Legacy single-step experiment

Measured, not cherry-picked.

These metrics predate the multi-agent Request Desk and are retained as historical single-step triage and regression evidence.

Looking for the current Request Desk evaluation? View the multi-agent workflow evaluation.

Optional internal-style rerun of the historical single-step path. It is not an evaluation of the current multi-agent workflow.

Category accuracy

Not run

Correct category / 25 cases

Priority accuracy

Not run

Correct priority / 25 cases

Valid output

Not run

Schema-valid responses / 25

Median latency

Not run

End-to-end request time

P95 latency

Not run

End-to-end request time

Dataset composition

25 cases with frozen expected labels, including deliberately ambiguous examples.

integration: 5support: 5partnership: 4marketing: 3media: 3billing: 3other: 2

Where the model failed

Errors are part of the proof

This section reports deterministic mismatches and request/output failures from the current run. It does not guess why the model failed.

Waiting for a real benchmark run.

Fixed dataset

25 labeled test cases

Expected labels are frozen before the model is evaluated.

MessageExpected categoryExpected priorityModel categoryModel priorityStatusLatency
#1Hi, we are building a wallet and are ready to integrate your swap API. Could you send the technical documentation and integration requirements? We want to start implementation this week.integrationhighPendingPendingNot run-
#2Do you have a JavaScript SDK, or should we integrate directly with the REST API?integrationmediumPendingPendingNot run-
#3We are testing the API integration in staging and every request to the quote endpoint is returning 401. Could your technical team help us check what is wrong?integrationhighPendingPendingNot run-
#4Which blockchain networks are currently supported by your API? We are comparing several providers before deciding whether to integrate.integrationlowPendingPendingNot run-
#5We need sandbox credentials for API testing. Our developer want to finish integration before tomorrow demo, can you provide access today?integrationhighPendingPendingNot run-
#6I cannot access my wallet after reinstalling the app and I need to recover access to my funds. Please help as soon as possible.supporthighPendingPendingNot run-
#7My swap shows completed but the funds have not arrived in my receiving wallet. Can you check what happened?supporthighPendingPendingNot run-
#8How can I change the receiving address before confirming a swap?supportmediumPendingPendingNot run-
#9The mobile app crashes every time I open it, so I cannot use the wallet at all.supporthighPendingPendingNot run-
#10Is there a way to export my previous swap history as a file?supportlowPendingPendingNot run-
#11We operate a crypto wallet and would like to discuss a possible long-term partnership with your team.partnershipmediumPendingPendingNot run-
#12We are choosing a swap partner for our wallet launch and need to finalize the decision by Friday. Could we discuss commercial terms and integration options this week?partnershiphighPendingPendingNot run-
#13Our company runs a crypto comparison platform. We would like to explore adding your service as one of our partners.partnershipmediumPendingPendingNot run-
#14Hello, do you have some referral or partner program for websites who send users to you?partnershiplowPendingPendingNot run-
#15Would your team be interested in running a joint giveaway with us on X next month?marketingmediumPendingPendingNot run-
#16We are planning an educational campaign about crypto wallets and would like to invite your brand to contribute content and cross-promote it.marketingmediumPendingPendingNot run-
#17Could you send us your logo pack and brand guidelines? We may mention your product in an upcoming social post.marketinglowPendingPendingNot run-
#18I am writing an article about self-custody wallets and would like a short comment from your team. My publication deadline is tomorrow at 12:00 UTC.mediahighPendingPendingNot run-
#19We would like to invite someone from your team to our podcast next month to discuss trends in crypto payments.mediamediumPendingPendingNot run-
#20I'm preparing a market overview and wanted to ask whether your team could provide a short quote about current swap trends.medialowPendingPendingNot run-
#21We still have not received the invoice for this month's service and our accounting team needs it before the end of today.billinghighPendingPendingNot run-
#22Could you resend the invoice for last month? I cannot find the original email.billingmediumPendingPendingNot run-
#23The amount on our latest invoice is different from what we expected. Could you explain how the total was calculated?billingmediumPendingPendingNot run-
#24Hello, I am interested in working for your company. Where can I see current job openings?otherlowPendingPendingNot run-
#25Do you have a public office address where physical correspondence can be sent?otherlowPendingPendingNot run-
Showing 25 of 25 cases.

Build it yourself

Don't trust the demo. Reproduce it.

The useful proof is the path another person can repeat: GPU, Ollama, open model, CUDA verification, server-side integration, and real measurements.

View the build path →
01

Rent a cloud GPU

02

Run Ollama

03

Pull an open model

04

Force / verify CUDA

05

Prove the HTTP API

06

Connect Next.js server-side

07

Measure actual results