Measured, not cherry-picked.
These metrics predate the multi-agent Request Desk and are retained as historical single-step triage and regression evidence.
Looking for the current Request Desk evaluation? View the multi-agent workflow evaluation.
Optional internal-style rerun of the historical single-step path. It is not an evaluation of the current multi-agent workflow.
Category accuracy
Not run
Correct category / 25 cases
Priority accuracy
Not run
Correct priority / 25 cases
Valid output
Not run
Schema-valid responses / 25
Median latency
Not run
End-to-end request time
P95 latency
Not run
End-to-end request time
Dataset composition
25 cases with frozen expected labels, including deliberately ambiguous examples.
Where the model failed
Errors are part of the proof
This section reports deterministic mismatches and request/output failures from the current run. It does not guess why the model failed.
Fixed dataset
25 labeled test cases
Expected labels are frozen before the model is evaluated.
| Message | Expected category | Expected priority | Model category | Model priority | Status | Latency |
|---|---|---|---|---|---|---|
| #1Hi, we are building a wallet and are ready to integrate your swap API. Could you send the technical documentation and integration requirements? We want to start implementation this week. | integration | high | Pending | Pending | Not run | - |
| #2Do you have a JavaScript SDK, or should we integrate directly with the REST API? | integration | medium | Pending | Pending | Not run | - |
| #3We are testing the API integration in staging and every request to the quote endpoint is returning 401. Could your technical team help us check what is wrong? | integration | high | Pending | Pending | Not run | - |
| #4Which blockchain networks are currently supported by your API? We are comparing several providers before deciding whether to integrate. | integration | low | Pending | Pending | Not run | - |
| #5We need sandbox credentials for API testing. Our developer want to finish integration before tomorrow demo, can you provide access today? | integration | high | Pending | Pending | Not run | - |
| #6I cannot access my wallet after reinstalling the app and I need to recover access to my funds. Please help as soon as possible. | support | high | Pending | Pending | Not run | - |
| #7My swap shows completed but the funds have not arrived in my receiving wallet. Can you check what happened? | support | high | Pending | Pending | Not run | - |
| #8How can I change the receiving address before confirming a swap? | support | medium | Pending | Pending | Not run | - |
| #9The mobile app crashes every time I open it, so I cannot use the wallet at all. | support | high | Pending | Pending | Not run | - |
| #10Is there a way to export my previous swap history as a file? | support | low | Pending | Pending | Not run | - |
| #11We operate a crypto wallet and would like to discuss a possible long-term partnership with your team. | partnership | medium | Pending | Pending | Not run | - |
| #12We are choosing a swap partner for our wallet launch and need to finalize the decision by Friday. Could we discuss commercial terms and integration options this week? | partnership | high | Pending | Pending | Not run | - |
| #13Our company runs a crypto comparison platform. We would like to explore adding your service as one of our partners. | partnership | medium | Pending | Pending | Not run | - |
| #14Hello, do you have some referral or partner program for websites who send users to you? | partnership | low | Pending | Pending | Not run | - |
| #15Would your team be interested in running a joint giveaway with us on X next month? | marketing | medium | Pending | Pending | Not run | - |
| #16We are planning an educational campaign about crypto wallets and would like to invite your brand to contribute content and cross-promote it. | marketing | medium | Pending | Pending | Not run | - |
| #17Could you send us your logo pack and brand guidelines? We may mention your product in an upcoming social post. | marketing | low | Pending | Pending | Not run | - |
| #18I am writing an article about self-custody wallets and would like a short comment from your team. My publication deadline is tomorrow at 12:00 UTC. | media | high | Pending | Pending | Not run | - |
| #19We would like to invite someone from your team to our podcast next month to discuss trends in crypto payments. | media | medium | Pending | Pending | Not run | - |
| #20I'm preparing a market overview and wanted to ask whether your team could provide a short quote about current swap trends. | media | low | Pending | Pending | Not run | - |
| #21We still have not received the invoice for this month's service and our accounting team needs it before the end of today. | billing | high | Pending | Pending | Not run | - |
| #22Could you resend the invoice for last month? I cannot find the original email. | billing | medium | Pending | Pending | Not run | - |
| #23The amount on our latest invoice is different from what we expected. Could you explain how the total was calculated? | billing | medium | Pending | Pending | Not run | - |
| #24Hello, I am interested in working for your company. Where can I see current job openings? | other | low | Pending | Pending | Not run | - |
| #25Do you have a public office address where physical correspondence can be sent? | other | low | Pending | Pending | Not run | - |
Build it yourself
Don't trust the demo. Reproduce it.
The useful proof is the path another person can repeat: GPU, Ollama, open model, CUDA verification, server-side integration, and real measurements.
View the build path →Rent a cloud GPU
Run Ollama
Pull an open model
Force / verify CUDA
Prove the HTTP API
Connect Next.js server-side
Measure actual results