Pegas field notes · October 2026
How We Ran an Open-Source LLM on a Cloud GPU
RunPod + Ollama + Qwen3 4B: a practical proof, not a benchmark
Can one person rent a GPU, run an open model, expose it through an API, and get a valid structured response without sending inference to a hosted LLM API?
The short answer
Yes. We launched Qwen3 4B on a rented NVIDIA L4, served it with Ollama, verified GPU inference, exposed it through RunPod, and sent a structured request from a normal Windows machine. The final showcase request completed in 2.602 seconds end to end and passed schema validation.
The setup
- Cloud GPU: RunPod, 1x NVIDIA L4 (24 GB VRAM)
- Runtime: Ollama 0.35.0
- Model: qwen3:4b
- External test: Windows PowerShell → RunPod HTTPS proxy
- Output contract: category, priority, summary, nextAction
What we actually did
1. Confirmed the GPU was real and visible
We ran nvidia-smi inside the pod and confirmed the NVIDIA L4, driver 570.195.03, and CUDA 12.8.

2. Installed Ollama and pulled Qwen3 4B
Ollama installed correctly, but the container had no running systemd, so we launched ollama serve manually. The qwen3:4b model download was about 2.5 GB.
3. Verified real GPU inference
ollama ps reported 100% GPU, confirming the model was not quietly falling back to CPU.
4. Learned that “valid JSON” is not enough
A plain JSON response can still break an application contract. We switched to JSON Schema and machine validation and got SCHEMA_VALID.

5. Exposed the API without resetting the pod
Instead of editing the pod and triggering a reset, we reused the already exposed port 8888, stopped Jupyter Lab, and temporarily moved Ollama there.
6. Sent the final request from Windows
The final path was:
Windows → RunPod HTTPS proxy → Ollama → Qwen3 4B → structured JSON
The Showcase Proof Run reported:
- Model: qwen3:4b
- Runtime: Ollama 0.35.0
- External HTTP: OK
- Schema valid: True
- Measured latency: 2602 ms

What surprised us
- Containers do not always behave like normal servers.
- JSON is not the same as a stable API contract.
- Infrastructure changes can be destructive.
- A public model endpoint is useful for a smoke test, not for the final app.
What we proved
- NVIDIA L4 was available inside the pod.
- Ollama reported 100% GPU inference.
- qwen3:4b returned structured JSON matching our schema.
- The API was reachable externally through RunPod HTTPS proxy.
- One end-to-end Showcase Proof Run measured 2602 ms.
Cost and next step
We stopped the GPU immediately after the proof run, so compute billing returned to $0.00/hour. RunPod billing data was still delayed, so we are waiting for the official billing record before publishing the final experiment cost.
The next application path is:
Browser → Next.js /api/analyze → RunPod → Ollama → Qwen3 4B → validated response → UI
After that comes the frozen 25-case benchmark for accuracy, valid-output rate, latency distribution, real misses, and actual cost.
For Pegas, this was the moment the project stopped being a polished frontend preview and became a real self-hosted AI proof.