You can now call models running on your own hardware through a hosted AI gateway. Privately, with one command:
`ngrok http 8000 --url https://vllm.internal`
Plus all the public models you know and love. ↓
260 chars. Fits.
Built Drawful clone from scratch in Claude Code, played it at family game night: phone controllers, shared TV, real-time sync, over a free ngrok tunnel. No subscriptions, no per-player fees, no app downloads. Code: github.com/averyduffin/fa…
Validating JWTs, restricting IPs, and running a WAF on your ngrok endpoints just got 10x cheaper.
Traffic Policy is now $0.10 per 100k TPUs, down from $1.
OAuth didn't get cheaper. It was already free. ↓
The next interactive essay from @samwhoo is about k8s probes, but even starting it took… 90k lines of TypeScript.
Here's webernetes episode 3: the first demo of a cluster in a browser tab. Deployment controllers due in 3 days, when Sam's summer holiday is supposed to start ↓
The first dollar at ngrok.ai is on us.
That's enough to send real requests through the gateway: route between hosted and self-hosted models, trigger a failover, peek at input/output tokens and see how much they cost. Then decide if you want to top up.
Join me & my guest @joelhans, Developer Relations Lead at @ngrokHQ, Thursday at 3PM ET, while we discuss all things developer relations, and learn more about building a working internet (on a piece of plywood) to better understand network packets work.
youtube.com/watch?v=uQ4KmE…
How does quantization work, and what are the different ways to measure it against the original model while maintaining confidence? At #P99CONF, @ngrokHQ's @samwhoo will share tools that gauge how much a quantized model differs from its original. ow.ly/4NJS50ZkG2A#ScyllaDB
ESTO ES UNA LOCURA
Un ingeniero de ngrok ha portado Kubernetes al navegador
→ 100.000 líneas de código escritas por LLMs en 2 meses
→ Revisó cada línea a mano y escribió 2.000+ tests
→ Los mismos tests pasan contra un cluster k3s real
→ Todo pesa 140KB, menos que un hello
Someone built Kubernetes that runs entirely in your browser.
This might be the easiest way to learn Kubernetes.
- No Docker.
- No Minikube.
- No Kind.
- No EKS.
- No installation.
Just open a webpage and start exploring Pods, Deployments, ReplicaSets, Nodes, scheduling,
We spent a lot of time listening to feedback, and observing how users were working with our early access release. Super proud of the team and their efforts that went into this. If you want to route traffic to AI in your applications this is the gateway for you.
You can now call models running on your own hardware through a hosted AI gateway. Privately, with one command:
`ngrok http 8000 --url https://vllm.internal`
Plus all the public models you know and love. ↓
ngrok.ai
I wired tiny OLED screens onto two Raspberry Pis so I can watch them find and forget each other.
It's a live view of the kernel's ARP table: how every machine tracks its neighbors on a local network and maps IPs → MAC addresses.
One baseURL change and an access key gets you:
→ routing to every model, even dedicated ones on AWS, Azure, etc
→ automatic failover + retries when a model slows or fails
→ scoped keys per app
→ token, latency, and cost visibility by app, dev, and model
When you route to a local model, it's full private connectivity: no publicly addressable URLs, no public IPs to allowlist, or ports to open, wherever you've got those GPUs humming.
Works with Ollama, vLLM, LM Studio, and the neoclouds, too.
