Pinned
You can now call models running on your own hardware through a hosted AI gateway. Privately, with one command:
`ngrok http 8000 --url https://vllm.internal`
Plus all the public models you know and love. ↓
ngrok
2,026 posts
ngrok
@ngrokHQ
- ngrok reposted
We spent a lot of time listening to feedback, and observing how users were working with our early access release. Super proud of the team and their efforts that went into this. If you want to route traffic to AI in your applications this is the gateway for you. - ngrok reposted
I wired tiny OLED screens onto two Raspberry Pis so I can watch them find and forget each other. It's a live view of the kernel's ARP table: how every machine tracks its neighbors on a local network and maps IPs → MAC addresses.
00:00 - Replying to @ngrokHQOne flat fee: $0.05 per million tokens with your inference billed on top. Buy credits as you go. Full writeup → ngrok.com/blog/new-ngrok… Docs → ngrok.com/docs/ai-gatewa…
- Yes, ngrok gives localhost a public URL. It's also developer infrastructure that routes and secures traffic to the wild thing you'll plug into prod next. Reach into customer networks. Put every bit of ingress behind one front door. Run one gateway for devices, APIs, and LLMs.
00:00 - Our own @samwhoo ported Kubernetes to the browser. Like a real flippin' cluster with lifecycles, DNS, and a simulated network. It's ~100k lines of TypeScript, almost all written by LLMs, but with every line reviewed by hand to keep it slop-free ↓Those dots moving around a webpage? Real (enough) pods sending each other requests over a simulated network, all in your browser. Hit pause and the whole cluster freezes, because it runs on a fake clock. Go poke at the demo for yourself: webernetes-demo.ngrok.app
00:00 - This prompt caching post is still a banger and perhaps the perfect read for a Friday afternoon if you're light on token spend.
- In 1996, AOL went down for 19 hours. It pushed "NASA finds evidence of life on Mars" off the front page of the New York Times. We sponsored an SRE to write a human postmortem of that outage: the stuff five whys leave out.
- Gather round and come on a journey with us back to 1996 when AOL went down ngrok.com/blog/aol-was-d…
GIF - Day 2 of vibe coding
