RunInfra (YC F26) (@runinfrai) / X

RunInfra (YC F26)

21 posts

RunInfra (YC F26)

@runinfrai

The inference platform that auto-optimizes your model by @rightnowai_co

Joined February 2026

Pinned
RunInfra (YC F26)
@runinfrai
Jun 30
RunInfra beta is live! Describe your use case. Tell us what to optimize for, which models, and your latency + cost targets. We handle the rest: kernels, quantization, serverless deploy, routing, autoscaling. And your optimized model now deploys straight through our connector
RunInfra (YC F26)
@runinfrai
Jul 17
01101011 01100101 01110010 01101110 01100101 01101100 01110011 00100000 01100001 01110010 01100101 00100000 01110100 01101000 01100101 00100000 01101101 01101111 01100001 01110100
RunInfra (YC F26) reposted
Jaber
@Akashi203
Jul 16
i open-sourced bonsai-turbo -- a batch-1 decode engine that runs @PrismML's Bonsai 27B 1.76x faster than the official llama.cpp fork. same outputs, token for token H100, tg128, greedy: ternary 85.5 >> 151 tok/s. 1-bit 90.1 >> 159 tok/s. logit parity with the fork on 32 of 32
RunInfra (YC F26) reposted
Jaber
@Akashi203
Jul 7
we built the agi compiler it watches your llm agent work, finds the parts that are secretly deterministic, and compiles them into verified binaries that cost nothing to run llms are just the first frontend, world models and new model types plug into the same toolchain, that's
RunInfra (YC F26) reposted
Jaber
@Akashi203
Jul 6
NVIDIA's Director of Accelerated Computing just said RightNow is seeing up to 14x more performance and 92% lower cost from AI-optimized CUDA kernels. the agents behind those numbers are now live on @runinfrai, and @runinfrai is now fully public >> runinfra.ai bring
00:00
RunInfra (YC F26)
@runinfrai
Jun 30
Replying to @runinfrai
New in beta: your optimized model deploys straight through our connector integration. Live in your stack, not stuck in a dashboard. More connectors landing soon. Any model on Hugging Face works: LLMs, embeddings, STT/TTS, diffusion.
RunInfra (YC F26)
@runinfrai
Jun 30
It's fast because of the kernels and that optimization ships under every model you deploy. Serverless. Pay per 1M tokens. Scale to zero. SOC 2 Type II.
Optimize open models for production - RunInfra
From runinfra.ai
RunInfra (YC F26) reposted
Jaber
@Akashi203
Jun 17
i open-sourced automegakernel -- compiles any huggingface model into a single persistent megakernel batch-1 decode is bandwidth-bound. normal execution launches one kernel per op and round-trips activations through HBM dozens of times a layer. that overhead is the whole problem
RunInfra (YC F26) reposted
Jaber
@Akashi203
Jun 8
i built autokernel. an agent that writes one cuda kernel and self-tunes it past nvidia's own libs. it hit 14x on some of their internal kernels and ended up inside big tech but it only optimizes one kernel at a time. an llm is hundreds of them, round-tripping through HBM between
RunInfra (YC F26) reposted
RightNow (YC F26)
@rightnowai_co
May 30
A new era of inference. no coordinates. it runs everywhere.
RunInfra (YC F26) reposted
Jaber
@Akashi203
May 23
deepseek v3.2 and v4 use compressed sparse attention. a "lightning indexer" scores compressed keys, you pick top-k per query, attention reads only those. problem is, the indexer materializes a [B, S, H_I, T] fp32 tensor before the top-k. at 64k context with v4-flash dims that's
RunInfra (YC F26) reposted
Jaber
@Akashi203
Apr 25
big models reason because they're deep. a 70b model has 80 layers, each doing something different. if you want a small model to do the same, you can take one layer and just run it 80 times. universal transformer did this in 2019. huginn did it in 2025. problem is, when you run
RunInfra (YC F26) reposted
Jaber
@Akashi203
Apr 21
we just released RightNow-Arabic-0.5B-Turbo, the smallest open Arabic model on HuggingFace it has 518M params, takes 398MB after quantization, and runs on a phone. it beats Qwen2.5-0.5B and Falcon-H1-0.5B on Arabic benchmarks, ties Falcon-H1-1.5B on COPA-ar at 1/3 the size, and
RunInfra (YC F26) reposted
Jaber
@Akashi203
Apr 20
runinfra (@runinfrai) isn't just llms anymore! voice, transcription, tts, embeddings, any model you bring, we build you a highly optimized inference pipeline for it just tested it with a voice agent. wrote the prompt, got back a fully tuned stt + llm + tts stack try it out :
00:00
RunInfra (YC F26) reposted
Jaber
@Akashi203
Apr 19
been thinking about how wasteful LLM inference is at the token level every token goes through every layer. "the" gets 32 matmuls. a hard reasoning step also gets 32 matmuls. same compute for wildly different information content. always a bit silly, but now it's actually