TypeLLM

LLMs with type-safe generation

TypeLLM extends autoregressive LLMs with type-safe generation. Models can still think and generate freely when needed, while producing guaranteed typed outputs when structure matters. Define the output with a JSON Schema, and TypeLLM returns values your software can use directly.

  • No out-of-schema hallucinations
  • Negligible output-token cost
  • Linear input computation cost
  • Batch or sequential execution
  • Made for open autoregressive LLMs
View on GitHubRequest early access →

Supported output types

Text
Free text (string).
Integer
Whole numbers (integer).
Number
Numeric values (number).
Boolean
true or false.
Enum choice
One of your allowed string or numeric values.

Example

result = client.generate(
    context="Bought 3 notebooks for $12.50. Paid in full.",
    questions={
        "item": {
            "type": "string",
            "instructions": "Name the item in one plural word.",
        },
        "quantity": {
            "type": "integer",
            "instructions": "How many items were bought?",
        },
        "total": {
            "type": "number",
            "instructions": "What is the total price?",
        },
        "paid": {
            "type": "boolean",
            "instructions": "Was the purchase paid in full?",
        },
        "category": {
            "type": "string",
            "enum": ["office", "travel", "food"],
            "instructions": "Classify the purchase.",
        },
    },
)
print(result)
{
    "item": "notebooks",
    "quantity": 3,
    "total": 12.5,
    "paid": True,
    "category": "office",
}