Information-Theoretic Context Pruning for LLM APIs
A unified monorepo containing a dark-themed Next.js landing page and a Python serverless reverse proxy that prunes redundant RAG context using Shannon entropy — deployed as a single Vercel app.
┌──────────────────────────────────────────────────────────┐
│ YOUR APPLICATION │
│ (RAG pipeline, agent, chatbot) │
└─────────────────────────┬────────────────────────────────┘
│ POST /api
│ { messages, context_chunks }
▼
┌──────────────────────────────────────────────────────────┐
│ VERCEL (Single App) │
│ │
│ ┌──────────────┐ ┌─────────────────────────────┐ │
│ │ Next.js │ │ Python Serverless (/api) │ │
│ │ Landing │ │ │ │
│ │ Page (/) │ │ 1. Parse payload │ │
│ │ │ │ 2. Shannon entropy prune │ │
│ │ app/ │ │ 3. Inject compressed ctx │ │
│ │ page.tsx │ │ 4. Forward to OpenAI │ │
│ │ │ │ 5. Stream response back │ │
│ └──────────────┘ └─────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘
│
▼
┌──────────────────┐
│ OpenAI API │
└──────────────────┘
ShannonProxy/
├── app/ # Next.js frontend
│ ├── layout.tsx # Root layout + SEO metadata
│ ├── page.tsx # Landing page (dark theme, emerald)
│ └── globals.css # Tailwind + custom design system
├── api/ # Python serverless backend
│ ├── index.py # Vercel serverless function (proxy)
│ └── shannon_core.py # Shannon entropy pruning engine
├── package.json # Next.js + Tailwind dependencies
├── requirements.txt # Python dependencies
├── vercel.json # Hybrid routing config
├── tsconfig.json # TypeScript config
├── next.config.ts # Next.js config
├── postcss.config.mjs # PostCSS / Tailwind v4
├── test_client.py # Test script
└── README.md
- Node.js 18+
- Python 3.9+
- An OpenAI API key
git clone https://github.com/your-org/ShannonProxy.git
cd ShannonProxy
# Install frontend dependencies
npm install
# Install backend dependencies
pip install -r requirements.txt# Linux / macOS
export OPENAI_API_KEY="sk-your-key-here"
# Windows PowerShell
$env:OPENAI_API_KEY = "sk-your-key-here"The best local dev experience uses the Vercel CLI, which runs both the Next.js frontend and Python serverless functions together:
npm i -g vercel
vercel devThis starts:
- Landing page at
http://localhost:3000 - Python API at
http://localhost:3000/api
Alternatively, run just the Next.js frontend:
npm run devWith curl:
curl -X POST http://localhost:3000/api \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [
{"role": "user", "content": "Explain backpropagation."}
],
"context_chunks": [
"Backpropagation computes gradients via the chain rule...",
"The Amazon rainforest covers 5.5 million square km...",
"Loss functions like cross-entropy guide weight updates..."
]
}'With the test script:
python test_client.py --sdk-onlyExpected output:
Pruning Statistics:
Input chunks : 7
Retained chunks : 4
Pruned chunks : 3
Input tokens : 511
Output tokens : 295
Token reduction : 42.3%
curl http://localhost:3000/api/health
# → {"status": "ok", "service": "ShannonProxy", "version": "0.1.0"}Accepts an OpenAI-compatible chat completion payload with an additional context_chunks field.
| Field | Type | Required | Description |
|---|---|---|---|
model |
string | No | Model name (default: gpt-4o-mini) |
messages |
array | Yes | Standard OpenAI messages |
context_chunks |
string[] | No | RAG chunks to prune |
stream |
boolean | No | SSE streaming (default: false) |
shannon_config |
object | No | Per-request overrides |
shannon_config options:
| Field | Default | Description |
|---|---|---|
threshold |
0.25 | Min composite score to retain (0–1) |
min_keep |
1 | Always retain at least N chunks |
Response includes a shannon_proxy metadata block:
{
"choices": [...],
"usage": {...},
"shannon_proxy": {
"input_chunks": 7,
"retained_chunks": 4,
"reduction_ratio": 0.42,
"latency_ms": 14.2
}
}# Install Vercel CLI
npm i -g vercel
# Set API key
vercel env add OPENAI_API_KEY
# Deploy
vercel --prodYour app will be live with:
- Landing page at
https://your-project.vercel.app - API endpoint at
https://your-project.vercel.app/api
- Shannon Entropy:
H(X) = −Σ P(xᵢ) log₂ P(xᵢ)over sliding token windows measures informational density - TF-Cosine Relevance: Sub-linear token-frequency vectors + cosine similarity scores chunk–query relevance
- Composite Score:
score = w_e · H_norm + w_r · R_norm— chunks below threshold are pruned - Min-Keep Guarantee: Always retains at least N top-scoring chunks
| Layer | Tech | Purpose |
|---|---|---|
| Frontend | Next.js 15, Tailwind v4, Lucide | Dark-themed landing page |
| Backend | Python 3.9+, NumPy, tiktoken | Entropy + relevance pruning |
| API Proxy | OpenAI SDK | Downstream LLM forwarding |
| Deploy | Vercel | Hybrid Next.js + Python serverless |
MIT
