基于 OpenResty 实现的最小可运行 LLM Gateway:
Authorization: Bearer <key>鉴权(与上游 key 解耦)- 根据请求体
model自动路由到支持该模型的上游 provider - 白名单透传接口
GET /v1/models聚合返回可用模型列表
+----------------------+
| Client/User |
| OpenAI-compatible |
+----------+-----------+
|
| HTTP: /v1/* (model in JSON)
v
+---------+----------+
| OpenResty GW |
| (Nginx + Lua) |
|--------------------|
| access_by_lua: |
| - whitelist |
| - parse model |
| - route provider |
| - inject upstream|
+----+----------+----+
| |
| | Redis (control-plane)
| v
| +------+------+
| | Redis |
| | gw:key:* |
| | gw:model:* |
| | gw:provider:|
| +-------------+
|
| proxy_pass (transparent)
v
+-------------------+ +-------------------+ +-------------------+
| Provider A | | Provider B | | Provider C |
| (OpenAI/...) | | (Anthropic/...) | | (Self-host/...) |
+-------------------+ +-------------------+ +-------------------+
docker compose up -d
export OPENAI_REAL_KEY="你的上游 key"
./scripts/redis_seed.sh网关监听:http://localhost:8080
curl -sS http://localhost:8080/v1/models | jq
curl -sS http://localhost:8080/health | jqcurl -sS http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer demo" \
-H "Content-Type: application/json" \
-d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'见 docs/design/openresty-llm-gateway.md
仓库内提供一个简单的运维工具,用于维护 Redis 中的 key/provider/model 映射:
./ops/gwctl help
./ops/gwctl redis ping
./ops/gwctl provider add openai --base-url https://api.openai.com --auth-type bearer --auth-value "$OPENAI_REAL_KEY"
./ops/gwctl provider list
./ops/gwctl provider models openai
./ops/gwctl provider edit openai --auth-value "$OPENAI_REAL_KEY"
./ops/gwctl provider del openai --purge
./ops/gwctl key add demo --providers openai --status active --policy rr --model-allow "*"
./ops/gwctl model bind gpt-4o-mini --provider openai将 GATEWAY_DEBUG=1 开启路由调试日志(输出到 docker compose logs gateway):
docker compose up -d
docker compose exec -T gateway sh -lc 'echo $GATEWAY_DEBUG'
docker compose logs -f gateway