Sydes test | Devpost

Intro of the project
Result postman collection

Inspiration

AI coding agents can edit code, but backend APIs are spread across routes, handlers, validators, models, and response logic. Sydes-test was inspired by the need to understand that structure before generating useful tests.

What it does

Sydes-test analyzes a backend repository, discovers API routes, reconstructs request and response contracts, generates test cases, and exports Postman files.

It uses deterministic extraction first, with an optional evidence-grounded LLM step for unresolved gaps.

How we built it

We built Sydes-test as a Python CLI using Codex as the primary coding agent.

**GPT-5.6 helped us define the architecture and make key decisions: deterministic extraction first, route-specific evidence retrieval, one optional LLM completion pass, and a strict merge layer that prevents model output from overwriting stronger facts.

Codex implemented the CLI, contract extraction, response normalization, test generation, Postman export, debug artifacts, and regression tests. We used it iteratively to diagnose failures across Express, FastAPI, and Spring Boot repositories.**

We built a Python CLI with three main commands:

init configures the repository
scan discovers routes with Codebase Memory
generate extracts a contract, creates a test matrix, and exports Postman artifacts

The workflow is:

init stores the repository path and base URL.
scan uses Codebase Memory to index the repository and discover routes.
generate selects one route and retrieves evidence from handlers, validators, controllers, schemas, and models.
Deterministic extractors build the initial API contract.
Depending on the selected mode, an OpenAI model receives a compact evidence bundle and fills unresolved gaps.
The final contract is converted into a deterministic test matrix.
Postman collection and environment files are generated from the test cases.

The LLM receives only route-specific evidence, and deterministic facts remain authoritative.

Challenges we ran into

The hardest problem was not generating test cases. It was retrieving the right code context for one route.

Route discovery tools sometimes returned only a small route marker rather than the full handler or controller source. We added fallback source resolution and evidence prioritization so the richer local source could reach the extraction and LLM stages.

We also encountered several LLM integration issues:

useful source snippets were initially removed by overly aggressive deduplication
the model sometimes returned valid data under a different JSON layout
response bodies could appear as fields, objects, arrays, or JSON schemas
empty fields arrays could incorrectly take precedence over a valid body schema

We addressed these with normalized response parsing, deterministic merge rules, detailed rejection diagnostics, and regression tests.

Another challenge was maintaining stable behaviour across different frameworks. Express, FastAPI, and Spring represent routes and contracts very differently, so we kept the core model framework-neutral while adding focused extraction strategies.

Accomplishments that we're proud of

Sydes-test works across Express, FastAPI, and Spring Boot examples.

It generated:

9 tests for an Express login route
16 tests for a FastAPI user route
a complete array-of-Book response contract for a Spring WebFlux route using LLM completion

The project currently has 161 passing tests.

What we learned

Good evidence selection matters more than sending large amounts of code to an LLM.

We also learned that deterministic analysis and LLM completion work best together, with a strict merge layer between them.

We also learned that Codex works best when given small, evidence-based implementation tasks, while GPT-5.6 is useful for architecture, debugging, and deciding what should be deterministic versus model-assisted.