Quality MCP | Devpost

Main Page
Evaluation Results

Inspiration

MCP Servers are awesome, but completely unregulated. There are maybe 20 mcp servers for excel, and none work well. Furthermore, with all of these being OSS, anyone can easily post malicious code under the guise of a useful MCP.

We wanted an easy way to verify, easier than opening up the MCP and looking under the hood.

What it does

Our tool ingests an MCP configuration and OpenAPI Spec, runs a security audit, then compares how the MCP should function versus what it actually achieves.By checking both its security posture and how fully it implements the spec, we generate a single score that reflects the server’s overall reliability.

How we built it

Our security scan uses a combination of libraries like Bandit and Semgrep, as well as Anthropic. For the coverage, we use the openAPI spec to generate tests that we then run against the MCP server.

Challenges we ran into

All of this was very new to us, and there were alot of different debugging issues that slowed us down

Accomplishments that we're proud of

We made it happen! And can absolutely use it now to test MCPs. We also didn't know each other until today, and this project was definitely a stretch for the time.

What we learned

What's next for Quality MCP

We definitely would like to open source it, or have a hosted service where we can charge companies to verify their MCPs. The code is pretty much "hackathon code" and needs major rewriting to be deployable as a service.