Support response schema, tools and runtime sampling parameters #81
Conversation
- Add Schema class for unified Zod/OpenAPI schema handling - Implement conversion between Zod and OpenAPI/JSON Schema formats - Add ToolsSchema class for managing AI tool definitions - Support conversion to Vercel AI SDK and OpenAI function calling formats - Include comprehensive test suite with 21 passing tests - Add JSDoc documentation for all public methods 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
…mplete-schema-implementation
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for response schema, tools, and runtime sampling parameters in POML. It extends the framework to handle advanced LLM configuration options including structured response schemas (JSON/Zod), function calling tools, and runtime parameters like temperature and model settings.
Key changes:
- Added schema utility classes for handling Zod and OpenAPI schema formats with conversion between them
- Enhanced POML file parser to recognize and process
<meta>elements for responseSchema, tool, and runtime configurations - Updated command line interface and VS Code extension to output and display these new schema and configuration options
Reviewed Changes
Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/poml/util/schema.ts | New schema utility classes for handling Zod/OpenAPI schema conversions and tool definitions |
| packages/poml/file.tsx | Added parsing logic for meta elements (responseSchema, tool, runtime) in POML files |
| packages/poml/index.ts | Updated CLI to output structured result with messages, schemas, tools, and runtime parameters |
| python/poml/api.py | Modified to extract messages from new structured output format for backward compatibility |
| packages/poml-vscode/command/testCommand.ts | Replaced LangChain with Vercel AI SDK for LLM interactions and added schema/tools support |
| packages/poml/tests/ | Added comprehensive test coverage for new schema functionality |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| const zodSchemaString = jsonSchemaToZod(this.openApiSchema); | ||
| // Create a function that has z in scope and evaluate the schema string | ||
| const evalWithZ = new Function('z', `return ${zodSchemaString}`); | ||
| return evalWithZ(z) as z.ZodTypeAny; |
There was a problem hiding this comment.
Using the Function constructor is a security risk as it's equivalent to eval(). This could allow code injection if zodSchemaString contains malicious code. Consider using a safer alternative like a predefined schema parser or validation library.
| return evalWithZ(z) as z.ZodTypeAny; | |
| // Use the safe parser method instead of generating and evaluating code | |
| return parse(this.openApiSchema) as z.ZodTypeAny; |
| return Schema.fromOpenAPI(JSON.parse(text)); | ||
| } else if (lang === 'zod') { | ||
| const zodObject = evalWithVariables(text.trim(), { z }); | ||
| return Schema.fromZod(zodObject); |
There was a problem hiding this comment.
Using evalWithVariables with user-provided text creates a security vulnerability. Malicious POML files could execute arbitrary JavaScript code. Consider using a safe Zod schema parser instead of eval-based execution.
| return Schema.fromZod(zodObject); | |
| this.reportError( | |
| 'The "zod" schema language is not supported due to security concerns. Please use "json" instead.', | |
| this.xmlElementRange(element) | |
| ); | |
| return undefined; |
| if (this.openApiSchema) { | ||
| return this.openApiSchema; | ||
| } else if (this.zodSchema) { | ||
| const schema = z.toJSONSchema(this.zodSchema); |
There was a problem hiding this comment.
The method z.toJSONSchema() may not exist on the Zod library. The correct method is typically zodToJsonSchema() from a separate library like 'zod-to-json-schema'. Verify the correct API usage.
TODOs: