Ideas & Plans for this Project

[chr-hertel]

Idea

This collection of libraries is meant to provide multiple things

• A agent/multi agent framework => Agent Component
• An abstraction layer for AI model (not only LLMs) and inference provider or libraries => Platform Component
• A store integration for RAG and similar pattern => Store Component
• Generalized tooling for chatbots => Chat Component
• A low level MCP library for server and client => MCP SDK

By design these components are intended to be framework and vendor agnostic - even if using Symfony components.
Meaning that specific integrations for models, inference provider or stores are moved to bridges, and additional bundles integrate those into the framework => AI Bundle and MCP Bundle

Current Status

The main focus currently is to settle public interfaces of this library and polish extension points, with the goal to have a first tagged release, see #877. Extension points to third-party services have been identified and bridges are mostly in place in Platform, Agent, and Store component. To be extracted to standalone packages.
Most work is rather stabilizing the components, leveling up the AI bundle, and incorporating feedback.

Known Problems (high-level instead of issue-level

• Platform Component
  • Redundancy around ModelClient and ResultConverter implementations
  • Weak adoption of Capability concept
  • Error Handling
  • Dynamic model catalog
• Store Component
  • Missing abstractions in querying for documents - only vague options
  • Doctrine ORM compatibility
• Agent Component
  • AgentInterface is hardly providing more than PlatformInterface
  • Insights into agent runtime, e.g. feedback on reasoning or progress
  • Tool calling loops

Future Plans

The AI development is moving fast, and adopting most common and promising trends is important.

Concrete Item

• Platform Component
  • Support of generalized OpenAI-based APIs
  • PHP-ORT support
• Store Component
  • More loader (web, rss, pdf, word, json, ...) and transformer implementation
  • Support and extension points for querying
    • Use-case multi-vector query
    • Use-case result reranking
• Agent Component
  • Redesign interface for looking for like an agent (instruction + tools + input + options)
  • Multi-Agent runner (orchestration, handoff), see [Agent] Multi-Agent Runner #19
  • Agent Trace collecting metadata about the agent runtime, see [Agent] Agent Execution Trace #17
  • Summarizer
  • Tool call limitation
  • Tool classifier (read vs write)
  • Agent Interaction/Interception (Human-in-the-loop)
• AI Bundle
  • Extend config to support library features
  • Profiler panel to visualize platform, tools and agents correctly
• MCP things, see [MCP SDK] Complete MCP protocol #5

Bigger Picture

Autonomous Agent
Agents are not about only executing task, but controlling workflows autonomously. Enabling that planning/reasoning layer in PHP apps with pre-defined pattern and APIs while providing transparency to that runtime, is quite a powerful vision.

Real-time & Multi-Modal
Voice and image/video in and out is great for modern and interactive UIs, but currently the support is quite limited and looking at protocols like WebRTC would enable more efficient interactions.

Generative UI
With MCP-UI/Apps, AG UI or OpenAI Apps, there is a strong movement towards generative UIs - in combination with Twig/Live Components, that could be a strong built-in feature for Symfony apps.

How to get started

If you are up for joining, that's super awesome - and I guess running the examples or the demo is a great start. I bet you will also find tiny things that need enhancements basically everywhere - PRs are welcome whereever.
I think to get familiar with the platform component implementing a bridge for unsupported models is a great start - or with a store.
Other than that, I'm happy to write down one of the bullet points from above in more detail if you're interested - or even jump on a call.

In general, if someone is interested and want to learn more about the status quo - I'm happy to do an intro call or session - just let me know :)

Interesting reads:

• https://github.com/humanlayer/12-factor-agents
• https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf
• https://speakerdeck.com/el_stoffel/agentic-applications-with-symfony

Cheers!