KRYXstal | Devpost

Trace showcase for countdown test
Trace showcase for synchronisation deconflict
External MCP tool integrations
Human-in-the-loop approval
Detailed agent information sidebar
Communication workflow example

Inspiration

AI Agents keep becoming increasingly capable, yet their behaviour often still remains a black box.

Imagine if multiple Agents, tools, policies, approvals, and communication channels become involved; the final output alone is no longer enough to figure out what actions happened along the way. It cannot explain which Agent acted, which tools it used, why an action was allowed or denied, how Agents influenced one another, or where a failure began.

This creates a governance problem. Operators may know that a task failed without being able to reconstruct what happened or prove which Agent performed each action.

KRYXstal was created as a way to solve this issue, and to make these opaque multi-Agent workflows understandable, accountable, and governable.

What it does

KRYXstal is a trace, audit, and coordination middleware for the Volc Agent Launchpad that turns every prompt, message, model run, MCP tool call, approvals, and final results into a single trace.

Users can easily inspect this evidence through tree and chronological timeline views without having to dig through logs segregated across agents, runtimes, or channels.

With regards to secure multi-Agent collaboration, KRYXstal provides:

distinct agent identity and IAM policies (presets such as reader, worker, deployer, admin, and custom presets)
explicit allow/deny and implicit deny policy evaluation
policy-filtered tools and external MCP exposure
human approval for new principals and missing capabilities
per-channel conflict/desync detection and synchronisation follow-up
automatic regeneration after lost races (concurrency/race condition issues)
immutable and integrity-checked artefacts for controlled cross-agent review
audit records that distinguish policy denials from synchronisation conflicts

Despite these features, KRYXstal still remains a single-user proof-of-concept (POC). It is not recommended for immediate usage in production workflows.

How we built it

We built KRYXstal around one core principle: interface explains decisions, but trusted middleware enforces them.

Each human-rooted task is attached with a traceId, while the derived messages retain their parent message links. Runs record their triggering message and reply channel. This allows a trace to follow work across agents, channels, model runs, tool usage, approval outcomes, and policy decisions.

The Fastify control plane validates requests and coordinates lifecycle state, scheduling, IAM, synchronisation, approvals, integrations, traces, and immutable review artefacts. Each agent Runs in a separate private workspace (inside the workspaces/ folder) with a Codex configuration containing its permitted tools, policy rules, runtime hooks, sandbox settings, and external MCP allowlists.

Before protected tool calls execute, a fail-closed PreToolUse hook asks the trusted control plane to evaluate the Agent’s short-lived Run identity, requested action, resource, policy, and any applicable human-approved grant. Agent-facing MCP APIs repeat policy and channel-membership checks server-side.

The state is stored using versioned JSON with serial copy-on-write mutations and atomic file replacement. Review artefacts are stored separately as immutable snapshots with file allowlists, sizes, SHA-256 hashes, and publisher Run and trace lineage.

Challenges we ran into

Multi-agent concurrency was a big challenge (if not our biggest challenge). Several agents can read the same channel state, work concurrently, and attempt conflicting writes.

To address this issue, KRYXstal introduces per-channel sequence numbers, read cursors, FIFO resource locks, bounded waits, expiring leases, and atomic compare-and-set writes.

When an agent loses a race, KRYXstal will:

reject the stale write
record a synchronisation decision and conflict event
identify the winning message and unseen state
withhold stale reply
start a bounded regeneration run using the newest state

This helps to prevent a stale answer from silently overwriting or duplicating another agent’s contribution.

Another big issue was the enforcement of IAM consistency. Simply hiding a tool that's unavailable in interface was insufficient, and our policy had to remain aligned with generated configurations, runtime hooks, MCP APIs, external MCP allowlists, channel membership, as well as human approvals.

We thus treated tool visibility as a 'defense-in-depth' rather than pure authorisation. Each protection is still evaluated at the trusted action boundary.

Accomplishments that we're proud of

We are proud of KRYXstal being able to demonstrate both successful and failed execution tasks, at an end-to-end level. We are also happy with the way we could make it distinguish between synchronisation conflicts and IAM denials, rather than just log as a failed run.

What we learned

We learnt that multi-agent systems behave very similarly to distributed systems as a whole. Both of these systems suffer heavily from synchronisation issues spanning from stale reads, partial failures, unclear ownership, and much more.

We also learnt the important of enforcing security at the action boundary. Even one unauthorised tool that bypasses this, even if it gets executed or used for a split second, could lead to disastrous consequences involving governance.

What's next for KRYXstal

Next step is to attempt at improving the intentional limitations of the baseline, to move it from a single-user POC to a production-ready middleware of an appropriate scale.

This includes ideas such as:

Database persistence
Tamper-evident audit storage
signed artefacts and externally durable evidence
durable queues, retries, and reconciliation of failures
policy versioning and rollback
approval expiry, approver identity
encrypted secrets and OAuth
trace export, advanced metrics, and alerts
stronger multi-tenant runtime isolation

All these aim to make KRYXstal become a dependable trust layer that sits between humans, AI agents, and external systems, in order to keep autonomous multi-agent systems understandable, accountable, and governable.