Skip to content

Conversation

@mattt
Copy link
Contributor

@mattt mattt commented Apr 10, 2025

Resolves #62

Because Transport brokers Data objects, it can support sending and receiving JSON-RPC arrays without modification.

Server adds an internal Server.Batch structure and adds logic to decode arrays received from its transport connection. All user code will work without additional changes.

Client adds a public, Client.Batch actor, which is passed as an argument to a new withBatch method. API consumers can now make multiple requests in a single batch and receive their results asynchronously.

Example 1: Batching multiple tool calls and collecting typed tasks:

// Array to hold the task handles for each tool call
var toolTasks: [Task<CallTool.Result, Error>] = []
try await client.withBatch { batch in
    for i in 0..<10 {
        toolTasks.append(
            try await batch.addRequest(
                CallTool.request(.init(name: "square", arguments: ["n": i]))
            )
        )
    }
}

// Process results after the batch is sent
print("Processing \(toolTasks.count) tool results...")
for (index, task) in toolTasks.enumerated() {
    do {
        let result = try await task.value
        print("\(index): \(result.content)")
    } catch {
        print("\(index) failed: \(error)")
    }
}

Example 2: Batching different request types and awaiting individual tasks:

// Declare optional task variables beforehand
var pingTask: Task<Ping.Result, Error>?
var promptTask: Task<GetPrompt.Result, Error>?

try await client.withBatch { batch in
    // Assign the tasks within the batch closure
    pingTask = try await batch.addRequest(Ping.request())
    promptTask = try await batch.addRequest(GetPrompt.request(.init(name: "greeting")))
}

// Await the results after the batch is sent
do {
    if let pingTask = pingTask {
        try await pingTask.value // Await ping result (throws if ping failed)
        print("Ping successful")
    }
    if let promptTask = promptTask {
        let promptResult = try await promptTask.value // Await prompt result
        print("Prompt description: \(promptResult.description ?? "None")")
    }
} catch {
    print("Error processing batch results: \(error)")
}

@mattt mattt merged commit 82f4fd2 into main Apr 10, 2025
4 checks passed
@mattt mattt deleted the mattt/jsonrpc-batch branch April 10, 2025 19:13
@mattt mattt self-assigned this Apr 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

Support JSON-RPC batching

2 participants