this part of the diff you are maybe not going to be a fan of, @Wauplin @hanouticelina...

Basically an OpenAI-compatible chat completion endpoint is stateless so we need to feed the full array of messages from the downstream application here.

Let me know what you think.

I'm not shocked by the logic. Maybe a bit clunky to mix the local behavior (stateful with only a string passed) and the server behavior (stateless messages) but not too problematic IMO

This is the interesting part of the PR.

I format the tool call info as a "regular" chunk as if it was content generated by the model itself. 🔥

And I send it as a SSE chunk.

Use a recent version of zod, you can import { z } from "zod/v4", and you can use z.prettifyError(err)

i tried but it requires ESM or something similar "node16" or something.. (but feel free to give it a try)

did you want to give it a try @coyotte508? would be cool if we were able to use Zod/v4

shouldn't we use delta.tool_calls (example here) rather than custom <tool_call_info>...</tool_call_info> tags ?

hmm it's not the same:

• delta.tool_calls are from the LLM asking for some tool calling (the LLM is asking "provide me the output of this function call so I can incorporate it into my thinking"), providing the inputs to the tool call.
• whereas here in <tool_call_info>...</tool_call_info> I send the tool outputs so it can be displayed in the UI.

Do you see what I mean?

The distinction between the LLM’s tool call intent versus returning the actual tool output for display definitely makes sense.

One thought: it could be cleaner (and more future-proof) to provide both the tool call delta and the tool call result as structured fields within the delta stream, rather than relying on custom tags in the output. That way, you avoid potential collisions with future model formats, and maximize compatibility with clients already following the OpenAI API spec. Both approaches have valid use cases, but leaning on structured responses might help with long-term maintainability.

it could be cleaner (and more future-proof) to provide both the tool call delta and the tool call result as structured fields within the delta stream, rather than relying on custom tags in the output

indeed. strong agree with the comment.

It should return 2 messages:

1. message 1: assistant msg with tool_calls
2. message 2: user message providing <tool_response>...</tool_response>

Example:

{  
  role: 'assistant',
  content: "<think>\nThe user is asking about the weather in New York. I should use the weather tool to get this information.\n</think>\nI'll check the current weather in New York for you.",
  tool_calls: [
    {
      function: {
        name: 'get_weather',
        arguments: {
          location: 'New York',
          unit: 'celsius',
        },
      },
    },
  ],
},
{
  role: 'user',
  content: '<tool_response>\n{"temperature": 22, "condition": "Sunny", "humidity": 45, "wind_speed": 10}\n</tool_response>',
},

Hmm i don't think it should be a user message @mishig25 – it's still an assistant message as well

No, i think the tool call intent is already sent by the regular LLM chunk i.e. the first clause of the if

But give it a try locally to check! you can use the example.ts script in debug mode if you want to try step by step (see PR description)

@julien-c

Yeah, it would be outside the OpenAI spec. Here’s one way it could look in the response, for example:

{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "delta": {
        "tool_calls": [
          {
            "id": "call_abc",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\": \"Paris\"}"
            }
          }
        ],
        "tool_call_results": [   // <- this would be the new property
          {
            "id": "call_abc",
            "output": "It's 24°C and sunny in Paris."
          }
        ]
      }
    }
  ],
  "created": 1234567890,
  "model": "gpt-4"
}

So, tool_calls would follow the OpenAI spec, and tool_call_results (or whatever the field is named) could be our extension for passing tool outputs, keeping everything structured and easy to parse.

A bit late to the discussion but the current behavior looks good to me now. Agree role: "tool" is a nice way of doing it. Re: "tool_call_results" (#1473 (comment)) I don't think it makes sense the returned a structured output of the tool result given there is no definition for that (worse case, this is handled/discussed in a follow-up PR.

For the record, here is the (truncated) output returned by the example from the PR description.

{ content: '', role: 'assistant' }
(...)
{ content: ' Models' }
{ content: ' page' }
{ content: '.\n' }
{
  tool_calls: [
    {
      index: 0,
      id: 'chatcmpl-tool-3437d43f4e4c4c4891736f547f3c2048',
      function: { name: 'browser_navigate' },
      type: 'function'
    }
  ]
}
{ tool_calls: [ { index: 0, function: { arguments: '{"url": "' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: 'https' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: '://' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: 'h' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: 'ugging' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: 'face' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: '.co' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: '/models' } } ] }
{ tool_calls: [ { index: 0, function: { arguments: '"}' } } ] }
{ content: '' }
{
  role: 'tool',
  content: 'Tool[browser_navigate] chatcmpl-tool-3437d43f4e4c4c4891736f547f3c2048\n' +
    '- Ran Playwright code:\n' +
    '```js\n' +
    '// Navigate to https://huggingface.co/models\n' +
    "await page.goto('https://huggingface.co/models');\n" +
    '```\n' +
    '\n' +
    '- Page URL: https://huggingface.co/models\n' +
    '- Page Title: Models - Hugging Face\n' +
    '- Page Snapshot\n' +
    '```yaml\n' +
    '- generic [ref=e2]:\n' +
    '  - banner [ref=e4]:\n' +
    '    - generic [ref=e6]:\n' +
    `      - link "Hugging Face's logo Hugging Face" [ref=e7] [cursor=pointer]:\n` +
    '        - /url: /\n' +
    `        - img "Hugging Face's logo" [ref=e8] [cursor=pointer]\n` +
    '        - generic [ref=e9] [cursor=pointer]: Hugging Face\n' +
    '      - generic [ref=e10]:\n' +
    '            - tex'... 22945 more characters
}
{ content: '', role: 'assistant' }
{ content: 'Here' }
{ content: ' are' }
{ content: ' the' }
{ content: ' top' }

fyi, we already have

Happy with either options!

I'm running with pnpm cli:watch serve ./src/agents/julien-c/local-coder/

when I change code in packages/mcp-client/src/Agent.ts, tsx build does NOT get triggered and I'd beed to manually cd packages/mcp-client && pnpm build. Am I doing something wrong or this cli:watch script could be better by watching its dependencies as well?

cc: @coyotte508

https://tsx.is/watch-mode

I guess you could include either node_modules or the mcp-client's directory

yes can you try adding --include ../mcp-client/src to this command then @mishig25?

(potentially in a later PR as i'm going to merge this one soon)