Hacker News Reader

Comment by calibas | Hacker News Reader

"Mistral Large 2 is equipped with enhanced function calling and retrieval skills and has undergone training to proficiently execute both parallel and sequential function calls, enabling it to serve as the power engine of complex business applications."

Why does the chart below say the "Function Calling" accuracy is about 50%? Does that mean it fails half the time with complex operations?

View on HN

Mistral forgot to say which benchmark they were using for that chart, without that information it's impossible to determine what it actually means.

Relatedly, what does "parallel" function calling mean in this context?

That's when the LLM can respond with multiple functions it wants you to call at once. You might send it:

    Location and population of Paris, France

A parallel function calling LLM could return:

    {
      "role": "assistant",
      "content": "",
      "tool_calls": [
        {
          "function": {
            "name": "get_city_coordinates",
            "arguments": "{\"city\": \"Paris\"}"
          }
        }, {
          "function": {
            "name": "get_city_population",
            "arguments": "{\"city\": \"Paris\"}"
          }
        }
      ]
    }

Indicating that you should execute both of those functions and return the results to the LLM as part of the next prompt.

Ah, thank you!

Reply on news.ycombinator.com