Unable to use tool calling when using Llama 4 scout via LiteLLM Proxy

[tewenhao]

Please read this first

• Have you read the custom model provider docs, including the 'Common issues' section? Model provider docs Yes
• Have you searched for related issues? Others may have faced similar issues. Yes

Describe the question

(This is my first ever issue posted ever! If anything, I am happy to receive advice about openai agents and the like, as well as how I should be describing/framing issues I post. I hope to be able to help everyone out!)

I was following this guide on using custom model providers with open ai agents, and for my model i am using llama-4-scout-17b-16e-instruct via the LiteLLM proxy.

When I try to run the code, I keep getting the following error:

openai.BadRequestError: Error code: 400 - {'error': {'message': 'litellm.BadRequestError: Hosted_vllmException - "auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set. Received Model Group=llama-4-scout-17b-16e-instruct\nAvailable Model Group Fallbacks=None', 'type': None, 'param': None, 'code': '400'}}

I want to be able to run everything within a script rather than rely on CLI calls. I was able to run the code when I use AWS Bedrock as my model instead, so I doubt it is an issue with the code, nor would it be the API Keys or the Base URL I am using.

I have also explored using VLLM, but unfortunately as I work with a pretty sizable team I am unable to downgrade my Python version to <3.13 as required by VLLM without subjecting the rest of my team to verifying and checking none of the other features break because of the downgrade.

I want to ask if it is possible to do tool calling and/or reasoning capabilities using Llama 4 Scout via the LiteLLM Proxy in OpenAI Agents SDK

Debug information

• Agents SDK version: 0.0.15
• Python version 3.13.3

Repro steps

Ideally provide a minimal python script that can be run to reproduce the issue.

from __future__ import annotations
import asyncio
import litellm

from agents import Agent, Runner, function_tool, set_default_openai_api
from agents.extensions.models.litellm_model import LitellmModel

set_default_openai_api('chat_completions')

# get litellm proxy
import os
from dotenv import load_dotenv, find_dotenv
from pathlib import Path

load_dotenv(Path("../.env"))
API_KEY = os.getenv("API_KEY")
BASE_URL = os.getenv("BASE_URL")
MODEL_NAME = "litellm_proxy/llama-4-scout-17b-16e-instruct"
# MODEL_NAME = "litellm_proxy/bedrock"

@function_tool
def get_weather(city: str):
    print(f"[debug] getting weather for {city}")
    return f"The weather in {city} is sunny."

async def main(model: str, base_url: str, api_key: str):
    agent = Agent(
        name="Assistant",
        instructions="You only respond in haikus.",
        model=LitellmModel(model=model, base_url=base_url, api_key=api_key),
        tools=[get_weather],
    )

    result = await Runner.run(agent, "What's the weather in Tokyo?")
    print(result.final_output)

if __name__ == '__main__':
    try:
        asyncio.run(main(model=MODEL_NAME, base_url=BASE_URL, api_key=API_KEY))
    except Exception as e:
        print(e)

Expected behavior

A clear and concise description of what you expected to happen.

I expected to see this response (which I got from running using bedrock):

[debug] getting weather for Tokyo
Sun bathes Tokyo
Warmth embraces the city
Clear skies reign supreme

[rm-openai]

@tewenhao am I understanding correctly that the code you provided works with the same llama model on bedrock, but not on litellm proxy?

[tewenhao]

@rm-openai what i meant was the code works when i use aws bedrock via litellm proxy, but it does not work when i use llama4 scout via litellm proxy

[rm-openai]

gotcha. which model on aws bedrock did you use? I'm trying to narrow down whether it's a model issue or a litellm issue or something else

[tewenhao]

@rm-openai claude3.5

[rm-openai]

I see. Tbh it sounds like a llama+litellm specific issue - it might make more sense to ask in https://github.com/BerriAI/litellm about this. Seems like it just doesn't support tool calling?

[tewenhao]

Hmm I shall try to do that then. Thank you!