tool use agent

[mathewpareles]

No description provided.

[abhicombine]

anyone working on this?

[ziiw]

Hello !

What kind of tool use do you have in mind ?

I believe that the Model Context Protocol, would be more appropriate than standard tool use.

• Tool use is limiting because the list of tools is fixed on the first call
• Tool call needs to be done on the model provider server (which can be problematic for accessing a non-public endpoint, or for privacy)

Happy to discuss more on the topic :)

Proposed Target

• Being able to plan after a user query
• Being able to search in the codebase with different type of search (vector distance, regex)
• Being able to react to lints errors
• Being able to execute commands in terminal

[abhicombine]

I agree. MCP would be more appropriate and cover a wider array of use cases. Anyone working on MCP integration?

[andrewpareles]

Great question @ziiw. Right now, we have tools for:

• Context-gathering:
  • read file contents
  • search for file contents
  • search for file name
  • list a directory
• Editing files:
  • add/delete a file
  • write to a file
  • propose CLI commands

We currently have no "lint error" tool - if a lint error exists, it's automatically passed to the LLM and prompted for a fix.

Tool use is limiting because the list of tools is fixed on the first call

We weren't worried about tools being fixed on the first call, because last time I experimented, it seemed OpenAI's models can only output a message OR a tool call (not both like Anthropic), so we were going to prompt after every message to deal with OpenAI's models anyway.

Happy to talk more about how this all works, would love to hear your thoughts.

@abhicombine:
We don't have an MCP integration right now, but we want to support it. Happy to talk about getting this set up as well.

[ziiw]

Thanks for the details @andrewpareles
I'll dig the codebase a bit more to check those details.

[hongkongkiwi]

This would be amazing if void can support MCP, it will unlock a lock of potential.

Here's a good youtube video on it: https://www.youtube.com/watch?v=oAoigBWLZgE

[sailorseashell]

I think we should be careful with how we get this to automatically react to linting errors. I've had multiple times where a model recognized a lint error where there was none, and kept automatically trying to fix it until the code was entirely broken and it tried to throw out an entire file. The cursor settings to turn off auto fix don't get respected either - so even time you try to alter a file with this thing the model thinks is a mistake but isn't, it goes into a self destructive loop. I've seen this tool calling be useful, but also spiral very quickly when it comes to linting. These models still do hallucinate. The setting to enable auto fixing and not seems like a good solution if we could get the model to actually respect that setting, or having a flow that allows the user to fix the issue before then continuing on.

[andrewpareles]

Great suggestion, totally agree that an LLM can just confuse itself and downward spiral. We can definitely add a setting for disabling lint-fixing, especially for OSS models in general which might not be as smart.

One guiding principle for us is that linting fixes should always be done by an agent with tool use, because you might need to go into another file to fix a lint error. I think if we passively give the agent tool use but don't force it to fix the error before moving on, that might help, but totally open to other ways of dealing with lint errors.