Last week we precisely defined an agent: an LLM, in a harness, that calls tools repeatedly in a loop. This week we'll fix a few deliberate simplifications and explore the consequences of the harness in a bit more detail.
One thing of note - while the llms cant do math they can create functions that can do math..It may be possible to tell llm via a suitable prompt to create such function and do the math using that. would that work better/easier
Thanks for this article. I knew generally how LLMs are combined with other tools like search, but I didn't know about harnesses specifically. So a harness turns an LLM into something more like a useable application. It adds tools, fixes problems, adds constraints, and more. I'm just thinking, though, flipping it around in a more negative light, it's sort of hiding problems with the LLM as well. Hmmm. But I'll stick with one of your conclusions for now: "you have the potential to create tailored systems that can do better than the best generic chatbots for your specific domain." Super interesting.
Hi Hadley, I've just opened a PR to hopefully turbocharge ellmer along these lines, with claude code execution (python on claude server side) and programmatic tool calling (so that tools such as sql query results go directly into code execution environment without going through LLM).
Please keep this going! Absolutely loving these intuitive explanations of overjargoned concepts!
One thing of note - while the llms cant do math they can create functions that can do math..It may be possible to tell llm via a suitable prompt to create such function and do the math using that. would that work better/easier
Yes! I might talk about that in a future post
thanks, please continue this journey of bringing Agentic AI to the masses
look forward to next episode in the series
Librans are nice people ..
Thanks for this article. I knew generally how LLMs are combined with other tools like search, but I didn't know about harnesses specifically. So a harness turns an LLM into something more like a useable application. It adds tools, fixes problems, adds constraints, and more. I'm just thinking, though, flipping it around in a more negative light, it's sort of hiding problems with the LLM as well. Hmmm. But I'll stick with one of your conclusions for now: "you have the potential to create tailored systems that can do better than the best generic chatbots for your specific domain." Super interesting.
Hi Hadley, I've just opened a PR to hopefully turbocharge ellmer along these lines, with claude code execution (python on claude server side) and programmatic tool calling (so that tools such as sql query results go directly into code execution environment without going through LLM).
https://github.com/tidyverse/ellmer/pull/1020