Join the Tool Use Discord: https://discord.gg/PnEGyXpjaX
Are you looking to optimize your AI applications with the best open source Large Language Models? This week, we explore open source AI, rapid model switching, and local self-hosting with Nathan Break, a Machine Learning Engineer at Mozilla AI and the creator of Any-LLM. Any-LLM is an open-source Python SDK that allows developers to seamlessly talk to any LLM provider through a single, unified interface.
Nathan demonstrates how developers can instantly swap between cloud-hosted AI providers and local models like Llamafile by changing just a single line of code. We dive into the growing capabilities of smaller, cost-effective models, how to reduce your API expenses, and the security benefits of running local AI on your own hardware. Nathan also introduces the Any-LLM Platform, a secure vault for managing API keys that tracks your token usage and enforces budget limits without ever accessing your prompt data. Finally, we check out Porch Songs, Nathan's open-source application that leverages LLMs to rewrite the lyrics of popular songs while maintaining their original rhyme schemes and chord charts.
Check out the projects mentioned in this episode:
Any-LLM GitHub: https://github.com/mozilla-ai/any-llm
Any-LLM Platform: https://any-llm.ai
Mozilla AI Blog Post: https://blog.mozilla.ai/introducing-any-llm-a-unified-api-to-access-any-llm-provider
Porch Songs GitHub: https://github.com/njbrake/porchsongs
Connect with us
https://x.com/ToolUsePodcast
https://x.com/MikeBirdTech
00:00:00 - Intro
00:01:25 - What is Any-LLM? Unified API for Open Source AI
00:05:32 - Benefits of Rapid Model Switching & Open Weight Models
00:11:06 - Any-LLM Python SDK Demo: Mistral Image Recognition
00:19:51 - Best Local AI Models: Llamafile, LM Studio & Local LLMs
00:24:38 - Any-LLM Platform Vault & Porch Songs App Demo
00:34:45 - AI Coding Agents, Claude Code & LLM Evaluations
Subscribe for more insights on AI tools, productivity, and open source development.
Tool Use is a weekly conversation with the top AI experts.
