As large language models make their way into everyday work, creative projects, and personal productivity, more people want to use them on their own terms – with control over their data, costs, and how they customize the tools. Hosted services like Gemini, Claude, and ChatGPT are convenient, but a rapidly maturing open source ecosystem now makes it practical to run capable models on the hardware you already own. This presentation surveys that landscape: how to pick a model that fits your machine (with a brief look at model types, quantization, and reasoning capabilities); how to run them locally using tools like llama.cpp, vLLM, and SGLang; and how to wire those models into real workflows through code editors, tool-use frameworks, and agentic systems. We’ll also touch on what to do when local compute hits its limits – from API aggregators like OpenRouter to on-demand cloud GPU rentals – with practical notes on the trade-offs at each step. Regardless of your technical background, you will walk away with a clearer map of what’s possible today with local AI and a sense of where to begin experimenting in your own work.
About the Speaker
Dillon Roach, Ph.D. is a Sr. AI Research Engineer at OpenTeams, where he helps organizations navigate the fast-moving GenAI landscape – from prototype to production. With a background in high energy nuclear physics and years in the open source PyData ecosystem, he specializes in translating emerging AI capabilities into practical, deployable solutions. Dillon has driven the AI and ML technical work across engagements for major financial institutions and government clients, building production RAG systems at scale, developing custom reinforcement learning models, and standing up end-to-end pipelines around both proprietary and open weight models. His work spans fine-tuning, agentic architectures, multimodal systems, and the messy real-world engineering that sits between a model checkpoint and a governed deployment.
About the Open Source Architect Community
The Open Source Architect (OSA) Community is an invitation-only group for seasoned software architects who are passionate about open source technology. Request to join the OSA Community: https://forms.gle/7efbynVzYhhH2LCQ7
We review each application carefully. If it’s a fit, you’ll get an invite to join the space where it all happens.
For the latest updates on all things open source, follow our public feed on LinkedIn.