What I learned running local LLMs

I have been running local LLMs every day for the past few weeks, and one thing became clear.

The improvement came from using the right model for the right job. I was looking for one model that could handle code generation, Terraform, Kubernetes, AWS, CI/CD, documentation, and architecture discussions.

Every model has its strengths and weaknesses

Some are great at writing code Some are better at reasoning through complex problems Some are better at understanding larger systems and infrastructure

I have started to treat them like specialists on a team.

The three models I use

Qwen 3.5 9B

This is my default model. I use it for Terraform, Kubernetes, AWS, GCP, architecture discussions, and anything that needs a broader understanding of a system.

DeepSeek R1 8B

This is the model I use when I need reasoning. Gitlab CI/CD, Github Actions, CI pipeline debugging, workflow design, or anything that needs multiple steps before getting to the answer.

Qwen 2.5 Coder 7B

This one handles most of my coding.

• Python
• Go
• JavaScript/ TypeScript
• React
• Refactoring and code reviews

It is fast and has become my go to local model for day to day development.

Benchmarks

I spent too much time comparing benchmarks

• Which model scored higher
• Which one had a bigger context window
• Which one was newer

In reality, none of this mattered as much as having the right workflow.

• A coding model writes better code than a larger general purpose model
• A reasoning model does a better job untangling a complicated CI pipeline
• A larger model has a better understanding of infrastructure and architecture

My setup

After all the experimenting, my configuration ended up being boring.

• Coding tasks go to the coding model
• Infrastructure questions go to the larger model
• Pipeline related work goes to the reasoning model

That is it.

Will it replace cloud models?

Simple answer. No.

And I do not think it needs to.

For most of my daily development work, local models are more than good enough.

They are fast They are private They work offline

When I run into something that needs deeper reasoning or a second opinion, I still use Claude or GPT.

Conclusion

The biggest lesson was not which models to run. It was using the right model for the right job.