How I Run Local LLMs

In discussions about my recent pile of benchmarks of LLMs ability to crack hard security bugs (Mythos-discovered bugs), such as here and here, more than a few folks have had questions about how to run models locally. It’s a broadly covered topic, but still a source of confusion for some folks, especially when it comes to running them optimally. I’m not an expert, by any means, but I’ll tell you what I know.