7 posts
llm How an 18.5 GB MoE LLM was made usable on a 4 GB gaming laptop: measured 1.53 to 5.11 tokens/sec, then wrapped in rootless systemd supervision with restart backoff, log rotation and boot persistence. Real numbers, real config.
llm A public benchmark showed 92% of coding-agent tokens were served from cache — a 12.5x cost difference. A teardown of oMLX's prefix cache, the six gaps in it, and the three ceilings that decide how many agents one machine can serve.
mlx What MLX, mlx-lm and oMLX each own, which layer oMLX actually replaces, and three findings from reading the code: a dead SLO subsystem, an auth model with no principal, and a Thunderbolt 4 misclassification.
llm Step-by-step deployment of Qwen3.5-7B across two Mac minis with tensor parallelism, memory optimization, and performance testing.
mlx Understanding tensor parallelism, attention heads, and model configuration in oMLX. Which models support it, how to read config.json, and why it matters.
llm While running opensre (an AI-SRE investigation tool) locally through oMLX — a macOS-native MLX inference server — on two Mac Minis, the same real Kubernetes investigation task was pointed at two different open-weight instruct models in turn. Same oMLX server, same tool schemas, same alert. Only the
llm A unified stack for running, routing, and observing LLMs — currently deployed on a laptop with RTX 3050 Ti, with planned expansions to K8s GPU nodes and Apple Silicon clusters.