Inference Efficiency Is the Hot Story: FP8 Kernels and Small MoE Models Outshine New Flagships

Today's AI news has a clear throughline. The conversation is moving away from "what can the biggest model do" and toward "how cheaply can we run intelligence?" Kernel-level engineering and small, efficient models are stealing the spotlight from the usual frontier-release hype - and the signal is consistent across GitHub, HuggingFace, and Hacker News.

FP8 Kernels Are the Real Innovation

The clearest signal came from GitHub trending, where DeepGEMM - DeepSeek's JIT-compiled FP8 matrix-multiply kernels for NVIDIA Hopper and Blackwell GPUs - hit #1 for the day. This is the low-level kernel work behind DeepSeek's famously cheap inference, the kind of unglamorous plumbing that lets a lab serve frontier models at a fraction of the usual cost. The day's verdict was blunt: inference efficiency is the hot story, and kernel-level FP8 work is outshining new models. That is a striking inversion - the substrate, not the flagship, is what is drawing the attention. The rest of GitHub trending is mostly the familiar agent-and-UI layer, platforms like AutoGPT that wrap existing LLMs into workflows rather than advancing the models themselves.

Small MoE Models Make "Big" Feel Local

On HuggingFace, the model to watch is Edge0-35B-A3B-preview, a 35B Mixture-of-Experts model with only 3B active parameters. That active-params design is exactly what makes a nominally "big" model run at small-model speed; at Q4 quantization it sits around 20GB, a prime candidate for a 128GB workstation. It pairs naturally with Qwen3.8-27B, which is dominating HuggingFace trending with 7.73M downloads and already fits comfortably in 128GB at Q8, leaving room for a large context window. It is a useful contrast to the other end of the trending page: DeepSeek-V4.1-Flash, a 763B model that needs well over 400GB even at Q4 - no-go for a 128GB box, API-only. The gap between "runs locally" and "API only" is now the axis the community sorts models by.

The Efficiency Push Is Getting Physical

The race to cheap inference is now spilling into silicon and consumer devices. OpenTPU, an open-source AI inference accelerator that was itself developed by AI, is a live experiment in the AI-designs-AI-hardware loop - no longer a pitch, but a buildable artifact. On the consumer side, a 19-year-old founder is raising $11M to build a $3,499 personal-AI computer, betting that people will buy dedicated boxes for their personal models. Flagship releases are still landing - Mistral's Large 4 "Le Chonk" even dominated the Hacker News front page - but the community's real energy is on making everything run cheaper and closer to the user.

Signal to watch: If FP8 kernel work and 3B-active MoE models keep compounding, the next "frontier" milestone may be a local one - a single workstation that serves frontier-class models at a fraction of the cloud cost.