Hot Chips：OpenAI 的 Jalapeño、Cerebras CS-5、Groq 3 LPX、Apple M6

By far the biggest announcement at the 37th Hot Chips conference was OpenAI’s stunning progress on their own chip, less than a year after the Broadcom announcement… and that it isn’t an ASIC; but a full on Blackwell-beating alternative.

The key metric now is shifting to performance per watt, and Jalapeno delivers:

The full Hot Chips presentation is not yet out but various takes are below. For a fuller breakdown, watch along with the rest of OpenAI:

AI News for 8/24/2026-8/25/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

OpenAI’s Jalapeño Inference Chip and the Shift in the Inference Stack

- Jalapeño’s published numbers are the day’s biggest technical story: OpenAI released first benchmark details for its custom inference chip Jalapeño, claiming materially better efficiency and latency than NVIDIA GB200/GB300 systems on real model workloads. In OpenAI’s tests, Jalapeño delivered 1.5–1.9× more work per watt at peak throughput and 1.7–3.6× lower end-to-end latency, with 2.1–4.1× higher performance for highly interactive workloads; the chip is rated at 700W but reportedly stayed at or below 550W on the tested runs. OpenAI says deployment into its own infrastructure begins by year-end, with Gen 2 already deep in development and Gen 3 underway (OpenAI announcement, deployment roadmap, Sam Altman).

- Why engineers care: the claim is not just raw perf, but a more balanced inference architecture that reduces the usual throughput/latency tradeoff. Multiple technical reactions highlighted that some comparison points are especially notable because Jalapeño reportedly performed well even without tricks like aggressive prefill/decode disaggregation or speculative decoding in some setups, while beating systems that did use them (gdb, kimmonismus summary, eliebakouch analysis, You Jiacheng). SemiAnalysis framed it as unusually strong for a first-generation ASIC and compared it directly against Blackwell and Rubin-class systems (SemiAnalysis, dylan522p).

- A second-order story is model-assisted systems optimization: OpenAI’s post also said GPT-Astra + Codex helped write and optimize low-level kernels, bringing three previously unplanned open-weight models to high performance on Jalapeño in about two months; for selected attention and MoE blocks, these implementations reportedly ran 1.5–1.8× faster than existing human-expert-written code (kimmonismus, eliebakouch). That is a meaningful signal that compiler/kernel work is increasingly being folded into the model improvement loop, not just application-layer coding.

- Broader infra implication: several posts tie Jalapeño to a larger industry transition in which frontier labs may no longer be strictly downstream of NVIDIA for inference economics, even if packaging and foundry capacity remain a hard bottleneck (Liam Fedus, teortaxesTex reaction, LearnOpenCV caveat on TSMC/CoWoS capacity).

Agent Harnesses, Memory Systems, and Eval Engineering Becoming First-Class

- Harness quality is increasingly as important as model choice: several papers and launches converged on the same theme: agent performance depends heavily on the surrounding system. A new Microsoft-led paper on AutoSaddler treats the harness as code and patches prompts, tool configs, and control logic offline using failure traces, reporting gains of +9.0 on GAIA2, +9.6 on SWE-Bench Pro, and +10.0 on Terminal-Bench 2.0 over base harnesses (paper summary). In parallel, another paper quantified harness variance directly, finding that swapping harnesses could move scores far more than swapping models, with model-pair rankings flipping across scaffolds; the proposed fix is a structured Harness Card disclosure standard (analysis, “There Is No Neutral Harness”).

- Long-horizon software engineering remains very unsolved: SWE Refactor Bench measures whole-repository migration tasks like C→Rust, Maven→Gradle, and POSIX→WebAssembly across real projects including SQLite, zlib, and libsodium. Across 520 runs, only 28 survived all three stages, for a 5.4% survival rate, and 13/20 tasks were solved by nobody (EinsiaAI). This is a useful corrective to strong bug-fix numbers on more local coding benchmarks.

- Memory systems are being redesigned as programmable state, not compressed chat history: one Alibaba paper summarized by DAIR backs agent sessions with an append-only event log plus a persistent Python kernel, binding tool outputs and derived state to typed variables instead of continually serializing them into prompts. Reported results include 94.8% on LongMemEval_S, 73.1% on BEAM_10M (+5.1 over the previous best published memory system), and 86.7% on LOCA_256K with Qwen3.8-Max (summary). Related work on Knowledge Triage showed that naive context compaction destroys exact-rule retention; after five rounds of compaction, one setup preserved only 10% of safety rules, while type-aware retention policies preserved 2–4× more (summary).

- Practical eval-engineering is moving from ad hoc to productized workflows: LangChain/partners shared a concrete loop for turning traces and human feedback into task specs, synthetic environments, and evals that can be used to measure and post-train agents over time (Vtrivedy10, hwchase17). LangSmith Engine also shipped >2× better performance on key internal benchmarks with better issue detection/clustering, SaaS and self-hosted support, Slack/Linear integrations, and cost-tiered analysis modes (LangChain).

Local-First Agents, On-Device Inference, and the New Personal Compute Stack

- Perplexity’s Portable Computer is the clearest local-agent product launch of the day: Perplexity launched Portable Computer on NVIDIA DGX Spark, positioning it as a fully local version of Perplexity Computer where the orchestrator LLM, subagent LLM, and agent harness all run on local hardware with no cloud dependency (Perplexity launch, model details, NVIDIA, Arav Srinivas). The initial local stack uses a post-trained PPLX 27B with Qwen 3.8 27B also available; Nemotron 3.5 Lightning support is coming.

- The deeper trend is persistent, always-on local agents: Srinivas explicitly sketched a future of background processes that continuously ingest context from connectors, perform multi-hop reasoning in a perpetual loop, and run on your own hardware (Arav Srinivas). Community reactions were split between excitement about privacy/control and skepticism that “local-first” should mean a $5k DGX Spark rather than commodity consumer devices (theo critique, theo follow-up).

- Apple/macOS local AI tooling is also maturing: exo said Apple featured it on new M5 Ultra Mac Studio and M6/M5 Pro Mac Mini pages, emphasizing low-latency RDMA over Thunderbolt 5 to cluster Macs and run models like Kimi K3 and GLM-5.3 at API-like speeds, with 4× M5 Ultra scaling to about 4.8 TB/s aggregate memory bandwidth (exo). Related posts pointed to Apple’s faster PCIe storage and ANE-based vision pipelines as making small local clusters and mixed CPU/ANE/GPU inference more practical (anemll, onirenaud).

- Tooling continues to fill in around local runtimes: Ollama v0.33 added one-toggle integration to let Claude Desktop use Ollama as a third-party gateway for cloud and local models (Ollama); OpenCode v2 was shown running inside a Cloudflare Durable Object, illustrating how small agent runtimes are becoming embeddable in edge environments (fayazara).

Models, Retrieval, and Search Infrastructure

- Qwen 3.8 is showing up across the stack: enthusiasm around the Qwen3.8 release was visible in both deployment and evaluation posts, with Together adding fine-tuning and dedicated inference support for Qwen3.8-27B (Together) and Unsloth claiming full QLoRA fine-tuning of the 27B model on free 2× Tesla T4 Kaggle instances using optimized kernels (danielhanchen). On the application side, Qwen3.8-27B reached #1 among open models in the Image-to-WebDev Arena and #7 overall, while priced at $0.40 / $3 per million input/output tokens (arena).

- Search and retrieval infra got multiple substantive updates: Hugging Face published a detailed architecture writeup for the Papers with Code search engine: PostgreSQL + pgvector, Qwen 3 Embedding 0.6B, hybrid retrieval, embeddings generated on an NVIDIA L4 via Hugging Face Jobs, artifacts in buckets, and live serving via Inference Endpoints; the same stack powers “related papers” on paper pages (Niels Rogge). Keenable came out of stealth with a Web Search API and Web Query Language for AI, built by former Yandex Search leaders and backed by a $26M seed, explicitly targeting agent-scale web retrieval (styskin).

- Retrieval model design remains active territory: there was renewed discussion around late interaction / multivector retrieval, with claims that scaling behavior is finally becoming visible in retrieval workloads and that model+DB co-design matters at least as much as storage format (mixedbread perspective, Silvio Martinico).

Robotics, Physical World Models, and Embodied Data

- Figure’s “Index” is a major robotics data announcement: Figure introduced Index, described as the largest and most diverse robot dataset in the world, with reported ingestion at 30 minutes of video uploads per second, 16M video uploads, $15M already paid out for data, and 264k downloads. The company also says it will spend $1B over the next 12 months on data and compute (Brett Adcock, follow-up). That scale matters because many robotics labs still appear more bottlenecked on demonstration and perception data than on architecture novelty.

- Large-scale physics/world modeling continues to push context limits: Anima Anandkumar highlighted Accelerated Understanding, a startup building large AI models for physical simulation across modalities and 4D spacetime, claiming 1T parameters during pretraining, 1T context during training, and >5T context at inference without subsampling or patching (Anima Anandkumar). The details are sparse, but the post is notable as a statement of where some frontier non-language modeling work is heading: massive-context multimodal simulation rather than only text/video generation.

- Embodied policy generalization remains an active benchmark target: a separate robotics post introduced S1, a manipulation model that can complete tasks from a single demonstration outside its training distribution (anag004). Google Research also shared AgentHands, an XR system that augments conversational agents with synchronized hand gestures for spatial guidance during physical tasks (Google Research).

Top tweets (by engagement)

- OpenAI chip launch: @sama on Jalapeño, @OpenAI benchmark announcement drove the largest technical conversation by far.

- Local agent launch: @perplexity_ai launching Portable Computer was the biggest product release outside the chip story.

- Developer platform / agent-native web: @OpenAIDevs announcing the WebMCP Challenge and WebMCP support in ChatGPT desktop signal OpenAI pushing websites toward explicit agent interfaces.

- Open-source local task agents: @AndrewYNg on OpenWorker stood out for combining open harnesses, local models, and security-focused workflows.

- Benchmark realism for coding agents: @EinsiaAI on SWE Refactor Bench is one of the more useful benchmark releases in the set because it targets whole-repo migrations instead of local edits.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Qwen3.8 Flash/27B Benchmarks and Local Fit

Read more