Model Signal logo Model Signal Fast, verified AI updates

Tag

Benchmarks

Published stories tagged with Benchmarks.

AI Models 4 min read

Previewing GPT‑5.6 Sol, Terra, and Luna

OpenAI has begun a limited preview of the GPT‑5.6 family: **Sol** (flagship), **Terra** (balanced, 2× cheaper than GPT‑5.5), and **Luna** (fast, low‑cost). Sol introduces a new “max” reasoning mode and an “ultra” mode that uses sub‑agents. Early benchmarks show state‑of‑the‑art performance on coding (Terminal‑Bench 2.1), biology (GeneBench v1), and cybersecurity (ExploitBench, ExploitGym). The models ship with OpenAI’s most robust safety stack to date, but the preview may block or delay some requests.

AI Models 4 min read

GLM‑5.2 Brings 1M‑Token Context to Coding Agents

GLM‑5.2 is Z.ai’s newest open‑source model designed for long‑horizon coding tasks. It supports a solid 1 million‑token context, introduces flexible “effort” levels for balancing speed and capability, and uses the IndexShare architecture to cut per‑token FLOPs by 2.9×. Benchmarks show it outperforms its predecessor (GLM‑5.1) and ranks as the strongest open‑source coding model, closing the gap to leading closed‑source systems.

Coding 3 min read

Latest from X - 2026-06-16 to 2026-06-20

Z.ai: announced GLM‑5.2, a frontier‑intelligence model with open weights, a 1 M‑token context window and two reasoning effort levels for coding and agentic tasks. OpenCode: reported that GLM‑5.2 has risen to 6th place on their leaderboard within three days of release. ollama: highlighted GLM‑5.2 as the strongest open‑source coding model yet, now available on Ollama’s US cloud powered by NVIDIA AI Blackwell GPUs. GitHub Changelog: warned that Opus 4.6 (fast) will be deprecated in all Copilot experiences on June 29 2026 and urged migration to Opus 4.8 (fast). Cursor: introduced a new /automate skill that lets agents set up automations from plain‑language task descriptions, configuring triggers, instructions and tools automatically.

AI Models 3 min read

Latest from X - 2026-06-10 to 2026-06-12

Kimi.ai: Released the open‑source Kimi‑K2.7‑Code model, reporting +21.8% on Kimi Code Bench v2, +11.0% on Program Bench, +31.5% on MLS Bench Lite, and 30% lower reasoning overthinking. Google Research: Launched Gemini‑SQL2, a text‑to‑SQL system built on Gemini 3.1 Pro that hits state‑of‑the‑art scores on the BIRD benchmark. OpenAI: Added saved Codex rate‑limit resets for Go, Plus, Pro, and Business tiers (one free reset) and a two‑week invite program letting Plus/Pro users earn extra resets by inviting friends. Claude: Made dynamic workflows in Claude Code generally available, letting the model orchestrate parallel sub‑agents for complex tasks like codebase‑wide bug hunts and verify work before returning results.

Coding 4 min read

NVIDIA AI announced the release of NVIDIA Nemotron‑3 Ultra 550B (55B active) – What Developers Need to Know

NVIDIA’s Nemotron‑3 Ultra is a frontier‑scale LLM with 550 B total (55 B active) parameters, a hybrid LatentMixture‑of‑Experts (LatentMoE) architecture, and up to 1 M token context length. It runs on NVIDIA GPUs (minimum 4 × B200/H100) and offers configurable reasoning traces, multi‑token speculative decoding, and multilingual support. Benchmarks show strong performance on agentic, reasoning, and long‑context tasks.

Coding 3 min read

Latest from X - 2026-06-04

NVIDIA AI announced the release of Nemotron 3 Ultra, a 550 B MoE model that speeds inference fivefold, lowers agentic task costs up to 30 % and excels at coding, deep research, and long‑horizon planning. OpenCode noted that Nemotron 3 Ultra is now free with 1 M context and fully open source. Ollama said the model is available on its cloud platform, offering launch commands for Claude, Hermes and OpenClaw. OpenAI introduced a new memory system for ChatGPT that automatically tracks important details, doubles memory capacity for Plus and Pro users in the US, and lets users review and steer remembered content via a summary. Cursor added an interactive context‑usage report in its canvas, breaking down token distribution across prompts, tools, rules and skills.

AI Tools 3 min read

Gemma 4 12B Brings Multimodal AI to Your Laptop

Gemma 4 12B is Google’s new 12‑billion‑parameter multimodal model that runs locally on consumer laptops (≈16 GB VRAM). It eliminates separate vision and audio encoders, delivers reasoning close to the larger 26 B Mixture‑of‑Experts model, and is released under an Apache 2.0 license with full tool‑chain support.

Coding 3 min read

Qwen 3.7‑Plus: Multimodal Coding Agent with Vision‑Language Upgrade

Qwen 3.7‑Plus is a new multimodal agent model that adds vision capabilities to the strong text backbone of Qwen 3.7. It can read screens, interact with GUIs, and generate code from visual references while keeping the coding and tool‑use strengths of its predecessor. Benchmarks show notable gains in several coding‑related tasks, especially in terminal‑based and spreadsheet benchmarks.

Coding 3 min read

MiniMax M3: New Coding‑Focused LLM for Long‑Context and Tool Use

MiniMax released its latest M‑series model, **MiniMax‑M3**, on June 1 2026. The model is marketed for agentic reasoning, tool use, coding, multimodal chat input, and long‑context tasks. It follows a series of MiniMax models (M2.5, M2.1) that already claimed state‑of‑the‑art (SOTA) performance in programming, code refactoring, and tool calling.

Coding 3 min read

Latest from X - 2026-06-01

LangChain – Introduces Managed Deep Agents that retain the familiar project layout (AGENTS.md, skills/, subagents/, tools.json) and adds a Context Hub for persisting and updating agent context across sessions; a technical roundtable on June 17 in Munich will cover production‑grade agents, agent harnesses, and the Deep Agents SDK. MiniMax – Launches MiniMax M3, an open‑weights model that merges a 1 million‑token context window, frontier coding, agentic abilities, and native multimodal (image/video) support, with benchmark scores highlighted and a 50 % discount for the first week; the model appears automatically in the Hermes Agent picker and credits the Teknium and Nous teams. OpenRouter – Announces that MiniMax‑M3 is now available on its platform, offering the same 1 M‑token context, frontier coding, agentic performance, and multimodal capabilities. Visual Studio Code – Promotes its new VS Code Learn Series episode “Extending Agents,” which teaches how to use tools, agent plugins, and third‑party agents within the editor.

Open Source 3 min read

Latest from X - 2026-05-27

GitHub is hosting a Maintainer AMA in their community on May 27 from 8 a.m. to 1 p.m. PT, where contributors can ask questions and show appreciation for open-source projects like OpenClaw and Kubernetes. LangChain is using traces to build evaluations for production agents and has developed Mission Control, a decoupled application for deploying, configuring, and troubleshooting self-hosted LangChain infrastructure within Kubernetes. xAI has reset Grok Build usage limits for all accounts after receiving feedback on the Grok Build Beta, with the team continuing to collect feedback to improve the service. Anthropic has published a blog post discussing the importance of evolving access and permissions for agents as their capabilities grow, using sandboxing to limit potentially destructive actions in their own products.