Venture Beat

After a stunning debut last week with cyber capabilities so advanced they reportedly found a previously undetected vulnerability in Cursor, GLM-5.3, the new frontier open source language model from Chinese startup z.ai, has now hit the application programming interface (API) — allowing developers the ability to build atop it and plug it into their agents and applications. Developers who previously subscribed to a GLM Coding Plan are currently limited to the OpenAI Chat Completions-compatible protocol. Z.ai said it plans to make the model’s weights openly available, but a precise date and licensing remain to be seen. On the API, the price is unchanged from GLM-5.2: $1.40 per million input tokens and $4.40 per million output tokens. Cached input costs $0.26 per million tokens, while Z.ai currently lists cached-input storage as free for a limited time. That means developers can move to the new generation without taking a higher posted per-token rate from Z.ai, even as the company claims substantially stronger coding and long-horizon agent performance.
At those rates, GLM-5.3 sits well below several of the highest-end frontier APIs. ModelInput ($/1M)Output ($/1M)Total ($/1M)SourceMuse Spark 1.2 Contributor$0.10$0.20$0.30MetaMiMo-V2.5 Flash$0.10$0.30$0.40XiaomiDeepSeek-V4-Flash — off-peak$0.22$0.66$0.88DeepSeekGPT-5.6 Luna$0.20$1.20$1.40OpenAIMiniMax-M3$0.30$1.20$1.50MiniMaxLongCat-2.0 — limited-time promo$0.30$1.20$1.50LongCatDeepSeek-V4-Flash — peak hours$0.44$1.32$1.76DeepSeekMiMo-V2.5$0.40$2.00$2.40XiaomiDeepSeek-V4-Pro — off-peak$0.66$1.98$2.64DeepSeekLongCat-2.0 — standard$0.75$2.95$3.70LongCatMiMo-V2.5 Pro (≤256K)$1.00$3.00$4.00XiaomiGemini 3.6 Flash — through Dec. 31, 2026$0.75$3.75$4.50GoogleGemini 3.7 Flash — through Dec. 31, 2026$0.75$3.75$4.50GoogleDeepSeek-V4-Pro — peak hours$1.32$3.96$5.28DeepSeekMuse Spark 1.1 / 1.2$1.25$4.25$5.50MetaGLM-5.3$1.40$4.40$5.80Z.AIGrok 4.6 — 256K)$2.00$6.00$8.00XiaomiQwen3.8-Max$2.00$6.00$8.00QwenCloudGemini 3.6 Flash — starting Jan. 1, 2027$1.50$7.50$9.00GoogleGemini 3.7 Flash — starting Jan. 1, 2027$1.50$7.50$9.00GoogleGPT-5.6 Terra$2.00$12.00$14.00OpenAIGrok 4.6 — ≥200K prompt tokens$4.00$12.00$16.00xAIGPT-5.4$2.50$15.00$17.50OpenAIKimi K3$3.00$15.00$18.00Moonshot AIClaude Opus 5$5.00$25.00$30.00AnthropicSakana Fugu Ultra (≤272K)$5.00$30.00$35.00Sakana AIGPT-5.6 Sol — Standard mode$5.00$30.00$35.00OpenAIClaude Fable 5 / Claude Mythos 5$10.00$50.00$60.00AnthropicGPT-5.6 Sol — Fast mode$10.00$60.00$70.00OpenAIUsing the simple VentureBeat comparison of one million input tokens plus one million output tokens, GLM-5.3 comes to $5.80, versus $8 for Grok 4.6 at its lower context rate, $18 for Kimi K3, $30 for Claude Opus 5 and $35 for GPT-5.6 Sol. That is not a workload-cost estimate — real bills depend heavily on the input/output mix, caching and token consumption — but it makes the relative API price tier easy to see.GLM-5.3 is not the cheapest capable model available. Google’s current introductory price for Gemini 3.7 Flash is $0.75 per million input tokens and $3.75 per million output tokens through Dec. 31, 2026, while OpenAI’s GPT-5.6 Luna is priced at $0.20 input and $1.20 output. Still, Z.ai’s price puts GLM-5.3 into a notably lower cost band than the premium frontier models it is increasingly benchmarked against.That comparison has become more relevant following the latest independent results. Artificial Analysis gives GLM-5.3 a score of 60 on its Intelligence Index, tying Kimi K3 as the top performing open weights model in the world, and scoring seven points higher than GLM-5.2. Its analysis also estimates GLM-5.3 at about $0.68 per Intelligence Index task, versus roughly $0.44 for GLM-5.2, despite the identical API token prices.The difference underscores an important caveat in headline API pricing: Artificial Analysis found GLM-5.3 more verbose than its predecessor, so flat per-token rates do not necessarily mean flat costs for a completed workload.For developers, though, the immediate change is straightforward: GLM-5.3 is now callable through Z.ai’s API at the same $1.40/$4.40 per-million-token rate as GLM-5.2, giving teams another relatively low-cost option for testing frontier-class coding and agent workloads.

Block, the technology company founded by former Twitter CEO Jack Dorsey that owns Square, Cash App and the music streaming service Tidal, is open-sourcing Berd, a desktop application it originally built to give its own employees a single environment for working with AI agents across different models, tools and projects.Berd is a locally installed graphical desktop application rather than a browser-based workspace. It is available now on GitHub under a permissive Apache 2.0 license — meaning anyone can use, modify and redistribute it, including commercially — with free downloadable builds for macOS, Windows and Linux. The repository reached version 0.6.2 on Aug. 18, its seventh public release, and lists 91 contributors.“Berd is desktop-first because much of its value comes from working directly with projects, local files, tools, repositories, and agents running on or connected to the user’s computer,” Brad Axen, Head of AI Capabilities at Block, told VentureBeat in emailed responses.A daily work surface for AI agentsBlock positions Berd as a “daily AI work surface”: a single place where users can start chats, attach files or folders, choose agents and models, work inside persistent projects, configure AI providers, manage skills and extensions, review session history and build automations.The design goal is not merely convenience. According to Berd’s product specification, users should always be able to tell which project, files, agent, model, provider and session state are actively shaping a conversation. That emphasis on visible operational state is what Block says separates Berd from a generic chatbot wrapper. Configuration — providers, extensions, skills, automations, projects — is treated as part of the workflow rather than buried in an administrative layer, and the interface is designed to expose failures, unavailable providers, and loading and streaming states plainly rather than smoothing them over with assistant-style friendliness.Berd grew out of a practical problem inside Block. Employees were already working with capable agents — Block’s own Goose, Anthropic’s Claude Code and OpenAI’s Codex — but the experience around them had become fragmented.“We had capable agents through goose, Claude Code, and Codex, but working across them meant navigating different interfaces, configuration systems, and ways of managing context,” Lucinda Bell, Technology Communications at Block, said in an email. Berd, she said, gives Block teams “one consistent desktop application across models and harnesses.”For enterprises, that layer may matter as much as another incremental gain in model capability. Persistent projects mean users can return to an established collection of files, instructions and agent configurations instead of reconstructing context for each task. Block also explicitly designed Berd to make agentic work accessible beyond engineering, letting people start with a conversation and add tools, context and structure as the work requires.Block is not positioning Berd as a mobile product. Axen said the company’s open source collaboration platform Buzz already provides a mobile experience and is Block’s preferred environment for work on the go.Block thinks agents should look different when they are differentBerd takes an unusual approach to a problem most enterprise AI products leave implicit: how users tell one configured agent apart from another.Instead of treating every agent as another blank chat window, Berd gives agents roles, instructions, skills, tools and recognizable visual identities. Block created collections of animated characters — its flagship designs are called “Gloopies” — and Berd’s public site showcases predefined personas such as Berdy, Pushback, Choosey, Copycat, Tinker and Wildcard, each built around a different style of work: Pushback plays devil’s advocate on drafts, Choosey helps narrow decisions, Copycat learns to write in the user’s style.The visual layer is not meant to substitute personality for substance. Block’s own formulation, from a company blog post on Berd’s design, is functional: “The avatars make the agent recognizable. Its role, skills, and tools make it useful.”That distinction could grow more important as workers juggle several specialized agents at once. A recognizable identity serves as visible shorthand for an otherwise invisible package of instructions, tools and permissions — agents appear on the workspace as distinct animated characters rather than identical entries in a chat list, so users can tell them apart before opening a conversation.The product specification guards against the characters turning Berd into a novelty. It describes the intended personality as “focused, capable, companionable,” while explicitly rejecting toy-like presentation, oversized decorative UI and interfaces that hide operational state behind friendliness.An orchestration layer, not another model or agent runtimeArchitecturally, Berd is neither a new foundation model nor a new agent runtime. The application is built with Tauri 2 and React 19 — frameworks for building lightweight desktop apps with web technologies — and communicates with Goose, the open source agent framework Block introduced in January 2025, through a standard called the Agent Client Protocol (ACP). Goose runs quietly alongside Berd as a bundled “sidecar” process and handles the underlying agent loop; Berd handles projects, sessions, context, agents and configuration on top.For readers unfamiliar with it, Goose is a model-agnostic agent framework and runtime, originally focused on software development workflows. It connects large language models to files, commands and outside systems — including through the Model Context Protocol (MCP), an open standard for wiring AI systems to external tools and data — so agents can take actions rather than merely generate text. Goose works with proprietary model providers as well as locally run open models. Block has since contributed Goose to the Agentic AI Foundation, the Linux Foundation body it helped establish with Anthropic, OpenAI and others in December 2025, giving the framework a vendor-neutral home alongside MCP.Axen drew the boundary explicitly: agent harnesses such as Goose, Claude Code and Codex manage the loop between a model, its context and its tools, while Berd provides the consistent desktop environment around those harnesses. In the public release, users configure whichever providers and harnesses they want rather than inheriting a Block-selected model stack.The project’s README adds an operational detail that matters for anyone evaluating the software: Berd does not simply pull whatever Goose build happens to be current. Its build process pins a specific Goose backend version through a lockfile, verifies the cached binary matches that pinned version, and packages it with the app. Developers can explicitly substitute another Goose binary for local testing. For organizations inspecting Berd, that creates a defined version boundary between the desktop application and its default agent backend rather than letting the backend silently drift.How Berd compares to Codex, Claude Code, Cursor and GooseThat separation matters because the agent-tooling market is already converging on products that manage increasingly autonomous AI workers. OpenAI describes its Codex app as a “command center” for agents, with separate project threads and multiple agents operating in parallel. Claude Code, Anthropic’s agent for reading codebases, editing files and executing commands, now spans the terminal, IDEs and a desktop app. Cursor describes its latest release, Cursor 3, as a unified workspace for software development with agents, including handoffs between local and cloud agents and multi-repository workflows. And Goose itself remains available directly, as a free command-line tool and desktop app for technically comfortable users.Berd’s differentiation is therefore less about inventing the agent workspace than about making that workspace open source, multi-harness and potentially useful outside coding. Every other product in the table below is either tied to a single vendor’s models, priced as a subscription, or aimed squarely at developers — and in most cases all three.ProductMakerPricingModel supportDistinguishing featuresBerdBlockFree and open source (Apache 2.0); users pay only their own model provider costsAny provider the user configures, across multiple agent harnessesDesktop workspace over agent harnesses rather than a harness itself; local-first data storage; visual agent identities; persistent projects; enterprise custom distributions; designed for non-engineers as well as developersCodex appOpenAIIncluded with paid ChatGPT plans, from Plus at $20/month to Pro at $200/monthOpenAI models only“Command center” for coding agents; multiple agents running in parallel across separate project threads; cloud and local executionClaude CodeAnthropicIncluded with paid Claude plans, from Pro at $20/month to Max at $100–$200/month; also usage-based via APIAnthropic Claude models onlyDeep autonomous coding — reads codebases, edits files, runs commands; available in terminal, IDEs, desktop and web; usage caps have drawn developer criticismCursorAnyspherePro at $20/month; Ultra at $200/monthMultiple frontier modelsAI-native code editor; Cursor 3 adds a unified agent workspace with local-to-cloud agent handoffs and multi-repository workflows; developer-focusedGooseBlock / Agentic AI FoundationFree and open source (Apache 2.0); bring your own model, including free local modelsAny provider, or fully local models via tools like OllamaThe model-agnostic agent framework and runtime underneath Berd; CLI and desktop versions; can run entirely offline with no data leaving the machinePortability is part of Block’s pitch. Axen said much of the underlying work in Berd is deliberately kept portable: folders, Git repositories, files and plain-text instructions remain accessible outside the application; skills are file-based; and Berd agents can be exported and shared between Berd users with their instructions and settings intact. He pointed to ACP and MCP as part of Block’s direction toward “clear, user-owned formats and open protocols wherever possible.” That does not make every piece of state interchangeable across every harness, but it makes portability an explicit architectural goal rather than a side effect of using local files.Local-first data, telemetry and enterprise distributionsBerd is designed around a local-first data model. Axen said conversation history is stored on the user’s device in the local Goose session database, while credentials live in the operating system keychain by default.That does not mean all AI processing happens locally. “Local-first does not mean that no data ever leaves the device,” Axen said. When a user asks a configured model to work on particular context, the relevant prompt and material are sent to that model’s provider — so the actual data path, and the contractual terms governing it, depend on which provider an organization chooses.Telemetry — the usage data software sends back to its maker — is disabled by default in official Berd distributions. If a user opts in, Block says it collects a random installation identifier and a limited set of predefined usage events, while excluding prompts, messages, files, source code, credentials, names, email addresses and IP addresses from the telemetry payload itself (Block and its service providers still process IP addresses for transmission, security and rate limiting). Users can switch telemetry off again in settings. Block cautions that third-party forks can modify or redirect the telemetry system, so enterprises evaluating a customized build need to review that build’s actual behavior rather than assume Block’s defaults carry over.The public repository builds a self-contained, general-purpose distribution that does not depend on private package registries or enterprise credentials — technical teams can build and evaluate it without access to Block’s infrastructure. But Block has also created what it calls “distribution seams”: defined points where organizations can overlay managed provider settings, private agents, runtime configuration, optional companion tools, their own update channels and their own signing and publishing infrastructure, all without adding private material to the public source tree.Block’s own deployment shows how an enterprise can put governance around that architecture. Axen said Block runs its managed distribution on approved devices using approved model-provider infrastructure, managed configuration and governed tool connections. “The application can remain open and inspectable, while an organization decides which models, tools, permissions, and policies are appropriate for its environment,” he said.One caveat: “enterprise distribution” should not be read as a complete enterprise management plane. The README does not describe capabilities such as single sign-on, centrally enforced administrator policies, organization-wide permission controls, data-retention rules or audit logs. It establishes that companies can customize and distribute Berd; it does not by itself explain how those installations are centrally governed after deployment. Berd should be understood as endpoint software installed on users’ computers, not a centrally hosted SaaS product.Pricing, models and a closed contribution modelBlock has not announced a subscription price for Berd, and there is no indication one is coming. That does not make agent usage costless: organizations still pay whatever model, API or provider charges they incur. The economic pitch on Berd’s site is flexibility — users can match the model to the job rather than paying for an expensive frontier model on every task. As the site puts it: “Some tasks need the big brain. Most don’t.”Neither Block’s announcement nor the README provides a definitive matrix of supported model providers or recommends particular models. Internally, Block does not treat any one model as a permanent default: Axen said the company’s managed distribution exposes employees to “a curated mix of frontier and open-weight models,” with the catalog changing as models improve. “The best choice depends on the work,” he said.There is also a governance distinction developers should note. Although Berd’s source is public and Block solicits issues and feedback, the repository does not accept outside pull requests — external code contributions are automatically closed. The README says Berd is built by “a small team at Block” and directs outside participants toward well-formed, reproducible bug reports instead. Users are free to inspect, build and fork the code, but Block retains a tightly controlled upstream. For companies considering Berd as infrastructure rather than an experiment, that makes Block’s long-term maintenance and support plans worth clarifying.Block is not presenting Berd as a revenue product today. Axen said the first return on investment comes from Block’s own use of the software — making internal agent work more consistent while teaching the company how employees actually use agents — and that open-sourcing it adds leverage by letting outsiders inspect, adapt and give feedback on the architecture. A commercial layer around enterprise deployment may eventually follow. “Supporting organizations that want to deploy agents at work is an area of interest for us,” Axen said, adding that Block will share more when there is more to share.From private agent work to multiplayer workflowsBerd supplies the individual desktop experience; another Block project, Buzz, focuses on shared rooms where people and agents collaborate against a common, searchable record.Buzz is Block’s free, Apache 2.0-licensed collaboration workspace for humans and AI agents. Built on the decentralized Nostr protocol, it combines familiar team features — channels, threads, direct messages, voice and media sharing — with agents that hold their own cryptographic identities and defined permissions, and that can participate in conversations, code review and approved automated workflows. Unlike Berd’s local, single-user focus, Buzz is explicitly multiplayer, and can be self-hosted or used through Block-hosted infrastructure. It is likewise model- and harness-agnostic: teams can bring agents powered by Goose, Claude Code, Codex or others.Axen framed the split between the two products partly as a question of trust boundaries. “A local agent may have access to your filesystem and credentials,” he said. “An agent in a shared Buzz room has its own identity and access scoped through the relay.” Berd begins with one person and comparatively privileged access to that person’s machine; Buzz begins with multiple participants, explicit identities and shared-workspace permissions.The more consequential roadmap detail is that Block does not intend the two products to remain separate. “Going forward, our focus is Buzz, and we plan to bring the best parts of Berd’s single-player experience into it,” Axen said. Block wants work to begin privately and become collaborative without users switching tools or rebuilding context, with cross-device continuity — including mobile via Buzz — part of that ambition. In the meantime, Axen said, Berd remains available as an open source desktop application, but Buzz is the product Block encourages people to download and adapt.What enterprise adopters should take awayThe short version for IT and security teams: conversation history stays local by default; credentials use the operating system keychain; context sent to a model leaves the machine on whatever terms the configured provider offers; official telemetry is off by default and excludes prompts, files and source code; and organizations can package their own governed distribution — though central management tooling is not yet documented.The larger idea behind the release may matter more. Models and coding agents are becoming increasingly interchangeable components inside larger workflows. Berd is Block’s bet that the surrounding context — the projects, skills, tools, identities and persistent working environment — becomes the stable layer instead.

Enterprises that already got burned by an AI agent passing its evals and then failing in production are moving faster toward removing humans from deployment decisions, not slower — even as trust in automated evaluation is rising across the board, new VB Pulse research shows.In July, 13% of 108 enterprises surveyed said they trust automated evaluation, up from just 5% the month prior. Meanwhile, survey respondents citing poor alignment between tests and real-world results as their biggest concern fell 10 points, from 29% to 19%, month over month. Yet, 49% of survey respondents said that an AI agent or LLM-powered feature that had cleared company testing subsequently created a problem visible to customers, essentially unchanged from 50% in June. And nearly a quarter, 24%, said this troubling outcome had occurred more than once.The latest findings from VentureBeat Intelligence uncovered a more troubling phase of the enterprise agent rollout: the gap is no longer only between how much autonomy companies give agents and how well they can verify them. It is increasingly a gap between confidence in the evaluation layer and evidence that the layer is getting better at preventing failures.The most revealing split appears inside the July data. Of the enterprises that experienced an AI feature clear testing only to go on to disappoint a customer, 4% placed complete faith in automated checks. Of those that had detected no comparable incident, 24% expressed full confidence — a sixfold difference.It makes sense: those who experienced test-passing agents failing in live production are, unsurprisingly, more likely to doubt the automated checking process. Perhaps it makes sense then, that companies geared toward tackling this problem — like automated agent error monitoring and mitigation platform Raindrop.ai — are seeing the market transform wildly from just a few months ago.”We are seeing the great-decline of evals as we know them,” Raindrop CTO Ben Hylak told VentureBeat in a direct message. “The Fortune 100 are increasingly reducing eval sets and deprioritizing maintenance. As systems grow more complex (MCPs, subagents, etc.) it becomes impossible to fully enumerate the failure cases. Instead, they’re leaning on anomaly and issue detection solutions, both before and after production.” A directional finding, not a market censusVentureBeat fielded the July wave among 108 people representing companies with workforces of at least 100. This is down from 157 respondents in June. Of the 108, 69% described themselves as final AI-buying authorities or people who recommend and influence those purchases. The sample skewed toward midsize organizations: 63% worked at companies with 100 to 2,499 employees.The findings should be read directionally. The survey is self-selected rather than a probability sample, and the burned-vs.-unburned splits cited throughout this piece rest on groups of 41 to 53 respondents, and other cross-tabs in the report range from 40 to 68. The industry mix also changed: technology and software participation declined nine points, ending at 14%, while the retail and consumer share added four points and ended at 19%.The report nevertheless identifies four month-to-month changes worth noticing: more respondents professing complete confidence, fewer naming poor real-world alignment, more choosing integration ease as the decisive buying factor, and Braintrust gaining primary-platform share.Confidence in automated evals improved, but outcomes stayed flatVentureBeat’s June research identified an enterprise evaluation gap: companies were granting agents more authority faster than they were developing reliable ways to test them.July preserves the key number from that first wave. Across 265 enterprise responses over the two months, the proportion reporting at least one test-approved system that disappointed customers stayed within a single percentage point: 50% in June and 49% in July.This figure does not mean that 49% of all agent runs fail, or that any particular evaluation product has a 49% failure rate. The survey asks whether an organization experienced at least one customer-facing incident in the previous year after an AI feature passed its internal tests. Companies that deploy far more agents have more opportunities to encounter such an incident.But that limitation does not make the result less important. An internal evaluation serves as a release gate. If roughly half of surveyed organizations have seen that gate approve a system that later fails in front of customers, a passing score cannot be treated as proof of production reliability.The cross-tab reinforces the point. Ten of the 41 enterprises with no identified testing miss placed complete faith in automation. Only two of the 53 previously burned enterprises said the same. Confidence is strongest among respondents with the least evidence that the release gate can fail.The enterprises that got burned are moving faster toward zero-human deploymentThe counterintuitive finding is what companies do after an evaluation miss.Overall, 67% either let an agent push code or change a system without a person’s approval in certain low-risk cases, or are modifying their pipelines to support that practice during the coming year. That is unchanged from June. In July, 37% already permitted it in limited cases and another 30% were building toward it.Among enterprises where a test-approved system had disappointed a customer, however, 85% were pursuing that no-approval model, compared with 61% in the group reporting no comparable incident. Only 11% of burned respondents rejected end-to-end deployment automation for the years ahead, versus 24% of unburned respondents.It would be easy to read that as recklessness, but the data supports another plausible explanation: deployment maturity. Organizations running more agents, at higher volume and across more consequential workflows, are more likely both to encounter failures and to have the engineering infrastructure needed for automated deployment.The survey cannot establish which explanation dominates. It does establish that a customer-visible incident does not appear to stop the move toward autonomy. Respondents with firsthand proof that testing can miss defects are also moving most aggressively to let those tests authorize production changes.If their per-deployment failure rate remains constant while deployment volume rises, the total incident count could grow even without the percentage of affected companies increasing. The July data does not measure incident volume, so that remains a risk implied by the pattern rather than a measured outcome.The release gate is automated, but production quality monitoring still lagsPre-deployment evaluation and production monitoring answer different questions. An evaluation asks whether an agent appears ready to ship. Production monitoring asks what the agent is doing after release and whether its live outputs remain correct.Most companies in the July sample still emphasize whether the system functions, not whether the answer is correct. Among the 106 valid responses to this question, 26% used inline quality assertions — automated judges or guardrails checking live traffic for output-quality problems. Another 26% focused on transaction traces such as infrastructure spans, token usage and raw inputs and outputs, while 24% mainly tracked gateway metrics such as latency, errors and cost.Trace and gateway data can reveal outages, slowdowns and broken requests. They may not flag a fluent, fast and confidently wrong answer. Grouped by the report according to what each architecture actually watches, half of respondents monitored whether an agent was functioning, while just over a quarter automatically monitored whether its production output was correct.The gap is sharpest among the 40 respondents already permitting no-approval deployment in limited cases. Only 28% of that group automatically checked the meaning and correctness of live answers. In other words, most enterprises that have eliminated a person from at least some release decisions have not installed automated semantic-quality monitoring as the production backstop.This is the clearest operational lesson in the data. A pre-deployment test suite and infrastructure observability are necessary, but they do not cover the same failure mode. Enterprises need a way to detect bad outputs after the agent begins interacting with real users, data and tools — especially when nobody reviews the deployment decision first.An independent agent-evaluation market begins to take shapeThe vendor data offers a more encouraging sign: enterprises are adding dedicated evaluation tools, and specialist platforms are gaining ground.OpenAI’s native evals and traces narrowly led as the primary platform at 18%, followed by Confident AI’s DeepEval at 17% and Braintrust at 15%. Anthropic’s Claude Console and Workbench held 12%, tied with organizations reporting no dedicated evaluation platform. Three options each held 6%: internally built tools, Promptfoo and LangSmith.Braintrust’s primary share increased from 8% in June to 15% in July, the biggest gain by one vendor and the one the report flags as statistically significant. DeepEval rose from 12% to 17%. Use of no purpose-built platform declined five points to 12%, although that smaller change does not by itself confirm a trend.Because many companies use more than one tool, the broader footprints are larger. OpenAI native evaluation appeared somewhere in 31% of stacks, DeepEval in 27%, Braintrust in 22% and Anthropic’s native tooling in 20%. Custom internal tooling reached 14%, while Weights & Biases Weave and open-source Langfuse each reached 11%.These are adoption figures, not product-performance scores. The survey does not establish that one vendor produces more reliable agents than another. Still, the results point toward evaluation becoming a distinct enterprise software layer rather than a loose collection of internal scripts or a feature used only inside a model provider’s platform.Purchasing priorities are changing with that market. The proportion choosing integration ease as the decisive factor climbed 12 points to 39%, displacing cost, which fell from 28% to 23%. Evaluation accuracy ranked second at 28%. The combined average for satisfaction, implementation simplicity and economic value was 3.9 out of five.The move from price toward integration suggests enterprises increasingly want a tool they can install into existing development and monitoring pipelines now. Yet their leading success metric remains evaluation consistency at 38%, followed by fewer failures and regressions at 20%. Buyers are selecting for fit while still judging results on repeatability.Switching intent also cooled: 56% still expected to add or replace a platform during the coming year, down from 64% in June.The proportion staying put increased eight points, reaching 44%. Together with specialist adoption, they suggest some buyers are moving from evaluation to implementation.Human review is becoming the hedge against automated missesThe budget data reveals how enterprises are managing the contradiction between greater autonomy and unreliable evaluation.People-centered review workflows edged narrowly ahead of production observability as the most frequently cited area for increased investment, 31% to 30%.Automated evaluation pipelines ranked third at 19%, followed by testing for safety and policy compliance at 16%. Only 6% said their reliability and evaluation budget was not increasing.Among enterprises that had experienced a testing miss, 38% said people-centered review would receive the fastest investment growth, compared with 24% of organizations that had not been burned.That produces an apparent paradox: the burned group is most likely to remove people from the release checkpoint and most likely to increase spending on people elsewhere in the process. The strategy appears to be automation with a human backstop — allow agents to move faster, then use reviewers to catch what automated evaluation misses.The open question is whether that model scales. Agent deployments and automated checks can grow with software volume. Reviewer hours do not fall at the same rate. Enterprises may therefore be replacing a human approval step with a larger downstream review function rather than eliminating human oversight.The narrow but consequential readJuly’s data does not show that enterprise agent evaluation is failing everywhere, nor does it prove automated judges are getting worse. It shows something more precise: confidence rose before the measured failure incidence improved.At the same time, the infrastructure around evaluation is maturing. More enterprises are adopting specialist tools, integration has become the leading purchase criterion and companies that have already experienced failures are increasing investment in human review. The market recognizes the problem and is spending against it.But the central reliability result remains stubborn. Nearly half of surveyed enterprises still report that an AI feature cleared internal checks before disappointing a customer. Respondents with that experience place less faith in automation — and move faster toward deployments with no human approval.The report frames this as an incomplete verification model: a passing pre-deployment score marks the start of monitoring, not the end of it. For most enterprises, the production quality checks and evaluation testing that would close that gap still aren’t in place.

Presented by Rezolve AiEnterprise AI investment in commerce has never been higher. And enterprise AI outcomes in commerce have rarely been more inconsistent. That gap is not a coincidence. It is the predictable result of a pattern that has repeated itself across every major technology shift in retail: the industry adds new capabilities faster than it integrates them.That pattern is now playing out in commerce AI.The point solution patternThe dominant approach to commerce AI over the past three years has been an additive one. Brands have layered AI-powered search on top of existing catalog infrastructure. They have added conversational interfaces on top of existing checkout flows. They have deployed recommendation engines alongside personalization tools that were themselves deployed alongside earlier recommendation engines. Each addition was justified by a discrete metric improvement, and none were designed to work as a cohesive system.This is the point solution pattern, and commerce has lived inside it for two decades. It produced genuine progress in isolated capabilities: faster search, better recommendations, lower friction at specific points in the journey. What it did not produce is coherence across the journey. Consumers experience that incoherence as inconsistency, context loss, and the feeling that each part of the shopping experience doesn’t know what the others are doing.AI amplifies the cost of that incoherence. When a general-purpose AI tool makes a recommendation based on incomplete or inconsistent data, it doesn’t surface a suboptimal product. It confidently surfaces the wrong one, and often excludes the incomplete one altogether. The hallucination problem in commerce AI is largely a data coherence problem in disguise. Tools that don’t share a common understanding of inventory, pricing, policy, and product truth will produce outputs that contradict each other and mislead consumers.Where the metrics lieThe fragmented approach to commerce AI creates a specific kind of reporting problem: individual tools perform well in isolation while the system underperforms in aggregate.A conversational AI tool can show strong engagement metrics. The search layer can show improved relevance scores. The checkout system can show reduced abandonment within its own funnel. None of these metrics captures what happens at the handoffs between them, where context breaks, sessions drop, and purchase intent that was successfully generated in one layer fails to convert in the next.This is why brands investing aggressively in commerce AI are sometimes reporting strong tool-level performance alongside flat or declining overall conversion. The tools are working. The system isn’t. And the standard analytics stack, built to measure individual touchpoints rather than journey coherence, will not surface that distinction.Bain research shows that organic web traffic to retail sites has declined 15 to 25% as AI-driven zero-click search has grown. Brands are losing top-of-funnel visibility to AI disintermediation at the same time their internal AI tools are generating positive performance reports. That combination — external pressure compressing the funnel while internal fragmentation leaks it — represents a structural problem that point-level optimization cannot solve.What separates the companies closing the gapThe brands that are generating consistent, measurable outcomes from commerce AI share a common architectural characteristic: they have built or adopted a unifying execution layer that sits across their AI investments rather than beneath them.This isn’t a new technology category. It is a different design philosophy. Instead of asking what AI capability to add next, these brands have asked what the connecting tissue between AI capabilities needs to look like in order for those capabilities to produce a coherent consumer experience and a reliable transaction outcome.The answer, in practice, involves three things: a shared data layer that gives every AI tool in the stack access to the same real-time product, pricing, and inventory truth; a policy and governance framework that ensures AI-generated recommendations operate within the brand’s established rules; and a transaction layer that can receive intent from any AI surface and convert it into a completed order without breaking context or requiring the consumer to restart.Brands that have those three things in place are not just getting better results from individual tools. They are compounding improvements across tools, because each capability in the stack is operating on consistent inputs and contributing to a coherent output.The architectural question commerce can’t deferThe window for treating commerce AI fragmentation as a temporary problem is closing. As agentic commerce matures and AI systems begin to initiate and complete transactions on behalf of consumers, the stakes of incoherence rise significantly. An AI agent acting on behalf of a consumer doesn’t have the patience to navigate a broken handoff between a recommendation layer and a checkout system. It will fail, and it will not return.The brands that establish architectural coherence now, before agentic transactions become the norm, will enter that era with a compounding advantage. Those that continue to add point solutions will find that each new tool adds a new potential point of failure.Commerce AI isn’t fragmenting because the tools are bad. It is fragmenting because the connective infrastructure was never built. The brands that recognize that distinction — and act on it — are the ones that will define what commerce looks like in the next decade.Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.

Enterprise teams running AI agents at scale are finding that a single model handles every task poorly — either the model is too expensive for simple questions or not capable enough for hard ones. Model routing, which picks the right model for each task automatically, is becoming the fix.Snowflake’s Cortex AI Gateway now offers dynamic model routing to address that: enterprises can select “auto” instead of a fixed model, and the system routes each task to whichever model offers the best combination of quality and cost. Snowflake said the capability can cut token costs by as much as 3x on some workloads — a figure from the company’s own internal testing — after finding that simple questions were often handled by its most capable model, making responses more expensive and slower than necessary.The move lands amid a broader industry shift toward automated model routing. Databricks, AWS, Google Cloud and Nvidia have all announced some form of model routing technology. Snowflake argues that model routing is more complex than just price and performance, it’s also about governance and context.”For high quality, enterprise grade agents to be built, it’s crucial to get the context and the governance right,” Baris Gultekin, vice president of AI at Snowflake, told VentureBeat. “Context, trust and model choice all go hand in hand.”Two mechanisms decide where a task goesThe capability builds on Cortex AI Gateway, which Snowflake launched in July 2026 as a governance layer for agent and model traffic. Before dynamic routing, model selection ran off a static list per task rather than a true fallback system, Gultekin said.Dynamic routing itself runs on two mechanisms, according to Gultekin.A small model tries first. Under what Snowflake calls an advisor pattern, a smaller model attempts a task first. If it cannot finish the job, it calls a larger model as a tool and continues from there.A classifier sorts by task history. A separate classifier, trained on past queries, automatically routes straightforward questions to simpler models.Customers can still pin a model. Auto routing is optional. Customers can restrict routing to one model or a defined set of models, and the system routes only within that boundary.There is no separate fee. Snowflake prices AI purely on token usage. Routing to a cheaper model produces a cheaper bill, with no additional charge for the routing decision itself.Access controls follow the task, not just the dataSnowflake ties routing to the same access controls it already uses for data governance.Governance starts at the data level with role-based access controls. It extends to models next, where customer roles map to buckets of approved models. It extends again to agents, where an agent can be restricted to narrower privileges than the user invoking it.Open models can run from a customer’s own region to satisfy data residency requirements. Gultekin said all inference, open and proprietary alike, stays inside Snowflake’s security boundary rather than routing out to an external provider. That regional and perimeter setup matters specifically for open models with non-U.S. origins, including DeepSeek-V4-Flash and GLM-5.3, both developed in China.Snowflake’s recent acquisition of Natoma adds another layer. The deal brings more than 100 MCP connectors with scoped, governed access. An agent could get read-only access to a connected tool like email, for example, rather than broader permissions.Context lets a cheaper model do the workSnowflake recently announced its Horizon Context and Cortex Sense tools that provide context capabilities.Without good context, a model has to do the exploratory work itself, writing and testing SQL, searching through data and retrying when something does not work. Gultekin explained that the process is expensive, and getting it right typically requires a more capable model. Packaging the context in advance removes that exploratory step, which means a simpler, cheaper model can often handle the same task.Snowflake also builds agent memory into that context. As an agent is used repeatedly, its memory updates and gets folded back into future queries. The system does not re-solve the same problem from scratch each time. Memory becomes part of the context passed to the model.OpenRouter, Databricks and Nvidia are chasing the same problemThere is no shortage of technologies in the model routing space. OpenRouter is one of the most widely known options, providing a platform that enables organizations to route based on cost and performance. Nvidia on August 11 announced Switchyard as a technology layer to help route AI model choice. Databricks has an offering as well with Smart Routing for its Unity AI Gateway.

“The interesting part is what it says about where differentiation has moved,” Sanjeev Mohan, Principal and Founder, SanjMo, told VentureBeat. “Snowflake isn’t really selling routing, it’s selling routing that never leaves the governed data boundary, with access controls, tagging, and cost attribution already attached.”Mohan added that for a company whose data and compliance already center on Snowflake, routing that keeps data in place and attributes spend by team is a real lever on that problem. For a company without that center of gravity, a neutral gateway may route across more models with less friction.Mohan frames the market as three distinct camps rather than one competitive field. Databricks approaches governance from data engineering and ML lineage. Its Unity Catalog governs data, models and pipelines for teams building and training models. Snowflake approaches governance from analytics and access control, governing who can touch which data and attributing usage across business units. A third camp includes neutral gateways such as OpenRouter, LiteLLM, Portkey and hyperscaler routers like Azure AI Foundry. These compete on model breadth and avoiding lock-in rather than deep governance.Choosing a router means choosing a governance modelModel routing is now table stakes for enterprises. The decision that matters is which governance model already fits how their data and teams are organized, not which vendor’s router is fastest or cheapest.Manual model selection is becoming a cost liability at agent scale. What worked when a team ran a handful of agents breaks down at scale. Hundreds of agents making routine model calls with no automated cost check in place adds up fast.Evaluate the governance model, not the router’s feature list. The real question, per Mohan, is which governance model matches the data estate already in place, and which one gives the cost visibility needed to avoid an unpleasant surprise.The right starting point depends on where an enterprise’s data already lives. A Snowflake shop gets more value from in-platform routing that respects its existing access model and bills back to cost centers than from raw model breadth, according to Mohan. A Databricks-centric team worried about lineage across training and deployment is better served by a gateway built around that same lineage. A multi-platform or model-first team that wants maximum choice with minimal lock-in fits better with a neutral gateway, the same pitch behind OpenRouter’s valuation.”For a practitioner, don’t start with the router, start with where your governed data and platform commitment already live, and with how exposed your margins are to inference cost,” Mohan said.

The biggest AI model release of the past few days, at least among the developers and AI power users on social media, wasn’t a frontier cloud model from OpenAI, Anthropic or Google.It was a 27-billion-parameter model from Alibaba: Qwen3.8-27B landed on Hugging Face on Friday under an enterprise-friendly, open source Apache 2.0 license, giving developers downloadable weights for a dense multimodal model.But Qwen3.8-27B isn’t a garden variety small local model: it includes native image and video understanding, a 262,144-token context window, configurable reasoning and support for coding and agentic workflows — a “compact, deployment-friendly” version of the capabilities developed for its Qwen3.8 generation.That unusually small hardware footprint is a major part of Qwen3.8-27B’s appeal. Running the model at full 16-bit precision requires roughly 56GB of GPU memory, while an FP8 version needs about 28GB. But 4-bit quantization cuts the model itself to roughly 17GB, putting it within reach of high-end consumer machines such as a powerful gaming desktop or well-equipped laptop. Hitting the sweet spot between capability and sizeThe outsized reaction among developers has been due to the dynamic combination of its capability and size.Alibaba’s own launch benchmarks immediately supplied the first jolt. The company reported 61.7 on SWE-bench Pro, 90.3 on LiveCodeBench v6, 70.7 on its CoWorkBench office-work benchmark and 84.3 on OSWorld-Verified. In Alibaba’s published comparison table, the 27B model even beats the listed Claude Opus 4.6 Max result on SWE-bench Pro and LiveCodeBench, although Opus remains ahead on Terminal-Bench, GPQA Diamond and Humanity’s Last Exam. Some of Alibaba’s evaluations are internal, and benchmark harnesses are not identical across every comparison, making the numbers poor grounds for declaring a universal winner.Third-party results show a powerful, local model with performance equivalent to proprietary models from months agoThe conversation changed Monday when third-party results began arriving.Third-party AI benchmarking outfit Artificial Analysis gave Qwen3.8-27B a score of 52 on its Intelligence Index, a composite of nine evaluations spanning coding, science, reasoning and professional tasks. That happens to be the same score Artificial Analysis currently assigns OpenAI’s mid-tier model GPT-5.6 Luna at its maximum reasoning setting — a proprietary offering only available over the cloud. As open source coding agent Cline put it on X: “This is the first time a local model has scored frontier model capability. We weren’t expecting this pace of local progress anywhere near this soon.”On Artificial Analysis’ Agentic Index measuring model performance on agentic tasks, meanwhile, Qwen3.8-27B scored 51, beating Claude Opus 4.8 on maximum reasoning effort — a frontier model Anthropic released less than three months ago. That doesn’t mean these models are equivalent, but it helps explain why developers and AI power users stood up and took notice. As developer and AI podcaster/YouTuber Sero (@0xSero on X, real name Sharif Cherf) wrote on X: “A model that runs on 3k USD of hardware is beating everything from 4 months ago. Including Opus. Permanent underclass is cancelled.” Developer Joshua “Xenova” Lochner, known for bringing machine-learning models into web browsers, highlighted the result Monday on X alongside an experiment running Qwen3.8-27B with custom WebGPU kernels. His reaction — “What a time to be alive!” — captures much of the mood: a model scoring in the vicinity of proprietary frontier systems can be downloaded, modified and executed locally rather than accessed only through a vendor API.The appeal becomes clearer when the model is compressed. Developer and AI writer Simon Willison tested a roughly 17GB Q4_K_M quantization on an M5 Max MacBook Pro and Nvidia DGX Spark. He found that it could write code, interpret images and operate a coding-agent loop through the Pi agent framework. In one experiment, the model navigated a codebase to explain how authentication worked; in another, it wrote and tested a Python utility Willison needed to convert an agent transcript from JSONL to Markdown.“The fact that a 17GB file can do all of this stuff on my home machines is a miracle,” Willison wrote. His broader point is the one resonating with power users: capabilities that recently felt inseparable from expensive hosted models are moving into files small enough to keep on a workstation.The reaction is showing up in usage as well. Cybernews reported Monday that Qwen3.8-27B passed 3 million Hugging Face downloads in its first three days, while quantized versions rapidly appeared for local inference tools. The LocalLLaMA community on Reddit created a dedicated release megathread simply to consolidate the flood of benchmarks, quantizations, configuration advice and comparisons. One user showing a locally generated game described the model as “a different beast.”Overthinking is an issueThat frenzy comes with an important caveat: Qwen3.8-27B appears to buy some of its quality by thinking a lot.Artificial Analysis says the model generated 160 million output tokens across its Intelligence Index testing, versus a 43 million median for comparable open-weight models. Willison encountered an extreme version of the same behavior because Qwen defaults to its xhigh reasoning setting. A request to generate an SVG of a pelican riding a bicycle took 21 minutes and consumed more than 22,000 reasoning tokens before producing the answer. He recommends starting with low or no reasoning for ordinary local use.Investor and developer Tomasz Tunguz found a similar trade-off in a small nine-task test against DeepSeek V4 Flash: with reasoning enabled, Qwen edged ahead on quality in his agent stack, but he reported that it was roughly 30 times slower and 4.5 times more expensive. He explicitly cautioned that nine tasks were not enough for a verdict.Inference software may narrow that gap. Qwen3.8-27B includes Multi-Token Prediction, and Willison reported about a 72% performance improvement on his DGX Spark after enabling MTP through llama.cpp compared with his default LM Studio configuration. Even then, his normal LM Studio runs were producing only around 15 to 30 tokens per second — far below the responsiveness of many hosted models.That tension is precisely why Qwen3.8-27B matters more than another leaderboard position.What enterprises should take away from Qwen3.8-27BFor enterprises, the relevant comparison is not simply whether a 27B model “beats” Claude or GPT on a benchmark. It is whether a model small enough to run inside an organization’s own infrastructure can now perform enough coding, document analysis, vision and agent work to replace API calls for meaningful classes of tasks.That proposition changes privacy, deployment and cost calculations. Apache 2.0 weights can be inspected, modified and hosted behind a company’s own controls, while Alibaba already documents compatibility with serving frameworks including vLLM, SGLang and TokenSpeed. Alibaba says a managed Qwen Cloud version with a 1-million-token default context and built-in tools is coming later.The small size and accessible hardware requirements mean that enterprises, indie developers, and even curious consumers can easily deploy the model locally without worrying about their data leaving their machine — ensuring greater privacy, information security, governance and control.There is a broader reason power users are paying attention. Hugging Face data reported by Business Insider this week shows that actual model usage skews dramatically toward smaller models even as enormous frontier releases dominate headlines; models above 70 billion parameters accounted for only a small share of 2026 downloads. Alibaba’s strategy of publishing Qwen models across multiple practical size classes has helped make the family a recurring part of developers’ local deployment workflows.Qwen3.8-27B pushes that logic further. Its benchmark scores still need more independent validation, its default reasoning behavior can be painfully inefficient, and no single leaderboard establishes frontier-model parity.But three days after release, developers are no longer reacting primarily to Alibaba’s benchmark table. They are reacting to the experience of putting a comparatively small file on hardware they control and watching it perform tasks that, not long ago, seemed to belong exclusively to the largest proprietary systems.For certain developers, AI power users—and yes, even enterprise deployments—that is the benchmark that matters most.

Cursor began rolling out Origin, its own code hosting platform, to paid users on Monday morning. Roughly three and a half hours later, GitHub’s status page lit up with what became a six-hour-and-forty-two-minute global degradation — error rates near 20% across pull requests, issues and the API, and near 50% on archive and raw file downloads, according to GitHub’s incident log. Enterprise single sign-on went down with it: SAML, OIDC, SCIM provisioning and Team Sync all failed. So did Copilot.The developer internet did what the developer internet does.”You can now host your repos in Cursor Origin and deploy to Vercel via Cursor Origin which is itself hosted on Vercel,” Vercel chief executive Guillermo Rauch posted on X. “And unlike GitHub, it’s online 😁” Asked why he was smiling, Rauch replied: “trying to make light of the situation. We ourselves are stuck because of github rn!”Matt Palmer, who works at Cursor, quote-tweeted his own company’s launch with the day’s best line: “We were going to ship this earlier, but GitHub was down.” A GitHub outage, in other words, delayed the launch of a GitHub competitor.Product launches get locked weeks in advance, and no evidence suggests Cursor timed this one. But the coincidence did the company an enormous favor, because it dramatized the argument Origin exists to make. For eighteen years, choosing where to host your team’s source code has been the least interesting decision an engineering organization makes. Cursor is betting that AI agents have made it interesting again — and for technical decision makers, that is the real news here. Not a new product, but a new procurement question with a governance problem attached.Inside Origin: what Cursor’s code hosting platform actually doesOrigin lives in a new Codebase tab inside Cursor. Teams name a codebase, which becomes part of its URL, then push to it over the command line. From there they get the machinery you would expect from a forge — the service layer that wraps Git and handles storage, permissions, checks and merges. Every repository comes with pull requests: timelines, commits, checks and files changed. Reviewers read the diff, leave comments and merge, without ever opening a browser tab.What Cursor built around that machinery is the part worth studying. Agents now operate in the same surface as the code and the pull requests they are modifying. “Your code, PRs, and agents are now in the same place,” the changelog reads. A developer can ask questions about the file on screen, hand an agent a review comment and have it revise the pull request in place, or tell it to push a branch — all inside the editor where the code was written.Three integrations shipped on day one, and the choice of partners is telling. Vercel spins up a preview deployment for every pull request and ships to production on merge, available in public beta for Pro and Enterprise customers, its developer account said. Depot and Buildkite run continuous integration, and critically, both execute existing GitHub Actions workflows unchanged. Buildkite adds native pipelines on top.That compatibility layer is the whole strategy in miniature. Cursor is not asking teams to rewrite their build system, retrain their engineers or rip out their deployment pipeline. It is asking them to try a second window onto code they already have — which is a far easier request to approve.More partners are coming, the company said, and the ones it landed first are the ones that matter to a platform team evaluating whether Origin can carry real work. A forge without deployments and CI is a code viewer. A forge that runs your existing Actions workflows and ships previews to the CDN you already pay for is a candidate.Why letting GitHub stay the source of truth is Origin’s smartest design choiceHere is the decision enterprise buyers should study most closely, because it determines whether Origin survives a security review at all.Cursor does not ask you to leave GitHub. Connect a GitHub organization, pick repositories, and they appear alongside Origin-native ones. “Pushes keep going to GitHub, which stays the source of truth for anything started there,” the changelog says. Access permissions mirror GitHub’s existing read and write settings rather than establishing a parallel system. Pull request conversations sync in both directions — comment in Cursor and it posts to GitHub; reply or react on GitHub and it surfaces in Cursor “within seconds.”This is a classic wedge, and a well-executed one. Rip-and-replace migration of source control ranks among the highest-risk projects an engineering organization can undertake. It touches continuous integration, compliance evidence, audit trails, branch protection rules, every integration in the toolchain and the muscle memory of every engineer on staff. Almost no chief technology officer approves that for a product in early beta.A read-mostly mirror that leaves GitHub authoritative approves itself. It costs nothing to try, breaks nothing if abandoned, and quietly relocates the place developers spend their working hours. If Cursor’s review experience proves better — and Cursor spent real money to make sure it would — the source of truth eventually follows the attention.That money went to Graphite, the code review startup Cursor bought in December 2025 for what Axios reported was well above its $290 million Series B valuation. Graphite built stacked pull requests, the workflow that lets developers keep shipping dependent changes without waiting on approvals. Announcing the deal, Cursor wrote that “the boundary between where you write code and where you collaborate on it feels increasingly arbitrary,” and promised “some more radical ideas we can’t share just yet.” Origin is the radical idea. Graphite co-founder Tomas Reimers unveiled it on stage at Cursor’s inaugural Compile conference in June and leads its development.How AI agents turned code review into software’s new bottleneckThe case for an agent-native forge rests on a claim that is easy to state and, unusually for this market, well supported by evidence: writing code stopped being the constraint. Reviewing and integrating it became one.Google’s 2025 DORA report, drawn from nearly 5,000 technology professionals, found that 90% of developers now use AI at work, spending a median of two hours a day with it, and more than 80% say it made them more productive. But AI adoption showed a positive relationship with software delivery throughput and a negative one with delivery stability. More output, more breakage. The report’s authors describe AI as “an amplifier” that “magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones.”Trust has not kept pace with volume. Stack Overflow’s 2025 developer survey of 49,009 respondents across 177 countries found 84% using or planning to use AI tools, while trust in their accuracy fell to 33% from 43% a year earlier and distrust climbed to 46% from 31%. Two-thirds named “AI solutions that are almost right, but not quite” as their leading frustration. GitLab’s ninth annual DevSecOps survey, of 3,266 practitioners polled by Harris, put numbers on the operational drag: 73% had hit problems with vibe-coded output, 70% said AI made compliance management harder, and only 37% would let AI handle daily tasks without human review.The volume climbs regardless. GitHub’s Octoverse 2025 counted 180 million developers, 630 million repositories and 43.2 million pull requests merged per month, up 23% year over year. And RuntimeWire reported the internal figure that best explains Origin’s existence: 35% of pull requests merged inside Cursor were opened by agents running autonomously in cloud virtual machines.A forge built for humans assumes a pull request represents human intent, opened by someone you can ask what they meant. Once a third of merged changes come from software, the queue stops being a conversation and becomes a scheduling problem. That is a real architectural argument, and it is the strongest thing Cursor has going for it.GitHub’s reliability crisis handed Cursor an opening it did not have to earnThe supply-side case for an alternative is simpler: GitHub has been unreliable, and its own executives have said so.An analysis by LeadDev counted 257 incidents between May 2025 and April 2026, 48 of them major — roughly one significant disruption per week. February was the worst month on record with 37. GitHub Actions alone accounted for 57 outages in twelve months. Chief technology officer Vlad Fedorov has said the platform “wasn’t built for the scale it’s now being asked to handle” and must design for 30 times today’s load. In an April engineering post covered by InfoQ, the company acknowledged it “failed to meet its own reliability standards,” citing rapid growth, tight architectural coupling and inadequate load shedding. Monday’s outage was the seventh incident on GitHub’s status page in fifteen days.The fatigue is audible. “GitHub really doesn’t feel built for the agent era,” one developer wrote on X as Origin went live. “It goes down way too often, but until now there haven’t been many real alternatives.”The defections started before Origin existed. The Zig programming language moved to Codeberg in November 2025, citing Actions failures among its reasons. In April, Mitchell Hashimoto announced that Ghostty — a terminal emulator with more than 52,000 stars — would leave too, pointing to near-daily outages that blocked reviews and CI for hours. And The Information reported in March that OpenAI, a company Microsoft holds a large stake in, began building its own GitHub alternative partly because outages left its engineers unable to commit for hours at a time, as Tom’s Hardware relayed.Microsoft’s structure has not helped. Thomas Dohmke resigned as GitHub chief executive in August 2025 and was never replaced; the unit’s leadership was absorbed into Microsoft’s CoreAI organization under executive vice president Jay Parikh. In a May report, The Information wrote that Parikh had warned deputies that coding tools from Cursor and Anthropic could eventually make GitHub obsolete. GitHub’s own answer to the agent era, Agent HQ, lets customers orchestrate third-party agents from Anthropic, OpenAI, Google, Cognition and xAI inside GitHub — a coherent strategy that concedes the agent layer and keeps the substrate underneath. Origin attacks precisely that substrate.Now that SpaceX owns Cursor, who actually holds your source code?Cursor’s rise has been extraordinary even by the standards of this cycle. Founded in 2022 by four MIT students, Anysphere raised $8 million from the OpenAI Startup Fund in October 2023, per TechCrunch, then $100 million at $2.5 billion, $900 million at $9.9 billion, and $2.3 billion at $29.3 billion last November. In May, Bloomberg reported annualized revenue of $3 billion and more than 3,000 customers paying at least $100,000 a year.Then, three days before Origin shipped, Bloomberg reported that SpaceX completed its $60 billion all-stock acquisition of Cursor — an agreement TechCrunch covered in June, days after SpaceX’s record IPO and six months after it absorbed xAI. Cursor now operates inside a division called SpaceXAI. The vendor asking to hold your proprietary source code became, last Friday, a unit of a rocket company with its own frontier-model division and a founder not known for institutional caution.Jason Andersen of Moor Insights & Strategy raised the model-routing question to Tech Times in June, before the deal closed: “xAI’s models and treatment of guardrails are very different than what Cursor has stood for.” That piece framed the question a chief information security officer now has to answer. When one company controls the editor where agents write code, the host where that code lives and the model those agents run on, what governs what it does with the code?Cursor has not published an answer. RuntimeWire noted before launch that Origin’s pricing, security architecture, data-handling terms and migration tooling were all unpublished, and Monday’s changelog adds none of them. It says only that Origin reaches “all paid plan users starting today, except enterprise orgs whose admins opt out.” Opt-out, not opt-in — a sentence administrators should read twice.There is also a track record to weigh. In July, researchers at Mindgard disclosed that Cursor would execute a malicious git.exe planted in a Windows project’s root the moment a user opened it, with no prompt — a repository-poisoning flaw they first reported in December 2025. The Hacker News reported that Cursor declined to patch it, calling the issue out of scope under a shared-responsibility model while conceding it had not “closed the loop with the researcher in a timely manner.” No CVE was issued. The same flaw class turned up unpatched in GitHub Copilot CLI, Google’s Gemini CLI and OpenAI’s Codex — but a vulnerability the vendor declined to fix makes an awkward footnote for a product whose pitch is basically “let us hold your repositories.”What engineering leaders should settle before they let Origin into the toolchainOrigin is a beta, not a migration, and treated as one it is worth evaluating. The sync mode gives platform teams a low-risk way to measure whether an agent-native review surface shortens cycle time, without touching a single branch protection rule. But three things deserve resolution before anything authoritative moves.The first is the default. Origin switches on for paid users unless an enterprise administrator opts out, which means an organization that has not made an affirmative decision about whether proprietary code may be mirrored to a new host has effectively had that decision made for it. Confirming your posture is a Monday-morning task, not a next-quarter one.The second is the paperwork. Retention, residency, training use, subprocessors and what changes now that Cursor reports into SpaceX are all unpublished, and a product page is not a contract. Until those terms exist in writing, the defensible position is to treat Origin as a convenience layer over GitHub rather than a system of record — which is, conveniently, exactly what its architecture already is.The third is the exit. Origin’s Actions compatibility and its GitHub-as-source-of-truth design are the properties that make it safe to adopt. They are also the ones most likely to erode as Cursor’s incentives shift toward owning the substrate rather than borrowing it. Ask what egress looks like now, while the mirror is still a mirror.None of which makes Cursor’s argument wrong. GitHub earned its incumbency by being boring, dependable infrastructure, and it has spent eighteen months being neither while a third of the code arriving at its front door stopped being written by people. Origin is a serious answer to a real problem, built by a team that bought the right company to build it.But GitHub’s failure and Cursor’s are different in kind, and enterprises should not confuse them. Monday’s outage resolved at 20:22 UTC. Availability is an engineering problem, and engineering problems close. The question of who holds your source code, what they may do with it and who they ultimately answer to carries no such timestamp — and on that one, the company that spent Monday selling trust has yet to publish its terms.

A retrieval-augmented generation (RAG) system is built to answer strictly from the documents it retrieves. But when engineers optimize these AI pipelines end-to-end, the reader module can learn a shortcut: instead of relying on retrieved evidence, it starts answering from its own internal memory — while the system’s overall accuracy keeps climbing. This is the hidden challenge of “role drift,” a failure mode in compound AI systems where individual modules learn to bypass their assigned tasks even as end-to-end performance improves.To address this, researchers at MIT and Harvard introduce Role Anchor, a technique that forces modules to stay in their lanes during training. When applied, the technique mitigates role drift. For example, it forces the RAG reader to rely on retrieved evidence instead of answering based on its internal knowledge.The primary takeaway for practitioners is that end-to-end accuracy alone can overstate how much a compound AI system has genuinely learned. Engineers must evaluate individual components and ensure they work as intended.Role Anchor serves as both a guardrail and a diagnostic tool when optimizing multi-step LLM pipelines. It can be essential for real-world AI applications that require a strict division of labor between modules.Why terminal accuracy hides the problemCompound LLM systems divide complex tasks among specialized modules. For example, a system designed for multi-hop reasoning might split a task between a “Decomposer” and a “Solver.” The Decomposer breaks a large problem down into manageable sub-tasks, while the Solver computes the answers to those sub-questions. This division of labor allows AI engineers to delegate execution to smaller, cheaper models, and makes it possible to process sub-tasks in parallel where possible.To improve the performance of AI pipelines, engineers typically optimize them using end-to-end reinforcement learning (RL) guided by a single “terminal reward.” This means the system is evaluated on whether or not the final answer is correct (the researchers call it “terminal accuracy”). When this terminal accuracy goes up, the system is considered to be learning and working as intended.However, terminal accuracy does not verify whether the modules properly executed the tasks they were assigned. As Xiaoyang Cao, co-author of the paper, told VentureBeat, “Terminal accuracy reduces the behavior of an entire multi-part AI system to a single number. It shows whether the final answer is correct, but says little about which components contributed or whether they followed their assigned roles.”This blind spot leads to role drift, a failure mode where a module’s behavior diverges from its assigned role during optimization, even though the system’s terminal accuracy continues to improve. “For engineering teams, the practical risk is that they can deploy a pipeline that passes every end-to-end evaluation even though its intended division of labor has silently broken down,” Cao said. Because the reward system only scores the final answer, it fails to detect or penalize the module for going rogue.Consider how this happens in the Decomposer-Solver pipeline. The Decomposer’s assigned role is to write abstract sub-questions without solving the task, leaving the reasoning to the Solver. Under end-to-end RL, the Decomposer quickly learns that the weaker Solver is prone to errors on abstract tasks. To maximize the reward, the Decomposer begins leaking or planting answers into the sub-questions it sends to the Solver. The Solver ends up parroting the answer the Decomposer fed it. Terminal accuracy goes up, but the intended architecture is compromised.But if the system is getting the right answers and accuracy is going up, why should we care if a module drifts from its role?Real-world deployment requires much more than just a correct final answer on a training dataset. The implicit roles assigned to these modules ensure scalability, reliability, and auditability. Consider what happens when role drift takes over:Loss of efficiency and auditability: In the reasoning example, role drift causes the Decomposer to do all the heavy lifting instead of planning and delegating. “Once the decomposer starts putting answers directly into its sub-questions, the solvers are reduced to copying those answers,” Cao said. “You are still paying to run [different modules], but they are no longer doing independent work.” The workload can no longer be parallelized across multiple Solvers, it cannot be delegated to cheaper models to save compute, and downstream human stakeholders can no longer audit the system’s logic step-by-step to verify how it arrived at the answer.Fragility in dynamic environments: Consider a RAG system, in which a Reader model is tasked to answer questions strictly using external retrieved documents. If the Reader drifts and learns to rely on its own internal parametric memory instead (because its memory happens to be accurate during training), the system becomes brittle. When the enterprise updates its database with new information, or a user asks a question about a novel topic outside the model’s pretraining, the system will fail because it abandoned the grounding mechanism it was built to use.How Role Anchor measures a role — and enforces it”Training only for the final outcome rewards a system for producing the right answer, regardless of how it gets there,” Cao said. To counter this, Role Anchor serves as a lightweight regularization technique that makes role instructions part of the training objective. It compares how the component behaves with and without those instructions and discourages training from weakening their effect. At a high level, it ensures the module continues to respect the steering influence of its original role prompt throughout the reinforcement learning optimization process, making role drift both measurable and controllable.A key insight of Role Anchor is that a role’s effect can be measured by comparing how a model behaves with and without the role prompt. The system evaluates two different prompts for each module:The specialized, instruction-heavy role prompt (e.g., “You are a careful Reader. Use the retrieved passages to answer the user’s questions…”).The neutral prompt (e.g., “Answer the user’s question…”).For any given input, the model outputs a probability distribution for the next token. When run under the role prompt, it will favor certain tokens. When run under the neutral prompt, it behaves like a generic assistant. The difference between these two probability distributions is the “role utility.”This utility measures the ”nudge,” or the direction and strength with which the role prompt shifts the LLM’s default predictions. If a token is highly aligned with the assigned role, the role prompt boosts its likelihood compared to the neutral baseline (or “nudges” the model toward that token).Before starting RL training, Role Anchor keeps a frozen copy of the model as reference and measures the role prompt’s original nudge on this reference model. This pre-RL nudge serves as the ground truth of the designer’s intent, acting as a proxy for how the role prompt is supposed to steer the model.During RL training, as the active model’s weights are updated, Role Anchor regularly calculates the current nudge and compares it to the reference nudge. If the current nudge starts to fade or deviate from the reference, Role Anchor applies a penalty to the model to prevent role drift.To see this practically, consider the RAG system evaluated by the researchers. In this pipeline, the Reader module is explicitly instructed to answer user questions based only on retrieved documents, rather than relying on its internal knowledge.During unconstrained, outcome-only RL, the reader learns that the upstream retriever is sometimes noisy. To maximize accuracy on the training set, it starts ignoring the retrieved passages and answering from memory. Consequently, the gap between its behavior under the role prompt and the neutral prompt shrinks to the point that the reader starts behaving identically under both, ignoring the grounding instructions.In contrast, Role Anchor detects when the reader’s nudge deviates from the reference nudge. It applies a penalty, redirecting the model’s parameters away from this memory-based shortcut. This forces the reader to find role-compliant ways to improve, such as learning how to extract answers from the retrieved passages more robustly or avoiding using its internal knowledge when the retrieved passages are faulty.The numbers: how much of the accuracy gain was realTo test the efficacy of Role Anchor, researchers evaluated it on the RAG and Decomposer-Solver (DEC) pipelines. The experiments compared systems trained with standard outcome-only reinforcement learning (no anchor) against systems trained with Role Anchor.Under outcome-only RL, the RAG system’s terminal accuracy rose, but its internal integrity collapsed. The researchers measured “Evidence-Following Accuracy,” a probe testing if the model changes its answer when the retrieved text is deliberately swapped to state the opposite. This metric plummeted from 0.86 to 0.54 (just above random chance), meaning the model learned to ignore retrieved passages and rely on its pre-trained parametric memory instead. In one test, researchers deliberately changed a piece of information in a retrieved document to contradict the model’s internal knowledge. The unanchored model did not update the response because it wasn’t using the external document.When Role Anchor was applied, the Reader’s Evidence-Following Accuracy remained at 0.869, proving it relied strictly on the retrieved text. When researchers fed the anchored model random passages that were unrelated to the input prompt, its accuracy correctly dropped because it refused to use its internal knowledge. The unanchored model scored higher on random passages because it was guessing from memory.The Decomposer (DEC) pipeline showed an even more dramatic failure mode. Under outcome-only RL, terminal accuracy shot up, but the “insertion rate” (i.e., the frequency at which the Decomposer leaked the answer into the sub-questions it sent to the Solver) surged from 0.143 to 0.596.In the RAG pipeline, preserving the intended role cost the system a very modest accuracy drop (-0.067). The Reader still learned to be better at extracting answers, but it did so legitimately rather than by cheating with its internal memory. This means it is more reliable on real-world tasks with novel knowledge it has not seen during training.In the DEC pipeline, unanchored RL improved accuracy by 0.310 above the base model, while Role Anchor only showed a 0.057 improvement. When diagnosed, it turned out that the underlying issue was that the Solver model was too small and couldn’t learn the problem-solving part. This forced the Decomposer model to cheat and provide the answer to boost the terminal accuracy. This meant 86% of the unanchored improvement was fake, and the system had simply learned to exploit a shortcut instead of learning how to reason or decompose problems better.However, this tradeoff is not a universal rule. In some cases, eliminating shortcuts can actually boost overall performance. “Role Anchor… does not necessarily reduce final accuracy,” Cao said. “In a coding pipeline we recently tested, the model had learned to manipulate its own test executor during reinforcement learning training. Adding Role Anchor completely eliminated that shortcut while slightly improving correctness on the final tests used to judge the code.”What it takes to add Role Anchor to an existing pipelineFor engineering teams looking to apply this technique, “Role Anchor can be added to an existing reinforcement learning fine-tuning process as an extra training objective for each component that a team wants to anchor,” Cao said. The main pipeline and deployment setup remain entirely unchanged.To implement it, engineers need three specific items for each anchored component: its original role instructions, a matched neutral version with the role information removed, and a saved copy of the model from before reinforcement learning fine-tuning.Importantly, there is no latency penalty at inference time. “Role Anchor runs only while the model is being trained, so it does not slow down the deployed system,” Cao said. He noted that their current implementation takes roughly 20 percent longer during training due to additional calculations, though there is likely room to optimize and reduce that overhead. The research code, training configurations, and selected model weights will be released publicly in the near future.Deciding when to use Role Anchor is a case-by-case decision based on whether final accuracy captures everything that matters. Cao points to a regulated legal RAG system as a prime candidate. “The component producing the answer may need to follow retrieved evidence, stay grounded in an approved set of documents, and produce answers that can be traced back to their sources,” he said. “Final accuracy alone cannot verify those properties, so the behavior of that component needs to be measured and enforced directly.”As enterprise AI evolves toward more complex compound pipelines, role enforcement will become harder, and relying on prompts alone will prove unreliable. “At larger scales, role specifications will need to be enforced through both training and system design,” Cao said. “Methods such as Role Anchor can help preserve intended behavior during training, while clear system boundaries, limited tool permissions, and monitoring during use can provide additional safeguards.”

A company builds a governed context layer specifically to stop its AI agents from confidently giving wrong answers. Once that layer is live, the company is more than twice as likely to report the failure happening — not less.In the past six months, 68% of enterprises have traced a confident but wrong AI agent answer to missing or inconsistent business context. Thirty-seven percent say it happened more than once, ahead of the 32% who saw it happen only once. The figures come from a VB Pulse July 2026 survey of 101 qualified enterprises with more than 100 employees. That’s up from 57% in a VB Pulse survey conducted in June. Recurring failures climbed too, from 31% then to 37% now.This is the second time VB Pulse has asked enterprises this exact question, once in June and now in July. The failure rate is climbing, not falling, even as more enterprises report a governed layer in production, up from 25% in June to 32% now.How agents get context determines whether they’re wrongEvery AI agent needs some way to know what the business actually means, whether a metric is defined consistently, whether a document is current. That’s the operation. The challenge is that enterprises hand agents that context in very different ways, and those ways are not equally reliable.Retrieval over documents remains the most common approach, the primary source for 31% of enterprises. But a real share of enterprises skip a structured approach altogether. Thirteen percent run agents primarily on long-context loading, feeding documents directly into the model’s context window rather than retrieving them. Five percent give agents no structured context at all, just the model’s general knowledge. Between them, nearly one in five enterprises are feeding agents business context by brute force or not feeding it at all.Even the leading approach can still produce a confidently wrong answer. Retrieval works by matching a question to text that looks similar in meaning. Similar wording doesn’t guarantee the same meaning. Srijith Rajamohan, an AI research leader at Redis, described exactly this gap in an interview with VentureBeat earlier this year. “If you have a sentence like ‘Rome is closer than Paris’ and another that says ‘Paris is closer than Rome,’ and you do an embedding retrieval followed by a text search, you’re not going to be able to tell the difference,” Rajamohan said. “The same words exist in both sentences.”Buying shifted to access control. Grading didn’t follow.The way enterprises choose a retrieval system doesn’t help close the gap. Access control and permissions now tie ease of data ingestion as the top selection criteria, at 24% each. It’s the first time in this survey series that a governance property has led to the buying decision. Retrieval accuracy trails at 15%. The property most directly tied to a confident wrong answer isn’t the property most enterprises are buying for.Once a system is running, correctness is still how enterprises judge it. Response correctness is the primary success metric for 38% of enterprises, twice the next closest answer, security and access control at 19%. Enterprises are shifting how they buy toward governance. They’re still grading success on whether the answer is right.The companies fixing this are the ones reporting it worstA governed context layer is meant to fix this. It’s one shared, agreed-on model of what the business’s data means, that every agent and BI tool references instead of guessing on its own. Adoption is far from settled.Thirty-two percent of enterprises run one in production. Thirty-one percent are piloting or building one right now. Twenty percent are evaluating one. Fourteen percent have no plans to, and 4% don’t know.Compare that adoption data against who’s actually had the failure, and the picture inverts. Among the 91 enterprises able to say whether they’d experienced the failure at all, those running or building a governed layer report it recurring at 50%. Those without one report it at 21%.A governed layer doesn’t cause the failure — it’s what makes the failure visible in the first place. Tracing a bad answer to a broken definition or a stale table requires a shared, governed reference point. A context layer provides that. Without one, the same wrong answer still happens — it just gets chalked up to the model, or never gets traced at all.The pain point predates AI by decades. Kyle Nesbit, founder of the semantic layer startup Credible Data, described it to VentureBeat last month. “It’s the same pain point people have had for 30 years, the lack of governed data analysis,” Nesbit said. “Now with AI, it’s the same problem, but orders of magnitude more chaos and pain.”Company size sharpens the same point. Enterprises with more than 1,000 employees report recurring failures at 55%, against 30% for those between 101 and 1,000 employees. That’s despite the bigger companies being less likely to have a layer already in production, 24% against 37%. More instrumentation and more people asking why a number was wrong turns up more failures, not fewer. A clean record is not evidence of a healthy context layer. It’s at least as likely to be evidence that nobody’s checking.What this means for enterprisesHere’s what this adds up to for enterprises building on this layer.Retrieval alone will not close the context gap. RAG remains the default context source, and nearly one in five enterprises are running agents on long-context loading or no structured context layer at all. More documents or a bigger index doesn’t fix a definition that means two different things in two different systems.The budget is moving faster than the infrastructure is shipping. Sixty-three percent of enterprises are already building or running a governed context layer. Only 32% have actually gotten one into production. That gap is where the spend is going, not where the problem has been solved.A clean failure record is a red flag, not a green one. The 22% of enterprises reporting no context failure at all are not the best-governed group. They’re the group least likely to be checking. The size data backs this up directly. Larger enterprises report recurring failures at nearly twice the rate of mid-market peers, despite being less likely to have a governed layer in production, not more.No one is planning to hand the layer to a single provider. Seventy-nine percent of enterprises intend to keep at least part of the context layer outside any one vendor’s stack, split between best-of-breed tools and an explicit mix. Just 12% plan to consolidate onto a single provider’s native context stack.The finding that organizations aren’t likely to hand over control to a single provider is a theme that VentureBeat has reported on consistently this year. Michael Ni, an analyst at Constellation Research, put it bluntly earlier this year when DataHub’s context layer push first landed. “Whoever controls runtime context, controls the AI decision layer for enterprise data,” Ni said.