How Coding Agents and Frontier Models Reshaped What’s Possible in q

A year ago, frontier language models struggled to write working q code. Today, when paired with modern coding agents, they solve more than 85% of the problems in KX’s Q Evaluation Harness (QEval), with Anthropic’s Claude Fable 5 exceeding 95%. The state of the art has advanced rapidly, and conclusions drawn from experimenting with LLMs in 2025 no longer reflect what is possible today.

This article reviews that progress through the lens of benchmark results and practical experience building production systems. We examine three questions.

1. Can current models understand and write q?
2. Can they solve useful capital-markets programming tasks?
3. Can they produce expert-quality, high-performance q?

The evidence shows substantial progress and clearly defines where the remaining challenges lie.

Why q Has Been a Difficult Language for LLMs

The relatively poor performance of frontier models on q in 2025 was not unexpected. q is an outlier among modern programming languages, and many of its defining characteristics differ from the languages that dominate public training corpora.

q’s right-to-left evaluation, heavy operator overloading, transparent vectorization, and terse syntax all demand precise contextual reasoning. Small misunderstandings can fundamentally change a program’s behavior. Compared to languages such as Python, Go, and JavaScript, there is also far less publicly available q code for models to learn from. Most production q code is proprietary, often implementing valuable financial algorithms that never become part of public training data.

These factors created a genuine challenge for language models. Even when generated code looked plausible, subtle semantic errors were common. The benchmark results published in 2025 confirmed what many developers had already observed in practice.

The 2025 Baseline

In August 2025, KX published QEval, the first open benchmark for evaluating LLM proficiency in q. It established a reproducible baseline for measuring progress in a language that had previously been evaluated largely through anecdotal experience.

At the time, even the strongest models achieved only 74% pass@10 on a benchmark that had already become largely saturated for Python code generation. q remained a genuine challenge.

That baseline proved valuable over the following year. Rather than relying on subjective impressions of improvement, we could measure how successive advances in coding agents, frontier models, and domain-specific tooling changed the state of the art.

Coding Agents Advanced the State of the Art

The largest advance over the past year came not from a new model, but from a new way of using them.

Modern coding agents combine frontier reasoning models with iterative execution, tool use, and the ability to refine their own work over multiple turns. That combination fundamentally changed what was possible for q. Problems that routinely failed with one-shot code generation became tractable when models could execute code, inspect the results, and revise their approach.

The change first became apparent in day-to-day engineering work. During the development of the Scalable Market Data Ingestion Blueprint and components of the Tick-X Reference Architecture, coding agents demonstrated a level of fluency in q that simply had not been possible a year earlier. They still required expert guidance and careful review, but the interaction itself had changed. Rather than repeatedly correcting basic misunderstandings of the language, engineers could increasingly collaborate with the agent on larger implementation tasks while providing architectural direction and validating the resulting code.

Those experiences suggested that something fundamental had changed, but anecdotal success was not enough. We extended QEval with an agent backend capable of evaluating coding agents directly. Each benchmark problem was executed within a bounded coding session using the same grading criteria as the original benchmark, allowing us to compare one-shot code generation with agentic development under controlled conditions.

The results confirmed what we had begun to observe in practice. Pairing frontier models with coding agents raised benchmark performance from roughly 50% for one-shot generation to more than 80%. The dominant limitation was no longer simply the model’s knowledge of q syntax or semantics. Giving models the opportunity to reason iteratively, execute code, inspect failures, and refine their solutions produced the largest single advance we measured over the past year.

Skills Closed Many of the Remaining Gaps

The improvements delivered by coding agents also made the remaining limitations easier to understand. Once models could reliably solve much of the benchmark, the failures became less random and more predictable. The same classes of mistakes appeared repeatedly, suggesting that many reflected missing domain knowledge rather than fundamental reasoning limitations.

Benchmark analysis and day-to-day engineering work pointed to a consistent set of failure modes. Some were straightforward and mechanical, such as invoking q incorrectly in a non-interactive environment. Others were subtle, including language-specific pitfalls like accidentally commenting out an entire source file with a single slash. Experienced q developers rarely make these mistakes because they are part of the language’s accumulated practice rather than its formal syntax.

Those observations led to a simple hypothesis: If these recurring patterns could be captured as reusable guidance, coding agents should perform more consistently.

We encoded that knowledge in a set of reusable q skills, now available as part of the open-source KX Claude Plugins. Rather than teaching the model new capabilities, the skills provide practical guidance drawn from experienced q development, helping the agent avoid common pitfalls and adopt established best practices.

The benchmark results supported that approach. Adding the q skills increased agent performance to approximately 85% while also making development sessions noticeably more productive. Instead of repeatedly rediscovering how to execute q scripts or navigate common language gotchas, the agents began each session with the practical knowledge an experienced q developer would arrive with.

The improvement was larger than the benchmark alone suggests. Eliminating repetitive corrections allowed engineers to spend more time on architecture, implementation, and validation, shifting the interaction further away from language tutoring and toward collaborative software development. It also reduced wasted token consumption at the start of each q coding session.

Frontier Models Continued the Progress

While coding agents accounted for the largest single advance over the past year, improvements in the underlying models continued to move the frontier.

The most striking example was Anthropic’s release of Claude Fable 5 in June 2026. Fable 5 scored more than 95% on QEval. That was another significant improvement over the previous generation of frontier models and suggested that q proficiency was becoming a native capability rather than something that depended entirely on supplemental guidance.

One result was particularly interesting. Because Fable 5 and Claude Opus 5 are so strong, adding our q skills provided almost no boost, possibly even introducing a small regression, although the change was small enough to fall within normal measurement variability. There appears to be something fundamentally different about this model, and we look forward to understanding it better as additional evaluation data becomes available.

Measuring the State of the Art

The past year produced improvements at every layer of the development workflow. Coding agents transformed what was possible with existing models. Domain-specific skills reduced recurring failure modes. Frontier models continued to raise the ceiling.

The detailed benchmark results are shown in Table 1.

Table 1: QEval leaderboard — coding agent and one-shot results

Applying These Techniques in Practice

Benchmarks provide an important measure of technical capability, but they are only one source of evidence. The more important question is whether those capabilities translate into real engineering productivity.

We applied the same tools and techniques while developing the Scalable Market Data Ingestion Blueprint and components of the Tick-X Reference Architecture. The objective was not to let an agent autonomously build a production system. Experienced q developers remained responsible for the architecture, design decisions, and technical review. The coding agents became collaborators within that engineering process. We also enabled the same q skills we used in the benchmarks.

The experience closely matched what the benchmarks predicted. Modern coding agents were able to produce fluent q, understand existing tick architecture, and implement substantial portions of the surrounding infrastructure. They were considerably more effective working within an established architectural direction than inventing one independently, making them well suited to accelerating implementation while leaving higher-level design decisions to experienced engineers.

Validating code, more than writing it, was one of the agents’ most valuable contributions. They proved particularly effective at generating comprehensive end-to-end tests, verifying expected data flow, exercising API boundaries, and identifying edge cases that would otherwise require significant manual effort. In practice, this often saved more engineering time than code generation itself.

The benchmark results suggested that modern models had become capable q programmers. Practical engineering work demonstrated where those capabilities created the most value.

Where the Expert Still Wins

The progress over the past year has been substantial, but it would be a mistake to conclude that expert q developers have become unnecessary.

The benchmark results and our practical experience point to the same conclusion. Current models understand q well enough to solve a broad range of programming tasks, especially when paired with coding agents and domain-specific guidance. They remain less reliable when confronted with the kinds of architectural tradeoffs, sophisticated financial algorithms, and complex distributed systems that characterize production capital-markets software. In fact, we are working internally with a new, more difficult benchmark which raises the bar and covers more of these complex problems, including qsql generation for common complex analytics use cases. The state of the art models are only able to succeed at about 80% of this harder set. We continue to work on tools and models to increase their effectiveness, and we’ll share results as they’re available.

One example from the Blueprint work illustrates the distinction. While the agents readily implemented stream processing components and generated comprehensive validation suites, they struggled to design a deferred synchronous gateway that correctly coordinated queries across multiple processes. That problem required architectural reasoning and experience beyond what today’s models consistently demonstrate.

Those limitations are important because they define where expert developers continue to add the greatest value. Modern coding agents are increasingly effective implementation partners, but they do not yet replace the judgment required to design complex systems, choose among competing architectures, or recognize when an apparently reasonable solution is fundamentally flawed.

Conclusion

This article began with three questions.

Can current models understand and write q?

Yes.

Can they solve useful capital-markets programming tasks?

Increasingly, yes.

Can they consistently produce expert-quality, high-performance q for the most demanding engineering problems?

Not yet.

Those answers are supported by three independent sources of evidence. Benchmark results demonstrate that modern models have made remarkable progress in understanding and generating q. Practical engineering work shows that those capabilities translate into meaningful productivity gains on real systems. At the same time, both the benchmarks and the case studies identify clear areas where experienced developers continue to outperform today’s models.

The past year fundamentally changed what is possible with AI-assisted q development. Coding agents produced the largest single advance, domain-specific skills improved consistency and developer productivity, and successive generations of frontier models continued to move the state of the art forward.

If your assessment of LLMs for q is based on experiments from 2025, the evidence now points to a different conclusion. Modern coding agents have become capable engineering tools for q development, and the frontier continues to move quickly. Check out the Q Evaluation Harness, an open-source framework we developed for evaluating Large Language Models on Q/kdb+ code generation tasks.