Prove where agents can work without additional human review.
We work with enterprises on their software factories. We start with code review, then bug fixing, feature implementation and incident response, with your agents measured on real engineering outcomes.
An illustration of agent pull requests flowing through agent review, human review and production. Each defect is filed by the stage that caught it: 58% caught by the agent, 22% caught by a human and 20%escaped, with 4 failure modes found. As evidence accumulates, a lane opens for pull requests that are safe to merge without additional human review.
Incoming agent PRs
#4830fix: null check in payment retry
#4831feat: export invoices as CSV
#4832refactor: split auth middleware
#4833fix: race in session refresh
Safe to automerge
Agent reviewCaught by the agent58%
Human reviewCaught by a human22%
ProductionEscaped20%
#4829#4828#4827#4826#4824
EvidenceFailure modes4
Your software engineering agents, measured on your own repositories and shipped outcomes. Backed by a decade of ETH Zurich research.
Customers, investors and the companies the team comes from
The problem
The software engineering agent does the work. An engineer still checks almost everything.
Agents now write, review and fix code faster than your team can check it. Without evidence of where they are reliable, every change still waits for a person, and the speed-up stalls at review.
Review
Review is the bottleneck.
Agents open more pull requests than engineers can review properly, so reviews get rushed and real defects slip through.
??%
Nobody knows what’s safe to skip.
You have no evidence of where your agents are reliable, so every change gets a human check, or none does.
New model
Changes break things quietly.
A new model, prompt or tool changes how your agents behave overnight, and you find out from the bill or a customer complaint.
Agentic SDLC
From running agents to engineering them: measure, improve, monitor.
Two parts that work together: a forward-deployed engineer who measures and optimizes your agents, and a platform that keeps monitoring them.
What we bringThe CTO-led scoping workshop, the forward-deployed engineer and the platform.
The Engagement
One agent at a time: measure, improve, monitor, more trust and less oversight. Works with the agents you already run; if you don’t have an autonomous coding agent yet, we establish the first one with you. Typically this starts with PR review.
How are your agents doing today? How much human oversight is actually needed? What are the failure modes?
Caught by the agentCaught by a humanEscapedFailure mode
Deliverable
an analysis report: defect coverage, fix rate, escaped defects, failure modes, cost, a safe automerge classifier.
Access
read-only
Is agent performance improving or degrading after a change? Is cost going up or down?
How do we fix the failure modes? Where can human oversight come down?
BeforeAfterHuman reviewAutomergeFailure mode
Deliverable
an optimized agent, an operational automerge classifier, and before-and-after results against the Historic Analysis baseline.
Access
plus agent config
The Platform
Monitoring
Every software engineering agent you run is monitored: produce results, observe real-world outcomes, find issues and suggest improvements. Compare different agent versions side by side: testing a new harness, a new model, or a new MCP? Evaluate before using it in production.
What you get
A live answer to “are the agents doing their job?”
Ground truth comes from your SDLC and your engineers, not from an LLM grading its own homework.
Defect coverage
58%
caught by the agent before merge
Caught by humans
22%
missed by the agent, caught by a human before merge
Escaped defects
20%
missed by agent and human, and deployed
Failure modes
4
failure modes found and rectified
From a case study at a large SaaS company. The platform tracks these for every agent you run.
865 PRs analyzed.
What a midsize SaaS company learned from its own history.
1 in 10
PRs shipped a severe bug to production
78%
of PRs: human review was redundant, yet still required
50%
of PRs: human review safely removed, by a classifier we designed
What teams say
01 / 03
“I have infinite things to do. Something that changes what I plan to do for the next few weeks provides value. And LogicStar consistently surfaces issues that do just that.
JTJohan TibellCTO, Benetics
“Your team has done a really great job helping us through the PoC. The agent’s quality right now is much better than when we started, and we’ve got some really good validation now.
WHWaldemar HummerCTO and co-founder, LocalStack
“We don’t need to spend all that time on employees and build a new process ourselves. LogicStar catches those bugs before they get to production. That’s a big advantage for us.
FMFederico MuñozCTO, Sequence
Research
Built on research
Proven on real-world systems, we publish the leading benchmarks for AI coding agents. That same expertise drives our internal evaluations, so LogicStar keeps getting better as models evolve.
NeurIPS 2024
84%
validating tests generated
SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents
NeurIPS 2024
LogicStar reproduces every bug with a failing test that proves it’s real and validates fixes actually resolve them. State-of-the-art performance on SWT-Bench Verified.
overestimation of success rate in SWE-Bench Verified
Automated Benchmark Generation for Repository-Level Coding Tasks
ICML 2025
Many AI coding agents overfit to a single benchmark. We automatically create new benchmarks for every use-case and show popular software engineering agents lose up to 60% of performance on an application focused benchmark of 366 diverse codebases.
BaxBench: Can LLMs Generate Correct and Secure Backends?
ICML 2025 Spotlight
Even frontier models produce exploitable backends. Across 392 tasks, one in three working solutions contains SQL injection, path traversal, or code injection vulnerabilities.
Are Repository-Level Context Files Helpful for Coding Agents?
February 2026
Over 60,000 repos include AGENTS.md files to guide AI agents. Our evaluation shows these files reduce success rates by up to 3% while adding 20% to inference costs.
CodeTaste: Can LLMs Generate Human-Level Code Refactorings?
March 2026
AI agents solve only 22% of multi-file refactoring tasks and introduce breakage in 63% of attempts. CodeTaste measures whether AI restructures code the way a senior engineer would.
Serial entrepreneur, co-founder DeepCode (acq. by Snyk), EMBA (TRIUM), MSc (MIT).
Mark Müller
Co-Founder and CTO
PhD from ETH Zurich, 30+ papers and 1300+ citations. Notable industry collaborations.
Martin Vechev
Co-Founder and Advisor
Professor at ETH Zurich. 200+ publications in AI, networking, programming paradigms and others.
Who this is for
Teams automating real engineering work.
Your first agentic workflows are taking shape. We set up the first agent with you, with evaluation built in from day one, so you build trust before you scale.
Pull request counts are up, and a model update can break your agents overnight. We measure the agents you run (in-house, Claude Code, Codex or bought) on real outcomes and catch regressions before your customers do.
Agents do the work, but an engineer still checks every outcome. We show where additional human review can safely go, with the evidence to defend it.
FAQ
Everything else teams ask us.
Collected from our customer conversations, answered once for everyone.
Read-only access to the repositories your agent works in. Stage one runs entirely on history you already have: PRs, commits, reviews, and the fixes that followed. Later stages add access only when they need it, and you see exactly what each one adds.
How long until we see something?
Stage one doesn’t wait for new incidents; it runs on history that already exists, so evidence starts arriving as soon as access does. Each stage is scoped in the opening workshop.
Which agents can you work with?
Any software engineering agent whose work lands in your repositories: in-house harnesses, Claude Code, Codex, or bought tools. If you don’t run an autonomous coding agent yet, we establish the first one with you.
Does our code or data leave our environment?
We’re flexible on where the data lives: bring your own LLM endpoint or use ours. Engagements start read-only, under NDA, and we never train on your code. The trust page covers the rest of what your security team will ask.
How is it priced?
A fixed fee for each engagement stage, plus a subscription for the platform. For a longer-term contract, one annual fee covers both. Every price is scoped and set up front in the CTO-led scoping workshop.
Our codebase and our agents keep changing. Do the numbers stay comparable?
Yes. Every prompt tweak, agent change, or model update is measured against the same history, before and after, and the measurement is refreshed as your code evolves. A model update that shifts behavior overnight shows up in the score, not in the bill.
Is the goal to remove humans from the loop?
Only where the evidence says it’s safe, the regulatory environment allows it and you want it. Human review load comes down step by step, each step backed by evidence from your own history, and humans keep control throughout.
Won’t the model providers build this themselves?
Measurement only works when it’s independent of the thing measured. We don’t sell you a model, so we have no stake in which one wins: your agents are scored on your outcomes, across whatever models you run, and the numbers stay honest when you switch. Also, the model providers have no incentive to build it: they want to sell you more tokens, not to be independently measured and optimized for fewer tokens or replaced by another model.
What happens if we stop?
You keep everything: the labels, the prompts, the reports. Exportable by design, so leaving is always cheap. We’d rather earn the renewal than lock the door.
The concerns we hear, answered.
We measure agents, not developers.
Humans decide where agents may merge on their own, and nobody’s personal stats end up on a dashboard.
Adoption without the backlash.
Evidence from your own SDLC gives teams a reason to adopt instead of a mandate to comply with.
Keep the agents you have.
In-house, Claude Code, Codex, or bought: we don’t replace your setup, we measure it and make it better.
Numbers you can defend.
Every score traces back to a real PR, fix, or incident in your history. When an agent missed something, reality already wrote it down.
No changes to your workflow.
Read-only access to PRs, review comments, code and CI results, on GitHub or GitLab. We never write to your code, and you approve every agent change.
Unusual setup? Expected.
The measurement is built from your own repos, your history and your agents, homegrown or off-the-shelf. It adapts to your setup; you don’t change your workflow.
Build an Agentic SDLC you can trust.
Prove where agents can work without additional human review.
We use cookies to run the site and understand how it’s used. You can accept all cookies or reject non-essential ones. See our Cookie Policy.
Cookie settings
By clicking 'Accept all cookies', you agree to storing cookies on your device to enhance site navigation, analyze site usage and assist in our marketing efforts as outlined in our cookie policy.