Updated
Updated · Mistral AI · Jul 2
Leanstral 1.5 Solves 587 PutnamBench Problems, Finds 5 New Bugs
Updated
Updated · Mistral AI · Jul 2

Leanstral 1.5 Solves 587 PutnamBench Problems, Finds 5 New Bugs

3 articles · Updated · Mistral AI · Jul 2

Summary

  • A free Apache-2.0 Lean 4 model with 6B active parameters posted a major formal-verification jump, saturating miniF2F at 100% and setting new highs of 87% on FATE-H and 34% on FATE-X.
  • 587 of 672 PutnamBench problems were solved after training through mid-training, supervised fine-tuning and CISPO-based reinforcement learning, with the model showing steady gains as token budgets rose to 4 million.
  • In practical proof engineering, Leanstral 1.5 lifted FLTEval pass@1 to 28.9 and pass@8 to 43.2, beating Opus 4.6's 39.6 at about one-seventh the cost.
  • Across 57 open-source repositories, an automated Rust-to-Lean verification pipeline flagged 47 violated properties and confirmed 11 real bugs, including 5 previously unreported issues.
  • Hugging Face weights, a free API and open-sourced FLTEval make the release broadly accessible, underscoring a push to move formal verification from benchmarks into real-world Lean 4 development.

Insights

If an AI aces benchmarks but fails at generalization, can we trust it with our critical code?
Why is a breakthrough AI model being retired just three months after its high-profile launch?
Do free AI provers signal innovation for all or the bursting of a big tech 'inference bubble'?

Leanstral 1.5 Launches with 119B Parameters and Free Labs Access, Advancing Formal Verification for All

Overview

Leanstral 1.5, released on July 1, 2026, marks a significant update in AI-assisted formal verification, but immediate confirmation of its performance improvements and open-source details remains limited. The model card does not yet provide new benchmarks since the March version, and publications like AI Weekly have not confirmed any new comparison results or policies about releasing the model’s weights. As a result, comprehensive data on Leanstral 1.5’s advancements and its full accessibility are still awaiting official disclosure, highlighting the need for further transparency before its impact can be fully assessed.

...