Skip to content

0.1.0

Autobench 0.1.0 is the first release-shaped core.

Included

  • YAML-first benchmark specs
  • deterministic task runtime
  • semantic observations and projection
  • scoring, derivation, post-derivation, and policies
  • immutable YAML recording and replay
  • Markdown, YAML, and CSV reporting
  • offline minimal, basic, mid, and advanced examples
  • real optional CodeMode dogfood integration
  • portable CLI source provenance
  • Python 3.11, 3.12, and 3.13 quality matrix

Intentionally Not Included

  • autoptimize orchestration
  • GEPA integration
  • OpenTelemetry bridge
  • hosted dashboard features
  • distributed execution
  • full Pydantic Evals dataset/evaluator execution

PydanticEvalsBridge in this release is an optional payload and availability bridge. It does not claim to execute Pydantic Evals datasets. The full internal evaluation runtime remains a later integration milestone.