Skip to content

0.2.0

Autobench 0.2.0 introduces the Autobench Instrumentation Protocol (ABP) and native evidence collection for supported AI and HTTP SDKs.

Included

  • immutable ABP signals, traces, scopes, links, references, and protocol diagnostics
  • task-local trace context with correct concurrent parentage and partial-run preservation
  • privacy-first capture policy with redaction, truncation, hashing, and artifact references
  • versioned semantic source maps and accounting-safe trace extraction
  • native Pydantic AI, OpenAI Python, OpenAI Agents, and HTTPX instrumentors
  • sync, async, iterator, context-manager, and streaming lifecycle preservation
  • typed fluent and YAML instrumentation configuration with versioned schema completion
  • autobench instrumentation doctor compatibility diagnostics
  • autobench instrumentation trace replay-only trace summaries
  • real offline instrumentation, layering, and replay/extraction examples
  • Python 3.11, 3.12, 3.13, and 3.14 quality matrix
  • built-wheel/no-extras and target-library compatibility gates

Compatibility

Existing 0.1.0 benchmark specs and RunRecords remain loadable. ABP evidence is additive: manual spans and method instrumentation now materialize through the same protocol used by native instrumentors. Replaying ABP records does not import optional provider SDKs.

Intentionally Deferred

  • OTLP and vendor exporters
  • distributed context propagation
  • import-hook auto-instrumentation
  • execution cassette replay
  • visualization

ABP protocol version 1 is the initial public protocol. Autobench 0.2.x will preserve its serialized meaning; additive fields remain forward-compatible through extension maps. A breaking wire-format change requires a new ABP protocol version.