apte
Tests and LLM evals in one async-first framework for Python 3.10+. Explicit DI, native concurrency, smart scoping. Formerly ProTest.
$ apte run tests:session -n 4Why
A colleague and I kept talking about the magic side of pytest. We love the framework, but fixtures resolved by name, no types, no Ctrl+Click.
I wanted something more declarative, closer to what FastAPI does with DI.
Evals
Since v0.2.0, apte treats LLM evals as first-class citizens: an eval is just a test that returns a value — scored, not asserted. Same fixtures, same DI, same parallelism; judge, scoring and short-circuit built in. Your evals live right next to the tests they ship with, not in a separate framework. I wrote a post about it.
What I learned
The project grew fast. Thread pools, async exit stacks, event bus, tree-based scoping... Low-level async stuff you never touch when building APIs.
Benchmarks
To validate the approach, I rewrote large parts of pydantic, httpx and starlette test suites with apte. Result: on httpx and starlette, tests run 20-30% faster than the official suites, thanks to native async.
Status
v0.3.1, closed in June 2026: release-please, CI, docs, asciinema demo, blog post. Zero users besides me, the promo never took off. apte was the eval harness of Felix for as long as Felix was running.