Topic

Unified Test Format

1 post about Unified Test Format. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

MongoDB
How to test that a dozen MongoDB drivers, in a dozen languages, conform to a dozen specs
MongoDB develops a dozen native client library drivers in a dozen languages, each idiomatic to its language but needing consistent API behavior. Instead of relying on English specs or Cucumber tests, engineers created a YAML-based domain-specific test language interpreted by runners in each driver. The Unified Test Format, built on this YAML language, now contains nearly 1,500 files testing conformance to specs covering retries, transactions, client-side encryption, and more. A natural experiment showed four of five drivers had lower nonconformance bug rates after adopting YAML tests; Node.js improved after fixing its test runner. The IEEE ISSRE 2026 accepted a paper on this approach, which has saved over 22,000 lines of test-runner code across seven drivers.

The best engineering reads, once a week.

Topic

1 post about Unified Test Format. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

MongoDB
How to test that a dozen MongoDB drivers, in a dozen languages, conform to a dozen specs
MongoDB develops a dozen native client library drivers in a dozen languages, each idiomatic to its language but needing consistent API behavior. Instead of relying on English specs or Cucumber tests, engineers created a YAML-based domain-specific test language interpreted by runners in each driver. The Unified Test Format, built on this YAML language, now contains nearly 1,500 files testing conformance to specs covering retries, transactions, client-side encryption, and more. A natural experiment showed four of five drivers had lower nonconformance bug rates after adopting YAML tests; Node.js improved after fixing its test runner. The IEEE ISSRE 2026 accepted a paper on this approach, which has saved over 22,000 lines of test-runner code across seven drivers.

The best engineering reads, once a week.