I'm the co-founder of Reflect (https://reflect.run) which is a no-code tool for creating automated regression tests. We've realized that an accurate tool for building and running tests is necessary, but not always sufficient when it comes to being successful with automated regression testing. If your tests can't make assumptions about the state of your application, it doesn't matter how accurate your testing tool is; your tests are going to be flaky.
Every end-to-end test makes some implicit assumptions about the state of the application. For example, if you were testing an e-commerce store, you'd create a test that clicks through the site, adds a product to the cart, enters a dummy payment method, and validates that an order has been placed. There's going to be lots of baked-in assumptions in this test about the state of the application. If that product goes out of stock next month, or its product name changes, or its size / color attributes are different, then the test will probably fail. We built tdm to help software teams manage the underlying data that their end-to-end tests depend on. It's meant to run in your test environment just before your test suite executes, and it gets your application in a consistent state so that the implicit assumptions in your tests are correct.
tdm operates like a Terraform for test data; you describe the state that your data should be in, and tdm takes care of putting your data into that state. Rather than accessing your database directly, tdm interfaces with your APIs. This means that the same approach to managing your first-party data can also be used to manage test data in third-party APIs. If the API has an OpenAPI spec, we can auto-gen Typescript bindings to make integration easier.
Test data is defined as fixtures that are checked into source code. These fixtures look like JSON but are actually Typescript objects, which means your data gets compile-time checks, and you get the structural-typing goodness of TS to wrangle your data as you see fit.
Similar to Terraform, you can run tdm in a dry-run mode to first check what changes will be applied, and then run a secondary command to apply those changes. With this "diffing" approach, any data that's generated by the tests themselves gets cleared out for the next run.
I'd love to get any feedback on this approach! Hopefully this is something that can be useful regardless of what you're using to build E2E tests.
