1
It has to be said that our systems are pretty stable, and the amount of bugs we ship is minimal, but the real pain is in the time it takes to validate whatever you are currently working on in an end-to-end environment, outside of your unit-tested world.
We are designing a plan to make real-world testing easier for our different teams, and asked them what their biggest pain points were:
- Missing real-world data to mimic production-usage
- Missing (or broken) key systems in staging, causing missing functionality
- Difficult to start testing from a particular customer state onwards
- Sometimes you really want to test the actual payment flow of customers, without using the sandbox environment of a PSP
We've been thinking on possible solutions, but wanted to survey the land first, and see how other companies (of different sizes) tackle these problems.I'm interested in your thoughts on this topic, how do you handle testing functionality outside of using unit and integration tests? Do you maintain two (or more) environments? Do you allow testing on your production environment, and if so, how do you model such a system to keep garbage data from impacting other systems?