Repeated verification of the same areas, looking for tiniest inconsistencies is a pain, as everyone knows. Here are most common challenges UI regression testing brings:
- Fatigue: having 100 screenshots in your project requirements, it will be difficult to notice a tiny UI glitch when it comes to the last one.
- Wrong assumptions: "I've checked it a dozen of times already and everything was OK" or "No changes were committed here in the latest build, so it just cannot be different" - let us be honest, all manual testers say this from time to time
- No time for creative tasks: you could perform some stress testing or develop some suggestions to optimize the application, but you have these 100 pages to verify - again!
Automating visual regression testing could help with the problems listed above, but it brings some new challenges:
- Time-consuming, especially on the initial stage
- Requires regular maintenance of automated tests
- Some operations are difficult to automate - e.g. when proceeding with an action on one page requires a confirmation on another one
So, should you automate the UI regression testing process on your web project? It depends on its scope, time limitations, UI complexity, motivation level of your QA team and many other factors. In any case, this requires careful approach, risks estimation and finding a proper test automation tool/framework which can do the trick.
And what do you think about this dilemma? Care to tell about your experience of solving manual regression testing problems?