There’s a common myth in software QA at the moment that if you hire an SDET and they whip you up some test automation, you’ll have achieved “good” testing. Honestly, I’m not quite sure why this myth still exists. Just common sense says it’s not true.
Test automation is a good thing but should be used with planning and a knowledge of the pros and cons of doing so. If used appropriately, it’s a definite bonus to your test efforts but anyone who judges their test efforts solely by the amount of test automation in place is deceiving themselves.
Developer and Tester Mindsets are Different
Developers have an ingrained mindset that slants toward examining code to see that it does what it is supposed to do. Testers have a slant toward examining code to see what it does that it shouldn’t. This just has to do with how they are trained to operate.
Automation Tends to Not Find New Bugs
Test automation generally finds new bugs only while being written. It can (and is good at) looking for recurrences of bugs already found and fixed but it doesn’t tend to find new bugs after the initial run.
Automation Cannot Judge Esthetics or UIs Well
Testing a UI or web page cannot be done solely with automation. This is because some of the important aspects like appearance, alignment, and visual appeal are either not able to be tested via automation or so very expensive to automate that they just aren’t worth it.
Automated Tests are Only as Good as the Test Case
Writing a bad test case and then automating it still means you have a bad test case – just a more expensive one. Test automation needs to be written by testers who know how to design test cases.
Automation is Expensive to Write and Maintain
Test automation is often very fragile and prone to breaking during changes to the product under test. The costs of writing new automation and maintaining old automation can be prohibitively expensive. So test automation needs to be worth the costs.
I do feel that smart use of automation can improve test efforts but having lines of code is not a way to accurately judge a tester or test effort. Unit tests and BVT tests are good examples of test automation that is worthwhile because it’s reused and run so frequently that it pays off.