Automated Testing

Some people love automated tests and won’t stop singing their praises. Other people hate automated tests and constantly point out what a waste of time they are. My opinion? Well, it’s complicated…

TL;DR:

• ❤ I love automated testing.
• 😡 I hate automated testing.
• 🎓 Learn it early before you establish bad habits, and you’ll hate it less.

🛟 Automated testing is a lifesaver

Refactoring

Consider this scenario: A serious flaw was just discovered in an important library your project uses. The only way to fix it is to upgrade to the newest version, but there will be many breaking changes.

How can you know what will actually break? Automated tests.

A third-party service you use just jacked up their prices, so your company has decided to switch vendors. If things break in Production, it would cost the company tons of money for every minute that your product is broken.

How can you be confident that you won’t cost the company millions? Automated tests!

Unintended consequences

Your code and database have become bloated over time. You’re pretty sure this code isn’t called by anything, and you don’t think we need these database tables anymore.

How can you be sure that these removals won’t crash the whole system? *Automated tests.”

Your software architecture is a bunch of microservices. As things expand, the number of possible connections between services grows exponentially. Your company is successful and keeps growing, and now it’s so big that no single person could possibly know every single thing that might be calling any one piece of code.

How can you not worry that changing literally anything might break something in some faraway code? Automated integration tests!!

🐢 but it takes so long!

The initial setup

“Okay, so automated tests can really save your bacon, I get it. But I’m just working on this small project, so I don’t need to write any tests right now. I’m going to keep things simple and write my code in a simple way.”

Well what happens when your “little project” proves useful and becomes its own fully fledged product in the company? Suddenly you want all those benefits, but you have to refactor literally everything in your project. You won’t know if it’ll break because (obviously) you don’t have any automated tests yet!

Getting started is hard enough. Adding tests makes that even harder. But adding tests when you’ve already got a complicated project is painful. I understand why nobody really wants to do it.

Doing it right is hard

“Somebody’s boss decided automated testing is important, and now we have to raise our code coverage to 100%. Now I have to write hundreds of tests to try *every single possibility and make sure every line of code gets hit. Ughhhh!”*

And what ends up happening is people write bad tests because they’re chasing an arbitrary metric. Their test calls every path of the code, but they don’t check the part of the ouput that actually changes in that path. They don’t use parameterized tests, so they create dozens of tests that only vary by one line. Now nobody wants to maintain those when code changes.

Flakiness

“I uploaded my code and ran all the tests. They passed! Somebody approved my pull request. I try to complete it, the integration tests run again. They failed! Wait, they failed? But I didn’t change anything! And now I can’t complete my pull request! Now what!?”

Sure, the integration tests help you make sure all your complicated services don’t break each other. But starting a bunch of test services processing power and memory. You don’t want to slow down Production, so those resources are given to tests at a lower priority. So it’s no surprise when things take too long and time out. Now you’re wondering, “can I ignore this? Do I have to fix their broken test? Or is something actually wrong with my code?”.

Flakiness is a huge frustration and wastes so much time. The more it happens, the more legitimate test failures get ignored. Because of the random nature of the errors, it can be a nightmare to diagnose the problem.

Bad habits are hard to break

Automated tests are like exercise. If you’re fit and in shape, it’s really easy to exercise. But if you slack off for years and let yourself get out of shape, it suddenly becomes an uphill battle. The longer you go without exercise, the more of a struggle it is.

I wish I’d learned about automated tests when I first learned to code. I wish I started every project by writing tests that fail. But I have bad habits. I have to remind myself before I get too far to stop and write some tests. I groan about it every time, but I’m always glad to have the tests when they’re done.

Summary

If you’re reading this, are a computer programmer, and aren’t already writing automated tests, you need to start. Don’t do anything else until you have written some tests for your code. Trust me, it’s worth it. The sooner you start, the better. You might not see the point yet, but I promise you: one day an automated test is going to avert a disaster and you will be thankful for them.