Tests don’t have to be slow and heavy. They can be fast to run and fast to fail. Here are 7 tips to boost and improve your Java Selenium tests performance and speed. Some of them are super easy and all can be done gradually. up to x10 times better performance.
Selenium based tests tend to get slower over time. They just add up.
The bad news are that the more tests you have, it’s most likely to have a speed problem, where developers start to feel the tests are slowing them down.
The good news are that you are probably sitting in a pool full of great (and slow) tests! If that’s the case, you have the legitimacy to ask for time to work on their performance. And.. it’s time to optimize!
Not convinced that you should invest in optimizing just yet? Here are some non-trivial reasons why you should optimize your test automation project
Even legendary basketball players always start their practice with the same basic drills. Go back to the basics.
By the way, this post was written based on my experience with Selenium over Java, but these tips should be relevant for Python or C# just the same.
Take the Low hanging fruits way
Consider this list as a way to provide you ways to improve the Selenium based tests performance and speed of the project gradually. One optimization step at a time.
I strongly recommend to start from the EASIEST STEP, sticking to the lowest hanging fruit principle. There’s always something that can be done with little work, that can increase and improve the selenium performance gradually.
Use the same browser instance in many test possible
One of most expensive things in testing with Selenium, or any other framework that opens a real browser, is the time it takes to open the browser.
Why? Because it opens a new process on the computer, and this takes time. Exactly like the time it takes you to open the Google Chrome browser manually.
We are talking about several seconds each time!
To handle this issue, prefer using the same browser instance for several tests instead of closing and opening each test.
When using Java and TestNG, we can open the browser in the test class inside the method annotated with @BeforeClass.
This will open the browser before the first test in the class runs. Only once. Much faster than opening it in every test all over again.
Don’t use Explicit waits such as Thread.sleep(XXX)
Thread.sleep is the root of all evil, but that’s a subject for another post 🙂
When used in a test it creates a lower boundary for its run time. With a sleep of 4 seconds there is no way the test can be under, well, 4 seconds. When not using it, you are removing the lower boundary, enabling yourself to improve the selenium tests performance.
If you are using Java, a simple search (ctrl+f) of “Thread.sleep” inside the project can show you all the usage of this method.
Instead of using Thread.sleep, one can use a conditional wait for a something to happen. An element to appear for example.
In case you are not sure why there’s a Thread.sleep where you found it (happened to me several times :(), simply try to delete it and rerun the test.
Run tests in parallel
The idea is fairly simple: Instead of running tests one by one, run them all at the same time.
In default. tests run one after the other. This means that if there are 10 tests that each take 5 seconds then the total run time will be 50 seconds.
But, if you run them in parallel (on the same time), it can take only 5 seconds! 10 times faster!!
You might need to do some tweaks in the tests to enable them to run in parallel:
- Tests cannot interfere one another, so no test can “delete all users in the system” for cleanup.
- Tests that depend on one another cannot run in parallel.
- In case the tests require a user’s login, it might just be that the system under test won’t allow this user (used for testing) to be connected in multiple sessions – devices/browsers in the same time. So creating different users might be needed.
How can this be done?
There are at least 2 options that pop to mind.
- Selenium Grid offers a way to run several browsers in parallel on the same machine. It can be even 3 firefoxes and 4 chromes. Some great documentation can be found here.
- If you are using Jenkins, you can use it’s own mechanism to run tests on several different slaves, controlled by Jenkins master. Documentation for a compatible plugin can be found here.
- These 2 methods can be combined to provide X browsers per machine times Y parallel machines.
This is a MAJOR performance boost!
Use server APIs for test setup
Another way to improve Selenium’s performance is to understand that not everything in the test must be done using Selenium.
One of the basic best practices of testing is to focus each test on 1 thing only. It should be super clear to the one who reads the test’s steps what is the main goal of test, and what is the preparation part.
Example: In a text editing app, there can be a test that checks the “undo” feature. The test itself involves creating a text document, editing it, maybe login and logout, and finally – the undo moment, and some verification.
It is super important to identify what is the main step in the test (clicking the UNDO button in the example above). Because sometimes you can reach it much faster by using the server APIs and not performing many things using a browser.
This point can sound a bit weird to people who worked as a manual QA tester in the past (I was one a long time a go). It creates a change between the manual testing script and the automated one.
Another example is a test that needs users in the system. One way to do it is to use Selenium to log into the admin application, create users there and then login with them to the system. This will be a very straight forward way of doing the manual tests automatically.
What about using the server APIs to create these users just like the front end application does? It can save lots and lots of precious seconds or minutes.
Do not copy the manual testing script as is. It is optimized for manual tests, not for machines.
Me. Now.
Use a profiler
Using a software profiler is an important tool in the utility belt of all programmers. Doesn’t matter if this is in automation or not. This tool is good of course also to improve Selenium tests performance.
When running a test and attaching a profiler to its process, you get a visual (yet, sometimes complex) image about the time each method spent running. Also, it adds up. If there’s a method tool 0.1 second, and was executed 100 times during the test, it will show it took 10 seconds.
To be honest, it can take some time learning how to read the result from the report the profiler generates, but it is a great way to learn how the code performs. And can help us focus on the slowest methods that run most times.
Delete old and\or not relevant tests
If the Selenium based tests performance starts to be an issue, it means that there are many many tests out there, running and doing stuff. Another way to improve the performance of the Selenium tests is to DELETE OLD IRRELEVANT TESTS.
It’s better to start with the slowest tests that take the most time. Write their steps down so it would be readable for a human and ask the one in charge – is this test relevant. It depends on the organization, it can be the the QA lead, Scrum master, or if it’s a startup.. the CEO? remove the un-needed tests, and poof – less code – less time to run it.
Optimize highly used test steps
In most cases there are some “popular steps” that repeat themselves in several tests. These steps can take a significant time.
Imagine that you are testing Netflix’s site, focusing on the movies section. The tests will start with: login to Netflix, waiting for it to load, clicking on the Movies section.
After spotting this step as an “expensive” one time wise, we can now think about ways to reduce its cost. Sometimes using a specific URL parameter can do the trick (?tab=movies for instance).
The point is that focusing on the costly step is the first step in optimizing it.
Another example is an art gallery app, where you can upload an image an, edit it and sell it. Each of our tests starts by uploading an image to manipulate.
After some analyzing, we understood that uploading the image takes lots of time, and it is not even the focus of our test! it’s only an enabler for it.
Now we can optimize upload time for the test (probably there’s a bug in the app also, I’m assuming it’s fine for real users). We can consider uploading a tiny image, checking if there’s an option to use an already uploaded one, using a base64 string, whatever.
To improve selenium tests performance – Find a shortcut to get where you want to go
Someone, sometime
Improving selenium performance – Conclusion
Selenium based tests don’t have to be slow and heavy. There are ways to improve their performance, and its not rocket science (unless you are testing a rocket factory of course).
Choosing to improve the Selenium tests performance is is the only way your customers (the developers) will be able to benefit from it as a super fast feedback look for their precious work.
And.. it is a great way to improve your programming skills, understanding of the system you are testing, and getting to know your echo system – CI tools, testing frameworks. And your boss will love it!