How to speed up test execution


#1

This is a topic that collects hints and ways to speed up test execution. It is Wiki style, hence you can edit and add information.

If not mentioned differently, this is mainly about the execution as WebDriver style test case with Firefox, Chrome or Internet Explorer.

Long option lists slow down test execution

Ever wondered why it takes so long to run validations on shipping or billing screens? Why it takes ages to select a country from a selectbox?

Well, that is something that is by design in Selenium. While you only see one command in XLT or in Selenium itself, under the hood, Selenium is firing many many commands against the browser to determine the correct element to select.

This especially happens why you use value, label, and text as selection criteria. Because the position is not clear, Selenium fetches every option with a new command (there is no bulk fetch) and compares it. When you have 163 countries in a list, this might take a while.

Solution: To speed things up, use a select by index. This does not require the fetching of all items before the selection is made. But the disadvantage is clearly the data dependency… select United Kingdom is easier than selecting index 85. This will break easily.

We saw a runtime advantage of 20 seconds for a single country select box in our tests. Maybe hard coding the index is an option after all.

Important: This does not apply to the execution via XLT Script Developer, because it can directly talk to the element and does not use the WebDriver API, hence it is blazing fast.

Run tests in parallel

Even though you do not run Selenium Grid, you can still run test in parallel on a single machine. We extended the ANT script of our SiteGenesis Community Test Suite to permit parallel execution. It permits to define how many browser should run your test suite in parallel. Normally one per physical core.

The test suite automatically slices your tests into buckets for each browser that runs the tests.

For more details, check our ANT script out: https://github.com/Xceptance/SiteGenesis-Community-TestSuite/blob/master/build.xml

It uses fancy JavaScript to make things easier to code compared to ANT tasks alone.