We all try to design and deliver perfect products using unit, integration and functional UI tests. But how do we make sure everything looks as intended (responsive design)?  With all the different screens, devices, platforms and browsers out there, there is just no way to manually test across all of them. Nowadays first impression is a matter of life and dead, screwing up means losing a lot of potential clients. Therefore companies need to overcome the challenges of this “trend of device fragmentation”, here’s how.

Functional UI testing vs. visual testing

Test automation is the answer. There are different types of automated tests: functional UI testing, API testing, performance testing, security testing, database  testing, … which can help us testing the application to assure it works as expected on all supported configurations.  But what about visual testing?  How can we verify the visual content, page layout and responsive design?  Sure, we can do this manually, but this will be very time-consuming when we want to do this for every required device configuration.  Isn’t it possible to automate this?  Fortunately, the answer is yes. As a test automation engineer you should look for the right tool to do each job, not everything is meant to be a Selenium test.  Functional automation tools (like Selenium) are designed for simulating the user, meaning executing mouse actions and entering key strokes but are not designed for handling visual validations of UI Elements (validations like color, shape, position and size).  Theoretically you would be able to do visual verifications using Selenium, but this will take a lot of time and effort.  Why not use specific tools and libraries which are designed for testing UI Elements.

The most popular visual validation tool on the market

There are many free visual automation solutions, sadly all with the same disadvantage. They are all limited to integrate with only one functional automation tool or programming language: f.e. Gemini (javascript library), Needle (only works with Selenium and Python), PixDiff (Protractor plugin),… Other solutions like Percy and Screener require a license, but have a lot more functionalities and integrate with numerous functional automation solutions.  The most popular visual validation tool on the market is Applitools, a cloud service for sophisticated image comparisons.

Like any other visual automation tool it is not capable of doing any user actions on the application. Applitools is only used for comparison of the images you provide it.  In fact it doesn’t matter if images come from a mobile, web or desktop application, just upload them to the service. By integrating it with a functional automation tool you are able to go through the application by clicking, entering text and taking screenshots. Applitools does the image comparison against a previously created baseline.  You can define areas within the image and assign a different comparison mode to each of these areas.  Applitools currently contains four modes: compare images pixel by pixel, an AI-powered comparison*, ignore text areas** or you can ignore certain areas completely.  This makes it possible to create highly configurable image verifications.
* mode which reports only the differences which are perceptible to the human eye and ignores invisible differences
** only verify the layout instead

Visual automation when executing functional tests

In some projects visual automation can be useful when executing functional tests.  Assume you can control your test data and set up a process where every test iteration will be executed with the same input data. In that case you can predict the expected result by verifying a lot of text fields (for which it is often difficult and a lot of work to generate stable identifiers). It could be a lot easier to use a visual automation tool to compare a screenshot against the baseline. When the screenshot is verified against the baseline and it’s pixel perfect, it does not only looks as expected but it also works as expected.

Next to the image comparison, the cloud service also offers a user friendly UI with lots of features like:

• easily zoom-in on the changes
• grouping similar differences and handling them all at one
• a very easy baseline management system where you can work with different branches and push up-to-date baselines along with your code
• ..

As I mentioned earlier, you can integrate Applitools with your functional automation tool. It has over 40 SDKs to embed visual automation in your automated functional tests (integration is available for Selenium, Cypress, Protractor, UFT/QTP, LeanFT, Coded UI, Appium, Espresso,… ). If you want to know how, you can find all the details on their tutorials page.  There’s also an interesting video channel where you can find a lot of video tutorials

Applitools Ultrafast Grid

This all sounds great, but don’t forget even with a tool like Applitools it will take some effort and time to verify all the supported configurations.  Running through the functional test and executing the functional steps is slow. Wouldn’t it be amazing if we could run our functional test and do all our visual validations for the supported configurations all at once?  This is where Applitools Ultrafast Grid comes in.  Normally every eyes.checkWindow() command will do a visual validation of a screen, page or element, but with the Ultrafast Grid all your configurations are validated in parallel in the cloud.  More information on this nice new feature can be found here.

Conclusion

The objective of visual testing is to find visual bugs before they are pushed to production and affect the user experience.  By using a visual automation tool like Applitools you can increase test coverage in an automated way and take pressure of manual testing.  Implementing visual automation in your test automation strategy, will result in more confident releases, together with a flawless user interface.