Technical Thursday
Playwright, a new competitor for web automation
02
Jun
Posted by: Berte Verhoeven
Category:
Test automation
No Comments
Playwright by Microsoft is the newest addition to the Headless Browser Testing frameworks. It is built by the same team which created Puppeteer, the headless browser testing framework for Google Chrome. Just like Puppeteer, Playwright is also an open-source NodeJS based framework. Playwright is a web automation tool supporting Chrome, Gecko (browser engine developed by Mozilla and used by Firefox) and WebKit (browser engine developed by Apple and used in Safari) and has a lot of key features which make it a valuable competitor.
Resilient test runs
The primary cause of flaky tests in automation is due to the speed of the automation tool. The tool tends to go faster than the browser is capable of loading and rendering the pages. This often results in synchronization issues, causing tests to fail. For example, when working with Selenium. To reduce the flakiness, we need to create our own set of artificial waiting mechanisms.
Auto-wait
One major benefit over Selenium is that Playwright has a feature called Auto-wait. Auto-wait performs all relevant checks for an element, and the requested action is performed only when all checks are duly passed. It automatically waits for elements to be ready to interact with prior to performing an action on the element. It ensures that the elements perform as expected and the test results are more accurate, thus less flaky. Together with a rich set of introspection events, it eliminates the need for artificial timeouts.
Web-first assertions
Modern web applications are often dynamic with the application state in constant change. Testing assertions can be challenging if the condition is not stable. Playwright assertions are created specifically for this type of dynamic webpages, called web-first assertions. It uses the expect library to create assertions allowing the automator to test if the current state of the application matches an expected result. To make these assertions more reliable, they are extended with async matchers that will wait and retry till the expected condition is met, or timeout otherwise.
Tracing
While the above features help to reduce flakiness, Playwright also offers tracing. By configuring your test retry strategy, capture the execution trace, together with videos and screenshots you have all the information you need to debug and eliminate the remaining flaky tests.
No limitations
While Playwright launches browsers by default in headless mode, it can also run the browsers in headful mode. For communication with the browser drivers, Selenium (only for Selenium versions earlier than Selenium 4) uses the JSON Wire Protocol. So for communicating with the real browser API encoding and decoding is needed. But this slows down the entire Selenium communication process. Instead, Playwright uses the DevTools protocol in combination with an event-driven architecture. The Playwright library talks directly with the browser which makes the communication process much faster (see speed comparison).
The Playwright browser allows you to create and run test scenarios spanning multiple tabs, multiple origins and multiple users. You can execute all kind of complex actions such as hovering elements and interact with dynamic controls. Playwright uses real browser input indistinguishable from a real user. It’s also possible to execute more sophisticated actions like simulating file uploads and downloads, handling various authentication methods, intercepting network requests and mocking corresponding responses.
Full isolation
For each test, Playwright creates a browser context. The browser context is equivalent to a new browser profile. Creating such a browser context only takes a matter of milliseconds and the result is full test isolation with no overhead. Such browser contexts allow the simulation of private sessions as well as multi-page scenarios.
Powerful tools
Playwright contains some very useful tools like the Test Generator, it’s their version of a record and playback tool. It allows you to generate tests (or code) and element selectors by recording the user interactions and to save them in any programming language you want to use. This tool is called codegen and has lots of nice features like emulating devices, viewport sizes, geolocation, language and timezone. There are other great tools like the Playwright Inspector, which is a GUI tool that helps authoring and debugging your scripts. Another one is the Trace Viewer, which is a GUI tool that helps you to explore and analyze traces after a test script has ran. It captures all information you need to investigate possible test failures. Such a trace contains a test execution screencast, live DOM snapshots, action explorer, test source, …
Downsides
Playwright is a powerful tool with many new features but its novelty does bring some limitations:
- It’s still quite new so the community is rather small
- It has a steeper learning curve
- No support for Internet Explorer 11
- DevTools are not exposed in WebKit and Gecko, so it uses approved patches to expose DevTools
- No mobile testing on real device
Last year Playwright has evolved a lot and each new version is loaded with tons of new features. Nevertheless, they still have a long way to go to become the number one reference tool for web automation. But they are definitely on the right track!
Written by Frederique De Winter
