This article is still in draft mode
On 24th April, Angular announced the depreciation of their E2E testing tool protractor. It was unclear if there will be a success or if Angular is delegating this to the various users. At the time of this writing, WebDriver.IO, TestCafé and Cypress have provided schematics for the Angular CLI.
In this article, I will provide a short overview over the differences between the various E2E frameworks, argue why you should use Cypress and help you with the first steps.
Landscape of E2E testing frameworks
We can roughly split the available tools into two groups. The ones that are based on WebDriver and the ones that operate on Chrome DevTools Protocol (CDP).
Selenium was released in 2004. It merged with a similar tool called WebDriver. It is still a major player and Protractor is also based on it.
With Selenium/WebDriver, developers have a common API to automate a browser. Every supported browser must provide a driver which WebDriver uses for the communication. This makes E2E testing tools based on WebDriver cross-platform by default.
Over time, WebDriver, the core technology in Selenium, became a W3C standard. That opened doors for other vendors to create their own frameworks which rely on WebDriver.
Unfortunately WebDriver-based tests have the reputation to be flaky. This means that if they run under the same circumstances multiple times, some of them just fail. That is extremely bad for a testing framework. You spend a considerable amount of time building something that should guarantee the stability of the application and then you can’t trust it anymore.
Chrome DevTools Protocol
Two quite popular alternatives are Puppeteer and Playwright. They don’t rely on WebDriver but talk directly to the browser via the Chrome DevTools Protocol (CDP). This gives them much better control which leads to more stable tests.
CDP is bound to the Chromium-based browsers (Chrome, Edge). Cross-browser tests, as we find them with WebDriver, are not possible. Firefox also supports CDP and therefore Puppeteer, etc. as well.
Whereas Puppeteer is CDP-only, Playwright goes one step further. Because some Puppeteer’s developers are behind that project, it is CDP-based but also supports Safari. That is possible because it ships with patched browsers.
WebdriverIO is a hybrid framework. It provides both access to CDP and WebDriver.
In which category does Cypress fall? Cypress relies on CDP as well. That’s the reason why you are mainly limited to Chromium based browsers. In contrast to Puppeteer, Cypress uses CDP only to manage the tests. The tests themselves don’t use CDP at all but run directly in the browser. There is no layer between the tests and the browser anymore. Test and application code are one. Your test code has direct access to the DOM and not an indirect one via CDP or WebDriver.
That’s the same approach that Selenium took before it merged with WebDriver. Looks like history repeats itself.
Flakiness vs. Cross-Browser
From a very high level view, the more abstraction we have between our tests and the browser, the flaky tests tend to become but the better we can do cross browser testing. The closer we get the browser the more stable the tests become but we have to accept that we can only run the test in one particular browser technology, i.e. Chromium.
Maybe all browsers will support CDP or the upcoming WebDriver BiDi will bring stability into the WebDriver world. Things are still evolving.
Why should you use Cypress?
As already explained Cypress is not using WebDriver but runs its tests directly in the browser. These make the tests stable and reliable. Exactly what we wanted in the first place.
Another reason is the Development Experience. You can really see that Cypress has been written to make it as easy as possible for us developers to use it. This does not just mean that its API is easy to use. This also means the tooling built around it, the community work and especially the documentation is just fantastic.
OK, so Cypress is better than Protractor. Why should you pick Cypress and not any other E2E library?
As Angular developers, we alway appreciated that we got an “opinionated framework”, which also pre-selected various tools for us. Although it might look like that Angular leaves the spot for E2E tool open, there is another big player in the Angular world which evolved in the latest years of providing us with the best tooling available: nx
Nx is a wrapper around the CLI. It started out as a candidate for MonoRepositories but over the years became more and more of an Angular CLI++. In fact, I personally would always pick nx, if I start a new Angular application. There is no lock-in, no overhead, I just get better tooling and can stay with the Angular CLI.
And guess what? Nx replaced Protractor with Cypress all time along. So there is somebody taking care that Angular and Cypress can play very well together and is maintaining that integration for us.
The last argument is the usage of Cypress among Angular developers. Even before Angular announced Protractor’s deprecation. The survey showed that 64% are already using Cypress. You call that landslide victory in politics. And please remember, that was at a time where Protractor was the official tool! Protractor ended up with only 19%.
First Steps in Cypress
Cypress is extremely easy to use. Starting from Angular 12, you just need to run the schematics and voilá, done. As described above, if you are on nx, Cypress is already pre-installed.
describe defines a new test suite and contains multiple test cases, where each one is defined by the
E2E tests behave like a normal end user. What are they doing? They are looking (obviously), clicking and typing. Each of these three actions has its own command in Cypress. Actually, they are methods of the global
Before we can do something with a DOM node, we have to get it first. This is done via
cy.get("some-selector"). Then we can already run an action on it. This can be a
type("some text"). These methods can be chained. A click on a button can be done by
cy.get('button').click(). Isn’t that easy?
Since we are in a test we want to verify that something has happened after the click. If we expect a text message within a
p.message showing up, that contains a text called “Changes have been saved”. We would write it like that:
cy.get('p.message').should('contain.text', 'Changes have been saved');.
… more to come, this is still a draft …
Why you should not use Cypress
Since Cypress tests run directly inside the browser, there are of course limitations. Depending on your use case, Cypress might not fit your needs:
- Only support for Chromium and Firefox: If you need to run your e2e tests in Safari, other browsers or even mobiles devices, you need to use another tool.
- Switching domains within one test.
- Switching tabs
There are workarounds for iFrames but not for switching domains or tabs. This is just natural if you think of how Cypress works. Tests are run inside the browser and vanish if you change the url or the tab.
Cypress gives you both stability and great developer experience. It is already used by the majority of Angular Developer which means you very easily find support from the community.
If you are running E2E in Protractor, now is the time to migrate. If you haven’t written E2E until now, take a look at Cypress. You will be surprised how easy it is.
Switching to Cypress means rewriting your E2E tests. But this can be done in an incremental style. Cypress and Protractor don’t exclude each other. They can be run in the same Angular project.
Be aware that Cypress might not fit your needs. If cross-browser support is an absolute must, you should check out alternatives like WebDriverIO.
- W3C WebDriver protocol
- Migrating from Protractor to Cypress | Cypress Documentation
- Future of Angular E2E & Plans for Protractor · Issue #5502 · angular/protractor
- Automation Protocols
- What is Selenium RC : Difference from Webdriver
- Nightwatch.js | Node.js powered End-to-End testing framework
- Book: Learning Selenium Testing Tools – Third Edition, Raghavendra Prasad MG