“Cypress or WebdriverIO”, Choosing the Perfect Automation framework

Cypress vs WebdriverIO

Year 2020 has shown an upward trend in the usage of JS Automation frameworks such as Cypress, WebdriverIO, TestCafe, Nightwatch, Protractor, and Puppeteer and so on.

credit: NPM trends

NPM usage trends for these packages from past 6 months show a higher adoption of Cypress and WebdriverIO over the other tools.

Over the last few months as a Test Automation Engineer I have been experimenting with Cypress and WebdriverIO to understand the features provided by these tools. This blog covers my views and findings which could be useful for an automating engineer looking to choose a framework at work.

Cypress

Cypress is the next generation E2E testing tool mainly designed for modern web applications. It is an open source and scripts are written in JavaScript programming language. It supports Mocha and Chai library out of the box and can be used by both developers and testers for writing E2E tests.

Architecture:

“Cypress is not Selenium , It is architecturally and fundamentally built differently as compared to Selenium”.

Cypress executes commands directly inside the browsers automation API which makes the execution of scripts more faster and efficient.

In every system we have an OS layer (Mac, Windows or Linux).This is the layer where we install the Node and Cypress along with all of its dependencies. Since, Both the Node and Cypress resides on the same OS layer, they share a common browser and proxy for running the Application. So, it becomes very easy for the Cypress to get access for the application’s DOM, Windows, Local storage, iframe, network layer dev tools, and so on.

Cypress Execution sequence and Architecture (credit: google.com)

Cypress framework setup:

Cypress installation steps

There are only 2 commands required to get started with Cypress, The first command npm init creates an empty package.json file and second command npm install cypress — save-dev installs all the necessary dependant packages for Cypress and we don’t have to install each and every dependencies separately as we do in case of WebdriverIO.

Code Sample:

Code snippet of Cypress test for searching an item on amazon.in written using Mocha and Chai

WebdriverIO

WebdriverIO is also another excellent open source E2E testing utility for Node.js. It allows you to control the browser or a mobile application behaviour with just a few lines of code written in JavaScript with selenium tests. It is most suitable for building both TDD and BDD frameworks.

Architecture:

In WebdriverIO all the code runs locally on the development machine. The Node runs using Mocha and uses an internal runner to run the test scripts. It makes use of chai’s should for assertion. Internally WebdriverIO communicates with a Selenium Server for execution using the JSON wire protocol. The Selenium server invokes the browser via a driver in order to test the web applications. Below is a simple high level view of WebdriverIO architecture.

WebdriverIO Execution Sequence and Architecture (credit: google.com)

Now that we have understood the internals of both the tools, Let’s Setup the Framework to run tests.

Note: For both Cypress and WDIO we need to have Node.js and npm installed in our machine. (If installing through npm, Please use Node.js 8 and above)

WebdriverIO framework setup:

WebdriverIO installation steps

To install WebdriverIO on your machine, goto your project directory and switch to terminal. Initialize the folder as a npm package using the npm init command. then a utility tool starts running on your terminal prompting you to select project specific dependencies.

Then you can see a new file generated inside your project folder called package.json. which is project configuration file. You can edit it anytime to make config level changes for the project.

To install WebdriverIO, lets run npm install webdriverio — save-dev command. Then we can install the CLI Runner using npm install @wdio/cli

Now, We need wdio.config.js file that helps in performing project level functions like taking screenshot, retry logic and so on. Run ./node_modules/.bin/wdio config will prompt you with some project specific options like Base URL, framework, location of the testScripts, Reporting, Async or synchronous mode of running wdio commands.

On successful completion of above, you will see a new wdio.conf.js file generated in the root directory. this config file can also be modified later.

Next We should install Chai assertions. Let’s use npm install chai — save-dev. For global assertions use npm install chai-webdriverio — save-dev.

Finally, we need a local Runner to run scripts locally. Use npm install local-runner — save-dev in terminal.Now we are all set to execute our wdio tests.

Code Sample:

Code snippet of WebdriverIO test for searching an item on amazon.in written using Mocha and Chai

Comparison between Cypress and WebdriverIO

Comparison chart

My Views:

Both Cypress and WebdriverIO are great automation tools, So it becomes difficult when it comes to choosing one, So I would say

Go for Cypress if you are looking for:

• A Non Selenium based automation tool with most of its features included and additionally have features like

Time Travel: Takes snapshots of each commands running as a part of test.

Debuggability: Is lightening fast with clear and concise error logs.

Automatic Waiting: Capability of waiting automatically with retry-ability.

Screenshots & Videos: Captures test failure screens and video for reference.

Realtime Reloads: Automatically reloads whenever test scripts are change.

• A tool that will cover all the E2E tests, Integration tests and Unit tests for your application.
• A tool that combines both functional testing and API testing.
• A tool to setup, implement, debug and execute test scripts in a faster way.
• A tool that provides freedom to use either stub responses or actually hit the server for fetching responses.

Go for WebdriverIO if you are looking for:

• A modern front-end automation JS tool that uses the Selenium under the hood.
• A tool that is suitable for automating both web applications and native mobile Apps.
• A simple and easily configurable tool which can be extended, allowing you to write custom commands.
• A tool that supports for cross browsers(chrome, Firefox, Safari, Edge, and Internet Explorer), and inbuilt support for iframes, shadow Dom and multiple JS frameworks like jasmine, mocha and cucumber.
• A tool which supports parallel execution of tests.

In short, both the tools are having their own advantages and flaws when it comes to test automation. Depending upon the testing requirements, one can intelligently choose either one of them.

The more that you read, the more things you will know. The more that you learn, the more places you’ll go.

Stay curious and keep learning!!