static-cms/cypress

Cypress Tests Guide

Introduction

Cypress is a JavaScript End to End Testing Framework that runs in the browser.

Cypress tests run with a local version of the CMS.

During the setup of a spec file, the relevant index.html and config.yml are copied from dev-test/backends/<backend> to dev-test.

Tests for the test backend use mock data generated in dev-test/backends/test/index.html.

Tests for the github backend use previously recorded data and stub fetch calls. See more about recording tests data here.

Run Tests Locally

yarn test:e2e # builds the demo site and runs Cypress in headless mode with mock data

Debug Tests

yarn develop # starts a local dev server with the demo site
yarn test:e2e:exec # runs Cypress in non-headless mode with mock data

Recording Tests Data

Currently only relevant for github backend tests.

When recording tests, access to the GitHub API is required, thus one must set up a .env file in the root project directory in the following format:

GITHUB_REPO_OWNER=owner
GITHUB_REPO_NAME=repo
GITHUB_REPO_TOKEN=tokenWithWritePermissions
GITHUB_OPEN_AUTHORING_OWNER=forkOwner
GITHUB_OPEN_AUTHORING_TOKEN=tokenWithWritePermissions

The structure of the repo designated by GITHUB_REPO_OWNER/GITHUB_REPO_NAME should match the settings in config.yml

To start a recording run the following commands:

yarn develop # starts a local dev server with the demo site
yarn mock:server:start # starts the recording proxy
yarn test:e2e:record-fixtures:dev # runs Cypress in non-headless and pass data through the recording proxy
yarn mock:server:stop # stops the recording proxy

During the recorded process a clone of GITHUB_REPO_NAME will be created under .temp and reset between tests.

Recordings are sanitized from any possible sensitive data and transformed into an easier to process format.

To avoid recording all the tests over and over again, a recommended process is to:

1. Mark the specific test as only by changing it("some test... to it.only("some test... for the relevant test.
2. Run the test in recording mode.
3. Exit Cypress and stop the proxy.
4. Run the test normally (with mock data) to verify the recording works.

Debugging Playback Failures

Most common failures are:

1. The recorded data is not transformed properly (e.g. sanitization broke something).
2. The stubbed requests and responses are not matched properly (e.g. timestamp changes in request body between recording and playback).

Dumping all recorded data as is to a file here and adding a debugger; statement here is useful to gain insights.

Also comparing console log messages between recording and playback is very useful (ordering of requests, etc.)