Tag: Testing

A/B Testing Instant.Page With Netlify and Speedcurve

Instant.Page does one special thing to make sites faster: it preloads the next page when it’s pretty sure you’re going to click a link (either by hovering over 65ms or mousedown on desktop, or touchstart on mobile), so when you do complete the click (probably a few hundred milliseconds later), it loads that much faster.

It’s one thing to understand that approach, buy into it, integrate it, and consider it a perf win. I have it installed here!

It’s another thing to actually get the data on your own site. Leave it to Tim Kadlec to get clever and A/B test it. Tim was able to do a 50/50 A/B split with performance-neutral Netlify split testing. Half loaded Instant.Page, the other half didn’t. And the same halves told SpeedCurve which half they were in, so performance charts could be built to compare.

Tim says it mostly looks good, but his site probably isn’t the best test:

It’s also worth noting that even if the results do look good, just because it does or doesn’t make an impact on my site doesn’t mean it won’t have a different impact elsewhere. My site has a short session length, typically, and very lightweight pages: putting this on a larger commercial site would inevitably yield much different results.

I’d love to see someone do this on a beefier site. I’m in the how could it not be faster?! camp, but with zero data.

Direct Link to ArticlePermalink

The post A/B Testing Instant.Page With Netlify and Speedcurve appeared first on CSS-Tricks.

CSS-Tricks

, , ,

React Integration Testing: Greater Coverage, Fewer Tests

Integration tests are a natural fit for interactive websites, like ones you might build with React. They validate how a user interacts with your app without the overhead of end-to-end testing. 

This article follows an exercise that starts with a simple website, validates behavior with unit and integration tests, and demonstrates how integration testing delivers greater value from fewer lines of code. The content assumes a familiarity with React and testing in JavaScript. Experience with Jest and React Testing Library is helpful but not required.

There are three types of tests:

  • Unit tests verify one piece of code in isolation. They are easy to write, but can miss the big picture.
  • End-to-end tests (E2E) use an automation framework — such as Cypress or Selenium — to interact with your site like a user: loading pages, filling out forms, clicking buttons, etc. They are generally slower to write and run, but closely match the real user experience.
  • Integration tests fall somewhere in between. They validate how multiple units of your application work together but are more lightweight than E2E tests. Jest, for example, comes with a few built-in utilities to facilitate integration testing; Jest uses jsdom under the hood to emulate common browser APIs with less overhead than automation, and its robust mocking tools can stub out external API calls.

Another wrinkle: In React apps, unit and integration are written the same way, with the same tools. 

Getting started with React tests

I created a simple React app (available on GitHub) with a login form. I wired this up to reqres.in, a handy API I found for testing front-end projects.

You can log in successfully:

…or encounter an error message from the API:

The code is structured like this:

LoginModule/ ├── components/ ⎪   ├── Login.js // renders LoginForm, error messages, and login confirmation ⎪   └── LoginForm.js // renders login form fields and button ├── hooks/ ⎪    └── useLogin.js // connects to API and manages state └── index.js // stitches everything together

Option 1: Unit tests

If you’re like me, and like writing tests — perhaps with your headphones on and something good on Spotify — then you might be tempted to knock out a unit test for every file. 

Even if you’re not a testing aficionado, you might be working on a project that’s “trying to be good with testing” without a clear strategy and a testing approach of “I guess each file should have its own test?”

That would look something like this (where I’ve added unit to test file names for clarity):

LoginModule/ ├── components/ ⎪   ├── Login.js ⎪   ├── Login.unit.test.js ⎪   ├── LoginForm.js ⎪   └── LoginForm.unit.test.js ├── hooks/ ⎪   ├── useLogin.js  ⎪   └── useLogin.unit.test.js ├── index.js └── index.unit.test.js

I went through the exercise of adding each of these unit tests on on GitHub, and created a test:coverage:unit  script to generate a coverage report (a built-in feature of Jest). We can get to 100% coverage with the four unit test files:

100% coverage is usually overkill, but it’s achievable for such a simple codebase.

Let’s dig into one of the unit tests created for the onLogin React hook. Don’t worry if you’re not well-versed in React hooks or how to test them.

test('successful login flow', async () => {   // mock a successful API response   jest     .spyOn(window, 'fetch')     .mockResolvedValue({ json: () => ({ token: '123' }) }); 
   const { result, waitForNextUpdate } = renderHook(() => useLogin()); 
   act(() => {     result.current.onSubmit({       email: 'test@email.com',       password: 'password',     });   }); 
   // sets state to pending   expect(result.current.state).toEqual({     status: 'pending',     user: null,     error: null,   }); 
   await waitForNextUpdate(); 
   // sets state to resolved, stores email address   expect(result.current.state).toEqual({     status: 'resolved',     user: {       email: 'test@email.com',     },     error: null,   }); });

This test was fun to write (because React Hooks Testing Library makes testing hooks a breeze), but it has a few problems. 

First, the test validates that a piece of internal state changes from 'pending' to 'resolved'; this implementation detail is not exposed to the user, and therefore, probably not a good thing to be testing. If we refactor the app, we’ll have to update this test, even if nothing changes from the user’s perspective.

Additionally, as a unit test, this is just part of the picture. If we want to validate other features of the login flow, such as the submit button text changing to “Loading,” we’ll have to do so in a different test file.

Option 2: Integration tests

Let’s consider the alternative approach of adding one integration test to validate this flow:

LoginModule/ ├── components/ ⎪   ├─ Login.js ⎪   └── LoginForm.js ├── hooks/ ⎪   └── useLogin.js  ├── index.js └── index.integration.test.js

I implemented this test and a test:coverage:integration script to generate a coverage report. Just like the unit tests, we can get to 100% coverage, but this time it’s all in one file and requires fewer lines of code.

Here’s the integration test covering a successful login flow:

test('successful login', async () => {   // mock a successful API response   jest     .spyOn(window, 'fetch')     .mockResolvedValue({ json: () => ({ token: '123' }) }); 
   const { getByLabelText, getByText, getByRole } = render(<LoginModule />); 
   const emailField = getByLabelText('Email');   const passwordField = getByLabelText('Password');   const button = getByRole('button'); 
   // fill out and submit form   fireEvent.change(emailField, { target: { value: 'test@email.com' } });   fireEvent.change(passwordField, { target: { value: 'password' } });   fireEvent.click(button); 
   // it sets loading state   expect(button.disabled).toBe(true);   expect(button.textContent).toBe('Loading...'); 
   await waitFor(() => {     // it hides form elements     expect(button).not.toBeInTheDocument();     expect(emailField).not.toBeInTheDocument();     expect(passwordField).not.toBeInTheDocument(); 
     // it displays success text and email address     const loggedInText = getByText('Logged in as');     expect(loggedInText).toBeInTheDocument();     const emailAddressText = getByText('test@email.com');     expect(emailAddressText).toBeInTheDocument();   }); });

I really like this test, because it validates the entire login flow from the user’s perspective: the form, the loading state, and the success confirmation message. Integration tests work really well for React apps for precisely this use case; the user experience is the thing we want to test, and that almost always involves several different pieces of code working together.

This test has no specific knowledge of the components or hook that makes the expected behavior work, and that’s good. We should be able to rewrite and restructure such implementation details without breaking the tests, so long as the user experience remains the same.

I’m not going to dig into the other integration tests for the login flow’s initial state and error handling, but I encourage you to check them out on GitHub.

So, what does need a unit test?

Rather than thinking about unit vs. integration tests, let’s back up and think about how we decide what needs to be tested in the first place. LoginModule needs to be tested because it’s an entity we want consumers (other files in the app) to be able to use with confidence.

The onLogin hook, on the other hand, does not need to be tested because it’s only an implementation detail of LoginModule. If our needs change, however, and onLogin has use cases elsewhere, then we would want to add our own (unit) tests to validate its functionality as a reusable utility. (We’d also want to move the file because it wouldn’t be specific to LoginModule anymore.)

There are still plenty of use cases for unit tests, such as the need to validate reusable selectors, hooks, and plain functions. When developing your code, you might also find it helpful to practice test-driven development with a unit test, even if you later move that logic higher up to an integration test.

Additionally, unit tests do a great job of exhaustively testing against multiple inputs and use cases. For example, if my form needed to show inline validations for various scenarios (e.g. invalid email, missing password, short password), I would cover one representative case in an integration test, then dig into the specific cases in a unit test.

Other goodies

While we’re here, I want to touch on few syntactic tricks that helped my integration tests stay clear and organized.

Big waitFor Blocks

Our test needs to account for the delay between the loading and success states of LoginModule:

const button = getByRole('button'); fireEvent.click(button); 
 expect(button).not.toBeInTheDocument(); // too soon, the button is still there!

We can do this with DOM Testing Library’s waitFor helper:

const button = getByRole('button'); fireEvent.click(button); 
 await waitFor(() => {   expect(button).not.toBeInTheDocument(); // ahh, that's better });

But, what if we want to test some other items too? There aren’t a lot of good examples of how to handle this online, and in past projects, I’ve dropped additional items outside of the waitFor:

// wait for the button await waitFor(() => {   expect(button).not.toBeInTheDocument(); }); 
 // then test the confirmation message const confirmationText = getByText('Logged in as test@email.com'); expect(confirmationText).toBeInTheDocument();

This works, but I don’t like it because it makes the button condition look special, even though we could just as easily switch the order of these statements:

// wait for the confirmation message await waitFor(() => {   const confirmationText = getByText('Logged in as test@email.com');   expect(confirmationText).toBeInTheDocument(); }); 
 // then test the button expect(button).not.toBeInTheDocument();

It’s much better, in my opinion, to group everything related to the same update together inside the waitFor callback:

await waitFor(() => {   expect(button).not.toBeInTheDocument();      const confirmationText = getByText('Logged in as test@email.com');   expect(confirmationText).toBeInTheDocument(); });

Interestingly, an empty waitFor will also get the job done, because waitFor has a default timeout of 50ms. I find this slightly less declarative than putting your expectations inside of the waitFor, but some indentation-averse developers may prefer it: 

await waitFor(() => {}); // or maybe a custom util, `await waitForRerender()` 
 expect(button).not.toBeInTheDocument(); // I pass!

For tests with a few steps, we can have multiple waitFor blocks in row:

const button = getByRole('button'); const emailField = getByLabelText('Email'); 
 // fill out form fireEvent.change(emailField, { target: { value: 'test@email.com' } }); 
 await waitFor(() => {   // check button is enabled   expect(button.disabled).toBe(false); }); 
 // submit form fireEvent.click(button); 
 await waitFor(() => {   // check button is no longer present   expect(button).not.toBeInTheDocument(); });

Inline it comments

Another testing best practice is to write fewer, longer tests; this allows you to correlate your test cases to significant user flows while keeping tests isolated to avoid unexpected behavior. I subscribe to this approach, but it can present challenges in keeping code organized and documenting desired behavior. We need future developers to be able to return to a test and understand what it’s doing, why it’s failing, etc.

For example, let’s say one of these expectations starts to fail:

it('handles a successful login flow', async () => {   // beginning of test hidden for clarity 
   expect(button.disabled).toBe(true);   expect(button.textContent).toBe('Loading...'); 
   await waitFor(() => {     expect(button).not.toBeInTheDocument();     expect(emailField).not.toBeInTheDocument();     expect(passwordField).not.toBeInTheDocument(); 
     const confirmationText = getByText('Logged in as test@email.com');     expect(confirmationText).toBeInTheDocument();   }); });

A developer looking into this can’t easily determine what is being tested and might have trouble deciding whether the failure is a bug (meaning we should fix the code) or a change in behavior (meaning we should fix the test).

My favorite solution to this problem is using the lesser-known test syntax for each test, and adding inline it-style comments describing each key behavior being tested:

test('successful login', async () => {   // beginning of test hidden for clarity 
   // it sets loading state   expect(button.disabled).toBe(true);   expect(button.textContent).toBe('Loading...'); 
   await waitFor(() => {     // it hides form elements     expect(button).not.toBeInTheDocument();     expect(emailField).not.toBeInTheDocument();     expect(passwordField).not.toBeInTheDocument(); 
     // it displays success text and email address     const confirmationText = getByText('Logged in as test@email.com');     expect(confirmationText).toBeInTheDocument();   }); });

These comments don’t magically integrate with Jest, so if you get a failure, the failing test name will correspond to the argument you passed to your test tag, in this case 'successful login'. However, Jest’s error messages contain surrounding code, so these it comments still help identify the failing behavior. Here’s the error message I got when I removed the not from one of my expectations:

For even more explicit errors, there’s package called jest-expect-message that allows you to define error messages for each expectation:

expect(button, 'button is still in document').not.toBeInTheDocument();

Some developers prefer this approach, but I find it a little too granular in most situations, since a single it often involves multiple expectations.

Next steps for teams

Sometimes I wish we could make linter rules for humans. If so, we could set up a prefer-integration-tests rule for our teams and call it a day.

But alas, we need to find a more analog solution to encourage developers to opt for integration tests in a situation, like the LoginModule example we covered earlier. Like most things, this comes down to discussing your testing strategy as a team, agreeing on something that makes sense for the project, and — hopefully — documenting it in an ADR.

When coming up with a testing plan, we should avoid a culture that pressures developers to write a test for every file. Developers need to feel empowered to make smart testing decisions, without worrying that they’re “not testing enough.” Jest’s coverage reports can help with this by providing a sanity check that you’re achieving good coverage, even if the tests are consolidated that the integration level.

I still don’t consider myself an expert on integration tests, but going through this exercise helped me break down a use case where integration testing delivered greater value than unit testing. I hope that sharing this with your team, or going through a similar exercise on your codebase, will help guide you in incorporating integration tests into your workflow.

The post React Integration Testing: Greater Coverage, Fewer Tests appeared first on CSS-Tricks.

CSS-Tricks

, , , , , ,
[Top]

Automated Selenium Testing with Jest and LambdaTest

You know what the best thing is about building and running automated browser tests is? It means that the site you’re doing it on really matters. It means you’re trying to take care of that site by making sure it doesn’t break, and it’s worth the time to put guards in place against that breakages. That’s awesome. It means you’re on the right track.

My second favorite thing about automated browser tests is just how much coverage you get for such little code. For example, if you write a script that goes to your homepage, clicks a button, and tests if a change happened, that covers a lot of ground. For one, your website works. It doesn’t error out when it loads. The button is there! The JavaScript ran! If that test passes, a lot of stuff went right. If that fails, you’ve just caught a major problem.

So that’s what we’re talking about here:

  1. Selenium is the tool that automates browsers. Go here! Click this!
  2. Jest is the testing framework that developers love. I expect this to be that, was it? Yes? PASS. No? ERROR.
  3. LambdaTest is the cloud cross-browser testing platform you run it all on.

Are you one of those folks who likes the concept of automated testing but might not know where to start? That’s what we’re going to check out in this post. By stitching together a few resources, we can take the heavy lifting out of cross-browser testing and feel more confident that the code we write isn’t breaking other things.

Serenity Selenium now!

If you’re new to Selenium, you’re in for a treat. It’s an open source suite of automated testing tools that can run tests on different browsers and platforms on virtual machines. And when we say it can run tests, we’re talking about running them all in parallel. We’ll get to that.

It’s able to do that thanks to one of its components, Selenium Grid. The grid is a self-hosted tool that creates a network of testing machines. As in, the browsers and operating systems we want to test automatically. All of those machines provide the environments we want to test and they are able to run simultaneously. So cool.

Jest you wait ’til you see this

Where Selenium is boss at running tests, Jest is the testing framework. Jest tops the charts for developer satisfaction, interest, and awareness. It provides a library that helps you run code, pointing out not only where things fall apart, but the coverage of that code as a way of knowing what code impacts what tests. This is an amazing feature. How many times have you made a change to a codebase and been completely unsure what parts will be affected? That’s what Jest provides: confidence.

Jest is jam-packed with a lot of testing power and its straightforward API makes writing unit tests a relative cinch. It comes from the Facebook team, who developed it as a way to test React applications, but it’s capable of testing more than React. It’s for literally any JavaScript, and as we’ll see, browser tests themselves.

So, let’s make Jest part of our testing stack.

Selenium for machines, Jest for testing code

If we combine the superpowers of Selenium with Jest, we get a pretty slick testing environment. Jest runs the unit tests and Selenium provides and automates the grounds for cross-browser testing. It’s really no more than that!

Let’s hit pause on developing our automated testing stack for a moment to grab Selenium and Jest. They’re going to be pre-requisites for us, so we may as well snag them.

Start by creating a new project and cd-ing into it. If you already have a project, you can cd into that instead.

Once we’re in the project folder, let’s make sure we have Node and npm at the ready.

## Run this or download the package yourself at: https://nodejs.org/brew install node  ## Then we'll install the latest version of npm npm install npm@latest -g

Okey-dokey, now let’s install Jest. If you happen to be running a React project that was created with create-react-app, then you’re in luck — Jest is already included, so you’re good to go!

For the rest of us mortals, we’re going back to the command line:

## Yarn is also supported npm install --save-dev jest

OK, we have the core dependencies we need to get to work, but there is one more thing to consider…

Scalability!

Yep, scale. If you’re running a large, complex site, then it’s not far-fetched to think that you might need to run thousands of tests. And, while Selenium Grid is a fantastic resources, it is hosted on whatever environment you put it on, meaning you may very well outgrow it and need something more robust.

That’s where LambdaTest comes into play. If you haven’t heard of it, LambdaTest is a cloud-based cross-browser testing tool with 2,000+ real browsers for both manual and Selenium automation testing. Not to mention, it plays well with a lot of other services, from communication tools like Slack and Trello to project management tools like Jira and Asana — and GitHub, Bitbucket, and such. It’s extensible like that.

Here’s an important thing to know: Jest doesn’t support running tests in parallel all by itself, which is really needed when you have a lot of tests and you’re running them on multiple browsers. But on LambdaTest, you can be running concurrent sessions, meaning different Jest scripts can be running on different sessions at the same time. That’s right, it can run multiple tests together, meaning the time to run tests is cut down dramatically compared to running them sequentially.

Integrating LambdaTest Into the Stack

We’ve already installed Jest. Let’s say Selenium is already set up somewhere. The first thing we need to do is sign up for LambdaTest and grab the account credentials. We’ll need to set them up as environment variables in our project.

From the command line:

## Mac/Linuxexport LT_USERNAME=<your lambdatest username> export LT_ACCESS_KEY=<your lambdatest access_key>  ## Windowsset LT_ACCESS_KEY=<your lambdatest access_key>set LT_ACCESS_KEY=<your lambdatest access_key>

LambdaTest has a repo that contains a sample of how to set things up from here. You could clone that as a starting point if you’re just interested in testing things out.

Running tests

The LambdaTest docs use this as a sample test script:

const webdriver = require('selenium-webdriver'); const { until } = require('selenium-webdriver'); const { By } = require('selenium-webdriver'); const LambdaTestRestClient = require('@lambdatest/node-rest-client');  const username = process.env.LT_USERNAME || '<your username>'; const accessKey = process.env.LT_ACCESS_KEY || '<your accessKey>';  const AutomationClient = LambdaTestRestClient.AutomationClient({   username,   accessKey }); const capabilities = {   build: 'jest-LambdaTest-Single',   browserName: 'chrome',   version: '72.0',   platform: 'WIN10',   video: true,   network: true,   console: true,   visual: true };  const getElementById = async (driver, id, timeout = 2000) => {   const el = await driver.wait(until.elementLocated(By.id(id)), timeout);   return await driver.wait(until.elementIsVisible(el), timeout); };  const getElementByName = async (driver, name, timeout = 2000) => {   const el = await driver.wait(until.elementLocated(By.name(name)), timeout);   return await driver.wait(until.elementIsVisible(el), timeout); };  const getElementByXpath = async (driver, xpath, timeout = 2000) => {   const el = await driver.wait(until.elementLocated(By.xpath(xpath)), timeout);   return await driver.wait(until.elementIsVisible(el), timeout); };  let sessionId = null;  describe('webdriver', () => {   let driver;   beforeAll(async () => {     driver = new webdriver.Builder()       .usingServer(         'https://' + username + ':' + accessKey + '@hub.lambdatest.com/wd/hub'       )       .withCapabilities(capabilities)       .build();     await driver.getSession().then(function(session) {       sessionId = session.id_;     });     // eslint-disable-next-line no-undef     await driver.get(`https://lambdatest.github.io/sample-todo-app/`);   }, 30000);    afterAll(async () => {     await driver.quit();   }, 40000);    test('test', async () => {     try {       const lnk = await getElementByName(driver, 'li1');       await lnk.click();        const lnk1 = await getElementByName(driver, 'li2');       await lnk1.click();        const inpf = await getElementById(driver, 'sampletodotext');       await inpf.clear();       await inpf.sendKeys("Yey, Let's add it to list");        const btn = await getElementById(driver, 'addbutton');       await btn.click();        const output = await getElementByXpath(         driver,         '//html/body/div/div/div/ul/li[6]/span'       );       const outputVal = await output.getText();       expect(outputVal).toEqual("Yey, Let's add it to list");       await updateJob(sessionId, 'passed');     } catch (err) {       await webdriverErrorHandler(err, driver);       throw err;     }   }, 35000); });  async function webdriverErrorHandler(err, driver) {   console.error('Unhandled exception! ' + err.message);   if (driver && sessionId) {     try {       await driver.quit();     } catch (_) {}     await updateJob(sessionId, 'failed');   } } function updateJob(sessionId, status) {   return new Promise((resolve, reject) => {     AutomationClient.updateSessionById(       sessionId,       { status_ind: status },       err => {         if (err) return reject(err);         return resolve();       }     );   }); }

The ‘Capabilities’ object look confusing? It’s actually a lot easier to write this sort of thing using the Selenium Desired Capabilities Generator that the LambdaTest team provides. That sample script defines a set of tests that can be run on a cloud machine that have browser configuration Chrome 72 and operating system Windows 10. You can run the script from the command line, like this:

npm test .single.test.js

The sample script also have an example that you can use to run the tests on your local machine like this:

npm test .local.test.js

Great, but what about test results?

Wouldn’t it be great to have a record of all your tests, which ones are running, logs of when they ran, and what their results were? This is where LambdaTest is tough to beat because it has a UI for all of that through their automation dashboard.

The dashboard provides all of those details and more, including analytics that show how many builds ran on a given day, how much time it took to run them, and which ones passed or failed. Pretty nice to have that nearby. LambdaTest even has super handy documentation for the Selenium Automation API that can be used to extract all this test execution data that you can use for any custom reporting framework that you may have.

Test all the things!

That’s the stack: Selenium for virtual machines, Jest for unit tests, and LambdaTest for automation, hosting and reporting. That’s a lot of coverage from only a few moving pieces.

If you’ve ever used online cross-browser tools, it’s kind of like having that… but running on your own local machine. Or a production environment.

LambdaTest is free to try and worth every minute of the trial.

Get Started

The post Automated Selenium Testing with Jest and LambdaTest appeared first on CSS-Tricks.

CSS-Tricks

, , , ,
[Top]

Getting Started with Front End Testing

Amy Kapernick covers four types of testing that front-end devs could and should be doing:

  1. Linting (There’s ESLint for JavaScript and Stylelint or Prettier for CSS.)
  2. Accessibility Testing (Amy recommends pa11y, and we’ve covered Axe.)
  3. Visual Regression Testing (Amy recommends Backstop, and we’ve covered Percy.)
  4. End to End Testing (There’s Cypress and stuff like jest-puppeteer.)

Amy published something similar over on 24 ways, listing out 12 different testing tools.

As long as we’re being comprehensive, we might consider performance testing to be part of all this, ala SpeedCurve or Calibre to mention some web services.

I’ve liked what Harry Roberts has said lately about performance budgets. They don’t need to be fancy; they just need to prevent you from bad screwups.

[…] most organisations aren’t ready for challenges, they’re in need of safety nets. Performance budgets should not be things to work toward, they should be things that stop us slipping past a certain point. They shouldn’t be aspirational, they should be preventative.

Direct Link to ArticlePermalink

The post Getting Started with Front End Testing appeared first on CSS-Tricks.

CSS-Tricks

, , ,
[Top]

Testing React Hooks With Enzyme and React Testing Library

As you begin to make use of React hooks in your applications, you’ll want to be certain the code you write is nothing short of solid. There’s nothing like shipping buggy code. One way to be certain your code is bug-free is to write tests. And testing React hooks is not much different from how React applications are tested in general.

In this tutorial, we will look at how to do that by making use of a to-do application built with hooks. We’ll cover writing of tests using Ezyme and React Testing Library, both of which are able to do just that. If you’re new to Enzyme, we actually posted about it a little while back showing how it can be used with Jest in React applications. It’s not a bad idea to check that as we dig into testing React hooks.

Here’s what we want to test

A pretty standard to-do component looks something like this:

import React, { useState, useRef } from "react"; const Todo = () => {   const [todos, setTodos] = useState([     { id: 1, item: "Fix bugs" },     { id: 2, item: "Take out the trash" }   ]);   const todoRef = useRef();   const removeTodo = id => {     setTodos(todos.filter(todo => todo.id !== id));   };   const addTodo = data => {     let id = todos.length + 1;     setTodos([       ...todos,       {         id,         item: data       }     ]);   };   const handleNewTodo = e => {     e.preventDefault();     const item = todoRef.current;     addTodo(item.value);     item.value = "";   };   return (     <div className="container">       <div className="row">         <div className="col-md-6">           <h2>Add Todo</h2>         </div>       </div>       <form>         <div className="row">           <div className="col-md-6">             <input               type="text"               autoFocus               ref={todoRef}               placeholder="Enter a task"               className="form-control"               data-testid="input"             />           </div>         </div>         <div className="row">           <div className="col-md-6">             <button               type="submit"               onClick={handleNewTodo}               className="btn btn-primary"             >               Add Task             </button>           </div>         </div>       </form>       <div className="row todo-list">         <div className="col-md-6">           <h3>Lists</h3>           {!todos.length ? (             <div className="no-task">No task!</div>           ) : (             <ul data-testid="todos">               {todos.map(todo => {                 return (                   <li key={todo.id}>                     <div>                       <span>{todo.item}</span>                       <button                         className="btn btn-danger"                         data-testid="delete-button"                         onClick={() => removeTodo(todo.id)}                       >                         X                       </button>                     </div>                   </li>                 );               })}             </ul>           )}         </div>       </div>     </div>   ); }; export default Todo; 

Testing with Enzyme

We need to install the packages before we can start testing. Time to fire up the terminal!

npm install --save-dev enzyme enzyme-adapter-16 

Inside the src directory, create a file called setupTests.js. This is what we’ll use to configure Enzyme’s adapter.

import Enzyme from "enzyme"; import Adapter from "enzyme-adapter-react-16"; Enzyme.configure({ adapter: new Adapter() }); 

Now we can start writing our tests! We want to test four things:

  1. That the component renders
  2. That the initial to-dos get displayed when it renders
  3. That we can create a new to-do and get back three others
  4. That we can delete one of the initial to-dos and have only one to-do left

In your src directory, create a folder called __tests__ and create the file where you’ll write your Todo component’s tests in it. Let’s call that file Todo.test.js.

With that done, we can import the packages we need and create a describe block where we’ll fill in our tests.

import React from "react"; import { shallow, mount } from "enzyme"; import Todo from "../Todo";  describe("Todo", () => {   // Tests will go here using `it` blocks });

Test 1: The component renders

For this, we’ll make use of shallow render. Shallow rendering allows us to check if the render method of the component gets called — that’s what we want to confirm here because that’s the proof we need that the component renders.

it("renders", () => {   shallow(<Todo />); });

Test 2: Initial to-dos get displayed

Here is where we’ll make use of the mount method, which allows us to go deeper than what shallow gives us. That way, we can check the length of the to-do items.

it("displays initial to-dos", () => {   const wrapper = mount(<Todo />);   expect(wrapper.find("li")).toHaveLength(2); });

Test 3: We can create a new to-do and get back three others

Let’s think about the process involved in creating a new to-do:

  1. The user enters a value into the input field.
  2. The user clicks the submit button.
  3. We get a total of three to-do items, where the third is the newly created one.
it("adds a new item", () => {   const wrapper = mount(<Todo />);   wrapper.find("input").instance().value = "Fix failing test";   expect(wrapper.find("input").instance().value).toEqual("Fix failing test");   wrapper.find('[type="submit"]').simulate("click");   expect(wrapper.find("li")).toHaveLength(3);   expect(     wrapper       .find("li div span")       .last()       .text()   ).toEqual("Fix failing test"); });

We mount the component then we make use of find() and instance() methods to set the value of the input field. We assert that the value of the input field is set to “Fix failing test” before going further to simulate a click event, which should add the new item to the to-do list.

We finally assert that we have three items on the list and that the third item is equal to the one we created.

Test 4: We can delete one of the initial to-dos and have only one to-do left

it("removes an item", () => {   const wrapper = mount(<Todo />);   wrapper     .find("li button")     .first()     .simulate("click");   expect(wrapper.find("li")).toHaveLength(1);   expect(wrapper.find("li span").map(item => item.text())).toEqual([     "Take out the trash"   ]); });

In this scenario, we return the to-do with a simulated click event on the first item. It’s expected that this will call the removeTodo() method, which should delete the item that was clicked. Then we’re checking the numbers of items we have, and the value of the one that gets returned.

The source code for these four tests are here on GitHub for you to check out.

Testing With react-testing-library

We’ll write three tests for this:

  1. That the initial to-do renders
  2. That we can add a new to-do
  3. That we can delete a to-do

Let’s start by installing the packages we need:

npm install --save-dev @testing-library/jest-dom @testing-library/react

Next, we can import the packages and files:

import React from "react"; import { render, fireEvent } from "@testing-library/react"; import Todo from "../Todo"; import "@testing-library/jest-dom/extend-expect";  test("Todo", () => {   // Tests go here }

Test 1: The initial to-do renders

We’ll write our tests in a test block. The first test will look like this:

it("displays initial to-dos", () => {   const { getByTestId } = render(<Todo />);   const todos = getByTestId("todos");   expect(todos.children.length).toBe(2); });

What’s happening here? We’re making use of getTestId to return the node of the element where data-testid matches the one that was passed to the method. That’s the <ul> element in this case. Then, we’re checking that it has a total of two children (each child being a <li> element inside the unordered list). This will pass as the initial to-do is equal to two.

Test 2: We can add a new to-do

We’re also making use of getTestById here to return the node that matches the argument we’re passing in.

it("adds a new to-do", () => {   const { getByTestId, getByText } = render(<Todo />);   const input = getByTestId("input");   const todos = getByTestId("todos");   input.value = "Fix failing tests";   fireEvent.click(getByText("Add Task"));   expect(todos.children.length).toBe(3); });

We use getByTestId to return the input field and the ul element like we did before. To simulate a click event that adds a new to-do item, we’re using fireEvent.click() and passing in the getByText() method, which returns the node whose text matches the argument we passed. From there, we can then check to see the length of the to-dos by checking the length of the children array.

Test 3: We can delete a to-do

This will look a little like what we did a little earlier:

it("deletes a to-do", () => {   const { getAllByTestId, getByTestId } = render(<Todo />);   const todos = getByTestId("todos");   const deleteButton = getAllByTestId("delete-button");   const first = deleteButton[0];   fireEvent.click(first);   expect(todos.children.length).toBe(1); });

We’re making use of getAllByTestId to return the nodes of the delete button. Since we only want to delete one item, we fire a click event on the first item in the collection, which should delete the first to-do. This should then make the length of todos children equal to one.

These tests are also available on GitHub.

Linting

There are two lint rules to abide by when working with hooks:

Rule 1: Call hooks at the top level

…as opposed to inside conditionals, loops or nested functions.

// Don't do this! if (Math.random() > 0.5) {   const [invalid, updateInvalid] = useState(false); }

This goes against the first rule. According to the official documentation, React depends on the order in which hooks are called to associate state and the corresponding useState call. This code breaks the order as the hook will only be called if the conditions are true.

This also applies to useEffect and other hooks. Check out the documentation for more details.

Rule 2: Call hooks from React functional components

Hooks are meant to be used in React functional components — not in React’s class component or a JavaScript function.

We’ve basically covered what not to do when it comes to linting. We can avoid these missteps with an npm package that specifically enforces these rules.

npm install eslint-plugin-react-hooks --save-dev

Here’s what we add to the package’s configuration file to make it do its thing:

{   "plugins": [     // ...     "react-hooks"   ],   "rules": {     // ...     "react-hooks/rules-of-hooks": "error",     "react-hooks/exhaustive-deps": "warn"   } }

If you are making use of Create React App, then you should know that the package supports the lint plugin out of the box as of v3.0.0.

Go forth and write solid React code!

React hooks are equally prone to error as anything else in your application and you’re gonna want to ensure that you use them well. As we just saw, there’s a couple of ways we can go about it. Whether you use Enzyme or You can either make use of enzyme or React Testing Library to write tests is totally up to you. Either way, try making use of linting as you go, and no doubt, you’ll be glad you did.

The post Testing React Hooks With Enzyme and React Testing Library appeared first on CSS-Tricks.

CSS-Tricks

, , , ,
[Top]

How We Perform Frontend Testing on StackPath’s Customer Portal

Nice post from Thomas Ladd about how their front-end team does testing. The list feels like a nice place to be:

  1. TypeScript – A language, but you’re essentially getting various testing for free (passing the right arguments and types of variables)
  2. Jest – Unit tests. JavaScript functions are doing the right stuff. Works with React.
  3. Cypress – Integration tests. Page loads, do stuff with page, expected things happen in DOM. Thomas says their end-to-end tests (e.g. hitting services) are also done in Cypress with zero mocking of data.

I would think this is reflective of a modern setup making its way across lots of front-end teams. If there is anything to add to it, I’d think visual regression testing (e.g. with a tool like Percy) would be the thing to add.

As an alternative to Cypress, jest-puppeteer is also worth mentioning because (1) Jest is already in use here and (2) Puppeteer is perhaps a more direct way of controlling the browser — no middleman language or Electron or anything.

Thomas even writes that there’s a downside here: too-many-tools:

Not only do we have to know how to write tests in these different tools; we also have to make decisions all the time about which tool to use. Should I write an E2E test covering this functionality or is just writing an integration test fine? Do I need unit tests covering some of these finer-grain details as well?

There is undoubtedly a mental load here that isn’t present if you only have one choice. In general, we start with integration tests as the default and then add on an E2E test if we feel the functionality is particularly critical and backend-dependent.

I’m not sure we’ll ever get to a point where we only have to write one kind of test, but having unit and integration tests share some common language is nice. I’m also theoretically opposite in my conclusion: integration/E2E tests are a better default, since they are closer to reality and prove that a ton is going right in just testing one thing. They should be the default. However, they are also slower and flakier, so sad trombone.

Direct Link to ArticlePermalink

The post How We Perform Frontend Testing on StackPath’s Customer Portal appeared first on CSS-Tricks.

CSS-Tricks

, , , , ,
[Top]

Recipes for Performance Testing Single Page Applications in WebPageTest

WebPageTest is an online tool and an Open Source project to help developers audit the performance of their websites. As a Web Performance Evangelist at Theodo, I use it every single day. I am constantly amazed at what it offers to the web development community at large and the web performance folks particularly — for free.

But things can get difficult pretty quickly when dealing with Single Page Applications — usually written with React, Vue, Svelte or any other front-end framework. How can you get through a log in page? How can you test the performance of your users’ flow, when most of it happens client-side and does not have a specific URL to point to?

Throughout this article, we are going to find out how to solve these problems (and many more), and you’ll be ready to test the performance of your Single Page Application with WebPageTest!

Note: This articles requires an intermediate understanding about some of WebPageTest advanced features.

If you are curious about web performance and want a good introduction to WebPageTest, I would highly recommend the following resources:

The problem with testing Single Page Applications

Single Page Applications (SPAs) radically changed the way websites work. Instead of letting the back end (e.g. Django, Rails and Laravel) do most of the grunt work and delivering “ready-to-use” HTML to the browser, SPAs rely heavily on JavaScript to have the browser compute HTML. Such front-end frameworks include React, Vue, Angular or Svelte.

The simplicity of WebPageTest is what makes part of its appeal to developers: head to http://webpagetest.org/easy, enter your URL, wait a little, and voilà! Your performance audit is ready.

If you are building an SPA and want to measure its performance, you could rely on end-to-end testing tools like Selenium, Cypress or Puppeteer. However, I have found that none of these has the amount of performance-related information and easy-to-use tooling that WebPageTest offers.

But testing SPAs with WebPageTest can be complex.

In many SPAs, most of the site is protected behind a log in form. I often use Netlify for hosting my sites (including my personal blog), and most of the time I spend in the application is on authenticated pages, like the dashboard listing all my websites. As the information on my dashboard is specific to me, any other user trying to access https://app.netlify.com/teams/phacks/sites is not going to see my dashboard, but will instead be redirected to either a login or 404 page.

The same goes for WebPageTest. If I enter my dashboard URL into http://webpagetest.org/easy, the audit will be performed against the login page.

Moreover, testing and monitoring the performance of dynamic interactions in SPAs cannot be achieved with simple WebPageTest audits.

Here’s an example. Nuage is a domain name registrar with fancy animations and a beautiful, dynamic interface. When you search for domain names to buy, an asynchronous call fetches the results of the request and the results are displayed as they are retrieved.

As you might have noticed in the video above, the URL of the page does not change as I type my search terms. As a consequence, it is not possible to test the performance of the search experience using a simple WebPageTest audit as we do not have a proper URL to point to the action of searching something — only to an empty search page.

Some other problems can arise from the SPA paradigm shift when using WebPageTest:

  • Clicking around to navigate a webpage is usually harder than merely heading to a new URL, but it is sometimes the only option in SPAs.
  • Authentication in SPAs is usually implemented using JSON Web Tokens instead of good ol’ cookies, which rules out the option of setting authentication cookies directly in WebPageTest (as described here).
  • Using React and Redux (or other application state management libraries) for your SPA can mean that forms are harder to fill out programmatically, since using .innerText() or .value() to set a field’s value may not forward it to the application store.
  • As API calls are often asynchronous and various loaders can be used to indicate a loading state, those can “trick” WebPageTest into believing the page has actually finished loading when it has, in fact, not. I have seen it happen with longer-than-usual API calls (5+ seconds).

As I have faced these problems on several projects, I have come up with a range of tips and techniques to counter them.

The many ways of selecting an element

Selecting DOM elements is a key part of doing all sorts of automated testing, be it for end-to-end testing with Selenium or Cypress or for performance testing with WebPageTest. Selecting DOM elements allows us to click on links and buttons, fill in forms and more generally interact with the application.

There are several ways of selecting a particular DOM elements using native browser APIs, that range from the straightforward document.getElementsByClassName to the thorny but really powerful XPath selectors. In this section, we will see three different possibilities, ordered by increasing complexity.

Get an element by id, className or tagName

If the element you want to select (say, an “Empty Cart” button) has a specific and unique id (e.g. #empty-cart), class name, or is the only button on the page, you can click on it using the getElementsBy methods:

const emptyCartButton = document.getElementsById("empty-cart")[0]; // or document.getElementsByClassName(".empty-cart-button")[0] // or document.getElementsByTagName("button")[0] emptyCartButton.click();

If you have several buttons on the same page, you can filter the resulting list before interacting with the element:

const buttons = document.getElementsByTagName("button"); const emptyCartButton = buttons.filter(button =>   button.innerText.includes("Empty Cart") )[0]; emptyCartButton.click();

Use complex CSS selectors

Sometimes, the particular element you want to interact with does not present an interesting unicity property in either its ID, class or tag.

One way to circumvent this issue is to add this unicity manually, for testing purposes only. Adding #perf-test-empty-cart-button to a specific button is innocuous for your website markup and can dramatically simplify your testing setup.

However, this solution can sometimes be out of reach: you may not have access to the source code of the application, or may not be able to deploy new versions quickly. In those situations, it is useful to know about document.querySelector (and its variant document.querySelectorAll) and using complex CSS selectors.

Here are a few examples of what can be achieved with document.querySelector:

// Select the first input with the `name="username"` property document.querySelector("input[name='username']"); // Select all number inputs document.querySelectorAll("input[type='number']");  // Select the first h1 inside the <section> document.querySelector("section h1");  // Select the first direct descendent of a <nav> which is of type <img> document.querySelector("nav > img");

What’s interesting here is you have the full power of CSS selectors at hand. I encourage you to have a look at the always-useful MDN’s reference table of selectors!

Going nuclear: XPath selectors

XML Path Language (XPath), albeit really powerful, is harder to grasp and maintain than the CSS solutions above. I rarely have to use it, but it is definitively useful to know that it exists.

One such instance is when you want to select a node by its text value, and can’t resort to CSS selectors. Here’s a handy snippet to use in those cases:

// Returns the  that has the exact content 'Sep 16, 2015' document.evaluate(   "//span[text()='Sep 16, 2015']",   document,   null,   XPathResult.FIRST_ORDERED_NODE_TYPE,   null ).singleNodeValue;

I will not go into details on how to use it as it would have me wander away from the goal of this article. To be fair, I don’t even know what many of the parameters above even mean. However, I can definitely recommend the MDN documentation should you want to read on the topic.

Recipes for common use cases

In the following section, we will see how to test the performance in common use cases of Single Page Applications. I call these my testing recipes.

In order to illustrate those recipes, I will use the React Admin demo website as an example. React Admin is an open source project aimed at building admin applications and back offices.

It is a typical example of a SPA because it uses React (as the name suggests), calls remote APIs, has a login interface, many forms and relies on client-side routing. I encourage you to go take a quick look at the website (the demo account is demo/demo ) in order to have an idea of what we will be trying to achieve.

Authentication and forms

The authentication page of React Admin requires the user to input a username and a password:

The authentication screen of React Admin

Intuitively, one could take the following approach to filling in the form and submit:

const [usernameInput, passwordInput] = document.getElementsByTagName("input"); usernameInput.value = "demo"; // innerText could also be used here passwordInput.value = "demo"; document.getElementsByTagName("button")[0].click();

If you run these commands sequentially in a DevTools console on the login page, you will see that all fields are reset and the login request fails upon submitting by clicking the button. The problem comes from the fact that the new values we set with .value() (or .innerText()) are not kicked back to the Redux store, and thus not “processed” by the application.

What we need to do then is explicitly tell React that the value has changed so that it will update internal bookkeeping accordingly. This can be achieved using the Event interface.

const updateInputValue = (input, newValue) => {   let lastValue = input.value;   input.value = newValue;   let event = new Event("input", { bubbles: true });   let tracker = input._valueTracker;   if (tracker) {     tracker.setValue(lastValue);   }   input.dispatchEvent(event); };

Note: this solution is pretty hacky (even according to its own author), however it works well for our purposes here.

Our updated script becomes:

const updateInputValue = (input, newValue) => {   let lastValue = input.value;   input.value = newValue;   let event = new Event("input", { bubbles: true });   let tracker = input._valueTracker;   if (tracker) {     tracker.setValue(lastValue);   }   input.dispatchEvent(event); };  const [usernameInput, passwordInput] = document.getElementsByTagName("input");  updateInputValue(usernameInput, "demo"); updateInputValue(passwordInput, "demo");  document.getElementsByTagName("button")[0].click();

Hurrah! You can try it in your browser’s console—It works like a charm.

Translating this to an actual WebPageTest script (with scripting keywords, single line commands and tab-separated parameters) would look like this:

setEventName    Go to Login  navigate    https://marmelab.com/react-admin-demo/  setEventName    Login      exec    const updateInputValue = (input, newValue) => {  let lastValue = input.value;  input.value = newValue;  let event = new Event("input", { bubbles: true });  let tracker = input._valueTracker;  if (tracker) {  tracker.setValue(lastValue);  }  input.dispatchEvent(event);};  exec    const [usernameInput, passwordInput] = document.getElementsByTagName("input")  exec    updateInputValue(usernameInput, "demo") exec    updateInputValue(passwordInput, "demo")  execAndWait document.getElementsByTagName("button")[0].click()

Note that clicking on the submit button leads us to a new page and triggers API calls, which means we need to use the execAndWait command.

You can see the full results of the test at this address. (Note: the results may have been archived by WebPageTest — you can, however, run the test again yourself!)

Here is a short video (captured by WebPageTest) in which you can see that we indeed passed the authentication step:

Navigating between pages

For traditional Server Rendered pages, navigating from one URL to the next in WebPageTest scripting is done via the navigate <url> command.

However, for SPAs, this does not reflect the experience of the user, as client-side routing means that the server has no role in navigation. Thus, hitting a URL directly would significantly slow down the measured performance (because of the time it takes for the JavaScript framework to be compiled, parsed and executed), a slowdown that the user does not experience when changing pages. As it is crucial to simulate the user flow the best we can, we need to handle the navigation on the client as well.

Hopefully, this is a lot simpler to do than filling up forms. We only need to select the link (or button) that will take us to the new page, and .click() on it! Let’s follow through our previous example, although now we want to test the performance of the Reviews list, and of a single Review page.

A user would typically click on the Reviews item on the left-hand navigation menu, then on any item in the list. Inspecting the elements in DevTools may lead us to a selection strategy as follows:

document.querySelector("a[href='#reviews']"); // select the Reviews link in the menu document.querySelector("table tr"); // select the first item in the Reviews list

As both clicks lead to page transition and API calls (to fetch the reviews), we need to use the execAndWait keyword for the script:

setEventName    Go to Login  navigate    https://marmelab.com/react-admin-demo/  setEventName    Login  exec    const updateInputValue = (input, newValue) => {  let lastValue = input.value;  input.value = newValue;  let event = new Event("input", { bubbles: true });  let tracker = input._valueTracker;  if (tracker) {    tracker.setValue(lastValue);  }  input.dispatchEvent(event);};  exec    const [usernameInput, passwordInput] = document.getElementsByTagName("input")  exec    updateInputValue(usernameInput, "demo") exec    updateInputValue(passwordInput, "demo")  execAndWait document.getElementsByTagName("button")[0].click()  setEventName    Go to Reviews  execAndWait document.querySelector("a[href='#/reviews']").click()  setEventName    Open a single Review  execAndWait document.querySelector("table tbody tr").click()

Here’s the video of the complete script running on WebPageTest:

The audit result from WebPageTest shows the performance metrics and waterfall graphs for each step of the script, allowing us to monitor the performance of each API call and interaction:

What about Internet Explorer 11 compatibility?

WebPageTest allows us to select which location, browser and network conditions the test will use. Internet Explorer 11 (IE11) is among the available browser options, and if you try the previous scripts on IE11, they will fail.

This is due to two reasons:

The ES6 syntax problem can be overcome by translating our scripts to ES5 syntax (no arrow functions, no let and const, no array destructuring), which might look like this:

setEventName    Go to Login  navigate    https://marmelab.com/react-admin-demo/  setEventName    Login  exec    var updateInputValue = function(input, newValue) {  var lastValue = input.value;  input.value = newValue;  var event = new Event("input", { bubbles: true });  var tracker = input._valueTracker;  if (tracker) {    tracker.setValue(lastValue);  }  input.dispatchEvent(event);};  exec    var usernameInput = document.getElementsByTagName("input")[0] exec    var passwordInput = document.getElementsByTagName("input")[1]  exec    updateInputValue(usernameInput, "demo") exec    updateInputValue(passwordInput, "demo")  execAndWait document.getElementsByTagName("button")[0].click()  setEventName    Go to Reviews  execAndWait document.querySelector("a[href='#/reviews']").click()  setEventName    Open a single Review  execAndWait document.querySelector("table tbody tr").click()

In order to bypass the absence of CustomEvent support, we can turn to polyfills and add one manually at the top of the script. This polyfill is available on MDN:

(function() {   if (typeof window.CustomEvent === "function") return false;   function CustomEvent(event, params) {     params = params || { bubbles: false, cancelable: false, detail: undefined };     var evt = document.createEvent("CustomEvent");     evt.initCustomEvent(       event,       params.bubbles,       params.cancelable,       params.detail     );     return evt;   }   CustomEvent.prototype = window.Event.prototype;   window.CustomEvent = CustomEvent; })();

We can then replace all mentions of Event by CustomEvent, set the polyfill to fit on a single line and we are good to go!

setEventName    Go to Login  navigate    https://marmelab.com/react-admin-demo/  exec    (function(){if(typeof window.CustomEvent==="function")return false;function CustomEvent(event,params){params=params||{bubbles:false,cancelable:false,detail:undefined};var evt=document.createEvent("CustomEvent");evt.initCustomEvent(event,params.bubbles,params.cancelable,params.detail);return evt}CustomEvent.prototype=window.Event.prototype;window.CustomEvent=CustomEvent})();  setEventName    Login  exec    var updateInputValue = function(input, newValue) {  var lastValue = input.value;  input.value = newValue;  var event = new CustomEvent("input", { bubbles: true });  var tracker = input._valueTracker;  if (tracker) {    tracker.setValue(lastValue);  }  input.dispatchEvent(event);};  exec    var usernameInput = document.getElementsByTagName("input")[0] exec    var passwordInput = document.getElementsByTagName("input")[1]  exec    updateInputValue(usernameInput, "demo") exec    updateInputValue(passwordInput, "demo")  execAndWait document.getElementsByTagName("button")[0].click()  setEventName    Go to Reviews  execAndWait document.querySelector("a[href='#/reviews']").click()  setEventName    Open a single Review  execAndWait document.querySelector("table tbody tr").click()

Et voilà!

General tips and tricks for WebPageTest scripting

One last thing I want to do is provide a few tips and tricks that make writing WebPageTest scripts easier. Feel free to DM me on Twitter if you have any suggestions!

Security first!

Remember to tick both privacy checkboxes if your script includes senstitive data, like credentials!

WebPageTest security controls

Browse the docs

The WebPageTest Scripting docs are full of features that I didn’t cover in this article, ranging from DNS Overriding to iPhone Spoofing and even if/else conditionals.

When you plan on writing a new script, I recommend to have a look at the available parameters first and see if any can help make your scripting easier or more robust.

Long loading states

Sometimes, a remote API call (say, for fetching the reviews) will take a long time. A loading indicator, such as a spinner, can be used to tell the user to wait a bit as something is happening.

WebPageTest tries to detect when a page has finished loading by figuring out if things are changing on the screen. If your loading indicator lasts a long time, WebPageTest might mistake it for an integral part of your page design and cut the audit before the API call returns — thus truncating your measures.

A way to circumvent this issue is to tell WebPageTest to wait at least a certain duration before stopping the test. This is a parameter available under the Advanced tab:

WebPageTest minimum test duration

Keeping your script (and results) human-readable

  • Use blank lines and comments (//) generously because single-line JavaScript commands can sometimes be hard to grasp.
  • Keep a multi-line version somewhere as your reference, and single-line everything as you are about to test. This helps readability. Like, a lot.
  • Use setEventName to name your different “steps.” This makes for more readable tests as it explicits the sequence of pages the audit goes through, and also appears in the WebPageTest results.

Iterating on your scripts

  • First, make sure that your script works in the browser. To do so, strip the WebPageTest keywords (the first word of every line of your script), then copy and paste each line in the browser console to verify that everything is working as expected at every step of the way.
  • Once you are ready to submit your test to WebPageTest, do it first with very light settings: only one run, a fast browser (cough — not IE11 — cough), no network throttling, no repeat view, a well-dimensioned instance (Dulles, VA, usually has good response times). This will help you detect and correct errors way faster.

Automating your scripts

Your test script is running smoothly, and you start getting performance reports of your Single Page App. As you ship new features, it is important that you monitor its performance regularly to catch regressions at the earliest.

To address this problem, I am currently working on Falco, a soon-to-be-open-sourced WebPageTest test runner. Falco takes care of automating your audits, then presents the results in an easy-to-read interface while letting you read the full reports when you need it. You can follow me on Twitter to know when it goes open source, and learn more about web performance and WebPageTest!

The post Recipes for Performance Testing Single Page Applications in WebPageTest appeared first on CSS-Tricks.

CSS-Tricks

, , , , , ,
[Top]

Lessons Learned from a Year of Testing the Web Platform

Mike Pennisi:

The web-platform-tests project is a massive suite of tests (over one million in total) which verify that software (mostly web browsers) correctly implement web technologies. It’s as important as it is ambitious: the health of the web depends on a plurality of interoperable implementations.

Although Bocoup has been contributing to the web-platform-tests, or “WPT,” for many years, it wasn’t until late in 2017 that we began collecting test results from web browsers and publishing them to wpt.fyi

Talk about doing God’s work.

The rest of the article is about the incredible pain of scaling a test suite that big. Ultimately Azure Pipelines was helpful.

Direct Link to ArticlePermalink

The post Lessons Learned from a Year of Testing the Web Platform appeared first on CSS-Tricks.

CSS-Tricks

, , , , ,
[Top]

Using Percy to add visual testing to a Jekyll site

Visual testing is the automated process of reviewing software from a purely visual standpoint. Instead of testing the code underneath, visual testing is all about what end users actually see and interact with.

Similar to functional testing, however, visual testing fits directly into your stack and workflow. And with Percy, it’s easy to add snapshots to and run visual reviews for anything that runs in a browser.

Let’s set up an app to test

In this tutorial, we’re going to clone an example Jekyll site, add Percy snapshots, make some visual changes, and review them in Percy.

Before we add Percy, let’s set up our example app:

git clone https://github.com/percy/example-percy-jekyll.git

Navigate to your local app, install dependencies, then build and run the site:

cd example-percy-jekyll/ bundle install bundle exec jekyll build bundle exec jekyll serve

Note: Ruby >= 2.3 is required

Open and click around the site locally to see what we’ll be taking Percy snapshots of.

Shoutout to CloudCannon for the Hydra theme.

Next, if you haven’t already, sign up for a free Percy account and create an organization and a new project:

Percy projects correspond to the web application, component library, or static site you’re testing. In this case, we’ll be connecting our Percy project to our Jekyll site.

Generating Percy snapshots

To authenticate your local environment with your new Percy project, copy the PERCY_TOKEN environment variable from the new project screen or your project settings, then run:

$  export PERCY_TOKEN=aaabbbcccdddeee $  npx percy snapshot _site/

Note: Be sure to replace the token with your project-specific token.

You’ll see output like this:

$  npx percy snapshot _site/ Downloading Chromium r662092 - 88.3 Mb [====================] 100% 0.0s  [percy] created build #1: https://percy.io/jekyll-test-site/hydra-theme/builds/2048096 [percy] percy has started. [percy] serving static site at http://localhost:5339/ [percy] snapshot taken: '/404.html' [percy] snapshot taken: '/index.html' [percy] snapshot taken: '/about/index.html' [percy] snapshot taken: '/blog/index.html' [percy] snapshot taken: '/contact/index.html' [percy] snapshot taken: '/contact-success/index.html' [percy] snapshot taken: '/pricing/index.html' [percy] snapshot taken: '/category/marketing/index.html' [percy] snapshot taken: '/category/sales/index.html' [percy] snapshot taken: '/category/tips/index.html' [percy] snapshot taken: '/sales/2016/07/20/the-process-for-direct-sales/index.html' [percy] snapshot taken: '/marketing/2016/08/12/the-history-of-marketing/index.html' [percy] snapshot taken: '/sales/2016/08/06/definition-of-sales/index.html' [percy] snapshot taken: '/sales/tips/2016/07/28/effective-upselling-techniques/index.html' [percy] snapshot taken: '/sales/tips/2016/08/02/sales-effectiveness/index.html' [percy] shutting down static site at http://localhost:5339/ [percy] stopping percy... [percy] waiting for 15 snapshots to complete... [percy] done. [percy] finalized build #1: https://percy.io/jekyll-test-site/hydra-theme/builds/2048096

Click the build link or head over to your Percy project to check out your first build.

What’s going on behind the scenes?

Once npx percy snapshot _site/ was called, Percy captured the DOM snapshots for each page of your Jekyll site. Percy then recreated the snapshots to compare against baseline snapshots and determine which pixels have changed.

Since this is the first build, there isn’t anything to compare it to. It has also been auto-approved because the commit was on master and we assume that master builds are production-ready.

Making and reviewing visual changes

Now that you’ve pushed your first build and established a baseline to compare your next snapshots to, let’s make a visual change to review.

Let’s make a wide sweeping CSS change. Head over to the cloudcannon.scss file and change the brand colors:

$ brand-color: #ff8a00; $ secondary-brand-color: #da1b60;

Since we’re pushing these changes directly to master, be sure to disable auto-approve on master branches from your Percy project settings.

Once those changes are saved, build your site and run Percy again:

$  bundle exec jekyll build $  npx percy snapshot _site/

Head back to Percy or click the Percy build link to see the visual changes!

If you haven’t been following along, you can check out the build.

What’s going on in the Percy UI?

The red areas in the right panel represent pixels that have changed—the visual diffs. Clicking that area (or pressing “d”) will toggle between the underlying snapshot and the overlaid diff so you can easily see what exactly has changed. You’ll also notice that the first several diffs have been matched and grouped to make it easier and faster to review.

Each snapshot has been rendered across both Chrome and Firefox, and at mobile and desktop widths. Rendering your site across these variations helps you detect subtle differences caused by browser rendering or responsive bugs.

Now that we’re happy with our fresh new look, hit “Approve All.” ✅

You’ve done your first visual review! Visual testing is great not only for catching visual bugs, but also for knowing the exact impact of any given code change. Seeing your UI visualized during code reviews is invaluable, helping you fix regressions before they make their way to production, or to deploy with complete confidence.

Configuring your snapshots

You can configure how and where Percy runs for each build by creating a global .percy.yml file in the root of your project.

You can customize the responsive widths at which your snapshots are rendered. For example, if you want to add a super wide snapshot in addition to our mobile and desktop widths:

version: 1 snapshot: 	widths: [375, 1280, 1920]

You can also ignore certain pages. For example, if you’d like to ignore multiple pages with the same layout like blog categories posts:

version: 1 static-snapshots: 	ignore-files: "/blog/category/*"

Learn more about Percy SDK configuration in our docs).

Adding Percy to your workflow

What we’ve done so far demonstrates how Percy generates snapshots and detects visual changes locally, but to get the most value out of automated visual testing, we recommend integrating Percy with your CI service.

For instructions and to see all of our supported CI services, check out our CI setup documentation. Here are a few of our most popular supported services:

Visual testing is best when done alongside code reviews. We support integrations with GitHub, GitLab, and Bitbucket (coming soon)! With an integration enabled, Percy will show up in your checks commit and pull request checks, notifying you if visual changes are detected:

Clicking “Details” will take you right to the Percy build where you can review visual changes.
After snapshots are approved, your commit status will change to green and the pull request can be merged.

With continuous visual reviews on every feature branch, it’s easy to have 100% confidence the visual changes you’ll be deploying.


We hope this tutorial has helped you get acquainted with Percy’s visual review platform and workflow. To learn more about how Percy works, feel free to check out these docs:

Happy testing! 💜

The post Using Percy to add visual testing to a Jekyll site appeared first on CSS-Tricks.

CSS-Tricks

, , , , ,
[Top]

Using Percy to add visual testing to a Jekyll site

Visual testing is the automated process of reviewing software from a purely visual standpoint. Instead of testing the code underneath, visual testing is all about what end users actually see and interact with.

Similar to functional testing, however, visual testing fits directly into your stack and workflow. And with Percy, it’s easy to add snapshots to and run visual reviews for anything that runs in a browser.

Let’s set up an app to test

In this tutorial, we’re going to clone an example Jekyll site, add Percy snapshots, make some visual changes, and review them in Percy.

Before we add Percy, let’s set up our example app:

git clone https://github.com/percy/example-percy-jekyll.git

Navigate to your local app, install dependencies, then build and run the site:

cd example-percy-jekyll/ bundle install bundle exec jekyll build bundle exec jekyll serve

Note: Ruby >= 2.3 is required

Open and click around the site locally to see what we’ll be taking Percy snapshots of.

Shoutout to CloudCannon for the Hydra theme.

Next, if you haven’t already, sign up for a free Percy account and create an organization and a new project:

Percy projects correspond to the web application, component library, or static site you’re testing. In this case, we’ll be connecting our Percy project to our Jekyll site.

Generating Percy snapshots

To authenticate your local environment with your new Percy project, copy the PERCY_TOKEN environment variable from the new project screen or your project settings, then run:

$  export PERCY_TOKEN=aaabbbcccdddeee $  npx percy snapshot _site/

Note: Be sure to replace the token with your project-specific token.

You’ll see output like this:

$  npx percy snapshot _site/ Downloading Chromium r662092 - 88.3 Mb [====================] 100% 0.0s  [percy] created build #1: https://percy.io/jekyll-test-site/hydra-theme/builds/2048096 [percy] percy has started. [percy] serving static site at http://localhost:5339/ [percy] snapshot taken: '/404.html' [percy] snapshot taken: '/index.html' [percy] snapshot taken: '/about/index.html' [percy] snapshot taken: '/blog/index.html' [percy] snapshot taken: '/contact/index.html' [percy] snapshot taken: '/contact-success/index.html' [percy] snapshot taken: '/pricing/index.html' [percy] snapshot taken: '/category/marketing/index.html' [percy] snapshot taken: '/category/sales/index.html' [percy] snapshot taken: '/category/tips/index.html' [percy] snapshot taken: '/sales/2016/07/20/the-process-for-direct-sales/index.html' [percy] snapshot taken: '/marketing/2016/08/12/the-history-of-marketing/index.html' [percy] snapshot taken: '/sales/2016/08/06/definition-of-sales/index.html' [percy] snapshot taken: '/sales/tips/2016/07/28/effective-upselling-techniques/index.html' [percy] snapshot taken: '/sales/tips/2016/08/02/sales-effectiveness/index.html' [percy] shutting down static site at http://localhost:5339/ [percy] stopping percy... [percy] waiting for 15 snapshots to complete... [percy] done. [percy] finalized build #1: https://percy.io/jekyll-test-site/hydra-theme/builds/2048096

Click the build link or head over to your Percy project to check out your first build.

What’s going on behind the scenes?

Once npx percy snapshot _site/ was called, Percy captured the DOM snapshots for each page of your Jekyll site. Percy then recreated the snapshots to compare against baseline snapshots and determine which pixels have changed.

Since this is the first build, there isn’t anything to compare it to. It has also been auto-approved because the commit was on master and we assume that master builds are production-ready.

Making and reviewing visual changes

Now that you’ve pushed your first build and established a baseline to compare your next snapshots to, let’s make a visual change to review.

Let’s make a wide sweeping CSS change. Head over to the cloudcannon.scss file and change the brand colors:

$ brand-color: #ff8a00; $ secondary-brand-color: #da1b60;

Since we’re pushing these changes directly to master, be sure to disable auto-approve on master branches from your Percy project settings.

Once those changes are saved, build your site and run Percy again:

$  bundle exec jekyll build $  npx percy snapshot _site/

Head back to Percy or click the Percy build link to see the visual changes!

If you haven’t been following along, you can check out the build.

What’s going on in the Percy UI?

The red areas in the right panel represent pixels that have changed—the visual diffs. Clicking that area (or pressing “d”) will toggle between the underlying snapshot and the overlaid diff so you can easily see what exactly has changed. You’ll also notice that the first several diffs have been matched and grouped to make it easier and faster to review.

Each snapshot has been rendered across both Chrome and Firefox, and at mobile and desktop widths. Rendering your site across these variations helps you detect subtle differences caused by browser rendering or responsive bugs.

Now that we’re happy with our fresh new look, hit “Approve All.” ✅

You’ve done your first visual review! Visual testing is great not only for catching visual bugs, but also for knowing the exact impact of any given code change. Seeing your UI visualized during code reviews is invaluable, helping you fix regressions before they make their way to production, or to deploy with complete confidence.

Configuring your snapshots

You can configure how and where Percy runs for each build by creating a global .percy.yml file in the root of your project.

You can customize the responsive widths at which your snapshots are rendered. For example, if you want to add a super wide snapshot in addition to our mobile and desktop widths:

version: 1 snapshot: 	widths: [375, 1280, 1920]

You can also ignore certain pages. For example, if you’d like to ignore multiple pages with the same layout like blog categories posts:

version: 1 static-snapshots: 	ignore-files: "/blog/category/*"

Learn more about Percy SDK configuration in our docs).

Adding Percy to your workflow

What we’ve done so far demonstrates how Percy generates snapshots and detects visual changes locally, but to get the most value out of automated visual testing, we recommend integrating Percy with your CI service.

For instructions and to see all of our supported CI services, check out our CI setup documentation. Here are a few of our most popular supported services:

Visual testing is best when done alongside code reviews. We support integrations with GitHub, GitLab, and Bitbucket (coming soon)! With an integration enabled, Percy will show up in your checks commit and pull request checks, notifying you if visual changes are detected:

Clicking “Details” will take you right to the Percy build where you can review visual changes.
After snapshots are approved, your commit status will change to green and the pull request can be merged.

With continuous visual reviews on every feature branch, it’s easy to have 100% confidence the visual changes you’ll be deploying.


We hope this tutorial has helped you get acquainted with Percy’s visual review platform and workflow. To learn more about how Percy works, feel free to check out these docs:

Happy testing! 💜

The post Using Percy to add visual testing to a Jekyll site appeared first on CSS-Tricks.

CSS-Tricks

, , , , ,
[Top]