Customizing Puppeteer Tests - Part 3

In our previous two  posts, we talked about why we switched to Puppeteer and how to get started running tests. Today, we are going to work on customizing tests by passing in custom parameters.

Reasons for Custom Parameters

We need to be able to pass in custom parameters for debugging and local testing. Our tests currently run through Travis CI, but if a developer needs to run the tests locally, the options are not exactly the same.

• The URL for the test will be different

• The developer usually needs to debug the tests to determine why they failed

We implemented three custom parameters to help with this problem:

1. Ability to pass in a custom URL

2. Ability to run Chrome in a non-headless state

3. Ability to have screenshots taken of failing tests

We are going to go through all of these custom parameters and learn how to implement them.

Pass in a Custom URL

At Outside, we run our tests on a development Tugboat Environment and our local machines. The two base URLS for these environments differ but the paths to specific pages do not. For example, our local machines point to http://outside.test while our Tugboat environments are unique for each build.

We are going to pass a parameter that looks like this: --url={URL}. For our local site, the full command ends up being npm test -- --url=http://outside.test.

Let's get started in setting this up.

1. We need to set up a variable that will be accessible across all files that contains the base URL. In bootstrap.js inside the before function, we are going to name the variable baseURL:

before (async function () {
  ...
  global.baseURL = '';
  ...
});

2. Now we need to access the variables that are passed into the before s function from the command line. In Javascript, these arguments are stored in process.argv. If we console.log them real quick, we can see all that we have access to:

global.baseURL = '';
console.log(process.argv);

3. Head back to your terminal and run npm test -- --url=http://www.outsideonline.com. You should see an array of values printed:

[ '/usr/local/Cellar/node/10.5.0_1/bin/node',
  'bootstrap.js',
  '--recursive',
  'test/',
  '--timeout',
  '30000',
  '--url=http://www.outsideonline.com' ]

4. From the above array, we can see that our custom parameter is the last element. But don't let that fool you! We cannot guarantee that the URL will be the last parameter in this array (remember, we have 2 more custom parameters to create). So we need a way to loop through this list and retrieve the URL:

5. Inside before in bootstrap.js we are going to loop through all the parameters and find the one we need by the url key:

for (var i = 0; i < process.argv.length; i++) {
  var arg = process.argv[i];
  if (arg.includes('--url')) {
    // This is the url argument
  }
}

6. In the above loop, we set arg to be the current iteration value and then check if that string includes url in it. Simple enough, right?

7. Now we need to set the global.baseURL to be the url passed in through the npm test command. However, we need to make note that the url argument right now is the whole string --url=www.outsideonline.com. Thus, we need to modify our code to retrieve only www.outsideonline.com. To retrieve only the url, we are going to split the string at the equal sign using the Javascript function split. split works by creating an array of the values before and after the defined string to split at. In our case, splitting --url=www.outsideonline.com with arg.split("=") will return ['--url', 'www.outsideonline.com']. We can then assume the URL will be at the first index of the split array.

if (arg.includes('url')) {
  // This is the url argument
  global.baseURL = arg.split("=")[1];
}

8. Now that we have our URL, we need to update our tests to use it.

Open up homepage.spec.js and we are going to edit the before function in here:

before (async () => {
  page = await browser.newPage();
  await page.goto(baseURL + '/', { waitUntil: 'networkidle2' });
});

9. We are also going to keep our test from the previous post on Puppeteer:

it("should have the title", async () => {
  expect(await page.title()).to.eql("Outside Online")
});

10. Now, if you run the tests with the url added it should work as it previously did! npm test -- --url=https://www.outsideonline.com

11. Let's create another test to show the value of passing the url through a custom parameter. Inside the test folder, create a file called contact.spec.js. We are going to test the "Contact Us" page found here: https://www.outsideonline.com/contact-us

12. In this test, we are going to make sure the page has the title "Contact Us" using a very similar method:

describe('Contact Page Test', function() {
  before (async () => {
    page = await browser.newPage();
    await page.goto(baseURL + '/contact-us', { waitUntil: 'networkidle2' });
  });

  it("should have the title", async () => {
    expect(await page.title()).to.eql("Contact Us | Outside Online")
  });
});

As you can see above, using the baseURL, it is very easy to change the page you want to test based on the path. If for some reason we needed to test in our local environment, we only have to change the --url parameter to the correct base URL!

View a Chrome Browser during Tests (non-headless)

Having the ability to visually see the Chrome browser instance that tests are running in helps developers quickly debug any problems. Luckily for us, this is an easy flag we just need to switch between true and false.

1. The parameter we are going to pass in is --head to indicate that we want to see the browser (instead of passing in --headless which should be the default).

2. Our npm test script will now look something like this:

npm test -- --url=http://outsideonline.com --head
 

3. Inside of before in bootstrap.js, we need to update that for loop we created before to also check for the head parameter:

global.headlessMode = true;
for (var i = 0; i < process.argv.length; i++) {
  var arg = process.argv[i];
  if (arg.includes('url')) {
    // This is the url argument
    global.baseURL = arg.split("=")[1];
  }
  if (arg.includes("--head")) {
    global.headlessMode = false;
    // Turn off headless mode.
  }
}

4. In this instance, we only need to check if the parameter exists to switch a flag! We are using the parameter headlessMode to determine what gets passed into the puppeteerlaunch command:

global.browser = await puppeteer.launch({headless: global.headlessMode});

5. Lastly, if we are debugging the browser we probably do not want the browser to close after the tests are finished, we want to see what it looks like. So inside the after function in bootstrap.js we just need to create a simple if statement:

if (global.headlessMode) { 
  browser.close();
}

6. And that's it! Go ahead and run npm test -- --url=http://www.outsideonline.com --head and you should see the tests in a browser!

Take Screenshots of Failing Tests

Our last custom parameter is to help us view screenshots of failing tests. Screenshots can be an important part of the workflow to help quickly debug errors or capture the state of a test. This is going to look very similar to the head parameter, we are going to pass a --screenshot parameter.

1. Let's again update before in bootstrap.js to take in this new parameter:

if (arg.includes("screenshot")) {
  // Set to debug mode.
  global.screenshot = true;
}

2. Next up, we are going to implement another mocha function - afterEach. afterEach runs after each test and inside the function, we can access specific parameters about the test. Mainly, we are going to check and see if a test failed or passed. If it failed, we then know we need a screenshot. The afterEach function can go in bootstrap.js because all tests we create will be using this:

afterEach (function() {
  if (global.screenshot && this.currentTest.state === 'failed') {
    global.testFailed = true;
  }
});

3. After a test has failed, we now has a global testFailed flag to trigger a screenshot in that specific test. Note - bootstrap.js does not have all the information for a test, just the base. We need to let the individual test files know if we need a screenshot of a failed test so we get a picture of the right page.

4. Head back to homepage.spec.js and we are going to implement and after function.

after (async () => {
  if (global.testFailed) {
    await page.screenshot({
      path: "homepage_failed.png",
      fullPage: true
    });
    global.testFailed = false;
    await page.close();
    process.exit(1);
  } else {
    await page.close();
  }
});

5. The above function checks if the test has failed based on the testFailed flag. If the test failed, we take a full page screenshot, reset the flag, close the page, and exit the process.

6. Unfortunately, the above code works best inside each test file so there will be some code duplication across tests. The path setting makes sure that no screenshot overrides another tests screenshot by setting the filename to be the one of the test. The screenshot will be saved in the base directory where we run the npm test command from.

7. To test and make sure this works, let's edit homepage.spec.js to expect a different title - like "Outside Magazine"

it("should have the title", async () => {
  expect(await page.title()).to.eql("Outside Magazine")
});

8. We know this one will fail, so when we run npm test -- --url=http://www.outsideonline.com --screenshot we should get a generated screenshot! Look for a file named homepage_failed.png.

Recap & Final Thoughts

Add custom parameters to your npm script is fairly simple once you get the hang of it. From there, you can easily customize your tests based on these parameters. Even with the custom parameters we have created, there is room for improvement. Stricter checking of the parameters would be a good first step to rule out any unintended use cases. With the custom url, headless mode, and screenshots, our tests are now easier to manage and debug if something ever fails. Check out the Puppeteer Documentation, Mocha, and Chai to learn more!

from Outside Magazine: All https://ift.tt/2PqmnN5