Automated Testing of Barcode Scanning Apps with Playwright

Motivation

Web apps that use inputs from physical sensors instead of only user inputs like mouse clicks, touches and keystrokes are inherently more difficult to test. It’s all to easy to throw your hands in the air, say I give up! and and resort to manual testing instead.

Barcode scanning apps recognize barcodes in the video stream provided by the smartphone’s camera. 📷

What if we had a way to test these kinds of apps in an automated fashion?

Playwright

Playwright is an open-source automation framework by Microsoft for end-to-end testing of web apps. It allows developers to write tests that interact with web pages in a way that mimics real user behavior, using a real browser.

Older folks like the author of this post may remember Selenium, another browser automation framework. Playwright is similar in spirit, but newer and enjoys commercial backing from Microsoft. Writing tests with Playwright feels like a breeze, and is… gasp… fun!

You can use Playwright locally on your machine, e.g. for development, and then use a modern application reliability monitoring platform like Checkly or BetterStack to run your tests on a schedule and get alerts if something breaks.

If you’re planning on writing automated tests for your web app, you should give Playwright a try. Shout out to Stefan Judis who is really knowledgeable on this topic.

Test Scenario: Scanning an ISBN Barcode

Assume we have an app that scans the ISBN barcodes printed on books, and we want to test that the app recognizes the barcode correctly, and performs some follow-up processing (e.g. a lookup to OpenLibrary).

First, we have to produce a video with a smartphone camera, simulating the user placing the barcode into the center of the picture. You can do this easily with the native iOS or Android camera app.

Here’s our simulated user interaction: placing the barcode of the excellent Digital Image Processing book into view.

How do we use this video for automated testing?

Playwright Script

I won’t go into details on how to write tests with Playwright, as I couldn’t do a better job than the official documentation. So without further ado, here’s the Playwright script that tests reading an ISBN barcode using the STRICH Demo App.

import {test, expect, chromium} from '@playwright/test';

test('Scans EAN/UPC Book Barcode', async () => {
  // mock the camera using a prepared video file
  const browser = await chromium.launch({
    args: [
      "--use-fake-ui-for-media-stream",
      "--use-fake-device-for-media-stream",
      "--use-file-for-fake-video-capture=./test-videos/book_barcode.mjpeg",
    ]
  });

  // launch a browser context that has permission to access the camera
  const context = await browser.newContext();
  const page = await context.newPage();

  // check start page is displayed
  await page.goto('https://demo.strich.io/');
  await expect(page).toHaveTitle(/JavaScript Barcode Scanner Demo/);

  // select 1D retail barcodes and start scanning
  await page.getByText("1D Retail").click();
  await page.getByText("START SCANNING").click();

  // fake video stream starts, barcode should be read
  await expect(page.getByText('9781292223049')).toBeVisible();

  // clean up
  await context.close();
});

The beef that enables testing the camera stream are the custom launch arguments provided to Chromium:

• --use-fake-ui-for-media-stream: overrides browser controls that ask the user for permission to access the camera.
• --use-fake-device-for-media-stream: provides a fake video device to the page.
• --use-file-for-fake-video-capture=./test-videos/book_barcode.mjpeg: supplies the video stream for the fake video device.

The video file must be provided in the MJPEG format. You can easily convert an existing H.264 video to MJPEG using ffmpeg, the Swiss Army Knife of video transcoding

ffmpeg -i my_video.mp4 my_video.mjpeg

Note: Some sources on the web mention that Chrome supports H.264 videos for video stream faking, but I didn’t manage to get it to work.

If you do not supply a file with the fake video stream, the video stream will contain a test pattern instead. That can be useful in itself: say you’re only interested in testing that an app can place a video call to someone, but you’re not interested in the video’s contents.

For more information on testing media devices with Chrome, see the WebRTC Testing documentation.

Running the Test

It’s time to run the test!

% npx playwright test

Running 1 test using 1 worker
  1 passed (3.9s)

To open last HTML report run:

  npx playwright show-report

I don’t know about you, but there’s something uniquely satisfying in seeing video proof of the test completing as expected. 🤩

Conclusion

Playwright is a neat tool for programmatic end-to-end testing of web apps. If we leverage Chromium’s WebRTC faking capabilities, we can use it to elegantly write end-to-end tests for apps that scan barcodes, QR Codes and other things using the camera. 🎉