At a glance
Summary version
DPC’s challenge
Victoria’s Department of Premier and Cabinet (DPC) is responsible for Victoria’s whole-of-government digital platform, Single Digital Presence. Like most active platforms, SDP requires a lot of testing. DPC was developing an automation framework to reduce reliance on manual testing.
DPC’s transformation
Salsa provided resources to support SDP’s internal QA team set up global, automated testing across both the backend and frontend. The automation covers end-to-end test runs using and , integrated with .
The outcomes
- Improved test stability
- Reusable test code that’s easy to maintain
- Improved test coverage, from 20 automated tests to 100 during the project
- Reduced testing time, which means releases can happen faster
Detailed version
DPC’s challenge — time-consuming manual testing
Victoria’s DPC is responsible for Victoria’s whole-of-government digital platform, Single Digital Presence.
SDP was very active, with a lot of dev work happening on the platform to improve existing features and add new features. However, all changes needed extensive testing. In the past, SDP had refactored existing tests to test new features or content types. SDP wanted to reduce the time to get code changes out the door and increase confidence for releases.
SDP needed an automation framework to reduce reliance on manual testing. They also wanted to:
- Improve test framework reliability
- Improve reusability of test code
- Create a testing architecture built on a component-based design
SDP’s internal team had been working on this problem and had come up with a solution. However, they needed extra resources to review the framework and then help them write the automated tests.
DPC’s transformation — developing sustainable, automated tests for SDP
The solution was to create more global test scenarios and map them to components that are reused across multiple content types. This leverages the component-based design by creating component-based automated tests. For example, an accordion component should have the same test on different content types, so it can be reused easily, rather than receiving its own test script over and over again with slight differences.
The framework was set up by SDP, including choosing the tools for the project — and , integrated with . The Salsa team then worked with SDP to extend on that framework, including adding visual testing and cross-browser testing. Salsa also helped in the research phase, comparing the cross-browser testing tools on the market and setting up a proof of concept. SDP then chose BrowserStack .
The SDP automated tests imitate user behaviour in both the backend (BE) and the frontend (FE). The BE is the content management system (CMS) that content authors use to create web pages, and the FE is what people see.
The automated tests imitate:
- An editor logging into the Drupal CMS and adding some content, using most of the fields available
- Saving the content
- Checking the decoupled FE (Ripple) to make sure the FE content matches the entered data
The automated tests are written in Nightwatch JS and the human-readable test scripts are written in Cucumber's Gherkin script. The tests are run locally first and then go into the test repository and are run on every pull request change in GitHub via CircleCI.
Salsa’s QA engineers were embedded into the SDP sprint team, working with SDP’s internal test engineers. SDP’s QA team provided test steps for Salsa to automate. Some of the specific features that Salsa created automated tests for include:
- Webforms (BE) and embedded webforms (FE)
- Image gallery
- Campaign with video (BE)
- Adding campaigns (BE)
- Campaigns (FE)
- Timeline (BE and FE)
- Site navigation tests
- Embedded video (BE and FE)
- The contact module (BE)
- Navigation cards (BE and FE)
- Promo card (BE and FE)
- Chrome error message
- Numbered lists (BE and FE)
- Callout and blockquotes
- Table of contents on landing pages
- Bullet formatting in WYSIWYG editor
- Paragraph formatting
- Footer elements
- Header elements
- Accordions (reusable across content types)
The project used the following step-by-step process provided by SDP:
- Pull the relevant code from the GitHub repository
- Make a separate feature branch for whatever feature was being worked on
- Write the code on that branch (using Nightwatch to create automated test with JavaScript coding)
- Push the code to GitHub for SDP review
- Code run on 5 different environments to test stability of the automated test
- Salsa PR branch merged into the master branch
This framework was also integrated with the pipeline (continuous integration with the project using CircleCI).
The outcomes — faster testing and shorter dev release cycle
The high-level outcomes for the QA automation project include:
- An improved testing framework
- Overall reduction in time taken to develop and test new features
- Improved test stability — refactoring the tests and test framework to improve stability
- Test maintainability — restructuring the tests and test framework so the code is reusable and easy to maintain
- Improved test coverage — from 20 automated tests to 100 during the project
- Reduced testing time, which means releases can happen faster
- Improved developer confidence when adding or changing features
About the Victorian Department of Premier and Cabinet
The Victorian Department of Premier and Cabinet (DPC) leads a whole-of-government approach to policy and deliverables. It’s responsible for several elements of Victoria’s digital engagement, including Single Digital Presence, vic.gov.au, data.vic.gov.au and engage.vic.gov.au.