MEES Test Automation Framework

Purpose: Automated testing framework for MEES (Minimum Energy Efficiency Standards) compliance application using Playwright.

What It Includes

• Functional Testing: End-to-end user flows and business logic validation
• Accessibility Testing: WCAG 2.2 AA compliance using axe-core
• API Testing: Both user-level (Page API) and service-level (DMS API) boundary testing
• Authentication: GOV.UK One Login integration with stored session state
• Parallel Execution: Multiple test accounts for faster CI/CD execution
• Page Object Model: Maintainable test structure with reusable components
• Browser Support: Configured for Desktop Chrome only (can be extended for multi-browser testing)
• CI/CD Integration: GitHub Actions pipeline with automated reporting

Local Setup

Prerequisites: Node.js 16+, 2+ GOV.UK One Login accounts with MFA enabled

1. Install dependencies:

   npm install
   npx playwright install

2. Setup test accounts Setup test accounts in the tests/config/test-accounts.json that will be used to access MEES application.

The accounts are used to access application when executing functional or non-fucntional tests. One account is used for one worker therefore make sure that number of workers and number of accounts match. Workers can be configured in the playwright.config.ts file, in the defineConfig section.

The noAccessAccount just needs one account that is valid One Login account without access to MEES Compliance Hub page.

3. Create .env file with test credentials:

   # Test Accounts (each needs separate GOV.UK One Login with MFA)
   TEST_ACCOUNT_1_NAME=test user1
   TEST_ACCOUNT_1_EMAIL=[email protected]
   TEST_ACCOUNT_1_PASSWORD=Password123!
   TEST_ACCOUNT_2_EMAIL=[email protected]
   TEST_ACCOUNT_2_PASSWORD=Password456!
   
   # No access account for specific tests
   TEST_NO_ACCESS_EMAIL=[email protected]
   TEST_NO_ACCESS_PASSWORD=NoAccess789!
   
   # Application URL
   BASE_URL=https://your-app-url.com
   
   # API key for DMS API tests
   PROPERTIES_KEY=your-api-key-here
   PROPERTY_KEY=your-api-property-key
   LOCAL_AUTHORITIES_KEY=your-la-key-here
   DMS_BASE_URL=url-to-dms-service

4. Run authentication setup (creates session files): Authentication system includes real-time recovery that handles session issues automatically. Run setup once initially or when switching environments:

   npx playwright test --project=setup

   Note: LandingPage methods handle authentication detection and recovery during test execution automatically.

5. Setup test data (for new environments): Tests contain hardcoded values that must be verified/updated for new environments.

   📄 See TestDataSetup.md for complete test data configuration guide

Essential Commands

# Setup authentication (run first time and when sessions expire)
# For more information look at the 'Authentication Setup' section
npx playwright test --project=setup

# Manual cleanup rarely needed (authentication recovery handles most cases)
# Remove-Item "playwright\auth-states\user-*.json"

# Run all tests
npx playwright test

# Run specific test types
npx playwright test --project=functional        # User flows
npx playwright test --project=non-functional    # Accessibility + validation  
npx playwright test --project=api              # API boundary tests

# Development & debugging
npx playwright test --ui                       # Interactive mode
npx playwright test --headed                   # See browser
npx playwright test -g "test name"            # Run specific test
npx playwright show-report                     # View test results

Authentication Setup

What it is: Uses GOV.UK One Login authentication with stored browser sessions to avoid repeated logins.

Why needed:

• Speed: Avoids 10+ seconds of login flow per test
• Parallel execution: Each worker uses separate authenticated session
• Reliability: Reduces authentication-related test failures

How it works:

1. Setup project logs into each test account and saves session cookies
2. Worker correlation system creates worker-email-map.json for efficient email lookup during test execution
3. Test workers load their assigned session file (user-0.json, user-1.json, etc.) via streamlined fixtures
4. Tests run immediately without login flow
5. LandingPage methods detect authentication loss and call AuthUtils.reAuthenticate() to handle re-authentication using context-stored worker info

Authentication lifecycle:

• Local development: Run setup once, reuse for multiple test runs. LandingPage methods detect session issues and automatically call AuthUtils.reAuthenticate() with zero manual intervention
• CI/CD: Each test project runs setup with streamlined fixtures and LandingPage-based detection to ensure maximum reliability
• Architecture: Context-based email retrieval, separated auth functions, and dedicated reAuthenticate() utility called by LandingPage for robust session management

Test account requirements: Each account must be a complete GOV.UK One Login with MFA enabled and registered in the application.

Reports

Playwright HTML Report: Detailed test execution results with traces and screenshots

npx playwright show-report  # Open after test run

Coverage Report: Auto-generated summary of accessibility and validation testing

• Location: test-results/non-functional-test-coverage.md (and .html)
• Content: Which pages tested, accessibility violations, test status
• Generated: After each non-functional test run

CI/CD Reports: GitHub Actions uploads test reports as downloadable artifacts with 30-day retention.

CI/CD Pipeline Setup

GitHub Actions: Configured in .github/workflows/playwright.yml

• Triggers: Push, PR, manual dispatch
• Jobs: Functional tests and Non-functional tests (each with setup + recovery)
• Workflow: setup → functional tests → setup → non-functional tests (with LandingPage detection calling AuthUtils.reAuthenticate() for robust session recovery)
• Secrets needed: Test account credentials, BASE_URL, API keys
• Reports: Downloadable artifacts with test results and coverage

📄 See CI-CD.md for complete pipeline configuration

Quick Troubleshooting

Authentication issues:

• LandingPage methods automatically detect and handle all authentication issues - zero manual intervention needed
• Context-based architecture - efficient email retrieval and worker correlation ensure maximum reliability
• Separated authentication functions - clean state management with dedicated utilities for setup and recovery
• Run npx playwright test --project=setup only when switching environments or updating credentials
• Ensure test accounts have completed MFA setup in GOV.UK One Login
• Check .env file has correct credentials

Test failures:

• View detailed reports: npx playwright show-report
• Debug with traces: npx playwright test --trace on
• Use interactive mode: npx playwright test --ui

Documentation

• Authentication.md: Detailed auth setup and troubleshooting
• Accessibility.md: WCAG 2.2 AA testing guide
• CI-CD.md: Pipeline configuration and secrets setup
• ContextVerification.md: DOM structure and content validation
• TestDataSetup.md: Test data configuration for new environments

Project Structure

├── tests/
│   ├── pages/           # Page Object Models
│   ├── test/
│   │   ├── functional/  # End-to-end user flows
│   │   ├── non-functional/ # Accessibility + validation
│   │   ├── api/         # API boundary testing
│   │   ├── setup/       # Authentication setup
│   │   └── setup/       # Authentication setup with recovery
│   └── utils/           # Test utilities
├── Documentation/       # Guides and setup instructions
└── playwright.config.ts # Test configuration