There will be organisational and project specific test processes underpinned by appropriate artefacts.

Example evidence could include:

• Test Policy
• Test Strategy
• Test Plans
• Test Specifications
• Risk identification process as applied to Solution testing 
• How is testing within development phases established, such as;
  • Unit,
  • Module
  • System
  • System Integration
  • Integration
  • System Integration
  • User Acceptance Testing
• How testing automation is incorporated into the software build and distribution process, such as continuous build integration

• Testing Qualifications within supplier organisation

• Retrospective process improvement

There will be Test Management Processes covering test planning and design.

Example evidence could include:

• Test Automation (automated scope as opposed to manual testing automated)
• Test Regression and process for establishing coverage scope during a specific and subsequent Solution releases
• Covering both Functional and non-Functional Testing
• Techniques used - Risk-based, static/dynamic testing, testing heuristics, exploratory testing, smoke testing,...

There will be Test Management Processes covering test execution and management.

Example evidence could include:

• Tooling used, explaining how results are captured,

• Test Execution Management

• Repeatability
• Defect/Incident Management
• Test Environment Management

There will be Test Management Processes covering test completion and Reporting.

Example evidence could include:

• Comprehensive test results captured within test tooling
• Incident management on completion
• Quality Criteria for exiting testing

Requirements

Level

TEST-ENV-1

A Digital Care Services supplier whose Solution interoperates with NHS Digital national services (such as PDS, ePS, SCR, eRS, NHAIS, GPES, etc), or provides IM1 or GP Connect APIs, shall provide access to instances of production versions of their Solution(s), such as a Virtual Machine (VM) image, to be installed and maintained in the NHS Digital test laboratory. Or alternatively, provide access to supplier data centre hosted, or cloud-hosted versions, which allow multiple concurrency of connecting systems, such as API consumers for testing.

In this context, the production version means a version of the Solution which is reflective of the supplier’s LIVE estate for that Solution. Where a supplier has multiple instances and version, the supplier will provide access to the majority installed version.

TEST-ENV-2

The Supplier must make access available to the latest version 5 days after release in live. If it is provisioned for installation by NHS Digital, then the software installation package must be accompanied with appropriate instructions for installation and the release notes for the release being installed.

TEST-ENV-3

The Supplier shall provide access or provision installation any additional requested versions of its Solution(s) within 10 working days of the NHS Digital’s request. These versions could either be compliant or versions for which compliance are being sought.

TEST-ENV-4

The Supplier shall ensure that an up to date version of each Supplier Release of its Compliant Solution (including Live APIs) in the suppliers’ LIVE estate is installed, maintained in their own test environment and connected to NHS Digitals’ Path to Live environments (DEV and INT) as required to support the Suppliers’ own testing of a Supplier Release or where required, to support the testing of another Supplier Release and for NHS Digitals’ assurance of the release.

TEST-ENV-5

The Supplier must ensure that a non-functional test environment is available to do Volumetrics and Performance testing for their Solution. 

TEST-ENV-6

The Supplier shall support end-user organisations to aid training of their users on the supplier Solution(s) with Spine connectivity, including all Core Messaging and CIS services, enabled through integration with the SPINE Training environments.

TEST-ENV-7

The Supplier must ensure that their test environments support robust access controls (security features) both at Solution level and Infrastructure level so that only authorised users and authorised external connecting supplier Solutions can access the test environment.

TEST-ENV-8

The Supplier must ensure that the test environment(s) availability times and any scheduled downtimes for maintenance activities are made known to all the external connecting Solution suppliers and NHS Digital accessing the supplier hosted test environment. Any changes to the known availability times or scheduled downtimes should be communicated to the suppliers and NHS Digital at least two working days in advance. It is acknowledged that there might be compelling reasons in certain circumstances to perform maintenance activities outside the scheduled slots.

TEST-ENV-9

The Supplier must either provide appropriate Solution test data (including patient clinical data) in their test environment(s) to aid testing of the external connecting Solution suppliers and for NHS Digital to do assurance activities, or when provided by NHS Digital, the supplier must preload any provided test data (such as PDS test patients). If specified by NHS Digital, the supplier must ensure patient test data meets a minimum set of clinical data.

The test data must contain Synthetic Data only. It must not contain any sensitive or personally identifiable patient data.

The Supplier must ensure that any maintenance releases / emergency patches deployed to their test environment does NOT erase any of the existing test data as this might adversely impact any ongoing testing.

The Supplier shall ensure that the test environment is able to support concurrent testing by multiple external connecting Solution suppliers. The Solution test data must be discrete for each external connecting Solution supplier such that they can test independently of others.

TEST-ENV-11

The Supplier shall support backup and restore functions, to allow restoration of the test environment(s) back to the original clean state or a previously backed-up state.

TEST-ENV-12

The Supplier shall ensure that the performance of the test Solution is comparable to the LIVE service levels for the restricted volumes in the Test environment, in order to provide a realistic expectation of performance behaviour in the LIVE estate

The Supplier must provide any necessary documentation pertaining to external interfaces to allow the integration of connecting systems.

Requirements

Level

The Supplier must support use of appropriate Solution test data (including patient clinical data) in their live environment(s). That support will ensure that test data can be used and exist in live environments in accordance with NHS Digital guidance on the Use of Synthetic Data in Live Environments which is published as part of Training for Providers.