Provider systems MUST NOT use this volumetric model as a ‘gold standard’, but MAY use it as the basis of developing their own volumetric models.
Provider systems MUST through V&P profiling and solution assurance activities demonstrate how the system can scale as demand increases.
A command API is any API which performs a user-initiated operation with a side-effect (for example, booking an appointment or registering a patient).
Provider systems MUST process command API calls in <250ms.
Provider systems SHOULD process command API calls in <100ms, as this is the limit beyond which a user no longer feels that the system is reacting instantaneously.
A query API is any API which performs a user-initiated retrieval of data without any side-effects (for example, searching for a patient’s medication history).
Provider systems MUST process query API calls in <3000ms.
Provider systems SHOULD process query API calls in <1000ms, as this is the limit beyond which a user feels their workflow has been interrupted.
Volume and performance testing
Suppliers of Provider solutions are expected to undertake provider led V&P testing of their solutions.
V&P testing model
Suppliers MUST submit for review a high-level V&P testing model document that covers the stages of testing, details of the environment and how they intend to test.
Suppliers’ test approaches SHOULD include a LOAD and a RAMP test and ideally a SOAK test. Suppliers MUST supply comprehensive results of the test including timings for round trip API call/response against the message sizes used and the TPS at the time the request was made.
V&P testing infrastructure scaling
Predictive models of load are provided to give an indication of the likely volume and profile of full production roll out.
If suppliers are scaling their infrastructure solution over time (such as during First of Type) then the V&P testing must be performed and submitted for review.
Suppliers of provider solutions are expected to scale their infrastructure solutions in line with delivering services within the SLA and using the V&P Testing to proactively monitor and add infrastructure before it is saturated/overloaded.
A plan outlining the points at which infrastructure will be scaled up should be provided after V&P testing is performed.
V&P test environment
Test environments MUST simulate consumer applications making API calls against simulated test data (patient records, diaries, tasks, and so on.)
Test data MUST be populated with realistic complexity, depth and volume. That is, in the case of patients, the data should be representative of clinical records of a mix of healthy patients and patients with multiple long-term conditions.
If a small set of test data is repeatedly used as part of the V&P tests then test setup SHOULD seek to minimise the effects of caching - for example, within API middleware (the data from a small number of patients repeatedly queried in quick succession could be served from cache which would invalidate test results).
Suppliers SHOULD test API call volumes against a refined Volumetric Model1.
During the LOAD test, the timings for end to end API calls MUST NOT exceed the maximum stated (250ms for command APIs, 3000ms for query APIs) and SHOULD NOT exceed the lower limits (100ms and 1000ms respectively).
Results for the RAMP test MUST include the tps and message size profile at the threshold where the above limits were exceeded, if applicable.
If a SOAK test is performed, results MUST be provided.
1Note, V&P test profiles will differ according to the supplier based on the proportion of patient population whose GP records are held with that supplier.
External documents/policy documents