Health Information: Use of Information for Health Service Planning and Evaluation

Introduction

Learning objectives: You will learn about the use of information for health service planning and evaluation.

Read the resource text below.

Resource text

Planning

Commissioning or planning health care begins with an assessment of need [1].

This requires information on:

Size and structure of underlying population
Areas of met and unmet need
Incidence and / or prevalence of disease(s) of interest
Effectiveness of interventions available
Relative cost-effectiveness of those interventions
Current services, including capacity, quality, effectiveness, efficiency
Prioritisation processes, including political priorities such as waiting times, service targets etc

Typically this would be in the context of an assessment of the population's overall health which requires identifying and reviewing data sources such as:

Local data, such as a local specialised survey
Routine local and / or national statistics
Ad hoc data
Relevant published surveys
Qualitative data as well as quantitative sources
Trends in incidence and outcome

When data are not available directly, such as when future demand needs to be predicted or when estimating the impact of service re-design, the available information may be used to formulate the assumptions of a model, and to calibrate it.

Issues to consider when using information for local health service planning include:

The applicability of national data to the local population and conditions. If only national data are available, when applied locally the data may need to be adjusted for the composition of the population such as age, ethnicity and deprivation.
Many data sources cannot give any indication of who has the health condition of interest but is not accessing services.
National or regional data may mask inequalities at smaller geographical levels such as wards.
Data are subject to random fluctuations. Small area data may have very small numbers. Indicators of precision, such as confidence intervals, should always be used in these cases.
Confidentiality issues with small numbers may mean the information is suppressed to prevent disclosure when the data source is used in combination with other sources.
How the data have been obtained.
For what purpose the data were originally collected. If different from the new purpose, there may be biases within the data that hamper a correct assessment of the new purpose. For example, a clinician records patient information to help treat that individual, not for population based analyses.
This could result in inconsistent coding. Also, there are legal restrictions on using personal data for purposes other than those for which it was collected.
How timely the data are.
How complete the data are.

Evaluation

The process of evaluation and monitoring should be a continuous one, and lessons learned should inform future developments [2]. A crucial part of evaluating health care is assessing the quality of both health and social care and public health interventions. In the UK, at least 10% of inpatient episodes lead to unintended harm. Around half of these cases are preventable [3].

Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. (Lohr, 1990)

Services can be evaluated using the Donabedian framework of Structure-Process-Outcome; all three elements are required to give an overall picture of the quality of a service [4].

Structure: examine the provision of facilities and staff available. This could include:

numbers of hospital beds
ratio of doctors, nurses and other staff to patients and to each other
comparisons across different geographical regions
ease of access and opening hours to a clinic or screening service

Process: what is done for and to a patient or a population, and how well. This could include:

methods by which patients are identified as suitable to be recorded on a disease register
how thoroughly the diagnostic criteria had been determined
validity and reliability of diagnostic tests
time taken from diagnosis to treatment
frequency of patient follow up
how people are recruited and involved in a process, e.g. a screening or immunisation programme or substance abuse treatment

Outcomes

Did the patient get better?
Were there complications?
Did the patient feel satisfied with the service?
Has there been a recurrence?
Was there a reduction in incidence in a population?
Has coverage improved?
Has life expectancy increased?

Maxwell's dimensions of quality take account of the population as well as patient care [5]. The dimensions include:

Access to services (for example, taking a population based approach, do some sub-groups find services easier to access than other sub-groups? This could be about the physical location of services as well as different attitudes on seeking care)
Relevance to need (for the whole community)
Effectiveness (on an individual patient basis)
Equity (could the service in any way be made more fair?)
Acceptability (is this a procedure which many find too uncomfortable/embarrassing/painful to undertake, and so avoid treatment?)
Efficiency and economy

Example of where structure, process, and outcome framework could be applied:

Death of a patient due to maladministration of an anti-cancer drug [2]:

In this case a drug which should only ever be administered into a vein was inserted into the spine. The patient died and the investigation found 40 system failures, including:

Structure

The syringes for the two different methods of injecting were labelled in a confusing manner. A senior doctor was not present.

Process

Two junior doctors were allowed to dispense these drugs without adequate training. No formal induction programme was in place for starting junior doctors.

Outcome

Death of a teenager.

Evaluations can lead to service re-design which then needs to be monitored regularly, such as:

Clearer protocols
Formal training
Induction programmes
Redesign of equipment
Re-balancing of junior and senior staff

Information sources for evaluating health services

Evaluation involves comparison of the actual outcome of an intervention with the intended outcome. While it often involves an economic component, this is by no means the whole of evaluation.

Depending on what needs to be evaluated, the information sources listed in the planning section above will be useful to varying degrees. For example, if patients' post-operative mortality rates are being measured and compared with other areas, Hospital Episode Statistics (HES) will be required with linkage to ONS mortality data, since HES only captures mortality within hospital stays.

Patient satisfaction may be determined by means of a survey which captures qualitative information.

If access to a service is under-represented from one ethnic group compared with what would be expected for that population, then ethnicity needs to be captured consistently as part of an appropriate process (e.g. when it is asked for, how it is asked for, which ethnic group categories are used). However, there are legal restrictions on collecting these data: hospitals may only collect them for admitted patients, not for outpatients or patients awaiting services.

Nationally, any health care system must have robust mechanisms in place for assuring the quality and safety of services provided to patients that enable comparisons across health care institutions and geographical regions. This requires consistent methods of capturing and coding data. Within the NHS, a framework for ensuring health care quality includes:

Clear national standards (e.g. from organisations that issue clinical guideline such as NICE or the medical Royal Colleges)
Local clinical governance mechanisms for implementing quality assurance programmes as well as patient safety and quality improvements
Inspection and audit programmes (Healthcare Commission)
National audits such as the confidential enquiries into Perioperative Deaths, Maternal deaths, Suicide and Homicide by People with Mental Illness

Monitoring

In addition to clinical audits and inspections, ongoing monitoring of public health issues can be undertaken using a variety of websites and databases.

NHS providers routinely send data to commissioners through the Connecting for Health Secondary Uses Service (formerly via the NHS-wide clearing service).

Independent providers of services, such as terminations of pregnancy, should supply datasets detailing their activities to support their invoices, and these data are used to monitor progress against plans. These data include clinical, personal, and financial data, but patients are anonymised before transmission.

Incidence of communicable disease is monitoring continuously by the Health Protection Agency, who are informed through the notifiable diseases process.

The London Health Observatory, alongside the other nine Public Health Observatories in England, produces public health performance management reports on a quarterly basis for each PCT in London, which monitor a range of indicators such as take up of screening services, immunisation uptake, and participants in smoking cessation services. It takes data from the Department of Health, Health Protection Agency and Compendium websites, as well as data from the UNIFY database via NHS Net. This database stores data on local delivery plan targets, and all PCTs need to upload data to this database on a regular basis.

The National Drug Treatment Agency collates and analyses data about substance misuse on a monthly basis, data that local drug action teams use to monitor their services against target.

Healthcare needs assessment

Health needs assessment (HNA) is "a systematic method of identifying the health and healthcare needs of a population and making recommendations for changes to meet these needs." (Wright J. Assessing health needs. In Pencheon D, Guest C, Melzer D, Gray JAM editors. The Oxford handbook of public health practice. Oxford: Oxford University Press 2001.)

In this context "need" is defined as the ability to benefit from a healthcare intervention. The aim of HNA is to maximise the appropriate delivery of effective healthcare, and minimise both the provision of ineffective care and the existence of unmet need for healthcare. HNA provides a systematic framework for undertaking a complex and important task in an evidence-based way. The principles are widely used by public health practitioners. Assessing the health needs of the population is one of the functions of PCTs.

The practice of HNA first became widespread in the early 1990s, at the time of the introduction of the NHS 'internal market'. HNA was used to improve efficiency-aiming to identify which cost-effective services should be provided to meet population health needs, and which ineffective services should not. More recently, with the rise of the inequalities agenda, the emphasis of HNA has shifted more towards achieving equity, i.e. fair user access to health services and fair allocation of resources by providers to services. The shift to a primary-care led NHS has seen Primary Care Trusts (PCTs) take on the function of assessing the health needs of the population. Primary care is seen as well placed to assess needs for health services at a local level.

Stevens and Raftery (Stevens A, Raftery J editors. Health care needs assessment: the epidemiologically based needs assessment reviews. Oxford: Radcliffe Medical Press; 1994) have described the common approaches to assessing population needs for healthcare. These are characterised as the corporate, comparative and epidemiological approaches to HNA.

The corporate approach "involves the structured collection of the knowledge and views of informants on health care services and needs". Informants are the stakeholders in the issue being addressed by the needs assessment. They might include clinical workers in both primary and secondary care, health service managers, commissioners of services, experts in the field and service users. Advantages of this approach include making the needs assessment responsive to local concerns and fostering "local ownership" of the issues. Disadvantages of this approach (if carried out in isolation) are that it determines demands rather than needs, and stakeholder concerns may be influenced by political agendas.
The comparative approach involves the comparison of levels of service provision between different localities. These could be cross-national comparisons of the levels of service provision (e.g. comparing England with other countries in Western Europe), or could be at a more local level (e.g. comparing the service provision in one town with another that has a similar demography). This approach is usually fairly quick to achieve and inexpensive, but the disadvantages include the problems of finding a sufficiently similar locality for an accurate comparison.
The epidemiological approach to HNA has three elements: (i) determining the incidence and/or prevalence of the health problem, (ii) identifying the effectiveness (and cost-effectiveness) of existing interventions for the problem, and (iii) identifying the current level of service provision. This combination of epidemiology (health status assessments) and evidence (effectiveness/cost-effectiveness) has also been described as the evidence-based approach to HNA. While this is a systematic and objective approach, its disadvantages lie in the frequent lack of existing local epidemiological data, and the lack of evidence for certain interventions - particularly for population subgroups such as the elderly (NB lack of evidence does not imply evidence of lack!). Carrying out new epidemiological work is also costly and time consuming.

See the Healthcare Needs Assessment

Health Impact Assessment (HIA)

Paragraph taken from APHO: The HIA Gateway

Health impact assessment is intended to help make decisions by predicting the health consequences if a proposal is implemented. In addition to assessing the health consequences, it also produces recommendations as to how the good consequences for health could be maximised and how the bad consequences could be avoided or minimised. It aims to predict not only the overall consequences for a population, but also the distribution of health impacts in that population, that is, which groups benefit and which groups lose, or at least benefit less. HIA may be used to assess policies, programmes or projects. Those undertaking the HIA should always carefully consider the requirements and the concerns of the decision makers whom they intend to inform. The principles and methods of HIA can be used to assess health consequences as part of another impact assessment (such as SEA or EIA). Whether the health consequences of a proposal are assessed with an HIA or as part of some other assessment is unimportant. The important thing is that they are thoroughly assessed.

There are three types of HIA:

Concurrent - conducted during the implementation of a proposal

Retrospective - conducted after the implementation of a proposal

For further information see the Health Management Specialist Library.

Public Health Observatories

There is a network of 12 public health observatories (PHOs) working across the five nations of England, Scotland, Wales, Northern Ireland and the Republic of Ireland. They produce information, data and intelligence on people's health and health care for practitioners, policy makers and the wider community. They were established in 2000. Among other activities they produce health profiles reports. There are also three topic-based observatories: on obesity, injury, and child and maternal health.

See the Association of Public Health Observatories.

Healthcare Commission

Since 2005, the Healthcare Commission has run an annual health check to assess and rate the performance of every NHS Trust. It scores Trusts on 4-point scales for quality of services and for use of resources (scored as excellent or good or fair or weak). In 2007/8 the Health Protection Agency, NHS Direct, and the NHS Blood and Transplant service were included for the first time.