Indicators for Healthcare Quality and Safety

Definitions  

Quality of care has been defined as 'the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge' [1], and can be divided into different dimensions according to the aspects of care being assessed [2].

Indicators have been defined in several different ways, as:

1. Measures that assess a particular health care process or outcome [3]
2. Quantitative measures that can be used to monitor and evaluate the quality of important governance, management, clinical, and support functions that affect patient outcomes [4]
3. Measurement tools, screens, or flags that are used as guides to monitor, evaluate, and improve the quality of patient care, clinical support services, and organizational function that affect patient outcomes [5].

Indicators provide a quantitative basis for clinicians, organizations, and planners aiming to achieve improvement in care and the processes by which patient care is provided.

Indicator measurement and monitoring serve many purposes. They make it possible to:

• Document the quality of care
• Make comparisons (benchmarking) over time and between places (e.g. hospitals)
• Make judgments and set priorities (e.g. choosing a hospital or surgery, or organizing medical care)
• Support accountability, regulation, and accreditation
• Support quality improvement
• Support patient choice of providers

However, indicators are not a direct measure of quality. Because quality is multi-dimensional, understanding quality requires many different measures.

Indicators are based on standards of care. These can be evidence-based and derived from the academic literature (e.g. Cochrane Collaboration, meta-analyses, or randomized controlled trials) or, when scientific evidence is lacking, determined by an expert panel of health professionals in a consensus process based on their experience. Thus, indicators and standards can be described according to the strength of scientific evidence for their ability to predict outcomes. [6]

Key Characteristics of an Ideal Indicator  

An ideal indicator would have the following key characteristics:

1. the indicator is based on agreed definitions, and described exhaustively and exclusively
2. the indicator is highly or optimally specific and sensitive, i.e. it detects few false positives and false negatives
3. the indicator is valid and reliable
4. the indicator is discriminates well
5. the indicator is relates to clearly identifiable events for the user (e.g. if meant for clinical providers, it is relevant to clinical practice)
6. the indicator is permits useful comparisons
7. the indicator is evidence-based

Each indicator must be defined in detail, with explicit data specifications in order to be specific and sensitive.

Indicators may vary in their validity and reliability.

• Validity: is the degree to which the indicator measures what it is intended to measure, i.e. the result of the measurement corresponds to the true state of the phenomenon being measured. A valid indicator discriminates between care otherwise known to be of good or bad quality and concurs with other measures that are intended to measure the same dimension of quality.
• Reliability: is the extent to which repeated measurements of a stable phenomenon by different data collectors, judges, or instruments, at different times and places, get similar results. Reliability is important when using an indicator to make comparisons among groups or within groups over time. A valid indicator must be reproducible and consistent.

Indicators should be based on the best available evidence, i.e. the integration of best research evidence with clinical expertise and patient values [7]. The strength of evidence for an indicator will determine its scientific soundness or the likelihood that improvement in the indicator will produce consistent and credible improvements in the quality of care.

Types of Indicators  

Rate-based indicators  

A rate-based indicator uses data about events that are expected to occur with some frequency. These can be expressed as proportions or rates (proportions within a given time period), ratios, or mean values for a sample population. To permit comparisons among providers or trends over time, proportion- or rate-based indicators need both a numerator and a denominator specifying the population at risk for an event and the period of time over which the event may take place.

Examples of Rate-based indicators

1. Clean and contaminated wound infection
   • Numerator: the number of patients who develop wound infection from the fifth post-operative day after clean surgery
   • Denominator: the total number of patients undergoing clean surgery within the time period under study who have a post-operative length of stay of 5 days.
2. Hospital-acquired bacteraemia
   • Numerator: total number of patients who acquire bacteraemia
   • Denominator: total number of patients in hospital during the study period

Sentinel Events  

A sentinel indicator identifies individual events or phenomena that are intrinsically undesirable, and always trigger further analysis and investigation. Each incident would trigger an investigation. Sentinel events represent the extreme of poor performance and they are generally used for risk management.

Examples of Sentinel Indicators

1. Numbers of patients who die during surgery
2. Numbers of patients who die during the perinatal period

Generic Indicators and Disease-Specific Indicators  

Generic indicators measure aspects of care that are relevant to most patients, while disease-specific indicators are diagnosis-specific and measure particular aspects of care related to specific diseases. Both can focus on structure, process, or outcome. Generic indicators may be difficult to interpret, especially when making comparisons among hospitals or providers, because there may be profound differences in patient mix. Disease-specific outcome indicators can be used to compare hospitals and plans, when data are risk-adjusted.

1. Examples of Generic Indicators
   • Proportion of specialists to other doctors
   • Registered patients in the emergency department > 6 hours
   • Unscheduled returns to the operating room
   • In-patient mortality
2. Examples of Disease-specific indicators
   • Proportion of cardiologists to other doctors treating patients with heart failure at the department of cardiology
   • Proportion of patients with stroke treated with thrombocyte inhibitor < 24 hours after admission
   • Proportion of patients with hip fracture who need a second operation
   • Proportion of patients with lung cancer who are alive 30 days after surgery
   • Proportion of patients with myocardial infarct who receive a beta-blocker within 24 hours of admission
   • Proportion of patients with diabetes mellitus who receive a retinal exam annually

Structure Indicators  

'Structure' denotes the attributes of the settings in which care occurs, affecting the system's ability to meet the health care needs of individual patients or a community. This includes the attributes of material resources (such as facilities, equipment, and financing), of human resources (such as the number and qualifications of personnel), and of organizational structure (such as medical staff, organization, methods of peer review, and methods or reimbursement). The assessment of structure is a judgment on whether care is being provided under conditions that are either conducive or inimical to the provision of good care.

Of the structural indicators, measures that predict variations in processes or outcomes of care have the greatest utility, and such measures often focus on hospital or provider characteristics. Regarding pediatric quality of care, one consistent finding has been that hospitals caring for higher volumes of patients with similar conditions have better adjusted mortality rates [8], which is also true for surgical procedures [9]. Fourteen structural characteristics that have been demonstrated to be related to evidence-based processes or to outcomes have been identified [10].

Examples of Indicators Related to Structure

1. Proportion of specialists to other doctors
2. Access to specific technologies (e.g. MRI scan)
3. Access of specific units (e.g. stroke units)
4. Clinical guidelines revised every 2nd year
5. Physiotherapists assigned to specific units

Process Indicators  

'Process' denotes what is actually done in giving and receiving care (i.e. the practitioner's activities in making a diagnosis, recommending or implementing treatment, or other interaction with the patient), and how well it was done. Processes are a series of inter-related activities undertaken to achieve objectives. Process indicators measure the activities and tasks in patient episodes of care.

Some authors include the patient's activities in seeking care and carrying it out in their definition of the health care process. Others limit this term to care that health care providers are giving. It may be argued that providers are not accountable for the patient’s activities and these, therefore, do not constitute part of the quality of care, but rather fall into the realm of patient characteristics and behavior that influence patients' health outcomes.

The advantages and disadvantages of process versus outcome measures are: first, elements of the process of care do not signify quality until demonstrating their relationship to desirable outcomes validates them. Once it has been established that certain procedures used in specified situations or for certain patients are clearly associated with good results, the presence or absence of these procedures for such patients or situations can be accepted as evidence of good or bad quality.

Process indicators are especially useful when:

• quality improvement is the goal of the measurement process
• an explanation is sought for why specific providers achieve particular outcomes
• short time frames are necessary
• performance of low volume providers is of interest
• when tools to adjust or stratify for patient factors are lacking

Comparisons of process data are easier to interpret and more sensitive to small differences than comparisons of outcomes data. A process indicator can measure whether or not a stroke patient receives the right medication, whereas 30-day mortality rates from stroke patients may be difficult to interpret.

Examples of Indicators Related to Process

1. Proportion of patients with diabetes given regular foot care
2. Proportion of patients with myocardial infarction who received thrombolyses
3. Proportion of patients assessed by a doctor within 24 hours of referral
4. Proportion of patients treated according to clinical guidelines

Outcomes Indicators  

'Outcome' measures attempt to describe the effects of care on the health status of patients and populations. Improvements in the patient's knowledge and salutary changes in the patient's behavior may be included under a broad definition of outcome, and so may represent the degree of the patient’s satisfaction with care.

Outcomes are states of health or events that follow care, and that may be affected by health care. An ideal outcome indicator would capture the effect of care processes on the health and well being of patients and populations. Outcomes can be expressed as 'the five Ds':

1. Death: a bad outcome if untimely
2. Disease: symptoms, physical signs, and laboratory abnormalities
3. Discomfort: symptoms such as pain, nausea, or dyspnea
4. Disability: impaired ability connected to usual activities at home, work, or in recreation
5. Dissatisfaction: emotional reactions to disease and its care, such as sadness and anger.

Intermediate outcome indicators reflect changes in biological status that affect subsequent health outcomes. Some outcomes can only be assessed after years (e.g. 5-year cancer survival). It is therefore important to assess intermediate outcome indicators. They should be evidence-based and reflect the outcome (e.g. HbA1c in diabetes). They can be regarded as short-term outcomes.

The outcome of care is determined by several factors related to the patient, the illness, and health care. Differences in outcome may be due to case mix and other confounding factors. Standardized data collection and risk adjustment are therefore important for interpreting outcomes data. In general it can be recommended that the broader the perspective required, the greater the relevance of outcome indicators. As the perspective narrows to hospitals and departments or providers, outcome measures become less useful, although still important.

Regardless of whether structural, process or outcome indicators are chosen, feasibility of measurement is always a key consideration. In addition, the frequency with which an event occurs in the population available for study may affect the usefulness of an indicator, unless it is a sentinel event.

Examples of Indicators Related to Outcome

1. Intermediate
   • HbA1c results for diabetics
   • Lipid profile results for patients with hyperlipidemia
   • Blood pressure results for hypertensive patients
2. End Result (should be specified for diseases)
   • Mortality
   • Morbidity
   • Functional status
   • Health status measurement
   • Work status
   • Quality of life
   • Patient satisfaction

Risk Adjustment  

In most cases, multiple factors contribute to a patient's survival and health outcomes. Therefore, outcome measures should be adjusted for factors outside the health system, if fair comparisons are to be made. Factors that are frequently included in risk adjustment models include patient demographic, psychosocial characteristics (such as age, sex, and functional status), lifestyle factors (smoking, alcohol use), and severity of the illness that is the focus for measurement, health status, and co-morbid conditions. Risk adjustment is essential before comparing patient outcomes across hospitals or providers.

Factors determining the outcome of care
The patient
      Demographic factors (age, sex, height)
      Lifestyle factors (smoking, alcohol use, weight, diet, physical exercise)
      Psychosocial factors (social status, education)
      Compliance
The illness
      Severity, prognosis
      Comorbidity
The treatment (prevention, diagnostics, care, rehabilitation, therapy and control)
      Competence
      Technical equipment
      Evidence based clinical practice
      Efficacy, accuracy
The organization
      Use of clinical guidelines
      Cooperation
      Delay

---

=   OUTCOME

Indicators related to Type of Care, Function, and Modality  

Indicators classified by type of care may be preventive, acute, or chronic. Function of care can relate to screening, diagnosis, treatment, and follow-up. The modality by which care can be delivered relates to physical examination of the patient, laboratory or radiology study, or prescription of medication, for example.

References  

1. Lohr KN (ed.) Medicare: A Strategy for Quality Assurance. Vols I and II. Washington, DC: National Academy Press, 1990.
2. Donabedian A. The quality of medical care. Science 1987; 200: 856-864.
3. Worning AM, Mainz J, Klazinga N, Gotrik JK, Johansen KS. Policy on quality development for the medical profession [in Danish]. Ugeskr Laeger 1992; 154: 3523-3533.
4. JCAHO. Characteristics of clinical indicators. Qual Rev Bull 1989; 11: 330-339.
5. Canadian Council on Health Services Accreditation. A guide to the development and use of performance indicators. Ottawa: Canadian Council on Health Services Accreditation, 1996. [Online] http://www.cchsa.ca (Accessed July 31 2003).
6. Mainz J. Developing clinical indicators. Int J Qual Health Care 2003; 15 (suppl. 1):i5 – i11.
7. Sackett DL, Straus SE, Richardson WS et al. Evidence-Based Medicine: How to Practice and Teach EBM, 2nd edition. London: Churchill Livingstone, 2000.
8. Mangione-Smith R, McGlynn EA. Assessing the quality of health care provided to children. Health Serv Res 1998; 33: 1063-1090.
9. Shahian DM, Normand SL. The volume-outcome relationship: from Luft to Leapfrog. Ann Thorac Surg 2003; 75: 1048-1058.
10. Palmer RH, Reilly MC. Individual and institutional variables which may serve as indicators of quality of medical care. Med Care 1979; 17: 693-717.
11. Palmer RH. Using health outcomes to compare plans, networks and providers. Int J Qual Health Care 1998; 10: 477-483.