Home
The University of Adelaide
| The University of Adelaide |
Home |
Departments |
Search
|
 |
||
 |
 |
  |
 |
||||||||||||||||
|
|
Simple calculations for risk analysis for workflow analysisMalcolm Pradhan In order to prioritise interventions for system improvement the risks and benefits of the process should be quantified in some way. The degree of detail in the quantification process depends very much on the purpose: managers may be satisfied with a broad indication of cost and benefit for small changes that require little investment; for expensive or risky interventions managers may require a much more detailed analysis with detailed figures. Our approach is to start simple and to respond with the amount of detail required by decision makers. We have found, however, that the more local data that are used in the quantification process the more compelling the analysis is to local stakeholders. Having an opinion leader supporting the proposed changes is also an important ingredient for success [1] . Accessing local data may be complicated and may require information technology skills to clean and analyse data from hospital information systems. A simple and often effective way of prioritising quality improvement initiatives is to use a Likelihood/Consequences grid as described in the Australian Standards Risk Management framework [2] . Table 1 is an example of the grid. Table 1. Likelihood/Consequence grid for risk management.
In Table 1 the consequences range from Insignificant to Catastrophic that may involve large loss of life. In a similar way costs and risks of interventions may be categorised. If we move beyond this categorisation approach to more detailed risk analysis we find that techniques may require complex mathematics to calculate accurately in the presence of interdependencies between processes, hierarchical models, and when forced to estimate variables from data or from indirect information. In recent years, computer intensive techniques have allowed expert modellers to construct and calculate complex risk models, even in cases where the distributions of information are non-linear and non-normal. However, these specialised techniques are well beyond the scope of this document, and we will concentrate on a simple method for calculating overall risk of failure in a healthcare process. In order to simplify the calculations we will assume that each step in the healthcare process is independent of other steps. So, if an individual patient has a blood test and an x-ray, the chance that the report is late in moving from the biochemistry laboratory to the primary doctor is not in any way influenced by the report from radiology to the primary doctor. While the calculations listed below may seem complicated, you will see from the worked examples that their use is relatively straightforward. Let us consider that an episode of patient care comprises a collection of m processes, where m is some number. The exact value of m can be obtained easily from workflow analysis. Let us define a process failure as an event when a process does not meet predefined quality standards, such as timeliness or accuracy. We will say that an Error occurs if any one of the m processes fails to meet predefined standards. Therefore, the probability of an Error occurring in an episode of patient care is the result of a failure in any of the 1 to m processes that comprise the episode. This can be calculated using equation as follows:
In words, the probability of Error is one minus the probability that no failure will occur in each of the m processes that comprise the episode of care. For example, if every 5 out of every 100 laboratory results were delayed the probability of failure for the lab is
The resulting error rate from this example is 27.8%. If there exist differences in the way patients are managed, then the expected error rates can be calculated for each situation in a similar way. To simplify calculations we have used point or discrete probability estimates that do not take account of variation, which may be non-normal. To achieve error bounds, the above calculation can be repeated for worst-case values and best-case values, with the median values the most likely. But what is the effect of this error rate? Is it acceptable or dangerous? Process control charts, including those used in infection control, use an arbitrary value of 2 standard deviations to determine if something is 'significant'. The use of 2 standard deviations as a cut-off for importance is mathematically convenient but it does not reflect the actual significance of the events in terms of the patient or the organisation, and is therefore inappropriate for risk management. In healthcare evaluation there is often confusion between a measurement, for example the number of infections that occur, and its value , the morbidity and increased length of stay resulting from the infection. Utility theory [3, 4] is a well-developed framework that assists us in moving from measurements, such as error rates, to values, or the impact of the errors. Multi-attribute utility theory is a method that incorporates complex value models in decision-making and is useful in healthcare contexts [5] . A fundamental question before commencing utility elicitation is to ask "whose utilities?" If a patient undergoes a surgical operation and suffers a preventable complication, the cost to the patient is measured in quality of life and perhaps financial loss if they are under-insured; but depending on the funding model, the hospital may not sustain any financial burden of the complication if they are able to obtain increased funding for the diagnostic related group by using complication codes and they avoid litigation (unfortunately funding models of health care are not always aligned to error reduction and patient safety). For illustration purposes we will take the perspective of an organisation and use dollars to signify values, or in this case the loss to the institution due to errors in health care delivery. By combining the error rates derived using equation and the loss to the institution due to errors we can obtain a prioritisation list to determine how important it is for us to improve a specific workflow, and how much we should spend in doing so. It is important to understand that the goal of risk management is to reduce the probability of errors occurring and to maximise the likelihood of a good outcome for the patient, however we cannot guarantee a good outcome; good decision making and healthcare processes can still lead to bad outcomes for the patient. Similarly, bad processes can still result in good outcomes for patients. Let us define a loss for an institution as the cost incurred for a preventable error as a result of a healthcare process. In the majority of cases, errors do not result in significant morbidity to the patient [6] , however in particular patients the same error can result in a fatal injury often because of patient factors such as comorbidities. Let us assume that the patient complexity ( Pt.Complexity ) determines how sensitive a patient is to errors in the healthcare process and is measured by the loss incurred for that subgroup, and that there are n subgroups of patients. The expected loss ( Eloss ) to the institution can be calculated by using the error rate previously calculated (
An example will make this clear. Let us say that in an adult tertiary hospital with an error rate for the timely checking of test results of 0.278 we have 2 subgroups of patients who are going for day surgery. The majority (90%) are uncomplicated patients with little comorbidity in which missing a test result may incur on average an increased length of stay of 0.2 days at $1000 per day, or $200. A second smaller group of patients (10%) have significant comorbidities, and failure to check test results before surgery can incur an increased length of stay of 4 days at $1000 per day, or $4000. The Expected Loss per patient is therefore:
The expected loss per patient is therefore $160. If the hospital sees approximately 10,000 patients per year, the loss will be $1,600,000. This figure is a direct cost ignoring the cost of litigation, patient morbidity, and costs to patients and employers. In a children's hospital where the majority of patients are uncomplicated (say 99%) and increased length of stay would be 0.1 days for this subgroup, the expected loss for 10,000 patients is approximately $400,000. The priority for improving this process would have lower priority in the children's hospital. The assumptions in our very simple example are very significant since we have not modelled explicitly the cost of death: note that death or severe morbidity must be carefully modelled as a special case, usually setting these events to a very high cost, even though the direct costs may be low - if a patient dies their length of stay will be lower than if they were mildly injured, consequently our use of length of stay as a surrogate measure of outcome will imply that death is a better outcome than mild injury. From a purely financial viewpoint this is may be an accurate picture for the institution, although it seems very harsh since the role of hospitals is to help people. To achieve a more balanced view of both the costs and the benefits of a situation multiple viewpoints should be considered where possible. Furthermore, as long as the modeller's assumptions are explicit, even financial models may take a more balanced view by assigning a penalty cost to death, even though it may not be a direct cost of the healthcare process.References
|
|
|||||||||||||
. 
. Similarly, if delays in radiology reports were also 5 in 100 then 
, and if doctors do not follow up 20 in every 100 test results in a timely manner then 
. Using equation we can calculate the error rate resulting from a failure in any of these steps: 
), the loss due to fixing the error for patients in a subgroup of a given complexity ( 
), and the proportion of patients within the subgroup ( 
): 
