A Decision Support System for Pre-Operative Screening

Edmonds M 1 , Crichton T 1 , Pradhan M 2 and Runciman W 3

1 Honours Student, Health Informatics, Faculty of Health Sciences, University of Adelaide , North Terrace, South Australia 5000

2 Health Informatics, Faculty of Health Sciences, University of Adelaide , North Terrace, South Australia 5000

3 Department of Anaesthesia and Intensive Care, University of Adelaide , North Terrace, South Australia 5000

Inadequate pre-operative assessment and planning is a major cause of preventable post-operative morbidity and mortality. Decision models can be used to represent uncertainty within a domain, such as pre-operative assessment and planning, where a decision is to be made. They handle uncertainty using Bayesian Theory, and the effects of the decision are mapped out to a utility measure. We explain the method of developing an evidence-based decision support system for pre-operative screening. An extensive literature review was performed to identify risk factors for specific post-operative adverse events, and examine the evidence to support this association. Using the best available evidence, including expert opinion, decision models have been developed to represent the occurrence of post-operative nausea and vomiting (PONV) and post-operative deep vein thrombosis (DVT). These models can predict each individual patient's risk based on known or assessable pre-operative risk factors, and the decision of administering a prophylactic agent is stratified by this risk to maximise utility. The utility measured is potential cost, although patient comfort can also be incorporated. Decision models have the added advantage of being easily updated and maintained to include the latest evidence and new prophylactic options. The use of an online database allows easy review and addition of evidence, and sensitivity analysis can show the impact new information has on the model results. Decision models are potentially beneficial in pre-operative screening as they collate best evidence to support decision making and improve workflow and communication. Incorporation of these decision models into a pre-operative assessment tool can reduce cost as well as improve patient comfort and quality of care.

1. Introduction

Preoperative screening, assessment and management has been an area of particular focus and interest in recent years, with a number of studies and reviews clearly indicating deficiencies within specific areas. Importantly, inadequate assessment and planning has been indicated as a major cause of preventable peri-operative morbidity and mortality. An analysis of the Australian Incident Monitoring Study (AIMS) data showed that in 1 in 10 adverse events involved inadequate preoperative patient preparation and/or evaluation [ 1 ] , and over 3% of all adverse events were clearly related to a deficiency in this area. Of these adverse events, 57% were judged by peers to be 'definitely preventable' and a further 21% were 'possibly preventable'.

Similarly, an Australian triennial report into anaesthetic related perioperative deaths indicated that inadequate preoperative assessment and management was implicated in 53 of 135 deaths attributable to anaesthesia [ 2 ] . A recent report from the Victorian Consultative Committee on Anaesthesia-Related Mortality found that similar problems with preoperative assessment were implicated in 18 out of 43 deaths in Victoria , Australia [ 3 ] .

From these reports it is evident that inadequate preoperative management is a highly significant problem, and recommendations have suggested that increased emphasis be placed on ensuring that preoperative assessment and management is appropriately performed, in order to minimise the occurrence of these anaesthesia related adverse events. Such events impose otherwise unnecessary costs onto the Australian health care system through unwarranted morbidity and mortality, and also reduce overall patient comfort and quality of care.

Decision support systems have gained increasing exposure over the last decade, with many different systems being developed for a wide variety of purposes [ 4 5 ] . These include decision support to aid in diagnosis, drug prescription and drug dosing. There have also been studies confirming the advantages of such decision support tools in improving the quality of patient care and reducing overall [ 5 ] . Inadequate guidelines/processes and poor communication have been identified as major factors in deficient pre-operative assessment [ 1 ] , and a well-structured decision support system could help to address both of these issues. We have focussed our decision support on risk assessment for individual patients and recommending prophylactic options from those that are currently available and in use. The emphasis has remained on using the best available evidence in order to ensure that the patients are assessed and managed according to best practice, while at the same time increasing quality of care and reducing costs through the judicious use of resources such that utility gained is maximised.

2. Decision Models

Decision models are represented using influence diagrams that define explicitly probabilistic dependencies in a domain of interest. Influence diagrams comprise chance nodes, decision nodes and value nodes. Chance nodes represent uncertainty, and an arc linking the two represents the relationship between different uncertainties. Each uncertainty or chance node has more than one possible state, and its parent nodes determine the probability of each state. This is the conditional probability and it is dependent on the states of the parent nodes. Where a chance node does not have any parent nodes, the probability is based upon population prevalence or prior probability . Probabilities are updated throughout the model using Bayesian theory[ 6 ] . Knowledge of a state will update the probabilities of uncertainties related to that state in both directions - parent nodes and downstream nodes.

Decision nodes represent the decisions under consideration in the influence diagram[ 7 8 ] . To monitor the effects of a decision, a value or utility node is included. Utility is the measure of outcome due to the decision[ 9 ] . The utility can be measured in any unit, such as cost, quality-adjusted-life-years, and quality of care or patient comfort depending on the perspective of the user. The best decision is that which has the maximum expected utility, or in the case of cost, the minimum expected cost.

In the case of pre-operative assessment, the uncertainty lies in the prediction of post-operative complications [figure 1]. The presence of risk factors is uncertain, and initially the probabilities of these are the prior probabilities, or prevalence in the population. As we observe the state of these risk factors, the model will update the predicted risk of the complications. At this stage we have taken the perspective of the health care provider (hospital) and modelled the utility as cost, and this cost will be a function of the pre-operative management decision (i.e. administration of prophylactics), and the post-operative pathway that is followed. Thus the effect of the decision will be shown in the change of utility gained. The best decision will be the one that maximises utility by resulting in the least cost, or in the highest patient comfort, depending on the utility measure. The decision model, then, uses population prevalence and conditional probability derived from evidence to predict the risk of the post-operative complication, and to show which decision will give the maximum utility in terms of cost and/or patient comfort for that risk.

Figure 1 - Simple Decision Model for Post-Operative Complications

3. Literature Review

For both PONV and DVT we performed an extensive literature review[ 10 11 ] using PubMed Medline, the Cochrane library, Internet searches and cross-referencing of bibliographies. We also consulted with experts in the relevant fields for their opinion and advice on reviewing the articles. General searches returned overview and consensus papers from which we identified specific areas to further research. Medline searches were performed using both free text and MeSH headings, as well as looking at 'related articles' from those already identified. We used PubMed Medline (http://www.ncbi.nlm.nih.gov/PubMed) as it is updated in real time. For our initial searches we used the MeSH headings of "Postoperative Complications" and "Embolism and Thrombosis" for DVT, and "Postoperative Nausea and Vomiting" for PONV. Specific topics and risk factors were searched for using MeSH headings, where identified, as well free text searches. All searches were of the entire Medline database, limited to English language and human, adult subjects. The Cochrane library was searched using the terms "thromboembolism" and "postoperative nausea and vomiting". Internet searches were performed using search engines such as Google ( http://www.google.com) , altavista ( http://www.altavista.com) , alltheweb (http://www.alltheweb.com) and Medscape (http://www.medscape.com).

The aim of our literature review was to identify the best available evidence for each topic or risk factor[ 12-14 ] . The choice of inclusion/exclusion criteria was dependent on each individual factor, as the level of evidence was variable for each different field. Preference was given to meta-analyses and systematic reviews that used randomised controlled primary studies on surgical patients. These were easier to identify for PONV, as this is purely a post-operative complication and its occurrence is discrete, whereas DVT can also be a medical or idiopathic condition with variable clinical presentations. Where meta-analyses or systematic reviews could not be identified, preference was given to randomised, controlled studies of surgical patients, and then, in order of decreasing preference, unrandomised/uncontrolled studies of surgical patients, case-control studies of surgical patients, studies of non-surgical patients or population studies. Expert opinion was used in evaluating the studies and for estimation where conclusive data was not available.

Using these criteria, at the time of writing we had retrieved over 60 articles relating to PONV, and nearly 200 relating to DVT. One or two of the authors (ME and/or TC) reviewed each article, and any useable data was included in the model. Useable data was in the form of raw data, odds ratios, relative risks and numbers-needed-to-treat (NNTs).

4. Database

We developed a database using Microsoft Access that incorporated all the details about the article, including citation, study type, sample and power. To integrate the results of different studies we used the database to track detailed information about each study, including inclusion/exclusion criteria, study design, and potential biases. The results of the studies were represented, where possible, as risk ratios, odds ratios, raw numbers, and continuous values where applicable.

5. Modelling

We used limited shareware versions of Hugin v5.2 Lite (http://www.hugin.dk) and Netica v1.12 (http://www.norsys.com) as well as GeNIe (a Graphical Network Interface) v1.0 to construct the influence diagrams and decision models. We used the tutorials available with each program as well as medical decision making texts to familiarise ourselves with the programs, and the concepts of influence diagram and decision model construction and use.

5.1 Qualitative Model

From our initial literature review we identified all factors suggested to contribute to the aetiology of PONV and DVT. From this we developed a qualitative model that illustrated the different relations, but did not include the statistical evidence. We also developed qualitative models that outlined the mechanisms of PONV and DVT using known physiological processes, such as the vomiting centre for PONV, and Virchow's triad for DVT. Unfortunately, many risk factors act through unknown mechanisms and could not be mapped.

Original qualitative models included all suggested factors, and further literature review was performed for each of these factors individually. Risk factors for PONV were divided into Patient, Operation, Anaesthetic and Post-operative factors. We did not divide risk factors for DVT into distinct categories.

5.2 Quantitative model

From our further literature review and incorporation of expert opinion we entered study data into the model. Those factors that did not have sufficient data, or had inconclusive data were excluded from the model, as were factors deemed not relevant to the population we are considering.

The probability for each node was defined. In the case of the risk factors, which have no parent nodes, this was the prior probability or population prevalence of the factor. The model, as such, does not include correlations between the presence of the risk factors. Correlation between factors could not be obtained from reported data, but we hope to analyse this using raw study data from a similar patient population. From the evidence base we then defined the conditional probability of the complication. This involved determining the effect of each risk factor in terms of the probability of the complication given each factor. Data reported in studies assumes that each factor is independent, and this leads to over-confidence in the model, which may be overcome by identifying correlations.

Data was weighted by study design, sample size and the presence of bias or confounding factors. A "Noisy OR" function was used in the GeNIe program to combine the individual influences of each risk factor.

Prophylactic agent cost and efficacies were entered into the model, as were costs of potential post-operative pathways, including investigation, treatment and readmission or prolonged stay in hospital. Cost utility is a function of these given that the prophylactic options have efficacies considerably below 100%. Neither model, as such, includes patient comfort as a utility, although this is planned and is an important consideration in pre-operative assessment and management.

6. Sensitivity analysis

We performed a one-way sensitivity analysis over the PONV model, testing the effect of each of the risk factors on the results, as well as the effect of different prophylactic agent costs, and variations in model design. The results of this show which factors are the most important in predicting the complication, and from this the value of information in knowing the state of this factor can be determined. Similar analysis of prophylactic efficacies or investigation sensitivity and specificity will show the effect that variability in these across different locations will have on the model output.

As new information is introduced to the model, similar sensitivity analysis can be performed to show the impact on the resulting outcomes of the model. This will show the importance of including new evidence, and the potential changes in practice that new information can bring about as the model is maintained.

7. Examples

7.1 Deep Vein Thrombosis (DVT)

Our DVT model [figure 2] includes the risk factors of age, obesity, varicose veins, a previous history of venous thromboembolism (VTE), use of the oral contraceptive pill, presence of Factor V Leiden, presence of a malignancy with or without chemotherapy, use of general anaesthesia over spinal anaesthesia and risk from the operation. Orthopaedic surgery is classed as high risk, most other surgeries including general and gynaecological are medium risk, and minor peripheral surgery is low risk. Risk factors initially suggested that were excluded include pregnancy, hormone replacement therapy, paralysis, heart disease, nephrotic syndrome and the rarer thrombophilias. These were excluded due to lack of relevance to the surgical population or insufficient/conflicting evidence in surgical patients and to simplify the model. The model only includes the most important factors so that it can feasibly be used in a clinical situation, but is still representative of the domain.

The model also maps out the potential post-operative clinical pathway starting with the chance of there being clinical signs or symptoms, objective testing and management. It also takes into account side effects, such as haemorrhage, of the prophylactic agents and treatment, and the possible progression of DVT onto pulmonary embolism (PE), the major clinical complication. The potential clinical pathway for PE is also mapped out, including testing, management and outcomes.

The costs of DVT arise from the prophylaxis agent itself, as well as the costs of testing, management and longer stay in hospital if a DVT does develop. Patient comfort can also be considered in the overall utility, taking into account invasive investigations, discomfort, prolonged morbidity and potential mortality. In this way, patient preference can be included into the decision, rather than a purely financial approach.

Figure 2 - DVT model

7.2 Post-operative nausea and vomiting (PONV)

Our final model for PONV [figure 3] includes patient risk factors of age, obesity, gender, smoking status, past history of PONV, anaesthetic risk factors of volatile agent or propofol, use of neostigmine, use of N 2 O and use of opioids, as well as operative risk. The anti-emetic agents considered in the model are Metoclopramide, Droperidol and a serotonin (5HT 3 ) antagonist, Tropisetron, as these are representative of the different classes used in our prototype hospital (RAH).

Figure 3 - PONV model

Using the set prior probabilities the model predicted a population risk in accordance with current literature and expert opinion. Individual patient risk ranges from 1.4% with no risk factors to 85.4% with all risk factors. The model also shows that anti-emetic choice is stratified by patient risk [figure 4], with more expensive, more effective agents becoming feasible at higher risk. The model predicts where on this graph each patient's risk lies, and therefore which prophylaxis will be the most cost-effective. Importantly results also show that the option of no prophylaxis is dominated throughout, suggesting that every patient should get some form of prophylaxis. It must be noted however, that this is a purely cost-driven approach, which has not yet incorporated side-effects of prophylactics or patient preferences.

Figure 4. Risk Stratification

One-way sensitivity analysis shows [figure 5] that of the individual factors, the most important is a past history of PONV, followed by female gender. Analysis of the risk factor categories shows that fixed patient characteristics are the most important in predicting the incidence of PONV, whereas anaesthetic agent choice has the least influence [figure 6]. These sensitivity analyses allow us to determine the value of information of knowing the state of each factor so we can ensure that the most important factors are covered in the pre-operative assessment.

8. Discussion

Pre-operative assessment and management represents an ideal opportunity to utilise the advantages that decision support systems can provide within a clinical situation. The current deficiencies in this area and the prevalence of adverse events, in particular those which are preventable given appropriate and adequate assessment and management, lend themselves particularly well to decision support.

There is currently an abundance of evidence available to anaesthetists and those involved in the area of assessment and management of patients pre-operatively, however it is often too time consuming and difficult to collate all of this information. Appropriately designed decision support tools can help in this situation by incorporating all of this evidence and making it explicit, so that those performing the assessment can critically examine the evidence behind the decision support tool, whilst still obtaining a global perspective from the complete tool.

There are a number of advantages associated with decision support tools such as those illustrated. Foremost is their inherent and explicit use of best available evidence. As mentioned, this consists of the most relevant literature sources, as well as input from experts within associated fields. This means that this decision support has a strong evidence base, and has been endorsed by those most knowledgeable in the particular area. This allows the patient to be assured by the fact that they are being managed according to the best current practice, and those in the relevant specialties are more likely to use this decision support technology if they have had input. Other advantages of decision support tools such as these include their ability to maximise utility in a specific given situation. This may involve minimising costs or increasing quality of care, however the end result is better management of available resources with management plans targeting those specific sub-populations of patients who will benefit most. This ability to propose individualised management plans for every patient based on their personal characteristics as well as their surgery is also a strong advantage of such tools. By incorporating the use of decision support tools into routine pre-operative assessment, patients are ensured a consistently adequate assessment, and as this has been a notable cause of post-operative adverse events, these can subsequently be reduced significantly. These decision models can also be localised to different hospitals and locations by using local risk factor and surgery prevalence and resource availability and costs.

These decision support tools in their current state are not complete, and may not cover every possible eventuality. They do not currently include specific contraindications for different prophylactic choices, or every prophylactic option. These models are for decision support, and still require expert interpretation and clinical experience to maximise the utility for each patient. As more work is done, and more evidence included, the more accurate the models will become, but the clinical situation is such that its complexity will require an expert's clinical judgement to interpret the recommendations.

The results from the PONV and DVT decision models have shown a number of interesting points. Of prime importance is the finding that choice of prophylaxis for PONV is stratified based on individual patient risk, and that every patient should receive some form of prophylaxis. Both models predict their respective post-operative complications at an acceptable level. This further supports the need for adequate patient assessment in order to minimise the costs associated with inappropriate prophylaxis and subsequent adverse events, and to reduce the absolute rate of post-operative complications to an acceptable level.

The aim of decision support is to determine the best decision in any situation given a degree of uncertainty which cannot otherwise be avoided. Decision support systems, such as those we have discussed, handle this uncertainty, and frame the recommendations based around a measure of utility. The best decision is that which maximises the utility gained. At the moment these models are aimed at minimising the costs associated with specific adverse events and their complications by appropriately prescribing prophylactics. Another important aspect of decision making is to incorporate patient perspective into the decision, including patient comfort and patient preference. It is important that decisions affecting patient wellbeing are not made entirely on a monetary basis, and it is for this reason that we plan to incorporate patient discomfort associated with extended hospitalisation, invasive procedures, adverse events or side effects of medication into the model. Obviously due to their subjective nature this is much more difficult to quantify than the costs associated with prophylaxis, however it is important to incorporate this into the final decision. It is important to directly ask the patient about these points in the assessment and management and use personal judgement to integrate this into the final decision regarding the management plan. The judgement call over where maximum utility gained lies within the balance of patient comfort and cost is one that will need to be assessed.

The objective is to include these decision models into an explicit pre-operative planning and assessment procedure. A computer-based tool 15 that incorporates these models would improve adherence to the pre-operative assessment process, and facilitate communication of results. A dedicated nurse or even rural GPs could perform a guided pre-operative questionnaire. This would elicit the information required for the decision models, as well as ensure that a rigorous and comprehensive assessment is performed. The model could then suggest which test and investigations, if any, are required, and the results of these could be relayed back into the system. This would avoid unnecessary investigations and cost. The results of this initial screening could then be assessed and interpreted by an anaesthetist, who could order further tests or examinations, and enter these into the system. The model would then create individual protocols based on that patient's risk. The advantages and disadvantages of each decision option could be viewed, and the final choice of which decision to take would lie with the anaesthetist, surgeon and patient. The anaesthetist could also query the system for the evidence supporting the decision, which could be retrieved from the online database. The system would suggest the best decision based on the utility measure(s), but the judgement call on how the utilities balance would lie with those involved.

Decision models, as demonstrated, are of huge potential benefit in pre-operative assessment and planning, not only in reducing cost, but also increasing patient comfort and quality of care by decreasing the incidence of adverse events. This same process can be used to model any area where there is uncertainty and decisions need to be made.

References:

1. Kluger MT, Tham EJ, Coleman NA, Runciman WB, Bullock MF. Inadequate preoperative evaluation and preparation; A review of 197 reports from the Australian Incident Monitoring Study. Submitted to Anaesthesia, 2000 .

2. Davis NJ. Anaesthesia related mortality in Australia 1994-1996. Report of the Committee convened under the auspices of the Australian and New Zealand College of Anaesthetists, 1999.

3. Report of the Victorian Consultative Committee on Anaesthesia-related Morbidity and Mortality, 2000.

4. Hotlist. Decision support. Health Manag Technol 2000;21(2):52-7.

5. Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Jr., et al. A computer-assisted management program for antibiotics and other antiinfective agents [see comments]. N Engl J Med 1998;338(4):232-8.

6. Andrade PJ. Specialized computer support systems for medical diagnosis. Relationship with the Bayes' theorem and with logical diagnostic thinking. Arq Bras Cardiol 1999;73(6):545-552.

7. Nease R, Owens D. Use of Influence Diagrams to Structure Medical Decisions. Medical Decision Making 1997;17(3):263-275.

8. Owens D, Shachter R, Nease R. Representation and Analysis of Medical Decision Problems with Influence Diagrams. Medical Decision Making 1997;17(3):241-262.

9. Sox H, Blatt M, Hiigins M, Marton K. Expected Value Decision Making. Medical Decision Making . Stoneham: Butterworth Publishers, 1988:147-166.

10. Chalmers I, Altman D. Systematic Reviews . London: BMJ Publishing Group, 1995.

11. Sutton A, Abrams K, Jones D, Sheldon T, Song F. Systematic reviews of trials and other studies: Health Technology Assessment, 1999.

12. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995;16(1):62-73.

13. Moher D, Cook D, Jada A, Tugwell P, Moher M, Jones A, et al. Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses: Health Technology Assessment, 1999.

14. Wortman P. Judging Research Quality. In: Cooper H, Hedges L, editors. The handbook of research synthesis . New York: Russell Sage Foundation, 1994:97-109.

15. Gawande A, Bates D. The use of information technology in improving medical performance Part II. Physician-Support tools: Medscape, 2000.

Acknowledgments:

Dr Guy Ludbrook, Dr George Osborne and Dr Andrew Bowie from the Department of Anaesthesia, Royal Adelaide Hospital

Dr John Lloyd, Director Department of Haematology, Royal Adelaide Hospital .

Home > Research > Pre-op

© 2004 The University of Adelaide Last Modified Wednesday, 28-Jun-2006 11:32:05 CST  Health Informatics	Copyright | Privacy | Disclaimer