A Comparison Of A Paper-Based Clinical Guideline And A Decision Support System For The Management of Community Acquired Pneumonia

Clark , S. 1 , Pradhan, M. 1 , Faunt, J. 2 and Swart, R. 2

1. Health Informatics, Division of Medicine, University of Adelaide , North Terrace, South Australia 5000.
2. Clinical Practice Unit, Royal Adelaide Hospital North Terrace, South Australia 5000.

Abstract

To compare the construction of a paper-based guideline to that of an evidence-based decision model we followed the development of a paper-based guideline at a major teaching hospital. Qualitative and quantitative models for decision analysis were developed from this guideline. Decision analysis models provide a dynamic and flexible alternative to paper -based guidelines. They enable evidence-based medicine to be applied in most instances, limited by only by the information available, are simple to update in comparison to paper-based guidelines and suffer less bias. Through review of the literature we identified that reporting of data in community acquired pneumonia (CAP) studies is inadequate for evidence-based medicine including decision analysis studies. More complete data via access to original data sets or by minimum standards of data reporting is required. Paper-based guidelines for the treatment of CAP have been shown to cut length of stay and cost, however they have no effect on mortality. An evidence-based decision support system, modeled with dependence of predictive factors in mind may provide the tool to reduce this mortality, along with cost, while considering patient utility. This remains to be assessed

Introduction

Practice guidelines are systematically developed statements designed to assist in clinical decision making under specific circumstances [1] . These documents have proliferated throughout clinical medicine, driven by the need to improve health outcomes, while controlling resource allocation and cost. They reflect the growing focus on evidence-based medicine, managed care, and concern over medico-legal issues.

Community-acquired pneumonia (CAP) adds a significant contribution to morbidity, mortality and health cost [ 2 3] . During the 1990's more than 30 guideline articles dealing with the treatment of CAP were published [4] , culminating in consensus statements from major organisations in pulmonary and infectious medicine such as the ATS [5] , the BTS [6] and the ISDA [2] . These documents represent attempts to reduce the high cost, both in dollars and morbidity, by reducing the high variability in clinical practice and introducing best evidence.

1.1 Guideline features

Walker [7] suggests that guidelines should be widely disseminated, based on methodology open to public scrutiny, revised regularly in response to new research and derived from research of the highest quality. They should be both comprehensive and specific - clearly describing all indications for a treatment or procedure. They should be manageable in understanding and practice, flexible, reliable and lead to improved outcomes.

1.2 Problems with guidelines

Guidelines generally lack the flexibility to deal with uncertain values in medicine. Taking a traditional statistical approach they divide patients into discrete categories, which may not truly represent the likelihood of certain outcomes. They contain hidden utility judgements of the value of medical outcomes. Once fixed in a guideline these judgements do not allow for situational variables such as patient preference [8] . It is a difficult task for busy clinicians involved in guideline construction to stay up to date with the most current information. Exposure to an incomplete picture of current data results in a bias of information. Integration of information from all studies is also difficult and time consuming. The evaluation of evidence is dependent on skills in evidence-based medicine, which vary amongst healthcare workers. Consequently, expression of biased opinions about papers, and partial recall of papers is common.

Guidelines not constructed from an evidence base suffer from selection, anchoring, and representativeness biases. These biases effect ideas of prior probability and the value of information. Prior probability assumptions implicit in guidelines limit their generalisability from global to local scales and between geographical locations [9]

The adoption of guidelines by practitioners is highly variable and dependent on characteristics of the physician, the patient and the medical system [10] Once in place guidelines require constant maintenance to keep pace with clinical science, drug development and demographic variation of illness [9] . This is especially important in the case of CAP due to changing antibiotic resistance profiles across the world [2] . The cost in lag time and dollars of assembling review committees and dispersing updates for paper-based guidelines is large when one considers the total amount of time spent by all staff.

1.3 Guidelines in CAP

CAP guidelines address a number of issues specific to this disease. They identify standards for diagnosis and investigation of microbiological aetiology. Many CAP guidelines also divide patients on the basis of risk. A subset of CAP patients with reduced risk of mortality has been identified [11] [12] . Research indicates that these individuals may be treated as outpatients, thus reducing the cost of treatment and allowing better utilization of resources. For those requiring hospitalisation average length of stay has been found to be highly variable [13] and in many cases stay extends beyond the need for hospital-based treatment [14] . Guidelines have been successfully implemented to shorten the average length of stay [ 15 16] . Mortality has not been reduced by guideline introduction [ 16 15] . Morbidity reduction as indicated by increased symptom absence at 6 weeks post therapy, has been associated with guideline implementation [17] . Cost savings form the use of guidelines are dependent on mortality risk. Treatment matching ATS guidelines resulted in 3 fold lower costs for treatment of low risk patients, but 10 fold higher cost for treatment of high risk patients [18] . Use of guidelines has been found to relate to risk factors such as aspiration [19] . Antibiotic treatment for CAP is usually empiric, as aetiological diagnosis is unreliable and delay in treatment is associated with poor outcomes. Consequently, CAP guidelines generally identify preferred antibiotics based on their spectrum of action against locally prevalent microbes, modified by patient risk factors and the severity of disease [ 2] .

1.4 Guideline development

Woolf [20] identifies 4 methods of guideline development: informal consensus, formal consensus, evidence-based, and explicit. Traditionally guidelines have been produced via informal consensus relying solely on expert opinion. This approach lacks clearly defined decision-making criteria. The formal consensus approach includes a more structured evaluation of expert evidence, such as weightings of appropriateness, however, such weightings are difficult to apply in clinical practice. All opinion-based methods suffer from biases. Evidence-based guidelines avoid bias by the use current scientific evidence from systematic review of the available literature. This evidence is evaluated on the basis of study design and sample population characteristics. The fourth method, explicit guideline development, involves defining the probability and utility of treatment outcomes and is also evidence based. This approach allows for the uncertainty in medical decision making and forms the basis of decision analysis modeling used in evidence-based decision systems [9] .

1.5 Advantages of evidence-based decision systems

In comparison to guidelines an evidence-based decision model may be constructed to provide a flexible, locally relevant, utility-based and patient specific, support for complex clinical decision-making. It may be used to provide ratings of the value of new information to assist in test ordering. Decision analysis allows simple access to probabilities for updating of the model to suit site and time dependant changes. The effectiveness of these changes (the "expected value of customization") can also be calculated [ 9] . The evidence base may be made accessible to users to encourage adoption, understanding and to provide education.

1.6 CAP guideline and process in a major teaching hospital

In order to compare the process and outcomes of paper-based guideline construction with that of an evidence-based decision model, we followed the development of a paper-based guideline for the treatment of community-acquired pneumonia at a large teaching hospital. An evidence-based decision model was derived from this guideline via a systematic review and construction of an influence diagram. This model considered the inter-relationship of CAP risk factors and findings, in order to reduce overconfidence due to assumptions of independence.

2. Methods

2.1 Systematic Review

We identified over 300 articles via Pubmed search. Other articles were identified in reference lists from these articles. Web-based references were also located using the Google search engine [21] , and the Medscape resource network [22] . Articles containing data relating to the prevalence of pneumonia risk factors, pneumonia findings, pneumonia complications, aetiology of pneumonia, decision to admit, microbiological testing, treatment of pneumonia, treatment of complications, length of stay, and costs were identified and included in the study. For the purpose of this study we restricted risk factor and finding variables to those identified by Fine et al. [11]

2.2 Qualitative modeling

An initial model was constructed using variables identified in the teaching hospital guideline. Basic medical texts [23] were used to identify mechanisms that grouped risk factors and findings. The qualitative model represents the important dependencies and independencies between the variables. Without this important 'structural' information we would need a vast number of probabilities to quantify the model.

2.3 Quantitative modeling

Influence diagrams, consist of chance (oval), decision (rectangle) and value (diamond) nodes. These are related via bayesian calculation of overall network probability of value node outcomes (expected value)[ 9] . The CAP influence diagram (see figure 1) includes the salient risk factors and findings identified by Fine [11] . These are also used for prediction in the teaching hospital guideline. Pneumonia severity and probability of complications then predicts length of stay, modified by the microbial aetiology, the decision to admit and the type of treatment. Admission, length of stay and treatment all influence outcome which may be considered as a financial or health value. Probability values for chance nodes were derived from the literature review. When data could not be located probability was determined by expert opinion. In order to simplify the initial model the influence of microbiological testing on cost and treatment revision was ignored.

Figure 1. Influence diagram of CAP

3. Results / Discussion

3.1 Data collection

Our CAP literature review indicated that data is generally not reported in a form useful for decision analysis or evidence-based medicine in general. This highlights a need for access to clinical datasets, or for minimum data reporting standards.

3.1.1 Risk factors

As there are few prospective population studies of CAP risk factors, data is only available as the probability of risk factor given CAP [ 11 24 25] . To model CAP correctly we then had to identify population values for each risk factor and calculate the probability of CAP given risk factors using Bayes' theorem.

Information regarding local population prevalence for risk factors was also difficult to find. An attempt was made to use Australian Case-mix data [26] , however, this data grossly underestimates the prevalence of risk factors as a large number of patients with these diseases are admitted with other principle diagnoses. This led to the use of U.S. population figures gathered from the NHANES study [27] for the prevalence of all risk factors except alcohol abuse [28] , age over 65 and nursing home residence [29] . To improve modeling of dependence between risk factors we intend to further investigate correlations using the NHANES dataset [27] .

3.1.2 Findings

Data for findings is in the form of probability of finding given CAP and this is consistent with our modeling [ 11 24 25] . Again, we require more information on correlation of certain findings, to avert overconfidence by assuming independence. There is some data available concerning the association of findings with specific aetiologies [30] , and length of stay. The concept of "time to resolution of symptoms" has been proposed as a valid indication of readiness for discharge and therefore links findings to an optimal length of stay [31] . We should be able to identify data that links findings to probability of complication.

3.1.3 Problems with pneumonia severity measures

A number of severity classifications have been formulated. These classifications have different weighting of factors. They also identify different risk factors and severity findings, based on statistical techniques such as regression analysis [32] . While these measures have been associated with mortality risk they hide valuable information relating specific risk factors and findings to outcomes. Comorbidities as risk factors, interact with pneumonia via different pathophysiological mechanisms, which are not accounted for by an overall risk score. Similarly critical findings indicate specific pathological mechanisms. Much of the recently published data relates outcomes to Fine et al.'s [11] mortality risk classes. This focus restricts the data's usefulness for decision modeling.

3.1.4 Aeitology

Most papers have some data on microbial aetiology of CAP and some focus on this specifically [33-35] . Some data exists concerning the probability of complication given aetiology [ 36 37] , aetiology given risk factors such as age and comorbidity [ 35 38] and infection with multiple pathogens [39] . These influences remain to be modeled. Aetiology figures that are both recent and local, are required to accurately predict local CAP risk. We hope to acquire up to date local data sets in the future. This information should incorporate antibiotic resistance data to provide a true indication of risk of complication.

3.1.5 Microbiological testing

There are a number of papers with sensitivity and specificity values [ 2] . Once again local information is required to make the model site specific [34]

3.1.6 Treatment

Clinical trials in the treatment of pneumonia have mostly been designed to test the efficacy of new antibiotics to "standard therapy" which varies greatly from paper to paper [40] . Treatment and outcome papers list drug usage but do not relate it back to patient, disease or organism characteristics [ 18 41] . For most of the combinations of antibiotics used in the teaching hospital guideline there is some clinical efficacy and length of treatment data (Amoxycillin [42] , Clarithromycin [43] , Erythromycin and Ceftriaxone [44] , Penicillin and Gentamycin [45] ) . However, this is not the case for regimes of Clarithromycin and Ceftriaxone or Clarithromycin and Augmentin [40] .

Treatment of pneumonia varies across the time course of the disease. Initially antibiotic therapy is intravenous. Switch to oral therapy once the patient is stable reduces the risks and costs of intravenous therapy and enables an earlier discharge [46] . Treatment is also modified by identification of microbiological aetiology or by the onset of a complication [ 2] . Delay to initial antibiotic treatment is associated with poor outcomes [47] . These factors, along with the use of other respiratory medications remain to be added to our model.

3.1.7 length of stay

Unfortunately much of data on these values relates back to measures of CAP severity that give no indication of individual patient, disease organism characteristics. However "high risk CAP" seems to result in a longer stay at hospital [14] . There is evidence that suggests that symptom resolution occurs in 3 to 5 days [31] , and that patients can be safely transferred to oral medication and discharged around this time [ 15 46] . Most papers report lengths of stay greater much greater than 3-5 days. There is high variation between hospitals [13] . Figures for length of stay relating to individual complications still need to be identified.

3.1.8 Complications

Fine et al. [37] in their meta-analysis of CAP studies provide figures for mortality given complication.

This group also indicates complications by treatment site by mortality from PORT study data [25] .

3.1.9 Cost

Data concerning drug and procedure costs was available from the online edition of the Australian Schedule of Pharmaceutical Benefits [48] . Cost of hospitalisation per day by care type is best represented via local figures and was estimated by expert opinion. Individual costs for treatment of complications remain to be identified.

3.2 Discussion

The intention of a guideline is to reduce the variation of medical practice and of patient outcomes for patients with similar characteristics. However, because of the limitations of the manual guideline process, the effect of guidelines is to reduce the range of medical practice for all patients because fewer patient-specific factors are considered during management selection.

The teaching hospital guideline was derived from a CAP treatment guideline used by Intermountain Health. This guideline uses predictor variables identified by Fine et al. [11] . The guideline committee used a relatively informal approach; only one paper containing evidence was used in its construction [25] . The guideline divides patients into 4 groups based on level of risk (high/low) and severity (severe/nonsevere). In contrast to the PORT studies patients were classed as high risk if they had one risk factor, and severe if they had one severity finding. Non-severe patients with low risk were to be treated as outpatients. Antibiotic treatment varied in 4 categories with increasing risk and severity.

The manual guideline development process is limited in the number of factors that committee members can consider and integrate statistically. In contrast, decision model construction allows us to systematically deconstruct the decision problem into smaller sub problems for which data can be collected and integrated using statistically rigorous techniques. The challenge to the model builder is in communicating the results of the decision model to the clinical experts involved, and to make sure the model complies with the local constraints of the organization, such as overall cost and resource availability. The decision model may be used as an educational resource for those outside the core committee, and may be used to resolve conflicts of opinion during the development process.

There is little data on the cost of guideline maintenance over a long period of time, when treatment and test choices will change in availability and in cost. We believe that the additional effort of decision model construction results in a system that is much easier to maintain both over time, and when guidelines are applied in new locations with different patient prevalence. Decision models provide a dynamic and flexible alternative to paper-based guidelines. They enable evidence-based medicine to be applied in most instances, limited by only by the information available, are simple to update in comparison to paper-based guidelines and suffer less bias.

Reporting of data in CAP studies is inadequate for evidence-based medicine including decision analysis studies. More complete data via access to original data sets or by minimum standards of data reporting is required.

Paper-based guidelines for the treatment of CAP have been shown to cut length of stay and cost, however they have no effect on mortality. An evidence-based decision support system, modeled with dependence of predictive factors in mind may provide the tool to reduce this mortality, along with cost, while considering patient utility. This remains to be assessed.

4. References

1. Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR. The relation between systematic reviews and practice guidelines. Annals of Internal Medicine 1997;127:210-216.

2. Bartlett JG, Breiman RF, Mandell LA, File TM, Jr. Community-acquired pneumonia in adults: guidelines for management. The Infectious Diseases Society of America . Clin Infect Dis 1998;26(4):811-38.

3. Guest JF, Morris A. Community-acquired pneumonia: the annual cost to the National Health Service in the UK . Eur Respir J 1997;10(7):1530-4.

4. Hackner D, Tu G, Weingarten S, Mohsenifar Z. Guidelines in pulmonary medicine: a 25-year profile. Chest 1999;116(4):1046-62.

5. Niederman MS, Bass JB, Jr., Campbell GD, Fein AM, Grossman RF, Mandell LA, et al. Guidelines for the initial management of adults with community-acquired pneumonia: diagnosis, assessment of severity, and initial antimicrobial therapy. American Thoracic Society. Medical Section of the American Lung Association. Am Rev Respir Dis 1993;148(5):1418-26.

6. BTS. Guidelines for the management of community-acquired pneumonia in adults admitted to hospital. The British Thoracic Society. Br J Hosp Med 1993;49(5):346-50.

7. Walker RD , Howard MO, M.D. L, Suchinsky R. Medical Practice Guidelines. Western Journal of Medicine 1994;161:39-44.

8. Protheroe J, Fahey T, Montgomery AA, Peters TJ. The impact of patients' preferences on the treatment of atrial fibrillation: observational study of patient based decision analysis [see comments]. Bmj 2000;320(7246):1380-4.

9. Owens DK, Nease RF, Jr. A normative analytic framework for development of practice guidelines for specific clinical populations. Med Decis Making 1997;17(4):409-26.

10. Halm EA, Atlas SJ, Borowsky LH, Benzer TI, Metlay JP, Chang YC, et al. Understanding physician adherence with a pneumonia practice guideline: effects of patient, system, and physician factors. Arch Intern Med 2000;160(1):98-104.

11. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, Singer DE, et al. A prediction rule to identify low-risk patients with community-acquired pneumonia [see comments]. N Engl J Med 1997;336(4):243-50.

12. Hoe LK, Keang LT. Hospitalized low-risk community-acquired pneumonia: outcome and potential for cost-savings. Respirology 1999;4(3):307-9

13. McCormick D, Fine MJ, Coley CM, Marrie TJ, Lave JR, Obrosky DS, et al. Variation in length of hospital stay in patients with community-acquired pneumonia: are shorter stays associated with worse medical outcomes? Am J Med 1999;107(1):5-12.

14. Fine MJ, Medsger AR , Stone RA, Marrie TJ, Coley CM, Singer DE, et al. The hospital discharge decision for patients with community-acquired pneumonia. Results from the Pneumonia Patient Outcomes Research Team cohort study [see comments]. Arch Intern Med 1997;157(1):47-56.

15. Weingarten SR, Riedinger MS, Hobson P, Noah MS, Johnson B, Giugliano G, et al. Evaluation of a pneumonia practice guideline in an interventional trial. Am J Respir Crit Care Med 1996;153(3):1110-5.

16. Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. CAPITAL Study Investigators. Community-Acquired Pneumonia Intervention Trial Assessing Levofloxacin Jama 2000;283(6):749-55.

17. Siegel RE. Community-Acquired Pneumonia in the 21st Century: Medscape.com, 2000.

18. Gleason PP, Kapoor WN, Stone RA, Lave JR, Obrosky DS, Schulz R, et al. Medical outcomes and antimicrobial costs with the use of the American Thoracic Society guidelines for outpatients with community-acquired pneumonia. Jama 1997;278(1):32-9.

19. Marras TK, Chan CK. Use of guidelines in treating community-acquired pneumonia. Chest 1998;113(6):1689-94.

20. Woolf SH. Practice guidelines, a new reality in medicine. II. Methods of developing guidelines

Arch Intern Med 1992;152(5):946-52.

21. google.com. , 2000.

22. Medscape.com. , 2000.

23. Govan A, Macfarlane PS, Callander R. Pathology Illustrated . 4th ed. London : Churchill Livingston, 1995.

24. Fine MJ, Singer DE, Hanusa BH, Lave JR, Kapoor WN. Validation of a pneumonia prognostic index using the MedisGroups Comparative Hospital Database. Am J Med 1993;94(2):153-9.

25. Fine MJ, Stone RA, Singer DE, Coley CM, Marrie TJ, Lave JR, et al. Processes and outcomes of care for patients with community-acquired pneumonia: results from the Pneumonia Patient Outcomes Research Team (PORT) cohort study. Arch Intern Med 1999;159(9):970-80.

26. CDHAC. Australian Casemix: Commonweatlth Department of Health and Aged Care, 2000.

27. CDC. National Health and Nutrition Examination Survey: Centers for disease control and prevention ( USA ), 2000.

28. NIAAA. Prevalence and population estimates of DSM-IV alcohol abuse and dependence by age, sex, and ethnicity, United States , 1992: National Institiute of Alcohol Abuse and Alcoholism, 1995.

29. AIHW. Residential Aged Care Facilities in Australia 1998: A Statistical Overview: Australian Institute of Health and Welfare, 2000.

30. Marrie TJ. Community-acquired pneumonia. Clin Infect Dis 1994;18(4):501-13; quiz 514-5.

31. Halm EA, Fine MJ, Marrie TJ, Coley CM, Kapoor WN, Obrosky DS, et al. Time to clinical stability in patients hospitalized with community-acquired pneumonia: implications for practice guidelines [see comments]. Jama 1998;279(18):1452-7.

32. Auble TE, Yealy DM, Fine MJ. Assessing prognosis and selecting an initial site of care for adults with community-acquired pneumonia. Infect Dis Clin North Am 1998;12(3):741-59.

33. Guglielmo L, Leone R. Aetiology and therapy of community-acquired pneumonia: a hospital study in northern Italy . Veneto Pneumonia Research Group. Eur J Clin Pharmacol 1997;51(6):437- 43.

34. Lim I, Shaw DR , Stanley DP, Lumb R, McLennan G. A prospective hospital study of the aetiology of community-acquired pneumonia. Med J Aust 1989;151(2):87-91.

35. Marrie TJ. Community-acquired pneumonia: epidemiology, etiology, treatment. Infect Dis Clin North Am 1998;12(3):723-40.

36. Campbell GD. Overview of community-acquired pneumonia. Prognosis and clinical features. Med Clin North Am 1994;78(5):1035-48.

37. Fine MJ, Smith MA, Carson CA , Mutha SS, Sankey SS, Weissfeld LA, et al. Prognosis and outcomes of patients with community acquired pneumonia. A meta-analysis [see comments] Jama 1996;275(2):134-41. Fein AM. Pneumonia in the elderly. Special diagnostic and therapeutic considerations. Med Clin North Am 1994;78(5):1015-33.

39. Lieberman D, Schlaeffer F, Boldur I, Horowitz S, Friedman MG, Leiononen M, et al. Multiple pathogens in adult patients admitted with community-acquired pneumonia: a one year prospective study of 346 consecutive patients. Thorax 1996;51(2):179-84.

40. Read RC. Evidence-based medicine: empiric antibiotic therapy in community-acquired pneumonia. J Infect 1999;39(3):171-8.

41. Gilbert K, Gleason PP, Singer DE, Marrie TJ, Coley CM, Obrosky DS, et al. Variations in antimicrobial use and cost in more than 2,000 patients with community-acquired pneumonia Am J Med 1998;104(1):17-27.

42. O'Doherty B, Dutchman DA, Pettit R, Maroli A. Randomized, double-blind, comparative study of grepafloxacin and amoxycillin in the treatment of patients with community-acquired pneumonia. J Antimicrob Chemother 1997;40 Suppl A:73-81.

43. Fogarty C, Grossman C, Williams J, Haverstock D, Church D. Efficacy and Safety of Moxifloxacin vs Clarithromycin for Community-Acquired Pneumonia. 16(11) ed: Infections in Medicine, 1999.

44. File TM. In vitro susceptibility and in vivo efficacy of Macrolides: A clinical contradiction: Medscape, 2000.

45. Speich R, Imhof E, Vogt M, Grossenbacher M, Zimmerli W. Efficacy, safety, and tolerance of piperacillin/tazobactam compared to co-amoxiclav plus an aminoglycoside in the treatment of severe pneumonia. Eur J Clin Microbiol Infect Dis 1998;17(5):313-7.

46. Ramirez JA, Vargas S, Ritter GW, Brier ME, Wright A, Smith S, et al. Early switch from intravenous to oral antibiotics and early hospital discharge: a prospective observational study of 200 consecutive patients with community-acquired pneumonia. Arch Intern Med 1999;159(20):2449-54.

47. Meehan TP, Fine MJ, Krumholz HM, Scinto JD, Galusha DH, Mockalis JT, et al. Quality of care, process, and outcomes in elderly patients with pneumonia [see comments]. Jama 1997;278(23):2080-4.

48. CDHAC. Schedule of Pharmaceutical Benefits: Commonwealth Department of Health and Aged Care, 2000.

Home > Research > CAP

© 2004 The University of Adelaide Last Modified Wednesday, 28-Jun-2006 11:32:05 CST  Health Informatics	Copyright | Privacy | Disclaimer