A Comparison with Statistical Models of an Evidence-Based Decision Support System for the Prophylaxis of Post-Operative Nausea and Vomiting

Pradhan M 1 , Crichton T 2 , Edmonds M 2 and Ludbrook G 3

1 Health Informatics, Faculty of Health Sciences, University of Adelaide , South Australia 5005

2 4th year medicine, School of Medicine , The University of Adelaide, South Australia 5005

3 Department of Anaesthetics & Intensive Care, The University of Adelaide , South Australia 5005

[Presented at HIC 1999]

Post-operative nausea and vomiting (PONV) represents a significant problem in anaesthetic procedures, particularly in the day-surgery patients. It is the most commonly stated fear of patients prior to elective surgery. In 1995 the Quality in Australian Health Care Study identified post-operative nausea and vomiting as the most common adverse event in the speciality of anaesthesia. It is estimated to cost over $1.2m per year in unanticipated hospital admissions following day surgery alone, which represents but a small percentage of the incidence of PONV. A large proportion of the cases of PONV do not result in delayed discharge or require readmission, but costs are felt in other areas, such as visits to general practitioners, slower recovery and loss of productivity by the patient. Despite this, there is no current general consensus regarding the prophylactic management of PONV.

We describe briefly the construction of an evidence-based decision model to predict the risk of PONV, and to determine the optimal prophylactic management in a specific patient. The model incorporates quantitative information about the factors that influence the incidence of PONV, the qualitative associations between these factors, and the values of potential outcomes. The model was constructed from extensive literature review and expert knowledge. Using a cross-validation experiment of a day surgery database, we compare the predictive accuracy of the decision model with a logistic regression model, a classification tree and feed-forward neural network. We discuss the relative merits and disadvantages of each approach in the context of a clinical decision support system.

The decision problem for the prophylaxis of post-operative nausea and vomiting

In 1995 the Quality in Australian Health Care Study identified post-operative nausea and vomiting (PONV) as the most common adverse event in the speciality of anaesthesia [1] . PONV has been found to be the cause of 17% of unanticipated readmissions following day surgery [2] - costing an estimated $1.2m per year, which represents but a small percentage of the incidence of PONV. A large proportion of the cases of PONV do not result in delayed discharge or require readmission, but costs are felt in other areas, such as visits to general practitioners, slower recovery and loss of productivity by the patient. In our study we have decided to define PONV as any episode of nausea, retching or vomiting in the 24 hours immediately following an operation, and the focus of this current study is the area of day surgery.

An additional cost of PONV is often not quantified: the effect of the condition on the patient's wellbeing and comfort. Kovac [3] stated that 38% of patients who experienced PONV perceived it to be more debilitating than the effects of the surgery itself. A recent survey of patients has shown that it is the single most commonly stated fear preceding elective surgery. Nausea alone can also limit patient activity, and severe nausea can be as, or more, distressing than vomiting or retching. There are also medical and surgical consequences of PONV. Vomiting may lead to dehydration, interference with nutrition and oral drug therapy or dehiscence of abdominal wounds [4] . It may also result in tension on suture lines, increased bleeding under skin flaps, venous hypertension and increased risk of pulmonary aspiration of vomitus [5, 6] .

Despite the large amount of research performed in this area there is currently no general consensus or guideline about the management and prophylaxis of PONV. Prevention of nausea and vomiting is important, not only in monetary value, but also in relation to the quality of care. However, routine anti-emetic prophylaxis is not been indicated since only 30% of patients experience post-operative emetic sequelae [6, 7] , and of these, many cases are only transient nausea, or only one or two bouts of vomiting or retching. Many of the anti-emetic agents also have a spectrum of associated adverse effects, potentially causing further discomfort for the patient [6] .

Under a policy of routine anti-emetic prophylaxis patients at low risk of developing PONV, faced the risk of the adverse effects of anti-emetics. This risk has been suggested to outweigh the potential benefits of routine prophylaxis, and the cost of administering the agents would amount to more than the cost of readmissions if no prophylaxis were given. On the other hand, use of anti-emetic agents in patients at high risk of developing PONV could greatly reduce the incidence of PONV, reducing cost, patient discomfort and medical complications, while increasing the quality of care. The decision problem we faced was how to identify the subset of the population that are at high risk of developing PONV, and would benefit from prophylactic anti-emetics. Our approach was to construct a decision model that would be used as a tool for communication and for quantitative analysis of the decision problem.

An introduction to decision models

The term 'evidence-based' may be defined as "the conscientious, explicit and judicious use of best current evidence in making decisions" [8] . In the case of the PONV decision model, we used evidence primarily studies involving randomised trials combined with expert opinion from the Royal Adelaide Hospital (RAH) Department of Anaesthesia and Intensive Care. The decision we wanted to model was which anti-emetic, if any, should be administered to patients at risk to reduce the incidence of PONV. Before describing in detail the PONV decision model, we introduce the concept of decision models using a simpler, non-medical problem.

A decision model or influence diagram is a graphical statistical model that defines explicitly the relationships, or influence, between variables of interest. The most common form of decision models is the decision tree. Influence diagrams and decision trees are mathematically equivalent but trees are often used in a patient specific manner as the layout of the tree imposes an implicit ordering of information gathering. In contrast to decision trees, influence diagrams emphasise the representation of conditional independence in a model. A number of tutorials are available for the construction of decision trees [9] and influence diagrams [10-12] .

Figure 1 a shows a simple influence diagram that models the difficult decision of 'the party problem' [13] , that is, where should we hold a party given uncertainty about the weather- in the backyard, on the patio, or inside? We obviously don't want the party to be a washout, nor do we want to be indoors when it's sunny outside. The oval nodes represent uncertain variables of interest. In this simple model the variable Weather can take on the values 'Rain' or 'Sun'. Since we do not know which state it will be we will assign probabilities to each state. As there are no incoming arcs to the variable Weather we must define prior probabilities to each state. The arc connecting Weather to the uncertain variable Forecast defines a conditional dependency between the two variables. In words, this arc tells us that the Forecast is dependent on what the Weather will be. In the case of our decision we may observe a forecast before we make the decision of where to hold the party, defined by the rectangular node Location? . The Value node is our preference order of possible outcomes.

Figure 1 . A simple influence diagram (a) and an equivalent decision tree (b) that model the party problem.

To specify the model shown in Figure 1 a we must define the probabilities outlined in Figure 2 . The specification of the decision node Location is a set of decisions we are considering; the Value node is a set of utilities or preferences over each possible outcome. The utilities shown in Figure state that the best outcome (value = 1) is if the party is held in the backyard and the weather is sunny, but the worse (value = 0) is if the party is held in the backyard and it rains. The utility does not have to be between 0 and 1, it can be in any units that are appropriate for the task, such as quality adjusted life years or dollars.

The probabilities that specify the node Forecast define the sensitivity and specificity of the weather forecast. For example, the sensitivity, or true positive rate, for the detection of rain is

.

Figure 2 . Parameters required to specify the party problem influence diagram.

Figure 1 b shows an equivalent decision tree for the influence diagram shown in Figure 1 a. Note that while mathematically equivalent the influence diagram highlights the qualitative structure of the problem, particularly as the number of variables grows. A decision tree displays explicitly asymmetry in the probability distributions. This occurs when decisions lead to different states of the world. Asymmetry is represented implicitly in the conditional probability tables of an influence diagram. In general, as a model grows in complexity it is the conditional dependencies that are important to communicate.

Another advantage of the influence diagram over decision trees is that we need not be concerned immediately with the nature of the probability distributions that specify arcs between uncertain variables. In the simple party problem all variables are specified as discrete probability tables; by using discrete (multinomial) distributions we can specify non-linear models. Probability distributions may also be specified as continuous parametric or semi-parametric multivariate distributions [14] . It is worth noting that many traditional statistical models may be represented using influence diagram notation, including regression models, ARIMA models, Markov models, and neural networks.

The purpose of these models is to facilitate decision-making. Inference is the calculation of the posterior probability for unobserved variables and the expected utility of decisions based on observations. For discrete models there exist efficient algorithms for inference [15] . In general, inference in continuous models requires simulation techniques, such as Monte Carlo Markov Chain simulation [16] . Beyond the ability to define sophisticated models, the decision theoretic framework provides a powerful perspective for the analysis of decisions such as sensitivity analysis and value of information. Sensitivity analysis allows us to assess the effect on our decision of variables in the model, while value of information informs us on how much effort we should spend on information gathering (for a good introduction to the topic see [17] ).

To summarise, recent advances in graphical statistical models allow us to specify complex clinical problem using influence diagrams. These models can be parametric, semi-parametric or non-parametric, and can be seen as a superset of traditional statistical models [18] . The advantage of the influence diagram formalism over traditional statistics is the ability to model explicitly dependencies between variables rather than having to avoid dependencies. Given this context, this work seeks to compare the difference between the manual construction of models and the automated parameter learning techniques such as logistic regression, classification trees and neural networks.

Creation of the PONV decision model

Two of the authors (TC and ME) conducted an extensive literary review to locate the best current knowledge regarding the aetiology, physiology, risk factors and treatments of PONV. We compiled literature from numerous sources, primarily the results of searches conducted in Medline, the Cochrane library and the Internet, as well as suggested reference texts supplied by the Department of Anaesthesia and Intensive Care, Royal Adelaide Hospital (RAH). This search resulted in a total of over 60 references, dating from 1957 to 1998. These references provided us with a comprehensive list of all factors thought to be associated with the development of PONV. Using this information an initial qualitative model was constructed.

The qualitative model

To be consistent with previous studies, and for simplicity, all of the suggested factors were categorised as contributing to patient, anaesthetic or operative risks. This association is represented in the influence diagram by an arc between the nodes of these factors. These three main categories combine to give an overall risk of PONV.

The anti-emetic agents included in this study were Metoclopramide, Droperidol and Tropisetron, as these agents are represent the major classes, and they are in use currently at the RAH. Since our model was designed primarily for a hospital policy decision, and partly due to time constraints, we defined the value function in terms of costs of drug treatment and readmission. The current model does not take into account patient preferences and quality of life beyond the fact that readmission implies a negative health state. We refined the model in response to expert opinion from anaesthetic specialists in the RAH Department of Anaesthetics and Intensive Care.

The quantitative model

Quantification involved a further literature review and consultation with an anaesthetic specialist. We located original evidence from studies and meta-analyses. We assessed evidence from these articles for relevance and validity; our assessment included the study design, the patient demographics of the study, the sample size, and various confounding factors such as operation types or anaesthetic techniques. Using this process we prioritised the studies by quality of evidence.

The evidence from the studies came in the form of percentages, odds-ratios and number-needed-to-treat (NNT). We standardised these presentations for use in the PONV decision model. We removed from the model factors that were not substantiated by study evidence, or had evidence that showed they were not a significant influence on the development of PONV. Examples of factors we excluded were gastroparesis and a past history of motion sickness.

The influence of each individual factor was weighted on the impact it would have on PONV. This was achieved by weighting the influence of each of the major categories of patient, operative and anaesthetic risk on the risk of PONV, and weighting the influence of each of the factors on the categorical risk they fell into. A 'noisy-or' function [19] was used to define the interaction between causal factors where these interactions were not available.

We determined the cost and efficacy of the different anti-emetic agents and entered these values into the model, along with the costs arising from each option and potential outcome. The anti-emetic agents used in this model were Metoclopramide, Droperidol and a 5HT 3 -antagonist, Tropisetron. The costs of each of the agents were obtained from the RAH Department of Anaesthesia and Intensive Care, and these were used in a function estimating the value of the cost of treatment node. The resulting model ( Figure 3 ) was then capable of predicting the risk of PONV as well as calculating the costs of each of the treatment options for that risk.

Figure 3 . The PONV decision model.

We also attempted to build a mechanistic model of PONV. Unfortunately, there did not exist in the literature enough information to allow us to quantify the mechanistic model of PONV, which may have lead to further insights regarding the difference in mechanisms of action for the different anti-emetics.

To construct our models we used a variety of software tools for the construction of decision models: Hugin v5.2 Lite (www.hugin.dk), Netica (www.norsys.com), and GeNIE (www2.sis.pitt.edu/~genie/).

Model output

Once completed we could use the model as real-time decision support tool, or as a basis of policy generation. Figure 4 shows the PONV decision model in use as a real-time decision support tool that recommends therapy for individual patients. The Figure demonstrates the following scenario: An obese (1) female (2) with a past medical history of nausea and vomiting (3) is having an operation in which Neostigmine (4) will be used. The system calculates the risk of PONV(5), and recommends the most cost-effective anti-emetic (6) based on the cost of the drugs and the potentially expensive outcome of readmission to hospital. In this case the best alternative is the serotonin antagonist (e.g. Tropisetron ).

Another use of the PONV decision model is in the formulation of policy. Figure 5 is an analysis of the decision thresholds for anti-emetic choice. Figure 5 a shows how the recommended anti-emetic treatment changes with the risk of PONV. This figure shows that a single drug will not be cost effective for all patients. It is easy to change the cost of a treatment, in this example we lowered the cost for the serotonin (5HT3) antagonist from the current level of $11 to $8, but kept the same efficacy. Figure 5 b shows how the role of the cheaper but slightly less effective anti-emetic Droperidol is reduced in the face the price change. Without a decision model, such changes in drug treatment may have required a significant cost as experts are reassembled and the literature is re-reviewed.

Figure 4 . The PONV decision model in action as a decision support tool.

See text for an explanation of the numbered steps.

The validation of our model requires a randomised controlled trial to test if the model's recommendations lead to decreased PONV compared to normal therapy-largely ad hoc treatment. While we are planning to conduct such a study our first goal was to validate the predictive power of the model compared to standard statistical models. We used a day surgery database comprising 6000 day surgery admissions. The database did not contain all the variables used in our model but those variables it did contain were used in the experiments.

Figure 5 . Decision thresholds for the PONV model (a) and the effect of changing the cost of a treatment (b).

Model comparison methods

We imported the day surgery (DSU) database into the statistical software package S-Plus [20] . Any obvious errors in encoding and inconsistencies in the database were fixed leaving just under 6000 cases. We split the dataset randomly into a training set of 4984 and a test set of 1009. Each statistical model would use the training set to learn parameters; the test set was used to assess each model's performance. We used the built-in logistic regression and classification tree functions of S-Plus for these experiments. To implement neural networks we used a separate add-on package [21] . One of the authors (MP) wrote S-Plus code to carry out model training, analysis, visualisation, and the ROC curve functions.

The evaluation consists of two components. The first is a model comparison for which we used two continuous variables, Age and Operation time , to predict the presence of PONV. The motivations for this experiment are (a) in lower dimensions it is easier to visualise how different statistical methods separate the feature space, and (b) we are able to get a general feel for the degree of difficulty of the classification task.

For each model we generated ROC curves [9] which plot a test's true positive rate (TPR or sensitivity) against its false positive rate (FPR or 1 - specificity). The ROC curve is a useful guide to the trade off between sensitivity and specificity (1-FPR), but it does not facilitate decision making directly. We wanted to assess the statistical models' ability for a treatment decision. The decision model already performs the task of treatment recommendation based on the principle of expected utility theory. However, to compare the models we ignored the decision and value nodes in the decision model and used its ROC curve only.

In the second component of the evaluation we assessed the cost of mistreatment and readmission of the models for a larger set of factors in the test dataset. The features we used to predict PONV were Age, Sex, Operative.risk, Opioids, Propofol, Volatile, N2O, and Neostigmine. For this experiment we used prevalence (prior probability) of PONV of 10%, a drug cost of $8, a drug efficacy rate of 60%, and a readmission cost of $500. We assumed that 20% of those with PONV were readmitted, and a population of 1000 patients. Except for the decision model, the statistical models did not recommend which therapy to give, but gave a probability of PONV given a set of patient features. We tested each point on the ROC curve to assess the best threshold value for the decision to administer an anti-emetic using the costs and parameters described (see Appendix for details). The ideal test would have a high TPR and low FPR, but this is rarely possible.

We conducted standard validation tests on the statistical models, including error diagnostics. We did not make any attempts to optimise the performance on the training set beyond pruning the decision tree; in this experiment we are concerned with comparing automated machine learning methods rather than seeking to optimise an individual algorithm.

Researchers have developed a variety of methods of learning parameters in decision models from data [22, 23] . We used a simple algorithm to update parameters in the model from the DSU database, but this did not improve the predictive accuracy of the model significantly because the algorithm did not update the intermediate (latent) variables. We plan to use a more sophisticated method based on the Expectation Maximisation (EM) algorithm [24] in future work.

Results of model comparison

Figure 6 to Figure 9 consists of two graphs each. In the graphs labelled (a), each patient from the test set is represented a black filled dot if they suffered PONV, or a light dot if the person did not suffer PONV. The patients are distributed on the graph by their age and the operation time. For each classification method the probability contours indicate the general shape each technique uses to partition the feature space. In simple classification tasks the population of interest will be linearly separable from the rest of the population. If this is the case then linear regression will partition the groups successfully. In more complex classification tasks the methods may need to generate non-linear cuts through the space. In each figure, the graphs labelled (b) shows the ROC curve that maps the FPR against the TPR. Each ROC curve plots the results of the training set and the test set. When comparing tests the area under the ROC curve is considered a measure of the discriminatory power of a test [25] , these results are listed in the summary table ( Figure 11 ).

Informally, the logistic regression model ( Figure 6 ) uses smooth curves to split the feature space. In the figure the curves appear here as straight lines but they may not always do so. Like logistic regression, the feed-forward neural network also uses the logistic function ( ) at its core but it has more parameters with which it can fit a surface. The number of parameters in a neural network can be adjusted by adding units to the hidden layer . It is the hidden layer that bestows upon neural networks the ability to fit non-linear surfaces, as can be seen in Figure 7 . There are a number of heuristics to guide the selection of hidden units; blithely adding units will result in overfitting. When a model overfits a training set it will perform badly on new data. To illustrate this phenomenon we trained a neural network with 20 hidden units ( Figure 9 ) that fit the training set well but its highly customised partitions perform badly on the test set, resulting in a large drop in the ROC curve between the training and test sets. Finally, the classification tree ( Figure 8 ) creates piece-wise linear partitions of the feature space; classification trees are not restricted to partitions based on the logistic function. The default tree was pruned based on residual deviation plots [26] to reduce overfitting. In this experiment the tree's performance was similar to that of the neural network. Interestingly, both models created similar partitions using different techniques.

It is important to note that this simple comparison is not designed to judge the different classification methods; each of these methods has many variations that we have not explored. As discussed in Section 2, these methods have an underlying relationship, and we can use them in combination. The comparison between these basic methods allows us to understand how the different techniques approach classification, and it verifies that the classification task is difficult because of the overlap between the target population (those who suffer PONV) and background.

The difficulty in classifying patients at risk of PONV extends to the multidimensional case. As we add factors to a model we can explain potentially more variance but we also require more data to fit the additional parameters of the model-the so-called 'curse of dimensionality'. We did not find factors that are particularly predictive for the event; hence, the ROC curves are similar in the higher dimensions.

The ROC curve of the decision model is shown in Figure 10 . In contrast to the other data-driven models, this performance was achieved without using the training set, but on literature review and expert knowledge alone.

Figure 6 . Logistic regression. Classification contours (a) and ROC curves (b).

Figure 7 . Neural network with 4 units in the hidden layer. Classification contours (a) and ROC curves (b).

Figure 8 . A classification tree. Classification contours (a) and ROC curves (b).

Figure 9 . An example of overfitting: a neural network with 20 units in the hidden layer.

Figure 10 . ROC curve for the decision model.

We ignored the decision model's ability to make one of four recommendations and considered only a single treatment option for the comparison. The formulae we used for the cost calculations are in the Appendix. The results of the comparison are shown in Figure 11 . We conducted extensive ROC curve analysis to determine the threshold probability that yields the minimum cost strategy for each model. The theoretical minimum with 100% sensitivity and specificity is $4,843.20 for 1000 patients. The relative error from this theoretical minimum is shown.

Figure 11 . A comparison of the predictive components of various statistical models.

*This tests the predictive component of the decision model only.

The results of the comparison show that all models performed in a similar fashion. Again, the decision model performed surprisingly well considering it had not been trained on data from this population.

Discussion and conclusions

In this comparison we have demonstrated visually how different statistical techniques approach the classification task. We assessed the use of a simple decision rule using ROC curve analysis. We did not test the models against the full decision making capability of the decision model, nor did we seek to optimise algorithms. In the comparisons the knowledge-derived decision model performed well compared to the data-driven methods. It is interesting to note that the logistic regression model was the simplest model, this simplicity worked in its favour in this difficult classification task because it made fewer assumptions about the dataset. Both the classification tree and neural network may be fine-tuned to create simpler fits to the data.

In comparison to data driven methods, such as neural networks, a significant advantage of the decision model is that the parameters maintain semantics, that is, parameters are named concepts. Furthermore, we are able to model explicitly features not in the dataset but which may influence the relationship between observed variables.

While decision theoretic models are powerful formalisms for decision support, classification and policy formulation, they do not have the same ease-of-use as traditional statistical packages. Decision modelling requires of the user an understanding of modelling and model critiquing. As tools emerge from the research labs decision theoretic techniques will become more widely used.

In pure classification tasks it is unlikely that decision models will perform as well as neural networks or similar models because the use in influence diagrams of causal structures and explicit prior probabilities generate more parameters to learn. However, the explicit representation of parameters in influence diagrams makes them modular. For example, if the incidence of a disease changes we can simply change the prior probability on a single node, whereas a neural network needs to be retrained. Modularity is important in medicine where systems must be customised for differing resource availability, or for different populations. The ability of influence diagrams to combine in a modular structure evidence from the literature, local data and patient preferences are paricularly powerful features.

Appendix

The cost calculations were as follows. Let Sn (TPR) be the sensitivity of the prediction for PONV, Sp (TNR) the specificity, the prior probability of PONV, and the number of patients N . Ef is the efficacy of our treatment, its cost is Cd , while Cr is the cost of readmission. The readmission rate to hospital for patients suffering from PONV is Rr . Then, , and .

References

[1] W. Runciman, "Anaesthetists and the "Quality in Australian Health Care Study"," Australian society of anaesthetists Newsletter , 15-17, 1997.

[2] B. Gold, D. Kitz, J. Lecky, and N. J, "Unanticipated Admission to the Hospital Following Ambulatory Surgery," Journal of the American Medical Association , 262, 3008-3010, 1989.

[3] A. Kovac, M. Pearman, S. Khalil, P. Scuderi, A. Joslyn, B. Prillaman, and F. Cox, "Odansetron prevents postoperative emesis in male outpatients," Journal of Clinical Anesthesia , 8, 644-651, 1996.

[4] K. Korttila, "The Study of Postoperative Nausea and Vomiting," British Journal of Anaesthesia , 69, 20s-23s, 1992.

[5] M. Watcha and P. White, "Postoperative Nausea and Vomiting: its etiology, treatment and prevention," Anesthesiology , 77, 162-184, 1992.

[6] M. Watcha and P. White, "Nausea and vomiting, pharmacology and clinical uses of antiemetic drugs," in International Practice of Anaesthesia , C. Prys-Roberts and B. J. Brown, Eds. Oxford: Butterworth-Heinemann, 1996.

[7] P. White and M. Watcha, "Are New Drugs Cost-effective for Patients Undergoing Ambulatory Surgery?," Anesthesiology , 78, 2-5, 1993.

[8] D. Sackett, W. Richardson, W. Rosenberg, and R. Haynes, Evidence-based Medicine . New York: Churchill Livingstone, 1997.

[9] H. C. Sox, M. A. Blatt, M. C. Higgins, and K. Marton, Medical Decision Making : Butterworth-Heinemann, 1988.

[10] R. Nease and D. Owens, "Use of Influence Diagrams to Structure Medical Decisions," Medical Decision Making , 17, 263-275, 1997.

[11] D. Owens, R. Shachter, and R. Nease, "Representation and Analysis of Medical Decision Problems with Influence Diagrams," Medical Decision Making , 17, 241-262, 1997.

[12] D. Owens and R. Nease, "A Normative Analytic Framework for Development of Practice Guidelines for Specific Clinical Populations," Medical Decision Making , 17, 409-426, 1997.

[13] R. Howard, "Decision analysis workbook," Stanford University, Stanford, CA 1993.

[14] J. P. Chevrolat, F. Rutigliano, and J. L. Golmard, "Mixed Bayesian networks: a mixture of Gaussian distributions," Methods of Information in Medicine , 33, 535-42, 1994.

[15] S. Lauritzen and D. Spiegelhalter, "Local computations with probabilities on graphical structures and their application to expert systems," Journal of the Royal Statistical Society , 50, 157-224, 1988.

[16] R. M. Neal, "Probabilistic inference using Markov chain Monte Carlo methods," Dept. of Computer Science, University of Toronto, Toronto CRG-TR-93-1, 1993.

[17] R. T. Clemen, Making hard decisions . Belmont, CA: Wadsworth Publishing, 1996.

[18] W. Buntine, "Graphical models for discovering knowlege," in Advances in knowledge discovery and data mining , U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy, Eds. Menlo Park, California: AAAI Press/MIT Press, 1996, 59-82.

[19] M. Henrion, "Practical issues in constructing a Bayes' belief network," in Uncertainty in Artificial Intelligence 3 , T. Levitt, J. F. Lemmer, and L. N. Kanal, Eds. Amsterdam: North Holland, 1988, 132-139.

[20] J. M. Chambers and T. J. Hastie, Statistical methods in S . New York: Chapman and Hall, 1992.

[21] W. N. Venables and B. D. Ripley, Modern applied statistics with S-Plus , Second ed. New York, NY: Springer-Verlag, 1997.

[22] W. Buntine, "A guide to the literature on learning probabilistic networks from data," IEEE Transactions on Knowledge and Data Engineering , 8, 195-210, 1996.

[23] D. Heckerman, D. Geiger, and D. M. Chickering, "Learning Bayesian networks: the combination of knowledge and statistical data," Uncertainty in Artificial Intelligence. Proceedings of the Tenth Conference , 293-301, 1994.

[24] A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society B , 39, 1-38, 1977.

[25] D. K. McClish, "Analyzing a portion of the ROC curve," Med Decis Making , 9, 190-5, 1989.

[26] B. D. Ripley, Pattern recognition and neural networks . Cambridge: Cambridge University Press, 1996.

Home > Topics > Decision Support

© 2004 The University of Adelaide Last Modified Wednesday, 28-Jun-2006 11:32:06 CST  Health Informatics	Copyright | Privacy | Disclaimer