
Home Research Topics Patient Safety Evidence Based Decision Support Clinical Workflow Other Core Topics Courses People Links

Health Informatics Unit
The University of Adelaide
SA 5005 Australia
Email

Telephone:
+61 8 8274 3713
Facsimile:
+61 8 8271 9158

The Crystal Ball - The Future Of Informatics And Decision Making

Malcolm Pradhan

Director of Health Informatics & Assoc. Dean IT, Faculty of Health Sciences, Adelaide University

Presented at the Australian Rheumatology Association 30 May 2001

The theme of this session is looking into the Crystal Ball. Some may find irony in sheer act of attempting to predict the future of science; perhaps in this age of evidence-based medicine I should assign 'p-values' to each of my predictions.

Making predictions about the future is fraught with peril, and the possibility of extreme embarrassment, particularly in the field of information technology. For example, in 1949 Popular Mechanics magazine predicted the future of computers. They prophesised:

"Where . the ENIAC is equipped with 18,000 vacuum tubes and weighs 30 tons, computers in the future may have only 1,000 vacuum tubes and perhaps weigh 2 tons."

Keep in mind that Palm computers today are equivalent to several thousand tons of ENIAC.

So, predicting the future is hard. The only assertion of which we are ensured is that whatever lies ahead is approaching at speeds we cannot cope with currently.

There are over 11 million citations in the Medline database today, and this number is growing by over 400,000 new articles every year. That translates into 1100 new articles cited every day, about 46 every hour, or about 2 articles since I began this talk.

How will we be able to deal with the future of health care and its information load? Over two thousand years, Marcus Aurelius Antoninus wrote about fear of the future:

"Never let the future disturb you. You will meet it, if you have to, with the same weapons of reason which today arm you against the present."

However, I assert that, in health care, the 'weapons of reason' of today are completely inadequate to meet the challenges of the future. I will go further and say that the evidence-based medicine ( EBM ) movement is entrenched in a misunderstanding of statistical methods, and that the techniques EBM supporters pontificate we should all use are impossible to use in practice - both from a cognitive and economic standpoint. I trust that this is controversial enough.

The good news is that we have the technology to deal with these problems. All pieces exist today: Evidence-based decision support systems, advanced techniques in research synthesis, alerting systems for clinical risk management, secure transactions for electronic data interchange, handheld computers, wireless networking. The good news is that we do not need to wait for breakthroughs in information technology or statistics to deal with challenges of today's health care system - most of the requisite theory and technology exist today .

The bad news is that the barriers to progress are much harder than making technological breakthroughs. The barriers lie in changing people, and in changing systems: the lie in changing the culture of health care organisations, they lie in enlightening politicians and high-level bureaucrats, they lie in changing the NHMRC's view of research, and the barriers lie in the form of corporate ownership of information.

Let us begin by reviewing, but not dwelling upon, the problems in health care that need improvement. You have probably read that widespread practice variation exists in most areas of health care. Practice variation has been reported by geographic region and by clinician experience. Practice variation by itself is not bad, but it is related to subsequent variation in outcome and cost. From an economic viewpoint, practice variation is a term that means unmanageable risk . From a financial viewpoint, this translates to uncontrollable cost and uncontrollable liability . This is not a good thing - business people hate this. This is why Health Maintenance Organisations (HMOs) in the USA are struggling for viability, and this is why health funders, such as governments and insurers and HMOs, try so hard to reduce practice variation.

An extensive range of interventions has been shown to reduce practice variation. Everything from IT systems, guidelines, seminars, and alerts have been successful in reducing variation, particularly in ambulatory settings. Very few interventions have shown sustainable changes in clinician practice patterns. The only exception to this has been academic detailing with respect to drug selection preferences.

A further problem only now reaching the public and legal consciousness is that of adverse events in health care that lead to premature deaths. A variety of studies estimate that around 14,000 people die prematurely in Australian hospitals every year. To put this figure in perspective, when Ford discovered a problem in the brake lining of some their models they recalled 90,000 vehicles; no deaths were recorded. Ansett grounded it's 767s when cracks appeared in the engine mounts; no deaths resulted from this error. Mitsubishi have recently recalled over 100,000 cars; no deaths have resulted from this error. The adverse events in health care are equivalent to a fully laden 767 crashing into the ground every week ! Both the National Health Service in the UK and HMOs in the United States are facing massive public liability, because they too have similar rates of adverse errors in their health systems.

Is the doctor to blame ? The assumption of the EBM movement back in the early nineties was that practice variation, and a large number of errors, resulted from the clinicians' inability to keep up with the latest medical research. What are the proposed EBM solutions? EBM proposes that when doctors face a challenging decision, they (1) formulate a clinical question, (2) get online, (3) formulate a query for a medical search engine, (4) filter and organise the results of the search, (5) perform a critical appraisal of the research, (6) check the Cochrane database for quantitative analyses, (7) integrate these incongruent data sources, and finally (8) take into account patient-specific features when applying the evidence. You have to fit a history and examination in there somewhere too. The practice of EBM is an economic Pyrrhic victory - the patient may be victorious, but our livelihoods will suffer.

EBM makes no mention of how we integrate patient preference or utility into the decision-making. Nor does it guide us in the application of research study results to a patient who would have been excluded from the study due to comorbidities or due to concurrent medications. The limitations of EBM are even more fundamental. From the late 1960's there has been a steady stream of quantitative research in cognitive science that has demonstrated systematically that we are not capable of manipulating subtle changes in probability. Evidence Based Medicine is a cognitive impossibility . The EBM movement has ignored this research. Our strength lies in our ability to integrate instantaneously a huge number of subtle queues and quickly converge to diagnostic and management decisions. However, this fascinating and complex phenomenon known as expertise may lead us astray on occasion, and that is why we need decision support.

From a personal viewpoint, my unit has expertise in research synthesis that considers such complex factors as heterogeneity in cohort populations, biases in study design, and studies with correlated variables. The mathematics required to perform this kind of research synthesis are complex, nonparametric, and computer intensive. To suggest that we can carry out this task in our heads during a patient encounter is, to me, fanciful.

The way studies are reported is a serious problem for research synthesis. To adequately make decisions we must consider the interaction of comorbidities and polypharmacy. However, published studies do not report any data on the interaction of clinical or outcome variables. At the Health Informatics unit at Adelaide University we have examined over 400 studies in detail , in a variety of areas such as pre-operative screening (including DVT, PONV), falls risk assessment, unstable angina, community-acquired pneumonia, and a variety of other infectious diseases.

Our aim is to construct evidence-based decision support systems that assist clinicians in applying evidence to individual patients, and to generate patient-specific recommendations that consider patient preference and resource availability. In these models we use best evidence in its true sense. If randomised trials are available we use them, but sometimes the best evidence available is expert opinion. Techniques such as 'value of information' tell us if particular variables require further investigation, and if so, how much we should spend improving our estimates.

In all of the 400+ publications we reviewed, we did not find a single report that published the all-important interaction information. Without this data we cannot perform accurate risk assessment or accurate research synthesis. Drug companies know this; they can access primary data because they fund studies. Thus, academic research is condemned to seek insight using incomplete data and often using simplistic statistical techniques that are acceptable to the NHMRC and to journals. Meanwhile, drug companies have access to detailed data and they use a wide variety of sophisticated techniques upon which to base their decisions.

So, Is the doctor to blame for variations in outcome and medical errors? I believe this is the wrong question. The right question is "Why does the health system provide so little support to doctors and to patients." When practice variation and medical errors are so pervasive across all parts of the health care system, the only conclusion we can draw is that the system is designed to work this way.

Professor Bill Runciman of the Australian Patient Safety Foundation, when talking about the health care system, claims a more accurate description is the 'health care activity'. Health care does not look or behave like a $40 billion company. Outside health care, there exists no other knowledge industry that provides so little support to its employees yet expects so much.

As an industry, health care spends about 2% of revenue of IT, which is much lower than other knowledge industries, and lower than all industries in general. Clinicians are expected to keep a vast amount of information in their heads, and then apply this information in real time and at great cost if errors are made. Most large supermarket chains spend more on IT to make sure their bread doesn't go stale, than health spends on IT to make sure that abnormal test results are seen, to assist clinicians to apply research results to specific problems, or to provide continuous quality improvement data back to clinicians.

Genomics is the straw that will break the camels back. The camel being current information management in health; I suppose genomics is more than a straw, it's more like a big bale of straw. [!]

The onset of cDNA micro-array technology will change the way diagnostic and management decisions are made in the future. Currently most diagnostic tests measure the endpoints of a long pathophysiological chain of events; for example liver function tests or thyroid function tests or even electrolytes. cDNA probes provide us with a picture of what proteins are active in our bodies. With micro-array technology we are already able to detect changes in the activity of proteins in specific diseases well before changes in endpoint chemistry or clinical symptoms. I believe these techniques will be the way therapy is monitored in the future - instead of waiting for symptoms to improve, we will be able to measure therapy effectiveness by measuring directly changes in protein levels the drugs are designed to influence. Based on technology available today, each micro-array test has the ability to return over 30,000 test results, per patient, per test. Researchers around the world are busy analysing micro-array data, matching up patterns in mRNA expression with diseases. We have trouble interpreting current test results, how will we be able to understand and act upon thousands of test results?

In the next 10 years the combination of health funders unwilling to accept uncontrollable risk, and the sheer volume of data available through array-technology will necessitate the use of decision support systems.

To participate in the process we have to make a subtle change from the way we conduct research and the way we use the results of research. Genomic information will improve our understanding in some areas, it will add to uncertainty in other. As decision-making becomes more complex, as more 'imperfect' information becomes available the response of EBM is to try to remove uncertainty rather than to manage it. The EBM notion of quality information is getting narrower, which means that more real world trials are rejected from review. We use arbitrary, utility-free decision rules such as setting alpha errors to 0.05 and beta errors to 0.8. Why?

In their original papers from the early 1930's, Neyman and Pearson who developed the hypothesis-testing framework along with Fisher, clearly state that the hypothesis-testing framework cannot tell whether a specific hypothesis is true or false, but that in the long run we correct more than we are wrong. They also state that alpha and beta errors should be set according to the decisions involved. If we are examining an intervention such as dietary change, then an alpha of 0.05 is too strict because the side effects of the therapy are minimal. If we are testing a chemotherapeutic agent that may result in death then 0.05 is too liberal. Somehow, all these fundamental limitations of hypothesis testing have been ignored by medical researchers and by the EBM movement. We are trying to prove truth, rather than trying to handle uncertainty. We are trying to use a hypothesis-testing framework for decision-making.

While the avalanche of new information, and now genomics, reveals the amazing complexity of biological systems and our interaction with them, EBM is trying to prove the answer to life, the universe and everything is 42, with a p-value of less than 0.05.

The technology and theory to support uncertainty management in health care is available through evidence-based decision support systems. We know how to improve IT system to support clinicians and to manage risk. We have the techniques to improve research synthesis. The bad news is that the barriers to progress are changes in attitude, and changes in systems, and these take time to change, but we must accept that change is necessary and we must begin moving in the right direction. To quote the late Douglas Adams again " Rome wasn't burned in a day"