Assessing the information needs of doctors

D. T. Leonello

July 1998

Supervisor: Malcolm Pradhan, Health Informatics, Faculty of Medicine

We constructed a study to firstly identify the information needs of doctors and secondly test different formats of information presentation. 11 doctors (5 physicians, 3 medical registrars, 2 interns and 1 GP) were asked to work through six different cases. Three different formats of information were employed. The doctors were asked to continually verbalise their thoughts and information requests and these were tape-recorded for the purpose of transcription.

The transcribed results were categorised by custom written computer software. The categories were analysed using logistic regression techniques.

1098 information requests were made by 11 doctors over the 6 cases. 43% of requests related to patient history, 6% of requests related to findings on physical examination and 51% of requests were for investigation results. The format of information presentation did not influence the test ordering behaviour over the six cases; however there were individual cases in which the format did influence information requesting. More experienced doctors requested more information and considered more differential diagnoses, as well as performing better in the treatment of the 6 cases.

In conclusion there was no difference in information requesting behaviour over the three formats, but there was evidence that different formats were beneficial for some individual cases. The different information requesting behaviour of doctors at varying levels of experience, suggests that customised information systems may be beneficial.

Aims

•  To determine the information needs of doctors.

•  To assess new methods of information presentation.

Background

Medicine is a progressively developing field. With this development comes increased data gathering and with this increased gathering of data, comes a need to store it. Computer technology over the recent decade has offered many solutions, firstly with magnetic tape storage and more recently digital CD-ROM technology, and the soon to be introduced DVD technology. Not only is the capacity of data storage improving, but the cost, reliability and speed of the systems accessing these storage devises are becoming better.

What does pose a problem is the retrieval of information. Data is stored for later use. How the data is stored, retrieved, processed and finally presented are all equally crucial steps in the delivery of information. At the end of this computerised assembly line, is the doctor; the human aspect that must interpret this information and translate it into patient management. It seems obvious that any attempt to model an information system must be done around its users and any measure of efficacy be made on the information system's ability to satisfy the information needs of these users.

The hospital case note is something that has been used for most of this century. More recently computer based information systems have been implemented. There is, however, evidence that suggests that the information needs of doctors are not met in current medical record systems or with paper based systems[ 1-3] . Paper based notes have been shown to be a difficult and unreliable storage medium [4] and coupled with research that suggests that computer information systems (CIS) are effective in improving the reliability of information delivery to doctors [5] , implementation of such a system must be strongly considered.

A hospital based CIS will be used for a range of purposes by varying types of health workers. Its ability to be customised and maximally efficacious to an individual user is important. For example, it has been shown that the poorer results of interns and non-specialist in general can be seen to reflect the degree of training that is required to effectively interpret rest results [6] . A CIS in this case might offer decision support components to the people that require them and the literature recognises the importance of integrating decision support with routine practice [7] .

On a local perspective, OACIS (Open Architecture Computer Information System) is the proposed statewide medical information system. It is currently trialled in the renal wards of the major teaching hospitals and soon the oncology wards hope to adopt it. There is no formal testing of OACIS in its ability to satisfy doctors' information needs. Any attempts to evaluate its efficacy have been based solely on its ability to store data. Before the multimillion-dollar investment proceeds further, it is crucial to identify and correct any faults, as the costs of changing a CIS during its design phase are anywhere between 10-200 times cheaper than implementing the same change during its operation [8] .

In order to test a CIS, the information needs of doctors must be recognised. Assessing information needs by means of a retrospective self reporting survey alone is problematic [9] . Potential bias may arise from several causes: firstly humans are shown to have selective recall of their previous behavior; secondly doctors may be unaware of an information deficit, thus making it impossible to report it; and thirdly, doctors' understanding of the term "information need" may differ, resulting in a variety of interpretations of survey questions.

Forsythe et al [2] overcame these problems by performing an observational study of doctors in a teaching hospital. The study identifies many ways in which information needs are expressed (verbally as well as non verbally).

Our study identified the information needs of doctors using a prospective interview based method while doctors were interpreting tests to reduce the bias of retrospective studies.

Method

Outline : The study involved interviewing 11 doctors. The interview involved six different cases, all with in an endocrine setting. Three different formats of information presentation were used for each case, but any one doctor would only see one of the formats. The three different formats were randomly rotated between the eleven doctors.

Sample : Eleven doctors were selected randomly and by availability. This group comprised of 5 physicians, 3 medical registrars, 2 interns and 1 GP. All but the GP were selected from the Royal Adelaide Hospital . The GP practices in metropolitan Adelaide . All were contacted several days before the interview and were given a brief description of the interview. They were informed of an approximate forty minute.

The cases : Six cases were selected, with the consultation of Professor A Need, Clinical Biochemistry, IMVS, in an attempt to include different aspects of the topic of osteoporosis. Case descriptions are included in Appendix I.

The interview : The interview ran through six different cases with each doctor. Every doctor was asked to work through the same six cases. However, the format of the method of information presentation changed.

•  Format 1:

This format was designed to best replicate the current information system of the RAH, i.e. the 'case-note'. The interview would commence by giving a verbal presenting complaint and then asking the doctor which other information he or she needed to progress to a diagnostic and management endpoint. 'History' and 'physical examination' information was given verbally. Investigations were given on single sheets of paper in the same batches used by the IMVS. The results were placed in a clip, so that the resulting 'booklet' of tests allowed the doctor to view only one result at a time, similar to the way in which they are pasted in the RAH case note.

Diagram 1: Example of batches or individual tests on single slips of paper.

•  Format 2:

Format 2 is a system that some of the endocrinologists at the RAH use. It involves the transcription of the single tests above onto a single sheet of paper. Therefore, not only can they view all the tests they want at the same time without having to flick through paper, but previous results are also displayed, giving a temporal perspective. After an information request for an available investigation was made, a 'snippet' of the results, along with any previous results were 'pasted' on a blank sheet of paper.

Diagram 2: Example of results on a single sheet of paper along with previous results.

•  Format 3:

This format was entirely written on a single A4 page. The top of the page contained a presenting complaint, history and findings on examination. Lower down the page, a set of investigations were given, along with any previous results for these investigations. The investigations that were included were derived from consultation with Professor Need, clinical biochemistry at the IMVS, as a good screening batch of tests for patients with suspected secondary causes of osteoporosis or related metabolic bone disease. The bottom of the page contained a brief comment on Professor Need's interpretation of the biochemical and clinical profile. It did not, however, recommend a management opinion and this was left up to the interviewed doctor to formulate.

Diagram 3: Example of single sheet containing previous and current investigation results and clinical information.

For each interview doctors were asked to verbalise their thoughts on the case. If they were silent for more than a few seconds they were asked to "think aloud". This method of interview does not have an adverse effect the analytical thought process[ 10] . The interviews were tape recorded, and the doctors' information requests and thoughts about the cases were transcribed for later analysis (please see Appendix II for an example of a case transcription). This act of verbalisation was particularly important during the third format. Due to its entirely written nature, verbal requests for information were not necessary to receive information and therefore doctors had to be continually reminded to verbalise what it was they were looking at and using on the sheet of paper.

Results

All eleven doctors were interviewed; however, three of the cases covered by one of the doctors were not available for analysis due to recording problems.

There were 1098 information requests recorded. These were broadly categorised into three different information types: history, examination and investigations. The proportions of each, over the 11 doctors and 6 cases, are demonstrated in diagram 4 .

Diagram 1

Each doctor was categorised into a group; registrar, physician, general practitioner and intern. The cases were classified by the initials of the real case from which they were constructed.

Table 1 demonstrates the number of information requests with respect to the type of format used.

Table 1 : Information Requests with respect to format

*treatment and referral were not considered as information requests

Table 2 demonstrates information requests with respect to the cases:

Table 2 : Information Requests with respect to case

*treatment and referral were not considered as information requests

Table 3 demonstrates the average information request per case with respect to doctor type:

Table 3 : Information requests with respect to doctor type

Logistic regression was done with request type as the dependent variable (Y) and analysed against doctor type, case and format type. The results are shown in Table 4:

Table 4 : Statistical analysis

Graphical representation of the number of information requests versus the doctor type is demonstrated in diagram 2 :

Diagram 2 .

p = 0.0005

The number of differential diagnoses that each doctor group considered over the six cases was also documented and is displayed in diagram 3 :

Diagram 3.

The performance of the different doctor types is shown in diagram 4:

Diagram 4.

p = 0.072

Discussion of results

Approximately half of the requested information was history and findings on examination. The remaining half were requests for investigations.

More highly trained doctors (e.g. physicians) require more information, consider a higher number of differential diagnoses and perform better than the other groups of doctors.

When viewed as a whole, with no consideration for doctor types, the format of the presentation did not signigicantly affect the number of information requests. However, different cases demonstrated varying significance for this effect; in particular, with Mrs RD , format and its influence on information requests was significant.

An interesting observation came from separating doctor type (physician, registrar and intern) and then comparing information requesting behaviour to format and patient.

For the interns, the type of format used and the information requesting behaviour did become important (p = 0.0156). It was also seen that the patient type had a bearing on information requesting, although not as significant (p=0.074) (Table 5).

Table 5.

The registrars demonstrated the same pattern; however the effect of format was not as strong (p = 0.052) but the type of patient did strongly influence the information requesting (p = 0.0022) (Table 6).

Table 6.

In the physician group, there was no observed significant correlation between firstly the format and information requests and secondly the patient and information requests. (Table 7)

Table 7.

Conclusion

Current reporting formats for investigations contain little clinical information, other than patient identifiers. It is often necessary, however, to evaluate investigation results in the context of such clinical information and in the past it has been necessary for the doctor to find the written clinical transcripts in the case notes; a troublesome and laborious task, especially with large notes that expand over several volumes and poor handwritten records.

Although not formally analysed, it was clear by looking at the order of information requests that "history" and "examination" did not always precede "investigation", suggesting that during the interpretation of investigation results, clinical information was being processing in conjunction with investigation findings. There is scope therefore, for an information system that displays clinical data together with investigation results to make this process of integrating history, examination and investigation more efficient and reliable.

However, our study demonstrated that formats incorporating history, examination and investigation did not significantly change the information requests of doctors in comparison to the existing 'case-note' style of information presentation. A point to consider in understanding this anomaly is the inherent 'time bias' of the study. Format 1 was constructed to simulate the current case-note information system, where individual investigation reports were layered in the case note. In reality, however, there is a time lag of days between the ordering of a test and the return of the report. By this time, most of the patient history and examination is likely to have been forgotten by the doctor. In our study, data on history and examination were, at the most, minutes old when results of investigations were given to the doctor; an effect which may have shrouded any benefit of the more concise and comprehensive formats 2 and 3. Similar future studies may return better results if run in real time.

Although the six cases, when analysed in their entirety, showed no significant change in information ordering behaviour, one of the six cases by itself did demonstrate significant difference in information requesting over the three formats. It is possible that this may be a chance occurrence (p = 0.0369); however, the nature of the case (primary hyperparathyroidism) is one which clinically presents ambiguously, yet on laboratory investigation appears as a simple combination of hypercalcaemia and hyperparathyroidism. The third format, which included the use of investigations from the very start of the case, may have displayed the case more clearly than other formats, resulting in the observed difference in information requests. With the perspective of instituting a hospital CIS, there may be scope for individualising format presentation in terms of case type, as well as user type.

Finally, the institution of a CIS is an extremely expensive task. These expenses become much more if mistakes are made during the design of the CIS. Identifying these mistakes, however, is only possible if we understand the information delivery and utility processes that doctors use in decision making. In essence, our study needed to create a measurement tool to record doctors' information needs. Having developed this 'tool', there is now the prospect of applying it, in a broader sense, to more doctors and patient cases, in order to gain an understanding of the true information needs of doctors. It is only through this process that we hope to develop effective CIS.

References

1. Covell, D.G., Uman, G.C.,& Manning, P.R. (1985). Information needs in office practice: are they being met? Ann Intern Med , 103(4):596-9.
2. Osheroff, J. A., Forsythe, D.E. et al. (1991). Physicians' information needs:analysis of questions posed during clinical teaching. Ann Intern Med , 114(7):576-81.
3. Tang, P.C., Jaworski, M.A. et al. (1995). Methods for assessing information needs of clinicians in ambulatory care. Proc Annu Symp Comput Appl Med Care ,:630-4
4. Tang, P.C., Fafchamps, D., & Shortliffe, E.H (1994). Traditional medical records as a source of clinical data in the outpatient setting. Proc Annu Symp Comput Appl Med Care , :575-9.
5. Sands, D.Z., Libman, H., & Safran, C. (1995). Meeting information needs: analysis of clinicians' use of an HIV database through an electronic medical record. Medinfo, 8 Pt 1:323-6.
6. Kirby, J.E. & Laposata, M, (1997). The nature and extent of training activities in clinical pathology required for effective consultation on laboratory test selection and interpretation. Arch Pathol Lab Med , 121(11):1163-7.
7. Shortliffe EH. Computer programs to support medical decision making. JAMA . 1987; 258;61-66.7.
8. McConnell S. Rapid Development. 1996. Microsoft Press, Redmond , WA .
9. Erricsson, K.A., and Simon, H.A. Verbal Reports As Data. Psychol . Rev. 87, 215 (1980).
10. Norris, S. Effect of eliciting verbal reports on thinking on critical thinking test performance. Educ Meas. 27(1):41-58 (1990)

Home > Research > Health Systems Analysis

© 2004 The University of Adelaide Last Modified Wednesday, 28-Jun-2006 11:32:05 CST  Health Informatics	Copyright | Privacy | Disclaimer