Use of simulation modelling to overcome operational and structural inefficiencies

Mackay M 1 and Pradhan M 2

1 Country and Disability Services Division, Department of Human Services, 11-13 Hindmarsh Square , Adelaide South Australia 5000

2 Health Informatics, Faculty of Health Sciences, University of Adelaide , North Terrace, South Australia 5000

Financial stress in the acute hospital environment can be broadly categorized as arising as a consequence of one of three types of problems: structural inefficiencies, operational inefficiencies, and activity problems. A review of the regional country hospitals in South Australia was initiated during August 1999. The focus of the review was to evaluate various aspects of service delivery, with the aim of identifying operational, structural and activity related problems.

Interviews with staff, combined with statistical analysis of activity data were used to identify the various types of existing norms that existed at each hospital. Many of the activities within a hospital influence other activities in the hospital. Modification of one activity can result in a flow on of consequences across a number of activities. Simulation models were used to enable the financial consequences of altering more than one activity at once to be evaluated.

We provide a framework for the categorization of problems encountered by hospital managers. Descriptions of how simulation models were used to determine the financial implications of simultaneously modifying more than one activity at once is then detailed. The use of simulation models in hospitals is then discussed in general. We highlight issues involved with using clinical information system data for use in organizational decision making.

Introduction

A broad ranging review of South Australian regional hospitals was commenced during August 1999 by the Department of Human Services. Regional hospitals are large country hospitals that provide a wide range of services, including a range of surgical and medical services, obstetric and gynaecological services, high dependency or intensive care services, as well as accident and emergency services. There are four regional hospitals in South Australia , although one sub-regional hospital, was also included in the review.

Health care providers are experiencing continued pressure to achieve increased activity levels while reducing expenditures (Duckett, 1995; Duckett, 1998; McClean and Millard, 1995; and Clerkin, Fos and Petry, 1995). Regional hospitals, like much of the Australian public health sector have also been under increased financial pressure. Over recent years some of the South Australian regional hospitals have experienced difficulty in containing the expenditure within the allocated budget. While the provision of additional funding may have resolved the financial difficulties experienced by these hospitals, such a remedy would not have necessarily represented an appropriate use of scarce resources.

The aims of the review of regional hospitals included an examination of the operations of these hospitals. In order to undertake the review it was necessary to establish a framework of how hospital operations should be examined. Statistical and simulation analyses were used to examine various aspects of hospital operations. We provide a case study of how the framework for examination of hospital operations was established, the role of simulation analysis in hospital operation research issues and the importance of data collections.

Framework for Analysing Problems

While a financial imperative existed for the review, the review was not conducted on the basis that a fixed sum of expenditure could be reduced from the annual expenditure of each hospital. Rather, the premise was that norms for various activities undertaken at each hospital existed, and that there may be different ways in achieving the same outcome. The different ways of achieving the same outcome, however, may result in different financial outcomes that are linked to expenditure.

In order to understand the financial consequences of running a hospital, it is necessary to first understand what drives the costs at the hospital. At the regional hospitals studied cost drivers were found to be nursing, medical, infrastructure, patients and activity. For each cost driver, there exists a range of cost influences. For example, while nursing is a cost driver, various factors influence the overall cost of nursing, including enterprise bargaining arrangements, the nursing hours per patient per day and the workforce profile.

To assist analyse the role of various cost influences, three types of problems were identified: structural inefficiencies, operational inefficiencies and activity problems. Structural inefficiencies are those inefficiencies arising as a consequence of the age and design of physical assets, external policy decisions or external milieu. Such inefficiencies may be outside the control of local management. Operational inefficiencies are generally those inefficiencies within the control of management. Such problems can arise wherever local management has options as to how a service may be delivered. Activity problems can arise when too much or too little activity for a given level expenditure occurs. Additionally, problems can arise when activity levels are capped as this may prevent economies of scale from being achieved. The categories are not mutually exclusive as shown in Figure 1.

Figure 1 : The framework established for the analysis of problems within hospitals. The categories are not mutually exclusive and problems may be considered to belong to one or more category of problem.

The identification of these problems is important, as hospital management may not have control over predefined activity levels or structural inefficiencies. Additionally, existing funding mechanisms may not provide adequate compensation when structural inefficiencies or activity problems occur. For example, the price paid per weighted separation may not take into account whether the predefined activity levels results in diseconomies of scale being occurring. If economies of scale cannot be achieved, the amount paid may not be adequate.

Simulation Analysis

Simple descriptive statistics of various individual activities undertaken in a hospital, such as patient admission or discharge, theatre start and finish times, the number of beds on a ward, etc can provide useful information to decision makers and resource users. Such statistics can be used to be relatively simple spreadsheet models that can be manipulated to determine the ramifications of making changes to existing operations. The limitation of using descriptive statistics and spreadsheet models, however, becomes readily apparent when dealing with a complex environment, such as a hospital, where interactions occur between various activities. For example, while changes in theatre management would be expected to have ramifications for hospital theatre staff and surgeons, the possibility of flow on effects for ward staff may not be easily anticipated. Additionally, it is difficult to create simple models that enable the measurement of outcomes of making changes to different activities within the hospital. For example, modelling the consequences of making simultaneous changes to ward sizes, theatre management and admission practices, is difficult with spreadsheets.

Simulation analysis provides another tool in the armoury of operational researchers and health managers. What is the difference between statistical analysis and simulation analysis? With statistical analysis, the relationship between the phenomenon under investigation and the model is reasonably well understood (Gilbert and Troitzsch,1999). With the statistical model, the model of the processes leading to the phenomenon is based upon equations (for example, a regression equation). Simulation modelling, however, might not involve statistical equations, but rather a computer program that represents the processes leading to the phenomenon of interest. The statistical model generates predicted data that can be compared to the collected data upon which the model has been developed. The simulated model generates simulated data that can be compared to the collected data used to establish the model. The simulated model provides a tool that enables disparate processes, for example theatre management and discharge processes, to be modelled concurrently. Such modelling is more difficult to achieve using individual statistical models linked together in a spreadsheet.

Thus, not only does simulation analysis facilitate the modelling of multiple changes to the hospital system, but it also has the potential to enable greater understanding of the environment by users. Linking behaviour with outcomes is important when suggesting that changes to existing work practices are desirable in order to make better use of scarce resources.

Methodology

Historic data was provided by each hospital. The data included:

• Patient admission summary information such as admission and discharge times and dates, patient gender, age, patient category (emergency or elective), etc
• Theatre data
• Accident and Emergency Department data
• Bed numbers for each ward, and
• Staffing arrangements for each ward.

The data was cleaned to overcome problems arising from multiple names for the same doctor. After data cleaning, the data was analyzed for trends and patterns using the SPSS V9.0 statistical package and Microsoft Excel 97 spreadsheet package. Distributions relating to particular activities or patient groups were generated for incorporation into the simulation model.

The first step in simulation analysis was to decide the level of detail or granularity the model would represent the activities in the hospital. This decision was based on the data provided by the hospitals, and the output required of the analysis. In the case of the regional hospitals, the absence of an integrated clinical information system meant that at some sites we could not track patients from Accident and Emergency (A&E) to discharge. Out attempts to model staff costs accurately was constrained by the lack of accurate data on nurse scheduling data and doctor visit times. For these reasons, we did not model hospital activity in a state transition model; instead, we created a distributional model for the financial activity in a 1-year period.

Analysis of the hospital data yields an empirical distribution. If the empirical distributions are used directly in the model this will result in overfitting. In other words, the resulting model would be an accurate representation of that particular data set, but the results of the model would have limited generalisability. To avoid overfitting, we analysed the empirical distributions for underlying patterns. In contrast to designed experiments, complex real-world data are rarely normally distributed and cannot be described simply by their mean and variance. In many cases the empirical distributions are multi-modal because we are measuring a combination of populations that we cannot differentiate due to limitations of the data. Our analysis included techniques to smooth the empirical distribution, and then model the non-parametric distributions using either mixture Gaussian distributions or polynomial functions if there were no obvious mixtures. Fitting mixtures and kernel smoothing was carried out in S-Plus software (Statistical Science, 1993).

The model was constructed in a software package called Analytica from Lumina Systems (Lumina, 1999). Our approach handles the uncertainty in the data because Analytica simulates across entire distributions in the model and returns the results with probability intervals - the Bayesian equivalent of confidence intervals. The software allows us to model a variety of probability distributions as well as logical statements and a wide range of mathematical and statistical functions. Using these facilities, Analytica allowed us to model location specific factors, such as: limitations imposed on staffing levels by the physical constraints of the hospital layout; minimum staffing requirements in particular areas of the hospital, for example the high dependency unit; and allowances for higher than expected staff levels due to mental health patients in wards that were not designed for such patients.

Results

The model developed for one hospital is illustrated in Figure 2. Underlying each node in the visual model is the distributions or basis of decision making relating to particular events in the hospital. A similar model has been developed at another hospital.

Figure 2 : Nodes of the simulation model developed to show the consequences of changing multiple norms. The simulation model enabled the analysis of simultaneously changing multiple norms, which was not easily done with less sophisticated tools such as spreadsheets.

By varying certain factors it was possible to develop the possible cost reductions that could be achieved by changing existing norms. At one hospital we used the model to analyze the following the effect of:

• Reducing the number of beds in one ward
• Reducing separation time through early ward rounds and discharge planning
• Reducing theatre over-runs on ward staff,
• Alteration of nurse staff ratio in one ward, and
• Various combinations of the above.

Individually, the potential savings arising from the four scenarios amounted to approximately $600,000 per annum. This excluded the savings attributable to theatre staff if theatre over-runs were reduced. The savings achievable if all four options were implemented, however, cannot be treated as purely additive as there is interaction between the various options. Thus, the achievable savings of the combined scenarios (excluding those attributable to theatre staff) amounted to approximately $480,000 per annum, which represented a saving of approximately 2.5% of expenditure. The identified savings did not reduce the existing level of activity.

Discussion

The framework for analysis of problems

While the framework was not necessary for the simulation modelling, it provided a context within which to structure the approach to the review for not only the reviewers, but also the management of each hospital. The categorization of problems identified during the review work enabled the discussion of the problems in a manner that should have reduced management's fear that everything would be perceived as being in their control.

The framework used in the review is portable and can be used as the basis for the review of many problems, regardless of the industry. In order to be relevant to the management of different industries, some of the categories, such as activity problems, may need modification. For example, activity problems may be more appropriately described as sales activity in a retail environment.

Simulation

The simulation analysis was employed to look at issues the reviewers believed were operational inefficiencies, that is, within the control of the local management. A large number of benefits can be seen to accrue from the use of simulation analysis, including:

• The provision of education about issues affecting the operations of the organization
• Enabling more informed debate regarding issues and the extent to which they affect the organization e.g. the implications of theatre over-runs on ward staff
• The ability to model multiple scenarios in a manner that is not readily achievable using spreadsheets, and
• The ability to attach costs to the behaviour of personnel working with the organization.

Hospital staff, including the doctors who were not employees of the regional hospitals, do not always appreciate the financial implications of their behaviour. For example, theatre over-runs had been the source of ongoing tension between various groups within one hospital. In order to try to reduce the number of occasions when the theatre operated after the designated closing time, simple statistical information had been displayed within the theatre area. Instead of resulting in the reduction of the unwanted behaviour, it exacerbated the behaviour as it became a contest among one group to see who could achieve the most over-runs. When the results of the simulation analysis were presented, however, the association between the behaviour of causing theatre over-runs and financial stress became apparent. The information from the simulation analysis resulted in some reduction in the number of occasions when the theatres continued to operate after hours and also focussed discussion on how much of the work was actually avoidable.

While the benefits of simulation analysis are potential great, there are disadvantages to using simulation analysis, including:

• Specific software must be purchased, and
• Training of staff must occur.

In regional settings, the ability to attract personnel that possess the necessary skills to undertake simulation modelling would appear to be limited. Such skills can, however, be purchased from consultants. For the purpose of the review, the local hospital management did not decide to purchase the skills required to undertake the analysis - it was provided for them. We believe that such purchases will not occur in the future unless the management of hospitals - regional or otherwise - gain an increased appreciation of the benefits of simulation modelling, and more generally the benefits of using the extensive data collections for decision making.

The role of information

One of the findings of the review has been that much data is being collected by regional hospitals, but there has been little effort by staff to convert the data into information for decision-making purposes. This is perhaps best illustrated by example. Databases have been established by hospital staff, with or without the aid of information technologist specialists, to collect data relating to specific activities within the hospitals. The data has been captured for many months or years and was analyzed for the review. We found that the quality of the data was poor. For example, if a doctor's name had been recorded as part of the data collection, the name may have been spelt in various ways. Consequently, when extracted, data cleaning was necessary before analysis of activity by doctor could occur. This would have become immediately apparent to those collecting the data had it been used.

The problem with the enormous data collections maintained by the hospitals was exacerbated by another three factors: variation in the data that was captured; the ability of staff to use the data for management purposes; and a lack of knowledge of what data was even captured. Data collections appeared to be created on the basis of one or two reasons: internal requirements or the requirement of an external agency. The number of external agencies able to influence the type of data collected appears to be large. The manner in which data is captured, however, varies considerably. Factors influencing the manner in which data was collected was the type of software used to capture the data, the willingness of staff to participate in the role of data collection and the interpretation of what data elements should be captured. This latter factor was particularly important if the data was not being used for internal management purposes. The data elements included in collections relating to operational activities were best when staff with some understanding of the activity were involved in the creation of the database. This provided some compensation when data was not used for management purposes. Our ability to gain a good understanding of some activities analyzed as part of the review was frustrated by this latter factor.

Hospitals employ a wide range of staff. Staff without training in the use of management information (as opposed to clinical information) enter the workforce in one professional capacity and as a result of promotion may attain management positions. While the staff are keen to use the data for management purposes, from the numerous interviews that were conducted as part of the review, it would appear that many lack the opportunity to obtain the necessary training or access to support staff to enable them to use the data for decision making purposes. There are short and long term solutions to this problem. The short-term solutions involve providing additional training or support to staff, while the long term solutions include amending the training programs of the professions. The implementation of these solutions can be made more difficult when organizations are under financial stress or located outside of metropolitan centres. It is at times of financial stress when the possession of such skills by staff is important, as the identification of potential solutions to financial problems can sometimes only be found when activities can be appropriately analyzed. Until this situation is remedied, it is likely that problems with data collections and data usage will continue.

The finding that managers were not aware of all the various data collections with the hospitals is not necessarily surprising, given the manner in which some of the data collections have been established. Davenport (1997) suggests that many organizations have not faired well in the management of information. According to Davenport , most organizations have achieved applying technology to information problems and attempted to use machine-engineering methods to convert data into something of use on a computer. The implementation of an appropriate knowledge management framework for managing the resources within the various hospitals may address this problem. Davenport suggests the adoption of an information ecology as the basis for managing the continuum of data to knowledge, and the adoption of this in the health sector may be appropriate.

Certainly, for simulation analysis to become more widespread, it is first necessary that hospital staff develop the necessary understanding of what data and information are and how they can be of value in the decision making process. This will only occur if hospitals begin to develop knowledge management frameworks, which recognize that staff require the appropriate support, be it via education or the provision of specialist support or both, to achieve this outcome.

The simulation model we used to analyse this complex data allowed us to model the patterns we found in the empirical data. Selective parameterisation of factors in the model enabled us to play "what if" scenarios with regards to potential interventions into the running of the hospitals. The power of this approach arises from two factors:

• For the first time we were able to attach dollar values to patterns in workflow that highlights clearly the magnitude of particular practices.
• We used local data and as much as possible modeled local factors to make the results of the analysis relevant to the stakeholders.

Conclusion

We have found that the analysis of hospital financial difficulties can be more easily performed if a suitable framework is adopted and more sophisticated modelling tools are employed. Given the expected ongoing financial difficulties that the health sector is likely to face, it is imperative that the adoption of such frameworks and modelling tools become widespread. The framework we have used provided a mechanism that enabled the identification of whether a problem is within the control of local management and also provided a focus on service delivery rather than on the direct financial implications of the service.

Simulation analysis provided us with a means by which the effects of changing multiple activities could be analyzed simultaneously. Such analysis can provide management with a means of attaching the financial implications of how resources are used and overcomes the shortfalls of simple spreadsheet models. Consequently, this is a powerful tool, which is suitable for assisting in the change management process. However, in order to be able to undertake such modelling, it is imperative that the data collections are properly maintained.

References

• Clerkin, D, Fos PJ and Petry, FE. (1995) A decision support system for hospital bed assignment. Hospital and Health Services Administration , 40(3), 386-400.

• Davenport , TH. (1997) Information Ecology - Mastering the Information and Knowledge Environment. New York : Oxford University Press.

• Duckett, SJ. (1995) Hospital payment arrangements to encourage efficiency: the case of Victoria , Australia . Health Policy , 34, 113-134.

• Duckett SJ. (1998) Casemix funding for acute hospital inpatient services in Australia . Medical Journal of Australia , 169, Supplement, S17-S21.

• Gilbert, N and Troitzsch, KG. (1999). Simulation for the social scientist. Buckingham: Open University Press.

• McClean, S I and Millard P H (1995a). Modelling in-patient bed usage behaviour in a department of geriatric medicine. Methods of Information in Medicine , vol 32, pp 79-81.

• Statistical Sciences (1993) S-Plus Guide to Statistical and Mathematical Analysis , Seattle: StatSci, a division of MathSoft, Inc. http://www.mathsoft.com .

• Lumina Decision Systems (1999) Analytica User's Guide. Lumina Decision Systems Inc. http://www.lumina.com/ .

Home > Research > Health Systems Analysis

© 2004 The University of Adelaide Last Modified Wednesday, 28-Jun-2006 11:32:05 CST  Health Informatics	Copyright | Privacy | Disclaimer