^{1}

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.

Our study investigates different models to forecast the total number of next-day discharges from an open ward having no real-time clinical data.

We compared 5 popular regression algorithms to model total next-day discharges: (1) autoregressive integrated moving average (ARIMA), (2) the autoregressive moving average with exogenous variables (ARMAX), (3) k-nearest neighbor regression, (4) random forest regression, and (5) support vector regression. Although the autoregressive integrated moving average model relied on past 3-month discharges, nearest neighbor forecasting used median of similar discharges in the past in estimating next-day discharge. In addition, the ARMAX model used the day of the week and number of patients currently in ward as exogenous variables. For the random forest and support vector regression models, we designed a predictor set of 20 patient features and 88 ward-level features.

Our data consisted of 12,141 patient visits over 1826 days. Forecasting quality was measured using mean forecast error, mean absolute error, symmetric mean absolute percentage error, and root mean square error. When compared with a moving average prediction model, all 5 models demonstrated superior performance with the random forests achieving 22.7% improvement in mean absolute error, for all days in the year 2014.

In the absence of clinical information, our study recommends using patient-level and ward-level data in predicting next-day discharges. Random forest and support vector regression models are able to use all available features from such data, resulting in superior performance over traditional autoregressive methods. An intelligent estimate of available beds in wards plays a crucial role in relieving access block in emergency departments.

Demand for health care services has become unsustainable [

Daily discharge rate can be a potential real-time indicator of operational efficiency [

Ward-level discharges incorporate far greater hospital dynamics that are often nonlinear [

The current practice of bed allocation in general wards of most hospitals involves a hospital staff/team, who use past information and experience, to schedule and assign beds [

Motivated by this result, we address the open problem of forecasting daily discharges from a ward with no real-time clinical data. Specifically, we compare the forecasting performance of 5 popular regression models: (1) the classical autoregressive integrated moving average (ARIMA), (2) the autoregressive moving average with exogenous variables (ARMAX), (3) k-nearest neighbor (kNN) regression, (4) random forest (RF) regression, and (v) support vector regression (SVR). Our experiments were conducted on commonly available data from a recovery ward (heath wing 5) in Barwon Health, a regional hospital in Victoria, Australia. The ARIMA and kNN models are built from daily discharges from ward. To account for the seasonal nature of discharges, the ARMAX model included day of the week and ward occupancy statistics. We identified and constructed 20 ward-level and 88 patient-level predictors to derive the RF and SVR models.

Forecasting accuracy was measured using 3 metrics on a held out set of 2511 patient visits in the year 2014. When compared with a naive forecasting method of using the mean of last week discharges, we demonstrate through our experiments that (1) using regression methods for forecasting discharge outperforms naive forecasting, (2) SVR and RF models outperform the autoregressive methods and kNN, (3) an RF model derived from 108 features has the minimum error for next-day forecasts.

The significance of our study is in identifying the importance of foreseeing available beds in wards, which could help relieve emergency access block [

Patient length of stay directly contributes to hospital costs and resource allocation. Long-term forecasting in health care aims to model bed and staffing needs over a period of months to years. Cote and Tucker categorize the common methods in health care demand forecasting as percent adjustment, 12-month moving average, trendline, and seasonalized forecast [

On the other hand, our work implements short-term forecasting. The short-term forecasting methods are concerned with hourly and daily forecasts from a single unit in a care environment. The most popular unit of interest is the emergency or acute care department because this is often a key performance indicator metric in assessing quality of care [

Decision tree modeling of total discharges from an open ward from day of the week and ward occupancy (previous day occupation) data for 5 years. The leaves represent total number of patient discharges.

When looking at discharges as time series, autoregressive moving average models are the most popular [

Jones et al used the classical ARIMA to forecast daily bed occupancy in emergency department of a European hospital [

Jones et al [

Modeling using simulation is typically used to study the behavior of complex systems. An early work in investigated the effects of emergency admissions on daily bed requirements in acute care, using discrete-event stochastic simulation modeling [

Regression models analyze the relationship between the forecasted variable and features in the data. Linear regression that encoded monthly variations was used to forecast patient admissions over a 6-month horizon and outperformed quadratic and autoregressive models [

Barnes et al used 10 predictors to model real-time inpatient length of stay in a 36-bed unit using an RF model [

Nonlinear regression is better suited to model the changing dynamics of patient flow. To characterize the outflow of patients from the ward, we resort to regression using RF, kNN, and SVR. In the area of pattern recognition, kNNs [

Another powerful and popular regression technique, SVR, uses kernel functions to map features into a higher dimensional space to perform linear regression. Though this technique has not seen much application in medical forecasting, support vector machines have been successful in financial market prediction, electricity forecasting, business forecasting, and reliability forecasting [

Apart from the standard autoregressive methods, we use kNN, RFs, and SVR in forecasting next-day discharges. Because discharge patterns repeat over time, kNN regression can be applied to search for a matching pattern from past discharges. RFs and SVR regression are powerful modelling techniques requiring minimum tuning to effectively handle nonlinearity in the hospital processes.

Recently, RF forecasting was used to predict total patient discharges from a 36 bed unit in an urban hospital [

The absence of real-time clinical information in our data makes calculating patient length of stay impossible. Instead, we resort to modelling next-day discharges by observing previous discharge patterns and examining demographics and flow characteristics in the ward.

Our study used retrospective data collected from a recovery ward in Barwon Health, a large public health provider in Victoria, Australia serving about 350,000 residents. Ethics approval was obtained from the Hospital and Research Ethics Committee at Barwon Health (number 12/83) and Deakin University. The total number of available beds depended on the number of staff assigned to the ward. On average, the ward had 36 staffed beds, but fluctuated between 20 and 80 beds with varying patient flow. The physicians in the ward had no teaching responsibilities.

Tables in hospital database used in our data collection.

Tables | Columns |

Patients | 1. Patient ID |

2. Age | |

3. Gender | |

Ward Stay | 1. Admission ID |

2. Name of the ward | |

3. Time (entry, exit) | |

4. Bed ID | |

Admissions | 1. Patient ID |

2. Admission ID | |

3. Time (admit, discharge) | |

4. Patient Class (21 categories) | |

5. Admission type (7 categories) |

Cohort details.

Cohort | Stats |

Total patient visits | 12,141 |

Unique patients | 10,610 |

Length of stay (mean, median, IQR^{a}) |
4.26, 3, 5 |

Discharges per day (mean, median, IQR) | 8.7, 8, 5 |

Admissions per day (mean, median, IQR) | 8.6, 8, 5 |

Mean ward occupancy, IQR | 30.9, 4 |

Gender | 54.8% Female |

Age (mean, median) | 66, 63.23 |

^{a}IQR, interquartile range.

The data for our study came from three tables in the hospital database, as shown in

A time series decomposition of our data revealed strong seasonal variations and high nonlinearity in daily discharge patterns. There was a defined weekly pattern–discharge from ward peaked on Fridays and dropped significantly on weekends (see

We describe the following diverse methods that are applicable to forecasting under complex data dynamics: (1) ARIMA, (2) autoregressive moving, (3) forecasting using kNN discharge patterns, (4) RF, and (5) SVR. Autoregressive methods model the temporal linear correlation between nearby data points in the time series. Nearest patterns lift this linearity assumption and assumes that short periods form repeated patterns. Finally, RF and SVR look for a nonlinear functional relationship between the future outcomes and descriptors in the past.

Mean admissions and discharges per day from ward.

Time series of monthly discharges from ward.

Time-series forecasting methods can analyze the pattern of past discharges and formulate a forecasting model from underlying temporal relationships [_{t}, as a linear combination of previous discharges. On the other hand, moving averages models characterize as linear combination of previous forecast errors. For ARIMA model, the discharge time series is made stationary using differencing. Let

Classical ARIMA model.

Dynamic regression techniques allow adding additional explanatory variables, like day of the week and number of current patients in the ward, to autoregressive models. The autoregressive moving ARMAX modifies ARIMA model by including depending external variable _{t} at time _{t} using features from the hospital database.

ARIMA model with exogenous variable xt.

The kNN algorithm takes advantage of the locality in data space. We assume that the next-day discharge depends on the discharges happening in previous days. Using kNN principles, we can do a regression to forecast the next-day discharge. Let _{d} represent number of discharges on the current day: _{d+1}, we look at the discharges over the past _{d-p}: _{d}]. Using Euclidean distance metric, we find _{d+1}, is calculated as a measure of the next-day discharges of the _{match})_{i}_{d-7}: _{d}] results in 3 matches from the training data. For simplicity, we have plotted the matched patterns alongside disch_vec, although they had occurred in the past. The next-day forecast _{d+1} becomes a measure of (_{match})_{i}, where (_{match})_{i}^{th}term of each of the matched patterns [

One popular method of calculating _{d+1} is by minimizing the weighted quadratic loss (_{i} takes values between 0 and 1, with ∑^{k}_{i=1}_{i}_{.} However, there are 2 main drawbacks making it less desirable for our data. First, the quadratic loss is sensitive to outliers. Second, a robust estimate of { _{i}} becomes difficult.

Our data contain significant noise, causing large variations in next-day forecasts of the _{t+1} by minimizing the robust loss (

k-nearest neighbor forecasting example with k=3 and

Calculating ŷd+1 by minimizing the weighted quadratic loss.

Scatterplot of next-day forecast using k-nearest neighbor for a given day. X-axis represents each matched nearest-neighbor pattern. Y-axis represents the next day forecast of that matched pattern.

Estimating ŷt+1 by minimizing the robust loss.

In this approach, we assume the next-day discharge as a function of historical descriptor vector: _{p}, the function is approximated as shown in _{p}| is the number of data point falling in region _{p}_{p}

The voting leads to great benefits: reduce the variations per tree. The randomness helps combat against overfitting. There is no assumption about the distribution of data or the form of the function (

Random forests formulation of next day discharges (y) from historical descriptors (x).

The historical descriptor vector _{1}, _{1}), (_{2}, _{2}), … (_{n}, _{n})}, where each _{i}ϵ ^{m} denotes the input descriptor for the corresponding next day forecast _{i}ϵ ^{1}, a regression function takes the form: _{i}= _{i}). SVR works by (1) mapping the input space of _{i} into a higher dimensional space using a nonlinear mapping function: ^{m} is the weights and ^{1}is the bias term. Vapnik [_{ϵ} tolerates errors that are smaller than the threshold:

In our work, we use an RBF kernel [

The SVR learning model.

We extracted all data from the database tables (as in

Features constructed from ward data in hospital database.^{a}

Type | Predictor | Description |

Ward-level | Seasonality | Current day-of-week, current month |

Trend | Calculated using locally weighted polynomial regression from past discharges on the same weekday | |

Admissions | Number of admissions during past 7 days | |

Discharges | Number of discharges during past 7 days, number of discharges in previous 14th day and 21st day | |

Occupancy | Ward occupancy in previous day | |

Patient-level | Admission type | 5 categories |

Patient referral | 49 categories | |

Patient class | 21 categories | |

Age category | 8 categories | |

Number of wards visited | 4 categories | |

Elapsed length of stay | Calculated daily for each patient in the ward |

^{a} The random forest and support vector regression models used the full set of features. The ARMAX (autoregressive moving average with exogenous variables) model used seasonality and occupancy. All other models were derived from daily discharges.

An example of the discharge trend, as derived from a locally weighted polynomial regression model.

Our training and testing sets are separated by time. This strategy reflects the common practice of training the model using data in the past and applying it on future data. Training data consisted of 1460 days from January 1, 2010, to December 31, 2013. Testing data consisted of 365 days in the year 2014. The characteristics of the training and validation cohort are shown in

Characteristics of training and validation cohorts.

Categorization | Training (2010-2013) | Testing (2014) | |

Total days | 1460 | 365 | |

Mean discharges per day | 8.47 | 9.17 | |

Number of admissions | 9630 | 2511 | |

Gender | |||

Male | 4329 (44.9%) | 1135 (45.2%) | |

Female | 5301 (55.1%) | 1376 (54.8%) | |

Mean age (years) | 63.65 | 61.62 | |

Length of stay | |||

1-4 days | 6377 (66.22%) | 1636 (65.15%) | |

5 or more days | 3253 (33.78%) | 875 (34.85%) |

The current hospital strategy involves using past experience to foresee available beds. To compare the efficiency of our proposed approaches, we model the following baselines: (1) Naive forecasting using the last day of week discharge: since our data were found to have defined weekly patterns, we model the next day discharge as the number of discharges for the same day during previous week; (2) naive forecasting using mean of last week discharges: to better model the variation and noise in weekly discharges, we model the next-day discharge as the mean of discharges during previous 7 days; and (3) naive forecasting using mean of last 3-week discharges: to account for the monthly and weekly variations in our data, we use mean of daily discharges over the past 3 weeks to model the next-day discharge.

We compare the next-day forecasts of our proposed approaches with the baseline methods on the measures of mean forecast error, mean absolute error, symmetric mean absolute percentage error, and root mean square error [_{t} is the measured discharge at time _{t} is the forecasted dishcharge at time

• Mean forecast error (MFE): is used to gauge model bias and is calculated as MFE = mean(_{t}- _{t})

• For an ideal model, MFE = 0. If MFE > 0, the model tends to underforecast. When MFE < 0, the model tends to overforecast.

• Mean absolute error (MAE): is the average of unsigned errors: MAE = mean| _{t}- _{t}|.

MAE indicates the absolute size of the errors.

• Root mean square error (RMSE) is a measure of the deviation of forecast errors. It is calculated as: RMSE = √mean(_{t}- _{t}^{2}

Due to squaring and averaging, large errors tend to have more influence over RMSE. In contrast, individual errors are weighted equally in MAE. There has been much debate on the choice of MAE or RMSE as an indicator of model performance [

•Symmetric mean absolute percentage error (sMAPE): It is scale independent and hence can be used to compare forecast performance between different data series. It overcomes 2 disadvantages of mean absolute percentage error (MAPE) namely, (1) the inability to calculate error when the true discharge is zero and (2) heavier penalties for positive errors than negative errors. sMAPE is a more robust estimate of forecast error and is calculated as: sMAPE = mean(200[| _{t}- _{t}|/ _{t}+ _{t}]). However, sMAPE ranges from −200% to 200%, giving it an ambiguous interpretation [

In this section, we describe the results of comparing our different forecasting methods. The model parameters for kNN forecast, RF, and SVR models were tuned to minimize forecast errors.

For kNN regression, the optimum value of pattern length:

The SVR parameters

We compared the naive forecasting methods with our proposed 5 models using MFE, MAE, RMSE, and sMAPE. The results are summarized in

Forecast accuracy of different models.

Model | Mean forecast error | Mean absolute error | Symmetric mean |
Root mean square error | Mean absolute error improve over naïve | |

Naive forecast | ||||||

Using discharge from last |
0.03 | 3.81 | 45.70 % | 4.95 | ||

Using mean of last week |
0.02 | 3.57 | 41.68 % | 4.42 | ||

Using mean of last 3-week |
0.04 | 3.44 | 40.14% | 4.34 | ||

ARIMA^{a} |
0.06 | 3.27 | 38.32 % | 4.15 | 4.9 % | |

ARMAX^{b} |
-0.01 | 2.99 | 34.86 % | 3.84 | 13.1 % | |

k-nearest neighbor | 1.09 | 2.88 | 34.92 % | 3.77 | 16.3 % | |

Support vector regression | 0.73 | 2.75 | 32.88% | 3.64 | 20.1 % | |

Random forest | 0.44 | 2.66 | 31.86 % | 3.49 | 22.7 % |

^{a} ARIMA: autoregressive integrated moving average

^{b} ARMAX: autoregressive moving average with exogenous variables

The naive forecasts are unable to capture all variations in the data and resulted in the maximum error when compared with other models.

The variations in seasonality and trend are better captured in ARIMA and ARMAX models. The time series consisting of past 3-month discharges were used to generate the next-day discharge forecast. The ARMAX model also included the day of week and ward occupancy as exogenous variables, which resulted in better forecast performance over ARIMA.

Interestingly, kNN was more successful than ARIMA and ARMAX in capturing the variations in discharge, demonstrating about 3% improvement in MAE, when compared with ARMAX. However, the kNN model tends to under forecast (MFE = 1.09), possibly because of resorting to median values for forecast. In comparison, RF and SVR forecast models demonstrated better performance. This can be expected because they are derived from all the 108 features. However, RF demonstrated a relative improvement of 3.3 % in MAE over SVR model (see

The process of SVR with RBF kernel maps all data into a higher dimensional space. Hence, the original features responsible for forecast cannot be recovered, and the model acts as a black box. Alternatively, RF algorithm returns an estimate of importance for each variable for regression. Examining the features with high importance could give us a better understanding of the discharge process.

Comparison of actual and forecasted discharges from ward for each day in 2014.

Forecast error in predicting each day of week in 2014.

The features in random forecast model were ranked on importance scores. The top 10 significant features are described as follows. The day of week for the forecast proved to be the most important feature. Other features were number of patients in the ward during the day of forecast, the trend of discharges measured using locally weighted polynomial regression, number of discharges in past 14th day, number of discharges in past 21st day, number of patients who had visited only one previous ward, the number of males in the ward, number of patients labelled as: “public standard,” and current month of forecast.

Improved patient flow and efficient bed management is key to counter escalating service and economic pressures in hospitals. Predicting next-day discharges is crucial but has been seldom studied for general wards. When compared with emergency and acute care wards, predicting next-day discharges from a general ward is more challenging because of the nonavailability of real-time clinical information. The daily discharge pattern is seasonal and irregular. This could be attributed to management of hospital processes such as ward rounds, inpatient tests, and medication. The nonlinear nature of these processes contributes to unpredictable length of stay even in patients with similar diagnosis.

Typically, for open wards, a floor manager uses previous experience to foresee the number of available beds. In this paper, we attempt to model total number of next-day discharges using 5 methods. We have compared the forecasting performance using MAE, RMSE, and sMAPE. Our predictors are extracted from commonly available data in the hospital database. Although the kNN method is simple to implement, requiring no special expertise, software packages for other models are available for all common platforms. These models can be implemented by the analytics staff in hospital IT department and can be easily integrated into existing health information systems.

In our experiments, forecast based on RF model outperformed all other models. Forecasting error rate is 31.9% (as measured by sMAPE) which is in the same ballpark as the recent work of [

The kNN regression also performed well as it assumes only the locality in the data. But it is not adaptive, and thus less flexible in capturing complex patterns. The kNN regression assumes similar patterns in past discharges extrapolate to similar future discharge, which is not true for daily discharges from ward. ARMAX model outperformed the traditional ARIMA forecasts since it incorporated seasonal information as external regressors. As expected, a naive forecast of using the median of past discharges performed worst.

We noticed a weekly pattern (

Feature importance score from an RF model helps in identifying the features contributing to the regression process. The day of forecast proved to be one of the most important features in the RF model. Other important features included trend based on nonlinear regression of past weekdays, number of discharges in the past days, ward occupancy in previous day, number of males in the ward, and number of general patients in ward.

When looking at for each day of the week, the RF and SVR model consistently outperformed other models. Sundays and Thursdays proved to be the easiest to predict for all models (

Patient length of stay is inherently variable, partly due to the complex nonlinear structure of medical care [

In our study, we were able to validate that the weekend patterns affect discharges from a general ward. The RF model was able to give a reasonable estimate of number of next-day discharges from the ward. Clinical staff can use this information as an aid to decisions regarding staffing and resource utilization. This foresight can also aid discharge planning such as communication and patient transfer between wards or between hospitals.

An estimate of number of free beds can also help reduce emergency department (ED) boarding time and improve patient flow [

We acknowledge the following limitations in our study. First, we focused only on a single ward. However, it was a ward with different patient types, and hence the results could be an indication for all general wards. Second, we did not use patient clinical data to model discharges. This was because clinical diagnosis data were available only for 42.81% of patients who came from emergency. In a general ward, clinical coding is not done in real time. However, we believe that incorporating clinical information to model patient length of stay could improve forecasting performance. Third, we did not compare our forecasts with clinicians/managing nurses. Finally, our study is retrospective. However, we have selected prediction period separated from development period. This has eliminated possible leakage and optimism.

This study set out to model patient outflow from an open ward with no real-time clinical information. We have demonstrated that using patient-level and ward-level features in modelling forecasts outperforms the traditional autoregressive methods. Our proposed models are built from commonly available data and hence could be easily extended to other wards. By supplementing patient-level clinical information when available, we believe that the forecasting accuracy of our models can be further improved.

autoregressive intensive moving average

autoregressive moving average with exogenous variables

emergency department

k-nearest neighbor

mean absolute error

mean absolute percentage error

mean forecast error

random forest

root mean square error

symmetric mean absolute percentage error

support vector regression

The authors would like to thank the anonymous reviewers for their comments and suggestions which greatly improved the quality of the paper. This work is partially supported by the Telstra-Deakin Centre of Excellence in Big Data and Machine Learning.

None declared.