On 2014 Jun 27, Lauren F Laker commented:

Thanks for feedback and interesting questions. In response to your comments regarding validation, Figure 3 illustrates the arrivals for the real-world system which was approximated in our simulation model. It was replicated by the simulation within 0.5% of the empirical data. Unfortunately, the decision to release actual operational data would require the approval of senior hospital administration, so we can't do that at this time. While we could have discussed validation more extensively, the manuscript length was a concern and the primary goal of the paper was to compare flexibility policies to each other rather than to a real-world benchmark. As for implementation, these results influenced part of a redesign of the UCMC ED's patient intake area, which has been modified to explicitly include some flexible beds that can be shared between the fast-track area and the regular ED. That project is ongoing and construction is expected to begin soon. Thanks again for your interest in the paper.

On 2014 Jun 24, T Eugene Day commented:

This is a really cool model, and the concept of a flex-bed in the fast track area is intriguing. Using simulation to determine the preferred number of bed under varying assumptions is an ideal use of the technology.

Triage time was estimated from conversations with providers. This is sub-ideal (but common practice - preferable would be real-world observations), but they found that the model was robust to changes in triage time, justifying the method. My main concern is that though the authors assert that the model was validated against real world data, that data isn't presented or described. No model perfectly tracks the real world system, and having that data available would provide boundaries on how well we'd expect the model results to translate back to the real world. Similarly, it is not always clear (for example for figure 3) when they are discussing simulated or real-world data.

With the caveat that I'd like to see the baseline real-world to simulated results presented clearly, this seems like a strong paper with a useful result. Up next: are there plans to test this solution in the real world? And how well do such tests compare with predicted results? How much has the demography of the patient population changed from the AY2011 data set to the current population? etc.

On 2014 Jun 24, T Eugene Day commented:

This is a really cool model, and the concept of a flex-bed in the fast track area is intriguing. Using simulation to determine the preferred number of bed under varying assumptions is an ideal use of the technology.

Triage time was estimated from conversations with providers. This is sub-ideal (but common practice - preferable would be real-world observations), but they found that the model was robust to changes in triage time, justifying the method. My main concern is that though the authors assert that the model was validated against real world data, that data isn't presented or described. No model perfectly tracks the real world system, and having that data available would provide boundaries on how well we'd expect the model results to translate back to the real world. Similarly, it is not always clear (for example for figure 3) when they are discussing simulated or real-world data.

With the caveat that I'd like to see the baseline real-world to simulated results presented clearly, this seems like a strong paper with a useful result. Up next: are there plans to test this solution in the real world? And how well do such tests compare with predicted results? How much has the demography of the patient population changed from the AY2011 data set to the current population? etc.

On 2014 Jun 27, Lauren F Laker commented:

Thanks for feedback and interesting questions. In response to your comments regarding validation, Figure 3 illustrates the arrivals for the real-world system which was approximated in our simulation model. It was replicated by the simulation within 0.5% of the empirical data. Unfortunately, the decision to release actual operational data would require the approval of senior hospital administration, so we can't do that at this time. While we could have discussed validation more extensively, the manuscript length was a concern and the primary goal of the paper was to compare flexibility policies to each other rather than to a real-world benchmark. As for implementation, these results influenced part of a redesign of the UCMC ED's patient intake area, which has been modified to explicitly include some flexible beds that can be shared between the fast-track area and the regular ED. That project is ongoing and construction is expected to begin soon. Thanks again for your interest in the paper.