class CustomResource(simpy.Resource):
def __init__(self, env, capacity, id_attribute=None):
super().__init__(env, capacity)
self.id_attribute = id_attribute
def request(self, *args, **kwargs):
# Add logic to handle the ID attribute when a request is made
return super().request(*args, **kwargs)
def release(self, *args, **kwargs):
# Add logic to handle the ID attribute when a release is made
return super().release(*args, **kwargs)Introduction
Visual display of the outputs of discrete event simulations in simpy have been identified as one of the limitations of simpy, potentially hindering adoption of FOSS simulation in comparison to commercial modelling offerings or GUI FOSS alternatives such as JaamSim.
When compared to commercial DES software packages that are commonly used in health research, such as Simul8, or AnyLogic, a limitation of our approach is that we do not display a dynamic patient pathway or queuing network that updates as the model runs a single replication. This is termed Visual Interactive Simulation (VIS) and can help users understand where process problems and delays occur in a patient pathway; albeit with the caveat that single replications can be outliers. A potential FOSS solution compatible with a browser-based app could use a Python package that can represent a queuing network, such as NetworkX, and displaying results via matplotlib. If sophisticated VIS is essential for a FOSS model then researchers may need to look outside of web apps; for example, salabim provides a powerful FOSS solution for custom animation of DES models. - Monks T and Harper A. Improving the usability of open health service delivery simulation models using Python and web apps [version 2; peer review: 3 approved]. NIHR Open Res 2023, 3:48 (https://doi.org/10.3310/nihropenres.13467.2)
This package allows visually appealing, flexible visualisations of the movement of entities through some kind of pathway.
It is primarily tested with discrete event simulations to be created from SimPy models, though has also been tested with the Ciw package.
Plotly is leveraged to create the final animation, meaning that users can benefit from the ability to further customise or extend the plotly plot, as well as easily integrating with web frameworks such as Streamlit, Dash or Shiny for Python.
The code has been designed to be flexible and could potentially be used with alternative simulation packages such as simmeR or Salabim if it is possible to provide all of the required details in the logs that are output.
Examples
To develop and demonstrate the concept, it has so far been used to incorporate visualisation into several existing simpy models that were not initially designed with this sort of visualisation in mind: - a minor injuries unit, showing the utility of the model at high resolutions with branching pathways and the ability to add in a custom background to clearly demarcate process steps
- an elective surgical pathway (with a focus on cancelled theatre slots due to bed unavailability in recovery areas), with length of stay displayed as well as additional text and graphical data
- a community mental health assessment pathway, showing the wait to an appointment as well as highlighting ‘urgent’ patients with a different icon and showing the time from referral to appointment below the patient icons when they attend the appointment.
- a community mental health assessment pathway with pooling of clinics, showing the ‘home’ clinic for clients via icon so the balance between ‘home’ and ‘other’ clients can be explored.
- a community mental health assessment and treatment pathway, showing the movement of clients between a wait list, a booking list, and returning for repeat appointments over a period of time while sitting on a caseload in between.
Usage Instructions with SimPy
Test, test, and test again!
Before you start trying to incorporate vidigi into your model, make sure you take a backup of your model as it currently is.
While vidigi has been tested to ensure that it’s special resource classes work the same as existing simpy resource classes, it’s still possible to accidentally change your model. There’s also a chance that the vidigi classes don’t work identically to simpy classes in more complex scenarios with reneging, baulking, or other conditional logic around resource allocation.
Therefore, it’s highly advisable to check the key output metrics from your model before and after incorporating vidigi!
Creating a visualisation from an existing model
Two key things need to happen to existing models to work with the visualisation code: 1. All simpy resources need to be changed to simpy stores containing a custom resource with an ID attribute. Vidigi provides two helper classes - VidigiStore and VidigiPriorityStore - to support with this. 2. Logging needs to be added at key points: arrival, (queueing, resource use start, resource use end), departure where the steps in the middle can be repeated for as many queues and resource types as required, with the minimum requirement being at least one of each of arrival, queueing, departure.
1. All simpy resources need to be changed to simpy stores containing a custom resource with an ID attribute
To allow the use of resources to be visualised correctly - with entities staying with the same resource throughout the time they are using it - it is essential to be able to identify and track individual resources.
By default, this is not possible with Simpy resources. They have no ID attribute or similar.
The easiest workaround which drops fairly painlessly into existing models is to use a simpy store with a custom resource class.
The custom resource is setup as follows:
The creation of simpy resources is then replaced with the following pattern:
beds = simpy.Store(environment)
for i in range(number_of_beds):
beds.put(
CustomResource(
environment,
capacity=1,
id_attribute=i+1)
)vidigi.resources provides a helper function for setting up simpy resources in the required manner.
For a given resource that would have been created like this:
nurses = simpy.Resource(simpy_environment, capacity=number_of_nurses)You would use
from vidigi.resources import VidigiStore
nurses = VidigiStore(simpy_environment, num_resources=number_of_nurses)While you are now using a simpy store, VidigiStore and VidigiPriorityStore have a number of helper methods to allow you to continue to use
This becomes slightly more complex with conditional requesting (for example, where a resource request is made but if it cannot be fulfilled in time, the requester will renege). This is covered to some extent in some of the provided examples, but further demonstrations of this are planned.
The benefit of this is that when we are logging, we can use the .id_attribute attribute of the custom resource to record the resource that was in use. This can have wider benefits for monitoring individual resource utilisation within your model as well.
2. Logging needs to be added at key points
The animation function needs to be passed an event log with the following layout:
| patient | pathway | event_type | event | time | resource_id |
|---|---|---|---|---|---|
| 15 | Primary | arrival_departure | arrival | 1.22 | |
| 15 | Primary | queue | enter_queue_for_bed | 1.35 | |
| 27 | Revision | arrival_departure | arrival | 1.47 | |
| 27 | Revision | queue | enter_queue_for_bed | 1.58 | |
| 12 | Primary | resource_use_end | post_surgery_stay_ends | 1.9 | 4 |
| 15 | Revision | resource_use | post_survery_stay_begins | 1.9 | 4 |
One easy way to achieve this is by appending dictionaries to a list at each important point in the process. For example:
event_log = []
...
...
event_log.append(
{'patient': id,
'pathway': 'Revision',
'event_type': 'resource_use',
'event': 'post_surgery_stay_begins',
'time': simpy_environment.now,
'resource_id': bed.id_attribute}
)The list of dictionaries could then be converted to a pandas dataframe using
pd.DataFrame(event_log)and passed to the animation function where required.
However, from vidigi 1.0.0, you can use the EventLogger helper class instead! This will ensure the expected data is available throughout.
from vidigi.logging import EventLogger
logger = EventLogger(
# Pass in the simulation environment if using simpy
# so that the current sim time will be used when logging
env=simpy_environment,
# Optionally, pass in a run number (when running multiple iterations of your simulation,
# ensuring you can tell logs from different runs apart easily)
run_number=run_number
)We will access these helpers like so:
logger.log_arrival()logger.log_queue()logger.log_resource_use_start()logger.log_resource_use_end()logger.log_departure()
These are all of the key steps you are likely to need to log in a standard model
- when people arrive
- when they begin waiting for something to happen
- when they are using a resource
- when they finish using that resource
- when they leave
You can have multiple instances of queues and resource use within your logs per entity.
However, each entity should have only one arrival and one departure.
For arrivals and departures, only the entity ID - e.g. a patient or customer identifier - needs to be passed in.
For queues, we also need to provide an event name to the event parameter to identify the step later on.
For resource use (both start and end), we need to provide an event name to the event parameter, and also provide a resource_id so that we are tracking which resource is in use when - which is why we needed to make the change to use the VidigiStore earlier.
Event types
arrival_departure
Within this, a minimum of two ‘arrival_departure’ events per entity are mandatory - arrival and depart, both with an event_type of arrival_departure, as shown below.
logger.log_arrival(
entity_id=patient.identifier
)logger.log_departure(
entity_id=patient.identifier
)These are critical as they are used to determine when patients should first and last appear in the model. Forgetting to include a departure step for all types of patients can lead to slow model performance as the size of the event logs for individual moments will continue to increase indefinitely.
Click to see how we’d do this with a dictionary instead
event_log.append(
{'patient': unique_entity_identifier,
'pathway': 'Revision',
'event_type': 'arrival_departure',
'event': 'arrival',
'time': env.now}
)event_log.append(
{'patient': unique_entity_identifier,
'pathway': 'Revision',
'event_type': 'arrival_departure',
'event': 'depart',
'time': env.now}
)queue
Queues are key steps in the model.
It is possible to solely use queues and never make use of a simpy resource.
By tracking each important step in the process as a ‘queue’ step, the movement of patients can be accurately tracked.
Entities will be ordered by the point at which they are added to the queue, with the first entries appearing at the front (bottom-right) of the queue.
logger.log_queue(
entity_id=patient.identifier,
event="treatment_wait_begins"
)While the keys shown above are mandatory, you can add as many additional keys to a step’s log as desired. This can allow you to flexibly make use of the event log for other purposes as well as the animation.
Click to see how we’d do this with a dictionary instead
event_log.append(
{'patient': unique_entity_identifier,
'pathway': 'High intensity',
'event_type': 'queue',
'event': 'appointment_booked_waiting',
'time': self.env.now
}
)resource_use and resource_use_end
Resource use is more complex to include but comes with two key benefits over the queue: - it becomes easier to monitor the length of time a resource is in use by a single entity as users won’t ‘move through’ the resource use stage (which can also prove confusing to less experienced viewers) - it becomes possible to show the total number of resources that are available, making it easier to understand how well resources are being utilised at different stages
with treatment_cubicles.request() as req:
# Make sure we assign the result of the yield to a variable
# Assuming we are using a VidigiStore or VidigiPriorityStore, this will allow us
# to access the useful ID attribute of the returned resource
treatment_cubicle = yield req
# As we've waited for a resource to become available
# with the `yield req`, we can now record
# that the user's resource use is starting
logger.log_resource_use_start(
entity_id=patient.identifier,
event="treatment_begins",
resource_id=treatment_cubicle.id_attribute
) #<<
yield self.env.timeout(1) # some amount of time
# Now that we have waited for the patient to be seen,
# we can log that their use of the resource has ended
self.logger.log_resource_use_end(
entity_id=patient.identifier,
event="treatment_complete",
resource_id=treatment_cubicle.id_attribute
)
Click to see how we’d do this with a dictionary instead
event_log.append(
{'patient': unique_entity_identifier,
'pathway': 'Trauma',
'event_type': 'resource_use',
'event': 'triage_begins',
'time': env.now,
'resource_id': triage_resource.id_attribute
}
)
yield self.env.timeout(1) # some amount of time
event_log.append(
{'patient': unique_entity_identifier,
'pathway': 'Trauma',
'event_type': 'resource_use_end',
'event': 'triage_complete',
'time': env.now,
'resource_id': triage_resource.id_attribute}
)When providing your event position details, it then just requires you to include an identifier for the resource.
This requires you to be using an class to manage your resource counts (if following HSMA simpy structure, this will be your g class).
Creating the animation
Determining event positioning in the animation
Once the event log has been created, the positions of each queue and resource must be set up.
An easy way to create this is passing a list of dictionaries to the pd.DataFrame function.
The columns required are event: This must match the label used for the event in the event log x: The x coordinate of the event for the animation. This will correspond to the bottom-right hand corner of a queue, or the rightmost resource. y: The y coordinate of the event for the animaation. This will correspond to the lowest row of a queue, or the central point of the resources. label: A label for the stage. This can be hidden at a later step if you opt to use a background image with labels built-in. Note that line breaks in the label can be created using the HTML tag <br>. resource (OPTIONAL): Only required if the step is a resource_use step. This looks at the ‘scenario’ object passed to the animate_activity_log() function and pulls the attribute with the given name, which should give the number of available resources for that step.
Vidigi provides some helper classes and functions for setting this up.
from vidigi.utils import create_event_position_df, EventPosition
event_position_df = create_event_position_df([
EventPosition(event='arrival', x=50, y=450, label="Arrival"),
EventPosition(event='treatment_wait_begins', x=205, y=275, label="Waiting for Treatment"),
EventPosition(event='treatment_begins', x=205, y=175, label="Being Treated", resource='n_cubicles'),
EventPosition(event='depart', x=270, y=70, label="Exit")
])
Click to see how we’d do this with a dictionary instead
If you’d rather not use the Vidigi helpers for this, you can just pass it as a dataframe, which can be easily generated with a list of dictionaries.
event_position_df = pd.DataFrame([
# Triage
{'event': 'triage_wait_begins',
'x': 160, 'y': 400, 'label': "Waiting for<br>Triage" },
{'event': 'triage_begins',
'x': 160, 'y': 315, 'resource':'n_triage', 'label': "Being Triaged" },
# Trauma pathway
{'event': 'TRAUMA_stabilisation_wait_begins',
'x': 300, 'y': 560, 'label': "Waiting for<br>Stabilisation" },
{'event': 'TRAUMA_stabilisation_begins',
'x': 300, 'y': 500, 'resource':'n_trauma', 'label': "Being<br>Stabilised" },
{'event': 'TRAUMA_treatment_wait_begins',
'x': 630, 'y': 560, 'label': "Waiting for<br>Treatment" },
{'event': 'TRAUMA_treatment_begins',
'x': 630, 'y': 500, 'resource':'n_cubicles', 'label': "Being<br>Treated" },
{'event': 'depart',
'x': 670, 'y': 330, 'label': "Exit"}
])Creating the animation
There are two main ways to create the animation:
- using the one-step function
animate_activity_log()- See this simple example or this slightly more complex example for a demonstration of this.
- using the functions
reshape_for_animations(),generate_animation_df()andgenerate_animation()separately, passing the output of each to the next step. This allows you to apply significant extra customisations to things such as entity icons for patients of different classes.- See this priority queueing example for a demonstration of this.
Models used as examples
Emergency department (Treatment Centre) model
Monks.T, Harper.A, Anagnoustou. A, Allen.M, Taylor.S. (2022) Open Science for Computer Simulation
https://github.com/TomMonks/treatment-centre-sim
The layout code for the emergency department model: https://github.com/hsma-programme/Teaching_DES_Concepts_Streamlit
The hospital efficiency project model
Harper, A., & Monks, T. Hospital Efficiency Project Orthopaedic Planning Model Discrete-Event Simulation [Computer software]. https://doi.org/10.5281/zenodo.7951080
https://github.com/AliHarp/HEP/tree/main
Simulation model with scheduling example
Monks, T.
https://github.com/health-data-science-OR/stochastic_systems
https://github.com/health-data-science-OR/stochastic_systems/tree/master/labs/simulation/lab5
Licences for adapted code
Resource and Store code has been adapted from SimPy. Licence code for SimPy is provided below.
The MIT License (MIT)
Copyright (c) 2013 Ontje Lünsdorf and Stefan Scherfke (also see AUTHORS.txt)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.