import plotly.io as pio
from simple_triage_assess_treat_model import Model
from vidigi.animation import animate_activity_log
from vidigi.utils import EventPosition, create_event_position_df
pio.renderers.default = "notebook"
import random
random.seed(42)Feature Breakdown: Comparing gauge, hybrid and gaugeless animations for simulations with high entity volumes
Warning
This example has not yet been updated to reflect all of the new features and recommendations from vidigi 2.0.0, which have further simplified the process of adding vidigi to your model and accessing and modifying your animation.
For now, all of the code below will still work, but check out the getting started page for a full guide to the recommended way to use vidigi 2.0.0.
Vidigi was originally designed to work with models where relatively small numbers of entities were being visualised at any time.
However, many models work at a larger scale, with hundreds or thousands of entities being tracked.
Due to the way Plotly animations work, there are some limitations to what works well when larger numbers of entities are involved - but we do have some options.
Let’s start by looking at a simple three-step model where patients are triaged, assessed and treated. We’ve used the vidigi EventLogger class and only done 1 run for demonstration purposes.
model = Model(run_number=1)
logs = model.run()We can use the EventLogger .summary() function to see a quick overview of our patients.
logs.summary(){'total_events': 12587,
'event_types': {'queue': 8839, 'arrival_departure': 3748},
'time_range': (np.float64(0.0), np.float64(363.9660096972095)),
'unique_entities': 3614,
'label': None}Let’s take a quick look at a sample of the dataframe.
logs_df = logs.to_dataframe()
logs_df| entity_id | event_type | event | time | run_number | |
|---|---|---|---|---|---|
| 0 | 1 | arrival_departure | arrival | 0.000000 | 1 |
| 1 | 1 | queue | queue_initial_review | 0.000000 | 1 |
| 2 | 1 | queue | start_initial_review | 0.000000 | 1 |
| 3 | 1 | queue | end_initial_review | 0.012664 | 1 |
| 4 | 1 | queue | queue_assessment | 0.012664 | 1 |
| ... | ... | ... | ... | ... | ... |
| 12582 | 3612 | queue | queue_initial_review | 363.846975 | 1 |
| 12583 | 3613 | arrival_departure | arrival | 363.961821 | 1 |
| 12584 | 3613 | queue | queue_initial_review | 363.961821 | 1 |
| 12585 | 3614 | arrival_departure | arrival | 363.966010 | 1 |
| 12586 | 3614 | queue | queue_initial_review | 363.966010 | 1 |
12587 rows × 5 columns
And let’s also just see what steps we have.
logs_df.event.unique()<ArrowStringArray>
[ 'arrival', 'queue_initial_review', 'start_initial_review',
'end_initial_review', 'queue_assessment', 'start_assessment',
'end_assessment', 'queue_treatment', 'start_treatment',
'end_treatment', 'depart']
Length: 11, dtype: strWe’ll choose a subset of these steps to represent in our animation, and create our event positioning dataframe to stagger them.
event_position_df = create_event_position_df(
[
EventPosition(
event="queue_initial_review",
x=200,
y=600,
label="Waiting for <br> Initial Review",
),
EventPosition(
event="start_initial_review",
x=200,
y=500,
label="Having <br> Initial Review",
),
EventPosition(
event="queue_assessment", x=350, y=400, label="Waiting for <br> Assessment"
),
EventPosition(
event="start_assessment", x=350, y=300, label="Having for <br> Assessment"
),
EventPosition(
event="queue_treatment", x=500, y=200, label="Waiting for <br> Treatment"
),
EventPosition(
event="start_treatment", x=500, y=100, label="Having <br> Treatment"
),
]
)
event_position_df| event | x | y | label | resource | direction | flip_icons | resource_icon | |
|---|---|---|---|---|---|---|---|---|
| 0 | queue_initial_review | 200 | 600 | Waiting for <br> Initial Review | None | None | None | None |
| 1 | start_initial_review | 200 | 500 | Having <br> Initial Review | None | None | None | None |
| 2 | queue_assessment | 350 | 400 | Waiting for <br> Assessment | None | None | None | None |
| 3 | start_assessment | 350 | 300 | Having for <br> Assessment | None | None | None | None |
| 4 | queue_treatment | 500 | 200 | Waiting for <br> Treatment | None | None | None | None |
| 5 | start_treatment | 500 | 100 | Having <br> Treatment | None | None | None | None |
Now let’s create an animation where the step_snapshot_max is set to 0 (meaning all patients are treated as ‘excess’ queue and not visualised as individual entities), and turn on gauges to replace the ‘+ x more’ text so we have a visual indicator of queue lengths across our steps.
animate_activity_log(
event_log=logs_df,
event_position_df=event_position_df,
simulation_time_unit="days",
every_x_time_units=1,
step_snapshot_limit_gauges=True,
step_snapshot_max=0,
limit_duration=365,
override_x_max=600,
override_y_max=700,
plotly_width=1600,
time_display_units="day_clock",
)/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: 719 row(s) across 1 event(s) with no matching `event_position_df` row were rendered anyway: 'depart' (719 entities).
An event with no position gets no coordinates, and a point with no coordinates can't be drawn - the entity's icon disappears for that frame, then flies in from the top-left corner once a positioned event takes over again.
Add a row to `event_position_df` for each event listed above.
full_entity_df_plus_pos = generate_animation_df(Why would we bother with this over a standard animated bar chart or some other form of representation?
In a more complex model with a large number of steps, some of which may represent different paths certain patients take, being able to arrange them in a logical order - particularly showing the potential parallel pathways.
We could also pair this with a background image that helps to represent elements of the pathway, such as the ways in which patients can branch off or circle back to different points.
Alternative representations
For comparison, let’s explore including some icons for individuals to try to demonstrate flow between steps.
However, as many of the individuals moving between the steps are not shown in the queue in the frame before they move, we observe a confusing ‘flying in’ effect from the top left of the animation - a Plotly limitation, not a choice vidigi made deliberately.
However, one benefit is that we can use the hover features to see the wait duration of different individuals in the queue, allowing us to drill down into entity experience at different points in the model in a way that we couldn’t otherwise.
animate_activity_log(
event_log=logs_df,
event_position_df=event_position_df,
simulation_time_unit="days",
every_x_time_units=1,
step_snapshot_limit_gauges=True,
step_snapshot_max=10,
limit_duration=365,
override_x_max=600,
override_y_max=700,
wrap_queues_at=10,
custom_entity_icon_list=["⚫"],
frame_duration=1000,
plotly_width=1600,
entity_icon_size=18,
time_display_units="day_clock",
warm_up=180,
)/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: 877 row(s) across 1 event(s) with no matching `event_position_df` row were rendered anyway: 'depart' (877 entities).
An event with no position gets no coordinates, and a point with no coordinates can't be drawn - the entity's icon disappears for that frame, then flies in from the top-left corner once a positioned event takes over again.
Add a row to `event_position_df` for each event listed above.
full_entity_df_plus_pos = generate_animation_df(Watch an individual queue for a few frames in a row and the fly-in gets worse than it first looks: an entity that has been queuing for a while, hidden behind step_snapshot_max, appears to arrive the moment it finally crosses back under the limit - as if it had just joined, when really it had been waiting all along.
step_snapshot_reveal_pop_in=True fixes this specific case - an entity that was already in the system but hidden by the cap now pops in at its queue position instead of flying in. A genuine new arrival still flies in, which is usually the clearer visual cue for “just joined the system” - only reveals from behind the cap are affected. It’s opt-in for now (default False, so nothing changes unless you ask for it) and costs one extra invisible row per reveal - negligible next to the step_snapshot_max savings this whole notebook is about.
animate_activity_log(
event_log=logs_df,
event_position_df=event_position_df,
simulation_time_unit="days",
every_x_time_units=1,
step_snapshot_limit_gauges=True,
step_snapshot_max=10,
step_snapshot_reveal_pop_in=True,
limit_duration=365,
override_x_max=600,
override_y_max=700,
wrap_queues_at=10,
custom_entity_icon_list=["⚫"],
frame_duration=1000,
plotly_width=1600,
entity_icon_size=18,
time_display_units="day_clock",
warm_up=180,
)/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: 877 row(s) across 1 event(s) with no matching `event_position_df` row were rendered anyway: 'depart' (877 entities).
An event with no position gets no coordinates, and a point with no coordinates can't be drawn - the entity's icon disappears for that frame, then flies in from the top-left corner once a positioned event takes over again.
Add a row to `event_position_df` for each event listed above.
full_entity_df_plus_pos = generate_animation_df(Let’s move away from the black dots to make the benefits of this easier to see.
animate_activity_log(
event_log=logs_df,
event_position_df=event_position_df,
simulation_time_unit="days",
every_x_time_units=1,
step_snapshot_limit_gauges=True,
step_snapshot_max=10,
step_snapshot_reveal_pop_in=True,
limit_duration=365,
override_x_max=600,
override_y_max=700,
wrap_queues_at=10,
frame_duration=1000,
plotly_width=1600,
entity_icon_size=18,
time_display_units="day_clock",
warm_up=180,
)/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: 877 row(s) across 1 event(s) with no matching `event_position_df` row were rendered anyway: 'depart' (877 entities).
An event with no position gets no coordinates, and a point with no coordinates can't be drawn - the entity's icon disappears for that frame, then flies in from the top-left corner once a positioned event takes over again.
Add a row to `event_position_df` for each event listed above.
full_entity_df_plus_pos = generate_animation_df(Setting the transition duration to 0 is an alternative way to keep the option to have the hover for the individuals in the queue while removing the odd animation effects without step_snapshot_reveal_pop_in=True.
animate_activity_log(
event_log=logs_df,
event_position_df=event_position_df,
simulation_time_unit="days",
every_x_time_units=1,
step_snapshot_limit_gauges=True,
step_snapshot_max=10,
limit_duration=365,
override_x_max=600,
override_y_max=700,
wrap_queues_at=10,
frame_duration=400,
frame_transition_duration=0,
plotly_width=1600,
entity_icon_size=18,
time_display_units="day_clock",
warm_up=180,
)/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: 877 row(s) across 1 event(s) with no matching `event_position_df` row were rendered anyway: 'depart' (877 entities).
An event with no position gets no coordinates, and a point with no coordinates can't be drawn - the entity's icon disappears for that frame, then flies in from the top-left corner once a positioned event takes over again.
Add a row to `event_position_df` for each event listed above.
full_entity_df_plus_pos = generate_animation_df(
WarningA second way to get the exact same symptom: a missing
event_position_df row
generate_animation_df resolves each snapshot’s coordinates with event_position_df.merge(..., on="event", how="left"). An event with no matching row gets NaN coordinates wherever it’s an entity’s most-recently-logged step at a rendered snapshot - and a point with no coordinates can’t be drawn, so Plotly removes it from the frame outright instead of placing it somewhere sensible. The entity’s icon just disappears, then reappears - flying in from the top-left corner exactly like the step_snapshot_max reveal bug above - once a positioned event takes over again. This is what actually cost us the most time while building this notebook: it looks identical to the reveal bug, but step_snapshot_reveal_pop_in does nothing for it, because there’s no cap involved at all.
It’s easy to end up here by accident, because most of the steps you’d naturally leave unpositioned - arrival, a resource_use_end step like end_initial_review above - are logged at the same simulated instant as the very next step, so the next step always wins the “most recent event” tie-break and the earlier one is never actually selected for rendering. That’s the coincidence “you don’t need coordinates for resource_use_end” relies on in practice, not a guarantee vidigi makes. depart gets no such protection - and, worth flagging honestly, event_position_df above doesn’t have a row for it either, so every animation in this notebook is quietly relying on the same coincidence for its final exit frame.
generate_animation_df now checks this for you and raises a UserWarning naming the offending event(s), row and entity counts included, whenever one of these gaps is actually rendered - so you no longer need to check by hand. It only looks at events that were genuinely selected for rendering, the same distinction drawn above, so it won’t fire on a resource_use_end step that’s always superseded by its successor. Every call in this notebook now triggers it, naming depart - the live gap described above, left in place deliberately so the warning has something real to show.
Finally, let’s try including all of our individuals in the plot - we’ll just make them very small and adjust our event position dataframe slightly to provide more space.
event_position_df = create_event_position_df(
[
EventPosition(
event="queue_initial_review",
x=200,
y=900,
label="Waiting for <br> Initial Review",
),
EventPosition(
event="queue_assessment", x=350, y=500, label="Waiting for <br> Assessment"
),
EventPosition(
event="queue_treatment", x=500, y=200, label="Waiting for <br> Treatment"
),
]
)
event_position_df| event | x | y | label | resource | direction | flip_icons | resource_icon | |
|---|---|---|---|---|---|---|---|---|
| 0 | queue_initial_review | 200 | 900 | Waiting for <br> Initial Review | None | None | None | None |
| 1 | queue_assessment | 350 | 500 | Waiting for <br> Assessment | None | None | None | None |
| 2 | queue_treatment | 500 | 200 | Waiting for <br> Treatment | None | None | None | None |
You may notice that the animation starts to struggle significantly, with slowdowns occurring, despite it generating very quickly. This is due to the sheer amount of data per frame that needs to be stored. In extreme cases, this can cause browser windows to freeze.
However, this does give us more of a sense of flow between steps, which can complement the gauge animation.
The gauge version of the animation can be utilised to better track the scale of queues, while the non-gauge animation may better serve as a demonstration of how the underlying model works in terms of potential paths for people to move between steps.
fig = animate_activity_log(
event_log=logs_df,
event_position_df=event_position_df,
simulation_time_unit="days",
every_x_time_units=7,
step_snapshot_limit_gauges=True,
step_snapshot_max=9999,
debug_mode=True,
limit_duration=365,
override_x_max=600,
override_y_max=1800,
wrap_queues_at=75,
custom_entity_icon_list=["⚫"],
entity_icon_size=6,
gap_between_entities=2,
gap_between_queue_rows=15,
frame_duration=1000,
frame_transition_duration=2000,
plotly_width=1600,
time_display_units="day_clock",
warm_up=180,
)
figAnimation function called at 14:04:23
Iteration through time-unit-by-time-unit logs complete 14:04:24
Snapshot df concatenation complete at 14:04:24
Reshaped animation dataframe finished construction at 14:04:24
Placement dataframe started construction at 14:04:24
Placement dataframe finished construction at 14:04:24/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: `step_snapshot_max` is not a multiple of `wrap_queues_at`.The animation will display better if this is resolved.
full_entity_df_plus_pos = generate_animation_df(
/home/runner/work/vidigi/vidigi/src/vidigi/animation.py:2609: UserWarning: 176 row(s) across 4 event(s) with no matching `event_position_df` row were rendered anyway: 'depart' (64 entities), 'start_initial_review' (56 entities), 'start_assessment' (28 entities), 'start_treatment' (27 entities).
An event with no position gets no coordinates, and a point with no coordinates can't be drawn - the entity's icon disappears for that frame, then flies in from the top-left corner once a positioned event takes over again.
Add a row to `event_position_df` for each event listed above.
full_entity_df_plus_pos = generate_animation_df(Output animation generation complete at 14:04:45
Total Time Elapsed: 21.94 seconds