Speed measurement
Here we will compare the speed of plotting UnfoldMakie with MNE (Python) and EEGLAB (MATLAB).
Three cases are measured:
- Single topoplot
- Topoplot series with 50 topoplots
- Topoplott animation with 50 timestamps
Note that the results of benchmarking on your computer and on Github may differ.
using UnfoldMakie
using TopoPlots
using BenchmarkTools
using Observables
using CairoMakie
using PythonPlot;
using PyMNE; CondaPkg Found dependencies: /home/runner/.julia/packages/CondaPkg/0UqYV/CondaPkg.toml
CondaPkg Found dependencies: /home/runner/.julia/packages/PyMNE/cNGDN/CondaPkg.toml
CondaPkg Found dependencies: /home/runner/.julia/packages/PythonCall/83z4q/CondaPkg.toml
CondaPkg Found dependencies: /home/runner/.julia/packages/PythonPlot/oS8x4/CondaPkg.toml
CondaPkg Dependencies already up to dateData input
dat, positions = TopoPlots.example_data()
df = UnfoldMakie.eeg_array_to_dataframe(dat[:, :, 1], string.(1:length(positions)));Topoplots
UnfoldMakie.jl
@benchmark plot_topoplot(dat[:, 320, 1]; positions = positions)BenchmarkTools.Trial: 102 samples with 1 evaluation per sample.
Range (min … max): 42.301 ms … 342.773 ms ┊ GC (min … max): 0.00% … 85.41%
Time (median): 46.291 ms ┊ GC (median): 0.00%
Time (mean ± σ): 49.134 ms ± 29.428 ms ┊ GC (mean ± σ): 5.84% ± 8.46%
▁▁ ▁ ▃█ ▁▁▁ ▁▃▁ █ ▁ █▁
▄▄▇▁▄▄▇▄██▇▄█▄▇▄▄▁▇▄██▄▄███▇███▄█▇▄▄▇▇█▇▁██▇▄▄▄▄▁▁▁▄▇▁▁▁▁▁▁▄ ▄
42.3 ms Histogram: frequency by time 51.1 ms <
Memory estimate: 11.59 MiB, allocs estimate: 190700.UnfoldMakie.jl with DelaunayMesh
@benchmark plot_topoplot(
dat[:, 320, 1];
positions = positions,
topo_interpolation = (; interpolation = DelaunayMesh()),
)BenchmarkTools.Trial: 95 samples with 1 evaluation per sample.
Range (min … max): 46.440 ms … 347.054 ms ┊ GC (min … max): 0.00% … 84.84%
Time (median): 49.343 ms ┊ GC (median): 0.00%
Time (mean ± σ): 52.784 ms ± 30.572 ms ┊ GC (mean ± σ): 5.87% ± 8.70%
▁ ▁ ▃ ▃▁█ ▃▁▆▃▁▁ █ ▁ ▁
█▄▁▄▁█▄▄█▇▇▁███▇██████▄█▇█▁▄█▇▄▄▄▇▇▇▄▇▄▁▁▄▁▄▁▁▄▁▁▁▄▄▁▁▁▁▄▁▁▇ ▁
46.4 ms Histogram: frequency by time 55.3 ms <
Memory estimate: 11.59 MiB, allocs estimate: 190707.MNE
posmat = collect(reduce(hcat, [[p[1], p[2]] for p in positions])')
pypos = Py(posmat).to_numpy()
pydat = Py(dat[:, 320, 1])
@benchmark begin
f = PythonPlot.figure()
PyMNE.viz.plot_topomap(
pydat,
pypos,
sphere = 1.1,
extrapolate = "box",
cmap = "RdBu_r",
sensors = false,
contours = 6,
)
f.show()
endBenchmarkTools.Trial: 387 samples with 1 evaluation per sample.
Range (min … max): 11.941 ms … 182.226 ms ┊ GC (min … max): 0.00% … 0.00%
Time (median): 12.265 ms ┊ GC (median): 0.00%
Time (mean ± σ): 12.912 ms ± 8.784 ms ┊ GC (mean ± σ): 0.00% ± 0.00%
▂█▇▆▅▄▄
▄▇████████▆▇█▆▆▅▅▆▅▄▃▄▃▃▃▃▂▁▂▃▁▂▁▁▁▂▂▁▂▁▁▁▃▁▁▁▂▁▂▁▁▁▁▁▁▁▁▁▁▂ ▃
11.9 ms Histogram: frequency by time 14.5 ms <
Memory estimate: 3.30 KiB, allocs estimate: 98.Topoplot series
Note that UnfoldMakie and MNE have different defaults for displaying topoplot series. UnfoldMakie in plot_topoplot averages over time samples. MNE in plot_topopmap displays single samples without averaging.
UnfoldMakie.jl
@benchmark begin
plot_topoplotseries(
df;
bin_num = 50,
positions = positions,
axis = (; xlabel = "Time windows [s]"),
)
endBenchmarkTools.Trial: 3 samples with 1 evaluation per sample.
Range (min … max): 2.028 s … 2.186 s ┊ GC (min … max): 0.00% … 0.00%
Time (median): 2.132 s ┊ GC (median): 0.00%
Time (mean ± σ): 2.115 s ± 80.591 ms ┊ GC (mean ± σ): 0.00% ± 0.00%
█ █ █
█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
2.03 s Histogram: frequency by time 2.19 s <
Memory estimate: 342.81 MiB, allocs estimate: 4928119.MNE
easycap_montage = PyMNE.channels.make_standard_montage("standard_1020")
ch_names = pyconvert(Vector{String}, easycap_montage.ch_names)[1:64]
info = PyMNE.create_info(PyList(ch_names), ch_types = "eeg", sfreq = 1)
info.set_montage(easycap_montage)
simulated_epochs = PyMNE.EvokedArray(Py(dat[:, :, 1]), info)
@benchmark simulated_epochs.plot_topomap(1:50)BenchmarkTools.Trial: 8 samples with 1 evaluation per sample.
Range (min … max): 686.545 ms … 717.607 ms ┊ GC (min … max): 0.00% … 0.00%
Time (median): 692.219 ms ┊ GC (median): 0.00%
Time (mean ± σ): 696.033 ms ± 9.843 ms ┊ GC (mean ± σ): 0.00% ± 0.00%
█ ██ █ █ █ █ █
█▁▁▁▁▁██▁█▁▁█▁▁▁▁▁▁▁▁▁▁█▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
687 ms Histogram: frequency by time 718 ms <
Memory estimate: 2.39 KiB, allocs estimate: 69.MATLAB
Running MATLAB on a GitHub Action is not easy. So we benchmarked three consecutive executions (on a screenshot) on a server with an AMD EPYC 7452 32-core processor. Note that Github and the server we used for MATLAB benchmarking are two different computers, which can give different timing results.
Animation
The main advantage of Julia is the speed with which the figures are updated.
timestamps = range(1, 50, step = 1)
framerate = 5050UnfoldMakie with .gif
@benchmark begin
f = Makie.Figure()
dat_obs = Observable(dat[:, 1, 1])
plot_topoplot!(f[1, 1], dat_obs, positions = positions)
record(f, "topoplot_animation_UM.gif", timestamps; framerate = framerate) do t
dat_obs[] = @view(dat[:, t, 1])
end
endBenchmarkTools.Trial: 2 samples with 1 evaluation per sample.
Range (min … max): 2.818 s … 2.854 s ┊ GC (min … max): 0.00% … 0.00%
Time (median): 2.836 s ┊ GC (median): 0.00%
Time (mean ± σ): 2.836 s ± 25.598 ms ┊ GC (mean ± σ): 0.00% ± 0.00%
█ █
█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
2.82 s Histogram: frequency by time 2.85 s <
Memory estimate: 178.42 MiB, allocs estimate: 602397.
MNE with .gif
@benchmark begin
fig, anim = simulated_epochs.animate_topomap(
times = Py(timestamps),
frame_rate = framerate,
blit = false,
image_interp = "cubic", # same as CloughTocher
)
anim.save("topomap_animation_mne.gif", writer = "ffmpeg", fps = framerate)
endBenchmarkTools.Trial: 1 sample with 1 evaluation per sample.
Single result which took 9.359 s (0.00% GC) to evaluate,
with a memory estimate of 3.03 KiB, over 96 allocations.Note, that due to some bugs in (probably) PythonCall topoplot is black and white.
This page was generated using Literate.jl.