tm_animate memory issue

[wrobell]

Please consider

library(sf)
library(tmap)

bbox <- st_bbox(c(xmin=-6.2,
                  ymin=53.4,
                  xmax=-6.1,
                  ymax=53.5),
                crs=st_crs(4326))
cp <- st_as_sfc(bbox) |> st_centroid()
cx = st_coordinates(cp)[,1]
cy = st_coordinates(cp)[,2]

tb = seq(as.POSIXct('2025-08-01 10:00:00', tz='UTC'),
         as.POSIXct('2025-08-01 12:00:00', tz='UTC'),
         by=1)
angle = seq(0, 2 * pi, length.out=length(tb))

data = st_sf(tb=tb,
             geometry=st_sfc(lapply(seq_len(length(tb)),
                                    function(i) st_point(c(cx + 0.025 * cos(angle[i]),
                                                           cy + 0.025 * sin(angle[i])))),
                             crs=4326),
             angle=angle)

p = (tm_basemap('CartoDB.Positron')
     + tm_shape(data, bbox=bbox, crs=4326)
     + tm_dots('location', fill='red', fill_alpha=0.75)
     + tm_options(facet.max=3600 * 3)
     + tm_animate_fast('tb', fps=30))

tmap_animation(p, file='plot-traffic-bug.mp4', width=1024, height=1024)

Running this script you can see memory usage steadily grow and the execution fails with error

*** recursive gc invocation

This also happens when I set TMP=~/tmp, so PNG files (frames) are stored on a hard drive instead of tmpfs.

tmap 4.3, sf 1.1

[wrobell]

I have found a solution for myself, which tmap project could consider.

What I want to achieve is presented in this tutorial: https://sq1090.diy-labs.eu/doc/traffic-anim.html. See the video at the end of the introduction section.

The script generating the video can be found at: https://gitlab.com/sq1090/sq1090/-/blob/master/scripts/traffic-on-hold-anim?ref_type=heads

The script

• creates base map consisting of positrion tiles from CartoDB project and additional static geographical features; it stores the map as a grob
• creates each animation frame; animation frames are rendered on top of the base map grob
• generates PNG file for each frame by using lapply and subset R functions
• uses av library to create MP4 file

Thanks to this approach

• memory is no longer an issue
• it is faster than using tm_animate/tm_animate_fast functions
• does not require changing facet options (see also tm_animate_fast requires facet.max option #1240)

I believe this approach could also help with #1241.

In the tutorial above, drawing static features once helps with performance. Maybe tmap could have tm_basemap_end function to make distinction between static and animated parts? For example

tm_basemap(...)
+ tm_shape()
...
+ tm_shape()
...
+ tm_basemap_end()   # render up to this point and store as grob
+ tm_shape()
...
+ tm_shape()
...
+ tm_animate()  # overlay each animation frame over stored grob

[mtennekes]

Thx @wrobell

Wondering what the bottlenecks are of the current implementation. I thought the basemaps are cached, so that should be be much slower than via a grob object. I'll check when time allows.

[wrobell]

IMHO, the main bottleneck seems to be storing all of the frames in memory.

[mtennekes]

The issue is indeed that all grob tree objects are created upfront, and saved to frames later in the process.
It's not just basemaps that are an issue. The basemap grob is generated only once, but added to the grob tree each frame (like all layers). It should be possible to change the process, but that is something to do carefully.