Repository navigation
on-disk/lazy draw backend for large fits #1041
Description
For large models (~13K paramemeters) the procedure
library(magrittr)
temp_rds_file <- tempfile(fileext = ".rds")
cmdstanr_fit$save_object(file = temp_rds_file)
standalone_fit <- readRDS(temp_rds_file)Takes an incredible amount of memory.
When loaded into memory, the cmdstanr_fit is around ~8Gb, and the operations pick at 54Gb.
This could be solved by handling ".csv" draws on-disk using DuckDB technology (toggling it with a parameter?)
Here an example of csv handling by DuckDB.
https://stackoverflow.com/questions/77797976/working-with-large-csv-file-using-duckdb-or-arrow-in-r
Reacted by Ben Schneider, Joff and Adam C Smith
Activity
jgabry commentedon Nov 20, 2024
We're definitely interested in ways to use less memory. I haven't used DuckDB but it seems promising. Right now a lot of the contributors to cmdstanr are pretty busy, but if someone else is interested and has time to work on this please let us know!
[-]allow reading and saving (`save_object()`) csv draws with DuckDB to avoid memory explosion[/-][+]on-disk/lazy draw backend for large fits[/+]jgabry commentedon Oct 5, 2026
It turns out the combination of stan-dev/posterior#467 and #1307 work better than a duckdb backend in the tests that I had Claude Fable run. The cmdstanr PR I'll get into the upcoming v1.0. The posterior PR will come whenever it's merged and will immediately work with cmdstanr for anyone with that version of posterior.
jgabry commentedon Oct 5, 2026
Here are measurements comparing the current code, the two open PRs (#1307 and stan-dev/posterior#467), and a DuckDB backend.
The fit has 65,001 variables with 4 chains of 1000 draws each. Memory is the peak resident memory of the R process (the baseline after loading packages is about 225 MB). The multiple in parentheses is the peak divided by the size of the draws.
$save_object()The current code peaks at 7.4x the size of the draws when some variables are read before saving, which matches the 54 GB peak for an 8 GB fit reported above by @stemangiola. The two PRs bring that down to 2.7x without slowing anything down.
DuckDB doesn't help with memory. Reading the draws back into R peaks higher than with the two PRs, the conversion needs about as much memory again, and the DuckDB file is larger than the CSV files. Querying the CSV files directly with DuckDB's
read_csv()didn't finish after 20 minutes on a smaller fit with 13,000 variables, where fread takes 4 seconds. DuckDB's advantage is speed when reading a few variables at a time after the conversion, not the memory problem in this issue.I'm also testing models 4x the size, similar to in the OP, and will report those results as well.
Update: results are consistent when using 2x and 4x the size of my simulation above
stemangiola commentedon Oct 5, 2026
Looks great. Thanks a lot. Looking forward to the new release then!
Maybe in the future, parquet + DuckDB would be a combination to benchmark against. However, I am not sure a Parquet file can be incrementally grown.
jgabry commentedon Oct 5, 2026
Sorry it took so long to get to this! Before coding agents became as capable as they are now, I just didn't have time to get to everything. Now if I have an idea for a memory improvement I can just tell Claude or Codex to try it and compare to DuckDB in a realistic simulation. I agree parquet + DuckDB would be worth looking into at some point.
jgabry commentedon Oct 12, 2026
stan-dev/posterior#467 was also merged, so the combination of the main posterior branch on GitHub + the v1.0 branch of cmdstanr from GitHub has all of this now. Not sure when the next posterior release will be. Hoping to announce the cmdstanr release candidate this week or next week at the latest.