Keep logs under a budget

A dataflash log is roughly 10 MB for a short SITL hop and hundreds of MB for a long flight, and an afternoon of runs fills a disk. mcardupilot keeps logs only when asked, and holds the ones it keeps to a budget.

Choose what each session keeps

open_sitl(keep_logs=...) sets the policy, last by default:

keep_logs	At close
never	every log in the session’s run directory is deleted
last	the newest log is kept, the rest deleted
all	every log is kept

Kept logs move to the server’s kept-log directory as <UTC time>-<label or session id>.BIN. close_sitl(keep_logs="never") overrides the policy for one throwaway flight. A session that died has already applied its own policy, so an override at close does nothing there.

Set the budget

export MCARDUPILOT_LOG_BUDGET_GB=5

The default is 20 GB. The budget is enforced at every close that keeps a log and once at server start: the oldest kept logs go first until the directory fits. A close reports what it took:

{ "session_id": "c37c5b", "kept": [".../logs/20261003T202159Z-gusty-b.BIN"], "pruned": [".../logs/20260919T081544Z-cruise.BIN"], "state": "closed" }

Never taken, however far over budget:

• a pinned log;
• a log written in the last ten minutes, which may belong to a SITL with no pid file;
• a log under a running SITL;
• the log this close just kept.

Only the kept-log directory is pruned. Logs elsewhere, in a harness’s results directory or a session’s live run directory, are never touched by the server.

server_info shows logs_used_gb beside log_budget_gb.

Pin what must survive

pin_log(log="20261003T202311Z-gusty-b.BIN")

{ "name": "20261003T202311Z-gusty-b.BIN", "pinned": true }

A pin is a <name>.BIN.pin file beside the log, so it survives server restarts and is visible to every server sharing the data directory. list_logs shows pinned for each log. pin_log(log=..., pinned=False) lets pruning take it again. Only logs inside the kept-log directory can be pinned.

Caution

Pinned logs still count against the budget. Pin enough of them and every unpinned log is pruned as soon as it is old enough. Keep the pinned set small, or raise the budget.

Prune by hand

uv run python -m mcardupilot.logs prune --root ./results/logs --budget-gb 2      # dry run
uv run python -m mcardupilot.logs prune --root ./results/logs --budget-gb 2 --apply

Without --apply nothing is deleted; the report lists what would go, oldest first, and how many logs were spared and why. --root repeats, and defaults to the kept-log directory. -v lists every log instead of the first ten.

Prunes across processes are serialized by a lock file in the directory. A prune that cannot get the lock within five seconds is skipped rather than stalling a close; the next close catches up.