Deletion Overview
Deletion in Rucio can be performed in broadly two ways.
Rule-based deletion
Each rule has an associated lifetime (default: None). The expires_at attribute of a rule is computed as the time when the lifetime was set plus the given lifetime.
Rules can also be locked for protection. Locked rules are excluded from expiration.
The Judge-cleaner daemon is responsible for handling expired rules. When a rule expires and is not locked, it is deleted by the daemon. When the rule is deleted, a tombstone is set on replicas that are no longer covered by any other rules.
Two types of tombstones are set:
-
Obsolete Tombstone (1970-01-01): Set when
purge_replicas=Trueor when the replica state isUNAVAILABLE. These replicas are prioritized for immediate deletion. -
Standard Tombstone: Set using the replica’s
accessed_at(orcreated_at) timestamp. This supports Least Recently Used (LRU) based cleanup.
A tombstone can be in the future (e.g. intermediate replicas in multi-hop transfers) or the replica may be used as a source in an active transfer. These are not eligible for deletion.
See the Judge-cleaner flow diagram.
DID-based deletion
Each DID also has an associated lifetime (default: None). The DID expires_at metadata is computed in a similar way to rules.
The Undertaker daemon processes expired DIDs. It first checks whether any associated rules are locked. If no locked rules exist, all rules for the DID are removed (see Rule-based deletion). The DID is then removed from the catalog.
If all DIDs in a dataset are deleted, the dataset itself is not removed unless its lifetime has also expired.
If undertaker.purge_all_replicas = True, the Undertaker sets an Obsolete tombstone on replicas. Otherwise, it sets a standard tombstone.
Users may also manually set a tombstone via client tools, which directly marks a replica as obsolete.
See the Undertaker flow diagram.
DID expiration takes precedence over rule expiration. However, locked rules are always protected.
After tombstones are set, physical deletion is performed by the Reaper daemon, which removes data from storage.
The deletion service supports two different modes: greedy and non-greedy.
-
Greedy
The Reaper deletes all replicas with eligible tombstone .
-
Non-greedy
The reaper daemon first checks if the free space is needed in the RSE. The needed free space is the difference of minimum free space (set with
Client().add_rse_attribute(rse=<RSE>, key='MinFreeSpace', value=<limit in bytes>)) and actual free space in RSE.- If space is not needed: Only Obsolete replicas are eligible for deletion.
- If space is needed: Both obsolete and standard tombstoned replicas are considered, and deletion follows an LRU strategy. Obsolete replicas are naturally deleted first as they have the oldest timestamps.
See the Reaper flow diagram.