Didclient

DIDClient(rucio_host=None, auth_host=None, account=None, ca_cert=None, auth_type=None, creds=None, timeout=600, user_agent='rucio-clients', vo=None, logger=LOG)

DataIdentifier client class for working with data identifiers

Constructor of the BaseClient.

Parameters:

Name	Type	Description	Default
rucio_host	Optional[str]	The address of the rucio server, if None it is read from the config file.	None
rucio_port		The port of the rucio server, if None it is read from the config file.	required
auth_host	Optional[str]	The address of the rucio authentication server, if None it is read from the config file.	None
auth_port		The port of the rucio authentication server, if None it is read from the config file.	required
account	Optional[str]	The account to authenticate to rucio.	None
use_ssl		Enable or disable ssl for commucation. Default is enabled.	required
ca_cert	Optional[str]	The path to the rucio server certificate.	None
auth_type	Optional[str]	The type of authentication (e.g.: 'userpass', 'kerberos' ...)	None
creds	Optional[dict[str, Any]]	Dictionary with credentials needed for authentication.	None
user_agent	Optional[str]	Indicates the client.	'rucio-clients'
vo	Optional[str]	The VO to authenticate into.	None
logger	Logger	Logger object to use. If None, use the default LOG created by the module	LOG

list_dids(scope, filters, did_type='collection', long=False, recursive=False)

List all data identifiers in a scope which match a given pattern.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
filters	Sequence[dict[str, Any]]	A nested dictionary of key/value pairs like [{'key1': 'value1', 'key2.lte': 'value2'}, {'key3.gte, 'value3'}]. Keypairs in the same dictionary are AND'ed together, dictionaries are OR'ed together. Keys should be suffixed like ., e.g. key1 >= value1 is equivalent to {'key1.gte': value}, where belongs to one of the set {'lte', 'gte', 'gt', 'lt', 'ne' or ''}. Equivalence doesn't require an operator.	required
did_type	Literal['all', 'collection', 'dataset', 'container', 'file']	The type of the did: 'all'(container, dataset or file)|'collection'(dataset or container)|'dataset'|'container'|'file'	'collection'
long	bool	Long format option to display more information for each DID.	False
recursive	bool	Recursively list DIDs content.	False

list_dids_extended(scope, filters, did_type='collection', long=False, recursive=False)

List all data identifiers in a scope which match a given pattern (DEPRECATED)

add_did(scope, name, did_type, statuses=None, meta=None, rules=None, lifetime=None, dids=None, rse=None)

Add data identifier for a dataset or container.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
did_type	Literal['DATASET', 'CONTAINER']	The data identifier type (dataset|container).	required
statuses	Optional[Mapping[str, Any]]	Dictionary with statuses, e.g. {'monotonic':True}.	None
meta	Optional[Mapping[str, Any]]	Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.	None
rules	Optional[Sequence[Mapping[str, Any]]]	Replication rules associated with the data identifier. A list of dictionaries, e.g., [{'copies': 2, 'rse_expression': 'TIERS1'}, ].	None
lifetime	Optional[int]	DID's lifetime (in seconds).	None
dids	Optional[Sequence[Mapping[str, Any]]]	The content.	None
rse	Optional[str]	The RSE name when registering replicas.	None

add_dids(dids)

Bulk add datasets/containers.

add_dataset(scope, name, statuses=None, meta=None, rules=None, lifetime=None, files=None, rse=None)

Add data identifier for a dataset.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
statuses	Optional[Mapping[str, Any]]	Dictionary with statuses, e.g.g {'monotonic':True}.	None
meta	Optional[Mapping[str, Any]]	Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.	None
rules	Optional[Sequence[Mapping[str, Any]]]	Replication rules associated with the data identifier. A list of dictionaries, e.g., [{'copies': 2, 'rse_expression': 'TIERS1'}, ].	None
lifetime	Optional[int]	DID's lifetime (in seconds).	None
files	Optional[Sequence[Mapping[str, Any]]]	The content.	None
rse	Optional[str]	The RSE name when registering replicas.	None

add_datasets(dsns)

Bulk add datasets.

Parameters:

Name	Type	Description	Default
dsns	Iterable[dict[str, Any]]	A list of datasets.	required

add_container(scope, name, statuses=None, meta=None, rules=None, lifetime=None)

Add data identifier for a container.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
statuses	Optional[Mapping[str, Any]]	Dictionary with statuses, e.g.g {'monotonic':True}.	None
meta	Optional[Mapping[str, Any]]	Meta-data associated with the data identifier is represented using key/value pairs in a dictionary.	None
rules	Optional[Sequence[Mapping[str, Any]]]	Replication rules associated with the data identifier. A list of dictionaries, e.g., [{'copies': 2, 'rse_expression': 'TIERS1'}, ].	None
lifetime	Optional[int]	DID's lifetime (in seconds).	None

add_containers(cnts)

Bulk add containers.

Parameters:

Name	Type	Description	Default
cnts	Sequence[dict[str, Any]]	A list of containers.	required

attach_dids(scope, name, dids, rse=None)

Attach data identifier.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
dids	Sequence[Mapping[str, Any]]	The content.	required
rse	Optional[str]	The RSE name when registering replicas.	None

detach_dids(scope, name, dids=None)

Detach data identifier

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
dids	Optional[Sequence[Mapping[str, Any]]]	The content.	None

attach_dids_to_dids(attachments, ignore_duplicate=False)

Add dids to dids.

Parameters:

Name	Type	Description	Default
attachments	Sequence[dict[str, Union[str, Sequence[dict[str, Any]]]]]	The attachments. attachments is: [attachment, attachment, ...] attachment is: {'scope': scope, 'name': name, 'dids': dids} dids is: [{'scope': scope, 'name': name}, ...]	required
ignore_duplicate	bool	If True, ignore duplicate entries.	False

add_files_to_datasets(attachments, ignore_duplicate=False)

Add files to datasets.

Parameters:

Name	Type	Description	Default
attachments	Sequence[dict[str, Union[str, Sequence[dict[str, Any]]]]]	The attachments. attachments is: [attachment, attachment, ...] attachment is: {'scope': scope, 'name': name, 'dids': dids} dids is: [{'scope': scope, 'name': name}, ...]	required
ignore_duplicate	bool	If True, ignore duplicate entries.	False

add_datasets_to_containers(attachments)

Add datasets_to_containers.

Parameters:

Name	Type	Description	Default
attachments	Sequence[dict[str, Union[str, Sequence[dict[str, Any]]]]]	The attachments. attachments is: [attachment, attachment, ...] attachment is: {'scope': scope, 'name': name, 'dids': dids} dids is: [{'scope': scope, 'name': name}, ...]	required

add_containers_to_containers(attachments)

Add containers_to_containers.

Parameters:

Name	Type	Description	Default
attachments	Sequence[dict[str, Union[str, Sequence[dict[str, Any]]]]]	The attachments. attachments is: [attachment, attachment, ...] attachment is: {'scope': scope, 'name': name, 'dids': dids} dids is: [{'scope': scope, 'name': name}, ...]	required

add_files_to_dataset(scope, name, files, rse=None)

Add files to datasets.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The dataset name.	required
files	Sequence[Mapping[str, Any]]	The content.	required
rse	Optional[str]	The RSE name when registering replicas.	None

add_files_to_archive(scope, name, files)

Add files to archive.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The dataset name.	required
files	Sequence[Mapping[str, Any]]	The content.	required

add_datasets_to_container(scope, name, dsns)

Add datasets to container.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The dataset name.	required
dsns	Sequence[Mapping[str, Any]]	The content.	required

add_containers_to_container(scope, name, cnts)

Add containers to container.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The dataset name.	required
cnts	Sequence[Mapping[str, Any]]	The content.	required

list_content(scope, name)

List data identifier contents.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required

list_content_history(scope, name)

List data identifier contents history.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required

list_files(scope, name, long=None)

List data identifier file contents.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
long	Optional[bool]	A boolean to choose if GUID is returned or not.	None

bulk_list_files(dids)

List data identifier file contents.

Parameters:

Name	Type	Description	Default
dids	list[dict[str, Any]]	The list of DIDs.	required

get_did(scope, name, dynamic=False, dynamic_depth=None)

Retrieve a single data identifier.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
dynamic_depth	Optional[str]	The DID type as string ('FILE'/'DATASET') at which to stop the dynamic length/bytes calculation. If not set, the size will not be computed dynamically.	None
dynamic	bool	(Deprecated) same as dynamic_depth = 'FILE'	False

get_metadata(scope, name, plugin='DID_COLUMN')

Get data identifier metadata

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
plugin	str	Backend Metadata plugin the Rucio server should use to query data.	'DID_COLUMN'

get_metadata_bulk(dids, inherit=False)

Bulk get data identifier metadata

Parameters:

Name	Type	Description	Default
inherit	bool	A boolean. If set to true, the metadata of the parent are concatenated.	False
dids	Sequence[Mapping[str, Any]]	A list of dids.	required

set_metadata(scope, name, key, value, recursive=False)

Set data identifier metadata

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
key	str	the key.	required
value	Any	the value.	required
recursive	bool	Option to propagate the metadata change to content.	False

set_metadata_bulk(scope, name, meta, recursive=False)

Set data identifier metadata in bulk.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
meta	Mapping[str, Any]	the metadata key-values.	required
recursive	bool	Option to propagate the metadata change to content.	False

set_dids_metadata_bulk(dids, recursive=False)

Set metadata to a list of data identifiers.

Parameters:

Name	Type	Description	Default
dids	Sequence[Mapping[str, Any]]	A list of dids including metadata, i.e. [{'scope': scope1, 'name': name1, 'meta': {key1: value1, key2: value2}] .	required
recursive	bool	Option to propagate the metadata update to content.	False

set_status(scope, name, **kwargs)

Set data identifier status

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required
kwargs		Keyword arguments of the form status_name=value.	{}

close(scope, name)

close dataset/container

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The dataset/container name.	required

delete_metadata(scope, name, key)

Delete data identifier metadata

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier.	required
key	str	the key.	required

list_did_rules(scope, name)

List the associated rules of a data identifier.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required

list_associated_rules_for_file(scope, name)

List the associated rules a file is affected from..

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The file name.	required

get_dataset_by_guid(guid)

Get the parent datasets for a given GUID.

Parameters:

Name	Type	Description	Default
guid	str	The GUID.	required

Returns:

Type	Description
Iterator[dict[str, Any]]	A did

scope_list(scope, name=None, recursive=False)

List data identifiers in a scope.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	Optional[str]	The data identifier name.	None
recursive	bool	boolean, True or False.	False

list_parent_dids(scope, name)

List parent dataset/containers of a did.

Parameters:

Name	Type	Description	Default
scope	str	The scope.	required
name	str	The name.	required

create_did_sample(input_scope, input_name, output_scope, output_name, nbfiles)

Create a sample from an input collection.

Parameters:

Name	Type	Description	Default
input_scope	str	The scope of the input DID.	required
input_name	str	The name of the input DID.	required
output_scope	str	The scope of the output dataset.	required
output_name	str	The name of the output dataset.	required
account		The account.	required
nbfiles	int	The number of files to register in the output dataset.	required

resurrect(dids)

Resurrect a list of dids.

Parameters:

Name	Type	Description	Default
dids	Sequence[Mapping[str, Any]]	A list of dids [{'scope': scope, 'name': name}, ...]	required

list_archive_content(scope, name)

List archive contents.

Parameters:

Name	Type	Description	Default
scope	str	The scope name.	required
name	str	The data identifier name.	required