Skip to main content

Configuration parameters

Description of the configuration parameters splitted in three different places: Rucio configuration file (rucio.cfg), Rucio configuration table and RSE attributes.

Rucio Configuration File (rucio.cfg)

  • Only the Configuration File provides information for the client connection information or the database configuration, this information cannot be set in a different location.
  • If not specified in the environmental variables (as $RUCIO_CONFIG), Rucio will look for the config in the following locations -
    • $RUCIO_HOME/etc/rucio.cfg
    • $VIRTUAL_ENV/etc/rucio.cfg
    • /opt/rucio/etc/rucio.cfg

Options and Defaults

accounts

  • special_accounts

alembic

  • cfg: Path to the configuration file (.ini) for Alembic. Example: /opt/rucio/etc/alembic.ini. No default.

api

  • endpoints: (Optional) Endpoints separated by commas. Default: ['accountlimits', 'accounts', 'config', 'credentials', 'dids', 'export', 'heartbeats', 'identities', 'import', 'lifetime_exceptions', 'locks', 'meta', 'ping', 'redirect', 'replicas', 'requests', 'rses', 'rules', 'scopes', 'subscriptions'].

auditor

  • cache: Path to the folder to store the rucio-auditor cache. Example: /opt/rucio/auditor-cache. No default.

  • results: Path to the folder to store the rucio-auditor results. Example: /opt/rucio/auditor-results. No default.

  • threshold: (Optional) Floating number used in a sanity check, comparing the number of entries with the total number of files on the RSE:

    if len(dark_replicas) > threshold * usage['files']
    if len(lost_replicas) > threshold * usage['files']

    Default: 0.2.

bb8

  • dump_production_day: (Optional) Day of the week of the most recent dump. Values: {Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday}. Default: None.
  • dump_url_template: (Optional) URL of the template (structure) of a dump. Default: http://rucio-analytix.cern.ch:8080/LOCKS/GetFileFromHDFS?date=${date}&rse=${rse}.

bootstrap

  • gss_email: (Optional) Email of the Kerberos auth method which identity is specified in gss_identity.
  • gss_identity: (Optional) Identity of the Kerberos auth method.
  • saml_email
  • saml_id
  • ssh_email: (Optional) Email of the SSH auth method which identity is specified in ssh_identity. Default: ph-adp-ddm-lab@cern.ch.
  • ssh_identity: (Optional) SSH auth using an RSA key. Default: ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq5LySllrQFpPL614sulXQ7wnIr1aGhGtl8b+HCB/0FhMSMTHwSjX78UbfqEorZV16rXrWPgUpvcbp2hqctw6eCbxwqcgu3uGWaeS5A0iWRw7oXUh6ydnVy89zGzX1FJFFDZ+AgiZ3ytp55tg1bjqqhK1OSC0pJxdNe878TRVVo5MLI0S/rZY2UovCSGFaQG2iLj14wz/YqI7NFMUuJFR4e6xmNsOP7fCZ4bGMsmnhR0GmY0dWYTupNiP5WdYXAfKExlnvFLTlDI5Mgh4Z11NraQ8pv4YE1woolYpqOc/IMMBBXFniTT4tC7cgikxWb9ZmFe+r4t6yCDpX4IL8L5GOQ== ddmlab
  • userpass_email: (Optional) Email of the root account which name is specified in userpass_identity. Default: ph-adp-ddm-lab@cern.ch.
  • userpass_identity: (Optional) Name of the root account. Default: ddmlab.
  • userpass_pwd: (Optional) Password of the root account which name is specified in userpass_identity. Default: secret.
  • x509_email: (Optional) Email of the X.509 identity specified in x509_identity. Default: ph-adp-ddm-lab@cern.ch.
  • x509_identity: (Optional) Identity of the X.509 certificate. Default: emailAddress=ph-adp-ddm-lab@cern.ch,CN=DDMLAB Client Certificate,OU=PH-ADP-CO,O=CERN,ST=Geneva,C=CH.

c3po

  • algorithms: (Optional) Placement algorithm.

    Only allows multiple algorithms if the dry_run mode is enabled (separated by commas).

    Values: {simple, t2_free_space, t2_free_space_only_pop, t2_free_space_only_pop_with_network}.

    Default: None.

  • ca_cert: (Optional) Path of the certificate for Elasticsearch. Default: False.

  • elastic_index: (Optional) Index (database) to use in Elasticsearch.

  • elastic_pass: (Optional) Password of the username defined in elastic_user to authenticate to Elasticsearch. No default.

  • elastic_url: URL of Elasticsearch. Example: http://aianalytics01.cern.ch:9200.

  • elastic_user: (Optional) Username to authenticate to Elasticsearch. No default.

  • placement_algorithm

  • redis_host: URL of the Redis database.

  • redis_port: Port of the Redis database defined in redis_host.

c3po-popularity

  • elastic_url

c3po-site-mapper

  • ddm_url: URL for DDM. Example: http://atlas-agis-api.cern.ch/request/ddmendpoint/query/list/?json.
  • panda_url: URL for PanDA. Example: http://atlas-agis-api.cern.ch/request/pandaqueue/query/list/?json.

c3po-workload

  • panda_url: URL for PanDA. Example: http://bigpanda.cern.ch/jobs/?category=analysis&jobstatus=running.
  • window: Integer.

c3po-network-metrics

  • prefix:
  • redis_host: URL of the Redis database.
  • redis_port: Port of the Redis database defined in redis_host.

cache

  • url: (Optional) URL of the cache. Default: 127.0.0.1:11211.
  • use_external_cache_for_auth_tokens: (Optional) if True, use remote cache provider for auth tokens. If False, use a private in-memory cache. Default: False

common

  • extract_scope: (Optional) Default: atlas.

  • logdir: Path of the directory for logs. Contains auditor.log.

  • logformat: (Optional) Formatter of the log. See the logging formatter documentation

  • loglevel: (Optional). Set the root logger level to the specified level.

    Values: {'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}. See documentation for logging levels

    Default: DEBUG. For rucio-auditor default is WARNING.

  • mailtemplatedir: (Optional) Path of the folder with mail templates (.tmpl). Example: /opt/rucio/etc/mail_templates.

  • multi_vo: (Optional) Default: False.

conveyor

  • allow_user_oidc_tokens: (Optional) Boolean. Default: False.
  • bring_online: Integer, bring online timeout. Default: 43200.
  • cacert
  • cache_time: (Optional) Integer, expiration time in seconds passed to the dogpile system. Default: 600.
  • failover_scheme: Failover schemes. Default: None.
  • filter_transfertool: (Optional) Default: None.
  • ftshosts: URL of the File Transfer Service (FTS) hosts (separated by commas). Example: https://fts3-pilot.cern.ch:8446, https://fts3-pilot.cern.ch:8446.
  • globus_auth_app: (Optional) Default: None.
  • max_time_in_queue: (Optional) (separated by commas). Default: {}.
  • poll_timeout: Float, timeout. Default: None.
  • queue_mode: (Optional) Values: {'strict', 'default'}Default: default.
  • request_oidc_audience: (Optional). Default: fts:example.
  • request_oidc_scope: (Optional). Default: fts:submit-transfer.
  • scheme: (Optional) Schemes to process (separated by commas). Default: None.
  • submit_timeout: (Optional) Timeout. Default: None.
  • transfertool: (Optional) Default: None.
  • transfertype: (Optional). Values: {bulk, single}. Default: single.
  • usercert: Path to the certificate for the FTS3 implementation of a Rucio transfertool. Default: None.
  • user_activities: (Optional) Default: ['user', 'user_test'].
  • user_transfers: (Optional) Default: None.
  • using_memcache: (Optional) Boolean. Default: False.
  • webdav_transfer_mode: (Optional). Default: None.

core

  • default_mail_from: (Optional) Default email. Default: None.
  • geoip_ignore_error: (Optional) Whether to ignore errors when downloading and parsing the GeoIP database. Otherwise exceptions will be raised for errors. Boolean. Default: True.
  • geoip_licence_key: (Optional) License key for GeoLite2. Get a free licence key at the signup page. Default: NOLICENCE.
  • use_temp_tables: (Optional) Use Rucio with temporary table workflows. Default: False.

client

  • account: Rucio account. Example: root.
  • auth_host: URL of the host of the rucio authentication server. Example: https://rucio-auth-prod.cern.ch:443.
  • auth_token_file_path: (Optional) If token file path is defined in the rucio.cfg file, use that file. Currently this prevents authenticating as another user or VO. Default: None.
  • auth_type: Type of authentication in rucio. Values: {userpass, x509, x509_proxy, gss, ssh, saml, oidc}.
  • ca_cert: Path of the cert file for HTTPS. Example: /opt/rucio/etc/web/ca.crt.
  • client_cert: Path of the X.509 client cert file. This can be overwitten by the RUCIO_CLIENT_CERT environment variable. Example: /opt/rucio/etc/web/client.crt.
  • client_key: Path of the X.509 client key file for the cert defined in client_cert. This can be overwitten by the RUCIO_CLIENT_KEY environment variable. Example: /opt/rucio/etc/web/client.key.
  • client_x509_proxy: Path of the X.509 client proxy. Mandatory if auth_type = x509_proxy.
  • oidc_audience: (Optional) Only used if auth_type = oidc. Default: None.
  • oidc_auto: (Optional) Boolean. Only used if auth_type = oidc. Default: False.
  • oidc_issuer: (Optional) Only used if auth_type = oidc. Default: None.
  • oidc_password: (Optional) Only used if auth_type = oidc. Default: None.
  • oidc_refresh_lifetime: (Optional) Only used if auth_type = oidc. Default: None.
  • oidc_scope: (Optional) Only used if auth_type = oidc. Default: openid profile.
  • oidc_username: (Optional) Only used if auth_type = oidc. Default: None.
  • password: Password of the user specified in username. Mandatory if auth_type = userpass or auth_type = saml.
  • protocol_stat_retries: (Optional) Integer, number of retries if stat file fails. The time of the retries are: 1s, 2s, 4s, 8s, 16s, 32s later. Default: 6.
  • request_retries: (Optional) Integer, number of retries if an unauthorized error is returned. Default: 3.
  • rucio_host: URL of rucio host. Example: https://rucio-server-prod.cern.ch:443.
  • ssh_private_key: Path of the SSH private key. Mandatory if auth_type = ssh. Example: $HOME/.ssh/id_rsa.
  • username: Mandatory if auth_type = userpass or auth_type = saml.
  • vo: (Optional) VO name. Default: def.

credentials

  • gcs: (Optional) Path of the Google Cloud Storage credentials. Default: /opt/rucio/etc/google-cloud-storage-test.json.
  • signature_lifetime: (Optional) ?. Default: 600.

database

download

  • transfer_speed_timeout: (Optional) Minimum allowed average transfer speed (in KBps). Default: 500. Used to dynamically compute the timeout if --transfer-timeout not set. Is not supported for --pfn.
  • transfer_timeout: (Optional) Transfer timeout (in seconds). Default: computed dynamically from --transfer-speed-timeout. If set to any value >= 0, --transfer-speed-timeout is ignored.

es-atlas

  • ca_cert: (Optional) Path of the certificate for Elasticsearch. No default.
  • password: (Optional) Password of the username defined in username to authenticate to Elasticsearch. No default.
  • url: (Optional) URL of Elasticsearch. Example: http://aianalytics01.cern.ch:9200. No default.
  • username: (Optional) Username to authenticate to Elasticsearch. No default.

hermes

  • elastic_endpoint: (Optional) URL of Elasticsearch. Example: http://aianalytics01.cern.ch:9200. Mandatory if elastic is specified in services_list.
  • influxdb_endpoint: (Optional) URL of InfluxDB. Mandatory if influx is specified in services_list.

importer

  • attr_sync_method: (Optional) Values: {append, edit, hard}. Default: edit.
  • rse_sync_method: (Optional) Values: {append, edit, hard}. Default: edit.

injector

  • adler32
  • bytes
  • file
  • md5

lifetime

  • directory: (Optional) Path to the policies directory with JSON files named config_DTYPE.json, where DTYPE is a value in {data, mc, valid, other}. Default: /opt/rucio/etc/policies.

logging

  • CFG_OPTION: (Optional) ?. Default: None.

messaging-cache

  • account
  • broker_virtual_host: (Optional) ?
  • brokers: Default message broker name for rucio-cache-client. Ignored if rucio-cache-client executed with --broker.
  • destination: Default message broker topic for rucio-cache-client. Ignored if rucio-cache-client executed with --destination.
  • port
  • ssl_cert_file: Default certificate file for rucio-cache-client. Ignored if rucio-cache-client executed with --certificate.
  • ssl_key_file: Default certificate key file for rucio-cache-client. Ignored if rucio-cache-client executed with --certificate-key.
  • voname

messaging-fts3

  • broker_virtual_host: (Optional) ?. No default.
  • brokers: Brokers separated by commas. Example: dashb-test-mb.cern.ch.
  • destination: Name of the destination topic. Example: /topic/transfer.fts_monitoring_queue_state.
  • nonssl_port: (Optional) Port of the broker if use_ssl is not set.
  • password: (Optional) Password of the username. Only used if use_ssl is not set. No default.
  • port
  • ssl_cert_file: (Optional) Path of the certificate file. No default.
  • ssl_key_file: (Optional) Path of the certificate key file defined in ssl_cert_file. No default.
  • use_ssl: (Optional) Boolean. Default: True.
  • username: (Optional) Username of the broker. Only used if use_ssl is not set. No default.
  • voname

messaging-hermes

  • broker_virtual_host: (Optional) No default.
  • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.
  • destination: Name of the destination topic. Example: /topic/rucio.events.
  • email_from: Example: Rucio <spamspamspam@cern.ch>.
  • email_test: Example: spamspamspam@cern.ch.
  • nonssl_port: (Optional) Port of the broker if use_ssl is not set.
  • password: (Optional) Password of the username. Mandatory if use_ssl is not set. No default.
  • port: (Optional) Port of the broker if use_ssl is set.
  • ssl_cert_file: (Optional) Path of the certificate file. No default. Mandatory if use_ssl is set.
  • ssl_key_file: (Optional) Path of the certificate key file defined in ssl_cert_file. No default. Mandatory if use_ssl is set.
  • use_ssl: (Optional) Boolean. Default: True.
  • username: (Optional) Username of the broker. Mandatory if use_ssl is not set. No default.
  • voname

metadata

  • plugins: (Optional) Metadata handler modules separated by commas. Default: rucio.core.did_meta_plugins.json_meta.JSONDidMeta.

monitor

  • enable_metrics: (Optional) Enable statsd metrics. Boolean. Default: False.
  • carbon_server: (Optional) Hostname or IP address of the statsd server. Default: localhost
  • carbon_port: (Optional) Port of the statsd server. Default: 8125.
  • user_scope: (Optional) Prefix to distinguish and group stats from an application or environment. Default: rucio.
  • metrics_port: (Optional) Port of Prometheus Python Client. Default: 8080.

nagios

  • fts_servers
  • proxy
  • rfcproxy

nongrid-trace

  • broker_virtual_host: (Optional) ?. No default.

  • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.

  • logformat: (Optional) Formatter of the log. See logging documentation.

  • loglevel: (Optional) Set the root logger level to the specified level.

    Values: {'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}. See logging documentation.

    Default: DEBUG.

  • password: Password of the username.

  • topic: Name of the destination topic.

  • tracedir: (Optional) Path of the directory for traces. Default: /var/log/rucio.

  • username: Username of the broker.

oidc

  • admin_issuer: Example: wlcg.
  • default_jwt_refresh_lifetime: (Optional) Integer. Default: 96.
  • exchange_grant_type: (Optional) Default: urn:ietf:params:oauth:grant-type:token-exchange.
  • expected_audience: (Optional) Default: rucio.
  • expected_scope: (Optional) Default: openid profile.
  • idpsecrets: Path of the idpsecrets JSON. Example: /opt/rucio/etc/idpsecrets.json.

permission

  • policy: (Optional) Permission policy. Values: {atlas, belleii, cms, generic, generic_multi_vo}. Default: generic.

policy

  • lfn2pfn_algorithm_default: (Optional) Default algorithm name for LFN2PFN translation for this server. Default: hash.
  • package
  • package-VO
  • permission: Same as policy/permission.
  • schema
  • scratchdisk_lifetime: (Optional) Integer. Default: 14.
  • support: (Optional) Contact information.
  • support_rucio: (Optional) Rucio contact information. Default: https://github.com/rucio/rucio/issues.

saml

  • config_path: Path to the SAML config folder. Example: /opt/rucio/lib/rucio/web/ui/common/saml/.

test

  • cacert: Path of the CA certificate for tests. Example: /opt/rucio/etc/web/ca.crt
  • usercert Path of the user certificate for tests. Example: /opt/rucio/etc/web/usercert.pem
  • userkey: Path of the user certificate key for tests.

trace

  • broker_virtual_host: (Optional) No default.

  • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.

  • logformat: (Optional) Formatter of the log. See logging documentation.

  • loglevel: (Optional) Set the root logger level to the specified level. Values: {'NOTSET', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'}. See logging documentation.

    Default: DEBUG.

  • password: Password of the username.

  • port: (Optional) Port of the broker. Example: 61013.

  • topic: Name of the destination topic. Example: /topic/rucio.tracer.

  • tracedir: (Optional) Path of the directory for traces. Default: /var/log/rucio/trace.

  • username: Username of the broker.

tracer-kronos

  • broker_virtual_host: (Optional) No default.
  • brokers: Brokers separated by commas. Example: atlas-test-mb.cern.ch.
  • chunksize: Integer
  • excluded_usrdns: Separated by commas. Example: CN=proxy,CN=Robot: Ganga Robot,CN=722147,CN=gangarbt,OU=Users,OU=Organic Units,DC=cern,DC=ch.
  • password: (Optional) Password of the username. Mandatory if use_ssl is not set. No default.
  • port: Port of the broker.
  • prefetch_size: activemq.prefetchSize, see activemq documentation
  • queue: The topic or queue to subscribe to. Example: /queue/Consumer.kronos.rucio.tracer.
  • reconnect_attempts: Maximum attempts to reconnect. Integer. Example: 100.
  • ssl_cert_file: (Optional) Path of the certificate file. No default. Mandatory if use_ssl is set.
  • ssl_key_file: (Optional) Path of the certificate key file defined in ssl_cert_file. No default. Mandatory if use_ssl is set.
  • subscription_id: A unique id to represent the subscription. Example: rucio-tracer-listener.
  • use_ssl: (Optional) Boolean. Default: True.
  • username: (Optional) Username of the broker. Mandatory if use_ssl is not set. No default.

transmogrifier

  • maxdids

upload

  • transfer_timeout: (Optional) Transfer timeout (seconds, integer). Default: 360.

webui

  • auth_issuer: (Optional) Mandatory if auth_type = oidc. No default.
  • auth_type: (Optional) Preferred server side config for webui authentication. Values: {oidc, None}. Default: None.
  • usercert
  • urls: A CSV specifying urls of Rucio WebUI 2.0 clients. Required for correctly handling pre-flight CORS requests.

Rucio configuration table

  • Checked only if Section/Option pair is not in the Configuration File

Updating

The table can be updated with the Rucio Client, using either

  • rucio-admin config set [section] [option] [value]
  • rucio.ConfigClient().set_config_option([section], [option], [value])

These changes take impact immediately without requiring a restart. Using the client only updates the Configuration Table and does not overwrite anything in the Configuration File, and thus will not change anything if there a setting already specified in the Configuration File.

Options and Defaults

automatix

  • account: (Optional) Account to use. Default: root.
  • dataset_lifetime: (Optional). Default: 0.
  • did_prefix: (Optional) Default: .
  • DIDTYPE_pattern: (Optional) Separated by separator char. No default.
  • rses: Separated by commas.
  • scope: (Optional) Default: test.
  • separator: (Optional) Separator char. No default.
  • set_metadata: (Optional) Default: True.
  • sites: Separated by commas (to be deprecated, please use rses).
  • sleep_time: (Optional) Integer. Default: 30.

clientcachemap

  • client_location['site']

conveyor

  • activity-source-strategy: (Optional) Default: {}.
  • default-source-strategy: (Optional) Default: orderly.

hermes

  • services_list: List of services separated by commas. Values: {activemq, elastic, influx}.

kronos

  • bad_files_patterns: (Optional) Patterns (regular expression) separated by commas for bad files. Default: [].

lifetime_model

  • approvers_email: (Optional) Separated by commas. Default: [].

reaper

  • auto_exclude_threshold: (Optional) Number of service unavailable exceptions after which the RSE gets temporarily excluded. Integer. Default: 100.
  • auto_exclude_timeout: Timeout for temporarily excluded RSEs. Integer. Default: 600.
  • max_deletion_threads_HOSTNAME: (Optional) Max number of deletion threads (integer). If nb_workers_by_hostname is also not defined, default: 5.
  • max_evaluator_backlog_count: (Optional) Integer. Default: None.
  • max_evaluator_backlog_duration: (Optional) Minutes (integer). Default: None.
  • nb_workers_by_hostname: (Optional) Integer. Default: 5.

root-proxy-internal

  • client_location['site']

rules

  • apply_rule_max_partition_size: (Optional) Integer . Default: 2000.
  • force_epoch_when_detach: (Optional) Purge setting of the rule. Boolean. Default: False.
  • use_new_rule_algorithm: (Optional) Boolean. Default: False.

subscriptions

  • keep_history: (Optional) Boolean. Default: False.
  • reevaluate_dids_at_close: (Optional) Flag to reevaluate the DID against all the subscriptions when the DID is closed. Boolean. Default: False.

transfers

  • hop_penalty: (Optional) Penalty to be applied to each further hop. Integer. Default: 10.
  • multihop_tombstone_delay: Seconds (integer). Default: 7200.
  • use_multihop: Boolean. Default: False.
  • fts3tape_metadata_plugins: (Optional) Plugins to use with FTS3 to include archive metadata in the transfer process. List[String]. Default: None.
  • metadata_byte_limit: (Optional) Limit applied to archive_metadata during a transfer. Only used with archive metadata plugins using FTS3. Integer. Default None.

virtual_placement

  • vp_endpoint: (Optional) Virtual Placement server. Once VP is integrated in Rucio it won't be needed. Default: .

vo-map

  • VO: (Optional) Internal short VO name. No default.

RSE attributes

The RSE Attributes are set separately using rucio.RSEClient or rucio-admin, and only contains information about the specific RSE's for the Rucio instance. Read more about RSEs here and how to set them up here. The RSE Attributes are set seperately using rucio.RSEClient or rucio-admin, and only contains information about the specific RSE's for the Rucio instance. Read more about RSEs here and how to set them up here.

  • associated_sites: Separated by commas. Default: None.
  • auto_approve_bytes: Auto Approve Limit. Example: 500GB. No default.
  • auto_approve_files: No default.
  • available_for_multihop: Boolean. If True, allow to use this RSE as an intermediate hop in a multi-hop transfer. Default: False.
  • block_manual_approval: Boolean. Default: False.
  • country: No default.
  • greedyDeletion: Boolean. Default: False.
  • hop_penalty: Integer. Cost of passing via this RSE in multi-hop transfers. Overrides the global transfers/hop_penalty configuration value for this particular RSE. Has only meaning if available_for_multihop attribute is True on the RSE. No default value.
  • istape: Boolean. Default: False.
  • mock: Boolean. Default: False.
  • naming_convention: Default: None.
  • physgroup: Default: .
  • quota_approvers: Separated by commas. Default: None.
  • restricted_read Boolean. If True, only allow transfers from this RSE if started by an account with admin privileges. Default: False
  • restricted_write Boolean. Same as restricted_read, but for transfers towards this RSE. Default: False
  • rule_approvers: Separated by commas. No default.
  • rule_deleters: Separated by commas. No default.
  • site
  • skip_upload_stat: Boolean. Default: False.
  • source_for_total_space: Default: storage.
  • source_for_used_space: Default: storage.
  • staging_buffer
  • tier: Datacenter TIER. Integer (from 1 to 4). No default.
  • type: Values: {LOCALGROUPDISK, LOCALGROUPTAPE, GROUPDISK, SCRATCHDISK, MOCK, TEST, DATADISK}. Default: .
  • verify_checksum: Boolean. No default.