Skip to main content

32.0.0rc2

Release Candidate​

This is a release candidate! Only to be used in testing environments!

Upgrade Instructions​

This feature release requires a database schema upgrade. Please consult the documentation about upgrading your database schema.

The following changes are necessary and are covered by the alembic upgrade head command:

  1. Renaming index RULES_STUCKSTATE_IDX to RULES_STATE_IDX (Alembic revision a6eb23955c28)

  2. Adding last_processed_by and last_processed_at column to requests table (Alembic revision 295289b5a800)

  3. Drop of indices REPLICAS_TOMBSTONE_IDX and REPLICAS_RSE_ID_IDX (Alembic revision 27e3a68927fb)

  4. Only for ATLAS and CMS, because they run specific oracle database schemas which diverge from the standard SQLAlchemy schema in alembic: Some additional steps have to be performed during database schema update by these two communities. See the relevant commit message: https://github.com/rucio/rucio/commit/81bacae53dbe2486bcff87d55feb0dc65daa7ac0

Important changes​

  • Drop support for python < 3.9.

  • The temporary tables workflow is now mandatory. This feature was gradually introduced over many previous releases. First it was opt-in, since 1.31.0 it is an opt-out. Starting the current release, the old behavior is completely removed and the usage of temporary tables is mandatory. The update should be transparent, but requires the database user used by rucio to be allowed to create temporary tables in the database. The configuration switch [core]use_temp_tables=[True|False] can now be removed, as it does not do anything.

  • The configuration switch [transfers]use_multihop=[True|False] isn't used by rucio anymore and can be removed. Use [transfers]multihop_rse_expression=\<some_rse_expression\> to define which RSEs can be used for multihop. It defaults to the RSE-expression available_for_multihop=true, meaning that the update should be transparent for everybody. Communities who did not use multi-hop should not have any RSEs with this attribute set. Communities who used multi-hop already had this attribute set on all required RSEs, as it was mandatory for multi-hopping to work correctly. To explicitly disable multi-hopping, set [transfers]multihop_rse_expression= to an empty value.

  • The transfers(conveyor) daemons where subjected to a huge internal re-design. The --threads N CLI switch now has a slightly different meaning. Instead of having N threads getting data from database and working on the retrieved data, there will now be 1 thread getting data from the database + N threads working on that data. This change should have no particular impact.

Helm chart changes​

Note: Technically, rucio 32.0.0 can still be run with the last helm charts of the 1.31 line. You are not required to update the helm values at the same time you update rucio if you want to dissociate updating rucio from updating the helm charts. However, it's strongly advised to update helm charts rapidly after rucio update. We don't guarantee that this compatibility will be maintained in rucio 32.1.0+

breaking change: rework of the additionalSecrets section​

This change was anounced in previous rucio release 1.31.0. The current release makes the new behavior mandatory. Follow the instructions from the previous release to substitute the additionalSecrets sections in helm values with secretMounts sections.

breaking change introduced in the previous release: 1.31.x of rucio-servers helm chart is incompatible with old values​

If you are updating from a helm chart prior to 1.31.0, please follow the instruction from the previous rucio release 1.31.0 to upgrade the value of the rucio-server helm charts.

General​

Features​

  • Authentication & Authorisation: Replace all instances of X509 identities presented in the legacy format #6182
  • Core & Internals: make temporary tables mandatory #6238
  • Database: Exchange of function-based indices with normal index #6143
  • Release management: drop support for python 3.6, 3.7 and 3.8 #6251
  • Transfers: New dispatcher logic for conveyors #6190

Enhancements​

  • Authentication & Authorisation: Extend /auth/x509 endpoint for browser-based clients #6048
  • Core & Internals: Protocol Priority Renumbering #6083
  • Core & Internals: finalize migration to unified config table + config file #6107
  • Core & Internals: Customisable replica sorter algorithm #6114
  • Core & Internals: Support protocols with NULL priority #6201
  • Core & Internals: Remove prefixes for unicode string literals #6268
  • Messaging: Get rid of hermes-legacy #6213
  • Remove deprecated constructs from the typing module #6276
  • Replicas: remove REPLICAS_TOMBSTONE_IDX and REPLICAS_RSE_ID_IDX indexes #6287
  • Transfers: get rid of the use_multihop configuration option #6164
  • Transfers: internal refactoring in the conveyor #6207

Bugs​

  • Authentication & Authorisation: Restrict permissions for adding scopes or manipulating identities #6267
  • Metadata: set_metadata_bulk doesn't work for external postgres / mongo metadata plugins #6246
  • x509 authentication fails for baseclient.py #6294
  • Protocols: CMS LFN to PFN translation gives double hosts #6096

Clients​

Bugs​

  • Clients: broken stats retrieval in the xrootd rse protocol #6265