Skip to main content

39.0.0

Changes to the RSE protocol priorities​

Summary​

This release contains the second part of a long-term effort to change how the RSE protocol priorities are interpreted. Rucio operators should review these instructions and plan accordingly.

  • Prior to Rucio 32.0.0, priority 0 meant disabled, 1 meant top-priority, 2 meant second-priority, etc.
  • Between Rucio 32.0.0 and 39.0.0 (not inclusive), priority 0 and/or None meant disabled, 1 meant top-priority, 2 meant second-priority, etc.
  • Since Rucio 39.0.0, priority None means disabled, 0 means top-priority, 1 means second-priority, etc.

Client Backward Compatibility​

Operators should be advised that the configuration that is applied affects which version of the Rucio client are supported.

  • When using 0 to denote disabled, 1 to denote top-priority, and None is not used at all, Rucio client earlier than 39.0.0 are supported. Client 39.0.0 and later may misbehave for some operations.
  • When using None to denote disabled, 1 to denote top-priority, and 0 is not used at all, Rucio client earlier than 32.0.0 may misbehave for some operations. Client 32.0.0 and later are supported.
  • When using None to denote disabled and 0 to denote top-priority, Rucio client earlier than 39.0.0 may misbehave for some operations. Client 39.0.0 and later are supported.

For established Rucio instances, it is generally recommended to choose the second option (i.e. not use 0 at all). The instructions that follow will assume as such. The third option is preferable if there are no concerns about backward compatibility.

Upgrade Instructions​

  1. (Optional) Review any systems that automatically configure RSEs or synchronise the Rucio instance with some external service.
  2. Apply the database schema upgrade.
  3. Update all existing RSE protocols in the database. This is NOT automatically done by alembic. It may look something like the following: 
    UPDATE rse_protocols SET read_lan = NULL WHERE read_lan = 0;
    UPDATE rse_protocols SET write_lan = NULL WHERE write_lan = 0;
    UPDATE rse_protocols SET delete_lan = NULL WHERE delete_lan = 0;
    UPDATE rse_protocols SET read_wan = NULL WHERE read_wan = 0;
    UPDATE rse_protocols SET write_wan = NULL WHERE write_wan = 0;
    UPDATE rse_protocols SET delete_wan = NULL WHERE delete_wan = 0;
    UPDATE rse_protocols SET third_party_copy_read = NULL WHERE third_party_copy_read = 0;
    UPDATE rse_protocols SET third_party_copy_write = NULL WHERE third_party_copy_write = 0;
  4. Follow other instructions and procedures to upgrade Rucio.
  5. (Optional) Make a reminder to start using 0 to denote top-priority at some point in the future when Rucio clients earlier than 39.0.0 are no longer used.

Upgrade Instructions​

This major release requires a database schema upgrade. Please consult the documentation about upgrading your database schema.

The following changes are necessary and are covered by the alembic upgrade head command:

  1. Add dids_opendata_record table (Alembic revision a7e76cf4881d)

  2. Add deleted_at columns to rules, rules_hist_recent and rules_history tables (Alembic revision 0548e199f091)

  3. Update column defaults in rse_protocols table (Alembic revision 3b943000da18)

General​

Enhancements​

  • Authentication & Authorisation: Enable OIDC authentication in development environment #8058
  • Database: Database: Improve Alembic migrations logging #8150
  • Database: Add rules deleted_at column #8242
  • Documentation: Rucio Python Client docstrings enhancement (Credentialclient) #7842
  • Monitoring & Traces: Kronos does not update the access_cnt for files #6832
  • Monitoring & Traces: TIME_ENTRY pattern doesn't fully cover traceTimeentry #8139
  • Support Open Data record ids #8080
  • Open Data CLI record id cannot be updated #8141
  • Dependency upgrade for Rucio 39 #8165
  • Add always a new line at the end of the generated webui_version. #8258
  • Protocols: Make 0 the top protocol priority #6312
  • Release management: Dependencies: deprecate setup.py and move to pyproject.toml #6971
  • Testing: Remove bare except clauses #7307
  • Testing: Remove client_syntax checks and ignoretool usage #7677
  • Testing: Ensure Python files end with a newline #8218
  • Transfers: Make automatix.rses an RSE expression #8064
  • Transfers: Group bulk transfers by authentication method #8199

Bugs​

  • Core & Internals: update_rules_for_lost_replica: bytes from lock might be None, but are always passed to the account counter's decrease #8174
  • Core & Internals: core/exporter functions use transactional_session, but they are just read operations #8177
  • Docker & Kubernetes: Docker & Kubernetes: Update the oracle container in the docker-compose #7918
  • Automatix IndexError if no scopes to list #8166
  • Policies: Downloading does not work with hierarchical did names and extract_scope change #8223

Clients​

Enhancements​

  • Clients: Support rich client in opendata CLI #8023
  • Clients: Explicit rucio list subscriptions #8206