swf-data-agent

About

This repository is for development of the swf-data-agent, which is the central data handling agent within the swf testbed.

Functionality:

• It receives the MQ messages from the swf-daqsim-agent and takes actions according to the messages.
• With respect to Rucio, it created new run datasets, and manages Rucio STF subscriptions in general.
• It sends messages to the swf-processing-agent for data processing and to the swf-fastmon-agent to signal new STF availability.
• It will eventually also have a 'watcher' role to identify and report stalls or anomalies.

Data Transport

The agent does not normally actuates the data transfer by itself. It instead relies on Rucio for that function, and Rucio in turn operates the FTS system to move data. The transport layer for the FTS is XRootD.

Rucio

Useful information for setting up the components such as Rucio can be found in the Startup Guide for NP PanDA & Rucio at BNL.

Quick summary:

• Set up is done using CVMFS (hence it must be mounted on the host)
• Setting up: source /cvmfs/eic.opensciencegrid.org/rucio-clients/alrb_setup.sh
• There will be error/warning messages during the setup that can be ignored in most cases.
• Specifying a valid Rucio user ID is important, there will be a prompt for that.
• The user needs a grid certificate and the key, with which to run voms-proxy-init.
• Access can be confirmed by using the command rucio whoami.
• $RUCIO_ACCOUNT contains the current user ID for Rucio.
• An example of a simple command listing Rucio items for a user: rucio list-dids --filter 'type=all' user.potekhin:*

Useful examples can be found on the official Rucio documentation pages.

The example below lists the available RSEs:

$ rucio list-rses
BNL_PROD_DISK_1
BNL_TEST_TAPE
DAQ_DISK_1
E1_BNL_DISK_1
E1_JLAB_DISK_1
JLAB_DISK_1

More examples:

rucio --help # a useful CLI help is available
rucio add-dataset user.potekhin:test # add dataset
rucio upload --rse BNL_PROD_DISK_1 --scope user.potekhin ./README.md # upload to a storage endpoint
rucio download user.potekhin:user.potekhin.74311a67-6e47-467d-b44a-244eac13c8be.log # download a container
rucio list-dids --filter 'type=FILE' user.potekhin:*
# This needs to be used prior to deleting rules, if unsure -
rucio list-rules --account my_account
# get useful info about an item:
rucio list-file-replicas user.potekhin:t1.txt

The way to delete files in Rucio is to delete associated rules.

# NB. Not every user has the privilege to do this, some accounts do e.g. swf.
rucio update-rule 680886366a584dcfb79b79c3f47af12d   --lifetime -1
# NB. The change won't be instantaneous.

Currently the CLI commands need to be run with the local Rucio imports deactivated, because of the clash with the global ATLAS-based setup.

For running tests and other Python functions which depend on the common SWF packages, please use a setting similar to this one:

export RUCIO_COMMS_PATH=/eic/u/eicmax/testbed/swf-common-lib/

And,

voms-proxy-init --cert ./mycert.pem --key ./mykey.pem

XRootD

XRootD is a high-performance, scalable, and fault-tolerant data access framework widely used in scientific computing, especially in high-energy physics. An XRootD server provides remote access to files and datasets over the network using the XRootD protocol. It supports efficient data transfers, authentication, authorization, and can be configured as a standalone server or as part of a distributed cluster for load balancing and redundancy. It is the data transport mechanism used in the test bed, and is actuated by the FTS system used for data distribution in Rucio.

The XRootD service been installed on the host pandaserver02.sdcc.bnl.gov in a standard type of a "standalone" configuration, meaning it's not a cluster. More information is contained in the README file in the xrootd folder.