Workflow creation and assignment definition
As of HG1705 CMSWEB cycle, we've tighten the request spec validation for both creation and assignment. In summary they are:
- there is a specific set of parameters that must be provided during request creation (optional=False). There are also optional parameters that have a default value in ReqMgr;
- there is a specific set of parameters that must be provided during request assignment (assign_optional=False). There are also optional parameters that have a default value in ReqMgr;
- unknown arguments (not defined in the spec files) are no longer accepted/removed and the request will fail validation. Moreover, arguments are case-sensitive;
- there are a few arguments that are allowed both at creation and at assignment. Besides those, you must provide only creation or only assignment parameters, according to the operation you want to accomplish.
- inner Task/Step dictionary (for TaskChain/StepChain) are also validated and they have their own arguments definition.
The definition of these arguments follow a hierarchy where StdBase contains the base definition for all workflows. See 'getWorkloadCreateArgs' for creation global arguments, 'getWorkloadAssignArgs' for assignment arguments and 'getChainCreateArgs' for either Task and Step argument definition. All the other request types inherit from StdBase and they can override the base/global arguments definition with specific arguments or data type (as well as adding new arguments), see each of them according to the request type you want to create. In addition to that, data processing requests (requests that always have an input dataset) inherit the arguments definition from DataProcessing, which also has StdBase as a super class. This flowchart should make these dependencies clear:
So, for instance, if one wants to create a MonteCarloFromGEN request, you need to:
- get the StdBase create args
- update them with the DataProcessing create args
- and finally update those with the MonteCarloFromGEN create args. this gives you the final list of arguments allowed, their default values, their expected data type, their validate function and so so.
We've built a json file for each of the supported request types, where one can find the final list of supported arguments and their definition. A few remarks must be made though:
- The https://github.com/dmwm/WMCore/tree/master/src/python/WMCore/WMSpec/StdSpecs files contain the authoritative argument definition. These json templates will be kept up-to-date in the best effort.
- Some spec files contain floating arguments (arguments with an integer suffix that starts in 1 and go up to X), e.g. for TaskChain, StepChain and ReReco. More on their sections.
blah
TaskChain spec contains the TaskX (where X is an integer) "floating" argument, which correspond to each task that is chained in the request. For "educational" purposes, the template below will