BASH_as_JCL

BASH as JCL

(quoted from Bash definition in Wikipedia)

Bash is a command processor that typically runs in a text window, where the user types commands that cause actions. Bash can also read commands from a file, called a script. Like all Unix shells, it supports filename wild-carding, piping, here documents, command substitution, variables and control structures for condition-testing and iteration. The keywords, syntax and other basic features of the language were all copied from sh. Other features, e.g., history, were copied from csh and ksh. Bash is a POSIX shell, but with a number of extensions.

The name itself is an acronym, a pun, and a description. As an acronym, it stands for Bourne-again shell, referring to its objective as a free replacement for the Bourne shell. As a pun, it expressed that objective in a phrase that sounds similar to born again, a term for spiritual rebirth. The name is also descriptive of what it did, bashing together the features of sh, csh, and ksh.

Bash file can be used as JEM JCL, adding metadata like comments to the script. The metadata could be the JEM properties, data descriptions and locks.

Special Tokens

All necessary information, which must be read by JEM to execute the script, are passed by comments inside the script. But JEM needs to be aware that what is managing is a Bash file. For this reason, all metadata written as comment must be included in 2 token (XML like):

#
# <JEM-BASH>
# jem.job.name=JOB-NAME
# jem.job.affinity=JOB-AFFINITY
# jem.job.user=JOB-USER
# </JEM-BASH>
#
....
...

JEM is able to read all comments, reading and using all information between <JEM-BASH> and </JEM-BASH> token. All properties are managed at row level, that means the JEM reads and interpret every line (not multi line properties). BE aware that Bash file will be executed inside of ANT file.

Properties definition

To use Bash file inside JEM, the file must contain the mandatory and optional properties that JEM needs, described in JCL Reference, as following:

• Job Name is optional string property, called for Bash jem.job.name . If missing, the file name will be used.
• Environment is optional string property, called for Bash jem.job.environment.If missing, the JEM node environment definition is used.
• Domain is optional string property, called for Bash jem.job.domain. If missing, default value (***) will be used.
• Affinity is optional string property, called for Bash jem.job.affinity. If missing, default value (***) will be used.
• User is optional string property, called for Bash jem.job.user. If missing, default value (null) will be used. An exception occurs if the user, who submitted the job, is not authorized to change the user job execution.
• Locking Scope is optional string property, called for Bash jem.job.lockingScope. If missing, default value (job) will be used. If the value is not equals to one the possible values (job, step , task), an exception occurs.
• Hold is optional boolean property, called for Bash jem.job.hold. If missing, default value (false) will be used.
• Priority is optional integer property, called for Bash jem.job.priority. If missing, default value (10) will be used (highest priority is 1).
• Memory is optional integer property, called for Bash jem.job.memory. If missing, default value (128) will be used. Be aware the unit is MegaBytes.
• Classpath is optional string property, called for Bash jem.job.classPath. Defining a classpath, it will be able to import ANT task definitions of 3rd parties. The value is a string and the files are separated by semicolons ;. If the file doesn't represent an absolute path, JEM will use relative position from jem.classpath folder. You could use variables that JEM substitutes in string value. If missing, default value (null) will be used. The paths will be added at the end of default classpath, build by JEM.
• Prior Classpath is optional string property, called for Bash jem.job.priorClassPath. Defining a classpath, it will be able to import ANT task definitions of 3rd parties. The value is a string and the files are separated by semicolons ;. If the file doesn't represent an absolute path, JEM will use relative position from jem.classpath folder. You could use variables that JEM substitutes in string value. If missing, default value (null) will be used. The paths will be added at the beginning of default classpath, build by JEM.
• Emails Notification is optional string property, called for Bash jem.job.emailsNotification. If missing, default value (null) will be used.

Here is the sample about the properties definition, in Bash file as comments:

#
# <JEM-BASH>
# jem.job.name=JOB1
# jem.job.environment=ENV1
# jem.job.domain=domain
# jem.job.affinity=classA
# jem.job.priority=99
# jem.job.user=newUSER
# jem.job.lockingScope=task
# jem.job.hold=true
# jem.job.memory=1024
# jem.job.classPath=/test/ant-taskdefs-lib.jar
# jem.job.priorClassPath=/test/commons.jar
# jem.job.emailsNotification=m1@pepstock.org;m2@pepstock.org
# </JEM-BASH>
#

Data Descriptions and Datasets in BASH

Data descriptions and datasets are implemented by properties, together with all the other properties (between <JEM-BASH> and </JEM-BASH> token). Data descriptions could be defined as following:

# <JEM-BASH>
# ...
# jem.dataDescription.name-of-data-description=SYSOUT,DISP=NEW
# jem.dataDescription.name-of-data-description=DSN=datasetName,DISP=NEW/MOD/OLD/SHR
# jem.dataDescription.name-of-data-description=DSN=gdgName,DISP=NEW/MOD/OLD/SHR
# jem.dataDescription.name-of-data-description=DSN=@@temp,DISP=NEW
# ...
# </JEM-BASH>

Data description needs a mandatory name, which must be inside of key name. All properties name for data descriptions, start with jem.dataDescription..

This name is used inside the Bash file to access to datasets therefore this name must be unique in the task definition. The SYSOUT attribute is optional one that if there is means that the data description is a sysout. The DISP attribute is optional and must be a valid value, one of the following string enumeration: NEW, MOD, OLD or SHR.

Datasets could be defined as following:

DSN=dsn, DISP=NEW/MOD/OLD/SHR

Datasets could represent:

• file name (absolute or relative path), composed by properties if necessary. If it's a relative path, JEM adds the content of jem.data variable, used to identify the global file system with all data.
• GDG file name, following the same rules of a normal file name (see previous item).
• Temporary prefix file name
• if the text assigned to the property doesn't match with the abpve description, a INLINE file will be created, with the content of the property

Using data descriptions, the programmer is able to develop own job batch programs, avoiding strong references with physical files but logical ones, by environment variables. To reduce the risk to use variables already used on system, all variables names have DD_prefix.

Here is an example, how to access to data description:

# <JEM-BASH>
# ...
# jem.dataDescription.INPUT=DSN=/gdg/jemtest(0),DISP=SHR
# jem.dataDescription.OUTPUT=DSN=/gdg/jemtest(+1),DISP=NEW
# ...
# </JEM-BASH>

cp ${DD_INPUT} ${DD_OUTPUT}

Locks in BASH

Locks are implemented by specific propertie. They could be defined as following:

# <JEM-BASH>
# ...
# jem.locks=LABEL1, LABEL2
# ...
# </JEM-BASH>

Lock names are defined a value, comma-separated, of a property named jem.locks. These names are used to create a lock inside the JEM environment in exclusive. This name must be unique in the task definition.