Difference: BasicUsage (1 vs. 2)

Log onto a User Interface (UI) machine

Log onto the Grid (get authenticated on the Grid)

Get your jobs authenticated on the Grid

Prepare and submit your job

https://edms.cern.ch/file/454439//LCG-2-UserGuide.html

-- AndrasLaszlo - 17 Sep 2007

How to submit simple jobs onto the Grid

On this page, we intend to provide a brief introduction on simple job submission onto the Grid. One can also find a brief introductory material on the Grid homepage of RMKI.

• Log onto a User Interface (UI) machine.

After you have logged onto a UI machine, you are able to submit commands to the Grid.

• Log onto the Grid (get authenticated on the Grid).

This means getting a so called user proxy. Commands are:

> grid-proxy-init Here, you will be prompted for your grid password. Or:

> grid-proxy-init  -valid 4:00 This is the same, but the authentication will expire in 4 hours (default lifetime is 12 hours).

If you are member of more then one VO, you can choose between them by using the voms-proxy-init for logging in, instead. E.g.:

> voms-proxy-init  -voms hungrid Or:

> voms-proxy-init  -voms hungrid  -valid 4:00

To get information on your user proxy, you can use the commands grid-proxy-info or voms-proxy-info.

• Get your jobs authenticated on the Grid.

This means getting a so called job proxy. Commands are:

> myproxy-init Here, you will be prompted for your grid password, and to specify an additional password (for extra protection) attached to your so called job proxy, to be created.

> myproxy-init  -n This is the same, but you won't be asked to specify an additional password for protecting your job proxy.

Running myproxy-init is necessary when you are running long-term jobs. Having a job proxy ensures that your jobs still will be authenticated on the Grid, even though your user proxy (used to perform interactive Grid manupulations) may have had expired. You can get information on your job proxy by myproxy-info. You can destroy your job proxy by myproxy-destroy. The default lifetime of a job proxy is 168 hours.

Note: If you don't get a job proxy, you may not be able to retrieve your job outputs for long-term jobs!

• Prepare and submit your job.

The programme which you want to run on the Grid is called a job. These consist of some executable(s) and some input(s), which can be submitted to the Grid system. The result shall be some output(s), which can be retrieved after your job has finished.

The job specifications are described for the Grid system by the so called Job Description Language (JDL). For each of your jobs, you should prepare a JDL file. An example for a typical simple JDL file content may be:

[

JobType = "Normal"

Executable = "testjob.sh";

StdOutput = "testjob.stdout";

StdError = "testjob.stdout";

InputSandbox = {"testjob.sh", "inputfile.dat"};

OutputSandbox = {"testjob.stdout", "outputfile.dat"};

Requirements = (
                 Member("AFS", other.GlueHostApplicationSoftwareRunTimeEnvironment) && 
                 other.GlueCEUniqueID=="grid109.kfki.hu:2119/jobmanager-lcgpbs-hungrid"
                     );

]

The meaning of the above variables are:

JobType

This optional variable describes whether your job is a normal job ("Normal"), or interactive ("Interactive"). If your job is interactive, the StdIn, StdOut and StdError shall be connected to the terminal, from where you submitted the job, so you are able to communicate with the job during the running time. If unspecified, defaults to "Normal".

Executable

This variable specifies the executable file of your job.

StdOutput

The StdOut of your program shall be written into this file.

StdError

The StdError of your program shall be written into this file.

InputSandbox

This is a list of files, which are sent to the system as the components of your job. Typically the executable of your program, and some supplementary files. The size of the files, sent via the InputSandbox, should be small (<10MB). Large files (as large input data files) should be communicated to the job by other ways, e.g. via AFS, NFS, or the internet (using e.g. wget), or via the Grid Storage System.

OutputSandbox

This is a list of files, which are retrieved after the job has finished. Typically the file containing the StdOut / StdError and some output files. The size of the files, retrieved via OutputSandbox, should be small (<100MB). Large files (as large output data files) should be transfered by other means, e.g. via the Grid Storage System.

Requirements

This optional variable may be a logical expression, specifying requirements for site or the node, where the job is going to be executed. Member("Some_software", other.GlueHostApplicationSoftwareRunTimeEnvironment) means the requirement of the software Some_software on the target node. other.GlueCEUniqueID=="grid109.kfki.hu:2119/jobmanager-lcgpbs-hungrid" means the requirement that the job should be sent to the computing element (queue) grid109.kfki.hu:2119/jobmanager-lcgpbs-hungrid.

Once you prepared the JDL file, you can look for the available queues, which are capable of running your job, by the command:

> edg-list-match  -vo your_vo  testjob.jdl

The job can be submitted by the command:

> edg-job-submit  -vo your_vo  testjob.jdl

The status of the job can be viewed by:

> edg-job-get-status  jobID

If your job has failed to be ran by the Grid system, the logging may be retrieved by:

> edg-job-get-logging-info  jobID

> edg-job-get-logging-info  -v 2  jobID  > log
> grep  "reason"  log  |  uniq

If your job has properly finished, you can retrieve the outputs by the command:

> edg-job-get-output jobID

For further information, look at the man pages of the above commands, and maybe also to the man pages of other edg- commands.