JobScheduler - Quickstart · April 2016 JobScheduler - Quickstart page: 1 JobScheduler - Job...

April 2016 JobScheduler - Quickstart page: 1

JobScheduler - Job Execution and Scheduling SystemSoftware Open Source

JobScheduler - Quickstart

An Introduction to Job SchedulingApril 2016

JobScheduler - Quickstart - Contact InformationSoftware Open Source


Contact Information

Software- und Organisations-Service GmbH

Last Updated: 04/22/2016 05:31 PM

This documentation is based on JobScheduler Version 1.10.4.

Copyright © 2005-2016 SOS GmbH Berlin.

All rights reserved. All trademarks or registered trademarks are the property of their respective holders. Allinformation and materials in this book are provided "as is" and without warranty of any kind. All information in thisdocument is subject to change without further notice.

This product includes software developed by the Apache Software Foundation (http://apache.org/)

We would appreciate any feedback you have, or suggestions for changes and improvements; please forward yourcomments to [email protected].

Giesebrechtstr. 15D-10629 BerlinGermany

Telephone +49 (0)30 86 47 90-0Telefax +49 (0)30 8 61 33 35Mail [email protected] http://www.sos-berlin.com

http://www.sos-berlin.com

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1 Required Knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 How are Jobs, Job Chains and Orders Configured? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6

4 Setting Up JobScheduler Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74.1 Setting up a Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

4.1.1 Setting up a Standalone Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74.1.2 Setting up an Order Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

4.2 Setting up a Job Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.3 Setting up an Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5 Scheduled Job and Order Starts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

6 Directory Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .246.1 Starting a Job Chain by Directory Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246.2 Starting a Standalone Job by Directory Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

7 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287.1 Setting Parameters for a Job or an Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287.2 Reading and Setting Parameters in Shell Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

8 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.1 Behavior of Stopped Order Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.2 Behavior of Stopped Standalone Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .328.3 Error Handling in Job Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

8.3.1 The Order is Immediately Forwarded to the End of the Job Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328.3.2 The Order Starts a Specified Follow-On Job in the Job Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338.3.3 Automatic Repetition of a Order Ending in Error (Setback) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338.3.4 Processing of the Order is Suspended . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

8.4 Error Handling for Standalone Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378.4.1 Scheduled Repetition of a Standalone Job Ending in Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

9 An Overview of the JobScheduler Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419.1 Installation and Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419.2 Using the JobScheduler API to Implement Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419.3 Advanced Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

10 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4211 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

JobScheduler - Quickstart - Table of ContentsSoftware Open Source


Table of Contents

Note that a quick overview of the JobScheduler documentation (page 41) has been included at the end of this

JobScheduler - Quickstart - IntroductionSoftware Open Source


1 Introduction

You have installed the JobScheduler and started it. What comes next?

To answer this question, we have prepared the following examples that are intended to provide a quick andcompact introduction to the use of the JobScheduler .

1.1 Required Knowledge

Before you start to work through the examples in this document, we strongly recommend that you take the time tobecome familiar with some of the other JobScheduler documentation, in particular, the Installation andConfiguration guide (scheduler_installation).

document.

http://www.sos-berlin.com/doc/en/scheduler_installation.pdf

JobScheduler - Quickstart - How are Jobs, Job Chains and Orders Configured?Software Open Source


2 How are Jobs, Job Chains and Orders Configured?

JobScheduler objects (jobs, job chains, orders, schedules, process classes and locks) are all configured usingplain XML files. These files are saved in a so-called Hot Folder s (the default location of this folder or directory is./config/live ). There is a file in this directory for every JobScheduler object, in which the object's configurationis saved.

Changes to these object configuration files become effective without a restart of the JobScheduler being necessary,although on Unix® systems it may take up to a minute for a change to become effective.

Objects can be created and modified using JOE (JobScheduler Object Editor). New objects are created by eitheropening an already existing configuration file in JOE, modifying it and then saving the file under a new name, or bycreating and configuring a completely new file.

A text editor could also be used to create and modify objects, but working in this way requires detailed knowledgeof the XML structure of each JobScheduler object (see the Reference documentation).

http://www.sos-berlin.com/doc/en/scheduler.doc/index.xml

JobScheduler - Quickstart - Getting StartedSoftware Open Source


3 Getting Started

1. Create a subdirectory named quickstart within the live folder ( ./config/live ) directory. Jobs and jobchains will be stored in this directory.

2. Start JOE (JobScheduler Object Editor) using one of the following scripts:

3. Use the key combination ctrl+D to open the live folder in JOE . You can also open this dialog by selectingFile->Open Hot Folder from the menu or by clicking the red folder icon. The live folder should have beenselected by default: if not, navigate to ./config/live . Select the quickstart subfolder. If required you canalso use this dialog to create the quickstart subdirectory.

4. Open the JOC (JobScheduler Operations Center) . You can use JOC to follow changes to Jobs and otherob jec ts as they happen . You open JOC i n a b rowser by en te r ing the addresshttp://[scheduler_host]:[scheduler_port] , where [scheduler_host] and [scheduler_port] are thehost name and the TCP port number of the JobScheduler (e.g. http://localhost:4444 ). Note that theJobScheduler must be running before JOC is called.

• %SCHEDULER_HOME% \bin\jobeditor.cmd ( Windows™ )

• $SCHEDULER_HOME /bin/jobeditor.sh ( Unix™ )

Have you finished making the steps described in the Getting Started (page 6) section above? Then we can

checkbox to be activated (see also Error Handling (page 32)). Now enter a shell script such as the following in the

JobScheduler - Quickstart - Setting Up JobScheduler ObjectsSoftware Open Source


4 Setting Up JobScheduler Objects

4.1 Setting up a Job

There are two types of jobs: Standalone Jobs and Order Jobs.

The key difference between Standalone and Order Jobs lies in the configuration of their order= "yes|no" attribute.

4.1.1 Setting up a Standalone Job

continue.

The JobScheduler object tree is shown in the left hand pane of the screenshot below. Click on the Jobs element inthis tree and then click on the New Standalone Job button in the right hand pane. A new Job will now be added. Thiswill be shown in both the left hand element tree and the list of jobs in the main pane. Click on this new job.

This will open an empty Job configuration element as shown in the next screenshot. Give this job a name byentering, for example, firstJob in the Job Name field. Note that for a Standalone Job the Order Job: No radio button(to the right of the name) has to be selected. As already mentioned this setting is the key difference in between theconfiguration of a Standalone Job and that of an Order Job. In addition, this setting causes the Stop On Error

Source Code text field:

• Standalone Jobs can be started independently of other Jobs but cannot be used in Job Chains.

• Order Jobs are used in Job Chains and are are started in a Job Chain by an Order for that particular JobChain.

Note that a Start Period (page 28) or Parameters can also be allocated to a Job (page 22).



@echo offecho %SCHEDULER_JOB_NAME% is running echo Current Timestamp: %DATE% %TIME%exit %ERRORLEVEL%

Example: for Microsoft® Windows®:

#!/bin/shecho "$SCHEDULER_JOB_NAME is running"echo "Current Timestamp: `date`"exit $?

Example: for Unix®:

Now save this configuration, either using the ctrl+S keys or File->Save, and JOE will create the./config/live/quickstart/firstJob.job.xml file.



Now change to JOC and update the window using the Update button at the very top of the window. where you willnow see the quickstart/firstJob Job in the Jobs tab. Click on the Job in the left hand part of the window to open adetailed view of the Job in the right hand part, as shown in the screenshot below. Start a task for this Job byclicking on ->Start task immediately in the Job menu.

The task will now start to run and you can follow its progress in the Task history by clicking on the Show log button.

The preparations (page 6) are done? Then we can continue.



This will open a new browser window with the log output, where, amongst other things, you will be able to read theoutput from the shell script.

Congratulations!You have successfully configured and run your first Standalone Job.

4.1.2 Setting up an Order Job

Click on the Jobs element in the JobScheduler object tree shown in the left hand pane (see the screenshot below)and then on New Order Job button in the right hand pane. A new Job will be added to both the left hand element treeand to the list of jobs in the right hand pane. Click on this new job.

This will open an empty Job configuration element as shown in the next screenshot. Give this job a name byentering, for example, firstOrderJob in the Job Name field. Note that for an Order Job the Order Job radio button (tothe right of the name) has to be set to Yes. As already mentioned, this setting is the key difference in between theconfiguration of an Order Job and that of a Standalone Job. In addition, this setting causes the Stop On Error

checkbox to be activated (see also Error Handling (page 32)). Now enter a shell script such as the following in the

Note that a Start Period (page 28) or Parameters can also be allocated to a Job (page 22).



Source Code text field:

@echo offecho %SCHEDULER_JOB_NAME% processed %SCHEDULER_ORDER_ID%echo Current Timestamp: %DATE% %TIME%exit %ERRORLEVEL%


#!/bin/shecho "$SCHEDULER_JOB_NAME processed $SCHEDULER_ORDER_ID"echo "Current Timestamp: `date`"exit $?


Now save this configuration, either using the ctrl+S keys or File->Save, and JOE will create the./config/live/quickstart/firstOrderJob.job.xml file.

after a delay). Order Jobs (page 10) should be configured to run with the Stop On Error parameter deactivated, so

Have you successfully set up the Order Job (page 10) described above? Then repeat the procedure for a second



Now change to JOC and update the window using the Update button at the very top of the window. Yourquickstart/firstOrderJob Job will now be shown in the Jobs tab. Click on the Job in the left hand part of the windowto open a detailed view of the Job in the right hand part, as shown in the screenshot below. Note that here the Start task immediately function in the Job Menu is not active, as Order Jobs can only be started by Orders within JobChains.

Congratulations!You have successfully configured your first Order Job.

4.2 Setting up a Job Chain

Job Chains are defined as a series of Jobs, which is started by an Order. Job dependencies are defined forsuccessful and unsuccessful execution. Only Order Jobs may be allocated to a Job Chain. A more exactdescription of a Chain would state that it has nodes and Jobs are allocated to these. The Job Chain nodes are usedto define the successor nodes an Order should be forwarded to after the current Job has been completed - eithersuccessfully or unsuccessfully. If a Job is defined for a successor node then this Job will be started: otherwise it isassumed that the Order has reached the end of the Chain. A Job may be configured so that if it runs unsuccessfully(i.e. ends with an error) the Order can stop at the current node. Alternatively, the Job could be restarted (possibly

that further Orders can be processed despite the unsuccessful run.

Order Job, so that the Job Chain can start two different Jobs. Name this second Order Job secondOrderJob andadd the following code:



@echo offecho %SCHEDULER_JOB_NAME% processed %SCHEDULER_ORDER_ID%dir .\config\live\quickstartexit %ERRORLEVEL%


#!/bin/shecho "$SCHEDULER_JOB_NAME processed $SCHEDULER_ORDER_ID"ls -l ./config/live/quickstartexit $?


Then we can continue.

Click on Job Chains in the JobScheduler object list and then on the New Job Chain button to the right. A new JobChain will then be added to the list of job chains and shown in the elements tree in the left hand pane. Click on thisnewly added element,

which will open an empty Job Chain Configuration element. Give this chain a name by entering, for example,firstJobChain in the configuration element Chain Name field.



Now move back to the element tree in the left hand pane and click on Nodes, which is a Job Chain subelement.Add Chain nodes using the New Chain Node menu item: Order Jobs will then be added to these nodes. In thisexample, we need to add two nodes to the Chain. Each node requires a State that is unique in the Chain. Set thestate for the first node to step1. The firstOrderJob Job should then be added to this node. The Next State and ErrorState determine the nodes the Order should be forwarded to, once the Job has been completed - eithersuccessfully or unsuccessfully. For the first node insert step2 and error in the appropriate fields as shown in thescreen-shot below.

Now add a second node and set the State to step2, the Job to secondOrderJob, the Next State to success andthe Error State to error.



There has to be a node in the Chain for every value of Next State and Error State. At this stage the nodes forsuccess and error are still missing. These missing nodes are shown in yellow as illustrated in the screen shotbelow and they can be easily added using the Add Missing Nodes button to the right. First of all, the node in the list,which the missing node is to be added for, has to be selected. In our example, select the node with the state step2.



Note that nodes generated in this manner do not have a Job and that the two nodes here both end the Chain.

Now save this configuration, either using the ctrl+S keys or File->Save, and _JOE will generate the./config/live/quickstart/firstJobChain.job_chain.xml file.

An Order for this Chain will start the first quickstart/firstOrderJob Job. It will then continue to thequickstart/secondOrderJob Job and end at the success node, if both of the Jobs run successfully. However, if oneof the Jobs causes an error, the Order will jump to the error node, where it will end.



Now change to JOC and update the window using the Update button at the very top of the window. Select the JobChains tab where you will be able to see the quickstart/firstJobChain Job Chain. Click on Job Chain in order to seea detailed view of the Job Chain in the right hand pane. Add an Order to the Job Chain by clicking on Job chain menu

->Add order. If you do not specify an Order Id for the Order here, the JobScheduler will automatically allocate anumber as the Order ID.

You can watch the progress of the Order along the Chain, either by observing the log, which can be seen byselecting Order menu ->Show log or in the order history (in which case the Show order history checkbox must beactivated).

You have set up the Job Chain (page 12) described above? Then we can continue.

all Orders (page 28), whether manual or persistent, can also contain Parameters.



The log will be shown in a new browser window and will show the script output.

Congratulations!You have successfully set up a Job Chain and an Order.

4.3 Setting up an Order

You can manually add anOrder to a Job Chain. Note that such manually added Orders are not persistent.Persistence is important when Orders, for example, are to be scheduled to start at a particular time. Note also that



Click in the JobScheduler object list in the left hand pane on Orders and then on New Order in the right hand pane.This will cause a new Order to be added to the order list and to the element tree in the left hand pane. Click on thisnewly added element,



This will now open an empty Order configuration element. Give this order an ID by entering, for example, firstOrderin the Order ID field and allocate it to a Job Chain by selecting the firstJobChain Job Chain.

Now save this configuration, either using the ctrl+S keys or File->Save, and JOE will create the./config/live/quickstart/firstJobChain,firstOrder.order.xml file.



Now change to JOC and update the window using the Update button at the very top of the window. Navigate to thedetailed view of the quickstart/firstJobChain Job-Kette as described above. Here you will see the Order, which youcan start by selecting Order menu ->Start order now. In addition, you can follow the progress of the Order in theOrders tab.

Note that the log will not read any differently to the log generated by the Order described in the previous section.

Congratulations!You have now successfully configured and run your first manual Order.

JobScheduler - Quickstart - Scheduled Job and Order StartsSoftware Open Source


5 Scheduled Job and Order Starts

As Order Jobs are started by Orders for a Job Chain, would it not be a good idea to set start times for Order Jobs?The answer is, "Yes" and, for this situation, the start times are set for the Order.

Note, however, that it is also possible to set a run-time period for Order Jobs. In this case, when an Order is to bestarted outside of an Order Job's time period, the Order will be queued and started once its run-time period hasbeen reached.

In the following example a Standalone Job or Order is to be started daily at 12:00 o'clock and at 14:00.

You will find the Run Time->Everyday function in the element tree in the left hand pane of JOE, this is a subelementavailable for both Jobs and Orders. Use the New Period button (positioned on the right hand side) to add a new startperiod. Use the Apply Period button to save your input after you have selected Single Start 12:00:00 in the StartTime field. Repeat this procedure for Single Start 14:00:00.

Note that in addition to the fixed start times (Single Start), it is also possible to set repeat intervals (Intervalend/start, Interval start/start).

set. It is also possible to combine the scheduled starting of a Standalone Job with Directory Monitoring (page 26),

JobScheduler - Quickstart - Scheduled Job and Order StartsSoftware Open Source


Standalone Jobs and Orders can still be started manually, regardless of whether a start time or run period has been

which will be described in the next section.

First of all, in JOE, we copy the Job Chain (page 12) we have already configured. This can be conveniently carried

JobScheduler - Quickstart - Directory MonitoringSoftware Open Source


6 Directory Monitoring

6.1 Starting a Job Chain by Directory Monitoring

With this method of operation, an Order is generated for a specified Job Chain each time a file is added to amonitored directory. The order ID is set to the name of the file that has been added to the monitored directory and afilter can be specified to restrict the monitoring to specific file names.

In the following example, the directory ./notification_dir will be monitored and only files with the extension txtwill be acted on.

out using the Copy and Paste functions in the object context menu in the left hand pane.

Select the new Chain in the element tree and enter a new name in the Chain Name field, such asfileOrderJobChain.



Now move back to the element tree in the left hand pane and click on Nodes, which is a Job Chain subelement. Inthe File Order Sources area you can now use the New File Order Source button to specify the directory that is to bemonitored. After you have set the notification_dir and \.txt$ values in the Directory and Regex fields, specify the Chain using the Apply File Order Source button. Repeat this procedure to add further directories or patterns.



Create the ./notification_dir directory. Now copy a file with a txt file extension into this directory and returningto JOC follow the history of the quickstart/fileOrderJobChain Job Chain in the log.

6.2 Starting a Standalone Job by Directory Monitoring

With this method of operation, a standalone job is automatically started every time a file is either added to ordeleted from a monitored directory. a filter can be specified to restrict the monitoring to specific file names.

Note that the JobScheduler will record whether the file that originated a file order is still in the monitored directorywhen the order has been processed to the end of its Job Chain. In this situation, the file will be added to a Blacklistto stop further Orders from being generated for this file.

It is important therefore that you either configure a Job in the Chain to remove the originating file or add a File Sinkto the chain. File Sinks delete or move the file that originated the order. In the following example, the originating fileshould be deleted if the order reaches the successnode and moved if the order reaches the error node. Click inthe node list on the success node and then activate the File Sink radio button and the Remove File checkbox.Save these configuration by clicking on the Apply Chain Node button. Repeat this procedure for the error node butthis time enter the directory which the file should be moved to in the Move to field.

Directory monitoring can also be combined with Scheduled Starts (page 22).

for Job Chains (page 24).



In the following example, the directory ./notification_dir will be monitored and only files with the extension txtwill be acted on.

Standalone Jobs qui sont démarrés avec le directory monitoring can also be started manually as required.

In the element tree in the left hand pane in JOE you will see under each job the Run Options function. Click on theNew Dir button in the Start When Directory Changed area of the right hand pane to add the directory that is to bemonitored. Save this configuration by clicking the Apply Dir button after you have entered the notification_dir and\.txt$ values in the Watch Directory and File Regex fields. Repeat this procedure to add further directories orpatterns.

Directory monitoring does not make sense for Order Jobs: it is however possible to configure Directory Monitoring

them using the Apply button. Add the testParam parameter with the value Hello World! (page 10) to thequickstart/firstJobStandalone Job and the quickstart/firstOrderJob (page 7) Order Job, both of which weregenerated in previous sections of this document. Add the testParam parameter with value Goodbye! (page 18) to

JobScheduler - Quickstart - ParametersSoftware Open Source


7 Parameters

Parameters can be set for Jobs, Orders and for job chain nodes.

Note that when Jobs are programmed using the JobScheduler API then Job and Order parameters are separateobjects. Node and Order parameters form the same object, although here node parameters have priority overOrder parameters of the same name.

This differentiation does not apply for shell scripts. In shell scripts, node parameters overwrite Order parameters ofthe same name and Order parameters overwrite Job parameters.

Order parameters are available to all the Jobs in a Job Chain. When the value of an Order parameter in a job chainchanges then this changed value will be applied at the start of the next (persistant) Order.

7.1 Setting Parameters for a Job or an Order

The Parameter function can be found in the JOE element tree as a subelement of every Job and Order. Eachparameter consists of a name and a value. Enter the name-value pairs in the Name and Value fields and save

the quickstart/firstOrder Order configured above.

Check to see if the testParam parameter, described in the last Kapitel (page 28), has been set. As an example of



7.2 Reading and Setting Parameters in Shell Scripts

An environment variable of the form:

can be used to read any parameter in a shell script.

Parameters in a shell script can be added and modified using:

the reading and writing of parameters, modify the scripts of the quickstart/firstOrderJob andquickstart/secondOrderJob Jobs as follows:

• $SCHEDULER_PARAM_[PARAM-NAME] (Unix®)

• %SCHEDULER_PARAM_[PARAM-NAME]% (Microsoft® Windows®)

• echo "[PARAM-NAME]=[PARAM-VALUE]" >> $SCHEDULER_RETURN_VALUES (on Unix® systems)

• echo [PARAM-NAME]=[PARAM-VALUE] >> %SCHEDULER_RETURN_VALUES% (on Microsoft® Windows®systems)



@echo offecho %SCHEDULER_JOB_NAME% processed %SCHEDULER_ORDER_ID%echo Current Timestamp: %DATE% %TIME%echo Parameter testParam = "%SCHEDULER_PARAM_TESTPARAM%"echo Parameter testParam is modified to "123456789"echo testParam=123456789 >> %SCHEDULER_RETURN_VALUES%exit %ERRORLEVEL%

Example: firstOrderJob for Microsoft® Windows®:

#!/bin/shecho "$SCHEDULER_JOB_NAME processed $SCHEDULER_ORDER_ID"echo "Current Timestamp: `date`"echo "Parameter testParam = \"$SCHEDULER_PARAM_TESTPARAM\""echo "Parameter testParam is modified to \"123456789\""echo "testParam=123456789" >> $SCHEDULER_RETURN_VALUESexit $?

Example: firstOrderJob for Unix®:

@echo offecho %SCHEDULER_JOB_NAME% processed %SCHEDULER_ORDER_ID%echo Parameter testParam = "%SCHEDULER_PARAM_TESTPARAM%"dir .\config\live\quickstartexit %ERRORLEVEL%

Example: secondOrderJob for Microsoft® Windows®:

#!/bin/shecho "$SCHEDULER_JOB_NAME processed $SCHEDULER_ORDER_ID"echo "Parameter testParam = \"$SCHEDULER_PARAM_TESTPARAM\""ls -l ./config/live/quickstartexit $?

Example: secondOrderJob for Unix®:

Both Jobs read the testParam parameter, whilst the quickstart/firstOrderJob Job changes the value of theparameter. testParam=Hello World! is the initial parameter setting for the quickstart/firstOrderJob Job . Thisvalue will be overridden by the Order.



Now start the Order for the quickstart/firstJobChain Chain in JOC and follow its progress in the log file.

stopped, unless Stop On Error (page 10) has been deactivated. Jobs that have been stopped are marked in red in

ending in error or, if required, a scheduled rerun (page 33) of an Order can take place. If this is not done, the Job

This error handling method is used in the example above (page 12). Here, an end node (i.e. a node without a Job )

JobScheduler - Quickstart - Error HandlingSoftware Open Source


8 Error Handling

A Job ends in error when the script or program it calls returns an exit code other than 0. The Job will then be

the log in JOC.

To illustrate this, change the scripts of the firstJob and firstOrderJob Jobs developed in the previous examples,so that they end in error.

... exit 5

Example: Forcing an error using exit != 0

8.1 Behavior of Stopped Order Jobs

Stop On Error has to be deactivated for Order Jobs, so that further Orders can be processed even after a run

ending in error will have to be unstopped using the Job menu->Unstop function in JOC.

8.2 Behavior of Stopped Standalone Jobs

Stopped Standalone Jobs can be manually restarted in JOC using the Job menu->Start job immediately function.However they cannot be restarted automatically, either with a scheduled start or by directory monitoring. Use thefunction Job menu->Unstop in JOC to unstop a job.

8.3 Error Handling in Job Chains

There are several possible ways to react to Jobs in Job Chains ending in error:

A different method of error handling can be specified at each node in a Chain.

Whenever the Job at the current Job Chain node ends in error, the Order will be either be forwarded to the nodeconfigured in the Error State, suspended or given a scheduled restart.

8.3.1 The Order is Immediately Forwarded to the End of the Job Chain

is defined for the Error State of the first node (step1).

Now start the quickstart/firstOrder Order in JOC and follow its behavior in the Order log, where you will seethat the second job (quickstart/secondOrderJob) does not run.

• End the Job Chain

• Start another Job in the Job Chain

• Repeat the Job that ended in error

• Suspend the Order



8.3.2 The Order Starts a Specified Follow-On Job in the Job Chain

You can also specify a node containing a Job for the Error State. The Chain will then be continued with this Job.

8.3.3 Automatic Repetition of a Order Ending in Error (Setback)

Order Jobs can be configured so that they will automatically be repeated if they end in error. This requires that anumber of settings are made at the relevant node in the Chain and in the Job in question.

Select the Nodes function in the JOE element tree, which is a subelement of the Job Chain firstJobChain. Nowselect the step1 node from the list of nodes. Select setback in the On Error field and save this new configurationusing Apply Chain Node .



Now open the Run Options function in the in JOE element tree, which is a subelement of the firstOrderJob. Inthe Delay Order After Setback area, you can now use the New Delay button to specify a scheduled (delayed) restartof the Order. Example settings for Set Back Count and Delay could be the values 2 and 00:00:10 respectively.Use the Apply Delay button to save these settings. Repeat this procedure, but this time set the Set Back Count to 5and activate the Max checkbox.

When the Job first ends in error, it will be immediately restarted. However, if the Job continues to end in error, afterthe 2nd rerun the restart will only take place after a delay of 10 seconds. After the 5th rerun the Order will move onto the node specified in the Error State.

Now start the quickstart/firstOrder Order in JOC and follow its behavior in the Order log,



8.3.4 Processing of the Order is Suspended

Select the Nodes function in the JOE element tree, which is a subelement of the firstJobChain Job Chain. Nowselect the step1 node from the list of nodes. Select suspend in the On Error field and save this change using the Apply Chain Node button.



This setting causes the Order to be suspended at an Order Job that ends in error. The Order can be continued inJOC once the cause of the error has been sorted. The order is restarted using the Resume order function in theOrder menu. The order will be restarted with the Job that caused the error being repeated.

8.4 Error Handling for Standalone Jobs

A Standalone Job can be configured to rerun after the Job has ended in error.

8.4.1 Scheduled Repetition of a Standalone Job Ending in Error

Select the Run Options function, which is a Job subelement in the JOE element tree. In the Delay After Error areayou can use the New Delay button to specify a scheduled (delayed) restart of the Job. Enter, for example, values forError Count and Delay of 2 and 00:00:10. Use the Apply Delay button to save these settings. Now repeat thisprocedure but enter an Error Count of 5 and check the stop radio button.



When the Job first ends in error, it will be immediately restarted. However, if the Job continues to end in error, afterthe 2nd rerun, the restart will only take place after a delay of 10 seconds and after the 5th attempt the Job will bestopped.



Now start the Job in JOC, and look in the Task History, where the repeated job starts can be seen along with thedelays they were subjected to.

JobScheduler - Quickstart - An Overview of the JobScheduler DocumentationSoftware Open Source


9 An Overview of the JobScheduler Documentation

The documentation can be found in the $SCHEDULER_HOME/doc/de directory.

9.1 Installation and Configuration

9.2 Using the JobScheduler API to Implement Jobs

The following documents are intended for job developers: It is not necessary to understand them before you start toconfigure automatic job starts for either executable programs or scripts.

9.3 Advanced Topics

The following documents describe more complex job scheduling scenarios.

Installation and Configuration (scheduler_installation)We recommend that you read this document before you install and configure the JobScheduler.

Quickstart Guide to Job Configuration (scheduler_quickstart)This document.

Reference Documentation (scheduler, reference)The handbook describes job configuration and the JobScheduler interfaces.

API Documentation (reference/api)The JobScheduler programming interface documentation.

API Job Implementation Tutorial (JobScheduler_API-Jobs-Implementation)An introduction to job development using the JobScheduler API.

Web Services Implementation Tutorial (scheduler_webservices)Explains the configuration and implementation of web services for your jobs

MySQL Job Scheduling (scheduler_managed_user_jobs)Describes the JobScheduler's SQL statement and procedure interface, which can be used with a database ina similar way to the Oracle® Database Job Scheduler.

http://www.sos-berlin.com/doc/en/scheduler.doc/api/api.xml

http://www.sos-berlin.com/doc/en/scheduler_managed_user_jobs.pdf

http://www.sos-berlin.com/doc/en/scheduler.doc/index.xml

http://www.sos-berlin.com/doc/en/scheduler.pdf

http://www.sos-berlin.com/doc/en/scheduler_quickstart.pdf

http://www.sos-berlin.com/doc/en/scheduler_installation.pdf

http://www.sos-berlin.com/doc/en/scheduler_webservices.pdf

http://www.sos-berlin.com/doc/en/JobScheduler_API-Jobs-Implementation.pdf

JobScheduler - Quickstart - GlossarySoftware Open Source


Glossary

Job Chains

Jobs

JOC (JobScheduler Operations Center)

JOE (JobScheduler Object Editor)

Orders

A series of jobs that process orders one after the other. The JobScheduler starts the jobs in a job chainautomatically, once a order has been started for the chain. Job chains allow a number of orders to beprocessed in parallel, by starting multiple instances of jobs (tasks).

Programs and scripts that are executed by the JobScheduler have to be embedded in jobs. Jobs cancontain either start executable files or contain job scripts that use the JobScheduler program interface.More than one instance of a job (task) may run at any one time, should this be required to scaleperformance.

There are two types of jobs: standalone and order jobs. Whereas order jobs are started by orders within ajob chain, standalone jobs can be started independently: either manually, scheduled or by directorymonitoring. Standalone jobs cannot be run in job chains.

JOC (JobScheduler Operations Center) is the JobScheduler interface for monitoring and controllingJobScheduler objects such as jobs, job chains and orders.

JOC is opened in a web browser using the address http://[scheduler_host]:[scheduler_port], where[scheduler_host] and [scheduler_port] are the host name and the TCP ports number of theJobScheduler (e.g. http://localhost:4444).

JOE is the JobScheduler Object Editor. This is used to configure JobScheduler objects (jobs, job chains,orders, schedules, process classes and locks).

JOE is started using the script:

Orders activate the processing of job chains. Orders may also contain parameters for the jobs in a job chain.Every job in a job chain has access to the order parameters. Order parameters overwrite job parameters ofthe same name. Orders can be started according to time.

An order processes the jobs in a job chain one after the other. Orders can be configured so that, if a error inprocessing a job occurs, the order ...

• $SCHEDULER_HOME \bin\jobeditor.cmd (Windows™)

• $SCHEDULER_HOME /bin/jobeditor.sh (Unix™)

• is removed from the job chain;

• continues with a further job in the chain;

• continues with the job that caused the initial error being repeated

• stands still - that is the order processing is suspended until it is restarted manually.

JobScheduler - Quickstart - GlossarySoftware Open Source


Schedules

Time-based starting of jobs or orders can either be directly specified for each job or order or can bedelegated to a schedule. Individual jobs or orders are then referred to this schedule. This means that ifseveral jobs or orders have the same start parameters, these need only be specified once in the schedule.In addition, one schedule can be replaced by another for a particular period of time, thereby increasing theflexibility of setting job and order start parameters.

Hot folder 6Hot Folder 5, 6

Job 5, 7, 10, 22, 28, 28, 28, 28, 32, 32,37Job chain 5, 6, 7, 7, 12, 12, 12, 13, 13,14, 15, 16, 16, 17, 17, 18, 18, 20, 21, 22, 24,24, 25, 26, 26, 27, 28, 31, 32, 32, 33, 33, 33,36

Lock 5

Order 5, 7, 12, 12, 14, 16, 17, 18, 18, 19,20, 21, 21, 21, 22, 22, 22, 22, 23, 24, 26, 28,28, 28, 28, 28, 30, 31, 32, 32, 32, 34, 34, 34,37Order job 7, 7, 10, 11, 12, 12, 12, 12, 14,16, 16, 22, 22, 26, 27, 28, 29, 30, 32, 32, 32,32, 33, 33, 34, 37

Process class 5

Schedule 5Standalone job 7, 7, 8, 9, 10, 10, 22, 23,32, 37, 37, 39, 40

JobScheduler - Quickstart - IndexSoftware Open Source


IndexH

J

L

O

P

S

JobScheduler - Quickstart · April 2016 JobScheduler - Quickstart page: 1 JobScheduler - Job...

Documents

Transcript of JobScheduler - Quickstart · April 2016 JobScheduler - Quickstart page: 1 JobScheduler - Job...