This project has retired. For details please refer to its Attic page.

org.apache.oozie.command.coord
Class CoordSubmitXCommand

java.lang.Object
  org.apache.oozie.command.XCommand<T>
      org.apache.oozie.command.TransitionXCommand<String>
          org.apache.oozie.command.SubmitTransitionXCommand
              org.apache.oozie.command.coord.CoordSubmitXCommand

All Implemented Interfaces:

Callable<String>, XCallable<String>

public class CoordSubmitXCommand
extends SubmitTransitionXCommand

This class provides the functionalities to resolve a coordinator job XML and write the job information into a DB table.

Specifically it performs the following functions: 1. Resolve all the variables or properties using job configurations. 2. Insert all datasets definition as part of the and tags. 3. Validate the XML at runtime.

Field Summary

static String	CONF_DEFAULT_CONCURRENCY
static String	CONF_DEFAULT_MAX_TIMEOUT Default MAX timeout in minutes, after which coordinator input check will timeout
static String	CONF_DEFAULT_THROTTLE
static String	CONF_DEFAULT_TIMEOUT_NORMAL Default timeout for normal jobs, in minutes, after which coordinator input check will timeout
static String	CONF_MAT_THROTTLING_FACTOR
static String	CONF_QUEUE_SIZE
static String	CONFIG_DEFAULT
String	COORD_INPUT_EVENTS
String	COORD_INPUT_EVENTS_DATA_IN
String	COORD_OUTPUT_EVENTS
String	COORD_OUTPUT_EVENTS_DATA_OUT
static String	COORDINATOR_XML_FILE

Fields inherited from class org.apache.oozie.command.TransitionXCommand

insertList, job, updateList

Fields inherited from class org.apache.oozie.command.XCommand

DEFAULT_LOCK_TIMEOUT, DEFAULT_REQUEUE_DELAY, instrumentation, INSTRUMENTATION_GROUP, LOG, logInfo

Constructor Summary

CoordSubmitXCommand(boolean dryrun, org.apache.hadoop.conf.Configuration conf, String authToken)
Constructor to create the Coordinator Submit Command.

CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf, String authToken)
Constructor to create the Coordinator Submit Command.

CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf, String authToken, String bundleId, String coordName)
Constructor to create the Coordinator Submit Command by bundle job.

Method Summary

org.jdom.Element	basicResolveAndIncludeDS(String appXml, org.apache.hadoop.conf.Configuration conf, CoordinatorJobBean coordJob) The method resolve all the variables that are defined in configuration.
String	getEntityKey() Return the entity key for the command.
org.apache.oozie.client.Job	getJob() Get the Job for the command.
protected void	includeDataSets(org.jdom.Element resolvedXml, org.apache.hadoop.conf.Configuration conf) Include referred datasets into XML.
protected void	initEvaluators() Initialize all the required EL Evaluators.
protected boolean	isLockRequired() Indicate if the the command requires locking.
protected void	loadState() Load the necessary state to perform the precondition check and to execute the command.
protected void	mergeDefaultConfig() Merge default configuration with user-defined configuration.
void	notifyParent() This will be used to notify the parent about the status of that perticular job.
void	performWrites() This will be used to perform atomically all the writes within this command.
protected String	readDefinition(String appPath) Read coordinator definition.
protected org.jdom.Element	resolveInitial(org.apache.hadoop.conf.Configuration conf, String appXml, CoordinatorJobBean coordJob) Resolve basic entities using job Configuration.
protected String	submit() Submit the job
void	updateJob() Update the parent job.
protected void	verifyPrecondition() Verify the precondition for the command after a lock has been obtain, just before executing the command.

Methods inherited from class org.apache.oozie.command.SubmitTransitionXCommand

execute, transitToNext

Methods inherited from class org.apache.oozie.command.TransitionXCommand

setJob

Methods inherited from class org.apache.oozie.command.XCommand

call, eagerLoadState, eagerVerifyPrecondition, executeInterrupts, getCreatedTime, getInstrumentation, getKey, getLockTimeOut, getLog, getName, getPriority, getRequeueDelay, getType, inInterruptMode, isReQueueRequired, queue, queue, resetUsed, setInterruptMode

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CONFIG_DEFAULT

public static final String CONFIG_DEFAULT

See Also:

Constant Field Values

COORDINATOR_XML_FILE

public static final String COORDINATOR_XML_FILE

See Also:

Constant Field Values

COORD_INPUT_EVENTS

public final String COORD_INPUT_EVENTS

See Also:

Constant Field Values

COORD_OUTPUT_EVENTS

public final String COORD_OUTPUT_EVENTS

See Also:

Constant Field Values

COORD_INPUT_EVENTS_DATA_IN

public final String COORD_INPUT_EVENTS_DATA_IN

See Also:

Constant Field Values

COORD_OUTPUT_EVENTS_DATA_OUT

public final String COORD_OUTPUT_EVENTS_DATA_OUT

See Also:

Constant Field Values

CONF_DEFAULT_TIMEOUT_NORMAL

public static final String CONF_DEFAULT_TIMEOUT_NORMAL

Default timeout for normal jobs, in minutes, after which coordinator input check will timeout

See Also:

Constant Field Values

CONF_DEFAULT_CONCURRENCY

public static final String CONF_DEFAULT_CONCURRENCY

See Also:

Constant Field Values

CONF_DEFAULT_THROTTLE

public static final String CONF_DEFAULT_THROTTLE

See Also:

Constant Field Values

CONF_MAT_THROTTLING_FACTOR

public static final String CONF_MAT_THROTTLING_FACTOR

See Also:

Constant Field Values

CONF_DEFAULT_MAX_TIMEOUT

public static final String CONF_DEFAULT_MAX_TIMEOUT

Default MAX timeout in minutes, after which coordinator input check will timeout

See Also:

Constant Field Values

CONF_QUEUE_SIZE

public static final String CONF_QUEUE_SIZE

See Also:

Constant Field Values

Constructor Detail

CoordSubmitXCommand

public CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf,
                           String authToken)

Constructor to create the Coordinator Submit Command.

Parameters:

conf - : Configuration for Coordinator job

authToken - : To be used for authentication

CoordSubmitXCommand

public CoordSubmitXCommand(org.apache.hadoop.conf.Configuration conf,
                           String authToken,
                           String bundleId,
                           String coordName)

Constructor to create the Coordinator Submit Command by bundle job.

Parameters:

conf - : Configuration for Coordinator job

authToken - : To be used for authentication

bundleId - : bundle id

coordName - : coord name

CoordSubmitXCommand

public CoordSubmitXCommand(boolean dryrun,
                           org.apache.hadoop.conf.Configuration conf,
                           String authToken)

Constructor to create the Coordinator Submit Command.

Parameters:

dryrun - : if dryrun

conf - : Configuration for Coordinator job

authToken - : To be used for authentication

Method Detail

submit

protected String submit()
                 throws CommandException

Description copied from class: SubmitTransitionXCommand

Submit the job

Specified by:

submit in class SubmitTransitionXCommand

Returns:

the id

Throws:

CommandException - thrown if unable to submit

mergeDefaultConfig

protected void mergeDefaultConfig()
                           throws CommandException

Merge default configuration with user-defined configuration.

Throws:

CommandException - thrown if failed to read or merge configurations

basicResolveAndIncludeDS

public org.jdom.Element basicResolveAndIncludeDS(String appXml,
                                                 org.apache.hadoop.conf.Configuration conf,
                                                 CoordinatorJobBean coordJob)
                                          throws CoordinatorJobException,
                                                 Exception

The method resolve all the variables that are defined in configuration. It also include the data set definition from dataset file into XML.

Parameters:

appXml - : Original job XML

conf - : Configuration of the job

coordJob - : Coordinator job bean to be populated.

Returns:

Resolved and modified job XML element.

Throws:

CoordinatorJobException - thrown if failed to resolve basic entities or include referred datasets

Exception - thrown if failed to resolve basic entities or include referred datasets

initEvaluators

protected void initEvaluators()

Initialize all the required EL Evaluators.

resolveInitial

protected org.jdom.Element resolveInitial(org.apache.hadoop.conf.Configuration conf,
                                          String appXml,
                                          CoordinatorJobBean coordJob)
                                   throws CoordinatorJobException,
                                          Exception

Resolve basic entities using job Configuration.

Parameters:

conf - :Job configuration

appXml - : Original job XML

coordJob - : Coordinator job bean to be populated.

Returns:

Resolved job XML element.

Throws:

CoordinatorJobException - thrown if failed to resolve basic entities

Exception - thrown if failed to resolve basic entities

includeDataSets

protected void includeDataSets(org.jdom.Element resolvedXml,
                               org.apache.hadoop.conf.Configuration conf)
                        throws CoordinatorJobException

Include referred datasets into XML.

Parameters:

resolvedXml - : Job XML element.

conf - : Job configuration

Throws:

CoordinatorJobException - thrown if failed to include referred datasets into XML

readDefinition

protected String readDefinition(String appPath)
                         throws CoordinatorJobException

Read coordinator definition.

Parameters:

appPath - application path.

Returns:

coordinator definition.

Throws:

CoordinatorJobException - thrown if the definition could not be read.

getEntityKey

public String getEntityKey()

Description copied from class: XCommand

Return the entity key for the command.

Specified by:

getEntityKey in interface XCallable<String>

Specified by:

getEntityKey in class XCommand<String>

Returns:

the entity key for the command.

isLockRequired

protected boolean isLockRequired()

Description copied from class: XCommand

Indicate if the the command requires locking.

Subclasses should override this method if they require locking.

Specified by:

isLockRequired in class XCommand<String>

Returns:

true/false

loadState

protected void loadState()
                  throws CommandException

Description copied from class: XCommand

Load the necessary state to perform the precondition check and to execute the command.

Subclasses must implement this method and load the state needed to do the precondition check and execute the command.

Specified by:

loadState in class XCommand<String>

Throws:

CommandException

verifyPrecondition

protected void verifyPrecondition()
                           throws CommandException

Description copied from class: XCommand

Verify the precondition for the command after a lock has been obtain, just before executing the command.

Specified by:

verifyPrecondition in class XCommand<String>

Throws:

CommandException - thrown if the precondition is not met.

notifyParent

public void notifyParent()
                  throws CommandException

Description copied from class: TransitionXCommand

This will be used to notify the parent about the status of that perticular job.

Specified by:

notifyParent in class TransitionXCommand<String>

Throws:

CommandException

updateJob

public void updateJob()
               throws CommandException

Description copied from class: TransitionXCommand

Update the parent job.

Specified by:

updateJob in class TransitionXCommand<String>

Throws:

CommandException

getJob

public org.apache.oozie.client.Job getJob()

Description copied from class: TransitionXCommand

Get the Job for the command.

Overrides:

getJob in class TransitionXCommand<String>

Returns:

the job

performWrites

public void performWrites()
                   throws CommandException

Description copied from class: TransitionXCommand

This will be used to perform atomically all the writes within this command.

Specified by:

performWrites in class TransitionXCommand<String>

Throws:

CommandException