org.apache.oozie

Class CoordinatorEngine

java.lang.Object

org.apache.oozie.BaseEngine

org.apache.oozie.CoordinatorEngine

public class CoordinatorEngine
extends BaseEngine

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CoordinatorEngine.FILTER_COMPARATORS

Field Summary

Fields

Modifier and Type	Field and Description
static String	COORD_ACTIONS_LOG_MAX_COUNT
static String[]	VALID_JOB_FILTERS

Fields inherited from class org.apache.oozie.BaseEngine

USE_XCOMMAND, user

Constructor Summary

Constructors

Constructor and Description
CoordinatorEngine() Create a system Coordinator engine, with no user and no group.
CoordinatorEngine(String user) Create a Coordinator engine to perform operations on behave of a user.

Method Summary

Modifier and Type	Method and Description
void	change(String jobId, String changeValue) Change a coordinator job.
void	changeSLA(String id, String actions, String dates, String childIds, String newParams) Change SLA properties for job
void	disableSLAAlert(String id, String actions, String dates, String childIds) Disable SLA alert for job
String	dryRunSubmit(org.apache.hadoop.conf.Configuration conf) Dry run a job; like BaseEngine.submitJob(org.apache.hadoop.conf.Configuration, boolean) but doesn't actually execute the job.
void	enableSLAAlert(String id, String actions, String dates, String childIds) Enable SLA alert for job
String	getActionStatus(String actionId) Return the status for an Action ID
CoordinatorActionBean	getCoordAction(String actionId)
List<Pair<CoordinatorActionBean,Map<String,ActionDependency>>>	getCoordActionMissingDependencies(String id, String actions, String dates) Get coord action missing dependencies
CoordinatorJobBean	getCoordJob(String jobId) Return the info about a coord job.
CoordinatorJobBean	getCoordJob(String jobId, String filter, int offset, int length, boolean desc) Return the info about a coord job with actions subset.
CoordinatorJobInfo	getCoordJobs(String filter, int start, int len)
String	getDefinition(String jobId) Return the a job definition.
WorkflowJob	getJob(String jobId) Return the info about a wf job.
WorkflowJob	getJob(String jobId, int start, int length) Return the info about a wf job with actions subset.
String	getJobIdForExternalId(String externalId) Return the workflow Job ID for an external ID.
String	getJobStatus(String jobId) Return the status for a Job ID
List<WorkflowJobBean>	getReruns(String coordActionId)
List<CoordinatorWfActionBean>	getWfActionByJobIdAndName(String jobId, String wfActionName, int offset, int len) get wf actions by action name in a coordinator job
CoordinatorActionInfo	ignore(String jobId, String type, String scope)
void	kill(String jobId) Kill a job.
CoordinatorActionInfo	killActions(String jobId, String rangeType, String scope)
CoordinatorJobInfo	killJobs(String filter, int start, int length) return a list of killed Coordinator job
Map<Pair<String,CoordinatorEngine.FILTER_COMPARATORS>,List<Object>>	parseJobFilter(String filter)
void	reRun(String jobId, org.apache.hadoop.conf.Configuration conf) Deprecated.
CoordinatorActionInfo	reRun(String jobId, String rerunType, String scope, boolean refresh, boolean noCleanup, boolean failed, org.apache.hadoop.conf.Configuration conf) Rerun coordinator actions for given rerunType
void	resume(String jobId) Resume a job.
CoordinatorJobInfo	resumeJobs(String filter, int start, int length) return the jobs that've been resumed
void	start(String jobId) Deprecated.
protected void	streamJobLog(XLogStreamer logStreamer, String jobId, Writer writer) Stream job log.
void	streamLog(String jobId, String logRetrievalScope, String logRetrievalType, Writer writer, Map<String,String[]> requestParameters) Add list of actions to the filter based on conditions
String	submitJob(org.apache.hadoop.conf.Configuration conf, boolean startJob) Submit a job.
void	suspend(String jobId) Suspend a job.
CoordinatorJobInfo	suspendJobs(String filter, int start, int length) return the jobs that've been suspended
String	updateJob(org.apache.hadoop.conf.Configuration conf, String jobId, boolean dryrun, boolean showDiff) Update coord job definition.

Methods inherited from class org.apache.oozie.BaseEngine

getJMSTopicName, getUser, streamAuditLog, streamErrorLog, streamLog

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COORD_ACTIONS_LOG_MAX_COUNT

public static final String COORD_ACTIONS_LOG_MAX_COUNT

See Also:

Constant Field Values

VALID_JOB_FILTERS

public static final String[] VALID_JOB_FILTERS

Constructor Detail

CoordinatorEngine

public CoordinatorEngine()

Create a system Coordinator engine, with no user and no group.

CoordinatorEngine

public CoordinatorEngine(String user)

Create a Coordinator engine to perform operations on behave of a user.

Parameters:

user - user name.

Method Detail

getDefinition

public String getDefinition(String jobId)
                     throws BaseEngineException

Description copied from class: BaseEngine

Return the a job definition.

Specified by:

getDefinition in class BaseEngine

Parameters:

jobId - job Id.

Returns:

the job definition.

Throws:

BaseEngineException - thrown if the job definition could no be obtained.

getCoordAction

public CoordinatorActionBean getCoordAction(String actionId)
                                     throws BaseEngineException

Parameters:

actionId - the ID of the action

Returns:

CoordinatorActionBean

Throws:

BaseEngineException - if the bean could not be retrieved

getCoordJob

public CoordinatorJobBean getCoordJob(String jobId)
                               throws BaseEngineException

Description copied from class: BaseEngine

Return the info about a coord job.

Specified by:

getCoordJob in class BaseEngine

Parameters:

jobId - job Id.

Returns:

the coord job info.

Throws:

BaseEngineException - thrown if the job info could not be obtained.

getCoordJob

public CoordinatorJobBean getCoordJob(String jobId,
                                      String filter,
                                      int offset,
                                      int length,
                                      boolean desc)
                               throws BaseEngineException

Description copied from class: BaseEngine

Return the info about a coord job with actions subset.

Specified by:

getCoordJob in class BaseEngine

Parameters:

jobId - job Id.

filter - the status filter

offset - starting from this index in the list of actions belonging to the job

length - number of actions to be returned

desc - true if actions are sorted in a descending order of nominal time, false if asc order

Returns:

the coord job info.

Throws:

BaseEngineException - thrown if the job info could not be obtained.

getJobIdForExternalId

public String getJobIdForExternalId(String externalId)
                             throws CoordinatorEngineException

Description copied from class: BaseEngine

Return the workflow Job ID for an external ID.

This is reverse lookup for recovery purposes.

Specified by:

getJobIdForExternalId in class BaseEngine

Parameters:

externalId - external ID provided at job submission time.

Returns:

the associated workflow job ID if any, null if none.

Throws:

CoordinatorEngineException

kill

public void kill(String jobId)
          throws CoordinatorEngineException

Description copied from class: BaseEngine

Kill a job.

Specified by:

kill in class BaseEngine

Parameters:

jobId - job Id.

Throws:

CoordinatorEngineException

killActions

public CoordinatorActionInfo killActions(String jobId,
                                         String rangeType,
                                         String scope)
                                  throws CoordinatorEngineException

Throws:

CoordinatorEngineException

change

public void change(String jobId,
                   String changeValue)
            throws CoordinatorEngineException

Description copied from class: BaseEngine

Change a coordinator job.

Specified by:

change in class BaseEngine

Parameters:

jobId - job Id.

changeValue - change value.

Throws:

CoordinatorEngineException

ignore

public CoordinatorActionInfo ignore(String jobId,
                                    String type,
                                    String scope)
                             throws CoordinatorEngineException

Throws:

CoordinatorEngineException

reRun

@Deprecated
public void reRun(String jobId,
                              org.apache.hadoop.conf.Configuration conf)
                       throws BaseEngineException

Deprecated.

Description copied from class: BaseEngine

Rerun a job.

Specified by:

reRun in class BaseEngine

Parameters:

jobId - job Id to rerun.

conf - configuration information for the rerun.

Throws:

BaseEngineException - thrown if the job could not be rerun.

reRun

public CoordinatorActionInfo reRun(String jobId,
                                   String rerunType,
                                   String scope,
                                   boolean refresh,
                                   boolean noCleanup,
                                   boolean failed,
                                   org.apache.hadoop.conf.Configuration conf)
                            throws BaseEngineException

Rerun coordinator actions for given rerunType

Parameters:

jobId - the job ID

rerunType - rerun type RestConstants.JOB_COORD_SCOPE_DATE or RestConstants.JOB_COORD_SCOPE_ACTION

scope - the rerun scope for given rerunType separated by ","

refresh - true if user wants to refresh input/output dataset urls

noCleanup - false if user wants to cleanup output events for given rerun actions

failed - true if user wants to rerun only failed nodes

conf - configuration values for actions

Returns:

the action info

Throws:

BaseEngineException - thrown if the actions could not be rerun

resume

public void resume(String jobId)
            throws CoordinatorEngineException

Description copied from class: BaseEngine

Resume a job.

Specified by:

resume in class BaseEngine

Parameters:

jobId - job Id.

Throws:

CoordinatorEngineException

start

@Deprecated
public void start(String jobId)
                       throws BaseEngineException

Deprecated.

Description copied from class: BaseEngine

Start a job.

Specified by:

start in class BaseEngine

Parameters:

jobId - job Id.

Throws:

BaseEngineException - thrown if the job could not be started.

streamJobLog

protected void streamJobLog(XLogStreamer logStreamer,
                            String jobId,
                            Writer writer)
                     throws IOException,
                            BaseEngineException

Description copied from class: BaseEngine

Stream job log.

Specified by:

streamJobLog in class BaseEngine

Parameters:

logStreamer - the log streamer

jobId - the job id

writer - the writer

Throws:

IOException - Signals that an I/O exception has occurred.

BaseEngineException - the base engine exception

streamLog

public void streamLog(String jobId,
                      String logRetrievalScope,
                      String logRetrievalType,
                      Writer writer,
                      Map<String,String[]> requestParameters)
               throws IOException,
                      BaseEngineException,
                      CommandException

Add list of actions to the filter based on conditions

Parameters:

jobId - Job Id

logRetrievalScope - Value for the retrieval type

logRetrievalType - Based on which filter criteria the log is retrieved

writer - writer to stream the log to

requestParameters - additional parameters from the request

Throws:

IOException - in case of IO error

BaseEngineException - if there is an error during streaming

CommandException - if a parameter could not be parsed

submitJob

public String submitJob(org.apache.hadoop.conf.Configuration conf,
                        boolean startJob)
                 throws CoordinatorEngineException

Description copied from class: BaseEngine

Submit a job.

It validates configuration properties.

Specified by:

submitJob in class BaseEngine

Parameters:

conf - job configuration.

startJob - indicates if the job should be started or not.

Returns:

the job Id.

Throws:

CoordinatorEngineException

dryRunSubmit

public String dryRunSubmit(org.apache.hadoop.conf.Configuration conf)
                    throws CoordinatorEngineException

Description copied from class: BaseEngine

Dry run a job; like BaseEngine.submitJob(org.apache.hadoop.conf.Configuration, boolean) but doesn't actually execute the job.

It validates configuration properties.

Specified by:

dryRunSubmit in class BaseEngine

Parameters:

conf - job configuration.

Returns:

the result of the dryrun

Throws:

CoordinatorEngineException

suspend

public void suspend(String jobId)
             throws CoordinatorEngineException

Description copied from class: BaseEngine

Suspend a job.

Specified by:

suspend in class BaseEngine

Parameters:

jobId - job Id.

Throws:

CoordinatorEngineException

getJob

public WorkflowJob getJob(String jobId)
                   throws BaseEngineException

Description copied from class: BaseEngine

Return the info about a wf job.

Specified by:

getJob in class BaseEngine

Parameters:

jobId - job Id.

Returns:

the workflow job info.

Throws:

DagEngineException - thrown if the job info could not be obtained.

BaseEngineException

getJob

public WorkflowJob getJob(String jobId,
                          int start,
                          int length)
                   throws BaseEngineException

Description copied from class: BaseEngine

Return the info about a wf job with actions subset.

Specified by:

getJob in class BaseEngine

Parameters:

jobId - job Id

start - starting from this index in the list of actions belonging to the job

length - number of actions to be returned

Returns:

the workflow job info.

Throws:

DagEngineException - thrown if the job info could not be obtained.

BaseEngineException

getCoordJobs

public CoordinatorJobInfo getCoordJobs(String filter,
                                       int start,
                                       int len)
                                throws CoordinatorEngineException

Parameters:

filter - he filter to parse. Elements must be semicolon-separated name=value pairs. Supported names are inFILTER_NAMES.

start - start from this job in the coordinator

len - maximum number of results

Returns:

CoordinatorJobInfo

Throws:

CoordinatorEngineException - if the job info could no be retrieved

parseJobFilter

public Map<Pair<String,CoordinatorEngine.FILTER_COMPARATORS>,List<Object>> parseJobFilter(String filter)
                                                                                   throws CoordinatorEngineException

Throws:

CoordinatorEngineException

getReruns

public List<WorkflowJobBean> getReruns(String coordActionId)
                                throws CoordinatorEngineException

Throws:

CoordinatorEngineException

updateJob

public String updateJob(org.apache.hadoop.conf.Configuration conf,
                        String jobId,
                        boolean dryrun,
                        boolean showDiff)
                 throws CoordinatorEngineException

Update coord job definition.

Parameters:

conf - the conf

jobId - the job id

dryrun - the dryrun

showDiff - the show diff

Returns:

the string

Throws:

CoordinatorEngineException - the coordinator engine exception

getJobStatus

public String getJobStatus(String jobId)
                    throws CoordinatorEngineException

Return the status for a Job ID

Specified by:

getJobStatus in class BaseEngine

Parameters:

jobId - job Id.

Returns:

the job's status

Throws:

CoordinatorEngineException - thrown if the job's status could not be obtained

getActionStatus

public String getActionStatus(String actionId)
                       throws CoordinatorEngineException

Return the status for an Action ID

Parameters:

actionId - action Id.

Returns:

the action's status

Throws:

CoordinatorEngineException - thrown if the action's status could not be obtained

disableSLAAlert

public void disableSLAAlert(String id,
                            String actions,
                            String dates,
                            String childIds)
                     throws BaseEngineException

Description copied from class: BaseEngine

Disable SLA alert for job

Specified by:

disableSLAAlert in class BaseEngine

Parameters:

id - job ID

actions - list of actions

dates - dates

childIds - child IDs

Throws:

BaseEngineException - thrown if SLA alert could not be disabled

changeSLA

public void changeSLA(String id,
                      String actions,
                      String dates,
                      String childIds,
                      String newParams)
               throws BaseEngineException

Description copied from class: BaseEngine

Change SLA properties for job

Specified by:

changeSLA in class BaseEngine

Parameters:

id - job ID

actions - list of actions

dates - dates

childIds - child IDs

newParams - parameters to

Throws:

BaseEngineException - thrown if SLA alert could not be enabled

enableSLAAlert

public void enableSLAAlert(String id,
                           String actions,
                           String dates,
                           String childIds)
                    throws BaseEngineException

Description copied from class: BaseEngine

Enable SLA alert for job

Specified by:

enableSLAAlert in class BaseEngine

Parameters:

id - job ID

actions - list of actions

dates - dates

childIds - child IDs

Throws:

BaseEngineException - thrown if SLA alert could not be enabled

killJobs

public CoordinatorJobInfo killJobs(String filter,
                                   int start,
                                   int length)
                            throws CoordinatorEngineException

return a list of killed Coordinator job

Parameters:

filter - the filter string for which the coordinator jobs are killed

start - the starting index for coordinator jobs

length - maximum number of jobs to be killed

Returns:

coordinatorJobInfo the list of jobs being killed

Throws:

CoordinatorEngineException - thrown if one or more of the jobs cannot be killed

suspendJobs

public CoordinatorJobInfo suspendJobs(String filter,
                                      int start,
                                      int length)
                               throws CoordinatorEngineException

return the jobs that've been suspended

Parameters:

filter - Filter for jobs that will be suspended, can be name, user, group, status, id or combination of any

start - Offset for the jobs that will be suspended

length - maximum number of jobs that will be suspended

Returns:

coordinatorJobInfo

Throws:

CoordinatorEngineException - if the jobs could not be suspended

resumeJobs

public CoordinatorJobInfo resumeJobs(String filter,
                                     int start,
                                     int length)
                              throws CoordinatorEngineException

return the jobs that've been resumed

Parameters:

filter - Filter for jobs that will be resumed, can be name, user, group, status, id or combination of any

start - Offset for the jobs that will be resumed

length - maximum number of jobs that will be resumed

Returns:

coordinatorJobInfo returns resumed jobs

Throws:

CoordinatorEngineException - if the jobs could not be resumed

getCoordActionMissingDependencies

public List<Pair<CoordinatorActionBean,Map<String,ActionDependency>>> getCoordActionMissingDependencies(String id,
                                                                                                        String actions,
                                                                                                        String dates)
                                                                                                 throws CommandException

Get coord action missing dependencies

Parameters:

id - jobID

actions - action list

dates - nominal time list

Returns:

CoordActionMissingDependenciesXCommand pair of coord action bean and list of missing input dependencies.

Throws:

CommandException - if the actions could not be retrieved

getWfActionByJobIdAndName

public List<CoordinatorWfActionBean> getWfActionByJobIdAndName(String jobId,
                                                               String wfActionName,
                                                               int offset,
                                                               int len)
                                                        throws CoordinatorEngineException

get wf actions by action name in a coordinator job

Parameters:

jobId - coordinator job id

wfActionName - workflow action name

offset - offset in the coordinator job

len - maximum number of results

Returns:

CoordWfActionInfoXCommand list of CoordinatorWfActionBean in a coordinator

Throws:

CoordinatorEngineException - if the actions could not be retrieved