org.apache.oozie
Class DagEngine

java.lang.Object
  extended by org.apache.oozie.BaseEngine
      extended by org.apache.oozie.DagEngine

public class DagEngine
extends BaseEngine

The DagEngine provides all the DAG engine functionality for WS calls.


Field Summary
 
Fields inherited from class org.apache.oozie.BaseEngine
USE_XCOMMAND, user
 
Constructor Summary
DagEngine()
          Create a system Dag engine, with no user and no group.
DagEngine(String user)
          Create a Dag engine to perform operations on behave of a user.
 
Method Summary
 void change(String jobId, String changeValue)
          Change a coordinator job.
 String dryRunSubmit(org.apache.hadoop.conf.Configuration conf)
          Dry run a job; like {@link BaseEngine#submitJob(org.apache.hadoop.conf.Configuration, boolean) but doesn't actually execute the job.
 org.apache.oozie.client.CoordinatorJob getCoordJob(String jobId)
          Return the info about a coord job.
 org.apache.oozie.client.CoordinatorJob getCoordJob(String jobId, String filter, int start, int length, boolean desc)
          Return the info about a coord job with actions subset.
 String getDefinition(String jobId)
          Return the a job definition.
 org.apache.oozie.client.WorkflowJob getJob(String jobId)
          Return the info about a job.
 org.apache.oozie.client.WorkflowJob getJob(String jobId, int start, int length)
          Return the info about a job with actions subset.
 String getJobIdForExternalId(String externalId)
          Return the workflow Job ID for an external ID.
 WorkflowsInfo getJobs(String filter, int start, int len)
          Return the info about a set of jobs.
 WorkflowActionBean getWorkflowAction(String actionId)
           
 void kill(String jobId)
          Kill a job.
protected  Map<String,List<String>> parseFilter(String filter)
          Validate a jobs filter.
 void processCallback(String actionId, String externalStatus, Properties actionData)
          Process an action callback.
 void reRun(String jobId, org.apache.hadoop.conf.Configuration conf)
          Rerun a job.
 void resume(String jobId)
          Resume a job.
 void start(String jobId)
          Start a job.
 void streamLog(String jobId, Writer writer)
          Stream the log of a job.
 String submitHttpJob(org.apache.hadoop.conf.Configuration conf, String jobType)
          Submit a pig/hive/mapreduce job through HTTP.
 String submitJob(org.apache.hadoop.conf.Configuration conf, boolean startJob)
          Submit a workflow job.
 String submitJobFromCoordinator(org.apache.hadoop.conf.Configuration conf, String parentId)
          Submit a workflow through a coordinator.
 void suspend(String jobId)
          Suspend a job.
 
Methods inherited from class org.apache.oozie.BaseEngine
getJMSTopicName, getUser
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DagEngine

public DagEngine()
Create a system Dag engine, with no user and no group.


DagEngine

public DagEngine(String user)
Create a Dag engine to perform operations on behave of a user.

Parameters:
user - user name.
Method Detail

submitJob

public String submitJob(org.apache.hadoop.conf.Configuration conf,
                        boolean startJob)
                 throws DagEngineException
Submit a workflow job.

It validates configuration properties.

Specified by:
submitJob in class BaseEngine
Parameters:
conf - job configuration.
startJob - indicates if the job should be started or not.
Returns:
the job Id.
Throws:
DagEngineException - thrown if the job could not be created.

submitJobFromCoordinator

public String submitJobFromCoordinator(org.apache.hadoop.conf.Configuration conf,
                                       String parentId)
                                throws DagEngineException
Submit a workflow through a coordinator. It validates configuration properties.

Parameters:
conf - job conf
parentId - parent of workflow
Returns:
Throws:
DagEngineException

submitHttpJob

public String submitHttpJob(org.apache.hadoop.conf.Configuration conf,
                            String jobType)
                     throws DagEngineException
Submit a pig/hive/mapreduce job through HTTP.

It validates configuration properties.

Parameters:
conf - job configuration.
jobType - job type - can be "pig", "hive, or "mapreduce".
Returns:
the job Id.
Throws:
DagEngineException - thrown if the job could not be created.

start

public void start(String jobId)
           throws DagEngineException
Start a job.

Specified by:
start in class BaseEngine
Parameters:
jobId - job Id.
Throws:
DagEngineException - thrown if the job could not be started.

resume

public void resume(String jobId)
            throws DagEngineException
Resume a job.

Specified by:
resume in class BaseEngine
Parameters:
jobId - job Id.
Throws:
DagEngineException - thrown if the job could not be resumed.

suspend

public void suspend(String jobId)
             throws DagEngineException
Suspend a job.

Specified by:
suspend in class BaseEngine
Parameters:
jobId - job Id.
Throws:
DagEngineException - thrown if the job could not be suspended.

kill

public void kill(String jobId)
          throws DagEngineException
Kill a job.

Specified by:
kill in class BaseEngine
Parameters:
jobId - job Id.
Throws:
DagEngineException - thrown if the job could not be killed.

change

public void change(String jobId,
                   String changeValue)
            throws DagEngineException
Description copied from class: BaseEngine
Change a coordinator job.

Specified by:
change in class BaseEngine
Parameters:
jobId - job Id.
changeValue - change value.
Throws:
DagEngineException

reRun

public void reRun(String jobId,
                  org.apache.hadoop.conf.Configuration conf)
           throws DagEngineException
Rerun a job.

Specified by:
reRun in class BaseEngine
Parameters:
jobId - job Id to rerun.
conf - configuration information for the rerun.
Throws:
DagEngineException - thrown if the job could not be rerun.

processCallback

public void processCallback(String actionId,
                            String externalStatus,
                            Properties actionData)
                     throws DagEngineException
Process an action callback.

Parameters:
actionId - the action Id.
externalStatus - the action external status.
actionData - the action output data, null if none.
Throws:
DagEngineException - thrown if the callback could not be processed.

getJob

public org.apache.oozie.client.WorkflowJob getJob(String jobId)
                                           throws DagEngineException
Return the info about a job.

Specified by:
getJob in class BaseEngine
Parameters:
jobId - job Id.
Returns:
the workflow job info.
Throws:
DagEngineException - thrown if the job info could not be obtained.

getJob

public org.apache.oozie.client.WorkflowJob getJob(String jobId,
                                                  int start,
                                                  int length)
                                           throws DagEngineException
Return the info about a job with actions subset.

Specified by:
getJob in class BaseEngine
Parameters:
jobId - job Id
start - starting from this index in the list of actions belonging to the job
length - number of actions to be returned
Returns:
the workflow job info.
Throws:
DagEngineException - thrown if the job info could not be obtained.

getDefinition

public String getDefinition(String jobId)
                     throws DagEngineException
Return the a job definition.

Specified by:
getDefinition in class BaseEngine
Parameters:
jobId - job Id.
Returns:
the job definition.
Throws:
DagEngineException - thrown if the job definition could no be obtained.

streamLog

public void streamLog(String jobId,
                      Writer writer)
               throws IOException,
                      DagEngineException
Stream the log of a job.

Specified by:
streamLog in class BaseEngine
Parameters:
jobId - job Id.
writer - writer to stream the log to.
Throws:
IOException - thrown if the log cannot be streamed.
DagEngineException - thrown if there is error in getting the Workflow Information for jobId.

parseFilter

protected Map<String,List<String>> parseFilter(String filter)
                                        throws DagEngineException
Validate a jobs filter.

Parameters:
filter - filter to validate.
Returns:
the parsed filter.
Throws:
DagEngineException - thrown if the filter is invalid.

getJobs

public WorkflowsInfo getJobs(String filter,
                             int start,
                             int len)
                      throws DagEngineException
Return the info about a set of jobs.

Parameters:
filter - job filter. Refer to the OozieClient for the filter syntax.
start - offset, base 1.
len - number of jobs to return.
Returns:
job info for all matching jobs, the jobs don't contain node action information.
Throws:
DagEngineException - thrown if the jobs info could not be obtained.

getJobIdForExternalId

public String getJobIdForExternalId(String externalId)
                             throws DagEngineException
Return the workflow Job ID for an external ID.

This is reverse lookup for recovery purposes.

Specified by:
getJobIdForExternalId in class BaseEngine
Parameters:
externalId - external ID provided at job submission time.
Returns:
the associated workflow job ID if any, null if none.
Throws:
DagEngineException - thrown if the lookup could not be done.

getCoordJob

public org.apache.oozie.client.CoordinatorJob getCoordJob(String jobId)
                                                   throws BaseEngineException
Description copied from class: BaseEngine
Return the info about a coord job.

Specified by:
getCoordJob in class BaseEngine
Parameters:
jobId - job Id.
Returns:
the coord job info.
Throws:
BaseEngineException - thrown if the job info could not be obtained.

getCoordJob

public org.apache.oozie.client.CoordinatorJob getCoordJob(String jobId,
                                                          String filter,
                                                          int start,
                                                          int length,
                                                          boolean desc)
                                                   throws BaseEngineException
Description copied from class: BaseEngine
Return the info about a coord job with actions subset.

Specified by:
getCoordJob in class BaseEngine
Parameters:
jobId - job Id.
filter - the status filter
start - starting from this index in the list of actions belonging to the job
length - number of actions to be returned
Returns:
the coord job info.
Throws:
BaseEngineException - thrown if the job info could not be obtained.

getWorkflowAction

public WorkflowActionBean getWorkflowAction(String actionId)
                                     throws BaseEngineException
Throws:
BaseEngineException

dryRunSubmit

public String dryRunSubmit(org.apache.hadoop.conf.Configuration conf)
                    throws BaseEngineException
Description copied from class: BaseEngine
Dry run a job; like {@link BaseEngine#submitJob(org.apache.hadoop.conf.Configuration, boolean) but doesn't actually execute the job.

It validates configuration properties.

Specified by:
dryRunSubmit in class BaseEngine
Parameters:
conf - job configuration.
Returns:
the result of the dryrun
Throws:
BaseEngineException - thrown if there was a problem doing the dryrun


Copyright © 2013 Apache Software Foundation. All Rights Reserved.