org.apache.oozie.action.hadoop
Class JavaActionExecutor

java.lang.Object
  extended by org.apache.oozie.action.ActionExecutor
      extended by org.apache.oozie.action.hadoop.JavaActionExecutor
Direct Known Subclasses:
DistcpActionExecutor, HiveActionExecutor, MapReduceActionExecutor, PigActionExecutor, ShellActionExecutor, SqoopActionExecutor

public class JavaActionExecutor
extends ActionExecutor


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.oozie.action.ActionExecutor
ActionExecutor.Context
 
Field Summary
protected  XLog log
           
static String MAX_EXTERNAL_STATS_SIZE
           
static int MAX_EXTERNAL_STATS_SIZE_DEFAULT
           
static String OOZIE_COMMON_LIBDIR
           
 
Fields inherited from class org.apache.oozie.action.ActionExecutor
CONF_PREFIX, ERROR_OTHER, MAX_RETRIES, RETRY_INTERVAL
 
Constructor Summary
  JavaActionExecutor()
           
protected JavaActionExecutor(String type)
           
 
Method Summary
protected  void addShareLib(org.apache.hadoop.fs.Path appPath, org.apache.hadoop.conf.Configuration conf, String actionShareLibPostfix)
           
 void check(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
          Check if an action has completed.
 org.apache.hadoop.mapred.JobConf createBaseHadoopConf(ActionExecutor.Context context, org.jdom.Element actionXml)
           
protected  org.apache.hadoop.mapred.JobClient createJobClient(ActionExecutor.Context context, org.apache.hadoop.mapred.JobConf jobConf)
          Create job client object
 void end(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
          End an action after it has executed.
protected  HashMap<String,CredentialsProperties> getActionCredentialsProperties(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
           
protected  void getActionData(org.apache.hadoop.fs.FileSystem actionFs, org.apache.hadoop.mapred.RunningJob runningJob, org.apache.oozie.client.WorkflowAction action, ActionExecutor.Context context)
          Get the output data of an action.
protected  org.apache.hadoop.fs.FileSystem getActionFileSystem(ActionExecutor.Context context, org.jdom.Element actionXml)
           
protected  org.apache.hadoop.fs.FileSystem getActionFileSystem(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
           
protected  boolean getCaptureOutput(org.apache.oozie.client.WorkflowAction action)
           
protected  CredentialsProperties getCredProperties(ActionExecutor.Context context, String credName)
           
protected  List<Class> getLauncherClasses()
           
protected  String getLauncherJarName()
           
protected  String getLauncherMain(org.apache.hadoop.conf.Configuration launcherConf, org.jdom.Element actionXml)
           
static int getMaxExternalStatsSize()
          Get the maximum allowed size of stats
protected  String getShareLibPostFix(ActionExecutor.Context context, org.jdom.Element actionXml)
          Return the sharelib postfix for the action.
 void initActionType()
          Invoked once at system initialization time.
 boolean isCompleted(String externalStatus)
          Return if the external status indicates that the action has completed.
 void kill(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
          Kill an action.
 void prepareActionDir(org.apache.hadoop.fs.FileSystem actionFs, ActionExecutor.Context context)
           
protected  HashMap<String,CredentialsProperties> setCredentialPropertyToActionConf(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action, org.apache.hadoop.conf.Configuration actionConf)
           
protected  void setCredentialTokens(org.apache.hadoop.mapred.JobConf jobconf, ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action, HashMap<String,CredentialsProperties> credPropertiesMap)
           
 void start(ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
          Start an action.
 void submitLauncher(org.apache.hadoop.fs.FileSystem actionFs, ActionExecutor.Context context, org.apache.oozie.client.WorkflowAction action)
           
 
Methods inherited from class org.apache.oozie.action.ActionExecutor
convertException, disableInit, enableInit, getActionDir, getActionDirPath, getActionSignal, getMaxRetries, getOozieConf, getOozieRuntimeDir, getOozieSystemId, getRetryInterval, getType, registerError, resetInitInfo, setMaxRetries, setRetryInterval
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

OOZIE_COMMON_LIBDIR

public static final String OOZIE_COMMON_LIBDIR
See Also:
Constant Field Values

MAX_EXTERNAL_STATS_SIZE_DEFAULT

public static final int MAX_EXTERNAL_STATS_SIZE_DEFAULT
See Also:
Constant Field Values

MAX_EXTERNAL_STATS_SIZE

public static final String MAX_EXTERNAL_STATS_SIZE
See Also:
Constant Field Values

log

protected XLog log
Constructor Detail

JavaActionExecutor

public JavaActionExecutor()

JavaActionExecutor

protected JavaActionExecutor(String type)
Method Detail

getLauncherJarName

protected String getLauncherJarName()

getLauncherClasses

protected List<Class> getLauncherClasses()

initActionType

public void initActionType()
Description copied from class: ActionExecutor
Invoked once at system initialization time.

It can be used to register error information for the expected exceptions. Exceptions should be register from subclasses to superclasses to ensure proper detection, same thing that it is done in a normal catch.

This method should invoke the ActionExecutor.registerError(java.lang.String, org.apache.oozie.action.ActionExecutorException.ErrorType, java.lang.String) method to register all its possible errors.

Subclasses overriding must invoke super.

Overrides:
initActionType in class ActionExecutor

getMaxExternalStatsSize

public static int getMaxExternalStatsSize()
Get the maximum allowed size of stats

Returns:
maximum size of stats

createBaseHadoopConf

public org.apache.hadoop.mapred.JobConf createBaseHadoopConf(ActionExecutor.Context context,
                                                             org.jdom.Element actionXml)

getActionFileSystem

protected org.apache.hadoop.fs.FileSystem getActionFileSystem(ActionExecutor.Context context,
                                                              org.apache.oozie.client.WorkflowAction action)
                                                       throws ActionExecutorException
Throws:
ActionExecutorException

getActionFileSystem

protected org.apache.hadoop.fs.FileSystem getActionFileSystem(ActionExecutor.Context context,
                                                              org.jdom.Element actionXml)
                                                       throws ActionExecutorException
Throws:
ActionExecutorException

prepareActionDir

public void prepareActionDir(org.apache.hadoop.fs.FileSystem actionFs,
                             ActionExecutor.Context context)
                      throws ActionExecutorException
Throws:
ActionExecutorException

addShareLib

protected void addShareLib(org.apache.hadoop.fs.Path appPath,
                           org.apache.hadoop.conf.Configuration conf,
                           String actionShareLibPostfix)
                    throws ActionExecutorException
Throws:
ActionExecutorException

getLauncherMain

protected String getLauncherMain(org.apache.hadoop.conf.Configuration launcherConf,
                                 org.jdom.Element actionXml)

submitLauncher

public void submitLauncher(org.apache.hadoop.fs.FileSystem actionFs,
                           ActionExecutor.Context context,
                           org.apache.oozie.client.WorkflowAction action)
                    throws ActionExecutorException
Throws:
ActionExecutorException

setCredentialPropertyToActionConf

protected HashMap<String,CredentialsProperties> setCredentialPropertyToActionConf(ActionExecutor.Context context,
                                                                                  org.apache.oozie.client.WorkflowAction action,
                                                                                  org.apache.hadoop.conf.Configuration actionConf)
                                                                           throws Exception
Throws:
Exception

setCredentialTokens

protected void setCredentialTokens(org.apache.hadoop.mapred.JobConf jobconf,
                                   ActionExecutor.Context context,
                                   org.apache.oozie.client.WorkflowAction action,
                                   HashMap<String,CredentialsProperties> credPropertiesMap)
                            throws Exception
Throws:
Exception

getActionCredentialsProperties

protected HashMap<String,CredentialsProperties> getActionCredentialsProperties(ActionExecutor.Context context,
                                                                               org.apache.oozie.client.WorkflowAction action)
                                                                        throws Exception
Throws:
Exception

getCredProperties

protected CredentialsProperties getCredProperties(ActionExecutor.Context context,
                                                  String credName)
                                           throws Exception
Throws:
Exception

start

public void start(ActionExecutor.Context context,
                  org.apache.oozie.client.WorkflowAction action)
           throws ActionExecutorException
Description copied from class: ActionExecutor
Start an action.

The ActionExecutor.Context.setStartData(java.lang.String, java.lang.String, java.lang.String) method must be called within this method.

If the action has completed, the ActionExecutor.Context.setExecutionData(java.lang.String, java.util.Properties) method must be called within this method.

Specified by:
start in class ActionExecutor
Parameters:
context - executor context.
action - the action to start.
Throws:
ActionExecutorException - thrown if the action could not start.

end

public void end(ActionExecutor.Context context,
                org.apache.oozie.client.WorkflowAction action)
         throws ActionExecutorException
Description copied from class: ActionExecutor
End an action after it has executed.

The ActionExecutor.Context.setEndData(org.apache.oozie.client.WorkflowAction.Status, java.lang.String) method must be called within this method.

Specified by:
end in class ActionExecutor
Parameters:
context - executor context.
action - the action to end.
Throws:
ActionExecutorException - thrown if the action could not end.

createJobClient

protected org.apache.hadoop.mapred.JobClient createJobClient(ActionExecutor.Context context,
                                                             org.apache.hadoop.mapred.JobConf jobConf)
                                                      throws HadoopAccessorException
Create job client object

Parameters:
context -
jobConf -
Returns:
Throws:
HadoopAccessorException

check

public void check(ActionExecutor.Context context,
                  org.apache.oozie.client.WorkflowAction action)
           throws ActionExecutorException
Description copied from class: ActionExecutor
Check if an action has completed. This method must be implemented by Async Action Executors.

If the action has completed, the ActionExecutor.Context.setExecutionData(java.lang.String, java.util.Properties) method must be called within this method.

If the action has not completed, the ActionExecutor.Context.setExternalStatus(java.lang.String) method must be called within this method.

Specified by:
check in class ActionExecutor
Parameters:
context - executor context.
action - the action to end.
Throws:
ActionExecutorException - thrown if the action could not be checked.

getActionData

protected void getActionData(org.apache.hadoop.fs.FileSystem actionFs,
                             org.apache.hadoop.mapred.RunningJob runningJob,
                             org.apache.oozie.client.WorkflowAction action,
                             ActionExecutor.Context context)
                      throws HadoopAccessorException,
                             org.jdom.JDOMException,
                             IOException,
                             URISyntaxException
Get the output data of an action. Subclasses should override this method to get action specific output data.

Parameters:
actionFs - the FileSystem object
runningJob - the runningJob
action - the Workflow action
context - executor context
Throws:
HadoopAccessorException
org.jdom.JDOMException
IOException
URISyntaxException

getCaptureOutput

protected boolean getCaptureOutput(org.apache.oozie.client.WorkflowAction action)
                            throws org.jdom.JDOMException
Throws:
org.jdom.JDOMException

kill

public void kill(ActionExecutor.Context context,
                 org.apache.oozie.client.WorkflowAction action)
          throws ActionExecutorException
Description copied from class: ActionExecutor
Kill an action.

The ActionExecutor.Context.setEndData(org.apache.oozie.client.WorkflowAction.Status, java.lang.String) method must be called within this method.

Specified by:
kill in class ActionExecutor
Parameters:
context - executor context.
action - the action to kill.
Throws:
ActionExecutorException - thrown if the action could not be killed.

isCompleted

public boolean isCompleted(String externalStatus)
Description copied from class: ActionExecutor
Return if the external status indicates that the action has completed.

Specified by:
isCompleted in class ActionExecutor
Parameters:
externalStatus - external status to check.
Returns:
if the external status indicates that the action has completed.

getShareLibPostFix

protected String getShareLibPostFix(ActionExecutor.Context context,
                                    org.jdom.Element actionXml)
Return the sharelib postfix for the action.

If NULL or emtpy, it means that the action does not use the action sharelib.

If a non-empty string, i.e. foo, it means the action uses the action sharelib subdirectory foo and all JARs in the foo directory will be in the action classpath.

Parameters:
context - executor context.
actionXml - the action XML.
Returns:
the action sharelib post fix, this implementation returns NULL.


Copyright © 2012 Apache Software Foundation. All Rights Reserved.