org.apache.oozie.action.hadoop

Class MapReduceActionExecutor

java.lang.Object

org.apache.oozie.action.ActionExecutor

org.apache.oozie.action.hadoop.JavaActionExecutor

org.apache.oozie.action.hadoop.MapReduceActionExecutor

public class MapReduceActionExecutor
extends JavaActionExecutor

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.oozie.action.ActionExecutor

ActionExecutor.Context, ActionExecutor.RETRYPOLICY

Field Summary

Fields

Modifier and Type	Field and Description
static String	HADOOP_COUNTERS
static String	JOB_END_NOTIFICATION_URL
static String	OOZIE_ACTION_EXTERNAL_STATS_WRITE
static String	OOZIE_MAPREDUCE_UBER_JAR_ENABLE

Fields inherited from class org.apache.oozie.action.hadoop.JavaActionExecutor

ACL_MODIFY_JOB, ACL_VIEW_JOB, DEFAULT_LAUNCHER_MAX_ATTEMPS, DEFAULT_LAUNCHER_MEMORY_MB, DEFAULT_LAUNCHER_PRIORITY, DEFAULT_LAUNCHER_QUEUE, DEFAULT_LAUNCHER_VCORES, FAILED, FAILED_KILLED, HADOOP_CHILD_JAVA_ENV, HADOOP_CHILD_JAVA_OPTS, HADOOP_JOB_CLASSLOADER, HADOOP_MAP_JAVA_ENV, HADOOP_MAP_JAVA_OPTS, HADOOP_MAP_MEMORY_MB, HADOOP_NAME_NODE, HADOOP_REDUCE_JAVA_OPTS, HADOOP_USER, HADOOP_USER_CLASSPATH_FIRST, HADOOP_YARN_KILL_CHILD_JOBS_ON_AMRESTART, HADOOP_YARN_RM, HADOOP_YARN_TIMELINE_SERVICE_ENABLED, HADOOP_YARN_UBER_MODE, KILLED, LAUNCER_MODIFY_ACL, LAUNCER_VIEW_ACL, LOG, MAPREDUCE_TO_CLASSPATH, MAX_EXTERNAL_STATS_SIZE, OOZIE_ACTION_LAUNCHER_PREFIX, OOZIE_COMMON_LIBDIR, OOZIE_CREDENTIALS_SKIP, OOZIE_LAUNCHER_ADD_MAPREDUCE_TO_CLASSPATH_PROPERTY, RUNNING, SUCCEEDED, workflowConf, YARN_AM_COMMAND_OPTS, YARN_AM_ENV, YARN_AM_RESOURCE_MB, YARN_MEMORY_MB_MIN

Fields inherited from class org.apache.oozie.action.ActionExecutor

ACTION_RETRY_INTERVAL, ACTION_RETRY_POLICY, CONF_PREFIX, ERROR_OTHER, MAX_RETRIES, OOZIE_ACTION_YARN_TAG, RETRY_INTERVAL

Constructor Summary

Constructors

Constructor and Description
MapReduceActionExecutor()

Method Summary

Modifier and Type	Method and Description
void	check(ActionExecutor.Context context, WorkflowAction action) Check if an action has completed.
protected org.apache.hadoop.conf.Configuration	createBaseHadoopConf(ActionExecutor.Context context, org.jdom.Element actionXml, boolean loadResources)
void	end(ActionExecutor.Context context, WorkflowAction action) End an action after it has executed.
protected String	getActualExternalId(WorkflowAction action) Useful for overriding in actions that do subsequent job runs such as the MapReduce Action, where the launcher job is not the actual job that then gets monitored.
protected String	getDefaultShareLibName(org.jdom.Element actionXml) Return the sharelib name for the action.
List<Class<?>>	getLauncherClasses()
protected String	getLauncherMain(org.apache.hadoop.conf.Configuration launcherConf, org.jdom.Element actionXml)
protected void	injectCallback(ActionExecutor.Context context, org.apache.hadoop.conf.Configuration conf)
static void	setStreaming(org.apache.hadoop.conf.Configuration conf, String mapper, String reducer, String recordReader, String[] recordReaderMapping, String[] env)

Methods inherited from class org.apache.oozie.action.hadoop.JavaActionExecutor

addActionLibs, addActionSpecificEnvVars, addAppNameContext, addShareLib, addSystemShareLibForAction, createBaseHadoopConf, createJobClient, createYarnClient, getActionCredentialsProperties, getActionData, getAppName, getCaptureOutput, getCommonLauncherClasses, getCredProperties, getMaxExternalStatsSize, getMaxOutputData, getShareLibFilesForActionConf, getShareLibNames, getTrimmedEncodedPath, initActionType, isCompleted, kill, loadHadoopDefaultResources, parseJobXmlAndConfiguration, parseJobXmlAndConfiguration, prepareActionDir, requiresNameNodeJobTracker, setActionCompletionData, setActionTokenProperties, setCredentialTokens, setLibFilesArchives, start, submitLauncher, supportsConfigurationJobXML

Methods inherited from class org.apache.oozie.action.ActionExecutor

convertException, disableInit, enableInit, getActionDir, getActionDirPath, getActionSignal, getActionYarnTag, getMaxRetries, getOozieConf, getOozieRuntimeDir, getOozieSystemId, getRetryInterval, getRetryPolicy, getType, registerError, resetInitInfo, setMaxRetries, setRetryInterval, setRetryPolicy

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OOZIE_ACTION_EXTERNAL_STATS_WRITE

public static final String OOZIE_ACTION_EXTERNAL_STATS_WRITE

See Also:

Constant Field Values

HADOOP_COUNTERS

public static final String HADOOP_COUNTERS

See Also:

Constant Field Values

OOZIE_MAPREDUCE_UBER_JAR_ENABLE

public static final String OOZIE_MAPREDUCE_UBER_JAR_ENABLE

See Also:

Constant Field Values

JOB_END_NOTIFICATION_URL

public static final String JOB_END_NOTIFICATION_URL

See Also:

Constant Field Values

Constructor Detail

MapReduceActionExecutor

public MapReduceActionExecutor()

Method Detail

getLauncherClasses

public List<Class<?>> getLauncherClasses()

Overrides:

getLauncherClasses in class JavaActionExecutor

getActualExternalId

protected String getActualExternalId(WorkflowAction action)

Description copied from class: JavaActionExecutor

Useful for overriding in actions that do subsequent job runs such as the MapReduce Action, where the launcher job is not the actual job that then gets monitored.

Overrides:

getActualExternalId in class JavaActionExecutor

Returns:

external ID.

getLauncherMain

protected String getLauncherMain(org.apache.hadoop.conf.Configuration launcherConf,
                                 org.jdom.Element actionXml)

Overrides:

getLauncherMain in class JavaActionExecutor

createBaseHadoopConf

protected org.apache.hadoop.conf.Configuration createBaseHadoopConf(ActionExecutor.Context context,
                                                                    org.jdom.Element actionXml,
                                                                    boolean loadResources)

Overrides:

createBaseHadoopConf in class JavaActionExecutor

end

public void end(ActionExecutor.Context context,
                WorkflowAction action)
         throws ActionExecutorException

Description copied from class: ActionExecutor

End an action after it has executed.

The ActionExecutor.Context.setEndData(org.apache.oozie.client.WorkflowAction.Status, java.lang.String) method must be called within this method.

Overrides:

end in class JavaActionExecutor

Parameters:

context - executor context.

action - the action to end.

Throws:

ActionExecutorException - thrown if the action could not end.

getDefaultShareLibName

protected String getDefaultShareLibName(org.jdom.Element actionXml)

Return the sharelib name for the action.

Overrides:

getDefaultShareLibName in class JavaActionExecutor

Parameters:

actionXml -

Returns:

returns streaming if mapreduce-streaming action, NULL otherwise.

setStreaming

public static void setStreaming(org.apache.hadoop.conf.Configuration conf,
                                String mapper,
                                String reducer,
                                String recordReader,
                                String[] recordReaderMapping,
                                String[] env)

injectCallback

protected void injectCallback(ActionExecutor.Context context,
                              org.apache.hadoop.conf.Configuration conf)

Overrides:

injectCallback in class JavaActionExecutor

check

public void check(ActionExecutor.Context context,
                  WorkflowAction action)
           throws ActionExecutorException

Description copied from class: ActionExecutor

Check if an action has completed. This method must be implemented by Async Action Executors.

If the action has completed, the ActionExecutor.Context.setExecutionData(java.lang.String, java.util.Properties) method must be called within this method.

If the action has not completed, the ActionExecutor.Context.setExternalStatus(java.lang.String) method must be called within this method.

Overrides:

check in class JavaActionExecutor

Parameters:

context - executor context.

action - the action to end.

Throws:

ActionExecutorException - thrown if the action could not be checked.