org.apache.oozie.service
Class HadoopAccessorService

java.lang.Object
  extended by org.apache.oozie.service.HadoopAccessorService
All Implemented Interfaces:
Service

public class HadoopAccessorService
extends Object
implements Service

The HadoopAccessorService returns HadoopAccessor instances configured to work on behalf of a user-group.

The default accessor used is the base accessor which just injects the UGI into the configuration instance used to create/obtain JobClient and FileSystem instances.


Field Summary
static String ACTION_CONFS
           
static String CONF_PREFIX
           
static String[] DEFAULT_SUPPORTED_SCHEMES
           
static String HADOOP_CONFS
           
protected static String HADOOP_JOB_TRACKER
           
protected static String HADOOP_JOB_TRACKER_2
           
protected static String HADOOP_YARN_RM
           
static String JOB_TRACKER_WHITELIST
           
protected static String JT_PRINCIPAL
          The Kerberos principal for the job tracker.
static String KERBEROS_AUTH_ENABLED
           
static String KERBEROS_KEYTAB
           
static String KERBEROS_PRINCIPAL
           
static org.apache.hadoop.io.Text MR_TOKEN_ALIAS
           
static String NAME_NODE_WHITELIST
           
protected static String OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED
           
protected static String RM_PRINCIPAL
          The Kerberos principal for the resource manager.
static String SUPPORTED_FILESYSTEMS
          Supported filesystem schemes for namespace federation
 
Fields inherited from interface org.apache.oozie.service.Service
DEFAULT_LOCK_TIMEOUT, lockTimeout, USE_XCOMMAND
 
Constructor Summary
HadoopAccessorService()
           
 
Method Summary
 void addFileToClassPath(String user, org.apache.hadoop.fs.Path file, org.apache.hadoop.conf.Configuration conf)
           
 void checkSupportedFilesystem(URI uri)
          checks configuration parameter if filesystem scheme is among the list of supported ones this makes system robust to filesystems other than HDFS also
 XConfiguration createActionDefaultConf(String hostPort, String action)
          Returns a Configuration containing any defaults for an action for a particular cluster.
 org.apache.hadoop.fs.FileSystem createFileSystem(String user, URI uri, org.apache.hadoop.conf.Configuration conf)
          Return a FileSystem created with the provided user for the specified URI.
 org.apache.hadoop.mapred.JobClient createJobClient(String user, org.apache.hadoop.mapred.JobConf conf)
          Return a JobClient created with the provided user/group.
 org.apache.hadoop.mapred.JobConf createJobConf(String hostPort)
          Creates a JobConf using the site configuration for the specified hostname:port.
 void destroy()
          Destroy the service.
 Class<? extends Service> getInterface()
          Return the public interface of the service.
 org.apache.hadoop.io.Text getMRDelegationTokenRenewer(org.apache.hadoop.mapred.JobConf jobConf)
           
 Set<String> getSupportedSchemes()
           
 void init(org.apache.hadoop.conf.Configuration conf)
           
 void init(Services services)
          Initialize the service.
protected  void validateJobTracker(String jobTrackerUri)
          Validate Job tracker
protected  void validateNameNode(String nameNodeUri)
          Validate Namenode list
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CONF_PREFIX

public static final String CONF_PREFIX
See Also:
Constant Field Values

JOB_TRACKER_WHITELIST

public static final String JOB_TRACKER_WHITELIST
See Also:
Constant Field Values

NAME_NODE_WHITELIST

public static final String NAME_NODE_WHITELIST
See Also:
Constant Field Values

HADOOP_CONFS

public static final String HADOOP_CONFS
See Also:
Constant Field Values

ACTION_CONFS

public static final String ACTION_CONFS
See Also:
Constant Field Values

KERBEROS_AUTH_ENABLED

public static final String KERBEROS_AUTH_ENABLED
See Also:
Constant Field Values

KERBEROS_KEYTAB

public static final String KERBEROS_KEYTAB
See Also:
Constant Field Values

KERBEROS_PRINCIPAL

public static final String KERBEROS_PRINCIPAL
See Also:
Constant Field Values

MR_TOKEN_ALIAS

public static final org.apache.hadoop.io.Text MR_TOKEN_ALIAS

OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED

protected static final String OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED
See Also:
Constant Field Values

JT_PRINCIPAL

protected static final String JT_PRINCIPAL
The Kerberos principal for the job tracker.

See Also:
Constant Field Values

RM_PRINCIPAL

protected static final String RM_PRINCIPAL
The Kerberos principal for the resource manager.

See Also:
Constant Field Values

HADOOP_JOB_TRACKER

protected static final String HADOOP_JOB_TRACKER
See Also:
Constant Field Values

HADOOP_JOB_TRACKER_2

protected static final String HADOOP_JOB_TRACKER_2
See Also:
Constant Field Values

HADOOP_YARN_RM

protected static final String HADOOP_YARN_RM
See Also:
Constant Field Values

SUPPORTED_FILESYSTEMS

public static final String SUPPORTED_FILESYSTEMS
Supported filesystem schemes for namespace federation

See Also:
Constant Field Values

DEFAULT_SUPPORTED_SCHEMES

public static final String[] DEFAULT_SUPPORTED_SCHEMES
Constructor Detail

HadoopAccessorService

public HadoopAccessorService()
Method Detail

init

public void init(Services services)
          throws ServiceException
Description copied from interface: Service
Initialize the service.

Invoked by the Service singleton at start up time.

Specified by:
init in interface Service
Parameters:
services - services singleton initializing the service.
Throws:
ServiceException - thrown if the service could not initialize.

init

public void init(org.apache.hadoop.conf.Configuration conf)
          throws ServiceException
Throws:
ServiceException

destroy

public void destroy()
Description copied from interface: Service
Destroy the service.

Invoked by the Service singleton at shutdown time.

Specified by:
destroy in interface Service

getInterface

public Class<? extends Service> getInterface()
Description copied from interface: Service
Return the public interface of the service.

Services are retrieved by its public interface. Specializations of services must return the public interface.

Specified by:
getInterface in interface Service
Returns:
the interface of the service.

createJobConf

public org.apache.hadoop.mapred.JobConf createJobConf(String hostPort)
Creates a JobConf using the site configuration for the specified hostname:port.

If the specified hostname:port is not defined it falls back to the '*' site configuration if available. If the '*' site configuration is not available, the JobConf has all Hadoop defaults.

Parameters:
hostPort - hostname:port to lookup Hadoop site configuration.
Returns:
a JobConf with the corresponding site configuration for hostPort.

createActionDefaultConf

public XConfiguration createActionDefaultConf(String hostPort,
                                              String action)
Returns a Configuration containing any defaults for an action for a particular cluster.

This configuration is used as default for the action configuration and enables cluster level default values per action.

Parameters:
hostPort - hostname"port to lookup the action default confiugration.
action - action name.
Returns:
the default configuration for the action for the specified cluster.

createJobClient

public org.apache.hadoop.mapred.JobClient createJobClient(String user,
                                                          org.apache.hadoop.mapred.JobConf conf)
                                                   throws HadoopAccessorException
Return a JobClient created with the provided user/group.

Parameters:
conf - JobConf with all necessary information to create the JobClient.
Returns:
JobClient created with the provided user/group.
Throws:
HadoopAccessorException - if the client could not be created.

createFileSystem

public org.apache.hadoop.fs.FileSystem createFileSystem(String user,
                                                        URI uri,
                                                        org.apache.hadoop.conf.Configuration conf)
                                                 throws HadoopAccessorException
Return a FileSystem created with the provided user for the specified URI.

Parameters:
uri - file system URI.
conf - Configuration with all necessary information to create the FileSystem.
Returns:
FileSystem created with the provided user/group.
Throws:
HadoopAccessorException - if the filesystem could not be created.

validateJobTracker

protected void validateJobTracker(String jobTrackerUri)
                           throws HadoopAccessorException
Validate Job tracker

Parameters:
jobTrackerUri -
Throws:
HadoopAccessorException

validateNameNode

protected void validateNameNode(String nameNodeUri)
                         throws HadoopAccessorException
Validate Namenode list

Parameters:
nameNodeUri -
Throws:
HadoopAccessorException

getMRDelegationTokenRenewer

public org.apache.hadoop.io.Text getMRDelegationTokenRenewer(org.apache.hadoop.mapred.JobConf jobConf)
                                                      throws IOException
Throws:
IOException

addFileToClassPath

public void addFileToClassPath(String user,
                               org.apache.hadoop.fs.Path file,
                               org.apache.hadoop.conf.Configuration conf)
                        throws IOException
Throws:
IOException

checkSupportedFilesystem

public void checkSupportedFilesystem(URI uri)
                              throws HadoopAccessorException
checks configuration parameter if filesystem scheme is among the list of supported ones this makes system robust to filesystems other than HDFS also

Throws:
HadoopAccessorException

getSupportedSchemes

public Set<String> getSupportedSchemes()


Copyright © 2013 Apache Software Foundation. All Rights Reserved.