org.apache.oozie.service
Class HadoopAccessorService

java.lang.Object
  extended by org.apache.oozie.service.HadoopAccessorService
All Implemented Interfaces:
Service

public class HadoopAccessorService
extends Object
implements Service

The HadoopAccessorService returns HadoopAccessor instances configured to work on behalf of a user-group.

The default accessor used is the base accessor which just injects the UGI into the configuration instance used to create/obtain JobClient and ileSystem instances.

The HadoopAccess class to use can be configured in the oozie-site.xml using the oozie.service.HadoopAccessorService.accessor.class property.


Field Summary
static String ACTION_CONFS
           
static String CONF_PREFIX
           
static String HADOOP_CONFS
           
static String JOB_TRACKER_WHITELIST
           
static String KERBEROS_AUTH_ENABLED
           
static String KERBEROS_KEYTAB
           
static String KERBEROS_PRINCIPAL
           
static org.apache.hadoop.io.Text MR_TOKEN_ALIAS
           
static String NAME_NODE_WHITELIST
           
static String SUPPORTED_FILESYSTEMS
          Supported filesystem schemes for namespace federation
 
Fields inherited from interface org.apache.oozie.service.Service
DEFAULT_LOCK_TIMEOUT, lockTimeout, USE_XCOMMAND
 
Constructor Summary
HadoopAccessorService()
           
 
Method Summary
 void addFileToClassPath(String user, org.apache.hadoop.fs.Path file, org.apache.hadoop.conf.Configuration conf)
           
 void checkSupportedFilesystem(URI uri)
          checks configuration parameter if filesystem scheme is among the list of supported ones this makes system robust to filesystems other than HDFS also
 XConfiguration createActionDefaultConf(String hostPort, String action)
          Returns a Configuration containing any defaults for an action for a particular cluster.
 org.apache.hadoop.fs.FileSystem createFileSystem(String user, URI uri, org.apache.hadoop.conf.Configuration conf)
          Return a FileSystem created with the provided user for the specified URI.
 org.apache.hadoop.mapred.JobClient createJobClient(String user, org.apache.hadoop.mapred.JobConf conf)
          Return a JobClient created with the provided user/group.
 org.apache.hadoop.mapred.JobConf createJobConf(String hostPort)
          Creates a JobConf using the site configuration for the specified hostname:port.
 void destroy()
          Destroy the service.
 Class<? extends Service> getInterface()
          Return the public interface of the service.
static org.apache.hadoop.io.Text getMRDelegationTokenRenewer(org.apache.hadoop.mapred.JobConf jobConf)
           
 void init(org.apache.hadoop.conf.Configuration conf)
           
 void init(Services services)
          Initialize the service.
protected  void validateJobTracker(String jobTrackerUri)
          Validate Job tracker
protected  void validateNameNode(String nameNodeUri)
          Validate Namenode list
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CONF_PREFIX

public static final String CONF_PREFIX
See Also:
Constant Field Values

JOB_TRACKER_WHITELIST

public static final String JOB_TRACKER_WHITELIST
See Also:
Constant Field Values

NAME_NODE_WHITELIST

public static final String NAME_NODE_WHITELIST
See Also:
Constant Field Values

HADOOP_CONFS

public static final String HADOOP_CONFS
See Also:
Constant Field Values

ACTION_CONFS

public static final String ACTION_CONFS
See Also:
Constant Field Values

KERBEROS_AUTH_ENABLED

public static final String KERBEROS_AUTH_ENABLED
See Also:
Constant Field Values

KERBEROS_KEYTAB

public static final String KERBEROS_KEYTAB
See Also:
Constant Field Values

KERBEROS_PRINCIPAL

public static final String KERBEROS_PRINCIPAL
See Also:
Constant Field Values

MR_TOKEN_ALIAS

public static final org.apache.hadoop.io.Text MR_TOKEN_ALIAS

SUPPORTED_FILESYSTEMS

public static final String SUPPORTED_FILESYSTEMS
Supported filesystem schemes for namespace federation

See Also:
Constant Field Values
Constructor Detail

HadoopAccessorService

public HadoopAccessorService()
Method Detail

init

public void init(Services services)
          throws ServiceException
Description copied from interface: Service
Initialize the service.

Invoked by the Service singleton at start up time.

Specified by:
init in interface Service
Parameters:
services - services singleton initializing the service.
Throws:
ServiceException - thrown if the service could not initialize.

init

public void init(org.apache.hadoop.conf.Configuration conf)
          throws ServiceException
Throws:
ServiceException

destroy

public void destroy()
Description copied from interface: Service
Destroy the service.

Invoked by the Service singleton at shutdown time.

Specified by:
destroy in interface Service

getInterface

public Class<? extends Service> getInterface()
Description copied from interface: Service
Return the public interface of the service.

Services are retrieved by its public interface. Specializations of services must return the public interface.

Specified by:
getInterface in interface Service
Returns:
the interface of the service.

createJobConf

public org.apache.hadoop.mapred.JobConf createJobConf(String hostPort)
Creates a JobConf using the site configuration for the specified hostname:port.

If the specified hostname:port is not defined it falls back to the '*' site configuration if available. If the '*' site configuration is not available, the JobConf has all Hadoop defaults.

Parameters:
hostPort - hostname:port to lookup Hadoop site configuration.
Returns:
a JobConf with the corresponding site configuration for hostPort.

createActionDefaultConf

public XConfiguration createActionDefaultConf(String hostPort,
                                              String action)
Returns a Configuration containing any defaults for an action for a particular cluster.

This configuration is used as default for the action configuration and enables cluster level default values per action.

Parameters:
hostPort - hostname"port to lookup the action default confiugration.
action - action name.
Returns:
the default configuration for the action for the specified cluster.

createJobClient

public org.apache.hadoop.mapred.JobClient createJobClient(String user,
                                                          org.apache.hadoop.mapred.JobConf conf)
                                                   throws HadoopAccessorException
Return a JobClient created with the provided user/group.

Parameters:
conf - JobConf with all necessary information to create the JobClient.
Returns:
JobClient created with the provided user/group.
Throws:
HadoopAccessorException - if the client could not be created.

createFileSystem

public org.apache.hadoop.fs.FileSystem createFileSystem(String user,
                                                        URI uri,
                                                        org.apache.hadoop.conf.Configuration conf)
                                                 throws HadoopAccessorException
Return a FileSystem created with the provided user for the specified URI.

Parameters:
uri - file system URI.
conf - Configuration with all necessary information to create the FileSystem.
Returns:
FileSystem created with the provided user/group.
Throws:
HadoopAccessorException - if the filesystem could not be created.

validateJobTracker

protected void validateJobTracker(String jobTrackerUri)
                           throws HadoopAccessorException
Validate Job tracker

Parameters:
jobTrackerUri -
Throws:
HadoopAccessorException

validateNameNode

protected void validateNameNode(String nameNodeUri)
                         throws HadoopAccessorException
Validate Namenode list

Parameters:
nameNodeUri -
Throws:
HadoopAccessorException

getMRDelegationTokenRenewer

public static org.apache.hadoop.io.Text getMRDelegationTokenRenewer(org.apache.hadoop.mapred.JobConf jobConf)
                                                             throws IOException
Throws:
IOException

addFileToClassPath

public void addFileToClassPath(String user,
                               org.apache.hadoop.fs.Path file,
                               org.apache.hadoop.conf.Configuration conf)
                        throws IOException
Throws:
IOException

checkSupportedFilesystem

public void checkSupportedFilesystem(URI uri)
                              throws HadoopAccessorException
checks configuration parameter if filesystem scheme is among the list of supported ones this makes system robust to filesystems other than HDFS also

Throws:
HadoopAccessorException


Copyright © 2013 Apache Software Foundation. All Rights Reserved.