org.apache.oozie.util

Class ZKUtils

java.lang.Object

org.apache.oozie.util.ZKUtils

public class ZKUtils
extends Object

This class provides a singleton for interacting with ZooKeeper that other classes can use. It handles connecting to ZooKeeper, service discovery, and publishing metadata about this server.

Users of this class should call register(java.lang.Object) to obtain the singleton. This will ensure that we're properly connected and ready to go with ZooKeeper. When the user is done (i.e. on shutdown), it should call unregister(java.lang.Object) to let this class know; once there are no more users, this class will automatically remove itself from ZooKeeper.

Each Oozie Server provides metadata that can be shared with the other Oozie Servers. To keep things simple and to make it easy to add additional metadata in the future, we share a Map. They keys are defined in ZKUtils.ZKMetadataKeys.

For the service discovery, the structure in ZooKeeper is /oozie.zookeeper.namespace/ZK_BASE_SERVICES_PATH/ (default is /oozie/services/). ZKUtils has a service named "servers" under which each Oozie server creates a ZNode named ${OOZIE_SERVICE_INSTANCE} (default is the hostname) that contains the metadata payload. For example, with the default settings, an Oozie server named "foo" would create a ZNode at /oozie/services/servers/foo where the foo ZNode contains the metadata.

If oozie.zookeeper.secure is set to true, then Oozie will (a) use jaas to connect to ZooKeeper using SASL/Kerberos based on Oozie's existing security configuration parameters (b) use/convert every znode under the namespace (including the namespace itself) to have ACLs such that only Oozie servers have access (i.e. if "service/host@REALM" is the Kerberos principal, then "service" will be used for the ACLs).

Oozie server will shutdown itself if ZK connection is lost for ${ZK_CONNECTION_TIMEOUT}.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ZKUtils.SASLOwnerACLProvider Simple implementation of an ACLProvider that simply returns saslACL.
class	ZKUtils.ZKMetadataKeys Keys used in the metadata provided by each Oozie Server to ZooKeeper and other Oozie Servers

Field Summary

Fields

Modifier and Type	Field and Description
static String	OOZIE_INSTANCE_ID oozie-env environment variable for specifying the Oozie instance ID
static String	ZK_BASE_SERVICES_PATH Services that need to put a node in zookeeper should go under here.
static String	ZK_CONNECTION_STRING oozie-site property for specifying the ZooKeeper connection string.
static String	ZK_CONNECTION_TIMEOUT Default ZK connection timeout ( in sec).
static String	ZK_NAMESPACE oozie-site property for specifying the ZooKeeper namespace to use (e.g.
static String	ZK_SECURE oozie-site property for specifying that ZooKeeper is secure.

Method Summary

Methods

Modifier and Type	Method and Description
List<org.apache.curator.x.discovery.ServiceInstance<Map>>	getAllMetaData() Returns a list of the metadata provided by all of the Oozie Servers.
org.apache.curator.framework.CuratorFramework	getClient() Returns the CuratorFramework used for managing the ZooKeeper connection; it can be used by calling classes to perform more direct operations on ZooKeeper.
static org.apache.curator.RetryPolicy	getRetryPolicy() Returns retry policy
static Set<Object>	getUsers() Useful for tests to get the registered classes
static int	getZKConnectionTimeout() Return ZK connection timeout
String	getZKId() Returns the ID of this Oozie Server as seen by ZooKeeper and other Oozie Servers
int	getZKIdIndex(List<org.apache.curator.x.discovery.ServiceInstance<Map>> oozies) Returns the index of this Oozie Server in ZooKeeper's list of Oozie Servers (ordered by registration time)
static String	getZKNameSpace() Returns configured zk namesapces
static ZKUtils	register(Object user) Classes that want to use ZooKeeper should call this method to get the ZKUtils singleton.
void	unregister(Object user) Classes should call this when they are done (i.e.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ZK_CONNECTION_STRING

public static final String ZK_CONNECTION_STRING

oozie-site property for specifying the ZooKeeper connection string. Comma-separated values of host:port pairs of the ZooKeeper servers.

See Also:

Constant Field Values

ZK_NAMESPACE

public static final String ZK_NAMESPACE

oozie-site property for specifying the ZooKeeper namespace to use (e.g. "oozie"). All of the Oozie servers that are planning on talking to each other should have the same value for this.

See Also:

Constant Field Values

ZK_CONNECTION_TIMEOUT

public static final String ZK_CONNECTION_TIMEOUT

Default ZK connection timeout ( in sec). If connection is lost for more than timeout, then Oozie server will shutdown itself.

See Also:

Constant Field Values

OOZIE_INSTANCE_ID

public static final String OOZIE_INSTANCE_ID

oozie-env environment variable for specifying the Oozie instance ID

See Also:

Constant Field Values

ZK_SECURE

public static final String ZK_SECURE

oozie-site property for specifying that ZooKeeper is secure.

See Also:

Constant Field Values

ZK_BASE_SERVICES_PATH

public static final String ZK_BASE_SERVICES_PATH

Services that need to put a node in zookeeper should go under here. Try to keep this area clean and organized.

See Also:

Constant Field Values

Method Detail

register

public static ZKUtils register(Object user)
                        throws Exception

Classes that want to use ZooKeeper should call this method to get the ZKUtils singleton.

Parameters:

user - The calling class

Returns:

the ZKUtils singleton

Throws:

Exception

unregister

public void unregister(Object user)

Classes should call this when they are done (i.e. shutdown).

Parameters:

user - The calling class

getAllMetaData

public List<org.apache.curator.x.discovery.ServiceInstance<Map>> getAllMetaData()

Returns a list of the metadata provided by all of the Oozie Servers. Note that the metadata is cached so it may be a second or two stale.

Returns:

a List of the metadata provided by all of the Oozie Servers.

getZKId

public String getZKId()

Returns the ID of this Oozie Server as seen by ZooKeeper and other Oozie Servers

Returns:

the ID of this Oozie Server

getClient

public org.apache.curator.framework.CuratorFramework getClient()

Returns the CuratorFramework used for managing the ZooKeeper connection; it can be used by calling classes to perform more direct operations on ZooKeeper. Most of the time, this shouldn't be needed.

Be careful not to close the connection.

Returns:

the CuratorFramework object

getZKIdIndex

public int getZKIdIndex(List<org.apache.curator.x.discovery.ServiceInstance<Map>> oozies)

Returns the index of this Oozie Server in ZooKeeper's list of Oozie Servers (ordered by registration time)

Parameters:

oozies - The collection of metadata provided by all of the Oozie Servers (from calling {@link ZKUtils#getAllMetaData())

Returns:

the index of this Oozie Server in ZooKeeper's list of Oozie Servers (ordered by registration time)

getUsers

public static Set<Object> getUsers()

Useful for tests to get the registered classes

Returns:

the set of registered classes

getRetryPolicy

public static org.apache.curator.RetryPolicy getRetryPolicy()

Returns retry policy

Returns:

RetryPolicy

getZKNameSpace

public static String getZKNameSpace()

Returns configured zk namesapces

Returns:

oozie.zookeeper.namespace

getZKConnectionTimeout

public static int getZKConnectionTimeout()

Return ZK connection timeout

Returns: