This project has retired. For details please refer to its Attic page.

org.apache.oozie.coord
Class CoordELFunctions

java.lang.Object
  org.apache.oozie.coord.CoordELFunctions

public class CoordELFunctions
extends Object

This class implements the EL function related to coordinator

Field Summary

static String	CONFIGURATION
static String	COORD_ACTION
static String	DATASET
static String	DIR_SEPARATOR
static String	INSTANCE_SEPARATOR
static String	LATEST_EL_USE_CURRENT_TIME

Constructor Summary

CoordELFunctions()

Method Summary

static void	configureEvaluator(ELEvaluator evaluator, SyncCoordDataset ds, SyncCoordAction coordAction) Configure an evaluator with data set and application specific information.
static String	coord_actualTime_echo()
static String	coord_conf(String property) Return a job configuration property for the coordinator.
static String	coord_days_echo(String n)
static String	coord_endOfDay_echo(String n)
static String	coord_endOfMonth_echo(String n)
static String	coord_hours_echo(String n)
static String	coord_minutes_echo(String n)
static String	coord_months_echo(String n)
static String	coord_user() Return the user that submitted the coordinator job.
static String	evalAndWrap(ELEvaluator eval, String expr) Helper method to wrap around with "${..}".
static List<Integer>	expandOffsetTimes(Calendar startCal, Calendar endCal, ELEvaluator eval) Takes two offset times and returns a list of multiples of the frequency offset from the effective nominal time that occur between them.
static Calendar	getCurrentInstance(Date effectiveTime, int[] instanceCount) Find the current instance based on effectiveTime (i.e Action_Creation_Time or Action_Start_Time)
static TimeZone	getDatasetTZ()
static Calendar	getEffectiveNominalTime()
static String	ph1_coord_actionId_echo()
static String	ph1_coord_actualTime_echo_wrap()
static String	ph1_coord_current_echo(String n)
static String	ph1_coord_currentRange_echo(String start, String end)
static String	ph1_coord_dataIn_echo(String n)
static String	ph1_coord_dataOut_echo(String n)
static String	ph1_coord_dateOffset_echo(String n, String offset, String unit)
static int	ph1_coord_days(int val) Used in defining the frequency in 'day' unit.
static String	ph1_coord_daysInMonth_echo(String n)
static int	ph1_coord_endOfDays(int val) Used in defining the frequency in 'day' unit and specify the "end of day" property.
static int	ph1_coord_endOfMonths(int val) Used in defining the frequency in 'month' unit and specify the "end of month" property.
static String	ph1_coord_formatTime_echo(String dateTime, String format)
static String	ph1_coord_future_echo(String n, String instance)
static String	ph1_coord_futureRange_echo(String start, String end, String instance)
static int	ph1_coord_hours(int val) Used in defining the frequency in 'hour' unit.
static String	ph1_coord_hoursInDay_echo(String n)
static String	ph1_coord_latest_echo(String n)
static String	ph1_coord_latestRange_echo(String start, String end)
static int	ph1_coord_minutes(int val) Used in defining the frequency in 'minute' unit.
static int	ph1_coord_months(int val) Used in defining the frequency in 'month' unit.
static String	ph1_coord_name_echo()
static String	ph1_coord_nominalTime_echo_fixed()
static String	ph1_coord_nominalTime_echo_wrap()
static String	ph1_coord_nominalTime_echo()
static String	ph1_coord_offset_echo(String n, String timeUnit)
static String	ph1_coord_tzOffset_echo()
static String	ph2_coord_actionId() Return Action Id.
static String	ph2_coord_actualTime() Return Action Start time.
static String	ph2_coord_current_echo(String n)
static String	ph2_coord_current(int n) Determine the date-time in Oozie processing timezone of n-th dataset instance.
static String	ph2_coord_currentRange_echo(String start, String end)
static String	ph2_coord_currentRange(int start, int end) Determine the date-time in Oozie processing timezone of current dataset instances from start to end offsets from the nominal time.
static String	ph2_coord_dateOffset(String strBaseDate, int offset, String unit) Returns the a date string while given a base date in 'strBaseDate', offset and unit (e.g.
static int	ph2_coord_daysInMonth(int n) Calculate number of days in one month for n-th dataset instance.
static String	ph2_coord_formatTime(String dateTimeStr, String format) Convert from standard date-time formatting to a desired format.
static String	ph2_coord_future_echo(String n, String instance)
static String	ph2_coord_futureRange_echo(String start, String end, String instance)
static int	ph2_coord_hoursInDay(int n) Determine how many hours is on the date of n-th dataset instance.
static String	ph2_coord_latest_echo(String n)
static String	ph2_coord_latestRange_echo(String start, String end)
static String	ph2_coord_name() Return Job Name.
static String	ph2_coord_nominalTime() Return nominal time or Action Creation Time.
static String	ph2_coord_offset_echo(String n, String timeUnit)
static String	ph2_coord_offset(int n, String timeUnit) Determine the date-time in Oozie processing timezone of the given offset from the dataset effective nominal time.
static int	ph2_coord_tzOffset() Calculate the difference of timezone offset in minutes between dataset and coordinator job.
static String	ph3_coord_actionId()
static String	ph3_coord_actualTime()
static String	ph3_coord_dataIn(String dataInName) Used to specify a list of URI's that are used as input dir to the workflow job.
static String	ph3_coord_dataOut(String dataOutName) Used to specify a list of URI's that are output dir of the workflow job.
static String	ph3_coord_dateOffset(String strBaseDate, int offset, String unit)
static int	ph3_coord_daysInMonth(int n)
static String	ph3_coord_formatTime(String dateTimeStr, String format)
static String	ph3_coord_future(int n, int instance) Determine the date-time in Oozie processing timezone of n-th future available dataset instance from nominal Time but not beyond the instance specified as 'instance.
static String	ph3_coord_futureRange(int start, int end, int instance) Determine the date-time in Oozie processing timezone of the future available dataset instances from start to end offsets from nominal Time but not beyond the instance specified as 'instance'.
static int	ph3_coord_hoursInDay(int n)
static String	ph3_coord_latest(int n) Determine the date-time in Oozie processing timezone of n-th latest available dataset instance.
static String	ph3_coord_latestRange(int start, int end) Determine the date-time in Oozie processing timezone of latest available dataset instances from start to end offsets from the nominal time.
static String	ph3_coord_name()
static String	ph3_coord_nominalTime()
static int	ph3_coord_tzOffset()
static Calendar	resolveOffsetRawTime(int n, TimeUnit timeUnit, ELEvaluator eval) Resolve the offset time from the effective nominal time

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DATASET

public static final String DATASET

See Also:

Constant Field Values

COORD_ACTION

public static final String COORD_ACTION

See Also:

Constant Field Values

CONFIGURATION

public static final String CONFIGURATION

See Also:

Constant Field Values

LATEST_EL_USE_CURRENT_TIME

public static final String LATEST_EL_USE_CURRENT_TIME

See Also:

Constant Field Values

INSTANCE_SEPARATOR

public static final String INSTANCE_SEPARATOR

See Also:

Constant Field Values

DIR_SEPARATOR

public static final String DIR_SEPARATOR

See Also:

Constant Field Values

Constructor Detail

CoordELFunctions

public CoordELFunctions()

Method Detail

ph1_coord_days

public static int ph1_coord_days(int val)

Used in defining the frequency in 'day' unit.

domain: val > 0 and should be integer.

Parameters:

val - frequency in number of days.

Returns:

number of days and also set the frequency timeunit to "day"

ph1_coord_months

public static int ph1_coord_months(int val)

Used in defining the frequency in 'month' unit.

domain: val > 0 and should be integer.

Parameters:

val - frequency in number of months.

Returns:

number of months and also set the frequency timeunit to "month"

ph1_coord_hours

public static int ph1_coord_hours(int val)

Used in defining the frequency in 'hour' unit.

parameter value domain: val > 0 and should be integer.

Parameters:

val - frequency in number of hours.

Returns:

number of minutes and also set the frequency timeunit to "minute"

ph1_coord_minutes

public static int ph1_coord_minutes(int val)

Used in defining the frequency in 'minute' unit.

domain: val > 0 and should be integer.

Parameters:

val - frequency in number of minutes.

Returns:

number of minutes and also set the frequency timeunit to "minute"

ph1_coord_endOfDays

public static int ph1_coord_endOfDays(int val)

Used in defining the frequency in 'day' unit and specify the "end of day" property.

Every instance will start at 00:00 hour of each day.

domain: val > 0 and should be integer.

Parameters:

val - frequency in number of days.

Returns:

number of days and also set the frequency timeunit to "day" and end_of_duration flag to "day"

ph1_coord_endOfMonths

public static int ph1_coord_endOfMonths(int val)

Used in defining the frequency in 'month' unit and specify the "end of month" property.

Every instance will start at first day of each month at 00:00 hour.

domain: val > 0 and should be integer.

Parameters:

val: - frequency in number of months.

Returns:

number of months and also set the frequency timeunit to "month" and end_of_duration flag to "month"

ph2_coord_tzOffset

public static int ph2_coord_tzOffset()

Calculate the difference of timezone offset in minutes between dataset and coordinator job.

Depends on:

1. Timezone of both dataset and job

2. Action creation Time

Returns:

difference in minutes (DataSet TZ Offset - Application TZ offset)

ph3_coord_tzOffset

public static int ph3_coord_tzOffset()

ph2_coord_dateOffset

public static String ph2_coord_dateOffset(String strBaseDate,
                                          int offset,
                                          String unit)
                                   throws Exception

Returns the a date string while given a base date in 'strBaseDate', offset and unit (e.g. DAY, MONTH, HOUR, MINUTE, MONTH).

Parameters:

strBaseDate - -- base date

offset - -- any number

unit - -- DAY, MONTH, HOUR, MINUTE, MONTH

Returns:

date string

Throws:

Exception

ph3_coord_dateOffset

public static String ph3_coord_dateOffset(String strBaseDate,
                                          int offset,
                                          String unit)
                                   throws Exception

Throws:

Exception

ph3_coord_future

public static String ph3_coord_future(int n,
                                      int instance)
                               throws Exception

Determine the date-time in Oozie processing timezone of n-th future available dataset instance from nominal Time but not beyond the instance specified as 'instance.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

7. Existence of dataset's directory

Parameters:

n - :instance count

domain: n >= 0, n is integer

instance: - How many future instance it should check? value should be >=0

Returns:

date-time in Oozie processing timezone of the n-th instance

Throws:

Exception

ph3_coord_futureRange

public static String ph3_coord_futureRange(int start,
                                           int end,
                                           int instance)
                                    throws Exception

Determine the date-time in Oozie processing timezone of the future available dataset instances from start to end offsets from nominal Time but not beyond the instance specified as 'instance'.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

7. Existence of dataset's directory

Parameters:

start - : start instance offset

domain: start >= 0, start is integer

end - : end instance offset

domain: end >= 0, end is integer

instance: - How many future instance it should check? value should be >=0

Returns:

date-time in Oozie processing timezone of the instances from start to end offsets delimited by comma.

Throws:

Exception

ph2_coord_nominalTime

public static String ph2_coord_nominalTime()
                                    throws Exception

Return nominal time or Action Creation Time.

Returns:

coordinator action creation or materialization date time

Throws:

Exception - if unable to format the Date object to String

ph3_coord_nominalTime

public static String ph3_coord_nominalTime()
                                    throws Exception

Throws:

Exception

ph2_coord_formatTime

public static String ph2_coord_formatTime(String dateTimeStr,
                                          String format)
                                   throws Exception

Convert from standard date-time formatting to a desired format.

Parameters:

dateTimeStr - - A timestamp in standard (ISO8601) format.

format - - A string representing the desired format.

Returns:

coordinator action creation or materialization date time

Throws:

Exception - if unable to format the Date object to String

ph3_coord_formatTime

public static String ph3_coord_formatTime(String dateTimeStr,
                                          String format)
                                   throws Exception

Throws:

Exception

ph2_coord_actionId

public static String ph2_coord_actionId()
                                 throws Exception

Return Action Id.

Returns:

coordinator action Id

Throws:

Exception

ph3_coord_actionId

public static String ph3_coord_actionId()
                                 throws Exception

Throws:

Exception

ph2_coord_name

public static String ph2_coord_name()
                             throws Exception

Return Job Name.

Returns:

coordinator name

Throws:

Exception

ph3_coord_name

public static String ph3_coord_name()
                             throws Exception

Throws:

Exception

ph2_coord_actualTime

public static String ph2_coord_actualTime()
                                   throws Exception

Return Action Start time.

Returns:

coordinator action start time

Throws:

Exception - if unable to format the Date object to String

ph3_coord_actualTime

public static String ph3_coord_actualTime()
                                   throws Exception

Throws:

Exception

ph3_coord_dataIn

public static String ph3_coord_dataIn(String dataInName)

Used to specify a list of URI's that are used as input dir to the workflow job.

Look for two evaluator-level variables

A) .datain. B) .datain..unresolved

A defines the current list of URI.

B defines whether there are any unresolved EL-function (i.e latest)

If there are something unresolved, this function will echo back the original function

otherwise it sends the uris.

Parameters:

dataInName - : Datain name

Returns:

the list of URI's separated by INSTANCE_SEPARATOR

if there are unresolved EL function (i.e. latest) , echo back

the function without resolving the function.

ph3_coord_dataOut

public static String ph3_coord_dataOut(String dataOutName)

Used to specify a list of URI's that are output dir of the workflow job.

Look for one evaluator-level variable

dataout.

It defines the current list of URI.

otherwise it sends the uris.

Parameters:

dataOutName - : Dataout name

Returns:

the list of URI's separated by INSTANCE_SEPARATOR

ph2_coord_current

public static String ph2_coord_current(int n)
                                throws Exception

Determine the date-time in Oozie processing timezone of n-th dataset instance.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

Parameters:

n - instance count domain: n is integer

Returns:

date-time in Oozie processing timezone of the n-th instance returns 'null' means n-th instance is earlier than Initial-Instance of DS

Throws:

Exception

ph2_coord_currentRange

public static String ph2_coord_currentRange(int start,
                                            int end)
                                     throws Exception

Determine the date-time in Oozie processing timezone of current dataset instances from start to end offsets from the nominal time.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

Parameters:

start - :start instance offset

domain: start <= 0, start is integer

end - :end instance offset

domain: end <= 0, end is integer

Returns:

date-time in Oozie processing timezone of the instances from start to end offsets delimited by comma.

If the current instance time of the dataset based on the Action Creation Time is earlier than the Initial-Instance of DS an empty string is returned. If an instance within the range is earlier than Initial-Instance of DS that instance is ignored

Throws:

Exception

ph2_coord_offset

public static String ph2_coord_offset(int n,
                                      String timeUnit)
                               throws Exception

Determine the date-time in Oozie processing timezone of the given offset from the dataset effective nominal time.

It depends on:

1. Data set frequency

2. Data set Time Unit

3. Data set Time zone/DST

4. Data set initial instance

5. Action Creation Time

Parameters:

n - offset amount (integer)

timeUnit - TimeUnit for offset n ("MINUTE", "HOUR", "DAY", "MONTH", "YEAR")

Returns:

date-time in Oozie processing timezone of the given offset from the dataset effective nominal time

Throws:

Exception - if there was a problem formatting

ph2_coord_hoursInDay

public static int ph2_coord_hoursInDay(int n)
                                throws Exception

Determine how many hours is on the date of n-th dataset instance.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

Parameters:

n - instance count

domain: n is integer

Returns:

number of hours on that day

returns -1 means n-th instance is earlier than Initial-Instance of DS

Throws:

Exception

ph3_coord_hoursInDay

public static int ph3_coord_hoursInDay(int n)
                                throws Exception

Throws:

Exception

ph2_coord_daysInMonth

public static int ph2_coord_daysInMonth(int n)
                                 throws Exception

Calculate number of days in one month for n-th dataset instance.

It depends on:

1. Data set frequency .

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

Parameters:

n - instance count. domain: n is integer

Returns:

number of days in that month

returns -1 means n-th instance is earlier than Initial-Instance of DS

Throws:

Exception

ph3_coord_daysInMonth

public static int ph3_coord_daysInMonth(int n)
                                 throws Exception

Throws:

Exception

ph3_coord_latest

public static String ph3_coord_latest(int n)
                               throws Exception

Determine the date-time in Oozie processing timezone of n-th latest available dataset instance.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

7. Existence of dataset's directory

Parameters:

n - :instance count

domain: n <= 0, n is integer

Returns:

date-time in Oozie processing timezone of the n-th instance

returns 'null' means n-th instance is earlier than Initial-Instance of DS

Throws:

Exception

ph3_coord_latestRange

public static String ph3_coord_latestRange(int start,
                                           int end)
                                    throws Exception

Determine the date-time in Oozie processing timezone of latest available dataset instances from start to end offsets from the nominal time.

It depends on:

1. Data set frequency

2. Data set Time unit (day, month, minute)

3. Data set Time zone/DST

4. End Day/Month flag

5. Data set initial instance

6. Action Creation Time

7. Existence of dataset's directory

Parameters:

start - :start instance offset

domain: start <= 0, start is integer

end - :end instance offset

domain: end <= 0, end is integer

Returns:

date-time in Oozie processing timezone of the instances from start to end offsets delimited by comma.

returns 'null' means start offset instance is earlier than Initial-Instance of DS

Throws:

Exception

configureEvaluator

public static void configureEvaluator(ELEvaluator evaluator,
                                      SyncCoordDataset ds,
                                      SyncCoordAction coordAction)

Configure an evaluator with data set and application specific information.

Helper method of associating dataset and application object

Parameters:

evaluator - : to set variables

ds - : Data Set object

coordAction - : Application instance

evalAndWrap

public static String evalAndWrap(ELEvaluator eval,
                                 String expr)
                          throws Exception

Helper method to wrap around with "${..}".

Parameters:

eval - :EL evaluator

expr - : expression to evaluate

Returns:

Resolved expression or echo back the same expression

Throws:

Exception

ph1_coord_current_echo

public static String ph1_coord_current_echo(String n)

ph1_coord_currentRange_echo

public static String ph1_coord_currentRange_echo(String start,
                                                 String end)

ph1_coord_offset_echo

public static String ph1_coord_offset_echo(String n,
                                           String timeUnit)

ph2_coord_current_echo

public static String ph2_coord_current_echo(String n)

ph2_coord_currentRange_echo

public static String ph2_coord_currentRange_echo(String start,
                                                 String end)

ph2_coord_offset_echo

public static String ph2_coord_offset_echo(String n,
                                           String timeUnit)

ph1_coord_dateOffset_echo

public static String ph1_coord_dateOffset_echo(String n,
                                               String offset,
                                               String unit)

ph1_coord_formatTime_echo

public static String ph1_coord_formatTime_echo(String dateTime,
                                               String format)

ph1_coord_latest_echo

public static String ph1_coord_latest_echo(String n)

ph2_coord_latest_echo

public static String ph2_coord_latest_echo(String n)

ph1_coord_future_echo

public static String ph1_coord_future_echo(String n,
                                           String instance)

ph2_coord_future_echo

public static String ph2_coord_future_echo(String n,
                                           String instance)

ph1_coord_latestRange_echo

public static String ph1_coord_latestRange_echo(String start,
                                                String end)

ph2_coord_latestRange_echo

public static String ph2_coord_latestRange_echo(String start,
                                                String end)

ph1_coord_futureRange_echo

public static String ph1_coord_futureRange_echo(String start,
                                                String end,
                                                String instance)

ph2_coord_futureRange_echo

public static String ph2_coord_futureRange_echo(String start,
                                                String end,
                                                String instance)

ph1_coord_dataIn_echo

public static String ph1_coord_dataIn_echo(String n)

ph1_coord_dataOut_echo

public static String ph1_coord_dataOut_echo(String n)

ph1_coord_nominalTime_echo

public static String ph1_coord_nominalTime_echo()

ph1_coord_nominalTime_echo_wrap

public static String ph1_coord_nominalTime_echo_wrap()

ph1_coord_nominalTime_echo_fixed

public static String ph1_coord_nominalTime_echo_fixed()

ph1_coord_actualTime_echo_wrap

public static String ph1_coord_actualTime_echo_wrap()

ph1_coord_actionId_echo

public static String ph1_coord_actionId_echo()

ph1_coord_name_echo

public static String ph1_coord_name_echo()

coord_minutes_echo

public static String coord_minutes_echo(String n)

coord_hours_echo

public static String coord_hours_echo(String n)

coord_days_echo

public static String coord_days_echo(String n)

coord_endOfDay_echo

public static String coord_endOfDay_echo(String n)

coord_months_echo

public static String coord_months_echo(String n)

coord_endOfMonth_echo

public static String coord_endOfMonth_echo(String n)

coord_actualTime_echo

public static String coord_actualTime_echo()

ph1_coord_hoursInDay_echo

public static String ph1_coord_hoursInDay_echo(String n)

ph1_coord_daysInMonth_echo

public static String ph1_coord_daysInMonth_echo(String n)

ph1_coord_tzOffset_echo

public static String ph1_coord_tzOffset_echo()

getCurrentInstance

public static Calendar getCurrentInstance(Date effectiveTime,
                                          int[] instanceCount)

Find the current instance based on effectiveTime (i.e Action_Creation_Time or Action_Start_Time)

Returns:

current instance i.e. current(0) returns null if effectiveTime is earlier than Initial Instance time of the dataset.

getEffectiveNominalTime

public static Calendar getEffectiveNominalTime()

getDatasetTZ

public static TimeZone getDatasetTZ()

Returns:

dataset TimeZone

coord_conf

public static String coord_conf(String property)

Return a job configuration property for the coordinator.

Parameters:

property - property name.

Returns:

the value of the property, null if the property is undefined.

coord_user

public static String coord_user()

Return the user that submitted the coordinator job.

Returns:

the user that submitted the coordinator job.

expandOffsetTimes

public static List<Integer> expandOffsetTimes(Calendar startCal,
                                              Calendar endCal,
                                              ELEvaluator eval)

Takes two offset times and returns a list of multiples of the frequency offset from the effective nominal time that occur between them. The caller should make sure that startCal is earlier than endCal.

As a simple example, assume its the same day: startCal is 1:00, endCal is 2:00, frequency is 20min, and effective nominal time is 1:20 -- then this method would return a list containing: -20, 0, 20, 40, 60

Parameters:

startCal - The earlier offset time

endCal - The later offset time

eval - The ELEvaluator to use; cannot be null

Returns:

A list of multiple of the frequency offset from the effective nominal time that occur between the startCal and endCal

resolveOffsetRawTime

public static Calendar resolveOffsetRawTime(int n,
                                            TimeUnit timeUnit,
                                            ELEvaluator eval)

Resolve the offset time from the effective nominal time

Parameters:

n - offset amount (integer)

timeUnit - TimeUnit for offset n ("MINUTE", "HOUR", "DAY", "MONTH", "YEAR")

eval - The ELEvaluator to use; or null to use the "current" eval

Returns:

A Calendar of the offset time