Search in sources :

Example 1 with CronExpression

use of org.quartz.CronExpression in project nifi by apache.

the class StandardProcessorDAO method validateProposedConfiguration.

private List<String> validateProposedConfiguration(final ProcessorNode processorNode, final ProcessorConfigDTO config) {
    List<String> validationErrors = new ArrayList<>();
    // validate settings
    if (isNotNull(config.getPenaltyDuration())) {
        Matcher penaltyMatcher = FormatUtils.TIME_DURATION_PATTERN.matcher(config.getPenaltyDuration());
        if (!penaltyMatcher.matches()) {
            validationErrors.add("Penalty duration is not a valid time duration (ie 30 sec, 5 min)");
        }
    }
    if (isNotNull(config.getYieldDuration())) {
        Matcher yieldMatcher = FormatUtils.TIME_DURATION_PATTERN.matcher(config.getYieldDuration());
        if (!yieldMatcher.matches()) {
            validationErrors.add("Yield duration is not a valid time duration (ie 30 sec, 5 min)");
        }
    }
    if (isNotNull(config.getBulletinLevel())) {
        try {
            LogLevel.valueOf(config.getBulletinLevel());
        } catch (IllegalArgumentException iae) {
            validationErrors.add(String.format("Bulletin level: Value must be one of [%s]", StringUtils.join(LogLevel.values(), ", ")));
        }
    }
    if (isNotNull(config.getExecutionNode())) {
        try {
            ExecutionNode.valueOf(config.getExecutionNode());
        } catch (IllegalArgumentException iae) {
            validationErrors.add(String.format("Execution node: Value must be one of [%s]", StringUtils.join(ExecutionNode.values(), ", ")));
        }
    }
    // get the current scheduling strategy
    SchedulingStrategy schedulingStrategy = processorNode.getSchedulingStrategy();
    // validate the new scheduling strategy if appropriate
    if (isNotNull(config.getSchedulingStrategy())) {
        try {
            // this will be the new scheduling strategy so use it
            schedulingStrategy = SchedulingStrategy.valueOf(config.getSchedulingStrategy());
        } catch (IllegalArgumentException iae) {
            validationErrors.add(String.format("Scheduling strategy: Value must be one of [%s]", StringUtils.join(SchedulingStrategy.values(), ", ")));
        }
    }
    // validate the concurrent tasks based on the scheduling strategy
    if (isNotNull(config.getConcurrentlySchedulableTaskCount())) {
        switch(schedulingStrategy) {
            case TIMER_DRIVEN:
            case PRIMARY_NODE_ONLY:
                if (config.getConcurrentlySchedulableTaskCount() <= 0) {
                    validationErrors.add("Concurrent tasks must be greater than 0.");
                }
                break;
            case EVENT_DRIVEN:
                if (config.getConcurrentlySchedulableTaskCount() < 0) {
                    validationErrors.add("Concurrent tasks must be greater or equal to 0.");
                }
                break;
        }
    }
    // validate the scheduling period based on the scheduling strategy
    if (isNotNull(config.getSchedulingPeriod())) {
        switch(schedulingStrategy) {
            case TIMER_DRIVEN:
            case PRIMARY_NODE_ONLY:
                final Matcher schedulingMatcher = FormatUtils.TIME_DURATION_PATTERN.matcher(config.getSchedulingPeriod());
                if (!schedulingMatcher.matches()) {
                    validationErrors.add("Scheduling period is not a valid time duration (ie 30 sec, 5 min)");
                }
                break;
            case CRON_DRIVEN:
                try {
                    new CronExpression(config.getSchedulingPeriod());
                } catch (final ParseException pe) {
                    throw new IllegalArgumentException(String.format("Scheduling Period '%s' is not a valid cron expression: %s", config.getSchedulingPeriod(), pe.getMessage()));
                } catch (final Exception e) {
                    throw new IllegalArgumentException("Scheduling Period is not a valid cron expression: " + config.getSchedulingPeriod());
                }
                break;
        }
    }
    final Set<String> autoTerminatedRelationships = config.getAutoTerminatedRelationships();
    if (isNotNull(autoTerminatedRelationships)) {
        for (final String relationshipName : autoTerminatedRelationships) {
            final Relationship relationship = new Relationship.Builder().name(relationshipName).build();
            final Set<Connection> connections = processorNode.getConnections(relationship);
            if (isNotNull(connections) && !connections.isEmpty()) {
                validationErrors.add("Cannot automatically terminate '" + relationshipName + "' relationship because a Connection already exists with this relationship");
            }
        }
    }
    return validationErrors;
}
Also used : Matcher(java.util.regex.Matcher) ArrayList(java.util.ArrayList) Connection(org.apache.nifi.connectable.Connection) ProcessorInstantiationException(org.apache.nifi.controller.exception.ProcessorInstantiationException) RejectedExecutionException(java.util.concurrent.RejectedExecutionException) ResourceNotFoundException(org.apache.nifi.web.ResourceNotFoundException) ParseException(java.text.ParseException) ComponentLifeCycleException(org.apache.nifi.controller.exception.ComponentLifeCycleException) NiFiCoreException(org.apache.nifi.web.NiFiCoreException) ValidationException(org.apache.nifi.controller.exception.ValidationException) SchedulingStrategy(org.apache.nifi.scheduling.SchedulingStrategy) Relationship(org.apache.nifi.processor.Relationship) CronExpression(org.quartz.CronExpression) ParseException(java.text.ParseException)

Example 2 with CronExpression

use of org.quartz.CronExpression in project Singularity by HubSpot.

the class SingularityJobPoller method getExpectedRuntime.

private Optional<Long> getExpectedRuntime(SingularityRequest request, SingularityTaskId taskId) {
    if (request.getScheduledExpectedRuntimeMillis().isPresent()) {
        return request.getScheduledExpectedRuntimeMillis();
    } else {
        final Optional<SingularityDeployStatistics> deployStatistics = deployManager.getDeployStatistics(taskId.getRequestId(), taskId.getDeployId());
        if (deployStatistics.isPresent() && deployStatistics.get().getAverageRuntimeMillis().isPresent()) {
            return deployStatistics.get().getAverageRuntimeMillis();
        }
        String scheduleExpression = request.getScheduleTypeSafe() == ScheduleType.RFC5545 ? request.getSchedule().get() : request.getQuartzScheduleSafe();
        Date nextRunAtDate;
        try {
            if (request.getScheduleTypeSafe() == ScheduleType.RFC5545) {
                final RFC5545Schedule rfc5545Schedule = new RFC5545Schedule(scheduleExpression);
                nextRunAtDate = rfc5545Schedule.getNextValidTime();
            } else {
                final CronExpression cronExpression = new CronExpression(scheduleExpression);
                final Date startDate = new Date(taskId.getStartedAt());
                nextRunAtDate = cronExpression.getNextValidTimeAfter(startDate);
            }
            if (nextRunAtDate == null) {
                String msg = String.format("No next run date found for %s (%s)", taskId, scheduleExpression);
                LOG.warn(msg);
                exceptionNotifier.notify(msg, ImmutableMap.of("taskId", taskId.toString()));
                return Optional.absent();
            }
        } catch (ParseException | InvalidRecurrenceRuleException e) {
            LOG.warn("Unable to parse schedule of type {} for expression {} (taskId: {}, err: {})", request.getScheduleTypeSafe(), scheduleExpression, taskId, e);
            exceptionNotifier.notify(String.format("Unable to parse schedule (%s)", e.getMessage()), e, ImmutableMap.of("taskId", taskId.toString(), "scheduleExpression", scheduleExpression, "scheduleType", request.getScheduleTypeSafe().toString()));
            return Optional.absent();
        }
        return Optional.of(nextRunAtDate.getTime() - taskId.getStartedAt());
    }
}
Also used : RFC5545Schedule(com.hubspot.singularity.helpers.RFC5545Schedule) CronExpression(org.quartz.CronExpression) ParseException(java.text.ParseException) SingularityDeployStatistics(com.hubspot.singularity.SingularityDeployStatistics) Date(java.util.Date) InvalidRecurrenceRuleException(org.dmfs.rfc5545.recur.InvalidRecurrenceRuleException)

Example 3 with CronExpression

use of org.quartz.CronExpression in project Singularity by HubSpot.

the class SingularityScheduler method getNextRunAt.

private Optional<Long> getNextRunAt(SingularityRequest request, RequestState state, SingularityDeployStatistics deployStatistics, SingularityPendingRequest pendingRequest, Optional<SingularityPendingDeploy> maybePendingDeploy) {
    PendingType pendingType = pendingRequest.getPendingType();
    final long now = System.currentTimeMillis();
    long nextRunAt = now;
    if (request.isScheduled()) {
        if (pendingType == PendingType.IMMEDIATE || pendingType == PendingType.RETRY) {
            LOG.info("Scheduling requested immediate run of {}", request.getId());
        } else {
            try {
                Date nextRunAtDate = null;
                Date scheduleFrom = null;
                if (request.getScheduleTypeSafe() == ScheduleType.RFC5545) {
                    final RFC5545Schedule rfc5545Schedule = new RFC5545Schedule(request.getSchedule().get());
                    nextRunAtDate = rfc5545Schedule.getNextValidTime();
                    scheduleFrom = new Date(rfc5545Schedule.getStartDateTime().getMillis());
                } else {
                    scheduleFrom = new Date(now);
                    final CronExpression cronExpression = new CronExpression(request.getQuartzScheduleSafe());
                    if (request.getScheduleTimeZone().isPresent()) {
                        cronExpression.setTimeZone(TimeZone.getTimeZone(request.getScheduleTimeZone().get()));
                    }
                    nextRunAtDate = cronExpression.getNextValidTimeAfter(scheduleFrom);
                }
                if (nextRunAtDate == null) {
                    return Optional.absent();
                }
                LOG.trace("Calculating nextRunAtDate for {} (schedule: {}): {} (from: {})", request.getId(), request.getSchedule(), nextRunAtDate, scheduleFrom);
                // don't create a schedule that is overdue as this is used to indicate that singularity is not fulfilling requests.
                nextRunAt = Math.max(nextRunAtDate.getTime(), now);
                LOG.trace("Scheduling next run of {} (schedule: {}) at {} (from: {})", request.getId(), request.getSchedule(), nextRunAtDate, scheduleFrom);
            } catch (ParseException | InvalidRecurrenceRuleException pe) {
                throw Throwables.propagate(pe);
            }
        }
    }
    if (!request.isLongRunning() && pendingRequest.getRunAt().isPresent()) {
        nextRunAt = Math.max(nextRunAt, pendingRequest.getRunAt().get());
    }
    if (pendingType == PendingType.TASK_DONE && request.getWaitAtLeastMillisAfterTaskFinishesForReschedule().or(0L) > 0) {
        nextRunAt = Math.max(nextRunAt, now + request.getWaitAtLeastMillisAfterTaskFinishesForReschedule().get());
        LOG.trace("Adjusted next run of {} to {} (by {}) due to waitAtLeastMillisAfterTaskFinishesForReschedule", request.getId(), nextRunAt, JavaUtils.durationFromMillis(request.getWaitAtLeastMillisAfterTaskFinishesForReschedule().get()));
    }
    if (state == RequestState.SYSTEM_COOLDOWN && pendingType != PendingType.NEW_DEPLOY) {
        final long prevNextRunAt = nextRunAt;
        nextRunAt = Math.max(nextRunAt, now + TimeUnit.SECONDS.toMillis(configuration.getCooldownMinScheduleSeconds()));
        LOG.trace("Adjusted next run of {} to {} (from: {}) due to cooldown", request.getId(), nextRunAt, prevNextRunAt);
    }
    return Optional.of(nextRunAt);
}
Also used : PendingType(com.hubspot.singularity.SingularityPendingRequest.PendingType) RFC5545Schedule(com.hubspot.singularity.helpers.RFC5545Schedule) CronExpression(org.quartz.CronExpression) ParseException(java.text.ParseException) Date(java.util.Date) InvalidRecurrenceRuleException(org.dmfs.rfc5545.recur.InvalidRecurrenceRuleException)

Example 4 with CronExpression

use of org.quartz.CronExpression in project jeesuite-libs by vakinge.

the class ZkJobRegistry method execCommond.

private void execCommond(MonitorCommond cmd) {
    if (cmd == null)
        return;
    JobConfig config = schedulerConfgs.get(cmd.getJobName());
    String key = cmd.getJobGroup() + ":" + cmd.getJobName();
    final AbstractJob abstractJob = JobContext.getContext().getAllJobs().get(key);
    if (MonitorCommond.TYPE_EXEC == cmd.getCmdType()) {
        if (config.isRunning()) {
            logger.info("任务正在执行中,请稍后再执行");
            return;
        }
        if (abstractJob != null) {
            JobContext.getContext().submitSyncTask(new Runnable() {

                @Override
                public void run() {
                    try {
                        logger.info("begin execute job[{}] by MonitorCommond", abstractJob.getJobName());
                        abstractJob.doJob(JobContext.getContext());
                    } catch (Exception e) {
                        logger.error(abstractJob.getJobName(), e);
                    }
                }
            });
        } else {
            logger.warn("Not found job by key:{} !!!!", key);
        }
    } else if (MonitorCommond.TYPE_STATUS_MOD == cmd.getCmdType() || MonitorCommond.TYPE_CRON_MOD == cmd.getCmdType()) {
        if (config != null) {
            if (MonitorCommond.TYPE_STATUS_MOD == cmd.getCmdType()) {
                config.setActive("1".equals(cmd.getBody()));
            } else {
                try {
                    new CronExpression(cmd.getBody().toString());
                } catch (Exception e) {
                    throw new RuntimeException("cron表达式格式错误");
                }
                abstractJob.resetTriggerCronExpr(cmd.getBody().toString());
                config.setCronExpr(cmd.getBody().toString());
            }
            updateJobConfig(config);
            if (JobContext.getContext().getConfigPersistHandler() != null) {
                JobContext.getContext().getConfigPersistHandler().persist(config);
            }
        }
    }
}
Also used : AbstractJob(com.jeesuite.scheduler.AbstractJob) CronExpression(org.quartz.CronExpression) JobConfig(com.jeesuite.scheduler.model.JobConfig)

Example 5 with CronExpression

use of org.quartz.CronExpression in project oozie by apache.

the class CoordCommandUtils method getNextValidActionTimeForCronFrequency.

/**
 * Get the next action time after a given time
 *
 * @param targetDate
 * @param coordJob
 * @return the next valid action time
 */
public static Date getNextValidActionTimeForCronFrequency(Date targetDate, CoordinatorJobBean coordJob) throws ParseException {
    String freq = coordJob.getFrequency();
    TimeZone tz = DateUtils.getOozieProcessingTimeZone();
    String[] cronArray = freq.split(" ");
    Date nextTime = null;
    // and return the earlier time
    if (!cronArray[2].trim().equals("?") && !cronArray[4].trim().equals("?")) {
        // we need to replace the wildcard with "?"
        if (cronArray[2].trim().equals("*") || cronArray[4].trim().equals("*")) {
            if (cronArray[2].trim().equals("*")) {
                cronArray[2] = "?";
            } else {
                cronArray[4] = "?";
            }
            freq = StringUtils.join(cronArray, " ");
            // The cronExpression class takes second
            // as the first field where oozie is operating on
            // minute basis
            CronExpression expr = new CronExpression("0 " + freq);
            expr.setTimeZone(tz);
            nextTime = expr.getNextValidTimeAfter(targetDate);
        } else // If both fields are specified by non-wildcards,
        // we need to split it into two expressions
        {
            String[] cronArray1 = freq.split(" ");
            String[] cronArray2 = freq.split(" ");
            cronArray1[2] = "?";
            cronArray2[4] = "?";
            String freq1 = StringUtils.join(cronArray1, " ");
            String freq2 = StringUtils.join(cronArray2, " ");
            // The cronExpression class takes second
            // as the first field where oozie is operating on
            // minute basis
            CronExpression expr1 = new CronExpression("0 " + freq1);
            expr1.setTimeZone(tz);
            CronExpression expr2 = new CronExpression("0 " + freq2);
            expr2.setTimeZone(tz);
            nextTime = expr1.getNextValidTimeAfter(targetDate);
            Date nextTime2 = expr2.getNextValidTimeAfter(targetDate);
            nextTime = nextTime.compareTo(nextTime2) < 0 ? nextTime : nextTime2;
        }
    } else {
        // The cronExpression class takes second
        // as the first field where oozie is operating on
        // minute basis
        CronExpression expr = new CronExpression("0 " + freq);
        expr.setTimeZone(tz);
        nextTime = expr.getNextValidTimeAfter(targetDate);
    }
    return nextTime;
}
Also used : TimeZone(java.util.TimeZone) CronExpression(org.quartz.CronExpression) Date(java.util.Date)

Aggregations

CronExpression (org.quartz.CronExpression)26 Date (java.util.Date)13 ParseException (java.text.ParseException)11 Test (org.junit.Test)8 ArrayList (java.util.ArrayList)6 Calendar (java.util.Calendar)5 DateTime (org.joda.time.DateTime)5 FeedOnTimeArrivalMetric (com.thinkbiganalytics.metadata.sla.api.core.FeedOnTimeArrivalMetric)4 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)3 RFC5545Schedule (com.hubspot.singularity.helpers.RFC5545Schedule)2 RejectedExecutionException (java.util.concurrent.RejectedExecutionException)2 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 Matcher (java.util.regex.Matcher)2 ComponentLifeCycleException (org.apache.nifi.controller.exception.ComponentLifeCycleException)2 ValidationException (org.apache.nifi.controller.exception.ValidationException)2 ProcessException (org.apache.nifi.processor.exception.ProcessException)2 SchedulingStrategy (org.apache.nifi.scheduling.SchedulingStrategy)2 NiFiCoreException (org.apache.nifi.web.NiFiCoreException)2 ResourceNotFoundException (org.apache.nifi.web.ResourceNotFoundException)2 InvalidRecurrenceRuleException (org.dmfs.rfc5545.recur.InvalidRecurrenceRuleException)2