Search in sources :

Example 11 with ScheduleEntity

use of pipelite.entity.ScheduleEntity in project pipelite by enasequence.

the class ScheduleRunner method executeSchedule.

protected void executeSchedule(ScheduleCron scheduleCron, ProcessEntity processEntity, ExecuteMode executeMode) {
    internalErrorHandler.execute(() -> {
        String pipelineName = processEntity.getPipelineName();
        String processId = processEntity.getProcessId();
        logContext(log.atInfo(), pipelineName, processId).log("Executing scheduled process");
        Schedule schedule = getSchedule(pipelineName);
        if (schedule == null) {
            return;
        }
        Process process = getProcess(processEntity, schedule);
        if (process == null) {
            return;
        }
        setMaximumRetries(process);
        if (executeMode != ExecuteMode.RESUME) {
            scheduleService.startExecution(pipelineName, processId);
        }
        scheduleCron.setLaunchTime(null);
        runProcess(pipelineName, process, (p) -> {
            ScheduleEntity scheduleEntity = scheduleService.getSavedSchedule(pipelineName).get();
            ZonedDateTime nextLaunchTime = CronUtils.launchTime(scheduleCron.getCron(), scheduleEntity.getStartTime());
            internalErrorHandler.execute(() -> scheduleService.endExecution(processEntity, nextLaunchTime));
            scheduleCron.setLaunchTime(nextLaunchTime);
            decreaseMaximumExecutions(pipelineName);
        });
    });
}
Also used : ZonedDateTime(java.time.ZonedDateTime) Schedule(pipelite.Schedule) Process(pipelite.process.Process) ScheduleEntity(pipelite.entity.ScheduleEntity)

Example 12 with ScheduleEntity

use of pipelite.entity.ScheduleEntity in project pipelite by enasequence.

the class RegisteredScheduleService method saveSchedules.

@PostConstruct
public void saveSchedules() {
    registeredPipelineService.getRegisteredPipelines(Schedule.class).forEach(schedule -> {
        String pipelineName = schedule.pipelineName();
        Optional<ScheduleEntity> savedScheduleEntityOpt = scheduleService.getSavedSchedule(pipelineName);
        if (!savedScheduleEntityOpt.isPresent()) {
            createSchedule(schedule);
        } else {
            ScheduleEntity savedScheduleEntity = savedScheduleEntityOpt.get();
            String registeredCron = savedScheduleEntity.getCron();
            String registeredServiceName = savedScheduleEntity.getServiceName();
            String cron = schedule.configurePipeline().cron();
            boolean isCronChanged = !registeredCron.equals(cron);
            boolean isServiceNameChanged = !registeredServiceName.equals(serviceName);
            if (isCronChanged) {
                log.atInfo().log("Cron changed for pipeline schedule: " + schedule.pipelineName());
            }
            if (isServiceNameChanged) {
                log.atInfo().log("Service name changed for pipeline schedule: " + schedule.pipelineName());
            }
            if (isServiceNameChanged && !pipeliteConfiguration.service().isForce()) {
                throw new PipeliteException("Forceful startup not requested. Service name changed for pipeline schedule " + pipelineName + " from " + registeredServiceName + " to " + serviceName);
            }
            if (isServiceNameChanged) {
                log.atWarning().log("Forceful startup requested. Changing service name for pipeline schedule " + pipelineName + " from " + registeredServiceName + " to " + serviceName);
            }
            if (isCronChanged || isServiceNameChanged) {
                log.atInfo().log("Updating pipeline schedule: " + schedule.pipelineName());
                try {
                    savedScheduleEntity.setCron(cron);
                    savedScheduleEntity.setDescription(CronUtils.describe(cron));
                    savedScheduleEntity.setServiceName(serviceName);
                    if (!savedScheduleEntity.isActive()) {
                        savedScheduleEntity.setNextTime(CronUtils.launchTime(savedScheduleEntity.getCron(), savedScheduleEntity.getStartTime()));
                    }
                    scheduleService.saveSchedule(savedScheduleEntity);
                } catch (Exception ex) {
                    throw new PipeliteException("Failed to update pipeline schedule: " + schedule.pipelineName(), ex);
                }
            }
        }
    });
}
Also used : Schedule(pipelite.Schedule) PipeliteException(pipelite.exception.PipeliteException) ScheduleEntity(pipelite.entity.ScheduleEntity) PipeliteException(pipelite.exception.PipeliteException) PostConstruct(javax.annotation.PostConstruct)

Example 13 with ScheduleEntity

use of pipelite.entity.ScheduleEntity in project pipelite by enasequence.

the class ScheduleService method startExecution.

/**
 * Called when the schedule execution starts. Sets the execution start time and process id.
 * Removes the execution end time and next execution time. Saves the schedule.
 *
 * @param pipelineName the pipeline name
 * @param processId the process id
 */
@Timed("pipelite.transactional")
public ScheduleEntity startExecution(String pipelineName, String processId) {
    log.atInfo().log("Starting scheduled process execution: " + pipelineName);
    ScheduleEntity scheduleEntity = getSavedSchedule(pipelineName).get();
    scheduleEntity.setStartTime(ZonedDateTime.now().truncatedTo(ChronoUnit.SECONDS));
    scheduleEntity.setProcessId(processId);
    scheduleEntity.setEndTime(null);
    scheduleEntity.setNextTime(null);
    return saveSchedule(scheduleEntity);
}
Also used : ScheduleEntity(pipelite.entity.ScheduleEntity) Timed(io.micrometer.core.annotation.Timed)

Example 14 with ScheduleEntity

use of pipelite.entity.ScheduleEntity in project pipelite by enasequence.

the class ScheduleService method endExecution.

/**
 * Called when the schedule execution ends. Sets the execution end, last completed and last failed
 * times. Increases the execution count and sets the completed and failed streak.
 *
 * @param processEntity the process entity
 * @param nextTime the next execution time
 */
@Timed("pipelite.transactional")
public ScheduleEntity endExecution(ProcessEntity processEntity, ZonedDateTime nextTime) {
    String pipelineName = processEntity.getPipelineName();
    log.atInfo().log("Ending scheduled process execution: " + pipelineName);
    ZonedDateTime now = ZonedDateTime.now().truncatedTo(ChronoUnit.SECONDS);
    ScheduleEntity scheduleEntity = getSavedSchedule(pipelineName).get();
    scheduleEntity.setEndTime(now);
    scheduleEntity.setNextTime(nextTime);
    scheduleEntity.setExecutionCount(scheduleEntity.getExecutionCount() + 1);
    if (processEntity.getProcessState() == ProcessState.COMPLETED) {
        scheduleEntity.setLastCompleted(now);
        scheduleEntity.setStreakCompleted(scheduleEntity.getStreakCompleted() + 1);
        scheduleEntity.setStreakFailed(0);
    } else {
        scheduleEntity.setLastFailed(now);
        scheduleEntity.setStreakCompleted(0);
        scheduleEntity.setStreakFailed(scheduleEntity.getStreakFailed() + 1);
    }
    return saveSchedule(scheduleEntity);
}
Also used : ZonedDateTime(java.time.ZonedDateTime) ScheduleEntity(pipelite.entity.ScheduleEntity) Timed(io.micrometer.core.annotation.Timed)

Example 15 with ScheduleEntity

use of pipelite.entity.ScheduleEntity in project pipelite by enasequence.

the class ScheduleRunner method createProcessEntity.

private ProcessEntity createProcessEntity(ScheduleCron scheduleCron) {
    String pipelineName = scheduleCron.getPipelineName();
    ScheduleEntity scheduleEntity = scheduleService.getSavedSchedule(pipelineName).get();
    String lastProcessId = scheduleEntity.getProcessId();
    String nextProcessId = nextProcessId(lastProcessId);
    Optional<ProcessEntity> processEntity = processService.getSavedProcess(pipelineName, nextProcessId);
    if (processEntity.isPresent()) {
        throw new PipeliteException("Scheduled new process already exists: " + pipelineName + " " + nextProcessId);
    }
    return processService.createExecution(pipelineName, nextProcessId, ProcessEntity.DEFAULT_PRIORITY);
}
Also used : ProcessEntity(pipelite.entity.ProcessEntity) PipeliteException(pipelite.exception.PipeliteException) ScheduleEntity(pipelite.entity.ScheduleEntity)

Aggregations

ScheduleEntity (pipelite.entity.ScheduleEntity)22 ProcessEntity (pipelite.entity.ProcessEntity)9 Test (org.junit.jupiter.api.Test)8 SpringBootTest (org.springframework.boot.test.context.SpringBootTest)8 ZonedDateTime (java.time.ZonedDateTime)5 Timed (io.micrometer.core.annotation.Timed)4 PipeliteProcessRetryException (pipelite.exception.PipeliteProcessRetryException)4 Schedule (pipelite.Schedule)2 StageEntity (pipelite.entity.StageEntity)2 PipeliteException (pipelite.exception.PipeliteException)2 Process (pipelite.process.Process)2 Duration (java.time.Duration)1 List (java.util.List)1 AtomicLong (java.util.concurrent.atomic.AtomicLong)1 PostConstruct (javax.annotation.PostConstruct)1 Getter (lombok.Getter)1 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)1 AfterEach (org.junit.jupiter.api.AfterEach)1 Autowired (org.springframework.beans.factory.annotation.Autowired)1 TestConfiguration (org.springframework.boot.test.context.TestConfiguration)1