Search in sources :

Example 1 with TaskContext

use of com.dangdang.ddframe.job.context.TaskContext in project elastic-job by dangdangdotcom.

the class LiteJobFacade method postJobStatusTraceEvent.

@Override
public void postJobStatusTraceEvent(final String taskId, final State state, final String message) {
    TaskContext taskContext = TaskContext.from(taskId);
    jobEventBus.post(new JobStatusTraceEvent(taskContext.getMetaInfo().getJobName(), taskContext.getId(), taskContext.getSlaveId(), Source.LITE_EXECUTOR, taskContext.getType(), taskContext.getMetaInfo().getShardingItems().toString(), state, message));
    if (!Strings.isNullOrEmpty(message)) {
        log.trace(message);
    }
}
Also used : TaskContext(com.dangdang.ddframe.job.context.TaskContext) JobStatusTraceEvent(com.dangdang.ddframe.job.event.type.JobStatusTraceEvent)

Example 2 with TaskContext

use of com.dangdang.ddframe.job.context.TaskContext in project elastic-job by dangdangdotcom.

the class SchedulerEngine method statusUpdate.

@Override
public void statusUpdate(final SchedulerDriver schedulerDriver, final Protos.TaskStatus taskStatus) {
    String taskId = taskStatus.getTaskId().getValue();
    TaskContext taskContext = TaskContext.from(taskId);
    String jobName = taskContext.getMetaInfo().getJobName();
    log.trace("call statusUpdate task state is: {}, task id is: {}", taskStatus.getState(), taskId);
    jobEventBus.post(new JobStatusTraceEvent(jobName, taskContext.getId(), taskContext.getSlaveId(), Source.CLOUD_SCHEDULER, taskContext.getType(), String.valueOf(taskContext.getMetaInfo().getShardingItems()), State.valueOf(taskStatus.getState().name()), taskStatus.getMessage()));
    switch(taskStatus.getState()) {
        case TASK_RUNNING:
            if (!facadeService.load(jobName).isPresent()) {
                schedulerDriver.killTask(Protos.TaskID.newBuilder().setValue(taskId).build());
            }
            if ("BEGIN".equals(taskStatus.getMessage())) {
                facadeService.updateDaemonStatus(taskContext, false);
            } else if ("COMPLETE".equals(taskStatus.getMessage())) {
                facadeService.updateDaemonStatus(taskContext, true);
            }
            break;
        case TASK_FINISHED:
            facadeService.removeRunning(taskContext);
            unAssignTask(taskId);
            statisticManager.taskRunSuccessfully();
            break;
        case TASK_KILLED:
            log.warn("task id is: {}, status is: {}, message is: {}, source is: {}", taskId, taskStatus.getState(), taskStatus.getMessage(), taskStatus.getSource());
            facadeService.removeRunning(taskContext);
            facadeService.addDaemonJobToReadyQueue(jobName);
            unAssignTask(taskId);
            break;
        case TASK_LOST:
        case TASK_FAILED:
        case TASK_ERROR:
            log.warn("task id is: {}, status is: {}, message is: {}, source is: {}", taskId, taskStatus.getState(), taskStatus.getMessage(), taskStatus.getSource());
            facadeService.removeRunning(taskContext);
            facadeService.recordFailoverTask(taskContext);
            unAssignTask(taskId);
            statisticManager.taskRunFailed();
            break;
        default:
            break;
    }
}
Also used : TaskContext(com.dangdang.ddframe.job.context.TaskContext) JobStatusTraceEvent(com.dangdang.ddframe.job.event.type.JobStatusTraceEvent)

Example 3 with TaskContext

use of com.dangdang.ddframe.job.context.TaskContext in project elastic-job by dangdangdotcom.

the class TaskLaunchScheduledService method getTaskInfo.

private Protos.TaskInfo getTaskInfo(final Protos.SlaveID slaveID, final TaskAssignmentResult taskAssignmentResult) {
    TaskContext taskContext = TaskContext.from(taskAssignmentResult.getTaskId());
    Optional<CloudJobConfiguration> jobConfigOptional = facadeService.load(taskContext.getMetaInfo().getJobName());
    if (!jobConfigOptional.isPresent()) {
        return null;
    }
    CloudJobConfiguration jobConfig = jobConfigOptional.get();
    Optional<CloudAppConfiguration> appConfigOptional = facadeService.loadAppConfig(jobConfig.getAppName());
    if (!appConfigOptional.isPresent()) {
        return null;
    }
    CloudAppConfiguration appConfig = appConfigOptional.get();
    taskContext.setSlaveId(slaveID.getValue());
    ShardingContexts shardingContexts = getShardingContexts(taskContext, appConfig, jobConfig);
    boolean useDefaultExecutor = CloudJobExecutionType.TRANSIENT == jobConfig.getJobExecutionType() && JobType.SCRIPT == jobConfig.getTypeConfig().getJobType();
    Protos.CommandInfo.URI uri = buildURI(appConfig, useDefaultExecutor);
    Protos.CommandInfo command = buildCommand(uri, appConfig.getBootstrapScript(), shardingContexts, useDefaultExecutor);
    return buildTaskInfo(taskContext, appConfig, jobConfig, shardingContexts, slaveID, command, useDefaultExecutor);
}
Also used : CloudAppConfiguration(com.dangdang.ddframe.job.cloud.scheduler.config.app.CloudAppConfiguration) TaskContext(com.dangdang.ddframe.job.context.TaskContext) CloudJobConfiguration(com.dangdang.ddframe.job.cloud.scheduler.config.job.CloudJobConfiguration) Protos(org.apache.mesos.Protos) ShardingContexts(com.dangdang.ddframe.job.executor.ShardingContexts)

Example 4 with TaskContext

use of com.dangdang.ddframe.job.context.TaskContext in project elastic-job by dangdangdotcom.

the class TaskLaunchScheduledService method runOneIteration.

@Override
protected void runOneIteration() throws Exception {
    try {
        LaunchingTasks launchingTasks = new LaunchingTasks(facadeService.getEligibleJobContext());
        List<VirtualMachineLease> virtualMachineLeases = LeasesQueue.getInstance().drainTo();
        Collection<VMAssignmentResult> vmAssignmentResults = taskScheduler.scheduleOnce(launchingTasks.getPendingTasks(), virtualMachineLeases).getResultMap().values();
        List<TaskContext> taskContextsList = new LinkedList<>();
        Map<List<Protos.OfferID>, List<Protos.TaskInfo>> offerIdTaskInfoMap = new HashMap<>();
        for (VMAssignmentResult each : vmAssignmentResults) {
            List<VirtualMachineLease> leasesUsed = each.getLeasesUsed();
            List<Protos.TaskInfo> taskInfoList = new ArrayList<>(each.getTasksAssigned().size() * 10);
            taskInfoList.addAll(getTaskInfoList(launchingTasks.getIntegrityViolationJobs(vmAssignmentResults), each, leasesUsed.get(0).hostname(), leasesUsed.get(0).getOffer().getSlaveId()));
            for (Protos.TaskInfo taskInfo : taskInfoList) {
                taskContextsList.add(TaskContext.from(taskInfo.getTaskId().getValue()));
            }
            offerIdTaskInfoMap.put(getOfferIDs(leasesUsed), taskInfoList);
        }
        for (TaskContext each : taskContextsList) {
            facadeService.addRunning(each);
            jobEventBus.post(createJobStatusTraceEvent(each));
        }
        facadeService.removeLaunchTasksFromQueue(taskContextsList);
        for (Entry<List<OfferID>, List<TaskInfo>> each : offerIdTaskInfoMap.entrySet()) {
            schedulerDriver.launchTasks(each.getKey(), each.getValue());
        }
    //CHECKSTYLE:OFF
    } catch (Throwable throwable) {
        //CHECKSTYLE:ON
        log.error("Launch task error", throwable);
    }
}
Also used : TaskContext(com.dangdang.ddframe.job.context.TaskContext) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) VirtualMachineLease(com.netflix.fenzo.VirtualMachineLease) TaskInfo(org.apache.mesos.Protos.TaskInfo) LinkedList(java.util.LinkedList) TaskInfo(org.apache.mesos.Protos.TaskInfo) Protos(org.apache.mesos.Protos) ArrayList(java.util.ArrayList) LinkedList(java.util.LinkedList) List(java.util.List) VMAssignmentResult(com.netflix.fenzo.VMAssignmentResult)

Example 5 with TaskContext

use of com.dangdang.ddframe.job.context.TaskContext in project elastic-job by dangdangdotcom.

the class ProducerManagerTest method assertDeregisterExisted.

@Test
public void assertDeregisterExisted() {
    when(configService.load("transient_test_job")).thenReturn(Optional.of(transientJobConfig));
    List<TaskContext> taskContexts = Arrays.asList(TaskContext.from("transient_test_job@-@0@-@READY@-@SLAVE-S0@-@UUID"), TaskContext.from("transient_test_job@-@1@-@READY@-@SLAVE-S0@-@UUID"));
    when(runningService.getRunningTasks("transient_test_job")).thenReturn(taskContexts);
    producerManager.deregister("transient_test_job");
    for (TaskContext each : taskContexts) {
        verify(schedulerDriver).killTask(Protos.TaskID.newBuilder().setValue(each.getId()).build());
    }
    verify(configService).remove("transient_test_job");
    verify(runningService).remove("transient_test_job");
    verify(readyService).remove(Lists.newArrayList("transient_test_job"));
}
Also used : TaskContext(com.dangdang.ddframe.job.context.TaskContext) Test(org.junit.Test)

Aggregations

TaskContext (com.dangdang.ddframe.job.context.TaskContext)19 Test (org.junit.Test)9 JobStatusTraceEvent (com.dangdang.ddframe.job.event.type.JobStatusTraceEvent)4 ArrayList (java.util.ArrayList)3 Protos (org.apache.mesos.Protos)3 TaskNode (com.dangdang.ddframe.job.cloud.scheduler.fixture.TaskNode)2 ByteString (com.google.protobuf.ByteString)2 TaskAssignmentResult (com.netflix.fenzo.TaskAssignmentResult)2 HashMap (java.util.HashMap)2 HashSet (java.util.HashSet)2 TaskInfo (org.apache.mesos.Protos.TaskInfo)2 CloudAppConfiguration (com.dangdang.ddframe.job.cloud.scheduler.config.app.CloudAppConfiguration)1 CloudJobConfiguration (com.dangdang.ddframe.job.cloud.scheduler.config.job.CloudJobConfiguration)1 RunningService (com.dangdang.ddframe.job.cloud.scheduler.state.running.RunningService)1 ShardingContexts (com.dangdang.ddframe.job.executor.ShardingContexts)1 JobRunningStatistics (com.dangdang.ddframe.job.statistics.type.job.JobRunningStatistics)1 TaskRunningStatistics (com.dangdang.ddframe.job.statistics.type.task.TaskRunningStatistics)1 Function (com.google.common.base.Function)1 TaskRequest (com.netflix.fenzo.TaskRequest)1 VMAssignmentResult (com.netflix.fenzo.VMAssignmentResult)1