Search in sources :

Example 11 with ShardingContexts

use of com.dangdang.ddframe.job.executor.ShardingContexts in project elastic-job by dangdangdotcom.

the class TaskLaunchScheduledService method getTaskInfo.

private Protos.TaskInfo getTaskInfo(final Protos.SlaveID slaveID, final TaskAssignmentResult taskAssignmentResult) {
    TaskContext taskContext = TaskContext.from(taskAssignmentResult.getTaskId());
    Optional<CloudJobConfiguration> jobConfigOptional = facadeService.load(taskContext.getMetaInfo().getJobName());
    if (!jobConfigOptional.isPresent()) {
        return null;
    }
    CloudJobConfiguration jobConfig = jobConfigOptional.get();
    Optional<CloudAppConfiguration> appConfigOptional = facadeService.loadAppConfig(jobConfig.getAppName());
    if (!appConfigOptional.isPresent()) {
        return null;
    }
    CloudAppConfiguration appConfig = appConfigOptional.get();
    taskContext.setSlaveId(slaveID.getValue());
    ShardingContexts shardingContexts = getShardingContexts(taskContext, appConfig, jobConfig);
    boolean useDefaultExecutor = CloudJobExecutionType.TRANSIENT == jobConfig.getJobExecutionType() && JobType.SCRIPT == jobConfig.getTypeConfig().getJobType();
    Protos.CommandInfo.URI uri = buildURI(appConfig, useDefaultExecutor);
    Protos.CommandInfo command = buildCommand(uri, appConfig.getBootstrapScript(), shardingContexts, useDefaultExecutor);
    return buildTaskInfo(taskContext, appConfig, jobConfig, shardingContexts, slaveID, command, useDefaultExecutor);
}
Also used : CloudAppConfiguration(com.dangdang.ddframe.job.cloud.scheduler.config.app.CloudAppConfiguration) TaskContext(com.dangdang.ddframe.job.context.TaskContext) CloudJobConfiguration(com.dangdang.ddframe.job.cloud.scheduler.config.job.CloudJobConfiguration) Protos(org.apache.mesos.Protos) ShardingContexts(com.dangdang.ddframe.job.executor.ShardingContexts)

Example 12 with ShardingContexts

use of com.dangdang.ddframe.job.executor.ShardingContexts in project elastic-job by dangdangdotcom.

the class TaskLaunchScheduledService method getShardingContexts.

private ShardingContexts getShardingContexts(final TaskContext taskContext, final CloudAppConfiguration appConfig, final CloudJobConfiguration jobConfig) {
    Map<Integer, String> shardingItemParameters = new ShardingItemParameters(jobConfig.getTypeConfig().getCoreConfig().getShardingItemParameters()).getMap();
    Map<Integer, String> assignedShardingItemParameters = new HashMap<>(1, 1);
    int shardingItem = taskContext.getMetaInfo().getShardingItems().get(0);
    assignedShardingItemParameters.put(shardingItem, shardingItemParameters.containsKey(shardingItem) ? shardingItemParameters.get(shardingItem) : "");
    return new ShardingContexts(taskContext.getId(), jobConfig.getJobName(), jobConfig.getTypeConfig().getCoreConfig().getShardingTotalCount(), jobConfig.getTypeConfig().getCoreConfig().getJobParameter(), assignedShardingItemParameters, appConfig.getEventTraceSamplingCount());
}
Also used : HashMap(java.util.HashMap) ShardingItemParameters(com.dangdang.ddframe.job.util.config.ShardingItemParameters) ByteString(com.google.protobuf.ByteString) ShardingContexts(com.dangdang.ddframe.job.executor.ShardingContexts)

Example 13 with ShardingContexts

use of com.dangdang.ddframe.job.executor.ShardingContexts in project elastic-job by dangdangdotcom.

the class TaskExecutorThreadTest method serialize.

private byte[] serialize(final Map<String, String> jobConfigurationContext) {
    // CHECKSTYLE:OFF
    LinkedHashMap<String, Object> result = new LinkedHashMap<>(2, 1);
    // CHECKSTYLE:ON
    ShardingContexts shardingContexts = new ShardingContexts(taskId, "test_job", 1, "", Collections.singletonMap(1, "a"));
    result.put("shardingContext", shardingContexts);
    result.put("jobConfigContext", jobConfigurationContext);
    return SerializationUtils.serialize(result);
}
Also used : ByteString(com.google.protobuf.ByteString) ShardingContexts(com.dangdang.ddframe.job.executor.ShardingContexts) LinkedHashMap(java.util.LinkedHashMap)

Example 14 with ShardingContexts

use of com.dangdang.ddframe.job.executor.ShardingContexts in project elastic-job by dangdangdotcom.

the class ShardingContextTest method assertNew.

@Test
public void assertNew() {
    ShardingContexts shardingContexts = ShardingContextsBuilder.getMultipleShardingContexts();
    ShardingContext actual = new ShardingContext(shardingContexts, 1);
    assertThat(actual.getJobName(), is(shardingContexts.getJobName()));
    assertThat(actual.getTaskId(), is(shardingContexts.getTaskId()));
    assertThat(actual.getShardingTotalCount(), is(shardingContexts.getShardingTotalCount()));
    assertThat(actual.getJobParameter(), is(shardingContexts.getJobParameter()));
    assertThat(actual.getShardingItem(), is(1));
    assertThat(actual.getShardingParameter(), is(shardingContexts.getShardingItemParameters().get(1)));
}
Also used : ShardingContexts(com.dangdang.ddframe.job.executor.ShardingContexts) Test(org.junit.Test)

Example 15 with ShardingContexts

use of com.dangdang.ddframe.job.executor.ShardingContexts in project elastic-job by dangdangdotcom.

the class SimpleJobExecutorTest method assertExecuteWhenRunOnceWithMisfireIsNotEmptyButIsNotEligibleForJobRunning.

@Test
public void assertExecuteWhenRunOnceWithMisfireIsNotEmptyButIsNotEligibleForJobRunning() {
    ShardingContexts shardingContexts = ShardingContextsBuilder.getMultipleShardingContexts();
    when(jobFacade.getShardingContexts()).thenReturn(shardingContexts);
    when(jobFacade.isExecuteMisfired(shardingContexts.getShardingItemParameters().keySet())).thenReturn(false);
    when(jobFacade.isEligibleForJobRunning()).thenReturn(false);
    simpleJobExecutor.execute();
    ElasticJobVerify.verifyForIsNotMisfire(jobFacade, shardingContexts);
    verify(jobCaller, times(2)).execute();
    verify(jobFacade, times(0)).clearMisfire(shardingContexts.getShardingItemParameters().keySet());
}
Also used : ShardingContexts(com.dangdang.ddframe.job.executor.ShardingContexts) Test(org.junit.Test)

Aggregations

ShardingContexts (com.dangdang.ddframe.job.executor.ShardingContexts)28 Test (org.junit.Test)23 SimpleJobConfiguration (com.dangdang.ddframe.job.config.simple.SimpleJobConfiguration)6 HashMap (java.util.HashMap)6 TestSimpleJob (com.dangdang.ddframe.job.lite.fixture.TestSimpleJob)4 DataflowJobConfiguration (com.dangdang.ddframe.job.config.dataflow.DataflowJobConfiguration)3 TestDataflowJob (com.dangdang.ddframe.job.lite.fixture.TestDataflowJob)3 ShardingItemParameters (com.dangdang.ddframe.job.util.config.ShardingItemParameters)2 ByteString (com.google.protobuf.ByteString)2 CloudAppConfiguration (com.dangdang.ddframe.job.cloud.scheduler.config.app.CloudAppConfiguration)1 CloudJobConfiguration (com.dangdang.ddframe.job.cloud.scheduler.config.job.CloudJobConfiguration)1 TaskContext (com.dangdang.ddframe.job.context.TaskContext)1 TestDistributeOnceElasticJobListener (com.dangdang.ddframe.job.lite.api.listener.fixture.TestDistributeOnceElasticJobListener)1 LiteJobConfiguration (com.dangdang.ddframe.job.lite.config.LiteJobConfiguration)1 LinkedHashMap (java.util.LinkedHashMap)1 Protos (org.apache.mesos.Protos)1 Before (org.junit.Before)1