Search in sources :

Example 6 with RealtimeIndexTask

use of org.apache.druid.indexing.common.task.RealtimeIndexTask in project druid by druid-io.

the class TaskAnnouncementTest method testBackwardsCompatibleSerde.

@Test
public void testBackwardsCompatibleSerde() throws Exception {
    final Task task = new RealtimeIndexTask("theid", new TaskResource("rofl", 2), new FireDepartment(new DataSchema("foo", null, new AggregatorFactory[0], null, null, new DefaultObjectMapper()), new RealtimeIOConfig(new LocalFirehoseFactory(new File("lol"), "rofl", null), (schema, config, metrics) -> null), null), null);
    final TaskStatus status = TaskStatus.running(task.getId());
    final TaskAnnouncement announcement = TaskAnnouncement.create(task, status, TaskLocation.unknown());
    final String statusJson = jsonMapper.writeValueAsString(status);
    final String announcementJson = jsonMapper.writeValueAsString(announcement);
    final TaskStatus statusFromStatus = jsonMapper.readValue(statusJson, TaskStatus.class);
    final TaskStatus statusFromAnnouncement = jsonMapper.readValue(announcementJson, TaskStatus.class);
    final TaskAnnouncement announcementFromStatus = jsonMapper.readValue(statusJson, TaskAnnouncement.class);
    final TaskAnnouncement announcementFromAnnouncement = jsonMapper.readValue(announcementJson, TaskAnnouncement.class);
    Assert.assertEquals("theid", statusFromStatus.getId());
    Assert.assertEquals("theid", statusFromAnnouncement.getId());
    Assert.assertEquals("theid", announcementFromStatus.getTaskStatus().getId());
    Assert.assertEquals("theid", announcementFromAnnouncement.getTaskStatus().getId());
    Assert.assertEquals("theid", announcementFromStatus.getTaskResource().getAvailabilityGroup());
    Assert.assertEquals("rofl", announcementFromAnnouncement.getTaskResource().getAvailabilityGroup());
    Assert.assertEquals(1, announcementFromStatus.getTaskResource().getRequiredCapacity());
    Assert.assertEquals(2, announcementFromAnnouncement.getTaskResource().getRequiredCapacity());
}
Also used : DataSchema(org.apache.druid.segment.indexing.DataSchema) FireDepartment(org.apache.druid.segment.realtime.FireDepartment) RealtimeIndexTask(org.apache.druid.indexing.common.task.RealtimeIndexTask) Task(org.apache.druid.indexing.common.task.Task) RealtimeIOConfig(org.apache.druid.segment.indexing.RealtimeIOConfig) RealtimeIndexTask(org.apache.druid.indexing.common.task.RealtimeIndexTask) TaskResource(org.apache.druid.indexing.common.task.TaskResource) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) LocalFirehoseFactory(org.apache.druid.segment.realtime.firehose.LocalFirehoseFactory) TaskStatus(org.apache.druid.indexer.TaskStatus) File(java.io.File) Test(org.junit.Test)

Example 7 with RealtimeIndexTask

use of org.apache.druid.indexing.common.task.RealtimeIndexTask in project druid by druid-io.

the class DruidJsonValidatorTest method testTaskValidator.

@Test
public void testTaskValidator() throws Exception {
    final ObjectMapper jsonMapper = new DefaultObjectMapper();
    for (final Module jacksonModule : new FirehoseModule().getJacksonModules()) {
        jsonMapper.registerModule(jacksonModule);
    }
    final RealtimeIndexTask task = new RealtimeIndexTask(null, new TaskResource("rofl", 2), new FireDepartment(new DataSchema("foo", null, new AggregatorFactory[0], new UniformGranularitySpec(Granularities.HOUR, Granularities.NONE, null), null, jsonMapper), new RealtimeIOConfig(new LocalFirehoseFactory(new File("lol"), "rofl", null), (schema, config, metrics) -> null), new RealtimeTuningConfig(null, 1, null, null, new Period("PT10M"), null, null, null, null, 1, NoneShardSpec.instance(), new IndexSpec(), new IndexSpec(), 0, 0, true, null, null, null, null)), null);
    File tmp = temporaryFolder.newFile("test_task.json");
    jsonMapper.writeValue(tmp, task);
    parseCommand("validator", "-f", tmp.getAbsolutePath(), "-t", "task").run();
}
Also used : RealtimeIOConfig(org.apache.druid.segment.indexing.RealtimeIOConfig) IndexSpec(org.apache.druid.segment.IndexSpec) RealtimeIndexTask(org.apache.druid.indexing.common.task.RealtimeIndexTask) TaskResource(org.apache.druid.indexing.common.task.TaskResource) Period(org.joda.time.Period) LocalFirehoseFactory(org.apache.druid.segment.realtime.firehose.LocalFirehoseFactory) RealtimeTuningConfig(org.apache.druid.segment.indexing.RealtimeTuningConfig) DataSchema(org.apache.druid.segment.indexing.DataSchema) FireDepartment(org.apache.druid.segment.realtime.FireDepartment) UniformGranularitySpec(org.apache.druid.segment.indexing.granularity.UniformGranularitySpec) FirehoseModule(org.apache.druid.guice.FirehoseModule) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) Module(com.fasterxml.jackson.databind.Module) FirehoseModule(org.apache.druid.guice.FirehoseModule) File(java.io.File) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DefaultObjectMapper(org.apache.druid.jackson.DefaultObjectMapper) Test(org.junit.Test)

Aggregations

RealtimeIndexTask (org.apache.druid.indexing.common.task.RealtimeIndexTask)7 Test (org.junit.Test)6 RealtimeIOConfig (org.apache.druid.segment.indexing.RealtimeIOConfig)5 FireDepartment (org.apache.druid.segment.realtime.FireDepartment)5 File (java.io.File)3 Executor (java.util.concurrent.Executor)3 TaskStatus (org.apache.druid.indexer.TaskStatus)3 Task (org.apache.druid.indexing.common.task.Task)3 TaskResource (org.apache.druid.indexing.common.task.TaskResource)3 DataSchema (org.apache.druid.segment.indexing.DataSchema)3 TaskRunnerListener (org.apache.druid.indexing.overlord.TaskRunnerListener)2 DefaultObjectMapper (org.apache.druid.jackson.DefaultObjectMapper)2 Monitor (org.apache.druid.java.util.metrics.Monitor)2 RealtimeTuningConfig (org.apache.druid.segment.indexing.RealtimeTuningConfig)2 UniformGranularitySpec (org.apache.druid.segment.indexing.granularity.UniformGranularitySpec)2 FireDepartmentTest (org.apache.druid.segment.realtime.FireDepartmentTest)2 LocalFirehoseFactory (org.apache.druid.segment.realtime.firehose.LocalFirehoseFactory)2 InitializedNullHandlingTest (org.apache.druid.testing.InitializedNullHandlingTest)2 DataSegment (org.apache.druid.timeline.DataSegment)2 Period (org.joda.time.Period)2