Search in sources :

Example 16 with Builder

use of org.apache.druid.indexing.common.task.CompactionTask.Builder in project druid by druid-io.

the class CompactionTaskTest method testCreateCompactionTaskWithGranularitySpec.

@Test
public void testCreateCompactionTaskWithGranularitySpec() {
    final Builder builder = new Builder(DATA_SOURCE, segmentCacheManagerFactory, RETRY_POLICY_FACTORY);
    builder.inputSpec(new CompactionIntervalSpec(COMPACTION_INTERVAL, SegmentUtils.hashIds(SEGMENTS)));
    builder.tuningConfig(createTuningConfig());
    builder.segmentGranularity(Granularities.HOUR);
    final CompactionTask taskCreatedWithSegmentGranularity = builder.build();
    final Builder builder2 = new Builder(DATA_SOURCE, segmentCacheManagerFactory, RETRY_POLICY_FACTORY);
    builder2.inputSpec(new CompactionIntervalSpec(COMPACTION_INTERVAL, SegmentUtils.hashIds(SEGMENTS)));
    builder2.tuningConfig(createTuningConfig());
    builder2.granularitySpec(new ClientCompactionTaskGranularitySpec(Granularities.HOUR, Granularities.DAY, null));
    final CompactionTask taskCreatedWithGranularitySpec = builder2.build();
    Assert.assertEquals(taskCreatedWithGranularitySpec.getSegmentGranularity(), taskCreatedWithSegmentGranularity.getSegmentGranularity());
}
Also used : Builder(org.apache.druid.indexing.common.task.CompactionTask.Builder) ClientCompactionTaskGranularitySpec(org.apache.druid.client.indexing.ClientCompactionTaskGranularitySpec) Test(org.junit.Test)

Example 17 with Builder

use of org.apache.druid.indexing.common.task.CompactionTask.Builder in project druid by druid-io.

the class CompactionTaskTest method testCreateCompactionTaskWithMetricsSpec.

@Test
public void testCreateCompactionTaskWithMetricsSpec() {
    AggregatorFactory[] aggregatorFactories = new AggregatorFactory[] { new CountAggregatorFactory("cnt") };
    final Builder builder = new Builder(DATA_SOURCE, segmentCacheManagerFactory, RETRY_POLICY_FACTORY);
    builder.inputSpec(new CompactionIntervalSpec(COMPACTION_INTERVAL, SegmentUtils.hashIds(SEGMENTS)));
    builder.tuningConfig(createTuningConfig());
    builder.metricsSpec(aggregatorFactories);
    final CompactionTask taskCreatedWithTransformSpec = builder.build();
    Assert.assertArrayEquals(aggregatorFactories, taskCreatedWithTransformSpec.getMetricsSpec());
}
Also used : CountAggregatorFactory(org.apache.druid.query.aggregation.CountAggregatorFactory) Builder(org.apache.druid.indexing.common.task.CompactionTask.Builder) DoubleLastAggregatorFactory(org.apache.druid.query.aggregation.last.DoubleLastAggregatorFactory) FloatMinAggregatorFactory(org.apache.druid.query.aggregation.FloatMinAggregatorFactory) FloatFirstAggregatorFactory(org.apache.druid.query.aggregation.first.FloatFirstAggregatorFactory) LongSumAggregatorFactory(org.apache.druid.query.aggregation.LongSumAggregatorFactory) AggregatorFactory(org.apache.druid.query.aggregation.AggregatorFactory) CountAggregatorFactory(org.apache.druid.query.aggregation.CountAggregatorFactory) DoubleMaxAggregatorFactory(org.apache.druid.query.aggregation.DoubleMaxAggregatorFactory) LongMaxAggregatorFactory(org.apache.druid.query.aggregation.LongMaxAggregatorFactory) Test(org.junit.Test)

Example 18 with Builder

use of org.apache.druid.indexing.common.task.CompactionTask.Builder in project druid by druid-io.

the class CompactionTaskTest method testCreateCompactionTaskWithTransformSpec.

@Test
public void testCreateCompactionTaskWithTransformSpec() {
    ClientCompactionTaskTransformSpec transformSpec = new ClientCompactionTaskTransformSpec(new SelectorDimFilter("dim1", "foo", null));
    final Builder builder = new Builder(DATA_SOURCE, segmentCacheManagerFactory, RETRY_POLICY_FACTORY);
    builder.inputSpec(new CompactionIntervalSpec(COMPACTION_INTERVAL, SegmentUtils.hashIds(SEGMENTS)));
    builder.tuningConfig(createTuningConfig());
    builder.transformSpec(transformSpec);
    final CompactionTask taskCreatedWithTransformSpec = builder.build();
    Assert.assertEquals(transformSpec, taskCreatedWithTransformSpec.getTransformSpec());
}
Also used : SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) Builder(org.apache.druid.indexing.common.task.CompactionTask.Builder) ClientCompactionTaskTransformSpec(org.apache.druid.client.indexing.ClientCompactionTaskTransformSpec) Test(org.junit.Test)

Example 19 with Builder

use of org.apache.druid.indexing.common.task.CompactionTask.Builder in project druid by druid-io.

the class CompactionTaskTest method testEmptyInterval.

@Test
public void testEmptyInterval() {
    expectedException.expect(IllegalArgumentException.class);
    expectedException.expectMessage(CoreMatchers.containsString("must specify a nonempty interval"));
    final Builder builder = new Builder(DATA_SOURCE, segmentCacheManagerFactory, RETRY_POLICY_FACTORY);
    @SuppressWarnings("unused") final CompactionTask task = builder.interval(Intervals.of("2000-01-01/2000-01-01")).build();
}
Also used : Builder(org.apache.druid.indexing.common.task.CompactionTask.Builder) Test(org.junit.Test)

Example 20 with Builder

use of org.apache.druid.indexing.common.task.CompactionTask.Builder in project druid by druid-io.

the class CompactionTaskTest method testSerdeWithInterval.

@Test
public void testSerdeWithInterval() throws IOException {
    final Builder builder = new Builder(DATA_SOURCE, segmentCacheManagerFactory, RETRY_POLICY_FACTORY);
    final CompactionTask task = builder.inputSpec(new CompactionIntervalSpec(COMPACTION_INTERVAL, SegmentUtils.hashIds(SEGMENTS))).tuningConfig(createTuningConfig()).context(ImmutableMap.of("testKey", "testContext")).build();
    final byte[] bytes = OBJECT_MAPPER.writeValueAsBytes(task);
    final CompactionTask fromJson = OBJECT_MAPPER.readValue(bytes, CompactionTask.class);
    assertEquals(task, fromJson);
}
Also used : Builder(org.apache.druid.indexing.common.task.CompactionTask.Builder) Test(org.junit.Test)

Aggregations

Builder (org.apache.druid.indexing.common.task.CompactionTask.Builder)40 Test (org.junit.Test)40 DataSegment (org.apache.druid.timeline.DataSegment)29 HashBasedNumberedShardSpec (org.apache.druid.timeline.partition.HashBasedNumberedShardSpec)18 NumberedShardSpec (org.apache.druid.timeline.partition.NumberedShardSpec)18 ImmutableList (com.google.common.collect.ImmutableList)17 ArrayList (java.util.ArrayList)17 List (java.util.List)17 TaskStatus (org.apache.druid.indexer.TaskStatus)15 AbstractParallelIndexSupervisorTaskTest (org.apache.druid.indexing.common.task.batch.parallel.AbstractParallelIndexSupervisorTaskTest)13 NumberedOverwriteShardSpec (org.apache.druid.timeline.partition.NumberedOverwriteShardSpec)13 ClientCompactionTaskGranularitySpec (org.apache.druid.client.indexing.ClientCompactionTaskGranularitySpec)12 DimensionsSpec (org.apache.druid.data.input.impl.DimensionsSpec)12 UniformGranularitySpec (org.apache.druid.segment.indexing.granularity.UniformGranularitySpec)12 HashMap (java.util.HashMap)11 Map (java.util.Map)11 CompactionState (org.apache.druid.timeline.CompactionState)11 ImmutableMap (com.google.common.collect.ImmutableMap)8 DynamicPartitionsSpec (org.apache.druid.indexer.partitions.DynamicPartitionsSpec)5 Interval (org.joda.time.Interval)5