Search in sources :

Example 1 with DatasourcePathSpec

use of io.druid.indexer.path.DatasourcePathSpec in project druid by druid-io.

the class HadoopIngestionSpecUpdateDatasourcePathSpecSegmentsTest method testupdateSegmentListIfDatasourcePathSpecIsUsedWithMultiplePathSpec.

@Test
public void testupdateSegmentListIfDatasourcePathSpecIsUsedWithMultiplePathSpec() throws Exception {
    PathSpec pathSpec = new MultiplePathSpec(ImmutableList.of(new StaticPathSpec("/xyz", null), new DatasourcePathSpec(jsonMapper, null, new DatasourceIngestionSpec(testDatasource, testDatasourceInterval, null, null, null, null, null, null, false), null)));
    HadoopDruidIndexerConfig config = testRunUpdateSegmentListIfDatasourcePathSpecIsUsed(pathSpec, testDatasourceInterval);
    Assert.assertEquals(ImmutableList.of(WindowedDataSegment.of(SEGMENT)), ((DatasourcePathSpec) ((MultiplePathSpec) config.getPathSpec()).getChildren().get(1)).getSegments());
}
Also used : DatasourceIngestionSpec(io.druid.indexer.hadoop.DatasourceIngestionSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) StaticPathSpec(io.druid.indexer.path.StaticPathSpec) MultiplePathSpec(io.druid.indexer.path.MultiplePathSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) StaticPathSpec(io.druid.indexer.path.StaticPathSpec) PathSpec(io.druid.indexer.path.PathSpec) MultiplePathSpec(io.druid.indexer.path.MultiplePathSpec) Test(org.junit.Test)

Example 2 with DatasourcePathSpec

use of io.druid.indexer.path.DatasourcePathSpec in project druid by druid-io.

the class HadoopIngestionSpecUpdateDatasourcePathSpecSegmentsTest method testupdateSegmentListIfDatasourcePathSpecIsUsedWithJustDatasourcePathSpecAndPartialInterval.

@Test
public void testupdateSegmentListIfDatasourcePathSpecIsUsedWithJustDatasourcePathSpecAndPartialInterval() throws Exception {
    PathSpec pathSpec = new DatasourcePathSpec(jsonMapper, null, new DatasourceIngestionSpec(testDatasource, testDatasourceIntervalPartial, null, null, null, null, null, null, false), null);
    HadoopDruidIndexerConfig config = testRunUpdateSegmentListIfDatasourcePathSpecIsUsed(pathSpec, testDatasourceIntervalPartial);
    Assert.assertEquals(ImmutableList.of(new WindowedDataSegment(SEGMENT, testDatasourceIntervalPartial)), ((DatasourcePathSpec) config.getPathSpec()).getSegments());
}
Also used : DatasourceIngestionSpec(io.druid.indexer.hadoop.DatasourceIngestionSpec) WindowedDataSegment(io.druid.indexer.hadoop.WindowedDataSegment) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) StaticPathSpec(io.druid.indexer.path.StaticPathSpec) PathSpec(io.druid.indexer.path.PathSpec) MultiplePathSpec(io.druid.indexer.path.MultiplePathSpec) Test(org.junit.Test)

Example 3 with DatasourcePathSpec

use of io.druid.indexer.path.DatasourcePathSpec in project druid by druid-io.

the class HadoopIngestionSpecUpdateDatasourcePathSpecSegmentsTest method testupdateSegmentListIfDatasourcePathSpecWithMatchingUserSegments.

@Test
public void testupdateSegmentListIfDatasourcePathSpecWithMatchingUserSegments() throws Exception {
    PathSpec pathSpec = new DatasourcePathSpec(jsonMapper, null, new DatasourceIngestionSpec(testDatasource, testDatasourceInterval, null, ImmutableList.<DataSegment>of(SEGMENT), null, null, null, null, false), null);
    HadoopDruidIndexerConfig config = testRunUpdateSegmentListIfDatasourcePathSpecIsUsed(pathSpec, testDatasourceInterval);
    Assert.assertEquals(ImmutableList.of(WindowedDataSegment.of(SEGMENT)), ((DatasourcePathSpec) config.getPathSpec()).getSegments());
}
Also used : DatasourceIngestionSpec(io.druid.indexer.hadoop.DatasourceIngestionSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) StaticPathSpec(io.druid.indexer.path.StaticPathSpec) PathSpec(io.druid.indexer.path.PathSpec) MultiplePathSpec(io.druid.indexer.path.MultiplePathSpec) DataSegment(io.druid.timeline.DataSegment) WindowedDataSegment(io.druid.indexer.hadoop.WindowedDataSegment) Test(org.junit.Test)

Example 4 with DatasourcePathSpec

use of io.druid.indexer.path.DatasourcePathSpec in project druid by druid-io.

the class HadoopIngestionSpecUpdateDatasourcePathSpecSegmentsTest method testupdateSegmentListThrowsExceptionWithUserSegmentsMismatch.

@Test(expected = IOException.class)
public void testupdateSegmentListThrowsExceptionWithUserSegmentsMismatch() throws Exception {
    PathSpec pathSpec = new DatasourcePathSpec(jsonMapper, null, new DatasourceIngestionSpec(testDatasource, testDatasourceInterval, null, ImmutableList.<DataSegment>of(SEGMENT.withVersion("v2")), null, null, null, null, false), null);
    testRunUpdateSegmentListIfDatasourcePathSpecIsUsed(pathSpec, testDatasourceInterval);
}
Also used : DatasourceIngestionSpec(io.druid.indexer.hadoop.DatasourceIngestionSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) StaticPathSpec(io.druid.indexer.path.StaticPathSpec) PathSpec(io.druid.indexer.path.PathSpec) MultiplePathSpec(io.druid.indexer.path.MultiplePathSpec) DataSegment(io.druid.timeline.DataSegment) WindowedDataSegment(io.druid.indexer.hadoop.WindowedDataSegment) Test(org.junit.Test)

Example 5 with DatasourcePathSpec

use of io.druid.indexer.path.DatasourcePathSpec in project druid by druid-io.

the class HadoopIngestionSpecUpdateDatasourcePathSpecSegmentsTest method testupdateSegmentListIfDatasourcePathSpecIsUsedWithJustDatasourcePathSpec.

@Test
public void testupdateSegmentListIfDatasourcePathSpecIsUsedWithJustDatasourcePathSpec() throws Exception {
    PathSpec pathSpec = new DatasourcePathSpec(jsonMapper, null, new DatasourceIngestionSpec(testDatasource, testDatasourceInterval, null, null, null, null, null, null, false), null);
    HadoopDruidIndexerConfig config = testRunUpdateSegmentListIfDatasourcePathSpecIsUsed(pathSpec, testDatasourceInterval);
    Assert.assertEquals(ImmutableList.of(WindowedDataSegment.of(SEGMENT)), ((DatasourcePathSpec) config.getPathSpec()).getSegments());
}
Also used : DatasourceIngestionSpec(io.druid.indexer.hadoop.DatasourceIngestionSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) DatasourcePathSpec(io.druid.indexer.path.DatasourcePathSpec) StaticPathSpec(io.druid.indexer.path.StaticPathSpec) PathSpec(io.druid.indexer.path.PathSpec) MultiplePathSpec(io.druid.indexer.path.MultiplePathSpec) Test(org.junit.Test)

Aggregations

DatasourceIngestionSpec (io.druid.indexer.hadoop.DatasourceIngestionSpec)5 DatasourcePathSpec (io.druid.indexer.path.DatasourcePathSpec)5 MultiplePathSpec (io.druid.indexer.path.MultiplePathSpec)5 PathSpec (io.druid.indexer.path.PathSpec)5 StaticPathSpec (io.druid.indexer.path.StaticPathSpec)5 Test (org.junit.Test)5 WindowedDataSegment (io.druid.indexer.hadoop.WindowedDataSegment)3 DataSegment (io.druid.timeline.DataSegment)2