Search in sources :

Example 6 with SegmentIndexCreationDriver

use of com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver in project pinot by linkedin.

the class DefaultSegmentNameGeneratorTest method testPostfix.

@Test
public void testPostfix() throws Exception {
    ColumnMetadataTest columnMetadataTest = new ColumnMetadataTest();
    // Build the Segment metadata.
    SegmentGeneratorConfig config = columnMetadataTest.CreateSegmentConfigWithoutCreator();
    SegmentNameGenerator segmentNameGenerator = new DefaultSegmentNameGenerator("daysSinceEpoch", "mytable", "1", -1);
    config.setSegmentNameGenerator(segmentNameGenerator);
    SegmentIndexCreationDriver driver = SegmentCreationDriverFactory.get(null);
    driver.init(config);
    driver.build();
    Assert.assertEquals(driver.getSegmentName(), "mytable_1756015683_1756015683_1");
}
Also used : SegmentIndexCreationDriver(com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) SegmentGeneratorConfig(com.linkedin.pinot.core.indexsegment.generator.SegmentGeneratorConfig) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) Test(org.testng.annotations.Test)

Example 7 with SegmentIndexCreationDriver

use of com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver in project pinot by linkedin.

the class DefaultSegmentNameGeneratorTest method testAlreadyNamedSegment.

@Test
public void testAlreadyNamedSegment() throws Exception {
    ColumnMetadataTest columnMetadataTest = new ColumnMetadataTest();
    // Build the Segment metadata.
    SegmentGeneratorConfig config = columnMetadataTest.CreateSegmentConfigWithoutCreator();
    SegmentNameGenerator segmentNameGenerator = new DefaultSegmentNameGenerator("mytable_1");
    config.setSegmentNameGenerator(segmentNameGenerator);
    SegmentIndexCreationDriver driver = SegmentCreationDriverFactory.get(null);
    driver.init(config);
    driver.build();
    Assert.assertEquals(driver.getSegmentName(), "mytable_1");
}
Also used : SegmentIndexCreationDriver(com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) SegmentGeneratorConfig(com.linkedin.pinot.core.indexsegment.generator.SegmentGeneratorConfig) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) Test(org.testng.annotations.Test)

Example 8 with SegmentIndexCreationDriver

use of com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver in project pinot by linkedin.

the class DefaultSegmentNameGeneratorTest method testNullTimeColumn.

@Test
public void testNullTimeColumn() throws Exception {
    ColumnMetadataTest columnMetadataTest = new ColumnMetadataTest();
    // Build the Segment metadata.
    SegmentGeneratorConfig config = columnMetadataTest.CreateSegmentConfigWithoutCreator();
    config.setTableName("mytable");
    config.setSegmentNamePostfix("postfix");
    config.setTimeColumnName(null);
    SegmentIndexCreationDriver driver = SegmentCreationDriverFactory.get(null);
    driver.init(config);
    driver.build();
    Assert.assertEquals(driver.getSegmentName(), "mytable_postfix");
}
Also used : SegmentIndexCreationDriver(com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) SegmentGeneratorConfig(com.linkedin.pinot.core.indexsegment.generator.SegmentGeneratorConfig) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) Test(org.testng.annotations.Test)

Example 9 with SegmentIndexCreationDriver

use of com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver in project pinot by linkedin.

the class DefaultSegmentNameGeneratorTest method testNullTimeColumnThroughDefaultSegment.

@Test
public void testNullTimeColumnThroughDefaultSegment() throws Exception {
    ColumnMetadataTest columnMetadataTest = new ColumnMetadataTest();
    // Build the Segment metadata.
    SegmentGeneratorConfig config = columnMetadataTest.CreateSegmentConfigWithoutCreator();
    SegmentNameGenerator segmentNameGenerator = new DefaultSegmentNameGenerator(null, "mytable", "1", 2);
    config.setSegmentNameGenerator(segmentNameGenerator);
    SegmentIndexCreationDriver driver = SegmentCreationDriverFactory.get(null);
    driver.init(config);
    driver.build();
    Assert.assertEquals(driver.getSegmentName(), "mytable_1_2");
}
Also used : SegmentIndexCreationDriver(com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) SegmentGeneratorConfig(com.linkedin.pinot.core.indexsegment.generator.SegmentGeneratorConfig) ColumnMetadataTest(com.linkedin.pinot.core.segment.index.ColumnMetadataTest) Test(org.testng.annotations.Test)

Example 10 with SegmentIndexCreationDriver

use of com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver in project pinot by linkedin.

the class ColumnMetadataTest method testAllFieldsExceptCreatorName.

@Test
public void testAllFieldsExceptCreatorName() throws Exception {
    // Build the Segment metadata.
    SegmentGeneratorConfig config = CreateSegmentConfigWithoutCreator();
    SegmentIndexCreationDriver driver = SegmentCreationDriverFactory.get(null);
    driver.init(config);
    driver.build();
    // Load segment metadata.
    IndexSegment segment = Loaders.IndexSegment.load(INDEX_DIR.listFiles()[0], ReadMode.mmap);
    SegmentMetadataImpl metadata = (SegmentMetadataImpl) segment.getSegmentMetadata();
    verifySegmentAfterLoading(metadata);
    // Make sure we get null for creator name.
    String creatorName = metadata.getCreatorName();
    Assert.assertEquals(creatorName, null);
}
Also used : SegmentIndexCreationDriver(com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver) IndexSegment(com.linkedin.pinot.core.indexsegment.IndexSegment) SegmentGeneratorConfig(com.linkedin.pinot.core.indexsegment.generator.SegmentGeneratorConfig) Test(org.testng.annotations.Test)

Aggregations

SegmentIndexCreationDriver (com.linkedin.pinot.core.segment.creator.SegmentIndexCreationDriver)37 SegmentGeneratorConfig (com.linkedin.pinot.core.indexsegment.generator.SegmentGeneratorConfig)34 File (java.io.File)21 Test (org.testng.annotations.Test)13 ColumnMetadataTest (com.linkedin.pinot.core.segment.index.ColumnMetadataTest)7 IndexSegment (com.linkedin.pinot.core.indexsegment.IndexSegment)6 BeforeClass (org.testng.annotations.BeforeClass)6 SegmentIndexCreationDriverImpl (com.linkedin.pinot.core.segment.creator.impl.SegmentIndexCreationDriverImpl)5 URL (java.net.URL)5 Schema (com.linkedin.pinot.common.data.Schema)3 HashMap (java.util.HashMap)3 Field (org.apache.avro.Schema.Field)3 GenericRecord (org.apache.avro.generic.GenericRecord)3 BeforeMethod (org.testng.annotations.BeforeMethod)3 StarTreeIndexSpec (com.linkedin.pinot.common.data.StarTreeIndexSpec)2 IndexLoadingConfigMetadata (com.linkedin.pinot.common.metadata.segment.IndexLoadingConfigMetadata)2 Configuration (org.apache.commons.configuration.Configuration)2 PropertiesConfiguration (org.apache.commons.configuration.PropertiesConfiguration)2 BeforeTest (org.testng.annotations.BeforeTest)2 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)1