Examples with SegmentWithHllIndexCreateHelper com.linkedin.pinot.core.startree.hll.SegmentWithHllIndexCreateHelper used on opensource projects

Example 1 with SegmentWithHllIndexCreateHelper

use of com.linkedin.pinot.core.startree.hll.SegmentWithHllIndexCreateHelper in project pinot by linkedin.

the class ColumnMetadataTest method testHllIndexRelatedMetadata.

@Test
public void testHllIndexRelatedMetadata() throws Exception {
    SegmentWithHllIndexCreateHelper helper = null;
    try {
        // Build the Segment metadata.
        helper = new SegmentWithHllIndexCreateHelper("testHllIndexRelatedMetadata", getClass().getClassLoader().getResource("data/test_data-sv.avro"), "daysSinceEpoch", TimeUnit.DAYS, "starTreeSegment");
        helper.build(true, new HllConfig(9, new HashSet<String>(Arrays.asList("column7")), "_hllSuffix"));
        // Load segment metadata.
        IndexSegment segment = Loaders.IndexSegment.load(helper.getSegmentDirectory(), ReadMode.mmap);
        SegmentMetadataImpl metadata = (SegmentMetadataImpl) segment.getSegmentMetadata();
        Assert.assertEquals(metadata.getHllLog2m(), 9);
        // Verify Hll Related Info
        StarTreeMetadata starTreeMetadata = metadata.getStarTreeMetadata();
        Assert.assertNotNull(starTreeMetadata);
        ColumnMetadata column = metadata.getColumnMetadataFor("column7_hllSuffix");
        Assert.assertEquals(column.getDerivedMetricType(), MetricFieldSpec.DerivedMetricType.HLL);
        Assert.assertEquals(column.getOriginColumnName(), "column7");
    } finally {
        if (helper != null) {
            helper.cleanTempDir();
        }
    }
}

Example 2 with SegmentWithHllIndexCreateHelper

use of com.linkedin.pinot.core.startree.hll.SegmentWithHllIndexCreateHelper in project pinot by linkedin.

the class TestHelper method buildLoadDefaultHllSegment.

SegmentWithHllIndexCreateHelper buildLoadDefaultHllSegment(HllConfig hllConfig) throws Exception {
    SegmentWithHllIndexCreateHelper helper = buildDefaultHllSegment(hllConfig);
    loadSegment(helper.getSegmentDirectory());
    return helper;
}

Example 3 with SegmentWithHllIndexCreateHelper

use of com.linkedin.pinot.core.startree.hll.SegmentWithHllIndexCreateHelper in project pinot by linkedin.

the class TestHelper method buildHllSegment.

SegmentWithHllIndexCreateHelper buildHllSegment(String tableName, String avroData, String timeColumn, TimeUnit timeUnit, String testSegment, HllConfig hllConfig, boolean enableStarTree) throws Exception {
    SegmentWithHllIndexCreateHelper helper = new SegmentWithHllIndexCreateHelper(tableName, avroData, timeColumn, timeUnit, testSegment);
    helper.build(enableStarTree, hllConfig);
    return helper;
}