Search in sources :

Example 6 with AbstractColumnStatisticsCollector

use of com.linkedin.pinot.core.segment.creator.AbstractColumnStatisticsCollector in project pinot by linkedin.

the class DictionariesTest method testDoubleColumnPreIndexStatsCollector.

@Test
public void testDoubleColumnPreIndexStatsCollector() throws Exception {
    FieldSpec spec = new DimensionFieldSpec("column1", DataType.DOUBLE, true);
    AbstractColumnStatisticsCollector statsCollector = new DoubleColumnPreIndexStatsCollector(spec);
    statsCollector.collect(new Integer(1));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Float(2));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Long(3));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Double(4));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Integer(4));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Float(2));
    Assert.assertFalse(statsCollector.isSorted());
    statsCollector.collect(new Double(40));
    Assert.assertFalse(statsCollector.isSorted());
    statsCollector.collect(new Double(20));
    Assert.assertFalse(statsCollector.isSorted());
    statsCollector.seal();
    Assert.assertEquals(statsCollector.getCardinality(), 6);
    Assert.assertEquals(((Number) statsCollector.getMinValue()).intValue(), 1);
    Assert.assertEquals(((Number) statsCollector.getMaxValue()).intValue(), 40);
    Assert.assertFalse(statsCollector.isSorted());
}
Also used : DoubleColumnPreIndexStatsCollector(com.linkedin.pinot.core.segment.creator.impl.stats.DoubleColumnPreIndexStatsCollector) AbstractColumnStatisticsCollector(com.linkedin.pinot.core.segment.creator.AbstractColumnStatisticsCollector) FieldSpec(com.linkedin.pinot.common.data.FieldSpec) DimensionFieldSpec(com.linkedin.pinot.common.data.DimensionFieldSpec) DimensionFieldSpec(com.linkedin.pinot.common.data.DimensionFieldSpec) Test(org.testng.annotations.Test)

Example 7 with AbstractColumnStatisticsCollector

use of com.linkedin.pinot.core.segment.creator.AbstractColumnStatisticsCollector in project pinot by linkedin.

the class DictionariesTest method testFloatColumnPreIndexStatsCollector.

@Test
public void testFloatColumnPreIndexStatsCollector() throws Exception {
    FieldSpec spec = new DimensionFieldSpec("column1", DataType.FLOAT, true);
    AbstractColumnStatisticsCollector statsCollector = new FloatColumnPreIndexStatsCollector(spec);
    statsCollector.collect(new Integer(1));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Float(2));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Long(3));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Double(4));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Integer(4));
    Assert.assertTrue(statsCollector.isSorted());
    statsCollector.collect(new Float(2));
    Assert.assertFalse(statsCollector.isSorted());
    statsCollector.collect(new Double(40));
    Assert.assertFalse(statsCollector.isSorted());
    statsCollector.collect(new Double(20));
    Assert.assertFalse(statsCollector.isSorted());
    statsCollector.seal();
    Assert.assertEquals(statsCollector.getCardinality(), 6);
    Assert.assertEquals(((Number) statsCollector.getMinValue()).intValue(), 1);
    Assert.assertEquals(((Number) statsCollector.getMaxValue()).intValue(), 40);
    Assert.assertFalse(statsCollector.isSorted());
}
Also used : AbstractColumnStatisticsCollector(com.linkedin.pinot.core.segment.creator.AbstractColumnStatisticsCollector) FieldSpec(com.linkedin.pinot.common.data.FieldSpec) DimensionFieldSpec(com.linkedin.pinot.common.data.DimensionFieldSpec) DimensionFieldSpec(com.linkedin.pinot.common.data.DimensionFieldSpec) FloatColumnPreIndexStatsCollector(com.linkedin.pinot.core.segment.creator.impl.stats.FloatColumnPreIndexStatsCollector) Test(org.testng.annotations.Test)

Aggregations

AbstractColumnStatisticsCollector (com.linkedin.pinot.core.segment.creator.AbstractColumnStatisticsCollector)7 DimensionFieldSpec (com.linkedin.pinot.common.data.DimensionFieldSpec)6 FieldSpec (com.linkedin.pinot.common.data.FieldSpec)6 Test (org.testng.annotations.Test)6 StringColumnPreIndexStatsCollector (com.linkedin.pinot.core.segment.creator.impl.stats.StringColumnPreIndexStatsCollector)2 HashBiMap (com.google.common.collect.HashBiMap)1 DoubleColumnPreIndexStatsCollector (com.linkedin.pinot.core.segment.creator.impl.stats.DoubleColumnPreIndexStatsCollector)1 FloatColumnPreIndexStatsCollector (com.linkedin.pinot.core.segment.creator.impl.stats.FloatColumnPreIndexStatsCollector)1 IntColumnPreIndexStatsCollector (com.linkedin.pinot.core.segment.creator.impl.stats.IntColumnPreIndexStatsCollector)1 LongColumnPreIndexStatsCollector (com.linkedin.pinot.core.segment.creator.impl.stats.LongColumnPreIndexStatsCollector)1 File (java.io.File)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 MutableLong (org.apache.commons.lang.mutable.MutableLong)1