Search in sources :

Example 21 with BenchmarkDataGenerator

use of io.druid.benchmark.datagen.BenchmarkDataGenerator in project druid by druid-io.

the class BenchmarkDataGeneratorTest method testNormal.

@Test
public void testNormal() throws Exception {
    List<BenchmarkColumnSchema> schemas = new ArrayList<>();
    RowValueTracker tracker = new RowValueTracker();
    schemas.add(BenchmarkColumnSchema.makeNormal("dimA", ValueType.FLOAT, false, 1, null, 8.0, 1.0, false));
    schemas.add(BenchmarkColumnSchema.makeNormal("dimB", ValueType.STRING, false, 1, 0.50, 88.0, 2.0, false));
    BenchmarkDataGenerator dataGenerator = new BenchmarkDataGenerator(schemas, 9999, 0, 0, 1000.0);
    for (int i = 0; i < 100; i++) {
        InputRow row = dataGenerator.nextRow();
        //System.out.println("N-ROW: " + row);
        tracker.addRow(row);
    }
    tracker.printStuff();
}
Also used : ArrayList(java.util.ArrayList) BenchmarkDataGenerator(io.druid.benchmark.datagen.BenchmarkDataGenerator) InputRow(io.druid.data.input.InputRow) BenchmarkColumnSchema(io.druid.benchmark.datagen.BenchmarkColumnSchema) Test(org.junit.Test)

Example 22 with BenchmarkDataGenerator

use of io.druid.benchmark.datagen.BenchmarkDataGenerator in project druid by druid-io.

the class BenchmarkDataGeneratorTest method testIntervalBasedTimeGeneration.

@Test
public void testIntervalBasedTimeGeneration() throws Exception {
    List<BenchmarkColumnSchema> schemas = new ArrayList<>();
    schemas.add(BenchmarkColumnSchema.makeEnumeratedSequential("dimB", ValueType.STRING, false, 1, null, Arrays.<Object>asList("Hello", "World", "Foo", "Bar")));
    BenchmarkDataGenerator dataGenerator = new BenchmarkDataGenerator(schemas, 9999, new Interval(50000, 600000), 100);
    for (int i = 0; i < 100; i++) {
        InputRow row = dataGenerator.nextRow();
    //System.out.println("S-ROW: " + row);
    }
    BenchmarkDataGenerator dataGenerator2 = new BenchmarkDataGenerator(schemas, 9999, new Interval(50000, 50001), 100);
    for (int i = 0; i < 100; i++) {
        InputRow row = dataGenerator2.nextRow();
    //System.out.println("S2-ROW: " + row);
    }
}
Also used : ArrayList(java.util.ArrayList) BenchmarkDataGenerator(io.druid.benchmark.datagen.BenchmarkDataGenerator) InputRow(io.druid.data.input.InputRow) BenchmarkColumnSchema(io.druid.benchmark.datagen.BenchmarkColumnSchema) Interval(org.joda.time.Interval) Test(org.junit.Test)

Aggregations

BenchmarkDataGenerator (io.druid.benchmark.datagen.BenchmarkDataGenerator)22 InputRow (io.druid.data.input.InputRow)22 HyperUniquesSerde (io.druid.query.aggregation.hyperloglog.HyperUniquesSerde)14 Setup (org.openjdk.jmh.annotations.Setup)14 IndexSpec (io.druid.segment.IndexSpec)10 File (java.io.File)9 ArrayList (java.util.ArrayList)9 BenchmarkColumnSchema (io.druid.benchmark.datagen.BenchmarkColumnSchema)8 IncrementalIndex (io.druid.segment.incremental.IncrementalIndex)8 OnheapIncrementalIndex (io.druid.segment.incremental.OnheapIncrementalIndex)8 Test (org.junit.Test)8 QueryableIndex (io.druid.segment.QueryableIndex)7 StupidPool (io.druid.collections.StupidPool)4 OffheapBufferGenerator (io.druid.offheap.OffheapBufferGenerator)4 Interval (org.joda.time.Interval)3 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)2 SmileFactory (com.fasterxml.jackson.dataformat.smile.SmileFactory)2 BenchmarkSchemaInfo (io.druid.benchmark.datagen.BenchmarkSchemaInfo)2 BlockingPool (io.druid.collections.BlockingPool)2 DefaultObjectMapper (io.druid.jackson.DefaultObjectMapper)2