Examples with TupleGenerator org.apache.hyracks.storage.am.common.datagen.TupleGenerator used on opensource projects

Example 6 with TupleGenerator

use of org.apache.hyracks.storage.am.common.datagen.TupleGenerator in project asterixdb by apache.

the class LSMInvertedIndexTestUtils method createStringDocumentTupleGen.

public static TupleGenerator createStringDocumentTupleGen(Random rnd) throws IOException {
    IFieldValueGenerator[] fieldGens = new IFieldValueGenerator[2];
    fieldGens[0] = new DocumentStringFieldValueGenerator(2, 10, 10000, rnd);
    fieldGens[1] = new SortedIntegerFieldValueGenerator(0);
    ISerializerDeserializer[] fieldSerdes = new ISerializerDeserializer[] { new UTF8StringSerializerDeserializer(), IntegerSerializerDeserializer.INSTANCE };
    TupleGenerator tupleGen = new TupleGenerator(fieldGens, fieldSerdes, 0);
    return tupleGen;
}

Example 7 with TupleGenerator

use of org.apache.hyracks.storage.am.common.datagen.TupleGenerator in project asterixdb by apache.

the class LSMInvertedIndexTestUtils method createPersonNamesTupleGen.

public static TupleGenerator createPersonNamesTupleGen(Random rnd) throws IOException {
    IFieldValueGenerator[] fieldGens = new IFieldValueGenerator[2];
    fieldGens[0] = new PersonNameFieldValueGenerator(rnd, 0.5f);
    fieldGens[1] = new SortedIntegerFieldValueGenerator(0);
    ISerializerDeserializer[] fieldSerdes = new ISerializerDeserializer[] { new UTF8StringSerializerDeserializer(), IntegerSerializerDeserializer.INSTANCE };
    TupleGenerator tupleGen = new TupleGenerator(fieldGens, fieldSerdes, 0);
    return tupleGen;
}

Example 8 with TupleGenerator

use of org.apache.hyracks.storage.am.common.datagen.TupleGenerator in project asterixdb by apache.

the class LSMInvertedIndexMultiThreadTest method wordTokensInvIndexTest.

@Test
public void wordTokensInvIndexTest() throws IOException, InterruptedException {
    String dataMsg = "Documents";
    int[] numThreads = new int[] { REGULAR_NUM_THREADS, EXCESSIVE_NUM_THREADS };
    for (int i = 0; i < numThreads.length; i++) {
        for (TestWorkloadConf conf : workloadConfs) {
            setUp();
            LSMInvertedIndexTestContext testCtx = LSMInvertedIndexTestUtils.createWordInvIndexTestContext(harness, getIndexType());
            TupleGenerator tupleGen = LSMInvertedIndexTestUtils.createStringDocumentTupleGen(harness.getRandom());
            runTest(testCtx, tupleGen, numThreads[i], conf, dataMsg);
            tearDown();
        }
    }
}

Example 9 with TupleGenerator

use of org.apache.hyracks.storage.am.common.datagen.TupleGenerator in project asterixdb by apache.

the class LSMInvertedIndexMultiThreadTest method hashedNGramTokensInvIndexTest.

@Test
public void hashedNGramTokensInvIndexTest() throws IOException, InterruptedException {
    String dataMsg = "Person Names";
    int[] numThreads = new int[] { REGULAR_NUM_THREADS, EXCESSIVE_NUM_THREADS };
    for (int i = 0; i < numThreads.length; i++) {
        for (TestWorkloadConf conf : workloadConfs) {
            setUp();
            LSMInvertedIndexTestContext testCtx = LSMInvertedIndexTestUtils.createHashedNGramInvIndexTestContext(harness, getIndexType());
            TupleGenerator tupleGen = LSMInvertedIndexTestUtils.createPersonNamesTupleGen(harness.getRandom());
            runTest(testCtx, tupleGen, numThreads[i], conf, dataMsg);
            tearDown();
        }
    }
}

Example 10 with TupleGenerator

use of org.apache.hyracks.storage.am.common.datagen.TupleGenerator in project asterixdb by apache.

the class AbstractInvertedIndexDeleteTest method hashedNGramTokensInvIndexTest.

@Test
public void hashedNGramTokensInvIndexTest() throws IOException {
    LSMInvertedIndexTestContext testCtx = LSMInvertedIndexTestUtils.createHashedNGramInvIndexTestContext(harness, invIndexType);
    TupleGenerator tupleGen = LSMInvertedIndexTestUtils.createPersonNamesTupleGen(harness.getRandom());
    runTest(testCtx, tupleGen);
}