Search in sources :

Example 6 with FileLineReferenceResolver

use of io.crate.expression.reference.file.FileLineReferenceResolver in project crate by crate.

the class JsonReaderBenchmark method measureFileReadingIteratorForJson.

@Benchmark()
public void measureFileReadingIteratorForJson(Blackhole blackhole) {
    Reference raw = createReference("_raw", DataTypes.STRING);
    InputFactory.Context<LineCollectorExpression<?>> ctx = inputFactory.ctxForRefs(txnCtx, FileLineReferenceResolver::getImplementation);
    List<Input<?>> inputs = Collections.singletonList(ctx.add(raw));
    BatchIterator<Row> batchIterator = FileReadingIterator.newInstance(Collections.singletonList(fileUri), inputs, ctx.expressions(), null, Map.of(LocalFsFileInputFactory.NAME, new LocalFsFileInputFactory()), false, 1, 0, CopyFromParserProperties.DEFAULT, JSON, Settings.EMPTY);
    while (batchIterator.moveNext()) {
        blackhole.consume(batchIterator.currentElement().get(0));
    }
}
Also used : LocalFsFileInputFactory(io.crate.execution.engine.collect.files.LocalFsFileInputFactory) InputFactory(io.crate.expression.InputFactory) Input(io.crate.data.Input) LineCollectorExpression(io.crate.execution.engine.collect.files.LineCollectorExpression) LocalFsFileInputFactory(io.crate.execution.engine.collect.files.LocalFsFileInputFactory) FileLineReferenceResolver(io.crate.expression.reference.file.FileLineReferenceResolver) Reference(io.crate.metadata.Reference) Row(io.crate.data.Row) Benchmark(org.openjdk.jmh.annotations.Benchmark)

Aggregations

InputFactory (io.crate.expression.InputFactory)6 FileLineReferenceResolver (io.crate.expression.reference.file.FileLineReferenceResolver)6 Input (io.crate.data.Input)5 Reference (io.crate.metadata.Reference)5 LineCollectorExpression (io.crate.execution.engine.collect.files.LineCollectorExpression)4 TestingHelpers.createReference (io.crate.testing.TestingHelpers.createReference)3 Row (io.crate.data.Row)2 FileInputFactory (io.crate.execution.engine.collect.files.FileInputFactory)2 LocalFsFileInputFactory (io.crate.execution.engine.collect.files.LocalFsFileInputFactory)2 ArrayList (java.util.ArrayList)2 Benchmark (org.openjdk.jmh.annotations.Benchmark)2 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 AmazonS3Client (com.amazonaws.services.s3.AmazonS3Client)1 ObjectListing (com.amazonaws.services.s3.model.ObjectListing)1 S3Object (com.amazonaws.services.s3.model.S3Object)1 S3ObjectSummary (com.amazonaws.services.s3.model.S3ObjectSummary)1 S3ClientHelper (io.crate.copy.s3.common.S3ClientHelper)1 FileUriCollectPhase (io.crate.execution.dsl.phases.FileUriCollectPhase)1 FileInput (io.crate.execution.engine.collect.files.FileInput)1 IOException (java.io.IOException)1