Examples with LineCollectorExpression io.crate.operation.collect.files.LineCollectorExpression used on opensource projects

Example 1 with LineCollectorExpression

use of io.crate.operation.collect.files.LineCollectorExpression in project crate by crate.

the class FileCollectSource method getCollector.

@Override
public CrateCollector getCollector(CollectPhase collectPhase, BatchConsumer consumer, JobCollectContext jobCollectContext) {
    FileUriCollectPhase fileUriCollectPhase = (FileUriCollectPhase) collectPhase;
    InputFactory.Context<LineCollectorExpression<?>> ctx = inputFactory.ctxForRefs(FileLineReferenceResolver::getImplementation);
    ctx.add(collectPhase.toCollect());
    String[] readers = fileUriCollectPhase.nodeIds().toArray(new String[fileUriCollectPhase.nodeIds().size()]);
    Arrays.sort(readers);
    List<String> fileUris;
    fileUris = targetUriToStringList(fileUriCollectPhase.targetUri());
    BatchIterator fileReadingIterator = FileReadingIterator.newInstance(fileUris, ctx.topLevelInputs(), ctx.expressions(), fileUriCollectPhase.compression(), fileInputFactoryMap, fileUriCollectPhase.sharedStorage(), readers.length, Arrays.binarySearch(readers, clusterService.state().nodes().getLocalNodeId()));
    return BatchIteratorCollectorBridge.newInstance(fileReadingIterator, consumer);
}

Example 2 with LineCollectorExpression

use of io.crate.operation.collect.files.LineCollectorExpression in project crate by crate.

the class FileLineReferenceResolver method getImplementation.

public static LineCollectorExpression<?> getImplementation(Reference refInfo) {
    ColumnIdent columnIdent = refInfo.ident().columnIdent();
    Supplier<LineCollectorExpression<?>> supplier = EXPRESSION_BUILDER.get(columnIdent.name());
    if (supplier == null) {
        return new ColumnExtractingLineExpression(columnIdent);
    }
    return supplier.get();
}