Search in sources :

Example 6 with UniqueNamer

use of com.bakdata.conquery.models.query.resultinfo.UniqueNamer in project conquery by bakdata.

the class ArrowRenderer method renderToStream.

public static void renderToStream(Function<VectorSchemaRoot, ArrowWriter> writerProducer, PrintSettings printSettings, int batchSize, List<ResultInfo> idHeaders, List<ResultInfo> resultInfo, Stream<EntityResult> results) throws IOException {
    // Combine id and value Fields to one vector to build a schema
    final UniqueNamer uniqNamer = new UniqueNamer(printSettings);
    final List<Field> idFields = generateFields(idHeaders, uniqNamer);
    List<Field> fields = new ArrayList<>(idFields);
    fields.addAll(generateFields(resultInfo, uniqNamer));
    VectorSchemaRoot root = VectorSchemaRoot.create(new Schema(fields, null), ROOT_ALLOCATOR);
    // Build separate pipelines for id and value, as they have different sources but the same target
    RowConsumer[] idWriters = generateWriterPipeline(root, 0, idHeaders.size(), printSettings, null);
    RowConsumer[] valueWriter = generateWriterPipeline(root, idHeaders.size(), resultInfo.size(), printSettings, resultInfo);
    // Write the data
    try (ArrowWriter writer = writerProducer.apply(root)) {
        write(writer, root, idWriters, valueWriter, printSettings.getIdMapper(), results, batchSize);
    }
}
Also used : Field(org.apache.arrow.vector.types.pojo.Field) VectorSchemaRoot(org.apache.arrow.vector.VectorSchemaRoot) Schema(org.apache.arrow.vector.types.pojo.Schema) ArrowWriter(org.apache.arrow.vector.ipc.ArrowWriter) UniqueNamer(com.bakdata.conquery.models.query.resultinfo.UniqueNamer)

Example 7 with UniqueNamer

use of com.bakdata.conquery.models.query.resultinfo.UniqueNamer in project conquery by bakdata.

the class CsvLineStreamRenderer method toStream.

public Stream<String> toStream(List<ResultInfo> idHeaders, List<ResultInfo> infos, Stream<EntityResult> resultStream) {
    final UniqueNamer uniqNamer = new UniqueNamer(cfg);
    Stream.concat(idHeaders.stream(), infos.stream()).map(uniqNamer::getUniqueName).forEach(writer::addValue);
    return Stream.concat(Stream.of(writer.writeValuesToString()), createCSVBody(cfg, infos, resultStream));
}
Also used : UniqueNamer(com.bakdata.conquery.models.query.resultinfo.UniqueNamer)

Aggregations

UniqueNamer (com.bakdata.conquery.models.query.resultinfo.UniqueNamer)7 ResultInfo (com.bakdata.conquery.models.query.resultinfo.ResultInfo)2 CQConcept (com.bakdata.conquery.apiv1.query.concept.specific.CQConcept)1 ConqueryConfig (com.bakdata.conquery.models.config.ConqueryConfig)1 ConceptId (com.bakdata.conquery.models.identifiable.ids.specific.ConceptId)1 SelectResultInfo (com.bakdata.conquery.models.query.resultinfo.SelectResultInfo)1 SimpleResultInfo (com.bakdata.conquery.models.query.resultinfo.SimpleResultInfo)1 ArrayList (java.util.ArrayList)1 Locale (java.util.Locale)1 VectorSchemaRoot (org.apache.arrow.vector.VectorSchemaRoot)1 ArrowWriter (org.apache.arrow.vector.ipc.ArrowWriter)1 Field (org.apache.arrow.vector.types.pojo.Field)1 Schema (org.apache.arrow.vector.types.pojo.Schema)1 Cell (org.apache.poi.ss.usermodel.Cell)1 Row (org.apache.poi.ss.usermodel.Row)1 Test (org.junit.jupiter.api.Test)1 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)1 MethodSource (org.junit.jupiter.params.provider.MethodSource)1 CTTableColumn (org.openxmlformats.schemas.spreadsheetml.x2006.main.CTTableColumn)1 CTTableColumns (org.openxmlformats.schemas.spreadsheetml.x2006.main.CTTableColumns)1