Examples with Schema org.apache.arrow.vector.types.pojo.Schema used on opensource projects

Example 6 with Schema

use of org.apache.arrow.vector.types.pojo.Schema in project parquet-mr by apache.

the class TestSchemaConverter method testArrowTimeMicrosecondToParquet.

@Test
public void testArrowTimeMicrosecondToParquet() {
    MessageType expected = converter.fromArrow(new Schema(asList(field("a", new ArrowType.Time(TimeUnit.MICROSECOND, 64))))).getParquetSchema();
    Assert.assertEquals(expected, Types.buildMessage().addField(Types.optional(INT64).as(timeType(false, MICROS)).named("a")).named("root"));
}

Example 7 with Schema

use of org.apache.arrow.vector.types.pojo.Schema in project beam by apache.

the class BigQueryStorageArrowReader method processReadRowsResponse.

@Override
public void processReadRowsResponse(ReadRowsResponse readRowsResponse) throws IOException {
    com.google.cloud.bigquery.storage.v1.ArrowRecordBatch recordBatch = readRowsResponse.getArrowRecordBatch();
    rowCount = recordBatch.getRowCount();
    this.alloc = new RootAllocator(Long.MAX_VALUE);
    InputStream input = protoSchema.getSerializedSchema().newInput();
    Schema arrowSchema = ArrowConversion.arrowSchemaFromInput(input);
    this.recordBatchIterator = ArrowConversion.rowsFromSerializedRecordBatch(arrowSchema, recordBatch.getSerializedRecordBatch().newInput(), this.alloc);
}

Example 8 with Schema

use of org.apache.arrow.vector.types.pojo.Schema in project parquet-mr by apache.

the class TestSchemaConverter method testSupportedParquetToArrow.

@Test
public void testSupportedParquetToArrow() throws IOException {
    Schema arrow = converter.fromParquet(supportedTypesParquetSchema).getArrowSchema();
    assertEquals(supportedTypesArrowSchema, arrow);
}

Example 9 with Schema

use of org.apache.arrow.vector.types.pojo.Schema in project parquet-mr by apache.

the class TestSchemaConverter method testParquetInt96ToArrowBinary.

@Test
public void testParquetInt96ToArrowBinary() {
    MessageType parquet = Types.buildMessage().addField(Types.optional(INT96).named("a")).named("root");
    Schema expected = new Schema(asList(field("a", new ArrowType.Binary())));
    Assert.assertEquals(expected, converter.fromParquet(parquet).getArrowSchema());
}

Example 10 with Schema

use of org.apache.arrow.vector.types.pojo.Schema in project parquet-mr by apache.

the class TestSchemaConverter method testParquetInt96ToArrowTimestamp.

@Test
public void testParquetInt96ToArrowTimestamp() {
    final SchemaConverter converterInt96ToTimestamp = new SchemaConverter(true);
    MessageType parquet = Types.buildMessage().addField(Types.optional(INT96).named("a")).named("root");
    Schema expected = new Schema(asList(field("a", new ArrowType.Timestamp(TimeUnit.NANOSECOND, null))));
    Assert.assertEquals(expected, converterInt96ToTimestamp.fromParquet(parquet).getArrowSchema());
}