Search in sources :

Example 1 with Phone

use of org.apache.parquet.thrift.test.Phone in project parquet-mr by apache.

the class ParquetScroogeSchemeTest method testNestedReadingInScrooge.

@Test
public void testNestedReadingInScrooge() throws Exception {
    Map<String, org.apache.parquet.thrift.test.Phone> phoneMap = new HashMap<String, Phone>();
    phoneMap.put("key1", new org.apache.parquet.thrift.test.Phone("111", "222"));
    org.apache.parquet.thrift.test.TestPersonWithAllInformation toWrite = new org.apache.parquet.thrift.test.TestPersonWithAllInformation(new org.apache.parquet.thrift.test.Name("first"), new Address("my_street", "my_zip"), phoneMap);
    toWrite.setInfo("my_info");
    String expected = "TestPersonWithAllInformation(Name(first,None),None,Address(my_street,my_zip),None,Some(my_info),Map(key1 -> Phone(111,222)),None,None)\n";
    verifyScroogeRead(thriftRecords(toWrite), TestPersonWithAllInformation.class, expected, "**");
    String expectedProjected = "TestPersonWithAllInformation(Name(first,None),None,Address(my_street,my_zip),None,Some(my_info),Map(),None,None)\n";
    verifyScroogeRead(thriftRecords(toWrite), TestPersonWithAllInformation.class, expectedProjected, "address/*;info;name/first_name");
}
Also used : Address(org.apache.parquet.thrift.test.Address) HashMap(java.util.HashMap) TestPersonWithAllInformation(org.apache.parquet.scrooge.test.TestPersonWithAllInformation) Phone(org.apache.parquet.thrift.test.Phone) Phone(org.apache.parquet.thrift.test.Phone) Test(org.junit.Test)

Example 2 with Phone

use of org.apache.parquet.thrift.test.Phone in project parquet-mr by apache.

the class TestProtocolReadToWrite method TestExtraFieldWhenFieldIndexIsNotStartFromZero.

@Test
public void TestExtraFieldWhenFieldIndexIsNotStartFromZero() throws Exception {
    CountingErrorHandler countingHandler = new CountingErrorHandler() {

        @Override
        public void handleFieldIgnored(TField field) {
            assertEquals(3, field.id);
            fieldIgnoredCount++;
        }
    };
    BufferedProtocolReadToWrite structForRead = new BufferedProtocolReadToWrite(ThriftSchemaConverter.toStructType(StructWithIndexStartsFrom4.class), countingHandler);
    // Data has an extra field of type struct
    final ByteArrayOutputStream in = new ByteArrayOutputStream();
    StructWithExtraField dataWithNewExtraField = new StructWithExtraField(new Phone("111", "222"), new Phone("333", "444"));
    dataWithNewExtraField.write(protocol(in));
    // read using the schema that doesn't have the extra field
    final ByteArrayOutputStream out = new ByteArrayOutputStream();
    structForRead.readOne(protocol(new ByteArrayInputStream(in.toByteArray())), protocol(out));
    assertEquals(1, countingHandler.recordCountOfMissingFields);
    assertEquals(1, countingHandler.fieldIgnoredCount);
}
Also used : TField(org.apache.thrift.protocol.TField) ByteArrayInputStream(java.io.ByteArrayInputStream) Phone(org.apache.parquet.thrift.test.Phone) StructWithIndexStartsFrom4(org.apache.parquet.thrift.test.StructWithIndexStartsFrom4) StructWithExtraField(org.apache.parquet.thrift.test.StructWithExtraField) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Test(org.junit.Test)

Aggregations

Phone (org.apache.parquet.thrift.test.Phone)2 Test (org.junit.Test)2 ByteArrayInputStream (java.io.ByteArrayInputStream)1 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 HashMap (java.util.HashMap)1 TestPersonWithAllInformation (org.apache.parquet.scrooge.test.TestPersonWithAllInformation)1 Address (org.apache.parquet.thrift.test.Address)1 StructWithExtraField (org.apache.parquet.thrift.test.StructWithExtraField)1 StructWithIndexStartsFrom4 (org.apache.parquet.thrift.test.StructWithIndexStartsFrom4)1 TField (org.apache.thrift.protocol.TField)1