Search in sources :

Example 1 with SequenceFileInputFilter

use of org.apache.hadoop.mapred.SequenceFileInputFilter in project pxf by greenplum-db.

the class ProfileFactoryTest method get.

@Test
public void get() {
    // if user specified vectorized ORC, no matter what the input format is, the profile (as in parameter) should be used
    String profileName = ProfileFactory.get(new TextInputFormat(), false, "HiveVectorizedORC");
    assertEquals("HiveVectorizedORC", profileName);
    profileName = ProfileFactory.get(new TextInputFormat(), false, "hivevectorizedorc");
    assertEquals("hivevectorizedorc", profileName);
    // For TextInputFormat when table has no complex types, HiveText profile should be used
    profileName = ProfileFactory.get(new TextInputFormat(), false);
    assertEquals("hive:text", profileName);
    // For TextInputFormat when table has complex types, Hive profile should be used, HiveText doesn't support complex types yet
    profileName = ProfileFactory.get(new TextInputFormat(), true);
    assertEquals("hive", profileName);
    // For RCFileInputFormat when table has complex types, HiveRC profile should be used
    profileName = ProfileFactory.get(new RCFileInputFormat(), true);
    assertEquals("hive:rc", profileName);
    // For RCFileInputFormat when table has no complex types, HiveRC profile should be used
    profileName = ProfileFactory.get(new RCFileInputFormat(), false);
    assertEquals("hive:rc", profileName);
    // For OrcInputFormat when table has complex types, HiveORC profile should be used
    profileName = ProfileFactory.get(new OrcInputFormat(), true);
    assertEquals("hive:orc", profileName);
    // For OrcInputFormat when table has no complex types, HiveORC profile should be used
    profileName = ProfileFactory.get(new OrcInputFormat(), false);
    assertEquals("hive:orc", profileName);
    // For other formats Hive profile should be used
    profileName = ProfileFactory.get(new SequenceFileInputFilter(), false);
    assertEquals("hive", profileName);
}
Also used : TextInputFormat(org.apache.hadoop.mapred.TextInputFormat) RCFileInputFormat(org.apache.hadoop.hive.ql.io.RCFileInputFormat) OrcInputFormat(org.apache.hadoop.hive.ql.io.orc.OrcInputFormat) SequenceFileInputFilter(org.apache.hadoop.mapred.SequenceFileInputFilter) Test(org.junit.jupiter.api.Test)

Aggregations

RCFileInputFormat (org.apache.hadoop.hive.ql.io.RCFileInputFormat)1 OrcInputFormat (org.apache.hadoop.hive.ql.io.orc.OrcInputFormat)1 SequenceFileInputFilter (org.apache.hadoop.mapred.SequenceFileInputFilter)1 TextInputFormat (org.apache.hadoop.mapred.TextInputFormat)1 Test (org.junit.jupiter.api.Test)1