Search in sources :

Example 1 with AccumuloPredicateHandler

use of org.apache.hadoop.hive.accumulo.predicate.AccumuloPredicateHandler in project hive by apache.

the class TestHiveAccumuloTableInputFormat method testIteratorNotInSplitsCompensation.

@Test
public void testIteratorNotInSplitsCompensation() throws Exception {
    FileInputFormat.addInputPath(conf, new Path("unused"));
    InputSplit[] splits = inputformat.getSplits(conf, 0);
    assertEquals(1, splits.length);
    InputSplit split = splits[0];
    IteratorSetting is = new IteratorSetting(1, PrimitiveComparisonFilter.FILTER_PREFIX + 1, PrimitiveComparisonFilter.class);
    is.addOption(PrimitiveComparisonFilter.P_COMPARE_CLASS, StringCompare.class.getName());
    is.addOption(PrimitiveComparisonFilter.COMPARE_OPT_CLASS, Equal.class.getName());
    is.addOption(PrimitiveComparisonFilter.CONST_VAL, new String(Base64.encodeBase64(new byte[] { '0' })));
    is.addOption(PrimitiveComparisonFilter.COLUMN, "cf:cq");
    // Mock out the predicate handler because it's just easier
    AccumuloPredicateHandler predicateHandler = Mockito.mock(AccumuloPredicateHandler.class);
    Mockito.when(predicateHandler.getIterators(Mockito.any(JobConf.class), Mockito.any(ColumnMapper.class))).thenReturn(Arrays.asList(is));
    // Set it on our inputformat
    inputformat.predicateHandler = predicateHandler;
    inputformat.getRecordReader(split, conf, null);
    // The code should account for the bug and update the iterators on the split
    List<IteratorSetting> settingsOnSplit = ((HiveAccumuloSplit) split).getSplit().getIterators();
    assertEquals(1, settingsOnSplit.size());
    assertEquals(is, settingsOnSplit.get(0));
}
Also used : Path(org.apache.hadoop.fs.Path) IteratorSetting(org.apache.accumulo.core.client.IteratorSetting) StringCompare(org.apache.hadoop.hive.accumulo.predicate.compare.StringCompare) GreaterThanOrEqual(org.apache.hadoop.hive.accumulo.predicate.compare.GreaterThanOrEqual) Equal(org.apache.hadoop.hive.accumulo.predicate.compare.Equal) AccumuloPredicateHandler(org.apache.hadoop.hive.accumulo.predicate.AccumuloPredicateHandler) InputSplit(org.apache.hadoop.mapred.InputSplit) JobConf(org.apache.hadoop.mapred.JobConf) ColumnMapper(org.apache.hadoop.hive.accumulo.columns.ColumnMapper) Test(org.junit.Test)

Aggregations

IteratorSetting (org.apache.accumulo.core.client.IteratorSetting)1 Path (org.apache.hadoop.fs.Path)1 ColumnMapper (org.apache.hadoop.hive.accumulo.columns.ColumnMapper)1 AccumuloPredicateHandler (org.apache.hadoop.hive.accumulo.predicate.AccumuloPredicateHandler)1 Equal (org.apache.hadoop.hive.accumulo.predicate.compare.Equal)1 GreaterThanOrEqual (org.apache.hadoop.hive.accumulo.predicate.compare.GreaterThanOrEqual)1 StringCompare (org.apache.hadoop.hive.accumulo.predicate.compare.StringCompare)1 InputSplit (org.apache.hadoop.mapred.InputSplit)1 JobConf (org.apache.hadoop.mapred.JobConf)1 Test (org.junit.Test)1