Search in sources :

Example 6 with LazyAccumuloRow

use of org.apache.hadoop.hive.accumulo.LazyAccumuloRow in project hive by apache.

the class AccumuloSerDe method initialize.

public void initialize(Configuration conf, Properties properties) throws SerDeException {
    accumuloSerDeParameters = new AccumuloSerDeParameters(conf, properties, getClass().getName());
    final LazySerDeParameters serDeParams = accumuloSerDeParameters.getSerDeParameters();
    final List<ColumnMapping> mappings = accumuloSerDeParameters.getColumnMappings();
    final List<TypeInfo> columnTypes = accumuloSerDeParameters.getHiveColumnTypes();
    final AccumuloRowIdFactory factory = accumuloSerDeParameters.getRowIdFactory();
    ArrayList<ObjectInspector> columnObjectInspectors = getColumnObjectInspectors(columnTypes, serDeParams, mappings, factory);
    cachedObjectInspector = LazyObjectInspectorFactory.getLazySimpleStructObjectInspector(serDeParams.getColumnNames(), columnObjectInspectors, serDeParams.getSeparators()[0], serDeParams.getNullSequence(), serDeParams.isLastColumnTakesRest(), serDeParams.isEscaped(), serDeParams.getEscapeChar());
    cachedRow = new LazyAccumuloRow((LazySimpleStructObjectInspector) cachedObjectInspector);
    serializer = new AccumuloRowSerializer(accumuloSerDeParameters.getRowIdOffset(), accumuloSerDeParameters.getSerDeParameters(), accumuloSerDeParameters.getColumnMappings(), accumuloSerDeParameters.getTableVisibilityLabel(), accumuloSerDeParameters.getRowIdFactory());
    if (log.isInfoEnabled()) {
        log.info("Initialized with {} type: {}", accumuloSerDeParameters.getSerDeParameters().getColumnNames(), accumuloSerDeParameters.getSerDeParameters().getColumnTypes());
    }
}
Also used : LazySimpleStructObjectInspector(org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector) ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) LazySimpleStructObjectInspector(org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector) LazySerDeParameters(org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters) LazyAccumuloRow(org.apache.hadoop.hive.accumulo.LazyAccumuloRow) TypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo) HiveAccumuloRowIdColumnMapping(org.apache.hadoop.hive.accumulo.columns.HiveAccumuloRowIdColumnMapping) ColumnMapping(org.apache.hadoop.hive.accumulo.columns.ColumnMapping)

Example 7 with LazyAccumuloRow

use of org.apache.hadoop.hive.accumulo.LazyAccumuloRow in project hive by apache.

the class TestAccumuloSerDe method invalidColMapping.

@Test(expected = InvalidColumnMappingException.class)
public void invalidColMapping() throws Exception {
    Properties properties = new Properties();
    Configuration conf = new Configuration();
    properties.setProperty(AccumuloSerDeParameters.COLUMN_MAPPINGS, "cf,cf:f2,cf:f3");
    properties.setProperty(serdeConstants.LIST_COLUMNS, "field2,field3,field4");
    serde.initialize(conf, properties);
    AccumuloHiveRow row = new AccumuloHiveRow();
    row.setRowId("r1");
    Object obj = serde.deserialize(row);
    assertTrue(obj instanceof LazyAccumuloRow);
    LazyAccumuloRow lazyRow = (LazyAccumuloRow) obj;
    lazyRow.getField(0);
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) LazyAccumuloRow(org.apache.hadoop.hive.accumulo.LazyAccumuloRow) Properties(java.util.Properties) AccumuloHiveRow(org.apache.hadoop.hive.accumulo.AccumuloHiveRow) Test(org.junit.Test)

Aggregations

LazyAccumuloRow (org.apache.hadoop.hive.accumulo.LazyAccumuloRow)7 Properties (java.util.Properties)6 Configuration (org.apache.hadoop.conf.Configuration)6 AccumuloHiveRow (org.apache.hadoop.hive.accumulo.AccumuloHiveRow)6 Test (org.junit.Test)6 LazyString (org.apache.hadoop.hive.serde2.lazy.LazyString)5 LazyMap (org.apache.hadoop.hive.serde2.lazy.LazyMap)2 LazySerDeParameters (org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters)2 LazyStruct (org.apache.hadoop.hive.serde2.lazy.LazyStruct)2 LazySimpleStructObjectInspector (org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector)2 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)2 TypeInfo (org.apache.hadoop.hive.serde2.typeinfo.TypeInfo)2 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 ColumnUpdate (org.apache.accumulo.core.data.ColumnUpdate)1 Mutation (org.apache.accumulo.core.data.Mutation)1 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)1 ColumnMapping (org.apache.hadoop.hive.accumulo.columns.ColumnMapping)1 HiveAccumuloRowIdColumnMapping (org.apache.hadoop.hive.accumulo.columns.HiveAccumuloRowIdColumnMapping)1 ByteArrayRef (org.apache.hadoop.hive.serde2.lazy.ByteArrayRef)1