Search in sources :

Example 31 with Mutation

use of org.apache.accumulo.core.data.Mutation in project gora by apache.

the class AccumuloStore method put.

@Override
public void put(K key, T val) {
    try {
        Mutation m = new Mutation(new Text(toBytes(key)));
        Schema schema = val.getSchema();
        List<Field> fields = schema.getFields();
        int count = 0;
        for (int i = 0; i < fields.size(); i++) {
            if (!val.isDirty(i)) {
                continue;
            }
            Field field = fields.get(i);
            Object o = val.get(field.pos());
            Pair<Text, Text> col = mapping.fieldMap.get(field.name());
            if (col == null) {
                throw new GoraException("Please define the gora to accumulo mapping for field " + field.name());
            }
            switch(field.schema().getType()) {
                case MAP:
                    count = putMap(m, count, field.schema().getValueType(), o, col, field.name());
                    break;
                case ARRAY:
                    count = putArray(m, count, o, col, field.name());
                    break;
                case // default value of null acts like union with null
                UNION:
                    Schema effectiveSchema = field.schema().getTypes().get(firstNotNullSchemaTypeIndex(field.schema()));
                    // map and array need to compute qualifier
                    if (effectiveSchema.getType() == Type.ARRAY) {
                        count = putArray(m, count, o, col, field.name());
                        break;
                    } else if (effectiveSchema.getType() == Type.MAP) {
                        count = putMap(m, count, effectiveSchema.getValueType(), o, col, field.name());
                        break;
                    }
                // continue like a regular top-level union
                case RECORD:
                    final SpecificDatumWriter<Object> writer = new SpecificDatumWriter<>(field.schema());
                    final byte[] byteData = IOUtils.serialize(writer, o);
                    m.put(col.getFirst(), col.getSecond(), new Value(byteData));
                    count++;
                    break;
                default:
                    m.put(col.getFirst(), col.getSecond(), new Value(toBytes(o)));
                    count++;
            }
        }
        if (count > 0)
            try {
                getBatchWriter().addMutation(m);
            } catch (MutationsRejectedException e) {
                LOG.error(e.getMessage(), e);
            }
    } catch (IOException e) {
        LOG.error(e.getMessage(), e);
    }
}
Also used : Schema(org.apache.avro.Schema) Text(org.apache.hadoop.io.Text) IOException(java.io.IOException) SpecificDatumWriter(org.apache.avro.specific.SpecificDatumWriter) Field(org.apache.avro.Schema.Field) GoraException(org.apache.gora.util.GoraException) Value(org.apache.accumulo.core.data.Value) Mutation(org.apache.accumulo.core.data.Mutation) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Aggregations

Mutation (org.apache.accumulo.core.data.Mutation)31 Value (org.apache.accumulo.core.data.Value)21 Key (org.apache.accumulo.core.data.Key)17 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)16 Authorizations (org.apache.accumulo.core.security.Authorizations)15 BatchWriter (org.apache.accumulo.core.client.BatchWriter)13 Entry (java.util.Map.Entry)12 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)12 Test (org.junit.Test)11 ByteArrayRef (org.apache.hadoop.hive.serde2.lazy.ByteArrayRef)10 Text (org.apache.hadoop.io.Text)10 Properties (java.util.Properties)8 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)8 Configuration (org.apache.hadoop.conf.Configuration)8 LazySerDeParameters (org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters)8 LazyStruct (org.apache.hadoop.hive.serde2.lazy.LazyStruct)8 LazySimpleStructObjectInspector (org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector)8 TypeInfo (org.apache.hadoop.hive.serde2.typeinfo.TypeInfo)8 Element (uk.gov.gchq.gaffer.data.element.Element)8 AccumuloException (org.apache.accumulo.core.client.AccumuloException)7