Search in sources :

Example 71 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project YCSB by brianfrankcooper.

the class AccumuloClient method update.

@Override
public Status update(String table, String key, Map<String, ByteIterator> values) {
    BatchWriter bw = null;
    try {
        bw = getWriter(table);
    } catch (TableNotFoundException e) {
        System.err.println("Error opening batch writer to Accumulo table " + table);
        e.printStackTrace();
        return Status.ERROR;
    }
    Mutation mutInsert = new Mutation(key.getBytes(UTF_8));
    for (Map.Entry<String, ByteIterator> entry : values.entrySet()) {
        mutInsert.put(colFamBytes, entry.getKey().getBytes(UTF_8), entry.getValue().toArray());
    }
    try {
        bw.addMutation(mutInsert);
    } catch (MutationsRejectedException e) {
        System.err.println("Error performing update.");
        e.printStackTrace();
        return Status.ERROR;
    }
    return Status.BATCHED_OK;
}
Also used : TableNotFoundException(org.apache.accumulo.core.client.TableNotFoundException) ByteIterator(site.ycsb.ByteIterator) ByteArrayByteIterator(site.ycsb.ByteArrayByteIterator) BatchWriter(org.apache.accumulo.core.client.BatchWriter) Mutation(org.apache.accumulo.core.data.Mutation) HashMap(java.util.HashMap) Map(java.util.Map) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) SortedMap(java.util.SortedMap) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 72 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project YCSB by brianfrankcooper.

the class AccumuloClient method getWriter.

/**
 * Called when the user specifies a table that isn't the same as the existing
 * table. Connect to it and if necessary, close our current connection.
 *
 * @param table
 *          The table to open.
 */
public BatchWriter getWriter(String table) throws TableNotFoundException {
    // tl;dr We're paying a cost for the ConcurrentHashMap here to deal with the DB api.
    // We know that YCSB is really only ever going to send us data for one table, so using
    // a concurrent data structure is overkill (especially in such a hot code path).
    // However, the impact seems to be relatively negligible in trivial local tests and it's
    // "more correct" WRT to the API.
    BatchWriter writer = writers.get(table);
    if (null == writer) {
        BatchWriter newWriter = createBatchWriter(table);
        BatchWriter oldWriter = writers.putIfAbsent(table, newWriter);
        // Someone beat us to creating a BatchWriter for this table, use their BatchWriters
        if (null != oldWriter) {
            try {
                // Make sure to clean up our new batchwriter!
                newWriter.close();
            } catch (MutationsRejectedException e) {
                throw new RuntimeException(e);
            }
            writer = oldWriter;
        } else {
            writer = newWriter;
        }
    }
    return writer;
}
Also used : BatchWriter(org.apache.accumulo.core.client.BatchWriter) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 73 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project apex-malhar by apache.

the class AbstractAccumuloOutputOperator method storeAggregate.

@Override
public void storeAggregate() {
    try {
        for (T tuple : tuples) {
            Mutation mutation = operationMutation(tuple);
            store.getBatchwriter().addMutation(mutation);
        }
        store.getBatchwriter().flush();
    } catch (MutationsRejectedException e) {
        logger.error("unable to write mutations", e);
        DTThrowable.rethrow(e);
    }
    tuples.clear();
}
Also used : Mutation(org.apache.accumulo.core.data.Mutation) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 74 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project apex-malhar by apache.

the class AccumuloWindowStore method storeCommittedWindowId.

@Override
public void storeCommittedWindowId(String appId, int operatorId, long windowId) {
    byte[] WindowIdBytes = toBytes(windowId);
    String columnKey = appId + "_" + operatorId + "_" + lastWindowColumnName;
    lastWindowColumnBytes = columnKey.getBytes();
    Mutation mutation = new Mutation(rowBytes);
    mutation.put(columnFamilyBytes, lastWindowColumnBytes, WindowIdBytes);
    try {
        batchwriter.addMutation(mutation);
        batchwriter.flush();
    } catch (MutationsRejectedException e) {
        logger.error("error getting committed window id", e);
        DTThrowable.rethrow(e);
    }
}
Also used : Mutation(org.apache.accumulo.core.data.Mutation) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 75 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project presto by prestodb.

the class AccumuloPageSink method appendPage.

@Override
public CompletableFuture<?> appendPage(Page page) {
    // For each position within the page, i.e. row
    for (int position = 0; position < page.getPositionCount(); ++position) {
        Row row = new Row();
        // For each channel within the page, i.e. column
        for (int channel = 0; channel < page.getChannelCount(); ++channel) {
            // Get the type for this channel
            Type type = columns.get(channel).getType();
            // Read the value from the page and append the field to the row
            row.addField(TypeUtils.readNativeValue(type, page.getBlock(channel), position), type);
        }
        try {
            // Convert row to a Mutation, writing and indexing it
            Mutation mutation = toMutation(row, rowIdOrdinal, columns, serializer);
            writer.addMutation(mutation);
            if (indexer.isPresent()) {
                indexer.get().index(mutation);
            }
            ++numRows;
        } catch (MutationsRejectedException e) {
            throw new PrestoException(UNEXPECTED_ACCUMULO_ERROR, "Mutation rejected by server", e);
        }
        // TODO Fix arbitrary flush every 100k rows
        if (numRows % 100_000 == 0) {
            flush();
        }
    }
    return NOT_BLOCKED;
}
Also used : Type(com.facebook.presto.common.type.Type) VarcharType(com.facebook.presto.common.type.VarcharType) PrestoException(com.facebook.presto.spi.PrestoException) Row(com.facebook.presto.accumulo.model.Row) Mutation(org.apache.accumulo.core.data.Mutation) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Aggregations

MutationsRejectedException (org.apache.accumulo.core.client.MutationsRejectedException)91 Mutation (org.apache.accumulo.core.data.Mutation)62 BatchWriter (org.apache.accumulo.core.client.BatchWriter)52 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)31 Text (org.apache.hadoop.io.Text)31 Value (org.apache.accumulo.core.data.Value)30 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)23 Key (org.apache.accumulo.core.data.Key)21 IOException (java.io.IOException)20 HashMap (java.util.HashMap)15 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)13 AccumuloException (org.apache.accumulo.core.client.AccumuloException)12 ConstraintViolationSummary (org.apache.accumulo.core.data.ConstraintViolationSummary)10 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)10 ArrayList (java.util.ArrayList)9 Scanner (org.apache.accumulo.core.client.Scanner)9 Set (java.util.Set)8 Test (org.junit.Test)8 List (java.util.List)7 Entry (java.util.Map.Entry)7