Search in sources :

Example 26 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project presto by prestodb.

the class Indexer method close.

/**
 * Flushes all remaining mutations via {@link Indexer#flush} and closes the index writer.
 */
@Override
public void close() {
    try {
        flush();
        indexWriter.close();
    } catch (MutationsRejectedException e) {
        throw new PrestoException(UNEXPECTED_ACCUMULO_ERROR, "Mutation was rejected by server on close", e);
    }
}
Also used : PrestoException(com.facebook.presto.spi.PrestoException) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 27 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project presto by prestodb.

the class Indexer method addIndexMutation.

private void addIndexMutation(ByteBuffer row, ByteBuffer family, ColumnVisibility visibility, byte[] qualifier) {
    // Create the mutation and add it to the batch writer
    Mutation indexMutation = new Mutation(row.array());
    indexMutation.put(family.array(), qualifier, visibility, EMPTY_BYTES);
    try {
        indexWriter.addMutation(indexMutation);
    } catch (MutationsRejectedException e) {
        throw new PrestoException(UNEXPECTED_ACCUMULO_ERROR, "Index mutation rejected by server", e);
    }
    // Increment the cardinality metrics for this value of index
    // metrics is a mapping of row ID to column family
    MetricsKey key = new MetricsKey(row, family, visibility);
    AtomicLong count = metrics.get(key);
    if (count == null) {
        count = new AtomicLong(0);
        metrics.put(key, count);
    }
    count.incrementAndGet();
}
Also used : AtomicLong(java.util.concurrent.atomic.AtomicLong) PrestoException(com.facebook.presto.spi.PrestoException) Mutation(org.apache.accumulo.core.data.Mutation) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 28 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project YCSB by brianfrankcooper.

the class AccumuloClient method cleanup.

@Override
public void cleanup() throws DBException {
    try {
        Iterator<BatchWriter> iterator = writers.values().iterator();
        while (iterator.hasNext()) {
            BatchWriter writer = iterator.next();
            writer.close();
            iterator.remove();
        }
    } catch (MutationsRejectedException e) {
        throw new DBException(e);
    }
}
Also used : DBException(site.ycsb.DBException) BatchWriter(org.apache.accumulo.core.client.BatchWriter) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 29 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project Gaffer by gchq.

the class AccumuloStore method insertGraphElements.

protected void insertGraphElements(final Iterable<? extends Element> elements) throws StoreException {
    // Create BatchWriter
    final BatchWriter writer = TableUtils.createBatchWriter(this);
    // too high a latency, etc.
    if (null != elements) {
        for (final Element element : elements) {
            final Pair<Key, Key> keys;
            try {
                keys = keyPackage.getKeyConverter().getKeysFromElement(element);
            } catch (final AccumuloElementConversionException e) {
                LOGGER.error(FAILED_TO_CREATE_AN_ACCUMULO_FROM_ELEMENT_OF_TYPE_WHEN_TRYING_TO_INSERT_ELEMENTS, "key", element.getGroup());
                continue;
            }
            final Value value;
            try {
                value = keyPackage.getKeyConverter().getValueFromElement(element);
            } catch (final AccumuloElementConversionException e) {
                LOGGER.error(FAILED_TO_CREATE_AN_ACCUMULO_FROM_ELEMENT_OF_TYPE_WHEN_TRYING_TO_INSERT_ELEMENTS, "value", element.getGroup());
                continue;
            }
            final Mutation m = new Mutation(keys.getFirst().getRow());
            m.put(keys.getFirst().getColumnFamily(), keys.getFirst().getColumnQualifier(), new ColumnVisibility(keys.getFirst().getColumnVisibility()), keys.getFirst().getTimestamp(), value);
            try {
                writer.addMutation(m);
            } catch (final MutationsRejectedException e) {
                LOGGER.error("Failed to create an accumulo key mutation");
                continue;
            }
            // If the GraphElement is an Edge then there will be 2 keys.
            if (null != keys.getSecond()) {
                final Mutation m2 = new Mutation(keys.getSecond().getRow());
                m2.put(keys.getSecond().getColumnFamily(), keys.getSecond().getColumnQualifier(), new ColumnVisibility(keys.getSecond().getColumnVisibility()), keys.getSecond().getTimestamp(), value);
                try {
                    writer.addMutation(m2);
                } catch (final MutationsRejectedException e) {
                    LOGGER.error("Failed to create an accumulo key mutation");
                }
            }
        }
    } else {
        throw new GafferRuntimeException("Could not find any elements to add to graph.", Status.BAD_REQUEST);
    }
    try {
        writer.close();
    } catch (final MutationsRejectedException e) {
        LOGGER.warn("Accumulo batch writer failed to close", e);
    }
}
Also used : Element(uk.gov.gchq.gaffer.data.element.Element) Value(org.apache.accumulo.core.data.Value) BatchWriter(org.apache.accumulo.core.client.BatchWriter) Mutation(org.apache.accumulo.core.data.Mutation) ColumnVisibility(org.apache.accumulo.core.security.ColumnVisibility) Key(org.apache.accumulo.core.data.Key) AccumuloElementConversionException(uk.gov.gchq.gaffer.accumulostore.key.exception.AccumuloElementConversionException) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException) GafferRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferRuntimeException)

Example 30 with MutationsRejectedException

use of org.apache.accumulo.core.client.MutationsRejectedException in project apex-malhar by apache.

the class AccumuloTestHelper method populateAccumulo.

public static void populateAccumulo() throws IOException {
    BatchWriterConfig config = new BatchWriterConfig();
    BatchWriter batchwriter = null;
    try {
        batchwriter = con.createBatchWriter("tab1", config);
    } catch (TableNotFoundException e) {
        logger.error("error in test helper");
        DTThrowable.rethrow(e);
    }
    try {
        for (int i = 0; i < 500; ++i) {
            String rowstr = "row" + i;
            Mutation mutation = new Mutation(rowstr.getBytes());
            for (int j = 0; j < 500; ++j) {
                String colstr = "col" + "-" + j;
                String valstr = "val" + "-" + i + "-" + j;
                mutation.put(colfam0_bytes, colstr.getBytes(), System.currentTimeMillis(), valstr.getBytes());
            }
            batchwriter.addMutation(mutation);
        }
        batchwriter.close();
    } catch (MutationsRejectedException e) {
        logger.error("error in test helper");
        DTThrowable.rethrow(e);
    }
}
Also used : TableNotFoundException(org.apache.accumulo.core.client.TableNotFoundException) BatchWriterConfig(org.apache.accumulo.core.client.BatchWriterConfig) BatchWriter(org.apache.accumulo.core.client.BatchWriter) Mutation(org.apache.accumulo.core.data.Mutation) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Aggregations

MutationsRejectedException (org.apache.accumulo.core.client.MutationsRejectedException)68 Mutation (org.apache.accumulo.core.data.Mutation)48 BatchWriter (org.apache.accumulo.core.client.BatchWriter)40 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)23 Value (org.apache.accumulo.core.data.Value)23 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)21 Text (org.apache.hadoop.io.Text)20 Key (org.apache.accumulo.core.data.Key)13 IOException (java.io.IOException)12 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)12 AccumuloException (org.apache.accumulo.core.client.AccumuloException)11 HashMap (java.util.HashMap)10 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)9 ArrayList (java.util.ArrayList)8 Test (org.junit.Test)8 Entry (java.util.Map.Entry)6 TableExistsException (org.apache.accumulo.core.client.TableExistsException)6 ConditionalMutation (org.apache.accumulo.core.data.ConditionalMutation)6 ConstraintViolationSummary (org.apache.accumulo.core.data.ConstraintViolationSummary)6 PrestoException (com.facebook.presto.spi.PrestoException)5