Search in sources :

Example 16 with Scanner

use of org.apache.accumulo.core.client.Scanner in project presto by prestodb.

the class AccumuloClient method getDefaultTabletLocation.

private Optional<String> getDefaultTabletLocation(String fulltable) {
    try {
        String tableId = connector.tableOperations().tableIdMap().get(fulltable);
        // Create a scanner over the metadata table, fetching the 'loc' column of the default tablet row
        Scanner scan = connector.createScanner("accumulo.metadata", connector.securityOperations().getUserAuthorizations(username));
        scan.fetchColumnFamily(new Text("loc"));
        scan.setRange(new Range(tableId + '<'));
        // scan the entry
        Optional<String> location = Optional.empty();
        for (Entry<Key, Value> entry : scan) {
            if (location.isPresent()) {
                throw new PrestoException(FUNCTION_IMPLEMENTATION_ERROR, "Scan for default tablet returned more than one entry");
            }
            location = Optional.of(entry.getValue().toString());
        }
        scan.close();
        return location;
    } catch (Exception e) {
        // Swallow this exception so the query does not fail due to being unable to locate the tablet server for the default tablet.
        // This is purely an optimization, but we will want to log the error.
        LOG.error("Failed to get tablet location, returning dummy location", e);
        return Optional.empty();
    }
}
Also used : Scanner(org.apache.accumulo.core.client.Scanner) Value(org.apache.accumulo.core.data.Value) Text(org.apache.hadoop.io.Text) PrestoException(com.facebook.presto.spi.PrestoException) Range(org.apache.accumulo.core.data.Range) Key(org.apache.accumulo.core.data.Key) PartialKey(org.apache.accumulo.core.data.PartialKey) InvalidParameterException(java.security.InvalidParameterException) PrestoException(com.facebook.presto.spi.PrestoException) AccumuloSecurityException(org.apache.accumulo.core.client.AccumuloSecurityException) AccumuloException(org.apache.accumulo.core.client.AccumuloException) TableNotFoundException(com.facebook.presto.spi.TableNotFoundException)

Example 17 with Scanner

use of org.apache.accumulo.core.client.Scanner in project gora by apache.

the class AccumuloStore method createScanner.

private Scanner createScanner(Query<K, T> query) throws TableNotFoundException {
    // TODO make isolated scanner optional?
    Scanner scanner = new IsolatedScanner(conn.createScanner(mapping.tableName, Authorizations.EMPTY));
    setFetchColumns(scanner, query.getFields());
    scanner.setRange(createRange(query));
    if (query.getStartTime() != -1 || query.getEndTime() != -1) {
        IteratorSetting is = new IteratorSetting(30, TimestampFilter.class);
        if (query.getStartTime() != -1)
            TimestampFilter.setStart(is, query.getStartTime(), true);
        if (query.getEndTime() != -1)
            TimestampFilter.setEnd(is, query.getEndTime(), true);
        scanner.addScanIterator(is);
    }
    return scanner;
}
Also used : IsolatedScanner(org.apache.accumulo.core.client.IsolatedScanner) Scanner(org.apache.accumulo.core.client.Scanner) IteratorSetting(org.apache.accumulo.core.client.IteratorSetting) IsolatedScanner(org.apache.accumulo.core.client.IsolatedScanner)

Example 18 with Scanner

use of org.apache.accumulo.core.client.Scanner in project gora by apache.

the class AccumuloStore method deleteByQuery.

@Override
public long deleteByQuery(Query<K, T> query) {
    try {
        Scanner scanner = createScanner(query);
        // add iterator that drops values on the server side
        scanner.addScanIterator(new IteratorSetting(Integer.MAX_VALUE, SortedKeyIterator.class));
        RowIterator iterator = new RowIterator(scanner.iterator());
        long count = 0;
        while (iterator.hasNext()) {
            Iterator<Entry<Key, Value>> row = iterator.next();
            Mutation m = null;
            while (row.hasNext()) {
                Entry<Key, Value> entry = row.next();
                Key key = entry.getKey();
                if (m == null)
                    m = new Mutation(key.getRow());
                // TODO optimize to avoid continually creating column vis? prob does not matter for empty
                m.putDelete(key.getColumnFamily(), key.getColumnQualifier(), new ColumnVisibility(key.getColumnVisibility()), key.getTimestamp());
            }
            getBatchWriter().addMutation(m);
            count++;
        }
        return count;
    } catch (TableNotFoundException e) {
        // TODO return 0?
        LOG.error(e.getMessage(), e);
        return 0;
    } catch (MutationsRejectedException e) {
        LOG.error(e.getMessage(), e);
        return 0;
    } catch (IOException e) {
        LOG.error(e.getMessage(), e);
        return 0;
    }
}
Also used : IsolatedScanner(org.apache.accumulo.core.client.IsolatedScanner) Scanner(org.apache.accumulo.core.client.Scanner) IOException(java.io.IOException) TableNotFoundException(org.apache.accumulo.core.client.TableNotFoundException) Entry(java.util.Map.Entry) IteratorSetting(org.apache.accumulo.core.client.IteratorSetting) SortedKeyIterator(org.apache.accumulo.core.iterators.SortedKeyIterator) RowIterator(org.apache.accumulo.core.client.RowIterator) Value(org.apache.accumulo.core.data.Value) Mutation(org.apache.accumulo.core.data.Mutation) ColumnVisibility(org.apache.accumulo.core.security.ColumnVisibility) Key(org.apache.accumulo.core.data.Key) MutationsRejectedException(org.apache.accumulo.core.client.MutationsRejectedException)

Example 19 with Scanner

use of org.apache.accumulo.core.client.Scanner in project gora by apache.

the class AccumuloStore method get.

@Override
public T get(K key, String[] fields) {
    try {
        // TODO make isolated scanner optional?
        Scanner scanner = new IsolatedScanner(conn.createScanner(mapping.tableName, Authorizations.EMPTY));
        Range rowRange = new Range(new Text(toBytes(key)));
        scanner.setRange(rowRange);
        setFetchColumns(scanner, fields);
        T persistent = newPersistent();
        ByteSequence row = populate(scanner.iterator(), persistent);
        if (row == null)
            return null;
        return persistent;
    } catch (TableNotFoundException e) {
        LOG.error(e.getMessage(), e);
        return null;
    } catch (IOException e) {
        LOG.error(e.getMessage(), e);
        return null;
    }
}
Also used : IsolatedScanner(org.apache.accumulo.core.client.IsolatedScanner) Scanner(org.apache.accumulo.core.client.Scanner) TableNotFoundException(org.apache.accumulo.core.client.TableNotFoundException) Text(org.apache.hadoop.io.Text) IOException(java.io.IOException) Range(org.apache.accumulo.core.data.Range) IsolatedScanner(org.apache.accumulo.core.client.IsolatedScanner) ByteSequence(org.apache.accumulo.core.data.ByteSequence)

Aggregations

Scanner (org.apache.accumulo.core.client.Scanner)19 Key (org.apache.accumulo.core.data.Key)15 Value (org.apache.accumulo.core.data.Value)15 IteratorSetting (org.apache.accumulo.core.client.IteratorSetting)13 Authorizations (org.apache.accumulo.core.security.Authorizations)11 Entry (java.util.Map.Entry)10 AccumuloException (org.apache.accumulo.core.client.AccumuloException)8 BatchWriterConfig (org.apache.accumulo.core.client.BatchWriterConfig)8 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)8 Mutation (org.apache.accumulo.core.data.Mutation)7 ColumnVisibility (org.apache.accumulo.core.security.ColumnVisibility)7 BatchWriter (org.apache.accumulo.core.client.BatchWriter)6 Range (org.apache.accumulo.core.data.Range)6 IteratorSettingBuilder (uk.gov.gchq.gaffer.accumulostore.utils.IteratorSettingBuilder)6 Edge (uk.gov.gchq.gaffer.data.element.Edge)6 Element (uk.gov.gchq.gaffer.data.element.Element)6 Connector (org.apache.accumulo.core.client.Connector)5 PasswordToken (org.apache.accumulo.core.client.security.tokens.PasswordToken)5 Text (org.apache.hadoop.io.Text)4 PrestoException (com.facebook.presto.spi.PrestoException)3