Search in sources :

Example 6 with PermanentBackendException

use of com.thinkaurelius.titan.diskstorage.PermanentBackendException in project atlas by apache.

the class HBaseStoreManager method ensureColumnFamilyExists.

private void ensureColumnFamilyExists(String tableName, String columnFamily, int ttlInSeconds) throws BackendException {
    AdminMask adm = null;
    try {
        adm = getAdminInterface();
        HTableDescriptor desc = ensureTableExists(tableName, columnFamily, ttlInSeconds);
        Preconditions.checkNotNull(desc);
        HColumnDescriptor cf = desc.getFamily(columnFamily.getBytes());
        // Create our column family, if necessary
        if (cf == null) {
            try {
                if (!adm.isTableDisabled(tableName)) {
                    adm.disableTable(tableName);
                }
            } catch (TableNotEnabledException e) {
                logger.debug("Table {} already disabled", tableName);
            } catch (IOException e) {
                throw new TemporaryBackendException(e);
            }
            try {
                HColumnDescriptor cdesc = new HColumnDescriptor(columnFamily);
                setCFOptions(cdesc, ttlInSeconds);
                adm.addColumn(tableName, cdesc);
                logger.debug("Added HBase ColumnFamily {}, waiting for 1 sec. to propogate.", columnFamily);
                adm.enableTable(tableName);
            } catch (TableNotFoundException ee) {
                logger.error("TableNotFoundException", ee);
                throw new PermanentBackendException(ee);
            } catch (org.apache.hadoop.hbase.TableExistsException ee) {
                logger.debug("Swallowing exception {}", ee);
            } catch (IOException ee) {
                throw new TemporaryBackendException(ee);
            }
        }
    } finally {
        IOUtils.closeQuietly(adm);
    }
}
Also used : TableNotFoundException(org.apache.hadoop.hbase.TableNotFoundException) TemporaryBackendException(com.thinkaurelius.titan.diskstorage.TemporaryBackendException) HColumnDescriptor(org.apache.hadoop.hbase.HColumnDescriptor) PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException) IOException(java.io.IOException) HTableDescriptor(org.apache.hadoop.hbase.HTableDescriptor) TableNotEnabledException(org.apache.hadoop.hbase.TableNotEnabledException)

Example 7 with PermanentBackendException

use of com.thinkaurelius.titan.diskstorage.PermanentBackendException in project incubator-atlas by apache.

the class Solr5Index method query.

@Override
public Iterable<RawQuery.Result<String>> query(RawQuery query, KeyInformation.IndexRetriever informations, BaseTransaction tx) throws BackendException {
    List<RawQuery.Result<String>> result;
    String collection = query.getStore();
    String keyIdField = getKeyFieldId(collection);
    SolrQuery solrQuery = new SolrQuery(query.getQuery()).addField(keyIdField).setIncludeScore(true).setStart(query.getOffset()).setRows(query.hasLimit() ? query.getLimit() : maxResults);
    try {
        QueryResponse response = solrClient.query(collection, solrQuery);
        if (logger.isDebugEnabled())
            logger.debug("Executed query [{}] in {} ms", query.getQuery(), response.getElapsedTime());
        int totalHits = response.getResults().size();
        if (!query.hasLimit() && totalHits >= maxResults) {
            logger.warn("Query result set truncated to first [{}] elements for query: {}", maxResults, query);
        }
        result = new ArrayList<>(totalHits);
        for (SolrDocument hit : response.getResults()) {
            double score = Double.parseDouble(hit.getFieldValue("score").toString());
            result.add(new RawQuery.Result<>(hit.getFieldValue(keyIdField).toString(), score));
        }
    } catch (IOException e) {
        logger.error("Query did not complete : ", e);
        throw new PermanentBackendException(e);
    } catch (SolrServerException e) {
        logger.error("Unable to query Solr index.", e);
        throw new PermanentBackendException(e);
    }
    return result;
}
Also used : PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException) SolrServerException(org.apache.solr.client.solrj.SolrServerException) IOException(java.io.IOException) SolrQuery(org.apache.solr.client.solrj.SolrQuery) SolrDocument(org.apache.solr.common.SolrDocument) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) RawQuery(com.thinkaurelius.titan.diskstorage.indexing.RawQuery)

Example 8 with PermanentBackendException

use of com.thinkaurelius.titan.diskstorage.PermanentBackendException in project incubator-atlas by apache.

the class Solr5Index method clearStorage.

@Override
public void clearStorage() throws BackendException {
    try {
        if (mode != Mode.CLOUD)
            throw new UnsupportedOperationException("Operation only supported for SolrCloud");
        logger.debug("Clearing storage from Solr: {}", solrClient);
        ZkStateReader zkStateReader = ((CloudSolrClient) solrClient).getZkStateReader();
        zkStateReader.updateClusterState();
        ClusterState clusterState = zkStateReader.getClusterState();
        for (String collection : clusterState.getCollections()) {
            logger.debug("Clearing collection [{}] in Solr", collection);
            UpdateRequest deleteAll = newUpdateRequest();
            deleteAll.deleteByQuery("*:*");
            solrClient.request(deleteAll, collection);
        }
    } catch (SolrServerException e) {
        logger.error("Unable to clear storage from index due to server error on Solr.", e);
        throw new PermanentBackendException(e);
    } catch (IOException e) {
        logger.error("Unable to clear storage from index due to low-level I/O error.", e);
        throw new PermanentBackendException(e);
    } catch (Exception e) {
        logger.error("Unable to clear storage from index due to general error.", e);
        throw new PermanentBackendException(e);
    }
}
Also used : ZkStateReader(org.apache.solr.common.cloud.ZkStateReader) ClusterState(org.apache.solr.common.cloud.ClusterState) UpdateRequest(org.apache.solr.client.solrj.request.UpdateRequest) PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException) SolrServerException(org.apache.solr.client.solrj.SolrServerException) IOException(java.io.IOException) TemporaryBackendException(com.thinkaurelius.titan.diskstorage.TemporaryBackendException) SolrServerException(org.apache.solr.client.solrj.SolrServerException) PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException) BackendException(com.thinkaurelius.titan.diskstorage.BackendException) KeeperException(org.apache.zookeeper.KeeperException) IOException(java.io.IOException) CloudSolrClient(org.apache.solr.client.solrj.impl.CloudSolrClient)

Example 9 with PermanentBackendException

use of com.thinkaurelius.titan.diskstorage.PermanentBackendException in project incubator-atlas by apache.

the class HBaseStoreManager method ensureColumnFamilyExists.

private void ensureColumnFamilyExists(String tableName, String columnFamily, int ttlInSeconds) throws BackendException {
    AdminMask adm = null;
    try {
        adm = getAdminInterface();
        HTableDescriptor desc = ensureTableExists(tableName, columnFamily, ttlInSeconds);
        Preconditions.checkNotNull(desc);
        HColumnDescriptor cf = desc.getFamily(columnFamily.getBytes());
        // Create our column family, if necessary
        if (cf == null) {
            try {
                if (!adm.isTableDisabled(tableName)) {
                    adm.disableTable(tableName);
                }
            } catch (TableNotEnabledException e) {
                logger.debug("Table {} already disabled", tableName);
            } catch (IOException e) {
                throw new TemporaryBackendException(e);
            }
            try {
                HColumnDescriptor cdesc = new HColumnDescriptor(columnFamily);
                setCFOptions(cdesc, ttlInSeconds);
                adm.addColumn(tableName, cdesc);
                logger.debug("Added HBase ColumnFamily {}, waiting for 1 sec. to propogate.", columnFamily);
                adm.enableTable(tableName);
            } catch (TableNotFoundException ee) {
                logger.error("TableNotFoundException", ee);
                throw new PermanentBackendException(ee);
            } catch (org.apache.hadoop.hbase.TableExistsException ee) {
                logger.debug("Swallowing exception {}", ee);
            } catch (IOException ee) {
                throw new TemporaryBackendException(ee);
            }
        }
    } finally {
        IOUtils.closeQuietly(adm);
    }
}
Also used : TableNotFoundException(org.apache.hadoop.hbase.TableNotFoundException) TemporaryBackendException(com.thinkaurelius.titan.diskstorage.TemporaryBackendException) HColumnDescriptor(org.apache.hadoop.hbase.HColumnDescriptor) PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException) IOException(java.io.IOException) HTableDescriptor(org.apache.hadoop.hbase.HTableDescriptor) TableNotEnabledException(org.apache.hadoop.hbase.TableNotEnabledException)

Example 10 with PermanentBackendException

use of com.thinkaurelius.titan.diskstorage.PermanentBackendException in project titan by thinkaurelius.

the class BerkeleyJEKeyValueStore method insert.

public void insert(StaticBuffer key, StaticBuffer value, StoreTransaction txh, boolean allowOverwrite) throws BackendException {
    Transaction tx = getTransaction(txh);
    try {
        OperationStatus status;
        log.trace("db={}, op=insert, tx={}", name, txh);
        if (allowOverwrite)
            status = db.put(tx, key.as(ENTRY_FACTORY), value.as(ENTRY_FACTORY));
        else
            status = db.putNoOverwrite(tx, key.as(ENTRY_FACTORY), value.as(ENTRY_FACTORY));
        if (status != OperationStatus.SUCCESS) {
            if (status == OperationStatus.KEYEXIST) {
                throw new PermanentBackendException("Key already exists on no-overwrite.");
            } else {
                throw new PermanentBackendException("Could not write entity, return status: " + status);
            }
        }
    } catch (DatabaseException e) {
        throw new PermanentBackendException(e);
    }
}
Also used : StoreTransaction(com.thinkaurelius.titan.diskstorage.keycolumnvalue.StoreTransaction) PermanentBackendException(com.thinkaurelius.titan.diskstorage.PermanentBackendException)

Aggregations

PermanentBackendException (com.thinkaurelius.titan.diskstorage.PermanentBackendException)14 IOException (java.io.IOException)10 TemporaryBackendException (com.thinkaurelius.titan.diskstorage.TemporaryBackendException)6 SolrServerException (org.apache.solr.client.solrj.SolrServerException)6 BackendException (com.thinkaurelius.titan.diskstorage.BackendException)4 SolrQuery (org.apache.solr.client.solrj.SolrQuery)4 QueryResponse (org.apache.solr.client.solrj.response.QueryResponse)4 SolrDocument (org.apache.solr.common.SolrDocument)4 StoreTransaction (com.thinkaurelius.titan.diskstorage.keycolumnvalue.StoreTransaction)3 HColumnDescriptor (org.apache.hadoop.hbase.HColumnDescriptor)3 HTableDescriptor (org.apache.hadoop.hbase.HTableDescriptor)3 TableNotEnabledException (org.apache.hadoop.hbase.TableNotEnabledException)3 TableNotFoundException (org.apache.hadoop.hbase.TableNotFoundException)3 IndexQuery (com.thinkaurelius.titan.diskstorage.indexing.IndexQuery)2 RawQuery (com.thinkaurelius.titan.diskstorage.indexing.RawQuery)2 CloudSolrClient (org.apache.solr.client.solrj.impl.CloudSolrClient)2 UpdateRequest (org.apache.solr.client.solrj.request.UpdateRequest)2 ClusterState (org.apache.solr.common.cloud.ClusterState)2 ZkStateReader (org.apache.solr.common.cloud.ZkStateReader)2 KeeperException (org.apache.zookeeper.KeeperException)2