Search in sources :

Example 46 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class SolrIndex method totals.

@Override
public Long totals(RawQuery query, KeyInformation.IndexRetriever information, BaseTransaction tx) throws BackendException {
    try {
        final String collection = query.getStore();
        final String keyIdField = getKeyFieldId(collection);
        final QueryResponse response = solrClient.query(collection, runCommonQuery(query, information, tx, collection, keyIdField));
        logger.debug("Executed query [{}] in {} ms", query.getQuery(), response.getElapsedTime());
        return response.getResults().getNumFound();
    } catch (final IOException e) {
        logger.error("Query did not complete : ", e);
        throw new PermanentBackendException(e);
    } catch (final SolrServerException e) {
        logger.error("Unable to query Solr index.", e);
        throw new PermanentBackendException(e);
    }
}
Also used : PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) SolrServerException(org.apache.solr.client.solrj.SolrServerException) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException)

Example 47 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class SolrIndex method clearStorage.

@Override
public void clearStorage() throws BackendException {
    try {
        if (mode != Mode.CLOUD) {
            logger.error("Operation only supported for SolrCloud. Cores must be deleted manually through the Solr API when using HTTP mode.");
            return;
        }
        logger.debug("Clearing storage from Solr: {}", solrClient);
        final ZkStateReader zkStateReader = ((CloudSolrClient) solrClient).getZkStateReader();
        zkStateReader.forciblyRefreshAllClusterStateSlow();
        final ClusterState clusterState = zkStateReader.getClusterState();
        for (final String collection : clusterState.getCollectionsMap().keySet()) {
            logger.debug("Clearing collection [{}] in Solr", collection);
            // Collection is not dropped because it may have been created externally
            final UpdateRequest deleteAll = newUpdateRequest();
            deleteAll.deleteByQuery("*:*");
            solrClient.request(deleteAll, collection);
        }
    } catch (final SolrServerException e) {
        logger.error("Unable to clear storage from index due to server error on Solr.", e);
        throw new PermanentBackendException(e);
    } catch (final IOException e) {
        logger.error("Unable to clear storage from index due to low-level I/O error.", e);
        throw new PermanentBackendException(e);
    } catch (final Exception e) {
        logger.error("Unable to clear storage from index due to general error.", e);
        throw new PermanentBackendException(e);
    }
}
Also used : ZkStateReader(org.apache.solr.common.cloud.ZkStateReader) ClusterState(org.apache.solr.common.cloud.ClusterState) UpdateRequest(org.apache.solr.client.solrj.request.UpdateRequest) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) SolrServerException(org.apache.solr.client.solrj.SolrServerException) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) SolrServerException(org.apache.solr.client.solrj.SolrServerException) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) TemporaryBackendException(org.janusgraph.diskstorage.TemporaryBackendException) BackendException(org.janusgraph.diskstorage.BackendException) KeeperException(org.apache.zookeeper.KeeperException) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) CloudSolrClient(org.apache.solr.client.solrj.impl.CloudSolrClient)

Example 48 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class ConsistentKeyLockerTest method expectDeleteLock.

private void expectDeleteLock(KeyColumn lockID, StaticBuffer lockKey, ConsistentKeyLockStatus lockStatus, BackendException... backendFailures) throws BackendException {
    List<StaticBuffer> deletions = ImmutableList.of(codec.toLockCol(lockStatus.getWriteTimestamp(), defaultLockRid, times));
    expect(times.getTime()).andReturn(currentTimeNS);
    store.mutate(eq(lockKey), eq(ImmutableList.of()), eq(deletions), eq(defaultTx));
    int backendExceptionsThrown = 0;
    for (BackendException e : backendFailures) {
        expectLastCall().andThrow(e);
        if (e instanceof PermanentBackendException) {
            break;
        }
        backendExceptionsThrown++;
        int maxTemporaryStorageExceptions = 3;
        if (backendExceptionsThrown < maxTemporaryStorageExceptions) {
            expect(times.getTime()).andReturn(currentTimeNS);
            store.mutate(eq(lockKey), eq(ImmutableList.of()), eq(deletions), eq(defaultTx));
        }
    }
    expect(mediator.unlock(lockID, defaultTx)).andReturn(true);
}
Also used : PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) StaticBuffer(org.janusgraph.diskstorage.StaticBuffer) TemporaryBackendException(org.janusgraph.diskstorage.TemporaryBackendException) BackendException(org.janusgraph.diskstorage.BackendException) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException)

Example 49 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class ConsistentKeyLockerTest method testCheckLocksDiesOnPermanentStorageException.

/**
 * A single PermanentStorageException on getSlice() for a single lock is
 * sufficient to make the method return immediately (regardless of whether
 * other locks are waiting to be checked).
 *
 * @throws InterruptedException shouldn't happen
 * @throws org.janusgraph.diskstorage.BackendException     shouldn't happen
 */
@Test
public void testCheckLocksDiesOnPermanentStorageException() throws InterruptedException, BackendException {
    // Setup a LockStatus for defaultLockID
    ConsistentKeyLockStatus lockStatus = makeStatusNow();
    currentTimeNS = currentTimeNS.plusNanos(1);
    expect(lockState.getLocksForTx(defaultTx)).andReturn(ImmutableMap.of(defaultLockID, lockStatus));
    expectSleepAfterWritingLock(lockStatus);
    // First and only getSlice call throws a PSE
    recordExceptionalLockGetSlice(new PermanentBackendException("Connection to storage cluster failed: peer is an IPv6 toaster"));
    ctrl.replay();
    PermanentBackendException pse = null;
    try {
        locker.checkLocks(defaultTx);
    } catch (PermanentBackendException e) {
        pse = e;
    }
    assertNotNull(pse);
}
Also used : PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) ConsistentKeyLockStatus(org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLockStatus) Test(org.junit.jupiter.api.Test)

Example 50 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class ConsistentKeyLockerTest method testWriteLockDiesOnPermanentStorageException.

/**
 * Test that the first {@link org.janusgraph.diskstorage.PermanentBackendException} thrown by the
 * locker's store causes it to attempt to delete outstanding lock writes and
 * then emit the exception without retrying.
 *
 * @throws org.janusgraph.diskstorage.BackendException shouldn't happen
 */
@Test
public void testWriteLockDiesOnPermanentStorageException() throws BackendException {
    PermanentBackendException errOnFire = new PermanentBackendException("Storage cluster is on fire");
    expect(lockState.has(defaultTx, defaultLockID)).andReturn(false);
    recordSuccessfulLocalLock();
    StaticBuffer lockCol = recordExceptionLockWrite(errOnFire);
    recordSuccessfulLockDelete(lockCol);
    recordSuccessfulLocalUnlock();
    ctrl.replay();
    BackendException expected = null;
    try {
        // SUT
        locker.writeLock(defaultLockID, defaultTx);
    } catch (PermanentLockingException e) {
        expected = e;
    }
    assertNotNull(expected);
    assertEquals(errOnFire, expected.getCause());
}
Also used : PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) StaticBuffer(org.janusgraph.diskstorage.StaticBuffer) TemporaryBackendException(org.janusgraph.diskstorage.TemporaryBackendException) BackendException(org.janusgraph.diskstorage.BackendException) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) Test(org.junit.jupiter.api.Test)

Aggregations

PermanentBackendException (org.janusgraph.diskstorage.PermanentBackendException)53 IOException (java.io.IOException)24 TemporaryBackendException (org.janusgraph.diskstorage.TemporaryBackendException)16 UncheckedIOException (java.io.UncheckedIOException)12 BackendException (org.janusgraph.diskstorage.BackendException)12 Configuration (org.janusgraph.diskstorage.configuration.Configuration)8 ConnectionException (com.netflix.astyanax.connectionpool.exceptions.ConnectionException)7 DatabaseException (com.sleepycat.je.DatabaseException)7 BaseTransactionConfig (org.janusgraph.diskstorage.BaseTransactionConfig)7 Duration (java.time.Duration)6 ArrayList (java.util.ArrayList)6 Map (java.util.Map)6 StaticBuffer (org.janusgraph.diskstorage.StaticBuffer)6 StoreTransaction (org.janusgraph.diskstorage.keycolumnvalue.StoreTransaction)6 SolrServerException (org.apache.solr.client.solrj.SolrServerException)5 KeeperException (org.apache.zookeeper.KeeperException)5 Transaction (com.sleepycat.je.Transaction)4 HashMap (java.util.HashMap)4 CloudSolrClient (org.apache.solr.client.solrj.impl.CloudSolrClient)4 KeyInformation (org.janusgraph.diskstorage.indexing.KeyInformation)4