Search in sources :

Example 11 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class BookKeeperLogTests method testAppendPermanentFailures.

/**
 * Tests the ability to retry writes when Bookies fail.
 */
@Test
public void testAppendPermanentFailures() throws Exception {
    try (DurableDataLog log = createDurableDataLog()) {
        log.initialize(TIMEOUT);
        List<CompletableFuture<LogAddress>> appendFutures = new ArrayList<>();
        try {
            // Suspend a bookie (this will trigger write errors).
            stopFirstBookie();
            // Issue appends in parallel.
            int writeCount = getWriteCount();
            for (int i = 0; i < writeCount; i++) {
                appendFutures.add(log.append(new ByteArraySegment(getWriteData()), TIMEOUT));
            }
            // Verify that all writes failed or got cancelled.
            AtomicBoolean cancellationEncountered = new AtomicBoolean(false);
            for (val f : appendFutures) {
                AssertExtensions.assertThrows("Write did not fail correctly.", () -> f.get(TIMEOUT.toMillis(), TimeUnit.MILLISECONDS), ex -> {
                    cancellationEncountered.set(cancellationEncountered.get() || ex instanceof CancellationException);
                    if (cancellationEncountered.get()) {
                        return ex instanceof CancellationException;
                    } else {
                        return ex instanceof RetriesExhaustedException || ex instanceof DurableDataLogException;
                    }
                });
            }
        } finally {
            // Don't forget to resume the bookie, but only AFTER we are done testing.
            restartFirstBookie();
        }
    }
}
Also used : DurableDataLog(io.pravega.segmentstore.storage.DurableDataLog) lombok.val(lombok.val) DurableDataLogException(io.pravega.segmentstore.storage.DurableDataLogException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) ByteArraySegment(io.pravega.common.util.ByteArraySegment) RetriesExhaustedException(io.pravega.common.util.RetriesExhaustedException) CancellationException(java.util.concurrent.CancellationException) ArrayList(java.util.ArrayList) Test(org.junit.Test)

Example 12 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class BookKeeperCommand method createContext.

/**
 * Creates a new Context to be used by the BookKeeper command.
 *
 * @return A new Context.
 * @throws DurableDataLogException If the BookKeeperLogFactory could not be initialized.
 */
protected Context createContext() throws DurableDataLogException {
    val serviceConfig = getServiceConfig();
    val bkConfig = getCommandArgs().getState().getConfigBuilder().include(BookKeeperConfig.builder().with(BookKeeperConfig.ZK_ADDRESS, serviceConfig.getZkURL())).build().getConfig(BookKeeperConfig::builder);
    val zkClient = createZKClient();
    val factory = new BookKeeperLogFactory(bkConfig, zkClient, getCommandArgs().getState().getExecutor());
    try {
        factory.initialize();
    } catch (DurableDataLogException ex) {
        zkClient.close();
        throw ex;
    }
    val bkAdmin = new BookKeeperAdmin(factory.getBookKeeperClient());
    return new Context(serviceConfig, bkConfig, zkClient, factory, bkAdmin);
}
Also used : lombok.val(lombok.val) DurableDataLogException(io.pravega.segmentstore.storage.DurableDataLogException) BookKeeperConfig(io.pravega.segmentstore.storage.impl.bookkeeper.BookKeeperConfig) BookKeeperLogFactory(io.pravega.segmentstore.storage.impl.bookkeeper.BookKeeperLogFactory) BookKeeperAdmin(org.apache.bookkeeper.client.BookKeeperAdmin)

Example 13 with DurableDataLogException

use of io.pravega.segmentstore.storage.DurableDataLogException in project pravega by pravega.

the class ContainerCommand method createContext.

/**
 * Creates a new Context to be used by the BookKeeper command.
 *
 * @return A new Context.
 * @throws DurableDataLogException If the BookKeeperLogFactory could not be initialized.
 */
@Override
protected Context createContext() throws DurableDataLogException {
    val serviceConfig = getServiceConfig();
    val containerConfig = getCommandArgs().getState().getConfigBuilder().build().getConfig(ContainerConfig::builder);
    val bkConfig = getCommandArgs().getState().getConfigBuilder().include(BookKeeperConfig.builder().with(BookKeeperConfig.ZK_ADDRESS, serviceConfig.getZkURL())).build().getConfig(BookKeeperConfig::builder);
    val zkClient = createZKClient();
    val factory = new BookKeeperLogFactory(bkConfig, zkClient, getCommandArgs().getState().getExecutor());
    try {
        factory.initialize();
    } catch (DurableDataLogException ex) {
        zkClient.close();
        throw ex;
    }
    val bkAdmin = new BookKeeperAdmin(factory.getBookKeeperClient());
    return new Context(serviceConfig, containerConfig, bkConfig, zkClient, factory, bkAdmin);
}
Also used : lombok.val(lombok.val) ContainerConfig(io.pravega.segmentstore.server.containers.ContainerConfig) DurableDataLogException(io.pravega.segmentstore.storage.DurableDataLogException) BookKeeperConfig(io.pravega.segmentstore.storage.impl.bookkeeper.BookKeeperConfig) BookKeeperLogFactory(io.pravega.segmentstore.storage.impl.bookkeeper.BookKeeperLogFactory) BookKeeperAdmin(org.apache.bookkeeper.client.BookKeeperAdmin)

Aggregations

DurableDataLogException (io.pravega.segmentstore.storage.DurableDataLogException)13 lombok.val (lombok.val)8 DataLogDisabledException (io.pravega.segmentstore.storage.DataLogDisabledException)5 DataLogWriterNotPrimaryException (io.pravega.segmentstore.storage.DataLogWriterNotPrimaryException)5 IOException (java.io.IOException)5 RetriesExhaustedException (io.pravega.common.util.RetriesExhaustedException)4 DataLogNotAvailableException (io.pravega.segmentstore.storage.DataLogNotAvailableException)4 CancellationException (java.util.concurrent.CancellationException)4 ObjectClosedException (io.pravega.common.ObjectClosedException)3 StreamSegmentException (io.pravega.segmentstore.contracts.StreamSegmentException)3 StreamSegmentNotExistsException (io.pravega.segmentstore.contracts.StreamSegmentNotExistsException)3 StreamSegmentSealedException (io.pravega.segmentstore.contracts.StreamSegmentSealedException)3 Operation (io.pravega.segmentstore.server.logs.operations.Operation)3 ProbeOperation (io.pravega.segmentstore.server.logs.operations.ProbeOperation)3 StorageOperation (io.pravega.segmentstore.server.logs.operations.StorageOperation)3 StreamSegmentAppendOperation (io.pravega.segmentstore.server.logs.operations.StreamSegmentAppendOperation)3 DurableDataLog (io.pravega.segmentstore.storage.DurableDataLog)3 ErrorInjector (io.pravega.test.common.ErrorInjector)3 IntentionalException (io.pravega.test.common.IntentionalException)3 CompletionException (java.util.concurrent.CompletionException)3