Search in sources :

Example 1 with DLInterruptedException

use of org.apache.distributedlog.exceptions.DLInterruptedException in project bookkeeper by apache.

the class ZKMetadataAccessor method getMetadata.

/**
 * Retrieve the metadata stored at the node.
 * @return byte array containing the metadata.
 * @throws IOException
 */
@Override
public byte[] getMetadata() throws IOException {
    checkClosedOrInError("createOrUpdateMetadata");
    String zkPath = getZKPath();
    LOG.debug("Getting application specific metadata from {}", zkPath);
    try {
        Stat currentStat = readerZKC.get().exists(zkPath, false);
        if (currentStat == null) {
            return null;
        } else {
            return readerZKC.get().getData(zkPath, false, currentStat);
        }
    } catch (InterruptedException ie) {
        Thread.currentThread().interrupt();
        throw new DLInterruptedException("Error reading the max tx id from zk", ie);
    } catch (Exception e) {
        throw new IOException("Error reading the max tx id from zk", e);
    }
}
Also used : Stat(org.apache.zookeeper.data.Stat) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) IOException(java.io.IOException) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) AlreadyClosedException(org.apache.distributedlog.exceptions.AlreadyClosedException) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) IOException(java.io.IOException)

Example 2 with DLInterruptedException

use of org.apache.distributedlog.exceptions.DLInterruptedException in project bookkeeper by apache.

the class DLAuditor method calculateLedgerSpaceUsage.

private long calculateLedgerSpaceUsage(BookKeeperClient bkc, final ExecutorService executorService) throws IOException {
    final AtomicLong totalBytes = new AtomicLong(0);
    final AtomicLong totalEntries = new AtomicLong(0);
    final AtomicLong numLedgers = new AtomicLong(0);
    LedgerManager lm = BookKeeperAccessor.getLedgerManager(bkc.get());
    final CompletableFuture<Void> doneFuture = FutureUtils.createFuture();
    final BookKeeper bk = bkc.get();
    BookkeeperInternalCallbacks.Processor<Long> collector = new BookkeeperInternalCallbacks.Processor<Long>() {

        @Override
        public void process(final Long lid, final AsyncCallback.VoidCallback cb) {
            numLedgers.incrementAndGet();
            executorService.submit(new Runnable() {

                @Override
                public void run() {
                    bk.asyncOpenLedgerNoRecovery(lid, BookKeeper.DigestType.CRC32, conf.getBKDigestPW().getBytes(UTF_8), new org.apache.bookkeeper.client.AsyncCallback.OpenCallback() {

                        @Override
                        public void openComplete(int rc, LedgerHandle lh, Object ctx) {
                            final int cbRc;
                            if (BKException.Code.OK == rc) {
                                totalBytes.addAndGet(lh.getLength());
                                totalEntries.addAndGet(lh.getLastAddConfirmed() + 1);
                                cbRc = rc;
                            } else {
                                cbRc = BKException.Code.ZKException;
                            }
                            executorService.submit(new Runnable() {

                                @Override
                                public void run() {
                                    cb.processResult(cbRc, null, null);
                                }
                            });
                        }
                    }, null);
                }
            });
        }
    };
    AsyncCallback.VoidCallback finalCb = new AsyncCallback.VoidCallback() {

        @Override
        public void processResult(int rc, String path, Object ctx) {
            if (BKException.Code.OK == rc) {
                doneFuture.complete(null);
            } else {
                doneFuture.completeExceptionally(BKException.create(rc));
            }
        }
    };
    lm.asyncProcessLedgers(collector, finalCb, null, BKException.Code.OK, BKException.Code.ZKException);
    try {
        doneFuture.get();
        logger.info("calculated {} ledgers\n\ttotal bytes = {}\n\ttotal entries = {}", new Object[] { numLedgers.get(), totalBytes.get(), totalEntries.get() });
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        throw new DLInterruptedException("Interrupted on calculating ledger space : ", e);
    } catch (ExecutionException e) {
        if (e.getCause() instanceof IOException) {
            throw (IOException) (e.getCause());
        } else {
            throw new IOException("Failed to calculate ledger space : ", e.getCause());
        }
    }
    return totalBytes.get();
}
Also used : LedgerManager(org.apache.bookkeeper.meta.LedgerManager) AsyncCallback(org.apache.zookeeper.AsyncCallback) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) ExecutionException(java.util.concurrent.ExecutionException) LedgerHandle(org.apache.bookkeeper.client.LedgerHandle) BookKeeper(org.apache.bookkeeper.client.BookKeeper) IOException(java.io.IOException) AtomicLong(java.util.concurrent.atomic.AtomicLong) BookkeeperInternalCallbacks(org.apache.bookkeeper.proto.BookkeeperInternalCallbacks) AtomicLong(java.util.concurrent.atomic.AtomicLong) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException)

Example 3 with DLInterruptedException

use of org.apache.distributedlog.exceptions.DLInterruptedException in project bookkeeper by apache.

the class ZKLogStreamMetadataStore method ensureReadLockPathExist.

// 
// Create Read Lock
// 
private CompletableFuture<Void> ensureReadLockPathExist(final LogMetadata logMetadata, final String readLockPath) {
    final CompletableFuture<Void> promise = new CompletableFuture<Void>();
    promise.whenComplete((value, cause) -> {
        if (cause instanceof CancellationException) {
            FutureUtils.completeExceptionally(promise, new LockCancelledException(readLockPath, "Could not ensure read lock path", cause));
        }
    });
    Optional<String> parentPathShouldNotCreate = Optional.of(logMetadata.getLogRootPath());
    Utils.zkAsyncCreateFullPathOptimisticRecursive(zooKeeperClient, readLockPath, parentPathShouldNotCreate, new byte[0], zooKeeperClient.getDefaultACL(), CreateMode.PERSISTENT, new org.apache.zookeeper.AsyncCallback.StringCallback() {

        @Override
        public void processResult(final int rc, final String path, Object ctx, String name) {
            if (KeeperException.Code.NONODE.intValue() == rc) {
                FutureUtils.completeExceptionally(promise, new LogNotFoundException(String.format("Log %s does not exist or has been deleted", logMetadata.getFullyQualifiedName())));
            } else if (KeeperException.Code.OK.intValue() == rc) {
                FutureUtils.complete(promise, null);
                LOG.trace("Created path {}.", path);
            } else if (KeeperException.Code.NODEEXISTS.intValue() == rc) {
                FutureUtils.complete(promise, null);
                LOG.trace("Path {} is already existed.", path);
            } else if (DistributedLogConstants.ZK_CONNECTION_EXCEPTION_RESULT_CODE == rc) {
                FutureUtils.completeExceptionally(promise, new ZooKeeperClient.ZooKeeperConnectionException(path));
            } else if (DistributedLogConstants.DL_INTERRUPTED_EXCEPTION_RESULT_CODE == rc) {
                FutureUtils.completeExceptionally(promise, new DLInterruptedException(path));
            } else {
                FutureUtils.completeExceptionally(promise, KeeperException.create(KeeperException.Code.get(rc)));
            }
        }
    }, null);
    return promise;
}
Also used : LockCancelledException(org.apache.distributedlog.exceptions.LockCancelledException) AsyncCallback(org.apache.zookeeper.AsyncCallback) ZooKeeperConnectionException(org.apache.distributedlog.ZooKeeperClient.ZooKeeperConnectionException) CompletableFuture(java.util.concurrent.CompletableFuture) CancellationException(java.util.concurrent.CancellationException) ZooKeeperClient(org.apache.distributedlog.ZooKeeperClient) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) LogNotFoundException(org.apache.distributedlog.exceptions.LogNotFoundException)

Example 4 with DLInterruptedException

use of org.apache.distributedlog.exceptions.DLInterruptedException in project bookkeeper by apache.

the class ZKLogStreamMetadataStore method logExists.

@Override
public CompletableFuture<Void> logExists(URI uri, final String logName) {
    final String logSegmentsPath = LogMetadata.getLogSegmentsPath(uri, logName, conf.getUnpartitionedStreamName());
    final CompletableFuture<Void> promise = new CompletableFuture<Void>();
    try {
        final ZooKeeper zk = zooKeeperClient.get();
        zk.sync(logSegmentsPath, new AsyncCallback.VoidCallback() {

            @Override
            public void processResult(int syncRc, String path, Object syncCtx) {
                if (KeeperException.Code.NONODE.intValue() == syncRc) {
                    promise.completeExceptionally(new LogNotFoundException(String.format("Log %s does not exist or has been deleted", logName)));
                    return;
                } else if (KeeperException.Code.OK.intValue() != syncRc) {
                    promise.completeExceptionally(new ZKException("Error on checking log existence for " + logName, KeeperException.create(KeeperException.Code.get(syncRc))));
                    return;
                }
                zk.exists(logSegmentsPath, false, new AsyncCallback.StatCallback() {

                    @Override
                    public void processResult(int rc, String path, Object ctx, Stat stat) {
                        if (KeeperException.Code.OK.intValue() == rc) {
                            promise.complete(null);
                        } else if (KeeperException.Code.NONODE.intValue() == rc) {
                            promise.completeExceptionally(new LogNotFoundException(String.format("Log %s does not exist or has been deleted", logName)));
                        } else {
                            promise.completeExceptionally(new ZKException("Error on checking log existence for " + logName, KeeperException.create(KeeperException.Code.get(rc))));
                        }
                    }
                }, null);
            }
        }, null);
    } catch (InterruptedException ie) {
        Thread.currentThread().interrupt();
        LOG.error("Interrupted while reading {}", logSegmentsPath, ie);
        promise.completeExceptionally(new DLInterruptedException("Interrupted while checking " + logSegmentsPath, ie));
    } catch (ZooKeeperClient.ZooKeeperConnectionException e) {
        promise.completeExceptionally(e);
    }
    return promise;
}
Also used : AsyncCallback(org.apache.zookeeper.AsyncCallback) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) ZooKeeperConnectionException(org.apache.distributedlog.ZooKeeperClient.ZooKeeperConnectionException) CompletableFuture(java.util.concurrent.CompletableFuture) ZKException(org.apache.distributedlog.exceptions.ZKException) ZooKeeper(org.apache.zookeeper.ZooKeeper) Stat(org.apache.zookeeper.data.Stat) ZooKeeperClient(org.apache.distributedlog.ZooKeeperClient) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) LogNotFoundException(org.apache.distributedlog.exceptions.LogNotFoundException)

Example 5 with DLInterruptedException

use of org.apache.distributedlog.exceptions.DLInterruptedException in project bookkeeper by apache.

the class Utils method sync.

/**
 * Sync zookeeper client on given <i>path</i>.
 *
 * @param zkc
 *          zookeeper client
 * @param path
 *          path to sync
 * @return zookeeper client after sync
 * @throws IOException
 */
public static ZooKeeper sync(ZooKeeperClient zkc, String path) throws IOException {
    ZooKeeper zk;
    try {
        zk = zkc.get();
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        throw new DLInterruptedException("Interrupted on checking if log " + path + " exists", e);
    }
    final CountDownLatch syncLatch = new CountDownLatch(1);
    final AtomicInteger syncResult = new AtomicInteger(0);
    zk.sync(path, new AsyncCallback.VoidCallback() {

        @Override
        public void processResult(int rc, String path, Object ctx) {
            syncResult.set(rc);
            syncLatch.countDown();
        }
    }, null);
    try {
        syncLatch.await();
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        throw new DLInterruptedException("Interrupted on syncing zookeeper connection", e);
    }
    if (KeeperException.Code.OK.intValue() != syncResult.get()) {
        throw new ZKException("Error syncing zookeeper connection ", KeeperException.Code.get(syncResult.get()));
    }
    return zk;
}
Also used : ZKException(org.apache.distributedlog.exceptions.ZKException) ZooKeeper(org.apache.zookeeper.ZooKeeper) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AsyncCallback(org.apache.zookeeper.AsyncCallback) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException) CountDownLatch(java.util.concurrent.CountDownLatch) DLInterruptedException(org.apache.distributedlog.exceptions.DLInterruptedException)

Aggregations

DLInterruptedException (org.apache.distributedlog.exceptions.DLInterruptedException)17 IOException (java.io.IOException)12 AsyncCallback (org.apache.zookeeper.AsyncCallback)6 Stat (org.apache.zookeeper.data.Stat)5 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)4 CompletableFuture (java.util.concurrent.CompletableFuture)3 CountDownLatch (java.util.concurrent.CountDownLatch)3 AtomicLong (java.util.concurrent.atomic.AtomicLong)3 ZooKeeperClient (org.apache.distributedlog.ZooKeeperClient)3 LogNotFoundException (org.apache.distributedlog.exceptions.LogNotFoundException)3 ZKException (org.apache.distributedlog.exceptions.ZKException)3 KeeperException (org.apache.zookeeper.KeeperException)3 ArrayList (java.util.ArrayList)2 ExecutionException (java.util.concurrent.ExecutionException)2 ExecutorService (java.util.concurrent.ExecutorService)2 LedgerHandle (org.apache.bookkeeper.client.LedgerHandle)2 LedgerManager (org.apache.bookkeeper.meta.LedgerManager)2 BookkeeperInternalCallbacks (org.apache.bookkeeper.proto.BookkeeperInternalCallbacks)2 ZooKeeperConnectionException (org.apache.distributedlog.ZooKeeperClient.ZooKeeperConnectionException)2 DistributedLogManager (org.apache.distributedlog.api.DistributedLogManager)2