Search in sources :

Example 1 with BookKeeperClient

use of org.apache.distributedlog.BookKeeperClient in project bookkeeper by apache.

the class DLAuditor method calculateLedgerSpaceUsage.

public long calculateLedgerSpaceUsage(URI uri) throws IOException {
    List<URI> uris = Lists.newArrayList(uri);
    String zkServers = validateAndGetZKServers(uris);
    RetryPolicy retryPolicy = new BoundExponentialBackoffRetryPolicy(conf.getZKRetryBackoffStartMillis(), conf.getZKRetryBackoffMaxMillis(), Integer.MAX_VALUE);
    ZooKeeperClient zkc = ZooKeeperClientBuilder.newBuilder().name("DLAuditor-ZK").zkServers(zkServers).sessionTimeoutMs(conf.getZKSessionTimeoutMilliseconds()).retryPolicy(retryPolicy).zkAclId(conf.getZkAclId()).build();
    ExecutorService executorService = Executors.newCachedThreadPool();
    try {
        BKDLConfig bkdlConfig = resolveBKDLConfig(zkc, uris);
        logger.info("Resolved bookkeeper config : {}", bkdlConfig);
        BookKeeperClient bkc = BookKeeperClientBuilder.newBuilder().name("DLAuditor-BK").dlConfig(conf).zkServers(bkdlConfig.getBkZkServersForWriter()).ledgersPath(bkdlConfig.getBkLedgersPath()).build();
        try {
            return calculateLedgerSpaceUsage(bkc, executorService);
        } finally {
            bkc.close();
        }
    } finally {
        zkc.close();
        executorService.shutdown();
    }
}
Also used : ZooKeeperClient(org.apache.distributedlog.ZooKeeperClient) BookKeeperClient(org.apache.distributedlog.BookKeeperClient) ExecutorService(java.util.concurrent.ExecutorService) BKDLConfig(org.apache.distributedlog.impl.metadata.BKDLConfig) BoundExponentialBackoffRetryPolicy(org.apache.bookkeeper.zookeeper.BoundExponentialBackoffRetryPolicy) URI(java.net.URI) RetryPolicy(org.apache.bookkeeper.zookeeper.RetryPolicy) BoundExponentialBackoffRetryPolicy(org.apache.bookkeeper.zookeeper.BoundExponentialBackoffRetryPolicy)

Example 2 with BookKeeperClient

use of org.apache.distributedlog.BookKeeperClient in project bookkeeper by apache.

the class DLAuditor method collectLedgers.

public Pair<Set<Long>, Set<Long>> collectLedgers(List<URI> uris, List<List<String>> allocationPaths) throws IOException {
    checkArgument(uris.size() > 0, "No uri provided to audit");
    String zkServers = validateAndGetZKServers(uris);
    RetryPolicy retryPolicy = new BoundExponentialBackoffRetryPolicy(conf.getZKRetryBackoffStartMillis(), conf.getZKRetryBackoffMaxMillis(), Integer.MAX_VALUE);
    ZooKeeperClient zkc = ZooKeeperClientBuilder.newBuilder().name("DLAuditor-ZK").zkServers(zkServers).sessionTimeoutMs(conf.getZKSessionTimeoutMilliseconds()).retryPolicy(retryPolicy).zkAclId(conf.getZkAclId()).build();
    ExecutorService executorService = Executors.newCachedThreadPool();
    try {
        BKDLConfig bkdlConfig = resolveBKDLConfig(zkc, uris);
        logger.info("Resolved bookkeeper config : {}", bkdlConfig);
        BookKeeperClient bkc = BookKeeperClientBuilder.newBuilder().name("DLAuditor-BK").dlConfig(conf).zkServers(bkdlConfig.getBkZkServersForWriter()).ledgersPath(bkdlConfig.getBkLedgersPath()).build();
        try {
            Set<Long> bkLedgers = collectLedgersFromBK(bkc, executorService);
            Set<Long> dlLedgers = collectLedgersFromDL(uris, allocationPaths);
            return Pair.of(bkLedgers, dlLedgers);
        } finally {
            bkc.close();
        }
    } finally {
        zkc.close();
        executorService.shutdown();
    }
}
Also used : ZooKeeperClient(org.apache.distributedlog.ZooKeeperClient) BookKeeperClient(org.apache.distributedlog.BookKeeperClient) ExecutorService(java.util.concurrent.ExecutorService) AtomicLong(java.util.concurrent.atomic.AtomicLong) BKDLConfig(org.apache.distributedlog.impl.metadata.BKDLConfig) BoundExponentialBackoffRetryPolicy(org.apache.bookkeeper.zookeeper.BoundExponentialBackoffRetryPolicy) RetryPolicy(org.apache.bookkeeper.zookeeper.RetryPolicy) BoundExponentialBackoffRetryPolicy(org.apache.bookkeeper.zookeeper.BoundExponentialBackoffRetryPolicy)

Aggregations

ExecutorService (java.util.concurrent.ExecutorService)2 BoundExponentialBackoffRetryPolicy (org.apache.bookkeeper.zookeeper.BoundExponentialBackoffRetryPolicy)2 RetryPolicy (org.apache.bookkeeper.zookeeper.RetryPolicy)2 BookKeeperClient (org.apache.distributedlog.BookKeeperClient)2 ZooKeeperClient (org.apache.distributedlog.ZooKeeperClient)2 BKDLConfig (org.apache.distributedlog.impl.metadata.BKDLConfig)2 URI (java.net.URI)1 AtomicLong (java.util.concurrent.atomic.AtomicLong)1