Search in sources :

Example 1 with RegistrationClient

use of org.apache.bookkeeper.discover.RegistrationClient in project pulsar by apache.

the class Bookies method getAllBookies.

@GET
@Path("/all")
@ApiOperation(value = "Gets raw information for all the bookies in the cluster", response = BookiesClusterInfo.class)
@ApiResponses(value = { @ApiResponse(code = 403, message = "Don't have admin permission") })
public BookiesClusterInfo getAllBookies() throws Exception {
    validateSuperUserAccess();
    BookKeeper bookKeeper = bookKeeper();
    MetadataClientDriver metadataClientDriver = bookKeeper.getMetadataClientDriver();
    RegistrationClient registrationClient = metadataClientDriver.getRegistrationClient();
    Set<BookieId> allBookies = registrationClient.getAllBookies().get().getValue();
    List<RawBookieInfo> result = new ArrayList<>(allBookies.size());
    for (BookieId bookieId : allBookies) {
        RawBookieInfo bookieInfo = new RawBookieInfo(bookieId.toString());
        result.add(bookieInfo);
    }
    return BookiesClusterInfo.builder().bookies(result).build();
}
Also used : BookieId(org.apache.bookkeeper.net.BookieId) RawBookieInfo(org.apache.pulsar.common.policies.data.RawBookieInfo) RegistrationClient(org.apache.bookkeeper.discover.RegistrationClient) ArrayList(java.util.ArrayList) BookKeeper(org.apache.bookkeeper.client.BookKeeper) MetadataClientDriver(org.apache.bookkeeper.meta.MetadataClientDriver) Path(javax.ws.rs.Path) GET(javax.ws.rs.GET) ApiOperation(io.swagger.annotations.ApiOperation) ApiResponses(io.swagger.annotations.ApiResponses)

Example 2 with RegistrationClient

use of org.apache.bookkeeper.discover.RegistrationClient in project pulsar by apache.

the class PulsarRegistrationClientTest method testGetAllBookies.

@Test(dataProvider = "impl")
public void testGetAllBookies(String provider, Supplier<String> urlSupplier) throws Exception {
    @Cleanup MetadataStoreExtended store = MetadataStoreExtended.create(urlSupplier.get(), MetadataStoreConfig.builder().build());
    String ledgersRoot = "/test/ledgers-" + UUID.randomUUID();
    @Cleanup RegistrationManager rm = new PulsarRegistrationManager(store, ledgersRoot, mock(AbstractConfiguration.class));
    @Cleanup RegistrationClient rc = new PulsarRegistrationClient(store, ledgersRoot);
    Set<BookieId> addresses = prepareNBookies(10);
    List<String> children = new ArrayList<>();
    for (BookieId address : addresses) {
        children.add(address.toString());
        boolean isReadOnly = children.size() % 2 == 0;
        rm.registerBookie(address, isReadOnly, new BookieServiceInfo());
    }
    Versioned<Set<BookieId>> result = result(rc.getAllBookies());
    assertEquals(addresses.size(), result.getValue().size());
}
Also used : BookieId(org.apache.bookkeeper.net.BookieId) RegistrationManager(org.apache.bookkeeper.discover.RegistrationManager) HashSet(java.util.HashSet) Set(java.util.Set) ArrayList(java.util.ArrayList) Cleanup(lombok.Cleanup) AbstractConfiguration(org.apache.bookkeeper.conf.AbstractConfiguration) RegistrationClient(org.apache.bookkeeper.discover.RegistrationClient) BookieServiceInfo(org.apache.bookkeeper.discover.BookieServiceInfo) MetadataStoreExtended(org.apache.pulsar.metadata.api.extended.MetadataStoreExtended) Test(org.testng.annotations.Test) BaseMetadataStoreTest(org.apache.pulsar.metadata.BaseMetadataStoreTest)

Example 3 with RegistrationClient

use of org.apache.bookkeeper.discover.RegistrationClient in project pulsar by apache.

the class PulsarRegistrationClientTest method testGetReadonlyBookies.

@Test(dataProvider = "impl")
public void testGetReadonlyBookies(String provider, Supplier<String> urlSupplier) throws Exception {
    @Cleanup MetadataStoreExtended store = MetadataStoreExtended.create(urlSupplier.get(), MetadataStoreConfig.builder().build());
    String ledgersRoot = "/test/ledgers-" + UUID.randomUUID();
    @Cleanup RegistrationManager rm = new PulsarRegistrationManager(store, ledgersRoot, mock(AbstractConfiguration.class));
    @Cleanup RegistrationClient rc = new PulsarRegistrationClient(store, ledgersRoot);
    Set<BookieId> addresses = prepareNBookies(10);
    List<String> children = new ArrayList<>();
    for (BookieId address : addresses) {
        children.add(address.toString());
        rm.registerBookie(address, true, new BookieServiceInfo());
    }
    Versioned<Set<BookieId>> result = result(rc.getReadOnlyBookies());
    assertEquals(addresses.size(), result.getValue().size());
}
Also used : BookieId(org.apache.bookkeeper.net.BookieId) RegistrationManager(org.apache.bookkeeper.discover.RegistrationManager) HashSet(java.util.HashSet) Set(java.util.Set) ArrayList(java.util.ArrayList) Cleanup(lombok.Cleanup) AbstractConfiguration(org.apache.bookkeeper.conf.AbstractConfiguration) RegistrationClient(org.apache.bookkeeper.discover.RegistrationClient) BookieServiceInfo(org.apache.bookkeeper.discover.BookieServiceInfo) MetadataStoreExtended(org.apache.pulsar.metadata.api.extended.MetadataStoreExtended) Test(org.testng.annotations.Test) BaseMetadataStoreTest(org.apache.pulsar.metadata.BaseMetadataStoreTest)

Example 4 with RegistrationClient

use of org.apache.bookkeeper.discover.RegistrationClient in project pulsar by apache.

the class PulsarRegistrationClientTest method testGetWritableBookies.

@Test(dataProvider = "impl")
public void testGetWritableBookies(String provider, Supplier<String> urlSupplier) throws Exception {
    @Cleanup MetadataStoreExtended store = MetadataStoreExtended.create(urlSupplier.get(), MetadataStoreConfig.builder().build());
    String ledgersRoot = "/test/ledgers-" + UUID.randomUUID();
    @Cleanup RegistrationManager rm = new PulsarRegistrationManager(store, ledgersRoot, mock(AbstractConfiguration.class));
    @Cleanup RegistrationClient rc = new PulsarRegistrationClient(store, ledgersRoot);
    Set<BookieId> addresses = prepareNBookies(10);
    List<String> children = new ArrayList<>();
    for (BookieId address : addresses) {
        children.add(address.toString());
        rm.registerBookie(address, false, new BookieServiceInfo());
    }
    Versioned<Set<BookieId>> result = result(rc.getWritableBookies());
    assertEquals(addresses.size(), result.getValue().size());
}
Also used : BookieId(org.apache.bookkeeper.net.BookieId) RegistrationManager(org.apache.bookkeeper.discover.RegistrationManager) HashSet(java.util.HashSet) Set(java.util.Set) ArrayList(java.util.ArrayList) Cleanup(lombok.Cleanup) AbstractConfiguration(org.apache.bookkeeper.conf.AbstractConfiguration) RegistrationClient(org.apache.bookkeeper.discover.RegistrationClient) BookieServiceInfo(org.apache.bookkeeper.discover.BookieServiceInfo) MetadataStoreExtended(org.apache.pulsar.metadata.api.extended.MetadataStoreExtended) Test(org.testng.annotations.Test) BaseMetadataStoreTest(org.apache.pulsar.metadata.BaseMetadataStoreTest)

Example 5 with RegistrationClient

use of org.apache.bookkeeper.discover.RegistrationClient in project pulsar by yahoo.

the class PulsarRegistrationClientTest method testWatchBookiesSuccess.

private void testWatchBookiesSuccess(String provider, Supplier<String> urlSupplier, boolean isWritable) throws Exception {
    @Cleanup MetadataStoreExtended store = MetadataStoreExtended.create(urlSupplier.get(), MetadataStoreConfig.builder().build());
    String ledgersRoot = "/test/ledgers-" + UUID.randomUUID();
    @Cleanup RegistrationManager rm = new PulsarRegistrationManager(store, ledgersRoot, mock(AbstractConfiguration.class));
    @Cleanup RegistrationClient rc = new PulsarRegistrationClient(store, ledgersRoot);
    // 
    // 1. test watch bookies with a listener
    // 
    Queue<Versioned<Set<BookieId>>> updates = new ConcurrentLinkedQueue<>();
    Map<BookieId, Boolean> bookies = new ConcurrentHashMap<>();
    RegistrationClient.RegistrationListener listener = (b) -> {
        updates.add(b);
        b.getValue().forEach(x -> bookies.put(x, true));
    };
    int BOOKIES = 10;
    Set<BookieId> addresses = prepareNBookies(BOOKIES);
    if (isWritable) {
        result(rc.watchWritableBookies(listener));
    } else {
        result(rc.watchReadOnlyBookies(listener));
    }
    for (BookieId address : addresses) {
        rm.registerBookie(address, !isWritable, new BookieServiceInfo());
    }
    Awaitility.await().untilAsserted(() -> {
        assertFalse(updates.isEmpty());
        assertEquals(bookies.size(), BOOKIES);
    });
}
Also used : BookieId(org.apache.bookkeeper.net.BookieId) BookieServiceInfo(org.apache.bookkeeper.discover.BookieServiceInfo) Assert.assertEquals(org.testng.Assert.assertEquals) Cleanup(lombok.Cleanup) MetadataStoreExtended(org.apache.pulsar.metadata.api.extended.MetadataStoreExtended) Test(org.testng.annotations.Test) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) RegistrationClient(org.apache.bookkeeper.discover.RegistrationClient) HashSet(java.util.HashSet) RegistrationManager(org.apache.bookkeeper.discover.RegistrationManager) Map(java.util.Map) Versioned(org.apache.bookkeeper.versioning.Versioned) Assert.assertFalse(org.testng.Assert.assertFalse) AbstractConfiguration(org.apache.bookkeeper.conf.AbstractConfiguration) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Set(java.util.Set) UUID(java.util.UUID) MetadataStoreConfig(org.apache.pulsar.metadata.api.MetadataStoreConfig) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) BookieSocketAddress(org.apache.bookkeeper.net.BookieSocketAddress) FutureUtils.result(org.apache.bookkeeper.common.concurrent.FutureUtils.result) BaseMetadataStoreTest(org.apache.pulsar.metadata.BaseMetadataStoreTest) Queue(java.util.Queue) Awaitility(org.awaitility.Awaitility) BookieId(org.apache.bookkeeper.net.BookieId) Mockito.mock(org.mockito.Mockito.mock) ConcurrentLinkedQueue(java.util.concurrent.ConcurrentLinkedQueue) RegistrationManager(org.apache.bookkeeper.discover.RegistrationManager) Versioned(org.apache.bookkeeper.versioning.Versioned) Cleanup(lombok.Cleanup) AbstractConfiguration(org.apache.bookkeeper.conf.AbstractConfiguration) RegistrationClient(org.apache.bookkeeper.discover.RegistrationClient) BookieServiceInfo(org.apache.bookkeeper.discover.BookieServiceInfo) ConcurrentLinkedQueue(java.util.concurrent.ConcurrentLinkedQueue) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) MetadataStoreExtended(org.apache.pulsar.metadata.api.extended.MetadataStoreExtended)

Aggregations

RegistrationClient (org.apache.bookkeeper.discover.RegistrationClient)21 BookieId (org.apache.bookkeeper.net.BookieId)19 ArrayList (java.util.ArrayList)16 Cleanup (lombok.Cleanup)15 BookieServiceInfo (org.apache.bookkeeper.discover.BookieServiceInfo)13 HashSet (java.util.HashSet)12 Set (java.util.Set)12 AbstractConfiguration (org.apache.bookkeeper.conf.AbstractConfiguration)12 RegistrationManager (org.apache.bookkeeper.discover.RegistrationManager)12 BaseMetadataStoreTest (org.apache.pulsar.metadata.BaseMetadataStoreTest)12 MetadataStoreExtended (org.apache.pulsar.metadata.api.extended.MetadataStoreExtended)12 Test (org.testng.annotations.Test)12 BookKeeper (org.apache.bookkeeper.client.BookKeeper)4 ApiOperation (io.swagger.annotations.ApiOperation)3 ApiResponses (io.swagger.annotations.ApiResponses)3 List (java.util.List)3 Map (java.util.Map)3 Queue (java.util.Queue)3 UUID (java.util.UUID)3 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)3