Search in sources :

Example 51 with LedgerEntry

use of org.apache.bookkeeper.client.LedgerEntry in project incubator-pulsar by apache.

the class EntryCacheTest method getLedgerHandle.

private static LedgerHandle getLedgerHandle() {
    final LedgerHandle lh = mock(LedgerHandle.class);
    final LedgerEntry ledgerEntry = mock(LedgerEntry.class, Mockito.CALLS_REAL_METHODS);
    doReturn(new byte[10]).when(ledgerEntry).getEntry();
    doReturn(Unpooled.wrappedBuffer(new byte[10])).when(ledgerEntry).getEntryBuffer();
    doReturn((long) 10).when(ledgerEntry).getLength();
    doAnswer(new Answer<Object>() {

        public Object answer(InvocationOnMock invocation) {
            Object[] args = invocation.getArguments();
            long firstEntry = (Long) args[0];
            long lastEntry = (Long) args[1];
            ReadCallback callback = (ReadCallback) args[2];
            Object ctx = args[3];
            Vector<LedgerEntry> entries = new Vector<LedgerEntry>();
            for (int i = 0; i <= (lastEntry - firstEntry); i++) {
                entries.add(ledgerEntry);
            }
            callback.readComplete(0, lh, entries.elements(), ctx);
            return null;
        }
    }).when(lh).asyncReadEntries(anyLong(), anyLong(), any(ReadCallback.class), any());
    return lh;
}
Also used : LedgerHandle(org.apache.bookkeeper.client.LedgerHandle) InvocationOnMock(org.mockito.invocation.InvocationOnMock) LedgerEntry(org.apache.bookkeeper.client.LedgerEntry) ReadCallback(org.apache.bookkeeper.client.AsyncCallback.ReadCallback) Vector(java.util.Vector)

Example 52 with LedgerEntry

use of org.apache.bookkeeper.client.LedgerEntry in project incubator-pulsar by apache.

the class CompactorTest method compactAndVerify.

private List<String> compactAndVerify(String topic, Map<String, byte[]> expected) throws Exception {
    BookKeeper bk = pulsar.getBookKeeperClientFactory().create(this.conf, null);
    Compactor compactor = new TwoPhaseCompactor(conf, pulsarClient, bk, compactionScheduler);
    long compactedLedgerId = compactor.compact(topic).get();
    LedgerHandle ledger = bk.openLedger(compactedLedgerId, Compactor.COMPACTED_TOPIC_LEDGER_DIGEST_TYPE, Compactor.COMPACTED_TOPIC_LEDGER_PASSWORD);
    // 0..lac
    Assert.assertEquals(// 0..lac
    ledger.getLastAddConfirmed() + 1, expected.size(), "Should have as many entries as there is keys");
    List<String> keys = new ArrayList<>();
    Enumeration<LedgerEntry> entries = ledger.readEntries(0, ledger.getLastAddConfirmed());
    while (entries.hasMoreElements()) {
        ByteBuf buf = entries.nextElement().getEntryBuffer();
        RawMessage m = RawMessageImpl.deserializeFrom(buf);
        String key = extractKey(m);
        keys.add(key);
        ByteBuf payload = extractPayload(m);
        byte[] bytes = new byte[payload.readableBytes()];
        payload.readBytes(bytes);
        Assert.assertEquals(bytes, expected.remove(key), "Compacted version should match expected version");
        m.close();
    }
    Assert.assertTrue(expected.isEmpty(), "All expected keys should have been found");
    return keys;
}
Also used : LedgerHandle(org.apache.bookkeeper.client.LedgerHandle) ArrayList(java.util.ArrayList) BookKeeper(org.apache.bookkeeper.client.BookKeeper) ByteBuf(io.netty.buffer.ByteBuf) LedgerEntry(org.apache.bookkeeper.client.LedgerEntry) RawMessage(org.apache.pulsar.client.api.RawMessage)

Example 53 with LedgerEntry

use of org.apache.bookkeeper.client.LedgerEntry in project incubator-pulsar by apache.

the class CompactedTopicImpl method readEntries.

private static CompletableFuture<List<Entry>> readEntries(LedgerHandle lh, long from, long to) {
    CompletableFuture<Enumeration<LedgerEntry>> promise = new CompletableFuture<>();
    lh.asyncReadEntries(from, to, (rc, _lh, seq, ctx) -> {
        if (rc != BKException.Code.OK) {
            promise.completeExceptionally(BKException.create(rc));
        } else {
            promise.complete(seq);
        }
    }, null);
    return promise.thenApply((seq) -> {
        List<Entry> entries = new ArrayList<Entry>();
        while (seq.hasMoreElements()) {
            ByteBuf buf = seq.nextElement().getEntryBuffer();
            try (RawMessage m = RawMessageImpl.deserializeFrom(buf)) {
                entries.add(EntryImpl.create(m.getMessageIdData().getLedgerId(), m.getMessageIdData().getEntryId(), m.getHeadersAndPayload()));
            } finally {
                buf.release();
            }
        }
        return entries;
    });
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) Entry(org.apache.bookkeeper.mledger.Entry) LedgerEntry(org.apache.bookkeeper.client.LedgerEntry) Enumeration(java.util.Enumeration) ArrayList(java.util.ArrayList) ByteBuf(io.netty.buffer.ByteBuf) RawMessage(org.apache.pulsar.client.api.RawMessage)

Aggregations

LedgerEntry (org.apache.bookkeeper.client.LedgerEntry)53 LedgerHandle (org.apache.bookkeeper.client.LedgerHandle)37 BKException (org.apache.bookkeeper.client.BKException)20 Test (org.junit.Test)20 BookKeeper (org.apache.bookkeeper.client.BookKeeper)10 ManagedLedgerException (org.apache.bookkeeper.mledger.ManagedLedgerException)9 IOException (java.io.IOException)8 ByteBuffer (java.nio.ByteBuffer)7 InvalidProtocolBufferException (com.google.protobuf.InvalidProtocolBufferException)6 File (java.io.File)5 ArrayList (java.util.ArrayList)5 List (java.util.List)5 CountDownLatch (java.util.concurrent.CountDownLatch)5 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)5 Bookie (org.apache.bookkeeper.bookie.Bookie)4 LedgerDirsManager (org.apache.bookkeeper.bookie.LedgerDirsManager)4 MLDataFormats (org.apache.bookkeeper.mledger.proto.MLDataFormats)4 ServerConfiguration (org.apache.bookkeeper.conf.ServerConfiguration)3 ManagedLedgerImpl.createManagedLedgerException (org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.createManagedLedgerException)3 Versioned (org.apache.bookkeeper.versioning.Versioned)3