Search in sources :

Example 61 with ManagedLedgerConfig

use of org.apache.bookkeeper.mledger.ManagedLedgerConfig in project incubator-pulsar by apache.

the class ManagedLedgerErrorsTest method recoverAfterZnodeVersionError.

@Test
public void recoverAfterZnodeVersionError() throws Exception {
    ManagedLedger ledger = factory.open("my_test_ledger", new ManagedLedgerConfig().setMaxEntriesPerLedger(1));
    zkc.failNow(Code.BADVERSION);
    // First write will succeed
    ledger.addEntry("test".getBytes());
    try {
        // This write will try to create a ledger and it will fail at it
        ledger.addEntry("entry".getBytes());
        fail("should fail");
    } catch (BadVersionException e) {
    // ok
    }
    try {
        // At this point the ledger should be fenced for good
        ledger.addEntry("entry".getBytes());
        fail("should fail");
    } catch (ManagedLedgerFencedException e) {
    // ok
    }
}
Also used : ManagedLedger(org.apache.bookkeeper.mledger.ManagedLedger) ManagedLedgerFencedException(org.apache.bookkeeper.mledger.ManagedLedgerException.ManagedLedgerFencedException) BadVersionException(org.apache.bookkeeper.mledger.ManagedLedgerException.BadVersionException) ManagedLedgerConfig(org.apache.bookkeeper.mledger.ManagedLedgerConfig) Test(org.testng.annotations.Test)

Example 62 with ManagedLedgerConfig

use of org.apache.bookkeeper.mledger.ManagedLedgerConfig in project incubator-pulsar by apache.

the class ManagedLedgerFactoryTest method testGetManagedLedgerInfoWithClose.

@Test(timeOut = 20000)
public void testGetManagedLedgerInfoWithClose() throws Exception {
    ManagedLedgerConfig conf = new ManagedLedgerConfig();
    conf.setMaxEntriesPerLedger(1);
    ManagedLedgerImpl ledger = (ManagedLedgerImpl) factory.open("testGetManagedLedgerInfo", conf);
    ManagedCursor c1 = ledger.openCursor("c1");
    PositionImpl p1 = (PositionImpl) ledger.addEntry("entry1".getBytes());
    PositionImpl p2 = (PositionImpl) ledger.addEntry("entry2".getBytes());
    PositionImpl p3 = (PositionImpl) ledger.addEntry("entry3".getBytes());
    ledger.addEntry("entry4".getBytes());
    c1.delete(p2);
    c1.delete(p3);
    ledger.close();
    ManagedLedgerInfo info = factory.getManagedLedgerInfo("testGetManagedLedgerInfo");
    assertEquals(info.ledgers.size(), 4);
    assertEquals(info.ledgers.get(0).ledgerId, 3);
    assertEquals(info.ledgers.get(1).ledgerId, 5);
    assertEquals(info.ledgers.get(2).ledgerId, 6);
    assertEquals(info.ledgers.get(3).ledgerId, 7);
    assertEquals(info.cursors.size(), 1);
    CursorInfo cursorInfo = info.cursors.get("c1");
    assertEquals(cursorInfo.markDelete.ledgerId, 3);
    assertEquals(cursorInfo.markDelete.entryId, -1);
    assertEquals(cursorInfo.individualDeletedMessages.size(), 1);
    MessageRangeInfo mri = cursorInfo.individualDeletedMessages.get(0);
    assertEquals(mri.from.ledgerId, p1.getLedgerId());
    assertEquals(mri.from.entryId, p1.getEntryId());
    assertEquals(mri.to.ledgerId, p3.getLedgerId());
    assertEquals(mri.to.entryId, p3.getEntryId());
}
Also used : CursorInfo(org.apache.bookkeeper.mledger.ManagedLedgerInfo.CursorInfo) ManagedLedgerConfig(org.apache.bookkeeper.mledger.ManagedLedgerConfig) ManagedCursor(org.apache.bookkeeper.mledger.ManagedCursor) ManagedLedgerInfo(org.apache.bookkeeper.mledger.ManagedLedgerInfo) MessageRangeInfo(org.apache.bookkeeper.mledger.ManagedLedgerInfo.MessageRangeInfo) Test(org.testng.annotations.Test)

Example 63 with ManagedLedgerConfig

use of org.apache.bookkeeper.mledger.ManagedLedgerConfig in project incubator-pulsar by apache.

the class ManagedLedgerSingleBookieTest method simple.

// (timeOut = 20000)
@Test
public void simple() throws Exception {
    ManagedLedgerConfig config = new ManagedLedgerConfig().setEnsembleSize(1).setWriteQuorumSize(1).setAckQuorumSize(1).setMetadataEnsembleSize(1).setMetadataWriteQuorumSize(1).setMetadataAckQuorumSize(1);
    ManagedLedger ledger = factory.open("my_test_ledger", config);
    assertEquals(ledger.getNumberOfEntries(), 0);
    assertEquals(ledger.getTotalSize(), 0);
    ledger.addEntry("dummy-entry-1".getBytes(Encoding));
    assertEquals(ledger.getNumberOfEntries(), 1);
    assertEquals(ledger.getTotalSize(), "dummy-entry-1".getBytes(Encoding).length);
    ManagedCursor cursor = ledger.openCursor("c1");
    assertEquals(cursor.hasMoreEntries(), false);
    assertEquals(cursor.readEntries(100), new ArrayList<Entry>());
    ledger.addEntry("dummy-entry-2".getBytes(Encoding));
    assertEquals(cursor.hasMoreEntries(), true);
    List<Entry> entries = cursor.readEntries(100);
    assertEquals(entries.size(), 1);
    entries.forEach(e -> e.release());
    entries = cursor.readEntries(100);
    assertEquals(entries.size(), 0);
    entries.forEach(e -> e.release());
    ledger.close();
}
Also used : Entry(org.apache.bookkeeper.mledger.Entry) ManagedLedger(org.apache.bookkeeper.mledger.ManagedLedger) ManagedLedgerConfig(org.apache.bookkeeper.mledger.ManagedLedgerConfig) ManagedCursor(org.apache.bookkeeper.mledger.ManagedCursor) Test(org.testng.annotations.Test)

Example 64 with ManagedLedgerConfig

use of org.apache.bookkeeper.mledger.ManagedLedgerConfig in project incubator-pulsar by apache.

the class ManagedLedgerTest method asyncAPI.

@Test(timeOut = 20000)
public void asyncAPI() throws Throwable {
    final CountDownLatch counter = new CountDownLatch(1);
    factory.asyncOpen("my_test_ledger", new ManagedLedgerConfig(), new OpenLedgerCallback() {

        @Override
        public void openLedgerComplete(ManagedLedger ledger, Object ctx) {
            ledger.asyncOpenCursor("test-cursor", new OpenCursorCallback() {

                @Override
                public void openCursorComplete(ManagedCursor cursor, Object ctx) {
                    ManagedLedger ledger = (ManagedLedger) ctx;
                    ledger.asyncAddEntry("test".getBytes(Encoding), new AddEntryCallback() {

                        @Override
                        public void addComplete(Position position, Object ctx) {
                            @SuppressWarnings("unchecked") Pair<ManagedLedger, ManagedCursor> pair = (Pair<ManagedLedger, ManagedCursor>) ctx;
                            ManagedLedger ledger = pair.first;
                            ManagedCursor cursor = pair.second;
                            assertEquals(ledger.getNumberOfEntries(), 1);
                            assertEquals(ledger.getTotalSize(), "test".getBytes(Encoding).length);
                            cursor.asyncReadEntries(2, new ReadEntriesCallback() {

                                @Override
                                public void readEntriesComplete(List<Entry> entries, Object ctx) {
                                    ManagedCursor cursor = (ManagedCursor) ctx;
                                    assertEquals(entries.size(), 1);
                                    Entry entry = entries.get(0);
                                    final Position position = entry.getPosition();
                                    assertEquals(new String(entry.getDataAndRelease(), Encoding), "test");
                                    log.debug("Mark-Deleting to position {}", position);
                                    cursor.asyncMarkDelete(position, new MarkDeleteCallback() {

                                        @Override
                                        public void markDeleteComplete(Object ctx) {
                                            log.debug("Mark delete complete");
                                            ManagedCursor cursor = (ManagedCursor) ctx;
                                            assertEquals(cursor.hasMoreEntries(), false);
                                            counter.countDown();
                                        }

                                        @Override
                                        public void markDeleteFailed(ManagedLedgerException exception, Object ctx) {
                                            fail(exception.getMessage());
                                        }
                                    }, cursor);
                                }

                                @Override
                                public void readEntriesFailed(ManagedLedgerException exception, Object ctx) {
                                    fail(exception.getMessage());
                                }
                            }, cursor);
                        }

                        @Override
                        public void addFailed(ManagedLedgerException exception, Object ctx) {
                            fail(exception.getMessage());
                        }
                    }, new Pair<ManagedLedger, ManagedCursor>(ledger, cursor));
                }

                @Override
                public void openCursorFailed(ManagedLedgerException exception, Object ctx) {
                    fail(exception.getMessage());
                }
            }, ledger);
        }

        @Override
        public void openLedgerFailed(ManagedLedgerException exception, Object ctx) {
            fail(exception.getMessage());
        }
    }, null);
    counter.await();
    log.info("Test completed");
}
Also used : ReadEntriesCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.ReadEntriesCallback) Position(org.apache.bookkeeper.mledger.Position) InitialPosition(org.apache.pulsar.common.api.proto.PulsarApi.CommandSubscribe.InitialPosition) ManagedLedger(org.apache.bookkeeper.mledger.ManagedLedger) CountDownLatch(java.util.concurrent.CountDownLatch) ManagedCursor(org.apache.bookkeeper.mledger.ManagedCursor) MarkDeleteCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.MarkDeleteCallback) OpenCursorCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.OpenCursorCallback) Entry(org.apache.bookkeeper.mledger.Entry) ManagedLedgerException(org.apache.bookkeeper.mledger.ManagedLedgerException) List(java.util.List) ArrayList(java.util.ArrayList) ManagedLedgerConfig(org.apache.bookkeeper.mledger.ManagedLedgerConfig) OpenLedgerCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.OpenLedgerCallback) AddEntryCallback(org.apache.bookkeeper.mledger.AsyncCallbacks.AddEntryCallback) Pair(org.apache.bookkeeper.mledger.util.Pair) ByteBufPair(org.apache.pulsar.common.api.ByteBufPair) Test(org.testng.annotations.Test)

Example 65 with ManagedLedgerConfig

use of org.apache.bookkeeper.mledger.ManagedLedgerConfig in project incubator-pulsar by apache.

the class ManagedLedgerTest method readFromOlderLedger.

@Test(timeOut = 20000)
public void readFromOlderLedger() throws Exception {
    ManagedLedgerConfig config = new ManagedLedgerConfig().setMaxEntriesPerLedger(1);
    ManagedLedger ledger = factory.open("my_test_ledger", config);
    ManagedCursor cursor = ledger.openCursor("test");
    ledger.addEntry("entry-1".getBytes(Encoding));
    ledger.addEntry("entry-2".getBytes(Encoding));
    assertEquals(cursor.hasMoreEntries(), true);
}
Also used : ManagedLedger(org.apache.bookkeeper.mledger.ManagedLedger) ManagedLedgerConfig(org.apache.bookkeeper.mledger.ManagedLedgerConfig) ManagedCursor(org.apache.bookkeeper.mledger.ManagedCursor) Test(org.testng.annotations.Test)

Aggregations

ManagedLedgerConfig (org.apache.bookkeeper.mledger.ManagedLedgerConfig)194 Test (org.testng.annotations.Test)182 ManagedLedger (org.apache.bookkeeper.mledger.ManagedLedger)159 ManagedCursor (org.apache.bookkeeper.mledger.ManagedCursor)147 Position (org.apache.bookkeeper.mledger.Position)85 ManagedLedgerException (org.apache.bookkeeper.mledger.ManagedLedgerException)77 Entry (org.apache.bookkeeper.mledger.Entry)63 ManagedLedgerFactory (org.apache.bookkeeper.mledger.ManagedLedgerFactory)54 CountDownLatch (java.util.concurrent.CountDownLatch)52 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)33 CyclicBarrier (java.util.concurrent.CyclicBarrier)20 ManagedLedgerFactoryConfig (org.apache.bookkeeper.mledger.ManagedLedgerFactoryConfig)19 AddEntryCallback (org.apache.bookkeeper.mledger.AsyncCallbacks.AddEntryCallback)18 List (java.util.List)17 AtomicReference (java.util.concurrent.atomic.AtomicReference)16 LedgerEntry (org.apache.bookkeeper.client.LedgerEntry)16 TimeUnit (java.util.concurrent.TimeUnit)15 AsyncCallbacks (org.apache.bookkeeper.mledger.AsyncCallbacks)15 BKException (org.apache.bookkeeper.client.BKException)14 Lists (com.google.common.collect.Lists)13