Search in sources :

Example 76 with DistributedLogManager

use of org.apache.distributedlog.api.DistributedLogManager in project incubator-pulsar by apache.

the class DLInputStreamTest method testRead.

/**
 * Test Case: read records from the input stream.
 */
@Test
public void testRead() throws Exception {
    DistributedLogManager dlm = mock(DistributedLogManager.class);
    LogReader reader = mock(LogReader.class);
    when(dlm.getInputStream(any(DLSN.class))).thenReturn(reader);
    byte[] data = "test-read".getBytes(UTF_8);
    LogRecordWithDLSN record = mock(LogRecordWithDLSN.class);
    when(record.getPayLoadInputStream()).thenReturn(new ByteArrayInputStream(data));
    when(reader.readNext(anyBoolean())).thenReturn(record).thenThrow(new EndOfStreamException("eos"));
    DLInputStream in = new DLInputStream(dlm);
    int numReads = 0;
    int readByte;
    while ((readByte = in.read()) != -1) {
        assertEquals(data[numReads], readByte);
        ++numReads;
    }
    assertEquals(data.length, numReads);
}
Also used : LogRecordWithDLSN(org.apache.distributedlog.LogRecordWithDLSN) DLSN(org.apache.distributedlog.DLSN) LogRecordWithDLSN(org.apache.distributedlog.LogRecordWithDLSN) ByteArrayInputStream(java.io.ByteArrayInputStream) DistributedLogManager(org.apache.distributedlog.api.DistributedLogManager) EndOfStreamException(org.apache.distributedlog.exceptions.EndOfStreamException) LogReader(org.apache.distributedlog.api.LogReader) Test(org.testng.annotations.Test)

Example 77 with DistributedLogManager

use of org.apache.distributedlog.api.DistributedLogManager in project incubator-pulsar by apache.

the class DLOutputStreamTest method testClose.

/**
 * Test Case: close output stream.
 */
@Test
public void testClose() throws Exception {
    DistributedLogManager dlm = mock(DistributedLogManager.class);
    AppendOnlyStreamWriter writer = mock(AppendOnlyStreamWriter.class);
    DLOutputStream out = new DLOutputStream(dlm, writer);
    out.close();
    verify(writer, times(1)).markEndOfStream();
    verify(writer, times(1)).close();
    verify(dlm, times(1)).close();
}
Also used : DistributedLogManager(org.apache.distributedlog.api.DistributedLogManager) AppendOnlyStreamWriter(org.apache.distributedlog.AppendOnlyStreamWriter) Test(org.testng.annotations.Test)

Example 78 with DistributedLogManager

use of org.apache.distributedlog.api.DistributedLogManager in project incubator-pulsar by apache.

the class DLOutputStreamTest method testFlush.

/**
 * Test Case: flush should force writing the data.
 */
@Test
public void testFlush() throws Exception {
    DistributedLogManager dlm = mock(DistributedLogManager.class);
    AppendOnlyStreamWriter writer = mock(AppendOnlyStreamWriter.class);
    DLOutputStream out = new DLOutputStream(dlm, writer);
    out.flush();
    verify(writer, times(1)).force(eq(false));
}
Also used : DistributedLogManager(org.apache.distributedlog.api.DistributedLogManager) AppendOnlyStreamWriter(org.apache.distributedlog.AppendOnlyStreamWriter) Test(org.testng.annotations.Test)

Example 79 with DistributedLogManager

use of org.apache.distributedlog.api.DistributedLogManager in project incubator-pulsar by apache.

the class DLOutputStreamTest method testWrite.

/**
 * Test Case: test writing the data.
 */
@Test
public void testWrite() throws Exception {
    DistributedLogManager dlm = mock(DistributedLogManager.class);
    AppendOnlyStreamWriter writer = mock(AppendOnlyStreamWriter.class);
    DLOutputStream out = new DLOutputStream(dlm, writer);
    byte[] data = new byte[16];
    out.write(data);
    verify(writer, times(1)).write(data);
}
Also used : DistributedLogManager(org.apache.distributedlog.api.DistributedLogManager) AppendOnlyStreamWriter(org.apache.distributedlog.AppendOnlyStreamWriter) Test(org.testng.annotations.Test)

Example 80 with DistributedLogManager

use of org.apache.distributedlog.api.DistributedLogManager in project incubator-pulsar by apache.

the class Utils method downloadFromBookkeeper.

public static void downloadFromBookkeeper(Namespace namespace, OutputStream outputStream, String packagePath) throws Exception {
    DistributedLogManager dlm = namespace.openLog(packagePath);
    InputStream in = new DLInputStream(dlm);
    int read = 0;
    byte[] bytes = new byte[1024];
    while ((read = in.read(bytes)) != -1) {
        outputStream.write(bytes, 0, read);
    }
    outputStream.flush();
}
Also used : DLInputStream(org.apache.pulsar.functions.worker.dlog.DLInputStream) DistributedLogManager(org.apache.distributedlog.api.DistributedLogManager) ObjectInputStream(java.io.ObjectInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) DLInputStream(org.apache.pulsar.functions.worker.dlog.DLInputStream) InputStream(java.io.InputStream)

Aggregations

DistributedLogManager (org.apache.distributedlog.api.DistributedLogManager)174 Test (org.junit.Test)139 AsyncLogReader (org.apache.distributedlog.api.AsyncLogReader)34 DynamicDistributedLogConfiguration (org.apache.distributedlog.config.DynamicDistributedLogConfiguration)33 URI (java.net.URI)29 LogReader (org.apache.distributedlog.api.LogReader)26 Namespace (org.apache.distributedlog.api.namespace.Namespace)26 AsyncLogWriter (org.apache.distributedlog.api.AsyncLogWriter)23 CountDownLatch (java.util.concurrent.CountDownLatch)18 DLSN (org.apache.distributedlog.DLSN)17 LogRecordWithDLSN (org.apache.distributedlog.LogRecordWithDLSN)16 EndOfStreamException (org.apache.distributedlog.exceptions.EndOfStreamException)14 IOException (java.io.IOException)13 AppendOnlyStreamWriter (org.apache.distributedlog.AppendOnlyStreamWriter)13 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)12 LogSegmentMetadata (org.apache.distributedlog.LogSegmentMetadata)12 ArrayList (java.util.ArrayList)11 CompletableFuture (java.util.concurrent.CompletableFuture)11 DistributedLogConfiguration (org.apache.distributedlog.DistributedLogConfiguration)11 List (java.util.List)8