Examples with JsonRecordFormatter org.komamitsu.fluency.JsonRecordFormatter used on opensource projects

Example 1 with JsonRecordFormatter

use of org.komamitsu.fluency.JsonRecordFormatter in project fluency by komamitsu.

the class BufferTest method setUp.

@BeforeEach
void setUp() {
    recordFormatter = new JsonRecordFormatter();
    ingester = mock(Ingester.class);
    bufferConfig = new Buffer.Config();
}

Example 2 with JsonRecordFormatter

use of org.komamitsu.fluency.JsonRecordFormatter in project fluency by komamitsu.

the class FlusherTest method testAsyncFlusher.

@Test
void testAsyncFlusher() throws IOException, InterruptedException {
    flusherConfig.setFlushAttemptIntervalMillis(500);
    Buffer buffer = spy(new Buffer(bufferConfig, new JsonRecordFormatter()));
    Flusher flusher = new Flusher(flusherConfig, buffer, ingester);
    verify(buffer, times(0)).flush(eq(ingester), anyBoolean());
    verify(buffer, times(0)).flush(eq(ingester), anyBoolean());
    flusher.flush();
    TimeUnit.MILLISECONDS.sleep(50);
    verify(buffer, times(0)).flush(eq(ingester), eq(false));
    verify(buffer, times(1)).flush(eq(ingester), eq(true));
    TimeUnit.SECONDS.sleep(1);
    verify(buffer, atLeast(1)).flush(eq(ingester), eq(false));
    verify(buffer, atMost(3)).flush(eq(ingester), eq(false));
    verify(buffer, times(1)).flush(eq(ingester), eq(true));
    verify(buffer, times(0)).close();
    flusher.close();
    verify(buffer, times(1)).close();
    verify(buffer, atLeast(2)).flush(eq(ingester), eq(false));
    verify(buffer, atMost(3)).flush(eq(ingester), eq(false));
    verify(buffer, atLeast(2)).flush(eq(ingester), eq(true));
    verify(buffer, atMost(3)).flush(eq(ingester), eq(true));
}

Example 3 with JsonRecordFormatter

use of org.komamitsu.fluency.JsonRecordFormatter in project fluency by komamitsu.

the class FlusherTest method queueSizeShouldNotExceedLimit.

@Test
void queueSizeShouldNotExceedLimit() throws Exception {
    flusherConfig.setFlushAttemptIntervalMillis(200);
    Buffer buffer = new Buffer(bufferConfig, new JsonRecordFormatter());
    Flusher flusher = new Flusher(flusherConfig, buffer, ingester);
    BlockingQueue<Boolean> eventQueue = (BlockingQueue<Boolean>) extractValue(flusher, "eventQueue");
    int nonZeroCounts = 0;
    for (int index = 0; index < 10_000; index++) {
        flusher.flush();
        if (!eventQueue.isEmpty()) {
            nonZeroCounts++;
        }
        // The eventQueue will always have less that 16 elements (the default max size)
        assertTrue(eventQueue.size() <= 16);
    }
    // The eventQueue will be non empty at least a couple of times
    assertTrue(nonZeroCounts > 0);
    // Wait for sufficiently long (amount of time > queue poll wait) and the queue should be polled completely to become empty
    Thread.sleep(1000);
    assertEquals(0, eventQueue.size());
}