Examples with Flusher org.komamitsu.fluency.flusher.Flusher used on opensource projects

Example 6 with Flusher

use of org.komamitsu.fluency.flusher.Flusher in project fluency by komamitsu.

the class FluencyTest method testWaitUntilFlushingAllBuffer.

@ParameterizedTest
@CsvSource({ "1, false", "3, true" })
void testWaitUntilFlushingAllBuffer(int waitUntilFlusherTerm, boolean expected) throws IOException, InterruptedException {
    flusherConfig.setFlushAttemptIntervalMillis(2000);
    Buffer buffer = new Buffer(bufferConfig, new JsonRecordFormatter());
    Flusher flusher = new Flusher(flusherConfig, buffer, ingester);
    try (Fluency fluency = new Fluency(buffer, flusher)) {
        fluency.emit("foo.bar", new HashMap<>());
        assertThat(fluency.waitUntilAllBufferFlushed(waitUntilFlusherTerm), is(expected));
    }
}

Example 7 with Flusher

use of org.komamitsu.fluency.flusher.Flusher in project fluency by komamitsu.

the class FluencyTest method testWaitUntilFlusherTerminated.

@ParameterizedTest
@CsvSource({ "1, false", "3, true" })
void testWaitUntilFlusherTerminated(int waitUntilFlusherTerm, boolean expected) throws IOException, InterruptedException {
    flusherConfig.setWaitUntilTerminated(1);
    // Wait before actually closing in Buffer
    int waitBeforeCloseMillis = 2000;
    Buffer buffer = spy(new Buffer(bufferConfig, new JsonRecordFormatter()));
    doAnswer((invocation) -> {
        long start = System.currentTimeMillis();
        try {
            TimeUnit.MILLISECONDS.sleep(waitBeforeCloseMillis);
        } catch (InterruptedException e) {
            long rest = waitBeforeCloseMillis - (System.currentTimeMillis() - start);
            if (rest > 0) {
                try {
                    TimeUnit.MILLISECONDS.sleep(rest);
                } catch (InterruptedException e1) {
                }
            }
        }
        return null;
    }).doCallRealMethod().when(buffer).close();
    Flusher flusher = new Flusher(flusherConfig, buffer, ingester);
    Fluency fluency = new Fluency(buffer, flusher);
    fluency.emit("foo.bar", new HashMap<>());
    fluency.close();
    assertThat(fluency.waitUntilFlusherTerminated(waitUntilFlusherTerm), is(expected));
}

Example 8 with Flusher

use of org.komamitsu.fluency.flusher.Flusher in project fluency by komamitsu.

the class FluencyTest method testBufferFullException.

@Test
public void testBufferFullException() throws IOException {
    final CountDownLatch latch = new CountDownLatch(1);
    StuckIngester stuckIngester = new StuckIngester(latch);
    bufferConfig.setChunkInitialSize(64);
    bufferConfig.setChunkExpandRatio(2);
    bufferConfig.setMaxBufferSize(256);
    bufferConfig.setChunkRetentionSize(196);
    flusherConfig.setFlushAttemptIntervalMillis(1000);
    Buffer buffer = new Buffer(bufferConfig, new JsonRecordFormatter());
    Flusher flusher = new Flusher(flusherConfig, buffer, stuckIngester);
    try (Fluency fluency = new Fluency(buffer, flusher)) {
        Map<String, Object> event = new HashMap<>();
        // '{"name":"xxxx"}' (length: 15 bytes)
        event.put("name", "xxxx");
        // 15 * (8 + 1) = 135
        for (int i = 0; i < 8; i++) {
            fluency.emit("tag", event);
        }
        try {
            fluency.emit("tag", event);
            fail();
        } catch (BufferFullException e) {
            assertTrue(true);
        } finally {
            latch.countDown();
        }
    }
}

Example 9 with Flusher

use of org.komamitsu.fluency.flusher.Flusher in project fluency by komamitsu.

the class FluencyBuilder method createFluency.

@VisibleForTesting
public Fluency createFluency(RecordFormatter recordFormatter, Ingester ingester, Buffer.Config bufferConfig, Flusher.Config flusherConfig) {
    Buffer buffer = new Buffer(bufferConfig, recordFormatter);
    Flusher flusher = new Flusher(flusherConfig, buffer, ingester);
    return new Fluency(buffer, flusher);
}