Examples with KafkaCheckpointManager io.cdap.cdap.logging.meta.KafkaCheckpointManager used on opensource projects

Example 1 with KafkaCheckpointManager

use of io.cdap.cdap.logging.meta.KafkaCheckpointManager in project cdap by caskdata.

the class TestDistributedLogReader method generateCheckpointTime.

private static void generateCheckpointTime(LoggingContext loggingContext, int numExpectedEvents, String kafkaTopic) throws IOException {
    FileLogReader logReader = injector.getInstance(FileLogReader.class);
    List<LogEvent> events = Lists.newArrayList(logReader.getLog(loggingContext, 0, Long.MAX_VALUE, Filter.EMPTY_FILTER));
    Assert.assertEquals(numExpectedEvents, events.size());
    // Save checkpoint (time of last event)
    TransactionRunner transactionRunner = injector.getInstance(TransactionRunner.class);
    CheckpointManager<KafkaOffset> checkpointManager = new KafkaCheckpointManager(transactionRunner, Constants.Logging.SYSTEM_PIPELINE_CHECKPOINT_PREFIX + kafkaTopic);
    long checkpointTime = events.get(numExpectedEvents - 1).getLoggingEvent().getTimeStamp();
    checkpointManager.saveCheckpoints(ImmutableMap.of(stringPartitioner.partition(loggingContext.getLogPartition(), -1), new Checkpoint<>(new KafkaOffset(numExpectedEvents, checkpointTime), checkpointTime)));
}