Search in sources :

Example 1 with DeletedRecords

use of org.apache.kafka.clients.admin.DeletedRecords in project apache-kafka-on-k8s by banzaicloud.

the class TaskManagerTest method shouldSendPurgeData.

@Test
public void shouldSendPurgeData() {
    final KafkaFutureImpl<DeletedRecords> futureDeletedRecords = new KafkaFutureImpl<>();
    final Map<TopicPartition, RecordsToDelete> recordsToDelete = Collections.singletonMap(t1p1, RecordsToDelete.beforeOffset(5L));
    final DeleteRecordsResult deleteRecordsResult = new DeleteRecordsResult(Collections.singletonMap(t1p1, (KafkaFuture<DeletedRecords>) futureDeletedRecords));
    futureDeletedRecords.complete(null);
    EasyMock.expect(active.recordsToDelete()).andReturn(Collections.singletonMap(t1p1, 5L)).times(2);
    EasyMock.expect(adminClient.deleteRecords(recordsToDelete)).andReturn(deleteRecordsResult).times(2);
    replay();
    taskManager.maybePurgeCommitedRecords();
    taskManager.maybePurgeCommitedRecords();
    verify(active, adminClient);
}
Also used : KafkaFuture(org.apache.kafka.common.KafkaFuture) TopicPartition(org.apache.kafka.common.TopicPartition) DeletedRecords(org.apache.kafka.clients.admin.DeletedRecords) RecordsToDelete(org.apache.kafka.clients.admin.RecordsToDelete) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) DeleteRecordsResult(org.apache.kafka.clients.admin.DeleteRecordsResult) Test(org.junit.Test)

Example 2 with DeletedRecords

use of org.apache.kafka.clients.admin.DeletedRecords in project apache-kafka-on-k8s by banzaicloud.

the class TaskManagerTest method shouldIgnorePurgeDataErrors.

@Test
public void shouldIgnorePurgeDataErrors() {
    final KafkaFutureImpl<DeletedRecords> futureDeletedRecords = new KafkaFutureImpl<>();
    final Map<TopicPartition, RecordsToDelete> recordsToDelete = Collections.singletonMap(t1p1, RecordsToDelete.beforeOffset(5L));
    final DeleteRecordsResult deleteRecordsResult = new DeleteRecordsResult(Collections.singletonMap(t1p1, (KafkaFuture<DeletedRecords>) futureDeletedRecords));
    futureDeletedRecords.completeExceptionally(new Exception("KABOOM!"));
    EasyMock.expect(active.recordsToDelete()).andReturn(Collections.singletonMap(t1p1, 5L)).times(2);
    EasyMock.expect(adminClient.deleteRecords(recordsToDelete)).andReturn(deleteRecordsResult).times(2);
    replay();
    taskManager.maybePurgeCommitedRecords();
    taskManager.maybePurgeCommitedRecords();
    verify(active, adminClient);
}
Also used : KafkaFuture(org.apache.kafka.common.KafkaFuture) TopicPartition(org.apache.kafka.common.TopicPartition) DeletedRecords(org.apache.kafka.clients.admin.DeletedRecords) RecordsToDelete(org.apache.kafka.clients.admin.RecordsToDelete) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) DeleteRecordsResult(org.apache.kafka.clients.admin.DeleteRecordsResult) StreamsException(org.apache.kafka.streams.errors.StreamsException) IOException(java.io.IOException) Test(org.junit.Test)

Example 3 with DeletedRecords

use of org.apache.kafka.clients.admin.DeletedRecords in project apache-kafka-on-k8s by banzaicloud.

the class TaskManagerTest method shouldNotSendPurgeDataIfPreviousNotDone.

@Test
public void shouldNotSendPurgeDataIfPreviousNotDone() {
    final KafkaFuture<DeletedRecords> futureDeletedRecords = new KafkaFutureImpl<>();
    final Map<TopicPartition, RecordsToDelete> recordsToDelete = Collections.singletonMap(t1p1, RecordsToDelete.beforeOffset(5L));
    final DeleteRecordsResult deleteRecordsResult = new DeleteRecordsResult(Collections.singletonMap(t1p1, futureDeletedRecords));
    EasyMock.expect(active.recordsToDelete()).andReturn(Collections.singletonMap(t1p1, 5L)).once();
    EasyMock.expect(adminClient.deleteRecords(recordsToDelete)).andReturn(deleteRecordsResult).once();
    replay();
    taskManager.maybePurgeCommitedRecords();
    // second call should be no-op as the previous one is not done yet
    taskManager.maybePurgeCommitedRecords();
    verify(active, adminClient);
}
Also used : TopicPartition(org.apache.kafka.common.TopicPartition) DeletedRecords(org.apache.kafka.clients.admin.DeletedRecords) RecordsToDelete(org.apache.kafka.clients.admin.RecordsToDelete) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) DeleteRecordsResult(org.apache.kafka.clients.admin.DeleteRecordsResult) Test(org.junit.Test)

Aggregations

DeleteRecordsResult (org.apache.kafka.clients.admin.DeleteRecordsResult)3 DeletedRecords (org.apache.kafka.clients.admin.DeletedRecords)3 RecordsToDelete (org.apache.kafka.clients.admin.RecordsToDelete)3 TopicPartition (org.apache.kafka.common.TopicPartition)3 KafkaFutureImpl (org.apache.kafka.common.internals.KafkaFutureImpl)3 Test (org.junit.Test)3 KafkaFuture (org.apache.kafka.common.KafkaFuture)2 IOException (java.io.IOException)1 StreamsException (org.apache.kafka.streams.errors.StreamsException)1