Search in sources :

Example 1 with ResetDataSourceMetadataAction

use of io.druid.indexing.common.actions.ResetDataSourceMetadataAction in project druid by druid-io.

the class KafkaIndexTask method sendResetRequestAndWait.

private void sendResetRequestAndWait(Map<TopicPartition, Long> outOfRangePartitions, TaskToolbox taskToolbox) throws IOException {
    Map<Integer, Long> partitionOffsetMap = Maps.newHashMap();
    for (Map.Entry<TopicPartition, Long> outOfRangePartition : outOfRangePartitions.entrySet()) {
        partitionOffsetMap.put(outOfRangePartition.getKey().partition(), outOfRangePartition.getValue());
    }
    boolean result = taskToolbox.getTaskActionClient().submit(new ResetDataSourceMetadataAction(getDataSource(), new KafkaDataSourceMetadata(new KafkaPartitions(ioConfig.getStartPartitions().getTopic(), partitionOffsetMap))));
    if (result) {
        log.makeAlert("Resetting Kafka offsets for datasource [%s]", getDataSource()).addData("partitions", partitionOffsetMap.keySet()).emit();
        // wait for being killed by supervisor
        try {
            Thread.sleep(Long.MAX_VALUE);
        } catch (InterruptedException e) {
            throw new RuntimeException("Got interrupted while waiting to be killed");
        }
    } else {
        log.makeAlert("Failed to send reset request for partitions [%s]", partitionOffsetMap.keySet()).emit();
    }
}
Also used : TopicPartition(org.apache.kafka.common.TopicPartition) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) ResetDataSourceMetadataAction(io.druid.indexing.common.actions.ResetDataSourceMetadataAction)

Aggregations

ImmutableMap (com.google.common.collect.ImmutableMap)1 ResetDataSourceMetadataAction (io.druid.indexing.common.actions.ResetDataSourceMetadataAction)1 Map (java.util.Map)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 TopicPartition (org.apache.kafka.common.TopicPartition)1