Search in sources :

Example 1 with PartitionRuntimeInformation

use of com.microsoft.azure.eventhubs.PartitionRuntimeInformation in project samza by apache.

the class EventHubSystemAdmin method getPartitionMetadata.

private Map<Partition, SystemStreamPartitionMetadata> getPartitionMetadata(String streamName, String[] partitionIds) {
    EventHubClientManager eventHubClientManager = getOrCreateStreamEventHubClient(streamName);
    Map<Partition, SystemStreamPartitionMetadata> sspMetadataMap = new HashMap<>();
    List<CompletableFuture<PartitionRuntimeInformation>> futureList = new ArrayList<>();
    for (String partition : partitionIds) {
        CompletableFuture<PartitionRuntimeInformation> partitionRuntimeInfo = eventHubClientManager.getEventHubClient().getPartitionRuntimeInformation(partition);
        futureList.add(partitionRuntimeInfo);
        partitionRuntimeInfo.thenAccept(ehPartitionInfo -> {
            LOG.info(printPartitionRuntimeInfo(ehPartitionInfo));
            // Set offsets
            String startingOffset = EventHubSystemConsumer.START_OF_STREAM;
            String newestOffset = ehPartitionInfo.getLastEnqueuedOffset();
            String upcomingOffset = EventHubSystemConsumer.END_OF_STREAM;
            SystemStreamPartitionMetadata sspMetadata = new SystemStreamPartitionMetadata(startingOffset, newestOffset, upcomingOffset);
            sspMetadataMap.put(new Partition(Integer.parseInt(partition)), sspMetadata);
        });
    }
    CompletableFuture<Void> futureGroup = CompletableFuture.allOf(futureList.toArray(new CompletableFuture[futureList.size()]));
    long timeoutMs = eventHubConfig.getRuntimeInfoWaitTimeMS(systemName);
    try {
        futureGroup.get(timeoutMs, TimeUnit.MILLISECONDS);
    } catch (InterruptedException | ExecutionException | TimeoutException e) {
        String msg = String.format("Error while fetching EventHubPartitionRuntimeInfo for System:%s, Stream:%s", systemName, streamName);
        LOG.error(msg, e);
        throw new SamzaException(msg, e);
    }
    return sspMetadataMap;
}
Also used : SystemStreamPartition(org.apache.samza.system.SystemStreamPartition) Partition(org.apache.samza.Partition) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) SystemStreamPartitionMetadata(org.apache.samza.system.SystemStreamMetadata.SystemStreamPartitionMetadata) SamzaException(org.apache.samza.SamzaException) CompletableFuture(java.util.concurrent.CompletableFuture) EventHubClientManager(org.apache.samza.system.eventhub.EventHubClientManager) PartitionRuntimeInformation(com.microsoft.azure.eventhubs.PartitionRuntimeInformation) ExecutionException(java.util.concurrent.ExecutionException) TimeoutException(java.util.concurrent.TimeoutException)

Aggregations

PartitionRuntimeInformation (com.microsoft.azure.eventhubs.PartitionRuntimeInformation)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 CompletableFuture (java.util.concurrent.CompletableFuture)1 ExecutionException (java.util.concurrent.ExecutionException)1 TimeoutException (java.util.concurrent.TimeoutException)1 Partition (org.apache.samza.Partition)1 SamzaException (org.apache.samza.SamzaException)1 SystemStreamPartitionMetadata (org.apache.samza.system.SystemStreamMetadata.SystemStreamPartitionMetadata)1 SystemStreamPartition (org.apache.samza.system.SystemStreamPartition)1 EventHubClientManager (org.apache.samza.system.eventhub.EventHubClientManager)1