Search in sources :

Example 6 with BatchClientFactory

use of io.pravega.client.BatchClientFactory in project pravega by pravega.

the class BatchClientSimpleTest method readFromRanges.

// Start utils region
private int readFromRanges(List<SegmentRange> ranges, BatchClientFactory batchClient) {
    List<CompletableFuture<Integer>> eventCounts = ranges.parallelStream().map(range -> CompletableFuture.supplyAsync(() -> batchClient.readSegment(range, new JavaSerializer<>())).thenApplyAsync(segmentIterator -> {
        log.debug("Thread " + Thread.currentThread().getId() + " reading events.");
        int numEvents = Lists.newArrayList(segmentIterator).size();
        segmentIterator.close();
        return numEvents;
    })).collect(Collectors.toList());
    return eventCounts.stream().map(CompletableFuture::join).mapToInt(Integer::intValue).sum();
}
Also used : StreamCut(io.pravega.client.stream.StreamCut) MarathonException(mesosphere.marathon.client.MarathonException) ConnectionFactory(io.pravega.client.connection.impl.ConnectionFactory) StreamManager(io.pravega.client.admin.StreamManager) RunWith(org.junit.runner.RunWith) Cleanup(lombok.Cleanup) CompletableFuture(java.util.concurrent.CompletableFuture) ReaderGroup(io.pravega.client.stream.ReaderGroup) JavaSerializer(io.pravega.client.stream.impl.JavaSerializer) StreamConfiguration(io.pravega.client.stream.StreamConfiguration) Service(io.pravega.test.system.framework.services.Service) Lists(com.google.common.collect.Lists) SegmentRange(io.pravega.client.batch.SegmentRange) ClientFactoryImpl(io.pravega.client.stream.impl.ClientFactoryImpl) ReaderGroupManager(io.pravega.client.admin.ReaderGroupManager) Stream(io.pravega.client.stream.Stream) After(org.junit.After) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Timeout(org.junit.rules.Timeout) Checkpoint(io.pravega.client.stream.Checkpoint) URI(java.net.URI) Utils(io.pravega.test.system.framework.Utils) SocketConnectionFactoryImpl(io.pravega.client.connection.impl.SocketConnectionFactoryImpl) BatchClientFactory(io.pravega.client.BatchClientFactory) ReaderGroupConfig(io.pravega.client.stream.ReaderGroupConfig) Before(org.junit.Before) Environment(io.pravega.test.system.framework.Environment) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) StreamInfo(io.pravega.client.admin.StreamInfo) Collectors(java.util.stream.Collectors) ControllerImplConfig(io.pravega.client.control.impl.ControllerImplConfig) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) Rule(org.junit.Rule) ControllerImpl(io.pravega.client.control.impl.ControllerImpl) ExecutorServiceHelpers(io.pravega.common.concurrent.ExecutorServiceHelpers) RandomFactory(io.pravega.common.hash.RandomFactory) ScalingPolicy(io.pravega.client.stream.ScalingPolicy) Futures(io.pravega.common.concurrent.Futures) SystemTestRunner(io.pravega.test.system.framework.SystemTestRunner) Assert.assertEquals(org.junit.Assert.assertEquals) ClientConfig(io.pravega.client.ClientConfig) CompletableFuture(java.util.concurrent.CompletableFuture) Checkpoint(io.pravega.client.stream.Checkpoint)

Aggregations

BatchClientFactory (io.pravega.client.BatchClientFactory)6 SegmentRange (io.pravega.client.batch.SegmentRange)6 Cleanup (lombok.Cleanup)6 Test (org.junit.Test)5 StreamCut (io.pravega.client.stream.StreamCut)4 ClientConfig (io.pravega.client.ClientConfig)3 EventStreamClientFactory (io.pravega.client.EventStreamClientFactory)3 StreamInfo (io.pravega.client.admin.StreamInfo)3 Segment (io.pravega.client.segment.impl.Segment)3 ReaderGroupManager (io.pravega.client.admin.ReaderGroupManager)2 StreamManager (io.pravega.client.admin.StreamManager)2 ConnectionFactory (io.pravega.client.connection.impl.ConnectionFactory)2 SocketConnectionFactoryImpl (io.pravega.client.connection.impl.SocketConnectionFactoryImpl)2 ControllerImpl (io.pravega.client.control.impl.ControllerImpl)2 Checkpoint (io.pravega.client.stream.Checkpoint)2 ReaderGroup (io.pravega.client.stream.ReaderGroup)2 Stream (io.pravega.client.stream.Stream)2 ClientFactoryImpl (io.pravega.client.stream.impl.ClientFactoryImpl)2 StreamCutImpl (io.pravega.client.stream.impl.StreamCutImpl)2 ArrayList (java.util.ArrayList)2