Search in sources :

Example 26 with PravegaNodeUri

use of io.pravega.shared.protocol.netty.PravegaNodeUri in project pravega by pravega.

the class ReaderGroupStateManagerTest method testSegmentMerge.

@Test(timeout = 20000)
public void testSegmentMerge() throws ReinitializationRequiredException {
    String scope = "scope";
    String stream = "stream";
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    Segment initialSegmentA = new Segment(scope, stream, 0);
    Segment initialSegmentB = new Segment(scope, stream, 1);
    Segment successor = new Segment(scope, stream, 2);
    MockController controller = new MockControllerWithSuccessors(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, new StreamSegmentsWithPredecessors(Collections.singletonMap(new SegmentWithRange(successor, 0.0, 1.0), ImmutableList.of(0, 1)), ""));
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    @Cleanup ClientFactory clientFactory = new ClientFactoryImpl(scope, controller, connectionFactory, streamFactory, streamFactory, streamFactory);
    SynchronizerConfig config = SynchronizerConfig.builder().build();
    @Cleanup StateSynchronizer<ReaderGroupState> stateSynchronizer = createState(stream, clientFactory, config);
    Map<Segment, Long> segments = new HashMap<>();
    segments.put(initialSegmentA, 1L);
    segments.put(initialSegmentB, 2L);
    ReaderGroupStateManager.initializeReaderGroup(stateSynchronizer, ReaderGroupConfig.builder().stream(Stream.of(scope, stream)).build(), segments);
    val readerState = new ReaderGroupStateManager("testReader", stateSynchronizer, controller, null);
    readerState.initializeReader(0);
    Map<Segment, Long> newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertEquals(2, newSegments.size());
    assertEquals(Long.valueOf(1), newSegments.get(initialSegmentA));
    assertEquals(Long.valueOf(2), newSegments.get(initialSegmentB));
    readerState.handleEndOfSegment(initialSegmentA);
    newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertTrue(newSegments.isEmpty());
    readerState.handleEndOfSegment(initialSegmentB);
    newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertEquals(1, newSegments.size());
    assertEquals(Long.valueOf(0), newSegments.get(successor));
    newSegments = readerState.acquireNewSegmentsIfNeeded(0);
    assertTrue(newSegments.isEmpty());
}
Also used : lombok.val(lombok.val) HashMap(java.util.HashMap) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) ClientFactory(io.pravega.client.ClientFactory) Cleanup(lombok.Cleanup) Segment(io.pravega.client.segment.impl.Segment) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) AtomicLong(java.util.concurrent.atomic.AtomicLong) MockController(io.pravega.client.stream.mock.MockController) SynchronizerConfig(io.pravega.client.state.SynchronizerConfig) Test(org.junit.Test)

Example 27 with PravegaNodeUri

use of io.pravega.shared.protocol.netty.PravegaNodeUri in project pravega by pravega.

the class MockController method deleteStream.

@Override
@Synchronized
public CompletableFuture<Boolean> deleteStream(String scope, String streamName) {
    Stream stream = new StreamImpl(scope, streamName);
    if (createdStreams.get(stream) == null) {
        return CompletableFuture.completedFuture(false);
    }
    for (Segment segment : getSegmentsForStream(stream)) {
        deleteSegment(segment.getScopedName(), new PravegaNodeUri(endpoint, port));
    }
    createdStreams.remove(stream);
    createdScopes.get(scope).remove(stream);
    return CompletableFuture.completedFuture(true);
}
Also used : PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) StreamImpl(io.pravega.client.stream.impl.StreamImpl) Stream(io.pravega.client.stream.Stream) CreateSegment(io.pravega.shared.protocol.netty.WireCommands.CreateSegment) Segment(io.pravega.client.segment.impl.Segment) DeleteSegment(io.pravega.shared.protocol.netty.WireCommands.DeleteSegment) Synchronized(lombok.Synchronized)

Example 28 with PravegaNodeUri

use of io.pravega.shared.protocol.netty.PravegaNodeUri in project pravega by pravega.

the class MockController method sendRequestOverNewConnection.

private <T> void sendRequestOverNewConnection(WireCommand request, ReplyProcessor replyProcessor, CompletableFuture<T> resultFuture) {
    ClientConnection connection = getAndHandleExceptions(connectionFactory.establishConnection(new PravegaNodeUri(endpoint, port), replyProcessor), RuntimeException::new);
    resultFuture.whenComplete((result, e) -> {
        connection.close();
    });
    try {
        connection.send(request);
    } catch (Exception e) {
        resultFuture.completeExceptionally(e);
    }
}
Also used : PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) ClientConnection(io.pravega.client.netty.impl.ClientConnection) AuthenticationException(io.pravega.common.auth.AuthenticationException) TxnFailedException(io.pravega.client.stream.TxnFailedException) ConnectionClosedException(io.pravega.client.stream.impl.ConnectionClosedException)

Example 29 with PravegaNodeUri

use of io.pravega.shared.protocol.netty.PravegaNodeUri in project pravega by pravega.

the class SegmentHelper method sendRequestAsync.

private <ResultT> void sendRequestAsync(final WireCommand request, final ReplyProcessor replyProcessor, final CompletableFuture<ResultT> resultFuture, final ConnectionFactory connectionFactory, final PravegaNodeUri uri) {
    CompletableFuture<ClientConnection> connectionFuture = connectionFactory.establishConnection(uri, replyProcessor);
    connectionFuture.whenComplete((connection, e) -> {
        if (connection == null) {
            resultFuture.completeExceptionally(new WireCommandFailedException(new ConnectionFailedException(e), request.getType(), WireCommandFailedException.Reason.ConnectionFailed));
        } else {
            try {
                connection.send(request);
            } catch (ConnectionFailedException cfe) {
                throw new WireCommandFailedException(cfe, request.getType(), WireCommandFailedException.Reason.ConnectionFailed);
            } catch (Exception e2) {
                throw new RuntimeException(e2);
            }
        }
    }).exceptionally(e -> {
        Throwable cause = Exceptions.unwrap(e);
        if (cause instanceof WireCommandFailedException) {
            resultFuture.completeExceptionally(cause);
        } else if (cause instanceof ConnectionFailedException) {
            resultFuture.completeExceptionally(new WireCommandFailedException(cause, request.getType(), WireCommandFailedException.Reason.ConnectionFailed));
        } else {
            resultFuture.completeExceptionally(new RuntimeException(cause));
        }
        return null;
    });
    resultFuture.whenComplete((result, e) -> {
        connectionFuture.thenAccept(ClientConnection::close);
    });
}
Also used : ClientConnection(io.pravega.client.netty.impl.ClientConnection) AuthenticationException(io.pravega.common.auth.AuthenticationException) Segment(io.pravega.client.segment.impl.Segment) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Exceptions(io.pravega.common.Exceptions) CompletableFuture(java.util.concurrent.CompletableFuture) Supplier(java.util.function.Supplier) FailingReplyProcessor(io.pravega.shared.protocol.netty.FailingReplyProcessor) Pair(org.apache.commons.lang3.tuple.Pair) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) ConnectionFactory(io.pravega.client.netty.impl.ConnectionFactory) Controller(io.pravega.controller.stream.api.grpc.v1.Controller) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) ModelHelper(io.pravega.client.stream.impl.ModelHelper) Host(io.pravega.common.cluster.Host) UUID(java.util.UUID) WireCommands(io.pravega.shared.protocol.netty.WireCommands) ImmutablePair(org.apache.commons.lang3.tuple.ImmutablePair) WireCommand(io.pravega.shared.protocol.netty.WireCommand) AtomicLong(java.util.concurrent.atomic.AtomicLong) Slf4j(lombok.extern.slf4j.Slf4j) HostControllerStore(io.pravega.controller.store.host.HostControllerStore) TxnStatus(io.pravega.controller.stream.api.grpc.v1.Controller.TxnStatus) ScalingPolicy(io.pravega.client.stream.ScalingPolicy) ReplyProcessor(io.pravega.shared.protocol.netty.ReplyProcessor) ClientConnection(io.pravega.client.netty.impl.ClientConnection) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) AuthenticationException(io.pravega.common.auth.AuthenticationException) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException)

Example 30 with PravegaNodeUri

use of io.pravega.shared.protocol.netty.PravegaNodeUri in project pravega by pravega.

the class ControllerImpl method getEndpointForSegment.

@Override
public CompletableFuture<PravegaNodeUri> getEndpointForSegment(final String qualifiedSegmentName) {
    Exceptions.checkNotClosed(closed.get(), this);
    Exceptions.checkNotNullOrEmpty(qualifiedSegmentName, "qualifiedSegmentName");
    long traceId = LoggerHelpers.traceEnter(log, "getEndpointForSegment", qualifiedSegmentName);
    final CompletableFuture<NodeUri> result = this.retryConfig.runAsync(() -> {
        RPCAsyncCallback<NodeUri> callback = new RPCAsyncCallback<>();
        Segment segment = Segment.fromScopedName(qualifiedSegmentName);
        client.getURI(ModelHelper.createSegmentId(segment.getScope(), segment.getStreamName(), segment.getSegmentNumber()), callback);
        return callback.getFuture();
    }, this.executor);
    return result.thenApply(ModelHelper::encode).whenComplete((x, e) -> {
        if (e != null) {
            log.warn("getEndpointForSegment failed: ", e);
        }
        LoggerHelpers.traceLeave(log, "getEndpointForSegment", traceId);
    });
}
Also used : PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) NodeUri(io.pravega.controller.stream.api.grpc.v1.Controller.NodeUri) Segment(io.pravega.client.segment.impl.Segment)

Aggregations

PravegaNodeUri (io.pravega.shared.protocol.netty.PravegaNodeUri)61 Test (org.junit.Test)54 MockController (io.pravega.client.stream.mock.MockController)48 MockConnectionFactoryImpl (io.pravega.client.stream.mock.MockConnectionFactoryImpl)47 Cleanup (lombok.Cleanup)42 ClientConnection (io.pravega.client.netty.impl.ClientConnection)33 UUID (java.util.UUID)24 WireCommands (io.pravega.shared.protocol.netty.WireCommands)22 SetupAppend (io.pravega.shared.protocol.netty.WireCommands.SetupAppend)20 Segment (io.pravega.client.segment.impl.Segment)17 AppendSetup (io.pravega.shared.protocol.netty.WireCommands.AppendSetup)17 InlineExecutor (io.pravega.test.common.InlineExecutor)17 CompletableFuture (java.util.concurrent.CompletableFuture)17 ClientFactory (io.pravega.client.ClientFactory)15 SynchronizerConfig (io.pravega.client.state.SynchronizerConfig)15 PendingEvent (io.pravega.client.stream.impl.PendingEvent)15 MockSegmentStreamFactory (io.pravega.client.stream.mock.MockSegmentStreamFactory)15 InOrder (org.mockito.InOrder)14 Append (io.pravega.shared.protocol.netty.Append)13 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)13