Search in sources :

Example 31 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class SegmentMetadataClientImpl method updatePropertyAsync.

private CompletableFuture<SegmentAttributeUpdated> updatePropertyAsync(UUID attributeId, long expected, long value) {
    log.trace("Updating segment attribute: {}", attributeId);
    RawClient connection = getConnection();
    long requestId = connection.getFlow().getNextSequenceNumber();
    return tokenProvider.retrieveToken().thenCompose(token -> connection.sendRequest(requestId, new UpdateSegmentAttribute(requestId, segmentId.getScopedName(), attributeId, value, expected, token))).thenApply(r -> transformReply(r, SegmentAttributeUpdated.class));
}
Also used : SneakyThrows(lombok.SneakyThrows) TokenExpiredException(io.pravega.auth.TokenExpiredException) Retry(io.pravega.common.util.Retry) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) Exceptions(io.pravega.common.Exceptions) RequiredArgsConstructor(lombok.RequiredArgsConstructor) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) GetSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.GetSegmentAttribute) GetStreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.GetStreamSegmentInfo) RawClient(io.pravega.client.connection.impl.RawClient) UpdateSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.UpdateSegmentAttribute) AccessOperation(io.pravega.shared.security.auth.AccessOperation) TokenException(io.pravega.auth.TokenException) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) StreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.StreamSegmentInfo) TruncateSegment(io.pravega.shared.protocol.netty.WireCommands.TruncateSegment) RetryWithBackoff(io.pravega.common.util.Retry.RetryWithBackoff) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) lombok.val(lombok.val) CompletionException(java.util.concurrent.CompletionException) DelegationTokenProviderFactory(io.pravega.client.security.auth.DelegationTokenProviderFactory) UUID(java.util.UUID) WireCommands(io.pravega.shared.protocol.netty.WireCommands) GuardedBy(javax.annotation.concurrent.GuardedBy) WrongHost(io.pravega.shared.protocol.netty.WireCommands.WrongHost) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) SegmentAttributeUpdated(io.pravega.shared.protocol.netty.WireCommands.SegmentAttributeUpdated) Slf4j(lombok.extern.slf4j.Slf4j) InvalidTokenException(io.pravega.auth.InvalidTokenException) SegmentTruncated(io.pravega.shared.protocol.netty.WireCommands.SegmentTruncated) SealSegment(io.pravega.shared.protocol.netty.WireCommands.SealSegment) SegmentSealed(io.pravega.shared.protocol.netty.WireCommands.SegmentSealed) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ConnectionClosedException(io.pravega.client.stream.impl.ConnectionClosedException) Controller(io.pravega.client.control.impl.Controller) Futures(io.pravega.common.concurrent.Futures) UpdateSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.UpdateSegmentAttribute) SegmentAttributeUpdated(io.pravega.shared.protocol.netty.WireCommands.SegmentAttributeUpdated) RawClient(io.pravega.client.connection.impl.RawClient)

Example 32 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class ConditionalOutputStreamImpl method closeConnection.

private void closeConnection(String message) {
    if (closed.get()) {
        log.debug("Closing connection as a result of receiving: {} for segment: {} and conditional writerId: {}", message, segmentId, writerId);
    } else {
        log.warn("Closing connection as a result of receiving: {} for segment: {} and conditional writerId: {}", message, segmentId, writerId);
    }
    RawClient c;
    synchronized (lock) {
        c = client;
        client = null;
    }
    if (c != null) {
        try {
            c.close();
        } catch (Exception e) {
            log.warn("Exception tearing down connection for writerId: {} ", writerId, e);
        }
    }
}
Also used : RawClient(io.pravega.client.connection.impl.RawClient) TokenExpiredException(io.pravega.auth.TokenExpiredException) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) AuthenticationException(io.pravega.auth.AuthenticationException)

Example 33 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class SegmentMetadataClientImpl method getStreamSegmentInfo.

@VisibleForTesting
CompletableFuture<StreamSegmentInfo> getStreamSegmentInfo() {
    log.debug("Getting segment info for segment: {}", segmentId);
    RawClient connection = getConnection();
    long requestId = connection.getFlow().getNextSequenceNumber();
    return tokenProvider.retrieveToken().thenCompose(token -> connection.sendRequest(requestId, new GetStreamSegmentInfo(requestId, segmentId.getScopedName(), token))).thenApply(r -> transformReply(r, StreamSegmentInfo.class));
}
Also used : SneakyThrows(lombok.SneakyThrows) TokenExpiredException(io.pravega.auth.TokenExpiredException) Retry(io.pravega.common.util.Retry) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) Exceptions(io.pravega.common.Exceptions) RequiredArgsConstructor(lombok.RequiredArgsConstructor) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) GetSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.GetSegmentAttribute) GetStreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.GetStreamSegmentInfo) RawClient(io.pravega.client.connection.impl.RawClient) UpdateSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.UpdateSegmentAttribute) AccessOperation(io.pravega.shared.security.auth.AccessOperation) TokenException(io.pravega.auth.TokenException) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) StreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.StreamSegmentInfo) TruncateSegment(io.pravega.shared.protocol.netty.WireCommands.TruncateSegment) RetryWithBackoff(io.pravega.common.util.Retry.RetryWithBackoff) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) lombok.val(lombok.val) CompletionException(java.util.concurrent.CompletionException) DelegationTokenProviderFactory(io.pravega.client.security.auth.DelegationTokenProviderFactory) UUID(java.util.UUID) WireCommands(io.pravega.shared.protocol.netty.WireCommands) GuardedBy(javax.annotation.concurrent.GuardedBy) WrongHost(io.pravega.shared.protocol.netty.WireCommands.WrongHost) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) SegmentAttributeUpdated(io.pravega.shared.protocol.netty.WireCommands.SegmentAttributeUpdated) Slf4j(lombok.extern.slf4j.Slf4j) InvalidTokenException(io.pravega.auth.InvalidTokenException) SegmentTruncated(io.pravega.shared.protocol.netty.WireCommands.SegmentTruncated) SealSegment(io.pravega.shared.protocol.netty.WireCommands.SealSegment) SegmentSealed(io.pravega.shared.protocol.netty.WireCommands.SegmentSealed) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ConnectionClosedException(io.pravega.client.stream.impl.ConnectionClosedException) Controller(io.pravega.client.control.impl.Controller) Futures(io.pravega.common.concurrent.Futures) GetStreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.GetStreamSegmentInfo) GetStreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.GetStreamSegmentInfo) StreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.StreamSegmentInfo) RawClient(io.pravega.client.connection.impl.RawClient) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 34 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class AppendTest method appendALotOfData.

@Test(timeout = 100000)
public void appendALotOfData() {
    String endpoint = "localhost";
    String scope = "Scope";
    String streamName = "appendALotOfData";
    int port = TestUtils.getAvailableListenPort();
    long heapSize = Runtime.getRuntime().maxMemory();
    long messageSize = Math.min(1024 * 1024, heapSize / 20000);
    ByteBuffer payload = ByteBuffer.allocate((int) messageSize);
    StreamSegmentStore store = SERVICE_BUILDER.createStreamSegmentService();
    TableStore tableStore = SERVICE_BUILDER.createTableStoreService();
    @Cleanup("shutdown") InlineExecutor tokenExpiryExecutor = new InlineExecutor();
    @Cleanup PravegaConnectionListener server = new PravegaConnectionListener(false, port, store, tableStore, tokenExpiryExecutor);
    server.startListening();
    ClientConfig config = ClientConfig.builder().build();
    SocketConnectionFactoryImpl clientCF = new SocketConnectionFactoryImpl(config);
    @Cleanup ConnectionPoolImpl connectionPool = new ConnectionPoolImpl(config, clientCF);
    Controller controller = new MockController(endpoint, port, connectionPool, true);
    @Cleanup StreamManagerImpl streamManager = new StreamManagerImpl(controller, connectionPool);
    streamManager.createScope(scope);
    @Cleanup ClientFactoryImpl clientFactory = new ClientFactoryImpl(scope, controller, config);
    streamManager.createStream("Scope", streamName, StreamConfiguration.builder().scalingPolicy(ScalingPolicy.fixed(1)).build());
    @Cleanup EventStreamWriter<ByteBuffer> producer = clientFactory.createEventWriter(streamName, new ByteBufferSerializer(), EventWriterConfig.builder().build());
    @Cleanup RawClient rawClient = new RawClient(new PravegaNodeUri(endpoint, port), connectionPool);
    for (int i = 0; i < 10; i++) {
        for (int j = 0; j < 100; j++) {
            producer.writeEvent(payload.slice());
        }
        producer.flush();
        long requestId = rawClient.getFlow().getNextSequenceNumber();
        String scopedName = new Segment(scope, streamName, 0).getScopedName();
        WireCommands.TruncateSegment request = new WireCommands.TruncateSegment(requestId, scopedName, i * 100L * (payload.remaining() + TYPE_PLUS_LENGTH_SIZE), "");
        Reply reply = rawClient.sendRequest(requestId, request).join();
        assertFalse(reply.toString(), reply.isFailure());
    }
    producer.close();
}
Also used : ConnectionPoolImpl(io.pravega.client.connection.impl.ConnectionPoolImpl) RawClient(io.pravega.client.connection.impl.RawClient) StreamManagerImpl(io.pravega.client.admin.impl.StreamManagerImpl) ByteBufferSerializer(io.pravega.client.stream.impl.ByteBufferSerializer) Cleanup(lombok.Cleanup) PravegaConnectionListener(io.pravega.segmentstore.server.host.handler.PravegaConnectionListener) CreateSegment(io.pravega.shared.protocol.netty.WireCommands.CreateSegment) Segment(io.pravega.client.segment.impl.Segment) NoSuchSegment(io.pravega.shared.protocol.netty.WireCommands.NoSuchSegment) ClientFactoryImpl(io.pravega.client.stream.impl.ClientFactoryImpl) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) InlineExecutor(io.pravega.test.common.InlineExecutor) ClientConfig(io.pravega.client.ClientConfig) WireCommands(io.pravega.shared.protocol.netty.WireCommands) SocketConnectionFactoryImpl(io.pravega.client.connection.impl.SocketConnectionFactoryImpl) MockController(io.pravega.client.stream.mock.MockController) Controller(io.pravega.client.control.impl.Controller) ByteBuffer(java.nio.ByteBuffer) TableStore(io.pravega.segmentstore.contracts.tables.TableStore) StreamSegmentStore(io.pravega.segmentstore.contracts.StreamSegmentStore) MockController(io.pravega.client.stream.mock.MockController) Reply(io.pravega.shared.protocol.netty.Reply) Test(org.junit.Test)

Aggregations

RawClient (io.pravega.client.connection.impl.RawClient)34 WireCommands (io.pravega.shared.protocol.netty.WireCommands)25 WireCommandType (io.pravega.shared.protocol.netty.WireCommandType)24 TokenExpiredException (io.pravega.auth.TokenExpiredException)17 ConnectionFailedException (io.pravega.shared.protocol.netty.ConnectionFailedException)17 Controller (io.pravega.controller.stream.api.grpc.v1.Controller)16 Reply (io.pravega.shared.protocol.netty.Reply)15 VisibleForTesting (com.google.common.annotations.VisibleForTesting)14 ConnectionPool (io.pravega.client.connection.impl.ConnectionPool)14 ConnectionClosedException (io.pravega.client.stream.impl.ConnectionClosedException)14 Exceptions (io.pravega.common.Exceptions)14 UUID (java.util.UUID)14 CompletableFuture (java.util.concurrent.CompletableFuture)14 CompletionException (java.util.concurrent.CompletionException)14 Futures (io.pravega.common.concurrent.Futures)12 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)12 SneakyThrows (lombok.SneakyThrows)12 AuthenticationException (io.pravega.auth.AuthenticationException)10 Unpooled (io.netty.buffer.Unpooled)9 PravegaNodeUri (io.pravega.shared.protocol.netty.PravegaNodeUri)8