Search in sources :

Example 11 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class SegmentHelper method mergeTxnSegments.

public CompletableFuture<List<Long>> mergeTxnSegments(final String scope, final String stream, final long targetSegmentId, final long sourceSegmentId, final List<UUID> txId, final String delegationToken, final long clientRequestId) {
    Preconditions.checkArgument(getSegmentNumber(targetSegmentId) == getSegmentNumber(sourceSegmentId));
    final Controller.NodeUri uri = getSegmentUri(scope, stream, sourceSegmentId);
    final String qualifiedNameTarget = getQualifiedStreamSegmentName(scope, stream, targetSegmentId);
    final List<String> transactionNames = txId.stream().map(x -> getTransactionName(scope, stream, sourceSegmentId, x)).collect(Collectors.toList());
    final WireCommandType type = WireCommandType.MERGE_SEGMENTS_BATCH;
    RawClient connection = new RawClient(ModelHelper.encode(uri), connectionPool);
    final long requestId = connection.getFlow().asLong();
    WireCommands.MergeSegmentsBatch request = new WireCommands.MergeSegmentsBatch(requestId, qualifiedNameTarget, transactionNames, delegationToken);
    return sendRequest(connection, clientRequestId, request).thenApply(r -> {
        handleReply(clientRequestId, r, connection, qualifiedNameTarget, WireCommands.MergeSegmentsBatch.class, type);
        return ((WireCommands.SegmentsBatchMerged) r).getNewTargetWriteOffset();
    });
}
Also used : SneakyThrows(lombok.SneakyThrows) TokenExpiredException(io.pravega.auth.TokenExpiredException) LoggerFactory(org.slf4j.LoggerFactory) TimeoutException(java.util.concurrent.TimeoutException) Unpooled(io.netty.buffer.Unpooled) TagLogger(io.pravega.common.tracing.TagLogger) Pair(org.apache.commons.lang3.tuple.Pair) Duration(java.time.Duration) Map(java.util.Map) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) HashTableIteratorItem(io.pravega.client.tables.impl.HashTableIteratorItem) Controller(io.pravega.controller.stream.api.grpc.v1.Controller) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Set(java.util.Set) CompletionException(java.util.concurrent.CompletionException) Request(io.pravega.shared.protocol.netty.Request) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) List(java.util.List) Config(io.pravega.controller.util.Config) Futures(io.pravega.common.concurrent.Futures) TableSegmentKeyVersion(io.pravega.client.tables.impl.TableSegmentKeyVersion) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) ModelHelper(io.pravega.client.control.impl.ModelHelper) Exceptions(io.pravega.common.Exceptions) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) NameUtils.getSegmentNumber(io.pravega.shared.NameUtils.getSegmentNumber) RawClient(io.pravega.client.connection.impl.RawClient) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) NameUtils.getQualifiedStreamSegmentName(io.pravega.shared.NameUtils.getQualifiedStreamSegmentName) RecordHelper(io.pravega.controller.store.stream.records.RecordHelper) Host(io.pravega.common.cluster.Host) TableSegmentKey(io.pravega.client.tables.impl.TableSegmentKey) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) AuthenticationException(io.pravega.auth.AuthenticationException) WireCommands(io.pravega.shared.protocol.netty.WireCommands) ImmutablePair(org.apache.commons.lang3.tuple.ImmutablePair) WireCommand(io.pravega.shared.protocol.netty.WireCommand) AbstractMap(java.util.AbstractMap) TableSegmentEntry(io.pravega.client.tables.impl.TableSegmentEntry) HostControllerStore(io.pravega.controller.store.host.HostControllerStore) TxnStatus(io.pravega.controller.stream.api.grpc.v1.Controller.TxnStatus) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ConnectionClosedException(io.pravega.client.stream.impl.ConnectionClosedException) ScalingPolicy(io.pravega.client.stream.ScalingPolicy) NameUtils.getTransactionNameFromId(io.pravega.shared.NameUtils.getTransactionNameFromId) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) RawClient(io.pravega.client.connection.impl.RawClient) Controller(io.pravega.controller.stream.api.grpc.v1.Controller) WireCommands(io.pravega.shared.protocol.netty.WireCommands)

Example 12 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class SegmentHelper method abortTransaction.

public CompletableFuture<TxnStatus> abortTransaction(final String scope, final String stream, final long segmentId, final UUID txId, final String delegationToken, final long clientRequestId) {
    final String transactionName = getTransactionName(scope, stream, segmentId, txId);
    final Controller.NodeUri uri = getSegmentUri(scope, stream, segmentId);
    final WireCommandType type = WireCommandType.DELETE_SEGMENT;
    RawClient connection = new RawClient(ModelHelper.encode(uri), connectionPool);
    final long requestId = connection.getFlow().asLong();
    WireCommands.DeleteSegment request = new WireCommands.DeleteSegment(requestId, transactionName, delegationToken);
    return sendRequest(connection, clientRequestId, request).thenAccept(r -> handleReply(clientRequestId, r, connection, transactionName, WireCommands.DeleteSegment.class, type)).thenApply(v -> TxnStatus.newBuilder().setStatus(TxnStatus.Status.SUCCESS).build());
}
Also used : SneakyThrows(lombok.SneakyThrows) TokenExpiredException(io.pravega.auth.TokenExpiredException) LoggerFactory(org.slf4j.LoggerFactory) TimeoutException(java.util.concurrent.TimeoutException) Unpooled(io.netty.buffer.Unpooled) TagLogger(io.pravega.common.tracing.TagLogger) Pair(org.apache.commons.lang3.tuple.Pair) Duration(java.time.Duration) Map(java.util.Map) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) HashTableIteratorItem(io.pravega.client.tables.impl.HashTableIteratorItem) Controller(io.pravega.controller.stream.api.grpc.v1.Controller) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Set(java.util.Set) CompletionException(java.util.concurrent.CompletionException) Request(io.pravega.shared.protocol.netty.Request) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) List(java.util.List) Config(io.pravega.controller.util.Config) Futures(io.pravega.common.concurrent.Futures) TableSegmentKeyVersion(io.pravega.client.tables.impl.TableSegmentKeyVersion) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) ModelHelper(io.pravega.client.control.impl.ModelHelper) Exceptions(io.pravega.common.Exceptions) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicReference(java.util.concurrent.atomic.AtomicReference) NameUtils.getSegmentNumber(io.pravega.shared.NameUtils.getSegmentNumber) RawClient(io.pravega.client.connection.impl.RawClient) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) NameUtils.getQualifiedStreamSegmentName(io.pravega.shared.NameUtils.getQualifiedStreamSegmentName) RecordHelper(io.pravega.controller.store.stream.records.RecordHelper) Host(io.pravega.common.cluster.Host) TableSegmentKey(io.pravega.client.tables.impl.TableSegmentKey) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) AuthenticationException(io.pravega.auth.AuthenticationException) WireCommands(io.pravega.shared.protocol.netty.WireCommands) ImmutablePair(org.apache.commons.lang3.tuple.ImmutablePair) WireCommand(io.pravega.shared.protocol.netty.WireCommand) AbstractMap(java.util.AbstractMap) TableSegmentEntry(io.pravega.client.tables.impl.TableSegmentEntry) HostControllerStore(io.pravega.controller.store.host.HostControllerStore) TxnStatus(io.pravega.controller.stream.api.grpc.v1.Controller.TxnStatus) Preconditions(com.google.common.base.Preconditions) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ConnectionClosedException(io.pravega.client.stream.impl.ConnectionClosedException) ScalingPolicy(io.pravega.client.stream.ScalingPolicy) NameUtils.getTransactionNameFromId(io.pravega.shared.NameUtils.getTransactionNameFromId) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) RawClient(io.pravega.client.connection.impl.RawClient) Controller(io.pravega.controller.stream.api.grpc.v1.Controller) WireCommands(io.pravega.shared.protocol.netty.WireCommands)

Example 13 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class SegmentHelper method getSegmentInfo.

public CompletableFuture<WireCommands.StreamSegmentInfo> getSegmentInfo(String qualifiedName, PravegaNodeUri uri, String delegationToken, long clientRequestId) {
    final WireCommandType type = WireCommandType.GET_STREAM_SEGMENT_INFO;
    RawClient connection = new RawClient(uri, connectionPool);
    final long requestId = connection.getFlow().asLong();
    WireCommands.GetStreamSegmentInfo request = new WireCommands.GetStreamSegmentInfo(requestId, qualifiedName, delegationToken);
    return sendRequest(connection, clientRequestId, request).thenApply(r -> {
        handleReply(clientRequestId, r, connection, qualifiedName, WireCommands.GetStreamSegmentInfo.class, type);
        assert r instanceof WireCommands.StreamSegmentInfo;
        return (WireCommands.StreamSegmentInfo) r;
    });
}
Also used : WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) RawClient(io.pravega.client.connection.impl.RawClient) WireCommands(io.pravega.shared.protocol.netty.WireCommands)

Example 14 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class SegmentHelper method createTableSegment.

/**
 * This method sends a WireCommand to create a table segment.
 *
 * @param tableName           Qualified table name.
 * @param delegationToken     The token to be presented to the segmentstore.
 * @param clientRequestId     Request id.
 * @param sortedTableSegment  Boolean flag indicating if the Table Segment should be created in sorted order.
 * @param keyLength           Key Length. If 0, a Hash Table Segment (Variable Key Length) will be created, otherwise
 *                            a Fixed-Key-Length Table Segment will be created with this value for the key length.
 * @param rolloverSizeBytes   The rollover size of segment in LTS.
 * @return A CompletableFuture that, when completed normally, will indicate the table segment creation completed
 * successfully. If the operation failed, the future will be failed with the causing exception. If the exception
 * can be retried then the future will be failed with {@link WireCommandFailedException}.
 */
public CompletableFuture<Void> createTableSegment(final String tableName, String delegationToken, final long clientRequestId, final boolean sortedTableSegment, final int keyLength, final long rolloverSizeBytes) {
    final Controller.NodeUri uri = getTableUri(tableName);
    final WireCommandType type = WireCommandType.CREATE_TABLE_SEGMENT;
    RawClient connection = new RawClient(ModelHelper.encode(uri), connectionPool);
    final long requestId = connection.getFlow().asLong();
    // All Controller Metadata Segments are non-sorted.
    return sendRequest(connection, clientRequestId, new WireCommands.CreateTableSegment(requestId, tableName, sortedTableSegment, keyLength, delegationToken, rolloverSizeBytes)).thenAccept(rpl -> handleReply(clientRequestId, rpl, connection, tableName, WireCommands.CreateTableSegment.class, type));
}
Also used : WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) RawClient(io.pravega.client.connection.impl.RawClient) Controller(io.pravega.controller.stream.api.grpc.v1.Controller)

Example 15 with RawClient

use of io.pravega.client.connection.impl.RawClient in project pravega by pravega.

the class LargeEventWriter method writeLargeEvent.

/**
 * Write the provided list of events (atomically) to the provided segment.
 *
 * @param segment The segment to write to
 * @param events The events to append
 * @param tokenProvider A token provider
 * @param config Used for retry configuration parameters
 * @throws NoSuchSegmentException If the provided segment does not exit.
 * @throws SegmentSealedException If the segment is sealed.
 * @throws AuthenticationException If the token can't be used for this segment.
 * @throws UnsupportedOperationException If the server does not support large events.
 */
public void writeLargeEvent(Segment segment, List<ByteBuffer> events, DelegationTokenProvider tokenProvider, EventWriterConfig config) throws NoSuchSegmentException, AuthenticationException, SegmentSealedException {
    List<ByteBuf> payloads = createBufs(events);
    int attempts = 1 + Math.max(0, config.getRetryAttempts());
    Retry.withExpBackoff(config.getInitialBackoffMillis(), config.getBackoffMultiple(), attempts, config.getMaxBackoffMillis()).retryWhen(t -> {
        Throwable ex = Exceptions.unwrap(t);
        if (ex instanceof ConnectionFailedException) {
            log.info("Connection failure while sending large event: {}. Retrying", ex.getMessage());
            return true;
        } else if (ex instanceof TokenExpiredException) {
            tokenProvider.signalTokenExpired();
            log.info("Authentication token expired while writing large event to segment {}. Retrying", segment);
            return true;
        } else {
            return false;
        }
    }).run(() -> {
        @Cleanup RawClient client = new RawClient(controller, connectionPool, segment);
        write(segment, payloads, client, tokenProvider);
        return null;
    });
}
Also used : Segment(io.pravega.client.segment.impl.Segment) TokenExpiredException(io.pravega.auth.TokenExpiredException) Retry(io.pravega.common.util.Retry) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) Exceptions(io.pravega.common.Exceptions) ConditionalCheckFailed(io.pravega.shared.protocol.netty.WireCommands.ConditionalCheckFailed) RequiredArgsConstructor(lombok.RequiredArgsConstructor) SegmentIsTruncated(io.pravega.shared.protocol.netty.WireCommands.SegmentIsTruncated) Cleanup(lombok.Cleanup) CompletableFuture(java.util.concurrent.CompletableFuture) SegmentSealedException(io.pravega.client.segment.impl.SegmentSealedException) ByteBuffer(java.nio.ByteBuffer) Unpooled(io.netty.buffer.Unpooled) CreateTransientSegment(io.pravega.shared.protocol.netty.WireCommands.CreateTransientSegment) ArrayList(java.util.ArrayList) RawClient(io.pravega.client.connection.impl.RawClient) ConditionalBlockEnd(io.pravega.shared.protocol.netty.WireCommands.ConditionalBlockEnd) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) ByteBuf(io.netty.buffer.ByteBuf) AuthTokenCheckFailed(io.pravega.shared.protocol.netty.WireCommands.AuthTokenCheckFailed) MergeSegments(io.pravega.shared.protocol.netty.WireCommands.MergeSegments) SegmentCreated(io.pravega.shared.protocol.netty.WireCommands.SegmentCreated) Futures.getThrowingException(io.pravega.common.concurrent.Futures.getThrowingException) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) Nonnull(javax.annotation.Nonnull) SegmentAlreadyExists(io.pravega.shared.protocol.netty.WireCommands.SegmentAlreadyExists) EventWriterConfig(io.pravega.client.stream.EventWriterConfig) Serializer(io.pravega.client.stream.Serializer) SegmentsMerged(io.pravega.shared.protocol.netty.WireCommands.SegmentsMerged) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) NoSuchSegmentException(io.pravega.client.segment.impl.NoSuchSegmentException) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) lombok.val(lombok.val) AuthenticationException(io.pravega.auth.AuthenticationException) UUID(java.util.UUID) WireCommands(io.pravega.shared.protocol.netty.WireCommands) WrongHost(io.pravega.shared.protocol.netty.WireCommands.WrongHost) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) InvalidEventNumber(io.pravega.shared.protocol.netty.WireCommands.InvalidEventNumber) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) OperationUnsupported(io.pravega.shared.protocol.netty.WireCommands.OperationUnsupported) VisibleForTesting(com.google.common.annotations.VisibleForTesting) DataAppended(io.pravega.shared.protocol.netty.WireCommands.DataAppended) SegmentIsSealed(io.pravega.shared.protocol.netty.WireCommands.SegmentIsSealed) Controller(io.pravega.client.control.impl.Controller) TokenExpiredException(io.pravega.auth.TokenExpiredException) RawClient(io.pravega.client.connection.impl.RawClient) ByteBuf(io.netty.buffer.ByteBuf) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Cleanup(lombok.Cleanup)

Aggregations

RawClient (io.pravega.client.connection.impl.RawClient)34 WireCommands (io.pravega.shared.protocol.netty.WireCommands)25 WireCommandType (io.pravega.shared.protocol.netty.WireCommandType)24 TokenExpiredException (io.pravega.auth.TokenExpiredException)17 ConnectionFailedException (io.pravega.shared.protocol.netty.ConnectionFailedException)17 Controller (io.pravega.controller.stream.api.grpc.v1.Controller)16 Reply (io.pravega.shared.protocol.netty.Reply)15 VisibleForTesting (com.google.common.annotations.VisibleForTesting)14 ConnectionPool (io.pravega.client.connection.impl.ConnectionPool)14 ConnectionClosedException (io.pravega.client.stream.impl.ConnectionClosedException)14 Exceptions (io.pravega.common.Exceptions)14 UUID (java.util.UUID)14 CompletableFuture (java.util.concurrent.CompletableFuture)14 CompletionException (java.util.concurrent.CompletionException)14 Futures (io.pravega.common.concurrent.Futures)12 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)12 SneakyThrows (lombok.SneakyThrows)12 AuthenticationException (io.pravega.auth.AuthenticationException)10 Unpooled (io.netty.buffer.Unpooled)9 PravegaNodeUri (io.pravega.shared.protocol.netty.PravegaNodeUri)8