Search in sources :

Example 1 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class SegmentMetadataClientImpl method sealSegmentAsync.

private CompletableFuture<SegmentSealed> sealSegmentAsync(Segment segment, DelegationTokenProvider tokenProvider) {
    log.trace("Sealing segment: {}", segment);
    RawClient connection = getConnection();
    long requestId = connection.getFlow().getNextSequenceNumber();
    return tokenProvider.retrieveToken().thenCompose(token -> connection.sendRequest(requestId, new SealSegment(requestId, segment.getScopedName(), token))).thenApply(r -> transformReply(r, SegmentSealed.class));
}
Also used : SneakyThrows(lombok.SneakyThrows) TokenExpiredException(io.pravega.auth.TokenExpiredException) Retry(io.pravega.common.util.Retry) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) Exceptions(io.pravega.common.Exceptions) RequiredArgsConstructor(lombok.RequiredArgsConstructor) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) CompletableFuture(java.util.concurrent.CompletableFuture) GetSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.GetSegmentAttribute) GetStreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.GetStreamSegmentInfo) RawClient(io.pravega.client.connection.impl.RawClient) UpdateSegmentAttribute(io.pravega.shared.protocol.netty.WireCommands.UpdateSegmentAttribute) AccessOperation(io.pravega.shared.security.auth.AccessOperation) TokenException(io.pravega.auth.TokenException) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) StreamSegmentInfo(io.pravega.shared.protocol.netty.WireCommands.StreamSegmentInfo) TruncateSegment(io.pravega.shared.protocol.netty.WireCommands.TruncateSegment) RetryWithBackoff(io.pravega.common.util.Retry.RetryWithBackoff) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) lombok.val(lombok.val) CompletionException(java.util.concurrent.CompletionException) DelegationTokenProviderFactory(io.pravega.client.security.auth.DelegationTokenProviderFactory) UUID(java.util.UUID) WireCommands(io.pravega.shared.protocol.netty.WireCommands) GuardedBy(javax.annotation.concurrent.GuardedBy) WrongHost(io.pravega.shared.protocol.netty.WireCommands.WrongHost) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) SegmentAttributeUpdated(io.pravega.shared.protocol.netty.WireCommands.SegmentAttributeUpdated) Slf4j(lombok.extern.slf4j.Slf4j) InvalidTokenException(io.pravega.auth.InvalidTokenException) SegmentTruncated(io.pravega.shared.protocol.netty.WireCommands.SegmentTruncated) SealSegment(io.pravega.shared.protocol.netty.WireCommands.SealSegment) SegmentSealed(io.pravega.shared.protocol.netty.WireCommands.SegmentSealed) VisibleForTesting(com.google.common.annotations.VisibleForTesting) ConnectionClosedException(io.pravega.client.stream.impl.ConnectionClosedException) Controller(io.pravega.client.control.impl.Controller) Futures(io.pravega.common.concurrent.Futures) SegmentSealed(io.pravega.shared.protocol.netty.WireCommands.SegmentSealed) RawClient(io.pravega.client.connection.impl.RawClient) SealSegment(io.pravega.shared.protocol.netty.WireCommands.SealSegment)

Example 2 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class BatchClientFactoryImpl method getStreamSegmentInfo.

private StreamSegmentsIterator getStreamSegmentInfo(final Stream stream, final StreamCut startStreamCut, final StreamCut endStreamCut) {
    log.debug("Start stream cut: {}, End stream cut: {}", startStreamCut, endStreamCut);
    StreamSegmentsInfoImpl.validateStreamCuts(startStreamCut, endStreamCut);
    StreamSegmentSuccessors segments = getAndHandleExceptions(controller.getSegments(startStreamCut, endStreamCut), RuntimeException::new);
    final SortedSet<Segment> segmentSet = new TreeSet<>(segments.getSegments());
    final DelegationTokenProvider tokenProvider = DelegationTokenProviderFactory.create(controller, stream.getScope(), stream.getStreamName(), AccessOperation.READ);
    log.debug("List of Segments between the start and end stream cuts : {}", segmentSet);
    val futures = segmentSet.stream().map(s -> getSegmentRange(s, startStreamCut, endStreamCut, tokenProvider)).collect(Collectors.toList());
    List<SegmentRange> results = Futures.getThrowingException(Futures.allOfWithResults(futures));
    return StreamSegmentsInfoImpl.builder().segmentRangeIterator(results.iterator()).startStreamCut(startStreamCut).endStreamCut(endStreamCut).build();
}
Also used : lombok.val(lombok.val) Segment(io.pravega.client.segment.impl.Segment) StreamCut(io.pravega.client.stream.StreamCut) SortedSet(java.util.SortedSet) ConnectionFactory(io.pravega.client.connection.impl.ConnectionFactory) SegmentMetadataClientFactoryImpl(io.pravega.client.segment.impl.SegmentMetadataClientFactoryImpl) CompletableFuture(java.util.concurrent.CompletableFuture) ConnectionPoolImpl(io.pravega.client.connection.impl.ConnectionPoolImpl) SegmentMetadataClientFactory(io.pravega.client.segment.impl.SegmentMetadataClientFactory) TreeSet(java.util.TreeSet) StreamCutHelper(io.pravega.client.admin.impl.StreamCutHelper) SegmentRange(io.pravega.client.batch.SegmentRange) Stream(io.pravega.client.stream.Stream) AccessOperation(io.pravega.shared.security.auth.AccessOperation) SegmentInputStreamFactoryImpl(io.pravega.client.segment.impl.SegmentInputStreamFactoryImpl) BatchClientFactory(io.pravega.client.BatchClientFactory) Serializer(io.pravega.client.stream.Serializer) SegmentInputStreamFactory(io.pravega.client.segment.impl.SegmentInputStreamFactory) Futures.getAndHandleExceptions(io.pravega.common.concurrent.Futures.getAndHandleExceptions) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) SegmentIterator(io.pravega.client.batch.SegmentIterator) lombok.val(lombok.val) SegmentInfo(io.pravega.client.segment.impl.SegmentInfo) DelegationTokenProviderFactory(io.pravega.client.security.auth.DelegationTokenProviderFactory) Collectors(java.util.stream.Collectors) Beta(com.google.common.annotations.Beta) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) SegmentMetadataClient(io.pravega.client.segment.impl.SegmentMetadataClient) StreamSegmentSuccessors(io.pravega.client.stream.impl.StreamSegmentSuccessors) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) StreamSegmentsIterator(io.pravega.client.batch.StreamSegmentsIterator) Controller(io.pravega.client.control.impl.Controller) Futures(io.pravega.common.concurrent.Futures) ClientConfig(io.pravega.client.ClientConfig) SegmentRange(io.pravega.client.batch.SegmentRange) StreamSegmentSuccessors(io.pravega.client.stream.impl.StreamSegmentSuccessors) TreeSet(java.util.TreeSet) Segment(io.pravega.client.segment.impl.Segment) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider)

Example 3 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class AsyncSegmentInputStreamTest method testProcessingFailure.

@Test
public void testProcessingFailure() throws ConnectionFailedException {
    Segment segment = new Segment("scope", "testRetry", 4);
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    @Cleanup MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    @Cleanup MockController controller = new MockController(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, true);
    DelegationTokenProvider tokenProvider = mock(DelegationTokenProvider.class);
    // return empty token
    when(tokenProvider.retrieveToken()).thenReturn(CompletableFuture.completedFuture(""));
    Semaphore dataAvailable = new Semaphore(0);
    @Cleanup AsyncSegmentInputStreamImpl in = new AsyncSegmentInputStreamImpl(controller, connectionFactory, segment, tokenProvider, dataAvailable);
    ClientConnection c = mock(ClientConnection.class);
    InOrder inOrder = Mockito.inOrder(c);
    connectionFactory.provideConnection(endpoint, c);
    WireCommands.SegmentRead segmentRead = new WireCommands.SegmentRead(segment.getScopedName(), 1234, false, false, Unpooled.EMPTY_BUFFER, in.getRequestId());
    Mockito.doAnswer(new Answer<Void>() {

        @Override
        public Void answer(InvocationOnMock invocation) throws Throwable {
            connectionFactory.getProcessor(endpoint).processingFailure(new ConnectionFailedException("Custom error"));
            return null;
        }
    }).doAnswer(new Answer<Void>() {

        @Override
        public Void answer(InvocationOnMock invocation) throws Throwable {
            connectionFactory.getProcessor(endpoint).process(segmentRead);
            return null;
        }
    }).when(c).send(any(ReadSegment.class));
    assertEquals(0, dataAvailable.availablePermits());
    CompletableFuture<SegmentRead> readFuture = in.read(1234, 5678);
    assertEquals(segmentRead, readFuture.join());
    assertTrue(Futures.isSuccessful(readFuture));
    assertEquals(1, dataAvailable.availablePermits());
    inOrder.verify(c).send(eq(new WireCommands.ReadSegment(segment.getScopedName(), 1234, 5678, "", in.getRequestId())));
    inOrder.verify(c).close();
    inOrder.verify(c).send(eq(new WireCommands.ReadSegment(segment.getScopedName(), 1234, 5678, "", in.getRequestId())));
    verifyNoMoreInteractions(c);
    // ensure retrieve Token is invoked for every retry.
    verify(tokenProvider, times(2)).retrieveToken();
}
Also used : InOrder(org.mockito.InOrder) ReadSegment(io.pravega.shared.protocol.netty.WireCommands.ReadSegment) Semaphore(java.util.concurrent.Semaphore) Cleanup(lombok.Cleanup) SegmentRead(io.pravega.shared.protocol.netty.WireCommands.SegmentRead) ReadSegment(io.pravega.shared.protocol.netty.WireCommands.ReadSegment) Answer(org.mockito.stubbing.Answer) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) SegmentRead(io.pravega.shared.protocol.netty.WireCommands.SegmentRead) InvocationOnMock(org.mockito.invocation.InvocationOnMock) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) MockController(io.pravega.client.stream.mock.MockController) ClientConnection(io.pravega.client.connection.impl.ClientConnection) WireCommands(io.pravega.shared.protocol.netty.WireCommands) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) Test(org.junit.Test)

Example 4 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class TransactionalEventStreamWriterImpl method beginTxn.

@Override
public Transaction<Type> beginTxn() {
    TxnSegments txnSegments = Futures.getThrowingException(controller.createTransaction(stream, config.getTransactionTimeoutTime()));
    log.info("Transaction {} created", txnSegments.getTxnId());
    UUID txnId = txnSegments.getTxnId();
    Map<Segment, SegmentTransaction<Type>> transactions = new HashMap<>();
    DelegationTokenProvider tokenProvider = null;
    for (Segment s : txnSegments.getStreamSegments().getSegments()) {
        if (tokenProvider == null) {
            tokenProvider = DelegationTokenProviderFactory.create(controller, s, AccessOperation.WRITE);
        }
        SegmentOutputStream out = outputStreamFactory.createOutputStreamForTransaction(s, txnId, config, tokenProvider);
        SegmentTransactionImpl<Type> impl = new SegmentTransactionImpl<>(txnId, out, serializer);
        transactions.put(s, impl);
    }
    pinger.startPing(txnId);
    return new TransactionImpl<Type>(writerId, txnId, transactions, txnSegments.getStreamSegments(), controller, stream, pinger);
}
Also used : HashMap(java.util.HashMap) SegmentOutputStream(io.pravega.client.segment.impl.SegmentOutputStream) UUID(java.util.UUID) Segment(io.pravega.client.segment.impl.Segment) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider)

Example 5 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class TransactionalEventStreamWriterImpl method getTxn.

@Override
public Transaction<Type> getTxn(UUID txId) {
    // check if the transaction is open.
    Status status = Futures.getThrowingException(controller.checkTransactionStatus(stream, txId));
    if (status != Status.OPEN) {
        return new TransactionImpl<>(writerId, txId, controller, stream);
    }
    // get the segments corresponding to the transaction.
    StreamSegments segments = Futures.getThrowingException(controller.getEpochSegments(stream.getScope(), stream.getStreamName(), getEpoch(txId)));
    assert segments != null : "Epoch segments returned is null";
    Preconditions.checkState(segments.getSegments().size() > 0, "There should be at least 1 epoch segment");
    // Create OutputStream for every segment.
    Map<Segment, SegmentTransaction<Type>> transactions = new HashMap<>();
    DelegationTokenProvider tokenProvider = null;
    for (Segment s : segments.getSegments()) {
        if (tokenProvider == null) {
            tokenProvider = DelegationTokenProviderFactory.create(controller, s, AccessOperation.WRITE);
        }
        SegmentOutputStream out = outputStreamFactory.createOutputStreamForTransaction(s, txId, config, tokenProvider);
        SegmentTransactionImpl<Type> impl = new SegmentTransactionImpl<>(txId, out, serializer);
        transactions.put(s, impl);
    }
    pinger.startPing(txId);
    return new TransactionImpl<Type>(writerId, txId, transactions, segments, controller, stream, pinger);
}
Also used : Status(io.pravega.client.stream.Transaction.Status) HashMap(java.util.HashMap) SegmentOutputStream(io.pravega.client.segment.impl.SegmentOutputStream) Segment(io.pravega.client.segment.impl.Segment) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider)

Aggregations

DelegationTokenProvider (io.pravega.client.security.auth.DelegationTokenProvider)12 Segment (io.pravega.client.segment.impl.Segment)6 WireCommands (io.pravega.shared.protocol.netty.WireCommands)5 ConnectionPool (io.pravega.client.connection.impl.ConnectionPool)4 Controller (io.pravega.client.control.impl.Controller)4 ConnectionFailedException (io.pravega.shared.protocol.netty.ConnectionFailedException)4 UUID (java.util.UUID)4 CompletableFuture (java.util.concurrent.CompletableFuture)4 Slf4j (lombok.extern.slf4j.Slf4j)4 lombok.val (lombok.val)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 TokenExpiredException (io.pravega.auth.TokenExpiredException)3 ClientConnection (io.pravega.client.connection.impl.ClientConnection)3 RawClient (io.pravega.client.connection.impl.RawClient)3 DelegationTokenProviderFactory (io.pravega.client.security.auth.DelegationTokenProviderFactory)3 SegmentMetadataClient (io.pravega.client.segment.impl.SegmentMetadataClient)3 EventWriterConfig (io.pravega.client.stream.EventWriterConfig)3 InvalidTokenException (io.pravega.auth.InvalidTokenException)2 TokenException (io.pravega.auth.TokenException)2 SegmentOutputStream (io.pravega.client.segment.impl.SegmentOutputStream)2