Search in sources :

Example 6 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class LargeEventWriter method writeLargeEvent.

/**
 * Write the provided list of events (atomically) to the provided segment.
 *
 * @param segment The segment to write to
 * @param events The events to append
 * @param tokenProvider A token provider
 * @param config Used for retry configuration parameters
 * @throws NoSuchSegmentException If the provided segment does not exit.
 * @throws SegmentSealedException If the segment is sealed.
 * @throws AuthenticationException If the token can't be used for this segment.
 * @throws UnsupportedOperationException If the server does not support large events.
 */
public void writeLargeEvent(Segment segment, List<ByteBuffer> events, DelegationTokenProvider tokenProvider, EventWriterConfig config) throws NoSuchSegmentException, AuthenticationException, SegmentSealedException {
    List<ByteBuf> payloads = createBufs(events);
    int attempts = 1 + Math.max(0, config.getRetryAttempts());
    Retry.withExpBackoff(config.getInitialBackoffMillis(), config.getBackoffMultiple(), attempts, config.getMaxBackoffMillis()).retryWhen(t -> {
        Throwable ex = Exceptions.unwrap(t);
        if (ex instanceof ConnectionFailedException) {
            log.info("Connection failure while sending large event: {}. Retrying", ex.getMessage());
            return true;
        } else if (ex instanceof TokenExpiredException) {
            tokenProvider.signalTokenExpired();
            log.info("Authentication token expired while writing large event to segment {}. Retrying", segment);
            return true;
        } else {
            return false;
        }
    }).run(() -> {
        @Cleanup RawClient client = new RawClient(controller, connectionPool, segment);
        write(segment, payloads, client, tokenProvider);
        return null;
    });
}
Also used : Segment(io.pravega.client.segment.impl.Segment) TokenExpiredException(io.pravega.auth.TokenExpiredException) Retry(io.pravega.common.util.Retry) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Reply(io.pravega.shared.protocol.netty.Reply) Exceptions(io.pravega.common.Exceptions) ConditionalCheckFailed(io.pravega.shared.protocol.netty.WireCommands.ConditionalCheckFailed) RequiredArgsConstructor(lombok.RequiredArgsConstructor) SegmentIsTruncated(io.pravega.shared.protocol.netty.WireCommands.SegmentIsTruncated) Cleanup(lombok.Cleanup) CompletableFuture(java.util.concurrent.CompletableFuture) SegmentSealedException(io.pravega.client.segment.impl.SegmentSealedException) ByteBuffer(java.nio.ByteBuffer) Unpooled(io.netty.buffer.Unpooled) CreateTransientSegment(io.pravega.shared.protocol.netty.WireCommands.CreateTransientSegment) ArrayList(java.util.ArrayList) RawClient(io.pravega.client.connection.impl.RawClient) ConditionalBlockEnd(io.pravega.shared.protocol.netty.WireCommands.ConditionalBlockEnd) SetupAppend(io.pravega.shared.protocol.netty.WireCommands.SetupAppend) ByteBuf(io.netty.buffer.ByteBuf) AuthTokenCheckFailed(io.pravega.shared.protocol.netty.WireCommands.AuthTokenCheckFailed) MergeSegments(io.pravega.shared.protocol.netty.WireCommands.MergeSegments) SegmentCreated(io.pravega.shared.protocol.netty.WireCommands.SegmentCreated) Futures.getThrowingException(io.pravega.common.concurrent.Futures.getThrowingException) WireCommandType(io.pravega.shared.protocol.netty.WireCommandType) Nonnull(javax.annotation.Nonnull) SegmentAlreadyExists(io.pravega.shared.protocol.netty.WireCommands.SegmentAlreadyExists) EventWriterConfig(io.pravega.client.stream.EventWriterConfig) Serializer(io.pravega.client.stream.Serializer) SegmentsMerged(io.pravega.shared.protocol.netty.WireCommands.SegmentsMerged) ConnectionPool(io.pravega.client.connection.impl.ConnectionPool) NoSuchSegmentException(io.pravega.client.segment.impl.NoSuchSegmentException) AppendSetup(io.pravega.shared.protocol.netty.WireCommands.AppendSetup) lombok.val(lombok.val) AuthenticationException(io.pravega.auth.AuthenticationException) UUID(java.util.UUID) WireCommands(io.pravega.shared.protocol.netty.WireCommands) WrongHost(io.pravega.shared.protocol.netty.WireCommands.WrongHost) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) InvalidEventNumber(io.pravega.shared.protocol.netty.WireCommands.InvalidEventNumber) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) OperationUnsupported(io.pravega.shared.protocol.netty.WireCommands.OperationUnsupported) VisibleForTesting(com.google.common.annotations.VisibleForTesting) DataAppended(io.pravega.shared.protocol.netty.WireCommands.DataAppended) SegmentIsSealed(io.pravega.shared.protocol.netty.WireCommands.SegmentIsSealed) Controller(io.pravega.client.control.impl.Controller) TokenExpiredException(io.pravega.auth.TokenExpiredException) RawClient(io.pravega.client.connection.impl.RawClient) ByteBuf(io.netty.buffer.ByteBuf) ConnectionFailedException(io.pravega.shared.protocol.netty.ConnectionFailedException) Cleanup(lombok.Cleanup)

Example 7 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class ClientFactoryImpl method createRevisionedStreamClient.

private <T> RevisionedStreamClient<T> createRevisionedStreamClient(Segment segment, Serializer<T> serializer, SynchronizerConfig config) {
    EventSegmentReader in = inFactory.createEventReaderForSegment(segment, config.getReadBufferSize());
    DelegationTokenProvider delegationTokenProvider = DelegationTokenProviderFactory.create(controller, segment, AccessOperation.READ_WRITE);
    ConditionalOutputStream cond = condFactory.createConditionalOutputStream(segment, delegationTokenProvider, config.getEventWriterConfig());
    SegmentMetadataClient meta = metaFactory.createSegmentMetadataClient(segment, delegationTokenProvider);
    return new RevisionedStreamClientImpl<>(segment, in, outFactory, cond, meta, serializer, config.getEventWriterConfig(), delegationTokenProvider);
}
Also used : RevisionedStreamClientImpl(io.pravega.client.state.impl.RevisionedStreamClientImpl) EventSegmentReader(io.pravega.client.segment.impl.EventSegmentReader) ConditionalOutputStream(io.pravega.client.segment.impl.ConditionalOutputStream) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) SegmentMetadataClient(io.pravega.client.segment.impl.SegmentMetadataClient)

Example 8 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class ByteStreanWriterImplTest method setup.

@Before
public void setup() {
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", 0);
    connectionFactory = new MockConnectionFactoryImpl();
    ClientConnection connection = mock(ClientConnection.class);
    connectionFactory.provideConnection(endpoint, connection);
    controller = new MockController(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, false);
    controller.createScope(SCOPE);
    controller.createStream(SCOPE, STREAM, StreamConfiguration.builder().scalingPolicy(ScalingPolicy.fixed(1)).build());
    MockSegmentStreamFactory streamFactory = new MockSegmentStreamFactory();
    StreamSegments segments = Futures.getThrowingException(controller.getCurrentSegments(SCOPE, STREAM));
    Preconditions.checkState(segments.getNumberOfSegments() > 0, "Stream is sealed");
    Preconditions.checkState(segments.getNumberOfSegments() == 1, "Stream is configured with more than one segment");
    Segment segment = segments.getSegments().iterator().next();
    EventWriterConfig config = EventWriterConfig.builder().retryAttempts(Integer.MAX_VALUE).build();
    DelegationTokenProvider tokenProvider = DelegationTokenProviderFactory.create(controller, segment, AccessOperation.WRITE);
    mockWriter = new ByteStreamWriterImpl(streamFactory.createOutputStreamForSegment(segment, config, tokenProvider), streamFactory.createSegmentMetadataClient(segment, tokenProvider));
}
Also used : PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) EventWriterConfig(io.pravega.client.stream.EventWriterConfig) MockSegmentStreamFactory(io.pravega.client.stream.mock.MockSegmentStreamFactory) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) MockController(io.pravega.client.stream.mock.MockController) ClientConnection(io.pravega.client.connection.impl.ClientConnection) StreamSegments(io.pravega.client.stream.impl.StreamSegments) Segment(io.pravega.client.segment.impl.Segment) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) Before(org.junit.Before)

Example 9 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class AsyncSegmentInputStreamTest method testRetry.

@Test(timeout = 10000)
public void testRetry() throws ConnectionFailedException {
    Segment segment = new Segment("scope", "testRetry", 4);
    PravegaNodeUri endpoint = new PravegaNodeUri("localhost", SERVICE_PORT);
    @Cleanup MockConnectionFactoryImpl connectionFactory = new MockConnectionFactoryImpl();
    @Cleanup MockController controller = new MockController(endpoint.getEndpoint(), endpoint.getPort(), connectionFactory, true);
    DelegationTokenProvider tokenProvider = mock(DelegationTokenProvider.class);
    // return empty token
    when(tokenProvider.retrieveToken()).thenReturn(CompletableFuture.completedFuture(""));
    Semaphore dataAvailable = new Semaphore(0);
    @Cleanup AsyncSegmentInputStreamImpl in = new AsyncSegmentInputStreamImpl(controller, connectionFactory, segment, tokenProvider, dataAvailable);
    ClientConnection c = mock(ClientConnection.class);
    InOrder inOrder = Mockito.inOrder(c);
    connectionFactory.provideConnection(endpoint, c);
    WireCommands.SegmentRead segmentRead = new WireCommands.SegmentRead(segment.getScopedName(), 1234, false, false, Unpooled.EMPTY_BUFFER, in.getRequestId());
    Mockito.doAnswer(new Answer<Void>() {

        @Override
        public Void answer(InvocationOnMock invocation) throws Throwable {
            connectionFactory.getProcessor(endpoint).connectionDropped();
            return null;
        }
    }).doAnswer(new Answer<Void>() {

        @Override
        public Void answer(InvocationOnMock invocation) throws Throwable {
            connectionFactory.getProcessor(endpoint).authTokenCheckFailed(new WireCommands.AuthTokenCheckFailed(in.getRequestId(), "SomeException", WireCommands.AuthTokenCheckFailed.ErrorCode.TOKEN_EXPIRED));
            return null;
        }
    }).doAnswer(new Answer<Void>() {

        @Override
        public Void answer(InvocationOnMock invocation) throws Throwable {
            connectionFactory.getProcessor(endpoint).process(segmentRead);
            return null;
        }
    }).when(c).send(any(ReadSegment.class));
    assertEquals(0, dataAvailable.availablePermits());
    CompletableFuture<SegmentRead> readFuture = in.read(1234, 5678);
    assertEquals(segmentRead, readFuture.join());
    assertTrue(Futures.isSuccessful(readFuture));
    assertEquals(1, dataAvailable.availablePermits());
    inOrder.verify(c).send(eq(new WireCommands.ReadSegment(segment.getScopedName(), 1234, 5678, "", in.getRequestId())));
    inOrder.verify(c).close();
    inOrder.verify(c).send(eq(new WireCommands.ReadSegment(segment.getScopedName(), 1234, 5678, "", in.getRequestId())));
    inOrder.verify(c).close();
    inOrder.verify(c).send(eq(new WireCommands.ReadSegment(segment.getScopedName(), 1234, 5678, "", in.getRequestId())));
    verifyNoMoreInteractions(c);
    // ensure retrieve Token is invoked for every retry.
    verify(tokenProvider, times(3)).retrieveToken();
}
Also used : InOrder(org.mockito.InOrder) ReadSegment(io.pravega.shared.protocol.netty.WireCommands.ReadSegment) Semaphore(java.util.concurrent.Semaphore) Cleanup(lombok.Cleanup) SegmentRead(io.pravega.shared.protocol.netty.WireCommands.SegmentRead) ReadSegment(io.pravega.shared.protocol.netty.WireCommands.ReadSegment) Answer(org.mockito.stubbing.Answer) PravegaNodeUri(io.pravega.shared.protocol.netty.PravegaNodeUri) SegmentRead(io.pravega.shared.protocol.netty.WireCommands.SegmentRead) InvocationOnMock(org.mockito.invocation.InvocationOnMock) MockConnectionFactoryImpl(io.pravega.client.stream.mock.MockConnectionFactoryImpl) MockController(io.pravega.client.stream.mock.MockController) ClientConnection(io.pravega.client.connection.impl.ClientConnection) WireCommands(io.pravega.shared.protocol.netty.WireCommands) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider) Test(org.junit.Test)

Example 10 with DelegationTokenProvider

use of io.pravega.client.security.auth.DelegationTokenProvider in project pravega by pravega.

the class ByteStreamClientImpl method createByteStreamWriter.

@Override
public ByteStreamWriter createByteStreamWriter(String streamName) {
    StreamSegments segments = Futures.getThrowingException(controller.getCurrentSegments(scope, streamName));
    Preconditions.checkState(segments.getNumberOfSegments() > 0, "Stream is sealed");
    Preconditions.checkState(segments.getNumberOfSegments() == 1, "Stream is configured with more than one segment");
    Segment segment = segments.getSegments().iterator().next();
    // The writer should not give up connecting to SegmentStore in the background until the ByteStreamWriter is closed.
    EventWriterConfig config = EventWriterConfig.builder().retryAttempts(Integer.MAX_VALUE).build();
    DelegationTokenProvider tokenProvider = DelegationTokenProviderFactory.create(controller, segment, AccessOperation.WRITE);
    return new BufferedByteStreamWriterImpl(new ByteStreamWriterImpl(outputStreamFactory.createOutputStreamForSegment(segment, config, tokenProvider), metaStreamFactory.createSegmentMetadataClient(segment, tokenProvider)));
}
Also used : EventWriterConfig(io.pravega.client.stream.EventWriterConfig) StreamSegments(io.pravega.client.stream.impl.StreamSegments) Segment(io.pravega.client.segment.impl.Segment) DelegationTokenProvider(io.pravega.client.security.auth.DelegationTokenProvider)

Aggregations

DelegationTokenProvider (io.pravega.client.security.auth.DelegationTokenProvider)12 Segment (io.pravega.client.segment.impl.Segment)6 WireCommands (io.pravega.shared.protocol.netty.WireCommands)5 ConnectionPool (io.pravega.client.connection.impl.ConnectionPool)4 Controller (io.pravega.client.control.impl.Controller)4 ConnectionFailedException (io.pravega.shared.protocol.netty.ConnectionFailedException)4 UUID (java.util.UUID)4 CompletableFuture (java.util.concurrent.CompletableFuture)4 Slf4j (lombok.extern.slf4j.Slf4j)4 lombok.val (lombok.val)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)3 TokenExpiredException (io.pravega.auth.TokenExpiredException)3 ClientConnection (io.pravega.client.connection.impl.ClientConnection)3 RawClient (io.pravega.client.connection.impl.RawClient)3 DelegationTokenProviderFactory (io.pravega.client.security.auth.DelegationTokenProviderFactory)3 SegmentMetadataClient (io.pravega.client.segment.impl.SegmentMetadataClient)3 EventWriterConfig (io.pravega.client.stream.EventWriterConfig)3 InvalidTokenException (io.pravega.auth.InvalidTokenException)2 TokenException (io.pravega.auth.TokenException)2 SegmentOutputStream (io.pravega.client.segment.impl.SegmentOutputStream)2