Search in sources :

Example 11 with GrpcAuthHelper

use of io.pravega.controller.server.security.auth.GrpcAuthHelper in project pravega by pravega.

the class PravegaTablesStoreHelperTest method testRetriesExhausted.

@Test
public void testRetriesExhausted() {
    SegmentHelper segmentHelper = spy(SegmentHelperMock.getSegmentHelperMockForTables(executor));
    GrpcAuthHelper authHelper = GrpcAuthHelper.getDisabledAuthHelper();
    PravegaTablesStoreHelper storeHelper = new PravegaTablesStoreHelper(segmentHelper, authHelper, executor, 2);
    CompletableFuture<Void> connectionDropped = Futures.failedFuture(new WireCommandFailedException(WireCommandType.CREATE_TABLE_SEGMENT, WireCommandFailedException.Reason.ConnectionDropped));
    doAnswer(x -> connectionDropped).when(segmentHelper).createTableSegment(anyString(), anyString(), anyLong(), anyBoolean(), anyInt(), anyLong());
    AssertExtensions.assertFutureThrows("ConnectionDropped", storeHelper.createTable("table", 0L), e -> Exceptions.unwrap(e) instanceof StoreException.StoreConnectionException);
    CompletableFuture<Void> connectionFailed = Futures.failedFuture(new WireCommandFailedException(WireCommandType.CREATE_TABLE_SEGMENT, WireCommandFailedException.Reason.ConnectionFailed));
    doAnswer(x -> connectionFailed).when(segmentHelper).createTableSegment(anyString(), anyString(), anyLong(), anyBoolean(), anyInt(), anyLong());
    AssertExtensions.assertFutureThrows("ConnectionFailed", storeHelper.createTable("table", 0L), e -> Exceptions.unwrap(e) instanceof StoreException.StoreConnectionException);
    CompletableFuture<Void> authFailed = Futures.failedFuture(new WireCommandFailedException(WireCommandType.CREATE_TABLE_SEGMENT, WireCommandFailedException.Reason.AuthFailed));
    doAnswer(x -> connectionFailed).when(segmentHelper).createTableSegment(anyString(), anyString(), anyLong(), anyBoolean(), anyInt(), anyLong());
    AssertExtensions.assertFutureThrows("AuthFailed", storeHelper.createTable("table", 0L), e -> Exceptions.unwrap(e) instanceof StoreException.StoreConnectionException);
}
Also used : GrpcAuthHelper(io.pravega.controller.server.security.auth.GrpcAuthHelper) WireCommandFailedException(io.pravega.controller.server.WireCommandFailedException) SegmentHelper(io.pravega.controller.server.SegmentHelper) StoreException(io.pravega.controller.store.stream.StoreException) Test(org.junit.Test)

Example 12 with GrpcAuthHelper

use of io.pravega.controller.server.security.auth.GrpcAuthHelper in project pravega by pravega.

the class PravegaTablesStreamTest method setup.

@Override
public void setup() throws Exception {
    SegmentHelper segmentHelper = SegmentHelperMock.getSegmentHelperMockForTables(executor);
    GrpcAuthHelper authHelper = GrpcAuthHelper.getDisabledAuthHelper();
    storeHelper = new PravegaTablesStoreHelper(segmentHelper, authHelper, executor);
    orderer = new ZkOrderedStore("txnOrderer", new ZKStoreHelper(PRAVEGA_ZK_CURATOR_RESOURCE.client, executor), executor);
    store = new PravegaTablesStreamMetadataStore(segmentHelper, PRAVEGA_ZK_CURATOR_RESOURCE.client, executor, Duration.ofSeconds(1), authHelper);
}
Also used : ZKStoreHelper(io.pravega.controller.store.ZKStoreHelper) PravegaTablesStoreHelper(io.pravega.controller.store.PravegaTablesStoreHelper) GrpcAuthHelper(io.pravega.controller.server.security.auth.GrpcAuthHelper) SegmentHelper(io.pravega.controller.server.SegmentHelper)

Example 13 with GrpcAuthHelper

use of io.pravega.controller.server.security.auth.GrpcAuthHelper in project pravega by pravega.

the class StreamTest method testConcurrentGetSuccessorScalePravegaTables.

@Test(timeout = 10000)
public void testConcurrentGetSuccessorScalePravegaTables() throws Exception {
    @Cleanup SegmentHelper segmentHelper = SegmentHelperMock.getSegmentHelperMockForTables(executorService());
    GrpcAuthHelper authHelper = GrpcAuthHelper.getDisabledAuthHelper();
    try (final StreamMetadataStore store = new PravegaTablesStreamMetadataStore(segmentHelper, PRAVEGA_ZK_CURATOR_RESOURCE.client, executorService(), authHelper)) {
        testConcurrentGetSuccessorScale(store, (x, y) -> {
            PravegaTablesStoreHelper storeHelper = new PravegaTablesStoreHelper(segmentHelper, authHelper, executorService());
            PravegaTablesScope scope = new PravegaTablesScope(x, storeHelper);
            OperationContext context = getContext();
            Futures.exceptionallyExpecting(scope.createScope(context), e -> Exceptions.unwrap(e) instanceof StoreException.DataExistsException, null).join();
            scope.addStreamToScope(y, context).join();
            return new PravegaTablesStream(x, y, storeHelper, orderer, () -> 0, scope::getStreamsInScopeTableName, executorService());
        });
    }
}
Also used : TestOperationContext(io.pravega.controller.store.TestOperationContext) Arrays(java.util.Arrays) SegmentHelper(io.pravega.controller.server.SegmentHelper) BiFunction(java.util.function.BiFunction) Exceptions(io.pravega.common.Exceptions) Cleanup(lombok.Cleanup) Mockito.spy(org.mockito.Mockito.spy) StreamConfiguration(io.pravega.client.stream.StreamConfiguration) ArrayList(java.util.ArrayList) RetryOneTime(org.apache.curator.retry.RetryOneTime) Lists(com.google.common.collect.Lists) VersionedMetadata(io.pravega.controller.store.VersionedMetadata) StateRecord(io.pravega.controller.store.stream.records.StateRecord) Map(java.util.Map) After(org.junit.After) ClassRule(org.junit.ClassRule) Before(org.junit.Before) EpochTransitionRecord(io.pravega.controller.store.stream.records.EpochTransitionRecord) NameUtils(io.pravega.shared.NameUtils) SegmentHelperMock(io.pravega.controller.mocks.SegmentHelperMock) Assert.assertTrue(org.junit.Assert.assertTrue) PravegaZkCuratorResource(io.pravega.controller.PravegaZkCuratorResource) Test(org.junit.Test) Collectors(java.util.stream.Collectors) PravegaTablesScope(io.pravega.controller.store.PravegaTablesScope) ExecutionException(java.util.concurrent.ExecutionException) ZKStoreHelper(io.pravega.controller.store.ZKStoreHelper) StreamConfigurationRecord(io.pravega.controller.store.stream.records.StreamConfigurationRecord) AbstractMap(java.util.AbstractMap) List(java.util.List) ThreadPooledTestSuite(io.pravega.test.common.ThreadPooledTestSuite) TestOperationContext(io.pravega.controller.store.TestOperationContext) RetryPolicy(org.apache.curator.RetryPolicy) PravegaTablesStoreHelper(io.pravega.controller.store.PravegaTablesStoreHelper) ScalingPolicy(io.pravega.client.stream.ScalingPolicy) Futures(io.pravega.common.concurrent.Futures) Assert.assertEquals(org.junit.Assert.assertEquals) GrpcAuthHelper(io.pravega.controller.server.security.auth.GrpcAuthHelper) PravegaTablesStoreHelper(io.pravega.controller.store.PravegaTablesStoreHelper) GrpcAuthHelper(io.pravega.controller.server.security.auth.GrpcAuthHelper) PravegaTablesScope(io.pravega.controller.store.PravegaTablesScope) SegmentHelper(io.pravega.controller.server.SegmentHelper) Cleanup(lombok.Cleanup) Test(org.junit.Test)

Example 14 with GrpcAuthHelper

use of io.pravega.controller.server.security.auth.GrpcAuthHelper in project pravega by pravega.

the class StreamMetadataTasksTest method testWorkflowCompletionTimeout.

@Test(timeout = 30000)
public void testWorkflowCompletionTimeout() {
    EventHelper helper = EventHelperMock.getEventHelperMock(executor, "host", ((AbstractStreamMetadataStore) streamStorePartialMock).getHostTaskIndex());
    StreamMetadataTasks streamMetadataTask = new StreamMetadataTasks(streamStorePartialMock, bucketStore, TaskStoreFactory.createZKStore(zkClient, executor), SegmentHelperMock.getSegmentHelperMock(), executor, "host", new GrpcAuthHelper(authEnabled, "key", 300), helper);
    streamMetadataTask.setCompletionTimeoutMillis(500L);
    StreamConfiguration configuration = StreamConfiguration.builder().scalingPolicy(ScalingPolicy.fixed(1)).build();
    String completion = "completion";
    streamStorePartialMock.createStream(SCOPE, completion, configuration, System.currentTimeMillis(), null, executor).join();
    streamStorePartialMock.setState(SCOPE, completion, State.ACTIVE, null, executor).join();
    WriterMock requestEventWriter = new WriterMock(streamMetadataTask, executor);
    streamMetadataTask.setRequestEventWriter(requestEventWriter);
    StreamConfiguration configuration2 = StreamConfiguration.builder().scalingPolicy(ScalingPolicy.fixed(3)).build();
    AssertExtensions.assertFutureThrows("update timedout", streamMetadataTask.updateStream(SCOPE, completion, configuration2, 0L), e -> Exceptions.unwrap(e) instanceof TimeoutException);
    ControllerEvent event = requestEventWriter.eventQueue.poll();
    assertTrue(event instanceof UpdateStreamEvent);
    VersionedMetadata<StreamConfigurationRecord> configurationRecord = streamStorePartialMock.getConfigurationRecord(SCOPE, completion, null, executor).join();
    assertTrue(configurationRecord.getObject().isUpdating());
    Map<Long, Long> streamCut = Collections.singletonMap(0L, 0L);
    AssertExtensions.assertFutureThrows("truncate timedout", streamMetadataTask.truncateStream(SCOPE, completion, streamCut, 0L), e -> Exceptions.unwrap(e) instanceof TimeoutException);
    event = requestEventWriter.eventQueue.poll();
    assertTrue(event instanceof TruncateStreamEvent);
    VersionedMetadata<StreamTruncationRecord> truncationRecord = streamStorePartialMock.getTruncationRecord(SCOPE, completion, null, executor).join();
    assertTrue(truncationRecord.getObject().isUpdating());
    AssertExtensions.assertFutureThrows("seal timedout", streamMetadataTask.sealStream(SCOPE, completion, 0L), e -> Exceptions.unwrap(e) instanceof TimeoutException);
    event = requestEventWriter.eventQueue.poll();
    assertTrue(event instanceof SealStreamEvent);
    VersionedMetadata<State> state = streamStorePartialMock.getVersionedState(SCOPE, completion, null, executor).join();
    assertEquals(state.getObject(), State.SEALING);
    streamStorePartialMock.setState(SCOPE, completion, State.SEALED, null, executor).join();
    AssertExtensions.assertFutureThrows("delete timedout", streamMetadataTask.deleteStream(SCOPE, completion, 0L), e -> Exceptions.unwrap(e) instanceof TimeoutException);
    event = requestEventWriter.eventQueue.poll();
    assertTrue(event instanceof DeleteStreamEvent);
}
Also used : DeleteStreamEvent(io.pravega.shared.controller.event.DeleteStreamEvent) StreamTruncationRecord(io.pravega.controller.store.stream.records.StreamTruncationRecord) SealStreamEvent(io.pravega.shared.controller.event.SealStreamEvent) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) ControllerEvent(io.pravega.shared.controller.event.ControllerEvent) TruncateStreamEvent(io.pravega.shared.controller.event.TruncateStreamEvent) GrpcAuthHelper(io.pravega.controller.server.security.auth.GrpcAuthHelper) State(io.pravega.controller.store.stream.State) StreamConfiguration(io.pravega.client.stream.StreamConfiguration) UpdateStreamEvent(io.pravega.shared.controller.event.UpdateStreamEvent) AtomicLong(java.util.concurrent.atomic.AtomicLong) ArgumentMatchers.anyLong(org.mockito.ArgumentMatchers.anyLong) EventHelper(io.pravega.controller.task.EventHelper) StreamConfigurationRecord(io.pravega.controller.store.stream.records.StreamConfigurationRecord) ControllerEventStreamWriterMock(io.pravega.controller.mocks.ControllerEventStreamWriterMock) EventStreamWriterMock(io.pravega.controller.mocks.EventStreamWriterMock) TimeoutException(java.util.concurrent.TimeoutException) Test(org.junit.Test)

Example 15 with GrpcAuthHelper

use of io.pravega.controller.server.security.auth.GrpcAuthHelper in project pravega by pravega.

the class StreamTransactionMetadataTasksTest method writerInitializationTest.

@Test(timeout = 10000)
public void writerInitializationTest() throws Exception {
    EventStreamWriterMock<CommitEvent> commitWriter = new EventStreamWriterMock<>();
    EventStreamWriterMock<AbortEvent> abortWriter = new EventStreamWriterMock<>();
    StreamMetadataStore streamStoreMock = spy(StreamStoreFactory.createZKStore(zkClient, executor));
    final long leasePeriod = 5000;
    // region close before initialize
    txnTasks = new StreamTransactionMetadataTasks(streamStoreMock, SegmentHelperMock.getSegmentHelperMock(), executor, "host", new GrpcAuthHelper(this.authEnabled, "secret", 300));
    CompletableFuture<Void> future = txnTasks.writeCommitEvent(new CommitEvent("scope", "stream", 0));
    assertFalse(future.isDone());
    txnTasks.close();
    AssertExtensions.assertFutureThrows("", future, e -> Exceptions.unwrap(e) instanceof CancellationException);
    // endregion
    // region test initialize writers with client factory
    txnTasks = new StreamTransactionMetadataTasks(streamStoreMock, SegmentHelperMock.getSegmentHelperMock(), executor, "host", new GrpcAuthHelper(this.authEnabled, "secret", 300));
    future = txnTasks.writeCommitEvent(new CommitEvent("scope", "stream", 0));
    EventStreamClientFactory cfMock = mock(EventStreamClientFactory.class);
    ControllerEventProcessorConfig eventProcConfigMock = mock(ControllerEventProcessorConfig.class);
    String commitStream = "commitStream";
    doAnswer(x -> commitStream).when(eventProcConfigMock).getCommitStreamName();
    doAnswer(x -> commitWriter).when(cfMock).createEventWriter(eq(commitStream), any(), any());
    String abortStream = "abortStream";
    doAnswer(x -> abortStream).when(eventProcConfigMock).getAbortStreamName();
    doAnswer(x -> abortWriter).when(cfMock).createEventWriter(eq(abortStream), any(), any());
    // future should not have completed as we have not initialized the writers.
    assertFalse(future.isDone());
    // initialize the writers. write future should have completed now.
    txnTasks.initializeStreamWriters(cfMock, eventProcConfigMock);
    assertTrue(Futures.await(future));
    txnTasks.close();
    // endregion
    // region test method calls and initialize writers with direct writer set up method call
    txnTasks = new StreamTransactionMetadataTasks(streamStoreMock, SegmentHelperMock.getSegmentHelperMock(), executor, "host", new GrpcAuthHelper(this.authEnabled, "secret", 300));
    streamStore.createScope(SCOPE, null, executor).join();
    streamStore.createStream(SCOPE, STREAM, StreamConfiguration.builder().scalingPolicy(ScalingPolicy.fixed(1)).build(), 1L, null, executor).join();
    streamStore.setState(SCOPE, STREAM, State.ACTIVE, null, executor).join();
    CompletableFuture<Pair<VersionedTransactionData, List<StreamSegmentRecord>>> createFuture = txnTasks.createTxn(SCOPE, STREAM, leasePeriod, 0L, 0L);
    // create and ping transactions should not wait for writer initialization and complete immediately.
    createFuture.join();
    assertTrue(Futures.await(createFuture));
    UUID txnId = createFuture.join().getKey().getId();
    CompletableFuture<PingTxnStatus> pingFuture = txnTasks.pingTxn(SCOPE, STREAM, txnId, leasePeriod, 0L);
    assertTrue(Futures.await(pingFuture));
    CompletableFuture<TxnStatus> commitFuture = txnTasks.commitTxn(SCOPE, STREAM, txnId, 0L);
    assertFalse(commitFuture.isDone());
    txnTasks.initializeStreamWriters(commitWriter, abortWriter);
    assertTrue(Futures.await(commitFuture));
    UUID txnId2 = txnTasks.createTxn(SCOPE, STREAM, leasePeriod, 0L, 1024 * 1024L).join().getKey().getId();
    assertTrue(Futures.await(txnTasks.abortTxn(SCOPE, STREAM, txnId2, null, 0L)));
}
Also used : ControllerEventProcessorConfig(io.pravega.controller.server.eventProcessor.ControllerEventProcessorConfig) EventStreamWriterMock(io.pravega.controller.mocks.EventStreamWriterMock) EventStreamClientFactory(io.pravega.client.EventStreamClientFactory) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) PingTxnStatus(io.pravega.controller.stream.api.grpc.v1.Controller.PingTxnStatus) StreamMetadataStore(io.pravega.controller.store.stream.StreamMetadataStore) StreamSegmentRecord(io.pravega.controller.store.stream.records.StreamSegmentRecord) GrpcAuthHelper(io.pravega.controller.server.security.auth.GrpcAuthHelper) CancellationException(java.util.concurrent.CancellationException) CommitEvent(io.pravega.shared.controller.event.CommitEvent) AbortEvent(io.pravega.shared.controller.event.AbortEvent) UUID(java.util.UUID) TxnStatus(io.pravega.controller.store.stream.TxnStatus) PingTxnStatus(io.pravega.controller.stream.api.grpc.v1.Controller.PingTxnStatus) Pair(org.apache.commons.lang3.tuple.Pair) ImmutablePair(org.apache.commons.lang3.tuple.ImmutablePair) Test(org.junit.Test)

Aggregations

GrpcAuthHelper (io.pravega.controller.server.security.auth.GrpcAuthHelper)28 Test (org.junit.Test)22 SegmentHelper (io.pravega.controller.server.SegmentHelper)10 Controller (io.pravega.controller.stream.api.grpc.v1.Controller)10 StreamMetadataStore (io.pravega.controller.store.stream.StreamMetadataStore)9 StreamConfiguration (io.pravega.client.stream.StreamConfiguration)8 AuthorizationResourceImpl (io.pravega.shared.security.auth.AuthorizationResourceImpl)8 UUID (java.util.UUID)8 TaskMetadataStore (io.pravega.controller.store.task.TaskMetadataStore)7 ArrayList (java.util.ArrayList)7 EventStreamWriterMock (io.pravega.controller.mocks.EventStreamWriterMock)6 EventHelper (io.pravega.controller.task.EventHelper)6 List (java.util.List)6 ScalingPolicy (io.pravega.client.stream.ScalingPolicy)5 AbortEvent (io.pravega.shared.controller.event.AbortEvent)5 CommitEvent (io.pravega.shared.controller.event.CommitEvent)5 ControllerEventStreamWriterMock (io.pravega.controller.mocks.ControllerEventStreamWriterMock)4 BucketStore (io.pravega.controller.store.stream.BucketStore)4 TableMetadataTasks (io.pravega.controller.task.KeyValueTable.TableMetadataTasks)4 StreamMetadataTasks (io.pravega.controller.task.Stream.StreamMetadataTasks)4