Search in sources :

Example 11 with MockStreamTaskBuilder

use of org.apache.flink.streaming.util.MockStreamTaskBuilder in project flink by apache.

the class StreamSourceOperatorLatencyMetricsTest method setupSourceOperator.

// ------------------------------------------------------------------------
@SuppressWarnings("unchecked")
private static <T> void setupSourceOperator(StreamSource<T, ?> operator, ExecutionConfig executionConfig, Environment env, TimerService timerService) {
    StreamConfig cfg = new StreamConfig(new Configuration());
    cfg.setStateBackend(new MemoryStateBackend());
    cfg.setTimeCharacteristic(TimeCharacteristic.EventTime);
    cfg.setOperatorID(new OperatorID());
    try {
        MockStreamTask mockTask = new MockStreamTaskBuilder(env).setConfig(cfg).setExecutionConfig(executionConfig).setTimerService(timerService).build();
        operator.setProcessingTimeService(mockTask.getProcessingTimeServiceFactory().createProcessingTimeService(null));
        operator.setup(mockTask, cfg, (Output<StreamRecord<T>>) mock(Output.class));
    } catch (Exception e) {
        e.printStackTrace();
        fail(e.getMessage());
    }
}
Also used : MockStreamTaskBuilder(org.apache.flink.streaming.util.MockStreamTaskBuilder) StreamRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord) Configuration(org.apache.flink.configuration.Configuration) MockStreamTask(org.apache.flink.streaming.util.MockStreamTask) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) StreamConfig(org.apache.flink.streaming.api.graph.StreamConfig) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID)

Example 12 with MockStreamTaskBuilder

use of org.apache.flink.streaming.util.MockStreamTaskBuilder in project flink by apache.

the class StreamingRuntimeContextTest method createDescriptorCapturingMockOp.

@SuppressWarnings("unchecked")
private static AbstractStreamOperator<?> createDescriptorCapturingMockOp(final AtomicReference<Object> ref, final ExecutionConfig config, Environment environment) throws Exception {
    AbstractStreamOperator<?> operator = new AbstractStreamOperator<Object>() {

        @Override
        public void setup(StreamTask<?, ?> containingTask, StreamConfig config, Output<StreamRecord<Object>> output) {
            super.setup(containingTask, config, output);
        }
    };
    StreamConfig streamConfig = new StreamConfig(new Configuration());
    streamConfig.setOperatorID(new OperatorID());
    operator.setup(new MockStreamTaskBuilder(environment).setExecutionConfig(config).build(), streamConfig, new CollectorOutput<>(new ArrayList<>()));
    StreamTaskStateInitializer streamTaskStateManager = new StreamTaskStateInitializerImpl(environment, new MemoryStateBackend());
    KeyedStateBackend keyedStateBackend = mock(KeyedStateBackend.class);
    DefaultKeyedStateStore keyedStateStore = new DefaultKeyedStateStore(keyedStateBackend, config);
    doAnswer(new Answer<Object>() {

        @Override
        public Object answer(InvocationOnMock invocationOnMock) throws Throwable {
            ref.set(invocationOnMock.getArguments()[2]);
            return null;
        }
    }).when(keyedStateBackend).getPartitionedState(Matchers.any(), any(TypeSerializer.class), any(StateDescriptor.class));
    operator.initializeState(streamTaskStateManager);
    operator.getRuntimeContext().setKeyedStateStore(keyedStateStore);
    return operator;
}
Also used : MockStreamTaskBuilder(org.apache.flink.streaming.util.MockStreamTaskBuilder) AbstractKeyedStateBackend(org.apache.flink.runtime.state.AbstractKeyedStateBackend) KeyedStateBackend(org.apache.flink.runtime.state.KeyedStateBackend) Configuration(org.apache.flink.configuration.Configuration) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) ArrayList(java.util.ArrayList) StreamConfig(org.apache.flink.streaming.api.graph.StreamConfig) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) InvocationOnMock(org.mockito.invocation.InvocationOnMock) CollectorOutput(org.apache.flink.streaming.util.CollectorOutput) TypeSerializer(org.apache.flink.api.common.typeutils.TypeSerializer) ReducingStateDescriptor(org.apache.flink.api.common.state.ReducingStateDescriptor) ListStateDescriptor(org.apache.flink.api.common.state.ListStateDescriptor) MapStateDescriptor(org.apache.flink.api.common.state.MapStateDescriptor) AggregatingStateDescriptor(org.apache.flink.api.common.state.AggregatingStateDescriptor) StateDescriptor(org.apache.flink.api.common.state.StateDescriptor) ValueStateDescriptor(org.apache.flink.api.common.state.ValueStateDescriptor) StreamTask(org.apache.flink.streaming.runtime.tasks.StreamTask) DefaultKeyedStateStore(org.apache.flink.runtime.state.DefaultKeyedStateStore)

Aggregations

MockStreamTaskBuilder (org.apache.flink.streaming.util.MockStreamTaskBuilder)12 MockEnvironment (org.apache.flink.runtime.operators.testutils.MockEnvironment)9 Test (org.junit.Test)7 StreamConfig (org.apache.flink.streaming.api.graph.StreamConfig)6 Configuration (org.apache.flink.configuration.Configuration)5 OperatorID (org.apache.flink.runtime.jobgraph.OperatorID)5 MailboxExecutor (org.apache.flink.api.common.operators.MailboxExecutor)4 CheckpointOptions (org.apache.flink.runtime.checkpoint.CheckpointOptions)4 StreamRecord (org.apache.flink.streaming.runtime.streamrecord.StreamRecord)4 ArrayList (java.util.ArrayList)3 Nullable (javax.annotation.Nullable)3 CheckpointMetaData (org.apache.flink.runtime.checkpoint.CheckpointMetaData)3 CheckpointMetricsBuilder (org.apache.flink.runtime.checkpoint.CheckpointMetricsBuilder)3 TaskIOMetricGroup (org.apache.flink.runtime.metrics.groups.TaskIOMetricGroup)3 MemoryStateBackend (org.apache.flink.runtime.state.memory.MemoryStateBackend)3 RunnableWithException (org.apache.flink.util.function.RunnableWithException)3 Arrays (java.util.Arrays)2 Collections (java.util.Collections)2 List (java.util.List)2 AtomicReference (java.util.concurrent.atomic.AtomicReference)2