Search in sources :

Example 1 with CollectResultIterator

use of org.apache.flink.streaming.api.operators.collect.CollectResultIterator in project flink by apache.

the class DataStream method executeAndCollectWithClient.

ClientAndIterator<T> executeAndCollectWithClient(String jobExecutionName) throws Exception {
    TypeSerializer<T> serializer = getType().createSerializer(getExecutionEnvironment().getConfig());
    String accumulatorName = "dataStreamCollect_" + UUID.randomUUID().toString();
    StreamExecutionEnvironment env = getExecutionEnvironment();
    CollectSinkOperatorFactory<T> factory = new CollectSinkOperatorFactory<>(serializer, accumulatorName);
    CollectSinkOperator<T> operator = (CollectSinkOperator<T>) factory.getOperator();
    CollectResultIterator<T> iterator = new CollectResultIterator<>(operator.getOperatorIdFuture(), serializer, accumulatorName, env.getCheckpointConfig());
    CollectStreamSink<T> sink = new CollectStreamSink<>(this, factory);
    sink.name("Data stream collect sink");
    env.addOperator(sink.getTransformation());
    final JobClient jobClient = env.executeAsync(jobExecutionName);
    iterator.setJobClient(jobClient);
    return new ClientAndIterator<>(jobClient, iterator);
}
Also used : ClientAndIterator(org.apache.flink.streaming.api.operators.collect.ClientAndIterator) CollectResultIterator(org.apache.flink.streaming.api.operators.collect.CollectResultIterator) CollectStreamSink(org.apache.flink.streaming.api.operators.collect.CollectStreamSink) JobClient(org.apache.flink.core.execution.JobClient) CollectSinkOperator(org.apache.flink.streaming.api.operators.collect.CollectSinkOperator) CollectSinkOperatorFactory(org.apache.flink.streaming.api.operators.collect.CollectSinkOperatorFactory) StreamExecutionEnvironment(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)

Aggregations

JobClient (org.apache.flink.core.execution.JobClient)1 StreamExecutionEnvironment (org.apache.flink.streaming.api.environment.StreamExecutionEnvironment)1 ClientAndIterator (org.apache.flink.streaming.api.operators.collect.ClientAndIterator)1 CollectResultIterator (org.apache.flink.streaming.api.operators.collect.CollectResultIterator)1 CollectSinkOperator (org.apache.flink.streaming.api.operators.collect.CollectSinkOperator)1 CollectSinkOperatorFactory (org.apache.flink.streaming.api.operators.collect.CollectSinkOperatorFactory)1 CollectStreamSink (org.apache.flink.streaming.api.operators.collect.CollectStreamSink)1