Search in sources :

Example 1 with ResultTypeQueryable

use of org.apache.flink.api.java.typeutils.ResultTypeQueryable in project flink by apache.

the class StreamExecutionEnvironment method addSource.

/**
	 * Ads a data source with a custom type information thus opening a
	 * {@link DataStream}. Only in very special cases does the user need to
	 * support type information. Otherwise use
	 * {@link #addSource(org.apache.flink.streaming.api.functions.source.SourceFunction)}
	 *
	 * @param function
	 * 		the user defined function
	 * @param sourceName
	 * 		Name of the data source
	 * @param <OUT>
	 * 		type of the returned stream
	 * @param typeInfo
	 * 		the user defined type information for the stream
	 * @return the data stream constructed
	 */
@SuppressWarnings("unchecked")
public <OUT> DataStreamSource<OUT> addSource(SourceFunction<OUT> function, String sourceName, TypeInformation<OUT> typeInfo) {
    if (typeInfo == null) {
        if (function instanceof ResultTypeQueryable) {
            typeInfo = ((ResultTypeQueryable<OUT>) function).getProducedType();
        } else {
            try {
                typeInfo = TypeExtractor.createTypeInfo(SourceFunction.class, function.getClass(), 0, null, null);
            } catch (final InvalidTypesException e) {
                typeInfo = (TypeInformation<OUT>) new MissingTypeInfo(sourceName, e);
            }
        }
    }
    boolean isParallel = function instanceof ParallelSourceFunction;
    clean(function);
    StreamSource<OUT, ?> sourceOperator;
    if (function instanceof StoppableFunction) {
        sourceOperator = new StoppableStreamSource<>(cast2StoppableSourceFunction(function));
    } else {
        sourceOperator = new StreamSource<>(function);
    }
    return new DataStreamSource<>(this, typeInfo, sourceOperator, isParallel, sourceName);
}
Also used : ParallelSourceFunction(org.apache.flink.streaming.api.functions.source.ParallelSourceFunction) SourceFunction(org.apache.flink.streaming.api.functions.source.SourceFunction) InputFormatSourceFunction(org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction) MissingTypeInfo(org.apache.flink.api.java.typeutils.MissingTypeInfo) ResultTypeQueryable(org.apache.flink.api.java.typeutils.ResultTypeQueryable) DataStreamSource(org.apache.flink.streaming.api.datastream.DataStreamSource) StoppableFunction(org.apache.flink.api.common.functions.StoppableFunction) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) ParallelSourceFunction(org.apache.flink.streaming.api.functions.source.ParallelSourceFunction) InvalidTypesException(org.apache.flink.api.common.functions.InvalidTypesException)

Example 2 with ResultTypeQueryable

use of org.apache.flink.api.java.typeutils.ResultTypeQueryable in project flink by apache.

the class Graph method mapEdges.

/**
 * Apply a function to the attribute of each edge in the graph.
 *
 * @param mapper the map function to apply.
 * @return a new graph
 */
@SuppressWarnings({ "unchecked", "rawtypes" })
public <NV> Graph<K, VV, NV> mapEdges(final MapFunction<Edge<K, EV>, NV> mapper) {
    TypeInformation<K> keyType = ((TupleTypeInfo<?>) edges.getType()).getTypeAt(0);
    TypeInformation<NV> valueType;
    if (mapper instanceof ResultTypeQueryable) {
        valueType = ((ResultTypeQueryable) mapper).getProducedType();
    } else {
        valueType = TypeExtractor.createTypeInfo(MapFunction.class, mapper.getClass(), 1, edges.getType(), null);
    }
    TypeInformation<Edge<K, NV>> returnType = (TypeInformation<Edge<K, NV>>) new TupleTypeInfo(Edge.class, keyType, keyType, valueType);
    return mapEdges(mapper, returnType);
}
Also used : ResultTypeQueryable(org.apache.flink.api.java.typeutils.ResultTypeQueryable) MapFunction(org.apache.flink.api.common.functions.MapFunction) FlatMapFunction(org.apache.flink.api.common.functions.FlatMapFunction) TupleTypeInfo(org.apache.flink.api.java.typeutils.TupleTypeInfo) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation)

Example 3 with ResultTypeQueryable

use of org.apache.flink.api.java.typeutils.ResultTypeQueryable in project flink by apache.

the class Graph method mapVertices.

/**
 * Apply a function to the attribute of each vertex in the graph.
 *
 * @param mapper the map function to apply.
 * @return a new graph
 */
@SuppressWarnings({ "unchecked", "rawtypes" })
public <NV> Graph<K, NV, EV> mapVertices(final MapFunction<Vertex<K, VV>, NV> mapper) {
    TypeInformation<K> keyType = ((TupleTypeInfo<?>) vertices.getType()).getTypeAt(0);
    TypeInformation<NV> valueType;
    if (mapper instanceof ResultTypeQueryable) {
        valueType = ((ResultTypeQueryable) mapper).getProducedType();
    } else {
        valueType = TypeExtractor.createTypeInfo(MapFunction.class, mapper.getClass(), 1, vertices.getType(), null);
    }
    TypeInformation<Vertex<K, NV>> returnType = (TypeInformation<Vertex<K, NV>>) new TupleTypeInfo(Vertex.class, keyType, valueType);
    return mapVertices(mapper, returnType);
}
Also used : ResultTypeQueryable(org.apache.flink.api.java.typeutils.ResultTypeQueryable) MapFunction(org.apache.flink.api.common.functions.MapFunction) FlatMapFunction(org.apache.flink.api.common.functions.FlatMapFunction) TupleTypeInfo(org.apache.flink.api.java.typeutils.TupleTypeInfo) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation)

Aggregations

TypeInformation (org.apache.flink.api.common.typeinfo.TypeInformation)3 ResultTypeQueryable (org.apache.flink.api.java.typeutils.ResultTypeQueryable)3 FlatMapFunction (org.apache.flink.api.common.functions.FlatMapFunction)2 MapFunction (org.apache.flink.api.common.functions.MapFunction)2 TupleTypeInfo (org.apache.flink.api.java.typeutils.TupleTypeInfo)2 InvalidTypesException (org.apache.flink.api.common.functions.InvalidTypesException)1 StoppableFunction (org.apache.flink.api.common.functions.StoppableFunction)1 MissingTypeInfo (org.apache.flink.api.java.typeutils.MissingTypeInfo)1 DataStreamSource (org.apache.flink.streaming.api.datastream.DataStreamSource)1 InputFormatSourceFunction (org.apache.flink.streaming.api.functions.source.InputFormatSourceFunction)1 ParallelSourceFunction (org.apache.flink.streaming.api.functions.source.ParallelSourceFunction)1 SourceFunction (org.apache.flink.streaming.api.functions.source.SourceFunction)1