Search in sources :

Example 1 with RowTypeInfo

use of org.apache.flink.api.java.typeutils.RowTypeInfo in project flink by apache.

the class HBaseRowInputFormat method getProducedType.

@Override
public TypeInformation<Row> getProducedType() {
    // split the fieldNames
    String[] famNames = schema.getFamilyNames();
    TypeInformation<?>[] typeInfos = new TypeInformation[famNames.length];
    int i = 0;
    for (String family : famNames) {
        typeInfos[i] = new RowTypeInfo(schema.getQualifierTypes(family), schema.getQualifierNames(family));
        i++;
    }
    return new RowTypeInfo(typeInfos, famNames);
}
Also used : RowTypeInfo(org.apache.flink.api.java.typeutils.RowTypeInfo) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation)

Example 2 with RowTypeInfo

use of org.apache.flink.api.java.typeutils.RowTypeInfo in project flink by apache.

the class MaxByOperatorTest method testMaxByRowTypeInfoKeyFieldsForUnsortedGrouping.

/**
 * Validates that no ClassCastException happens should not fail e.g. like in FLINK-8255.
 */
@Test(expected = InvalidProgramException.class)
public void testMaxByRowTypeInfoKeyFieldsForUnsortedGrouping() {
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    TypeInformation[] types = new TypeInformation[] { Types.INT, Types.INT };
    String[] fieldNames = new String[] { "id", "value" };
    RowTypeInfo rowTypeInfo = new RowTypeInfo(types, fieldNames);
    UnsortedGrouping groupDs = env.fromCollection(Collections.singleton(new Row(2)), rowTypeInfo).groupBy(0);
    groupDs.maxBy(1);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) UnsortedGrouping(org.apache.flink.api.java.operators.UnsortedGrouping) Row(org.apache.flink.types.Row) RowTypeInfo(org.apache.flink.api.java.typeutils.RowTypeInfo) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 3 with RowTypeInfo

use of org.apache.flink.api.java.typeutils.RowTypeInfo in project flink by apache.

the class MaxByOperatorTest method testMaxByRowTypeInfoKeyFieldsDataset.

/**
 * Validates that no ClassCastException happens should not fail e.g. like in FLINK-8255.
 */
@Test(expected = InvalidProgramException.class)
public void testMaxByRowTypeInfoKeyFieldsDataset() {
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    TypeInformation[] types = new TypeInformation[] { Types.INT, Types.INT };
    String[] fieldNames = new String[] { "id", "value" };
    RowTypeInfo rowTypeInfo = new RowTypeInfo(types, fieldNames);
    DataSet tupleDs = env.fromCollection(Collections.singleton(new Row(2)), rowTypeInfo);
    tupleDs.maxBy(0);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) DataSet(org.apache.flink.api.java.DataSet) Row(org.apache.flink.types.Row) RowTypeInfo(org.apache.flink.api.java.typeutils.RowTypeInfo) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 4 with RowTypeInfo

use of org.apache.flink.api.java.typeutils.RowTypeInfo in project flink by apache.

the class MinByOperatorTest method testMinByRowTypeInfoKeyFieldsDataset.

/**
 * Validates that no ClassCastException happens should not fail e.g. like in FLINK-8255.
 */
@Test(expected = InvalidProgramException.class)
public void testMinByRowTypeInfoKeyFieldsDataset() {
    final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
    TypeInformation[] types = new TypeInformation[] { Types.INT, Types.INT };
    String[] fieldNames = new String[] { "id", "value" };
    RowTypeInfo rowTypeInfo = new RowTypeInfo(types, fieldNames);
    DataSet tupleDs = env.fromCollection(Collections.singleton(new Row(2)), rowTypeInfo);
    tupleDs.minBy(0);
}
Also used : ExecutionEnvironment(org.apache.flink.api.java.ExecutionEnvironment) DataSet(org.apache.flink.api.java.DataSet) Row(org.apache.flink.types.Row) RowTypeInfo(org.apache.flink.api.java.typeutils.RowTypeInfo) TypeInformation(org.apache.flink.api.common.typeinfo.TypeInformation) Test(org.junit.Test)

Example 5 with RowTypeInfo

use of org.apache.flink.api.java.typeutils.RowTypeInfo in project flink by apache.

the class PythonKeyedProcessOperator method open.

@Override
public void open() throws Exception {
    internalTimerService = getInternalTimerService("user-timers", namespaceSerializer, this);
    keyTypeInfo = new RowTypeInfo(((RowTypeInfo) this.getInputTypeInfo()).getTypeAt(0));
    keyTypeSerializer = PythonTypeUtils.TypeInfoToSerializerConverter.typeInfoSerializerConverter(keyTypeInfo);
    timerDataTypeInfo = createTimerDataTypeInfo(keyTypeInfo);
    timerDataSerializer = PythonTypeUtils.TypeInfoToSerializerConverter.typeInfoSerializerConverter(timerDataTypeInfo);
    timerHandler = new TimerHandler();
    super.open();
}
Also used : TimerHandler(org.apache.flink.streaming.api.operators.python.timer.TimerHandler) RowTypeInfo(org.apache.flink.api.java.typeutils.RowTypeInfo)

Aggregations

RowTypeInfo (org.apache.flink.api.java.typeutils.RowTypeInfo)50 Test (org.junit.Test)34 Row (org.apache.flink.types.Row)32 TypeInformation (org.apache.flink.api.common.typeinfo.TypeInformation)26 Configuration (org.apache.flink.configuration.Configuration)16 FileInputSplit (org.apache.flink.core.fs.FileInputSplit)15 ArrayList (java.util.ArrayList)10 Transformation (org.apache.flink.api.dag.Transformation)8 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)8 SourceTransformation (org.apache.flink.streaming.api.transformations.SourceTransformation)8 TwoInputTransformation (org.apache.flink.streaming.api.transformations.TwoInputTransformation)8 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)6 PythonKeyedProcessOperator (org.apache.flink.streaming.api.operators.python.PythonKeyedProcessOperator)6 IOException (java.io.IOException)4 MapTypeInfo (org.apache.flink.api.java.typeutils.MapTypeInfo)4 File (java.io.File)3 FileOutputStream (java.io.FileOutputStream)3 OutputStreamWriter (java.io.OutputStreamWriter)3 LocalDateTime (java.time.LocalDateTime)3 PrimitiveArrayTypeInfo (org.apache.flink.api.common.typeinfo.PrimitiveArrayTypeInfo)3