Search in sources :

Example 56 with StringValue

use of org.apache.flink.types.StringValue in project flink by apache.

the class GenericCsvInputFormatTest method readWithHeaderLine.

@Test
public void readWithHeaderLine() {
    try {
        final String fileContent = "colname-1|colname-2|some name 3|column four|\n" + "123|abc|456|def|\n" + "987|xyz|654|pqr|\n";
        final FileInputSplit split = createTempFile(fileContent);
        final Configuration parameters = new Configuration();
        format.setFieldDelimiter("|");
        format.setFieldTypesGeneric(IntValue.class, StringValue.class, IntValue.class, StringValue.class);
        format.setSkipFirstLineAsHeader(true);
        format.configure(parameters);
        format.open(split);
        Value[] values = new Value[] { new IntValue(), new StringValue(), new IntValue(), new StringValue() };
        // first line is skipped as header
        // first row (= second line)
        assertNotNull(format.nextRecord(values));
        // second row (= third line)
        assertNotNull(format.nextRecord(values));
        // exhausted
        assertNull(format.nextRecord(values));
        // exhausted
        assertTrue(format.reachedEnd());
    } catch (Exception ex) {
        fail("Test failed due to a " + ex.getClass().getSimpleName() + ": " + ex.getMessage());
    }
}
Also used : FileInputSplit(org.apache.flink.core.fs.FileInputSplit) Configuration(org.apache.flink.configuration.Configuration) IntValue(org.apache.flink.types.IntValue) DoubleValue(org.apache.flink.types.DoubleValue) LongValue(org.apache.flink.types.LongValue) Value(org.apache.flink.types.Value) StringValue(org.apache.flink.types.StringValue) StringValue(org.apache.flink.types.StringValue) IntValue(org.apache.flink.types.IntValue) IOException(java.io.IOException) Test(org.junit.Test)

Example 57 with StringValue

use of org.apache.flink.types.StringValue in project flink by apache.

the class GenericCsvInputFormatTest method readWithEmptyField.

@Test
public void readWithEmptyField() {
    try {
        final String fileContent = "abc|def|ghijk\nabc||hhg\n|||";
        final FileInputSplit split = createTempFile(fileContent);
        final Configuration parameters = new Configuration();
        format.setFieldDelimiter("|");
        format.setFieldTypesGeneric(StringValue.class, StringValue.class, StringValue.class);
        format.configure(parameters);
        format.open(split);
        Value[] values = new Value[] { new StringValue(), new StringValue(), new StringValue() };
        values = format.nextRecord(values);
        assertNotNull(values);
        assertEquals("abc", ((StringValue) values[0]).getValue());
        assertEquals("def", ((StringValue) values[1]).getValue());
        assertEquals("ghijk", ((StringValue) values[2]).getValue());
        values = format.nextRecord(values);
        assertNotNull(values);
        assertEquals("abc", ((StringValue) values[0]).getValue());
        assertEquals("", ((StringValue) values[1]).getValue());
        assertEquals("hhg", ((StringValue) values[2]).getValue());
        values = format.nextRecord(values);
        assertNotNull(values);
        assertEquals("", ((StringValue) values[0]).getValue());
        assertEquals("", ((StringValue) values[1]).getValue());
        assertEquals("", ((StringValue) values[2]).getValue());
    } catch (Exception ex) {
        fail("Test failed due to a " + ex.getClass().getSimpleName() + ": " + ex.getMessage());
    }
}
Also used : FileInputSplit(org.apache.flink.core.fs.FileInputSplit) Configuration(org.apache.flink.configuration.Configuration) IntValue(org.apache.flink.types.IntValue) DoubleValue(org.apache.flink.types.DoubleValue) LongValue(org.apache.flink.types.LongValue) Value(org.apache.flink.types.Value) StringValue(org.apache.flink.types.StringValue) StringValue(org.apache.flink.types.StringValue) IOException(java.io.IOException) Test(org.junit.Test)

Example 58 with StringValue

use of org.apache.flink.types.StringValue in project flink by apache.

the class ToNullValueTest method testTranslation.

@Test
public void testTranslation() throws Exception {
    NullValue reuse = NullValue.getInstance();
    assertEquals(NullValue.getInstance(), new ToNullValue<>().translate(new DoubleValue(), reuse));
    assertEquals(NullValue.getInstance(), new ToNullValue<>().translate(new FloatValue(), reuse));
    assertEquals(NullValue.getInstance(), new ToNullValue<>().translate(new IntValue(), reuse));
    assertEquals(NullValue.getInstance(), new ToNullValue<>().translate(new LongValue(), reuse));
    assertEquals(NullValue.getInstance(), new ToNullValue<>().translate(new StringValue(), reuse));
}
Also used : NullValue(org.apache.flink.types.NullValue) DoubleValue(org.apache.flink.types.DoubleValue) LongValue(org.apache.flink.types.LongValue) FloatValue(org.apache.flink.types.FloatValue) StringValue(org.apache.flink.types.StringValue) IntValue(org.apache.flink.types.IntValue) Test(org.junit.Test)

Example 59 with StringValue

use of org.apache.flink.types.StringValue in project flink by apache.

the class InstantiationUtilTest method testInstantiationOfStringValueAndCastToValue.

@Test
public void testInstantiationOfStringValueAndCastToValue() {
    StringValue stringValue = InstantiationUtil.instantiate(StringValue.class, Value.class);
    assertNotNull(stringValue);
}
Also used : StringValue(org.apache.flink.types.StringValue) Test(org.junit.Test)

Example 60 with StringValue

use of org.apache.flink.types.StringValue in project flink by apache.

the class InstantiationUtilTest method testInstantiationOfStringValue.

@Test
public void testInstantiationOfStringValue() {
    StringValue stringValue = InstantiationUtil.instantiate(StringValue.class, null);
    assertNotNull(stringValue);
}
Also used : StringValue(org.apache.flink.types.StringValue) Test(org.junit.Test)

Aggregations

StringValue (org.apache.flink.types.StringValue)88 Test (org.junit.Test)61 IntValue (org.apache.flink.types.IntValue)35 LongValue (org.apache.flink.types.LongValue)21 IOException (java.io.IOException)17 ArrayList (java.util.ArrayList)15 Record (org.apache.flink.types.Record)13 TupleTypeInfo (org.apache.flink.api.java.typeutils.TupleTypeInfo)12 ExecutionEnvironment (org.apache.flink.api.java.ExecutionEnvironment)11 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)11 DoubleValue (org.apache.flink.types.DoubleValue)11 Value (org.apache.flink.types.Value)10 Tuple3 (org.apache.flink.api.java.tuple.Tuple3)9 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)7 Plan (org.apache.flink.api.common.Plan)7 Configuration (org.apache.flink.configuration.Configuration)7 FileInputSplit (org.apache.flink.core.fs.FileInputSplit)7 NoSuchElementException (java.util.NoSuchElementException)6 File (java.io.File)5 JobExecutionResult (org.apache.flink.api.common.JobExecutionResult)5