Examples with RowFormatInfo org.apache.inlong.sort.formats.common.RowFormatInfo used on opensource projects

Example 21 with RowFormatInfo

use of org.apache.inlong.sort.formats.common.RowFormatInfo in project incubator-inlong by apache.

the class CsvFormatFactory method createProjectedDeserializationSchema.

@Override
public DeserializationSchema<Row> createProjectedDeserializationSchema(Map<String, String> properties, int[] fields) {
    final DescriptorProperties descriptorProperties = getValidatedProperties(properties);
    final RowFormatInfo rowFormatInfo = TableFormatUtils.getRowFormatInfo(descriptorProperties);
    final RowFormatInfo projectedRowFormatInfo = TableFormatUtils.projectRowFormatInfo(rowFormatInfo, fields);
    return buildDeserializationSchema(descriptorProperties, projectedRowFormatInfo);
}

Example 22 with RowFormatInfo

use of org.apache.inlong.sort.formats.common.RowFormatInfo in project incubator-inlong by apache.

the class CsvFormatFactory method createSerializationSchema.

@Override
public CsvSerializationSchema createSerializationSchema(Map<String, String> properties) {
    final DescriptorProperties descriptorProperties = getValidatedProperties(properties);
    final RowFormatInfo rowFormatInfo = TableFormatUtils.getRowFormatInfo(descriptorProperties);
    return buildSerializationSchema(descriptorProperties, rowFormatInfo);
}

Example 23 with RowFormatInfo

use of org.apache.inlong.sort.formats.common.RowFormatInfo in project incubator-inlong by apache.

the class CsvFormatFactory method buildDeserializationSchema.

private static CsvDeserializationSchema buildDeserializationSchema(DescriptorProperties descriptorProperties, RowFormatInfo rowFormatInfo) {
    for (FormatInfo formatInfo : rowFormatInfo.getFieldFormatInfos()) {
        if (!(formatInfo instanceof BasicFormatInfo)) {
            throw new ValidationException("Currently only basic formats " + "are supported in csv formats.");
        }
    }
    CsvDeserializationSchema.Builder builder = new CsvDeserializationSchema.Builder(rowFormatInfo);
    descriptorProperties.getOptionalString(TableFormatConstants.FORMAT_CHARSET).ifPresent(builder::setCharset);
    descriptorProperties.getOptionalCharacter(TableFormatConstants.FORMAT_DELIMITER).ifPresent(builder::setDelimiter);
    descriptorProperties.getOptionalCharacter(TableFormatConstants.FORMAT_ESCAPE_CHARACTER).ifPresent(builder::setEscapeCharacter);
    descriptorProperties.getOptionalCharacter(TableFormatConstants.FORMAT_QUOTE_CHARACTER).ifPresent(builder::setQuoteCharacter);
    descriptorProperties.getOptionalString(TableFormatConstants.FORMAT_NULL_LITERAL).ifPresent(builder::setNullLiteral);
    return builder.build();
}

Example 24 with RowFormatInfo

use of org.apache.inlong.sort.formats.common.RowFormatInfo in project incubator-inlong by apache.

the class CsvFormatFactory method createProjectedSerializationSchema.

@Override
public SerializationSchema<Row> createProjectedSerializationSchema(Map<String, String> properties, int[] fields) {
    final DescriptorProperties descriptorProperties = getValidatedProperties(properties);
    final RowFormatInfo rowFormatInfo = TableFormatUtils.getRowFormatInfo(descriptorProperties);
    final RowFormatInfo projectedRowFormatInfo = TableFormatUtils.projectRowFormatInfo(rowFormatInfo, fields);
    return buildSerializationSchema(descriptorProperties, projectedRowFormatInfo);
}

Example 25 with RowFormatInfo

use of org.apache.inlong.sort.formats.common.RowFormatInfo in project incubator-inlong by apache.

the class CsvDeserializationSchemaTest method testBasicDeserialization.

private static <T> void testBasicDeserialization(Consumer<CsvDeserializationSchema.Builder> config, BasicFormatInfo<T> basicFormatInfo, T expectedRecord, String text) throws IOException {
    RowFormatInfo rowFormatInfo = new RowFormatInfo(new String[] { "f" }, new FormatInfo[] { basicFormatInfo });
    CsvDeserializationSchema.Builder builder = new CsvDeserializationSchema.Builder(rowFormatInfo);
    config.accept(builder);
    CsvDeserializationSchema deserializer = builder.build();
    Row row = deserializer.deserialize(text.getBytes());
    assertEquals(1, row.getArity());
    assertEquals(expectedRecord, row.getField(0));
}