Search in sources :

Example 1 with InLongMsgCsvMixedFormatConverter

use of org.apache.inlong.sort.formats.inlongmsgcsv.InLongMsgCsvMixedFormatConverter in project incubator-inlong by apache.

the class MultiTenancyInLongMsgMixedDeserializer method generateDeserializer.

@VisibleForTesting
Pair<AbstractInLongMsgMixedFormatDeserializer, InLongMsgMixedFormatConverter> generateDeserializer(FieldInfo[] fields, InLongMsgDeserializationInfo inLongMsgDeserializationInfo) {
    final RowFormatInfo rowFormatInfo = CommonUtils.generateDeserializationRowFormatInfo(fields);
    final AbstractInLongMsgMixedFormatDeserializer preDeserializer;
    final InLongMsgMixedFormatConverter deserializer;
    if (inLongMsgDeserializationInfo instanceof InLongMsgCsvDeserializationInfo) {
        final InLongMsgCsvDeserializationInfo csvDeserializationInfo = (InLongMsgCsvDeserializationInfo) inLongMsgDeserializationInfo;
        preDeserializer = new InLongMsgCsvMixedFormatDeserializer(StandardCharsets.UTF_8.name(), csvDeserializationInfo.getDelimiter(), null, null, csvDeserializationInfo.isDeleteHeadDelimiter(), false);
        deserializer = new InLongMsgCsvMixedFormatConverter(rowFormatInfo, DEFAULT_TIME_FIELD_NAME, DEFAULT_ATTRIBUTES_FIELD_NAME, null, false);
    } else {
        throw new UnsupportedOperationException("Not supported yet " + inLongMsgDeserializationInfo.getClass().getSimpleName());
    }
    return Pair.of(preDeserializer, deserializer);
}
Also used : InLongMsgMixedFormatConverter(org.apache.inlong.sort.formats.inlongmsg.InLongMsgMixedFormatConverter) InLongMsgCsvMixedFormatDeserializer(org.apache.inlong.sort.formats.inlongmsgcsv.InLongMsgCsvMixedFormatDeserializer) RowFormatInfo(org.apache.inlong.sort.formats.common.RowFormatInfo) InLongMsgCsvMixedFormatConverter(org.apache.inlong.sort.formats.inlongmsgcsv.InLongMsgCsvMixedFormatConverter) InLongMsgCsvDeserializationInfo(org.apache.inlong.sort.protocol.deserialization.InLongMsgCsvDeserializationInfo) AbstractInLongMsgMixedFormatDeserializer(org.apache.inlong.sort.formats.inlongmsg.AbstractInLongMsgMixedFormatDeserializer) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Aggregations

VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 RowFormatInfo (org.apache.inlong.sort.formats.common.RowFormatInfo)1 AbstractInLongMsgMixedFormatDeserializer (org.apache.inlong.sort.formats.inlongmsg.AbstractInLongMsgMixedFormatDeserializer)1 InLongMsgMixedFormatConverter (org.apache.inlong.sort.formats.inlongmsg.InLongMsgMixedFormatConverter)1 InLongMsgCsvMixedFormatConverter (org.apache.inlong.sort.formats.inlongmsgcsv.InLongMsgCsvMixedFormatConverter)1 InLongMsgCsvMixedFormatDeserializer (org.apache.inlong.sort.formats.inlongmsgcsv.InLongMsgCsvMixedFormatDeserializer)1 InLongMsgCsvDeserializationInfo (org.apache.inlong.sort.protocol.deserialization.InLongMsgCsvDeserializationInfo)1