Examples with DataOutputSerializer org.apache.flink.core.memory.DataOutputSerializer used on opensource projects

Example 61 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class FileSourceSplitSerializer method serialize.

@Override
public byte[] serialize(FileSourceSplit split) throws IOException {
    checkArgument(split.getClass() == FileSourceSplit.class, "Cannot serialize subclasses of FileSourceSplit");
    // optimization: the splits lazily cache their own serialized form
    if (split.serializedFormCache != null) {
        return split.serializedFormCache;
    }
    final DataOutputSerializer out = SERIALIZER_CACHE.get();
    out.writeUTF(split.splitId());
    split.path().write(out);
    out.writeLong(split.offset());
    out.writeLong(split.length());
    out.writeLong(split.fileModificationTime());
    out.writeLong(split.fileSize());
    writeStringArray(out, split.hostnames());
    final Optional<CheckpointedPosition> readerPosition = split.getReaderPosition();
    out.writeBoolean(readerPosition.isPresent());
    if (readerPosition.isPresent()) {
        out.writeLong(readerPosition.get().getOffset());
        out.writeLong(readerPosition.get().getRecordsAfterOffset());
    }
    final byte[] result = out.getCopyOfBuffer();
    out.clear();
    // optimization: cache the serialized from, so we avoid the byte work during repeated
    // serialization
    split.serializedFormCache = result;
    return result;
}

Example 62 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class TestManagedSinkCommittableSerializer method serialize.

@Override
public byte[] serialize(TestManagedCommittable committable) throws IOException {
    final DataOutputSerializer out = new DataOutputSerializer(64);
    out.writeInt(committable.getToAdd().size());
    for (Map.Entry<CatalogPartitionSpec, List<RowData>> entry : committable.getToAdd().entrySet()) {
        serializePartitionSpec(out, entry.getKey());
        serializeRowDataElements(out, entry.getValue());
    }
    out.writeInt(committable.getToDelete().size());
    for (Map.Entry<CatalogPartitionSpec, Set<Path>> entry : committable.getToDelete().entrySet()) {
        serializePartitionSpec(out, entry.getKey());
        serializePaths(out, entry.getValue());
    }
    final byte[] result = out.getCopyOfBuffer();
    out.clear();
    return result;
}

Example 63 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class RawFormatSerDeSchemaTest method serializeLocalDateTime.

private static byte[] serializeLocalDateTime(LocalDateTime localDateTime) {
    DataOutputSerializer dos = new DataOutputSerializer(16);
    LocalDateTimeSerializer serializer = new LocalDateTimeSerializer();
    try {
        serializer.serialize(localDateTime, dos);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
    return dos.getCopyOfBuffer();
}