Examples with DataOutputSerializer org.apache.flink.core.memory.DataOutputSerializer used on opensource projects

Example 31 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class KvStateSerializer method serializeValue.

/**
 * Serializes the value with the given serializer.
 *
 * @param value Value of type T to serialize
 * @param serializer Serializer for T
 * @param <T> Type of the value
 * @return Serialized value or <code>null</code> if value <code>null</code>
 * @throws IOException On failure during serialization
 */
public static <T> byte[] serializeValue(T value, TypeSerializer<T> serializer) throws IOException {
    if (value != null) {
        // Serialize
        DataOutputSerializer dos = new DataOutputSerializer(32);
        serializer.serialize(value, dos);
        return dos.getCopyOfBuffer();
    } else {
        return null;
    }
}

Example 32 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class JobConfWrapper method writeObject.

private void writeObject(ObjectOutputStream out) throws IOException {
    out.defaultWriteObject();
    // we write the jobConf through a separate serializer to avoid cryptic exceptions when it
    // corrupts the serialization stream
    final DataOutputSerializer ser = new DataOutputSerializer(256);
    jobConf.write(ser);
    out.writeInt(ser.length());
    out.write(ser.getSharedBuffer(), 0, ser.length());
}

Example 33 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class NullableSerializer method checkIfNullSupported.

/**
 * This method checks if {@code serializer} supports {@code null} value.
 *
 * @param serializer serializer to check
 */
public static <T> boolean checkIfNullSupported(@Nonnull TypeSerializer<T> serializer) {
    int length = serializer.getLength() > 0 ? serializer.getLength() : 1;
    DataOutputSerializer dos = new DataOutputSerializer(length);
    try {
        serializer.serialize(null, dos);
    } catch (IOException | RuntimeException e) {
        return false;
    }
    checkArgument(serializer.getLength() < 0 || serializer.getLength() == dos.getCopyOfBuffer().length, "The serialized form of the null value should have the same length " + "as any other if the length is fixed in the serializer");
    DataInputDeserializer dis = new DataInputDeserializer(dos.getSharedBuffer());
    try {
        checkArgument(serializer.deserialize(dis) == null);
    } catch (IOException e) {
        throw new RuntimeException(String.format("Unexpected failure to deserialize just serialized null value with %s", serializer.getClass().getName()), e);
    }
    checkArgument(serializer.copy(null) == null, "Serializer %s has to be able properly copy null value if it can serialize it", serializer.getClass().getName());
    return true;
}

Example 34 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class PassThroughPythonStreamGroupWindowAggregateOperator method emitTimerData.

private void emitTimerData(RowData key, TimeWindow window, byte timerOperand) throws IOException {
    reusePythonTimerData.setByte(0, timerOperand);
    reusePythonTimerData.setField(1, key);
    reusePythonTimerData.setLong(2, window.maxTimestamp());
    windowSerializer.serialize(window, windowBaosWrapper);
    reusePythonTimerData.setField(3, windowBaos.toByteArray());
    windowBaos.reset();
    DataOutputSerializer output = new DataOutputSerializer(1);
    udfOutputTypeSerializer.serialize(reusePythonTimerRowData, output);
    resultBuffer.add(output.getCopyOfBuffer());
}

Example 35 with DataOutputSerializer

use of org.apache.flink.core.memory.DataOutputSerializer in project flink by apache.

the class TypeInformationKeyValueSerializationSchema method serializeValue.

@Override
public byte[] serializeValue(Tuple2<K, V> element) {
    // if the value is null, its serialized value is null as well.
    if (element.f1 == null) {
        return null;
    }
    if (valueOutputSerializer == null) {
        valueOutputSerializer = new DataOutputSerializer(16);
    }
    try {
        valueSerializer.serialize(element.f1, valueOutputSerializer);
    } catch (IOException e) {
        throw new RuntimeException("Unable to serialize record", e);
    }
    byte[] res = valueOutputSerializer.getByteArray();
    if (res.length != valueOutputSerializer.length()) {
        byte[] n = new byte[valueOutputSerializer.length()];
        System.arraycopy(res, 0, n, 0, valueOutputSerializer.length());
        res = n;
    }
    valueOutputSerializer.clear();
    return res;
}