Examples with DataInputDeserializer org.apache.flink.core.memory.DataInputDeserializer used on opensource projects

Example 6 with DataInputDeserializer

use of org.apache.flink.core.memory.DataInputDeserializer in project flink by apache.

the class CompositeTypeSerializerSnapshotTest method testRestoreCompositeTypeSerializer.

// ------------------------------------------------------------------------------------------------
// Scope: tests CompositeTypeSerializerSnapshot#restoreSerializer
// ------------------------------------------------------------------------------------------------
@Test
public void testRestoreCompositeTypeSerializer() throws IOException {
    // the target compatibilities of the nested serializers doesn't matter,
    // because we're only testing the restore serializer
    TypeSerializer<?>[] testNestedSerializers = { new NestedSerializer(TargetCompatibility.COMPATIBLE_AS_IS), new NestedSerializer(TargetCompatibility.INCOMPATIBLE), new NestedSerializer(TargetCompatibility.COMPATIBLE_AFTER_MIGRATION) };
    TestCompositeTypeSerializer testSerializer = new TestCompositeTypeSerializer(testNestedSerializers);
    TypeSerializerSnapshot<String> testSerializerSnapshot = testSerializer.snapshotConfiguration();
    DataOutputSerializer out = new DataOutputSerializer(128);
    TypeSerializerSnapshot.writeVersionedSnapshot(out, testSerializerSnapshot);
    DataInputDeserializer in = new DataInputDeserializer(out.getCopyOfBuffer());
    testSerializerSnapshot = TypeSerializerSnapshot.readVersionedSnapshot(in, Thread.currentThread().getContextClassLoader());
    // now, restore the composite type serializer;
    // the restored nested serializer should be a RestoredNestedSerializer
    testSerializer = (TestCompositeTypeSerializer) testSerializerSnapshot.restoreSerializer();
    Assert.assertTrue(testSerializer.getNestedSerializers()[0].getClass() == RestoredNestedSerializer.class);
    Assert.assertTrue(testSerializer.getNestedSerializers()[1].getClass() == RestoredNestedSerializer.class);
    Assert.assertTrue(testSerializer.getNestedSerializers()[2].getClass() == RestoredNestedSerializer.class);
}

Example 7 with DataInputDeserializer

use of org.apache.flink.core.memory.DataInputDeserializer in project flink by apache.

the class SerializableHadoopConfigWrapper method readObject.

private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
    in.defaultReadObject();
    final byte[] data = new byte[in.readInt()];
    in.readFully(data);
    final DataInputDeserializer deser = new DataInputDeserializer(data);
    this.hadoopConfig = new Configuration();
    try {
        this.hadoopConfig.readFields(deser);
    } catch (IOException e) {
        throw new IOException("Could not deserialize Hadoop Configuration, the serialized and de-serialized don't match.", e);
    }
}

Example 8 with DataInputDeserializer

use of org.apache.flink.core.memory.DataInputDeserializer in project flink by apache.

the class LinkedOptionalMapSerializer method tryReadFrame.

@Nullable
private static <T> T tryReadFrame(DataInputView in, String keyName, BiFunctionWithException<DataInputView, String, T, IOException> reader) throws IOException {
    final int bufferSize = in.readInt();
    final byte[] buffer = new byte[bufferSize];
    in.readFully(buffer);
    DataInputDeserializer frame = new DataInputDeserializer(buffer);
    return reader.apply(frame, keyName);
}

Example 9 with DataInputDeserializer

use of org.apache.flink.core.memory.DataInputDeserializer in project flink by apache.

the class PulsarTypeInformationWrapper method deserialize.

@Override
public void deserialize(Message<byte[]> message, Collector<T> out) throws Exception {
    DataInputDeserializer dis = DESERIALIZER.get();
    dis.setBuffer(message.getData());
    T instance = serializer.deserialize(dis);
    out.collect(instance);
}

Example 10 with DataInputDeserializer

use of org.apache.flink.core.memory.DataInputDeserializer in project flink by apache.

the class KvStateSerializer method deserializeMap.

/**
 * Deserializes all kv pairs with the given serializer.
 *
 * @param serializedValue Serialized value of type Map<UK, UV>
 * @param keySerializer Serializer for UK
 * @param valueSerializer Serializer for UV
 * @param <UK> Type of the key
 * @param <UV> Type of the value.
 * @return Deserialized map or <code>null</code> if the serialized value is <code>null</code>
 * @throws IOException On failure during deserialization
 */
public static <UK, UV> Map<UK, UV> deserializeMap(byte[] serializedValue, TypeSerializer<UK> keySerializer, TypeSerializer<UV> valueSerializer) throws IOException {
    if (serializedValue != null) {
        DataInputDeserializer in = new DataInputDeserializer(serializedValue, 0, serializedValue.length);
        Map<UK, UV> result = new HashMap<>();
        while (in.available() > 0) {
            UK key = keySerializer.deserialize(in);
            boolean isNull = in.readBoolean();
            UV value = isNull ? null : valueSerializer.deserialize(in);
            result.put(key, value);
        }
        return result;
    } else {
        return null;
    }
}