Examples with ByteArrayInputStreamWithPos org.apache.flink.core.memory.ByteArrayInputStreamWithPos used on opensource projects

Example 6 with ByteArrayInputStreamWithPos

use of org.apache.flink.core.memory.ByteArrayInputStreamWithPos in project flink by apache.

the class TypeSerializerSerializationProxyTest method testStateSerializerSerializationProxy.

@Test
public void testStateSerializerSerializationProxy() throws Exception {
    TypeSerializer<?> serializer = IntSerializer.INSTANCE;
    TypeSerializerSerializationProxy<?> proxy = new TypeSerializerSerializationProxy<>(serializer);
    byte[] serialized;
    try (ByteArrayOutputStreamWithPos out = new ByteArrayOutputStreamWithPos()) {
        proxy.write(new DataOutputViewStreamWrapper(out));
        serialized = out.toByteArray();
    }
    proxy = new TypeSerializerSerializationProxy<>(Thread.currentThread().getContextClassLoader());
    try (ByteArrayInputStreamWithPos in = new ByteArrayInputStreamWithPos(serialized)) {
        proxy.read(new DataInputViewStreamWrapper(in));
    }
    Assert.assertEquals(serializer, proxy.getTypeSerializer());
}

Example 7 with ByteArrayInputStreamWithPos

use of org.apache.flink.core.memory.ByteArrayInputStreamWithPos in project flink by apache.

the class VersionedIOWriteableTest method testReadSameVersion.

@Test
public void testReadSameVersion() throws Exception {
    String payload = "test";
    TestWriteable testWriteable = new TestWriteable(1, payload);
    byte[] serialized;
    try (ByteArrayOutputStreamWithPos out = new ByteArrayOutputStreamWithPos()) {
        testWriteable.write(new DataOutputViewStreamWrapper(out));
        serialized = out.toByteArray();
    }
    testWriteable = new TestWriteable(1);
    try (ByteArrayInputStreamWithPos in = new ByteArrayInputStreamWithPos(serialized)) {
        testWriteable.read(new DataInputViewStreamWrapper(in));
    }
    Assert.assertEquals(payload, testWriteable.getData());
}

Example 8 with ByteArrayInputStreamWithPos

use of org.apache.flink.core.memory.ByteArrayInputStreamWithPos in project flink by apache.

the class VersionedIOWriteableTest method testReadMismatchVersion.

@Test
public void testReadMismatchVersion() throws Exception {
    String payload = "test";
    TestWriteable testWriteable = new TestWriteable(1, payload);
    byte[] serialized;
    try (ByteArrayOutputStreamWithPos out = new ByteArrayOutputStreamWithPos()) {
        testWriteable.write(new DataOutputViewStreamWrapper(out));
        serialized = out.toByteArray();
    }
    testWriteable = new TestWriteable(2);
    try (ByteArrayInputStreamWithPos in = new ByteArrayInputStreamWithPos(serialized)) {
        testWriteable.read(new DataInputViewStreamWrapper(in));
        Assert.fail("Version mismatch expected.");
    } catch (VersionMismatchException ignored) {
    }
    Assert.assertEquals(null, testWriteable.getData());
}

Example 9 with ByteArrayInputStreamWithPos

use of org.apache.flink.core.memory.ByteArrayInputStreamWithPos in project flink by apache.

the class RocksDBAggregatingState method get.

@Override
public R get() throws IOException {
    try {
        // prepare the current key and namespace for RocksDB lookup
        writeCurrentKeyWithGroupAndNamespace();
        final byte[] key = keySerializationStream.toByteArray();
        // get the current value
        final byte[] valueBytes = backend.db.get(columnFamily, key);
        if (valueBytes == null) {
            return null;
        }
        ACC accumulator = valueSerializer.deserialize(new DataInputViewStreamWrapper(new ByteArrayInputStreamWithPos(valueBytes)));
        return aggFunction.getResult(accumulator);
    } catch (IOException | RocksDBException e) {
        throw new IOException("Error while retrieving value from RocksDB", e);
    }
}

Example 10 with ByteArrayInputStreamWithPos

use of org.apache.flink.core.memory.ByteArrayInputStreamWithPos in project flink by apache.

the class RocksDBAggregatingState method add.

@Override
public void add(T value) throws IOException {
    try {
        // prepare the current key and namespace for RocksDB lookup
        writeCurrentKeyWithGroupAndNamespace();
        final byte[] key = keySerializationStream.toByteArray();
        keySerializationStream.reset();
        // get the current value
        final byte[] valueBytes = backend.db.get(columnFamily, key);
        // deserialize the current accumulator, or create a blank one
        final ACC accumulator = valueBytes == null ? aggFunction.createAccumulator() : valueSerializer.deserialize(new DataInputViewStreamWrapper(new ByteArrayInputStreamWithPos(valueBytes)));
        // aggregate the value into the accumulator
        aggFunction.add(value, accumulator);
        // serialize the new accumulator
        final DataOutputViewStreamWrapper out = new DataOutputViewStreamWrapper(keySerializationStream);
        valueSerializer.serialize(accumulator, out);
        // write the new value to RocksDB
        backend.db.put(columnFamily, writeOptions, key, keySerializationStream.toByteArray());
    } catch (IOException | RocksDBException e) {
        throw new IOException("Error while adding value to RocksDB", e);
    }
}