Search in sources :

Example 1 with SpillCipher

use of io.prestosql.spi.spiller.SpillCipher in project hetu-core by openlookeng.

the class TestSpillCipherPagesSerde method test.

@Test
public void test() {
    SpillCipher cipher = new AesSpillCipher();
    PagesSerde serde = TESTING_SERDE_FACTORY.createPagesSerdeForSpill(Optional.of(cipher), false, false);
    List<Type> types = ImmutableList.of(VARCHAR);
    Page emptyPage = new Page(VARCHAR.createBlockBuilder(null, 0).build());
    assertPageEquals(types, serde.deserialize(serde.serialize(emptyPage)), emptyPage);
    BlockBuilder blockBuilder = VARCHAR.createBlockBuilder(null, 2);
    VARCHAR.writeString(blockBuilder, "hello");
    VARCHAR.writeString(blockBuilder, "world");
    Page helloWorldPage = new Page(blockBuilder.build());
    SerializedPage serialized = serde.serialize(helloWorldPage);
    assertPageEquals(types, serde.deserialize(serialized), helloWorldPage);
    assertTrue(serialized.isEncrypted(), "page should be encrypted");
    cipher.close();
    assertFailure(() -> serde.serialize(helloWorldPage), "Spill cipher already closed");
    assertFailure(() -> serde.deserialize(serialized), "Spill cipher already closed");
}
Also used : Type(io.prestosql.spi.type.Type) PagesSerde(io.hetu.core.transport.execution.buffer.PagesSerde) Page(io.prestosql.spi.Page) SerializedPage(io.hetu.core.transport.execution.buffer.SerializedPage) SerializedPage(io.hetu.core.transport.execution.buffer.SerializedPage) SpillCipher(io.prestosql.spi.spiller.SpillCipher) BlockBuilder(io.prestosql.spi.block.BlockBuilder) Test(org.testng.annotations.Test)

Example 2 with SpillCipher

use of io.prestosql.spi.spiller.SpillCipher in project hetu-core by openlookeng.

the class FileSingleStreamSpillerFactory method create.

@Override
public synchronized SingleStreamSpiller create(List<Type> types, SpillContext spillContext, LocalMemoryContext memoryContext) {
    createSpillDirectories();
    Optional<SpillCipher> spillCipher = Optional.empty();
    if (spillEncryptionEnabled) {
        spillCipher = Optional.of(new AesSpillCipher());
    }
    PagesSerde serde = serdeFactory.createPagesSerdeForSpill(spillCipher, spillDirectSerdeEnabled, useKryo);
    return new FileSingleStreamSpiller(serde, executor, getNextSpillPath(), spillerStats, spillContext, memoryContext, spillCipher, spillCompressionEnabled, spillDirectSerdeEnabled, spillPrefetchReadPages, useKryo, spillToHdfs, spillProfile, fileSystemClientManager);
}
Also used : PagesSerde(io.hetu.core.transport.execution.buffer.PagesSerde) SpillCipher(io.prestosql.spi.spiller.SpillCipher)

Aggregations

PagesSerde (io.hetu.core.transport.execution.buffer.PagesSerde)2 SpillCipher (io.prestosql.spi.spiller.SpillCipher)2 SerializedPage (io.hetu.core.transport.execution.buffer.SerializedPage)1 Page (io.prestosql.spi.Page)1 BlockBuilder (io.prestosql.spi.block.BlockBuilder)1 Type (io.prestosql.spi.type.Type)1 Test (org.testng.annotations.Test)1