Examples with BlockOutStream alluxio.client.block.stream.BlockOutStream used on opensource projects

Example 6 with BlockOutStream

use of alluxio.client.block.stream.BlockOutStream in project alluxio by Alluxio.

the class AlluxioBlockStore method getOutStream.

/**
 * Gets a stream to write data to a block. The stream can only be backed by Alluxio storage.
 *
 * @param blockId the block to write
 * @param blockSize the standard block size to write
 * @param address the address of the worker to write the block to, fails if the worker cannot
 *        serve the request
 * @param options the output stream options
 * @return an {@link BlockOutStream} which can be used to write data to the block in a streaming
 *         fashion
 */
public BlockOutStream getOutStream(long blockId, long blockSize, WorkerNetAddress address, OutStreamOptions options) throws IOException {
    // No specified location to write to.
    Preconditions.checkNotNull(address, "address");
    LOG.debug("Create BlockOutStream for {} of block size {} at address {}, using options: {}", blockId, blockSize, address, options);
    DataWriter dataWriter = DataWriter.Factory.create(mContext, blockId, blockSize, address, options);
    return new BlockOutStream(dataWriter, blockSize, address);
}

Example 7 with BlockOutStream

use of alluxio.client.block.stream.BlockOutStream in project alluxio by Alluxio.

the class FileOutStreamTest method cacheWriteExceptionSyncPersist.

/**
 * Tests that if an exception is thrown by the underlying out stream, and the user is using
 * {@link UnderStorageType#SYNC_PERSIST} for their under storage type, the error is recovered
 * from by writing the data to the under storage out stream.
 */
@Test
public void cacheWriteExceptionSyncPersist() throws IOException {
    BlockOutStream stream = mock(BlockOutStream.class);
    when(mBlockStore.getOutStream(anyLong(), anyLong(), any(OutStreamOptions.class))).thenReturn(stream);
    when(stream.remaining()).thenReturn(BLOCK_LENGTH);
    doThrow(new IOException("test error")).when(stream).write((byte) 7);
    mTestStream.write(7);
    mTestStream.write(8);
    assertArrayEquals(new byte[] { 7, 8 }, mUnderStorageOutputStream.getWrittenData());
    // The cache stream is written to only once - the FileInStream gives up on it after it throws
    // the first exception.
    verify(stream, times(1)).write(anyInt());
}

Example 8 with BlockOutStream

use of alluxio.client.block.stream.BlockOutStream in project alluxio by Alluxio.

the class AlluxioBlockStoreTest method getOutStreamRemote.

@Test
public void getOutStreamRemote() throws Exception {
    WorkerNetAddress worker1 = new WorkerNetAddress().setHost("worker1");
    WorkerNetAddress worker2 = new WorkerNetAddress().setHost("worker2");
    OutStreamOptions options = OutStreamOptions.defaults(mClientContext).setBlockSizeBytes(BLOCK_LENGTH).setLocationPolicy(new MockBlockLocationPolicy(Arrays.asList(worker1, worker2))).setWriteType(WriteType.MUST_CACHE);
    BlockOutStream stream1 = mBlockStore.getOutStream(BLOCK_ID, BLOCK_LENGTH, options);
    assertEquals(worker1, stream1.getAddress());
    BlockOutStream stream2 = mBlockStore.getOutStream(BLOCK_ID, BLOCK_LENGTH, options);
    assertEquals(worker2, stream2.getAddress());
}