Search in sources :

Example 36 with WritableByteChannel

use of java.nio.channels.WritableByteChannel in project storm by apache.

the class Nimbus method uploadChunk.

@SuppressWarnings("deprecation")
@Override
public void uploadChunk(String location, ByteBuffer chunk) throws AuthorizationException, TException {
    try {
        uploadChunkCalls.mark();
        checkAuthorization(null, null, "fileUpload");
        WritableByteChannel channel = uploaders.get(location);
        if (channel == null) {
            throw new RuntimeException("File for that location does not exist (or timed out)");
        }
        channel.write(chunk);
        uploaders.put(location, channel);
    } catch (Exception e) {
        LOG.warn("uploadChunk exception.", e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : TException(org.apache.thrift.TException) WritableByteChannel(java.nio.channels.WritableByteChannel) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException)

Example 37 with WritableByteChannel

use of java.nio.channels.WritableByteChannel in project storm by apache.

the class Nimbus method finishFileUpload.

@SuppressWarnings("deprecation")
@Override
public void finishFileUpload(String location) throws AuthorizationException, TException {
    try {
        finishFileUploadCalls.mark();
        checkAuthorization(null, null, "fileUpload");
        WritableByteChannel channel = uploaders.get(location);
        if (channel == null) {
            throw new RuntimeException("File for that location does not exist (or timed out)");
        }
        channel.close();
        LOG.info("Finished uploading file from client: {}", location);
        uploaders.remove(location);
    } catch (Exception e) {
        LOG.warn("finish file upload exception.", e);
        if (e instanceof TException) {
            throw (TException) e;
        }
        throw new RuntimeException(e);
    }
}
Also used : TException(org.apache.thrift.TException) WritableByteChannel(java.nio.channels.WritableByteChannel) AuthorizationException(org.apache.storm.generated.AuthorizationException) NotAliveException(org.apache.storm.generated.NotAliveException) InterruptedIOException(java.io.InterruptedIOException) TException(org.apache.thrift.TException) IOException(java.io.IOException) AlreadyAliveException(org.apache.storm.generated.AlreadyAliveException) KeyAlreadyExistsException(org.apache.storm.generated.KeyAlreadyExistsException) KeyNotFoundException(org.apache.storm.generated.KeyNotFoundException) InvalidTopologyException(org.apache.storm.generated.InvalidTopologyException) BindException(java.net.BindException)

Example 38 with WritableByteChannel

use of java.nio.channels.WritableByteChannel in project LuaViewSDK by alibaba.

the class ChannelTools method fastCopy.

/**
     * fast copy
     *
     * @param inputStream
     * @param outputStream
     * @throws IOException
     */
public static void fastCopy(InputStream inputStream, OutputStream outputStream) throws IOException {
    final ReadableByteChannel input = Channels.newChannel(inputStream);
    final WritableByteChannel output = Channels.newChannel(outputStream);
    fastCopy(input, output);
}
Also used : ReadableByteChannel(java.nio.channels.ReadableByteChannel) WritableByteChannel(java.nio.channels.WritableByteChannel)

Example 39 with WritableByteChannel

use of java.nio.channels.WritableByteChannel in project neo4j by neo4j.

the class StreamToDisk method write.

@Override
public void write(String destination, int requiredAlignment, byte[] data) throws IOException {
    File fileName = new File(storeDir, destination);
    fs.mkdirs(fileName.getParentFile());
    fileCopyMonitor.copyFile(fileName);
    if (StoreType.shouldBeManagedByPageCache(destination)) {
        WritableByteChannel channel = channels.get(destination);
        if (channel == null) {
            int filePageSize = pageCache.pageSize() - pageCache.pageSize() % requiredAlignment;
            PagedFile pagedFile = pageCache.map(fileName, filePageSize, StandardOpenOption.CREATE);
            channel = pagedFile.openWritableByteChannel();
            pagedFiles.put(destination, pagedFile);
            channels.put(destination, channel);
        }
        ByteBuffer buffer = ByteBuffer.wrap(data);
        while (buffer.hasRemaining()) {
            channel.write(buffer);
        }
    } else {
        try (OutputStream outputStream = fs.openAsOutputStream(fileName, true)) {
            outputStream.write(data);
        }
    }
}
Also used : PagedFile(org.neo4j.io.pagecache.PagedFile) OutputStream(java.io.OutputStream) WritableByteChannel(java.nio.channels.WritableByteChannel) PagedFile(org.neo4j.io.pagecache.PagedFile) File(java.io.File) ByteBuffer(java.nio.ByteBuffer)

Example 40 with WritableByteChannel

use of java.nio.channels.WritableByteChannel in project neo4j by neo4j.

the class PageCacheTest method writingToClosedWritableByteChannelMustThrow.

@Test(timeout = SHORT_TIMEOUT_MILLIS)
public void writingToClosedWritableByteChannelMustThrow() throws Exception {
    File file = file("a");
    configureStandardPageCache();
    try (PagedFile pf = pageCache.map(file, filePageSize)) {
        WritableByteChannel channel = pf.openWritableByteChannel();
        channel.close();
        expectedException.expect(ClosedChannelException.class);
        channel.write(ByteBuffer.allocate(recordSize));
        fail("That read should have thrown");
    }
}
Also used : AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) WritableByteChannel(java.nio.channels.WritableByteChannel) AdversarialPagedFile(org.neo4j.adversaries.pagecache.AdversarialPagedFile) File(java.io.File) Test(org.junit.Test)

Aggregations

WritableByteChannel (java.nio.channels.WritableByteChannel)118 ByteBuffer (java.nio.ByteBuffer)35 ReadableByteChannel (java.nio.channels.ReadableByteChannel)30 ByteArrayOutputStream (java.io.ByteArrayOutputStream)28 FileOutputStream (java.io.FileOutputStream)24 Test (org.testng.annotations.Test)19 ByteArrayInputStream (java.io.ByteArrayInputStream)16 IOException (java.io.IOException)16 Test (org.junit.Test)15 File (java.io.File)13 OutputStream (java.io.OutputStream)12 DbusEventGenerator (com.linkedin.databus.core.test.DbusEventGenerator)10 Vector (java.util.Vector)10 FileInputStream (java.io.FileInputStream)7 Writer (java.io.Writer)6 FileChannel (java.nio.channels.FileChannel)6 DbusEventIterator (com.linkedin.databus.core.DbusEventBuffer.DbusEventIterator)4 PhysicalPartition (com.linkedin.databus.core.data_model.PhysicalPartition)4 DbusEventsStatisticsCollector (com.linkedin.databus.core.monitoring.mbean.DbusEventsStatisticsCollector)4 ClosedChannelException (java.nio.channels.ClosedChannelException)4