Search in sources :

Example 1 with MappedByteBuffer

use of java.nio.MappedByteBuffer in project hadoop by apache.

the class MappableBlock method load.

/**
   * Load the block.
   *
   * mmap and mlock the block, and then verify its checksum.
   *
   * @param length         The current length of the block.
   * @param blockIn        The block input stream.  Should be positioned at the
   *                       start.  The caller must close this.
   * @param metaIn         The meta file input stream.  Should be positioned at
   *                       the start.  The caller must close this.
   * @param blockFileName  The block file name, for logging purposes.
   *
   * @return               The Mappable block.
   */
public static MappableBlock load(long length, FileInputStream blockIn, FileInputStream metaIn, String blockFileName) throws IOException {
    MappableBlock mappableBlock = null;
    MappedByteBuffer mmap = null;
    FileChannel blockChannel = null;
    try {
        blockChannel = blockIn.getChannel();
        if (blockChannel == null) {
            throw new IOException("Block InputStream has no FileChannel.");
        }
        mmap = blockChannel.map(MapMode.READ_ONLY, 0, length);
        NativeIO.POSIX.getCacheManipulator().mlock(blockFileName, mmap, length);
        verifyChecksum(length, metaIn, blockChannel, blockFileName);
        mappableBlock = new MappableBlock(mmap, length);
    } finally {
        IOUtils.closeQuietly(blockChannel);
        if (mappableBlock == null) {
            if (mmap != null) {
                // unmapping also unlocks
                NativeIO.POSIX.munmap(mmap);
            }
        }
    }
    return mappableBlock;
}
Also used : MappedByteBuffer(java.nio.MappedByteBuffer) FileChannel(java.nio.channels.FileChannel) IOException(java.io.IOException)

Example 2 with MappedByteBuffer

use of java.nio.MappedByteBuffer in project hadoop by apache.

the class ShortCircuitReplica method munmap.

/**
   * Free the mmap associated with this replica.
   *
   * Must be called with the cache lock held.
   */
void munmap() {
    MappedByteBuffer mmap = (MappedByteBuffer) mmapData;
    NativeIO.POSIX.munmap(mmap);
    mmapData = null;
}
Also used : MappedByteBuffer(java.nio.MappedByteBuffer)

Example 3 with MappedByteBuffer

use of java.nio.MappedByteBuffer in project kafka by apache.

the class Utils method readFileAsString.

/**
     * Attempt to read a file as a string
     * @throws IOException
     */
public static String readFileAsString(String path, Charset charset) throws IOException {
    if (charset == null)
        charset = Charset.defaultCharset();
    try (FileInputStream stream = new FileInputStream(new File(path))) {
        FileChannel fc = stream.getChannel();
        MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
        return charset.decode(bb).toString();
    }
}
Also used : MappedByteBuffer(java.nio.MappedByteBuffer) FileChannel(java.nio.channels.FileChannel) File(java.io.File) FileInputStream(java.io.FileInputStream)

Example 4 with MappedByteBuffer

use of java.nio.MappedByteBuffer in project druid by druid-io.

the class SmooshedFileMapper method close.

@Override
public void close() {
    Throwable thrown = null;
    for (MappedByteBuffer mappedByteBuffer : buffersList) {
        try {
            ByteBufferUtils.unmap(mappedByteBuffer);
        } catch (Throwable t) {
            if (thrown == null) {
                thrown = t;
            } else {
                thrown.addSuppressed(t);
            }
        }
    }
    Throwables.propagateIfPossible(thrown);
}
Also used : MappedByteBuffer(java.nio.MappedByteBuffer)

Example 5 with MappedByteBuffer

use of java.nio.MappedByteBuffer in project druid by druid-io.

the class ByteBufferUtilsTest method testUnmapDoesntCrashJVM.

@Test
public void testUnmapDoesntCrashJVM() throws Exception {
    final File file = temporaryFolder.newFile("some_mmap_file");
    try (final OutputStream os = new BufferedOutputStream(new FileOutputStream(file))) {
        final byte[] data = new byte[4096];
        Arrays.fill(data, (byte) 0x5A);
        os.write(data);
    }
    final MappedByteBuffer mappedByteBuffer = Files.map(file);
    Assert.assertEquals((byte) 0x5A, mappedByteBuffer.get(0));
    ByteBufferUtils.unmap(mappedByteBuffer);
    ByteBufferUtils.unmap(mappedByteBuffer);
}
Also used : MappedByteBuffer(java.nio.MappedByteBuffer) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) BufferedOutputStream(java.io.BufferedOutputStream) FileOutputStream(java.io.FileOutputStream) File(java.io.File) BufferedOutputStream(java.io.BufferedOutputStream) Test(org.junit.Test)

Aggregations

MappedByteBuffer (java.nio.MappedByteBuffer)147 FileChannel (java.nio.channels.FileChannel)71 IOException (java.io.IOException)40 File (java.io.File)34 RandomAccessFile (java.io.RandomAccessFile)33 FileInputStream (java.io.FileInputStream)27 ByteBuffer (java.nio.ByteBuffer)24 Test (org.junit.Test)18 Path (java.nio.file.Path)11 ProjectWorkspace (com.facebook.buck.testutil.integration.ProjectWorkspace)9 Elf (com.facebook.buck.cxx.elf.Elf)8 FileOutputStream (java.io.FileOutputStream)7 ElfSection (com.facebook.buck.cxx.elf.ElfSection)6 ProjectFilesystem (com.facebook.buck.io.ProjectFilesystem)4 FileNotFoundException (java.io.FileNotFoundException)4 UnsafeBuffer (org.agrona.concurrent.UnsafeBuffer)4 Pair (com.facebook.buck.model.Pair)3 Date (java.util.Date)3 NulTerminatedCharsetDecoder (com.facebook.buck.charset.NulTerminatedCharsetDecoder)2 ElfDynamicSection (com.facebook.buck.cxx.elf.ElfDynamicSection)2