Examples with InputStreamFSInputWrapper org.apache.flink.api.common.io.InputStreamFSInputWrapper used on opensource projects

Example 1 with InputStreamFSInputWrapper

use of org.apache.flink.api.common.io.InputStreamFSInputWrapper in project flink by apache.

the class StreamFormatAdapter method openStream.

private static TrackingFsDataInputStream openStream(final Path file, final Configuration config, final long seekPosition) throws IOException {
    final FileSystem fs = file.getFileSystem();
    final long fileLength = fs.getFileStatus(file).getLen();
    final int fetchSize = MathUtils.checkedDownCast(config.get(StreamFormat.FETCH_IO_SIZE).getBytes());
    if (fetchSize <= 0) {
        throw new IllegalConfigurationException(String.format("The fetch size (%s) must be > 0, but is %d", StreamFormat.FETCH_IO_SIZE.key(), fetchSize));
    }
    final InflaterInputStreamFactory<?> deCompressor = StandardDeCompressors.getDecompressorForFileName(file.getPath());
    final FSDataInputStream inStream = fs.open(file);
    return doWithCleanupOnException(inStream, () -> {
        final FSDataInputStream in = deCompressor == null ? inStream : new InputStreamFSInputWrapper(deCompressor.create(inStream));
        in.seek(seekPosition);
        return new TrackingFsDataInputStream(in, fileLength, fetchSize);
    });
}