Search in sources :

Example 1 with AvroFSInput

use of org.apache.hadoop.fs.AvroFSInput in project samza by apache.

the class AvroFileHdfsReader method open.

@Override
public void open(String pathStr, String singleFileOffset) {
    LOG.info(String.format("%s: Open file [%s] with file offset [%s] for read", systemStreamPartition, pathStr, singleFileOffset));
    Path path = new Path(pathStr);
    try {
        AvroFSInput input = new AvroFSInput(FileContext.getFileContext(path.toUri()), path);
        fileReader = new DataFileReader<>(input, new GenericDatumReader<>());
        seek(singleFileOffset);
    } catch (IOException e) {
        throw new SamzaException(e);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) GenericDatumReader(org.apache.avro.generic.GenericDatumReader) IOException(java.io.IOException) AvroFSInput(org.apache.hadoop.fs.AvroFSInput) SamzaException(org.apache.samza.SamzaException)

Aggregations

IOException (java.io.IOException)1 GenericDatumReader (org.apache.avro.generic.GenericDatumReader)1 AvroFSInput (org.apache.hadoop.fs.AvroFSInput)1 Path (org.apache.hadoop.fs.Path)1 SamzaException (org.apache.samza.SamzaException)1