Search in sources :

Example 1 with SimpleReadSupport

use of org.apache.parquet.tools.read.SimpleReadSupport in project parquet-mr by apache.

the class HeadCommand method execute.

@Override
public void execute(CommandLine options) throws Exception {
    super.execute(options);
    long num = DEFAULT;
    if (options.hasOption('n')) {
        num = Long.parseLong(options.getOptionValue('n'));
    }
    String[] args = options.getArgs();
    String input = args[0];
    ParquetReader<SimpleRecord> reader = null;
    try {
        PrintWriter writer = new PrintWriter(Main.out, true);
        reader = ParquetReader.builder(new SimpleReadSupport(), new Path(input)).build();
        for (SimpleRecord value = reader.read(); value != null && num-- > 0; value = reader.read()) {
            value.prettyPrint(writer);
            writer.println();
        }
    } finally {
        if (reader != null) {
            try {
                reader.close();
            } catch (Exception ex) {
            }
        }
    }
}
Also used : Path(org.apache.hadoop.fs.Path) SimpleReadSupport(org.apache.parquet.tools.read.SimpleReadSupport) SimpleRecord(org.apache.parquet.tools.read.SimpleRecord) PrintWriter(java.io.PrintWriter)

Example 2 with SimpleReadSupport

use of org.apache.parquet.tools.read.SimpleReadSupport in project parquet-mr by apache.

the class CatCommand method execute.

@Override
public void execute(CommandLine options) throws Exception {
    super.execute(options);
    String[] args = options.getArgs();
    String input = args[0];
    ParquetReader<SimpleRecord> reader = null;
    try {
        PrintWriter writer = new PrintWriter(Main.out, true);
        reader = ParquetReader.builder(new SimpleReadSupport(), new Path(input)).build();
        ParquetMetadata metadata = ParquetFileReader.readFooter(new Configuration(), new Path(input));
        JsonRecordFormatter.JsonGroupFormatter formatter = JsonRecordFormatter.fromSchema(metadata.getFileMetaData().getSchema());
        for (SimpleRecord value = reader.read(); value != null; value = reader.read()) {
            if (options.hasOption('j')) {
                writer.write(formatter.formatRecord(value));
            } else {
                value.prettyPrint(writer);
            }
            writer.println();
        }
    } finally {
        if (reader != null) {
            try {
                reader.close();
            } catch (Exception ex) {
            }
        }
    }
}
Also used : Path(org.apache.hadoop.fs.Path) Configuration(org.apache.hadoop.conf.Configuration) ParquetMetadata(org.apache.parquet.hadoop.metadata.ParquetMetadata) SimpleReadSupport(org.apache.parquet.tools.read.SimpleReadSupport) SimpleRecord(org.apache.parquet.tools.read.SimpleRecord) JsonRecordFormatter(org.apache.parquet.tools.json.JsonRecordFormatter) PrintWriter(java.io.PrintWriter)

Aggregations

PrintWriter (java.io.PrintWriter)2 Path (org.apache.hadoop.fs.Path)2 SimpleReadSupport (org.apache.parquet.tools.read.SimpleReadSupport)2 SimpleRecord (org.apache.parquet.tools.read.SimpleRecord)2 Configuration (org.apache.hadoop.conf.Configuration)1 ParquetMetadata (org.apache.parquet.hadoop.metadata.ParquetMetadata)1 JsonRecordFormatter (org.apache.parquet.tools.json.JsonRecordFormatter)1