Examples with FilesForCommit org.apache.iceberg.mr.hive.FilesForCommit used on opensource projects

Example 1 with FilesForCommit

use of org.apache.iceberg.mr.hive.FilesForCommit in project hive by apache.

the class HiveIcebergWriterBase method close.

@Override
public void close(boolean abort) throws IOException {
    writer.close();
    FilesForCommit result = files();
    // If abort then remove the unnecessary files
    if (abort) {
        Tasks.foreach(result.allFiles()).retry(3).suppressFailureWhenFinished().onFailure((file, exception) -> LOG.debug("Failed on to remove file {} on abort", file, exception)).run(file -> io.deleteFile(file.path().toString()));
    }
    LOG.info("HiveIcebergWriter is closed with abort={}. Created {} data files and {} delete files", abort, result.dataFiles().size(), result.deleteFiles().size());
}