Search in sources :

Example 1 with ProblemReportingIterator

use of org.apache.accumulo.server.problems.ProblemReportingIterator in project accumulo by apache.

the class Compactor method openMapDataFiles.

private List<SortedKeyValueIterator<Key, Value>> openMapDataFiles(String lgName, ArrayList<FileSKVIterator> readers) throws IOException {
    List<SortedKeyValueIterator<Key, Value>> iters = new ArrayList<>(filesToCompact.size());
    for (FileRef mapFile : filesToCompact.keySet()) {
        try {
            FileOperations fileFactory = FileOperations.getInstance();
            FileSystem fs = this.fs.getVolumeByPath(mapFile.path()).getFileSystem();
            FileSKVIterator reader;
            reader = fileFactory.newReaderBuilder().forFile(mapFile.path().toString(), fs, fs.getConf()).withTableConfiguration(acuTableConf).withRateLimiter(env.getReadLimiter()).build();
            readers.add(reader);
            SortedKeyValueIterator<Key, Value> iter = new ProblemReportingIterator(context, extent.getTableId(), mapFile.path().toString(), false, reader);
            if (filesToCompact.get(mapFile).isTimeSet()) {
                iter = new TimeSettingIterator(iter, filesToCompact.get(mapFile).getTime());
            }
            iters.add(iter);
        } catch (Throwable e) {
            ProblemReports.getInstance(context).report(new ProblemReport(extent.getTableId(), ProblemType.FILE_READ, mapFile.path().toString(), e));
            log.warn("Some problem opening map file {} {}", mapFile, e.getMessage(), e);
            // failed to open some map file... close the ones that were opened
            for (FileSKVIterator reader : readers) {
                try {
                    reader.close();
                } catch (Throwable e2) {
                    log.warn("Failed to close map file", e2);
                }
            }
            readers.clear();
            if (e instanceof IOException)
                throw (IOException) e;
            throw new IOException("Failed to open map data files", e);
        }
    }
    return iters;
}
Also used : FileSKVIterator(org.apache.accumulo.core.file.FileSKVIterator) ArrayList(java.util.ArrayList) FileOperations(org.apache.accumulo.core.file.FileOperations) SortedKeyValueIterator(org.apache.accumulo.core.iterators.SortedKeyValueIterator) TimeSettingIterator(org.apache.accumulo.core.iterators.system.TimeSettingIterator) IOException(java.io.IOException) ProblemReport(org.apache.accumulo.server.problems.ProblemReport) FileRef(org.apache.accumulo.server.fs.FileRef) FileSystem(org.apache.hadoop.fs.FileSystem) ProblemReportingIterator(org.apache.accumulo.server.problems.ProblemReportingIterator) Value(org.apache.accumulo.core.data.Value) DataFileValue(org.apache.accumulo.core.metadata.schema.DataFileValue) Key(org.apache.accumulo.core.data.Key)

Aggregations

IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Key (org.apache.accumulo.core.data.Key)1 Value (org.apache.accumulo.core.data.Value)1 FileOperations (org.apache.accumulo.core.file.FileOperations)1 FileSKVIterator (org.apache.accumulo.core.file.FileSKVIterator)1 SortedKeyValueIterator (org.apache.accumulo.core.iterators.SortedKeyValueIterator)1 TimeSettingIterator (org.apache.accumulo.core.iterators.system.TimeSettingIterator)1 DataFileValue (org.apache.accumulo.core.metadata.schema.DataFileValue)1 FileRef (org.apache.accumulo.server.fs.FileRef)1 ProblemReport (org.apache.accumulo.server.problems.ProblemReport)1 ProblemReportingIterator (org.apache.accumulo.server.problems.ProblemReportingIterator)1 FileSystem (org.apache.hadoop.fs.FileSystem)1