Examples with CarbonDeleteFilesDataReader org.apache.carbondata.core.reader.CarbonDeleteFilesDataReader used on opensource projects

Example 1 with CarbonDeleteFilesDataReader

use of org.apache.carbondata.core.reader.CarbonDeleteFilesDataReader in project carbondata by apache.

the class CarbonDataMergerUtil method startCompactionDeleteDeltaFiles.

/**
   * this method compact the delete delta files.
   * @param deleteDeltaFiles
   * @param blockName
   * @param fullBlockFilePath
   * @return
   */
public static Boolean startCompactionDeleteDeltaFiles(List<String> deleteDeltaFiles, String blockName, String fullBlockFilePath) throws IOException {
    DeleteDeltaBlockDetails deleteDeltaBlockDetails = null;
    CarbonDeleteFilesDataReader dataReader = new CarbonDeleteFilesDataReader();
    try {
        deleteDeltaBlockDetails = dataReader.getCompactedDeleteDeltaFileFromBlock(deleteDeltaFiles, blockName);
    } catch (Exception e) {
        String blockFilePath = fullBlockFilePath.substring(0, fullBlockFilePath.lastIndexOf(CarbonCommonConstants.FILE_SEPARATOR));
        LOGGER.error("Error while getting the delete delta blocks in path " + blockFilePath);
        throw new IOException();
    }
    CarbonDeleteDeltaWriterImpl carbonDeleteWriter = new CarbonDeleteDeltaWriterImpl(fullBlockFilePath, FileFactory.getFileType(fullBlockFilePath));
    try {
        carbonDeleteWriter.write(deleteDeltaBlockDetails);
    } catch (IOException e) {
        LOGGER.error("Error while writing compacted delete delta file " + fullBlockFilePath);
        throw new IOException();
    }
    return true;
}

Example 2 with CarbonDeleteFilesDataReader

use of org.apache.carbondata.core.reader.CarbonDeleteFilesDataReader in project carbondata by apache.

the class SegmentUpdateStatusManager method getDeleteDeltaDataFromAllFiles.

/**
   * Returns all deleted records of specified block
   *
   * @param tupleId
   * @return
   * @throws Exception
   */
public int[] getDeleteDeltaDataFromAllFiles(String tupleId) throws Exception {
    List<String> deltaFiles = getDeltaFiles(tupleId, CarbonCommonConstants.DELETE_DELTA_FILE_EXT);
    CarbonDeleteFilesDataReader dataReader = new CarbonDeleteFilesDataReader();
    String blockletId = CarbonUpdateUtil.getRequiredFieldFromTID(tupleId, TupleIdEnum.BLOCKLET_ID);
    return dataReader.getDeleteDataFromAllFiles(deltaFiles, blockletId);
}