Search in sources :

Example 6 with FileFooter

use of org.apache.carbondata.format.FileFooter in project carbondata by apache.

the class CarbonFactDataWriterImplV1 method writeBlockletInfoToFile.

/**
   * This method will write metadata at the end of file file format in thrift format
   */
protected void writeBlockletInfoToFile(FileChannel channel, String filePath) throws CarbonDataWriterException {
    try {
        long currentPosition = channel.size();
        CarbonFooterWriter writer = new CarbonFooterWriter(filePath);
        FileFooter convertFileMeta = CarbonMetadataUtil.convertFileFooter(blockletInfoList, localCardinality.length, localCardinality, thriftColumnSchemaList, dataWriterVo.getSegmentProperties());
        fillBlockIndexInfoDetails(convertFileMeta.getNum_rows(), carbonDataFileName, currentPosition);
        writer.writeFooter(convertFileMeta, currentPosition);
    } catch (IOException e) {
        throw new CarbonDataWriterException("Problem while writing the carbon file: ", e);
    }
}
Also used : FileFooter(org.apache.carbondata.format.FileFooter) IOException(java.io.IOException) CarbonFooterWriter(org.apache.carbondata.core.writer.CarbonFooterWriter) CarbonDataWriterException(org.apache.carbondata.processing.store.writer.exception.CarbonDataWriterException)

Example 7 with FileFooter

use of org.apache.carbondata.format.FileFooter in project carbondata by apache.

the class CarbonFooterReader method readFooter.

/**
   * It reads the metadata in FileFooter thrift object format.
   *
   * @return
   * @throws IOException
   */
public FileFooter readFooter() throws IOException {
    ThriftReader thriftReader = openThriftReader(filePath);
    thriftReader.open();
    //Set the offset from where it should read
    thriftReader.setReadOffset(offset);
    FileFooter footer = (FileFooter) thriftReader.read();
    thriftReader.close();
    return footer;
}
Also used : FileFooter(org.apache.carbondata.format.FileFooter)

Aggregations

FileFooter (org.apache.carbondata.format.FileFooter)7 IOException (java.io.IOException)2 ArrayList (java.util.ArrayList)2 BlockletInfoColumnar (org.apache.carbondata.core.metadata.BlockletInfoColumnar)2 BlockletInfo (org.apache.carbondata.core.metadata.blocklet.BlockletInfo)2 DataFileFooter (org.apache.carbondata.core.metadata.blocklet.DataFileFooter)2 BlockletIndex (org.apache.carbondata.core.metadata.blocklet.index.BlockletIndex)2 ColumnSchema (org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema)2 CarbonFooterReader (org.apache.carbondata.core.reader.CarbonFooterReader)2 CarbonFooterWriter (org.apache.carbondata.core.writer.CarbonFooterWriter)2 CarbonDataWriterException (org.apache.carbondata.processing.store.writer.exception.CarbonDataWriterException)2 FileHolder (org.apache.carbondata.core.datastore.FileHolder)1 ColumnarFormatVersion (org.apache.carbondata.core.metadata.ColumnarFormatVersion)1 SegmentInfo (org.apache.carbondata.format.SegmentInfo)1