Search in sources :

Example 21 with CompressionFileState

use of org.smartdata.model.CompressionFileState in project SSM by Intel-bigdata.

the class MetaStore method insertUpdateFileState.

public void insertUpdateFileState(FileState fileState) throws MetaStoreException {
    try {
        // Update corresponding tables according to the file state
        fileStateDao.insertUpdate(fileState);
        switch(fileState.getFileType()) {
            case COMPACT:
                CompactFileState compactFileState = (CompactFileState) fileState;
                smallFileDao.insertUpdate(compactFileState);
                break;
            case COMPRESSION:
                CompressionFileState compressionFileState = (CompressionFileState) fileState;
                compressionFileDao.insertUpdate(compressionFileState);
                break;
            case S3:
                break;
            default:
        }
    } catch (Exception e) {
        throw new MetaStoreException(e);
    }
}
Also used : CompressionFileState(org.smartdata.model.CompressionFileState) EmptyResultDataAccessException(org.springframework.dao.EmptyResultDataAccessException) SQLException(java.sql.SQLException) CompactFileState(org.smartdata.model.CompactFileState)

Example 22 with CompressionFileState

use of org.smartdata.model.CompressionFileState in project SSM by Intel-bigdata.

the class MetaStore method getFileState.

/**
 * Get FileState of the given path.
 *
 * @param path
 * @return
 * @throws MetaStoreException
 */
public FileState getFileState(String path) throws MetaStoreException {
    FileState fileState;
    try {
        fileState = fileStateDao.getByPath(path);
        // Fetch info from corresponding table to regenerate a specific file state
        switch(fileState.getFileType()) {
            case NORMAL:
                fileState = new NormalFileState(path);
                break;
            case COMPACT:
                fileState = smallFileDao.getFileStateByPath(path);
                break;
            case COMPRESSION:
                CompressionFileState compressionFileState = getCompressionInfo(path);
                if (compressionFileState != null) {
                    compressionFileState.setFileStage(fileState.getFileStage());
                    fileState = compressionFileState;
                }
                break;
            case S3:
                fileState = new S3FileState(path);
                break;
            default:
        }
    } catch (EmptyResultDataAccessException e1) {
        fileState = new NormalFileState(path);
    } catch (Exception e2) {
        throw new MetaStoreException(e2);
    }
    return fileState;
}
Also used : S3FileState(org.smartdata.model.S3FileState) NormalFileState(org.smartdata.model.NormalFileState) CompactFileState(org.smartdata.model.CompactFileState) FileState(org.smartdata.model.FileState) CompressionFileState(org.smartdata.model.CompressionFileState) NormalFileState(org.smartdata.model.NormalFileState) CompressionFileState(org.smartdata.model.CompressionFileState) EmptyResultDataAccessException(org.springframework.dao.EmptyResultDataAccessException) S3FileState(org.smartdata.model.S3FileState) EmptyResultDataAccessException(org.springframework.dao.EmptyResultDataAccessException) SQLException(java.sql.SQLException)

Aggregations

CompressionFileState (org.smartdata.model.CompressionFileState)22 FileState (org.smartdata.model.FileState)13 Test (org.junit.Test)7 CompactFileState (org.smartdata.model.CompactFileState)7 IOException (java.io.IOException)6 NormalFileState (org.smartdata.model.NormalFileState)5 HdfsFileStatus (org.apache.hadoop.hdfs.protocol.HdfsFileStatus)4 Gson (com.google.gson.Gson)3 CompressionFileInfo (org.smartdata.model.CompressionFileInfo)3 OutputStream (java.io.OutputStream)2 SQLException (java.sql.SQLException)2 ArrayList (java.util.ArrayList)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 LocatedFileStatus (org.apache.hadoop.fs.LocatedFileStatus)2 DFSInputStream (org.apache.hadoop.hdfs.DFSInputStream)2 HdfsLocatedFileStatus (org.apache.hadoop.hdfs.protocol.HdfsLocatedFileStatus)2 HdfsNamedFileStatus (org.apache.hadoop.hdfs.protocol.HdfsNamedFileStatus)2 ActionException (org.smartdata.action.ActionException)2 CompactFileStateProto (org.smartdata.protocol.ClientServerProto.CompactFileStateProto)2 CompressionFileStateProto (org.smartdata.protocol.ClientServerProto.CompressionFileStateProto)2