Search in sources :

Example 46 with ErasureCodingPolicy

use of org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy in project SSM by Intel-bigdata.

the class AddErasureCodingPolicy method execute.

@Override
public void execute() throws Exception {
    this.setDfsClient(HadoopUtil.getDFSClient(HadoopUtil.getNameNodeUri(conf), conf));
    if (codecName == null || numDataUnits <= 0 || numParityUnits <= 0 || cellSize <= 0 || cellSize % 1024 != 0) {
        throw new ActionException("Illegal EC policy Schema! " + "A valid codec name should be given, " + "the dataNum, parityNum and cellSize should be positive and " + "the cellSize should be divisible by 1024.");
    }
    ECSchema ecSchema = new ECSchema(codecName, numDataUnits, numParityUnits);
    ErasureCodingPolicy ecPolicy = new ErasureCodingPolicy(ecSchema, cellSize);
    AddErasureCodingPolicyResponse addEcResponse = dfsClient.addErasureCodingPolicies(new ErasureCodingPolicy[] { ecPolicy })[0];
    if (addEcResponse.isSucceed()) {
        appendLog(String.format("EC policy named %s is added successfully!", addEcResponse.getPolicy().getName()));
    } else {
        appendLog(String.format("Failed to add the given EC policy!"));
        throw new ActionException(addEcResponse.getErrorMsg());
    }
}
Also used : ECSchema(org.apache.hadoop.io.erasurecode.ECSchema) ErasureCodingPolicy(org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy) AddErasureCodingPolicyResponse(org.apache.hadoop.hdfs.protocol.AddErasureCodingPolicyResponse) ActionException(org.smartdata.action.ActionException)

Aggregations

ErasureCodingPolicy (org.apache.hadoop.hdfs.protocol.ErasureCodingPolicy)46 Path (org.apache.hadoop.fs.Path)18 Test (org.junit.Test)16 IOException (java.io.IOException)9 DistributedFileSystem (org.apache.hadoop.hdfs.DistributedFileSystem)5 LocatedBlocks (org.apache.hadoop.hdfs.protocol.LocatedBlocks)5 DatanodeInfo (org.apache.hadoop.hdfs.protocol.DatanodeInfo)4 HdfsFileStatus (org.apache.hadoop.hdfs.protocol.HdfsFileStatus)4 LocatedBlock (org.apache.hadoop.hdfs.protocol.LocatedBlock)4 ServiceException (com.google.protobuf.ServiceException)3 MiniDFSCluster (org.apache.hadoop.hdfs.MiniDFSCluster)3 BlockType (org.apache.hadoop.hdfs.protocol.BlockType)3 ExtendedBlock (org.apache.hadoop.hdfs.protocol.ExtendedBlock)3 BlockInfo (org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo)3 BlockInfoStriped (org.apache.hadoop.hdfs.server.blockmanagement.BlockInfoStriped)3 ActionException (org.smartdata.action.ActionException)3 ByteString (com.google.protobuf.ByteString)2 HashSet (java.util.HashSet)2 List (java.util.List)2 Random (java.util.Random)2