Search in sources :

Example 6 with BlockWithLocations

use of org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations in project hadoop by apache.

the class TestPBHelper method getBlockWithLocations.

private static BlockWithLocations getBlockWithLocations(int bid, boolean isStriped) {
    final String[] datanodeUuids = { "dn1", "dn2", "dn3" };
    final String[] storageIDs = { "s1", "s2", "s3" };
    final StorageType[] storageTypes = { StorageType.DISK, StorageType.DISK, StorageType.DISK };
    final byte[] indices = { 0, 1, 2 };
    final short dataBlkNum = 6;
    BlockWithLocations blkLocs = new BlockWithLocations(new Block(bid, 0, 1), datanodeUuids, storageIDs, storageTypes);
    if (isStriped) {
        blkLocs = new StripedBlockWithLocations(blkLocs, indices, dataBlkNum, StripedFileTestUtil.getDefaultECPolicy().getCellSize());
    }
    return blkLocs;
}
Also used : StorageType(org.apache.hadoop.fs.StorageType) StripedBlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations) StripedBlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations) BlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations) Block(org.apache.hadoop.hdfs.protocol.Block) ExtendedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock) RecoveringBlock(org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand.RecoveringBlock) LocatedBlock(org.apache.hadoop.hdfs.protocol.LocatedBlock) ByteString(com.google.protobuf.ByteString)

Example 7 with BlockWithLocations

use of org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations in project hadoop by apache.

the class TestPBHelper method testConvertBlocksWithLocations.

@Test
public void testConvertBlocksWithLocations() {
    boolean[] testSuite = new boolean[] { false, true };
    for (int i = 0; i < testSuite.length; i++) {
        BlockWithLocations[] list = new BlockWithLocations[] { getBlockWithLocations(1, testSuite[i]), getBlockWithLocations(2, testSuite[i]) };
        BlocksWithLocations locs = new BlocksWithLocations(list);
        BlocksWithLocationsProto locsProto = PBHelper.convert(locs);
        BlocksWithLocations locs2 = PBHelper.convert(locsProto);
        BlockWithLocations[] blocks = locs.getBlocks();
        BlockWithLocations[] blocks2 = locs2.getBlocks();
        assertEquals(blocks.length, blocks2.length);
        for (int j = 0; j < blocks.length; j++) {
            compare(blocks[j], blocks2[j]);
        }
    }
}
Also used : BlocksWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations) BlocksWithLocationsProto(org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlocksWithLocationsProto) StripedBlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations) BlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations) Test(org.junit.Test)

Example 8 with BlockWithLocations

use of org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations in project hadoop by apache.

the class PBHelper method convert.

public static BlockWithLocations convert(BlockWithLocationsProto b) {
    final List<String> datanodeUuids = b.getDatanodeUuidsList();
    final List<String> storageUuids = b.getStorageUuidsList();
    final List<StorageTypeProto> storageTypes = b.getStorageTypesList();
    BlockWithLocations blk = new BlockWithLocations(PBHelperClient.convert(b.getBlock()), datanodeUuids.toArray(new String[datanodeUuids.size()]), storageUuids.toArray(new String[storageUuids.size()]), PBHelperClient.convertStorageTypes(storageTypes, storageUuids.size()));
    if (b.hasIndices()) {
        blk = new StripedBlockWithLocations(blk, b.getIndices().toByteArray(), (short) b.getDataBlockNum(), b.getCellSize());
    }
    return blk;
}
Also used : StripedBlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations) StripedBlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations) BlockWithLocations(org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations) StorageTypeProto(org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageTypeProto) ByteString(com.google.protobuf.ByteString)

Aggregations

BlockWithLocations (org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.BlockWithLocations)8 StripedBlockWithLocations (org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations.StripedBlockWithLocations)7 BlocksWithLocations (org.apache.hadoop.hdfs.server.protocol.BlocksWithLocations)3 Test (org.junit.Test)3 ByteString (com.google.protobuf.ByteString)2 StorageType (org.apache.hadoop.fs.StorageType)2 LocatedBlock (org.apache.hadoop.hdfs.protocol.LocatedBlock)2 BlockWithLocationsProto (org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlockWithLocationsProto)2 InetSocketAddress (java.net.InetSocketAddress)1 ArrayList (java.util.ArrayList)1 HadoopIllegalArgumentException (org.apache.hadoop.HadoopIllegalArgumentException)1 Configuration (org.apache.hadoop.conf.Configuration)1 Path (org.apache.hadoop.fs.Path)1 Block (org.apache.hadoop.hdfs.protocol.Block)1 DatanodeInfo (org.apache.hadoop.hdfs.protocol.DatanodeInfo)1 ExtendedBlock (org.apache.hadoop.hdfs.protocol.ExtendedBlock)1 StorageTypeProto (org.apache.hadoop.hdfs.protocol.proto.HdfsProtos.StorageTypeProto)1 BlocksWithLocationsProto (org.apache.hadoop.hdfs.protocol.proto.HdfsServerProtos.BlocksWithLocationsProto)1 ReportedBlockInfo (org.apache.hadoop.hdfs.server.blockmanagement.PendingDataNodeMessages.ReportedBlockInfo)1 RecoveringBlock (org.apache.hadoop.hdfs.server.protocol.BlockRecoveryCommand.RecoveringBlock)1