Search in sources :

Example 11 with HDFSBlocksDistribution

use of org.apache.hadoop.hbase.HDFSBlocksDistribution in project hbase by apache.

the class HRegion method getHDFSBlocksDistribution.

@Override
public HDFSBlocksDistribution getHDFSBlocksDistribution() {
    HDFSBlocksDistribution hdfsBlocksDistribution = new HDFSBlocksDistribution();
    synchronized (this.stores) {
        for (Store store : this.stores.values()) {
            Collection<StoreFile> storeFiles = store.getStorefiles();
            if (storeFiles == null)
                continue;
            for (StoreFile sf : storeFiles) {
                HDFSBlocksDistribution storeFileBlocksDistribution = sf.getHDFSBlockDistribution();
                hdfsBlocksDistribution.add(storeFileBlocksDistribution);
            }
        }
    }
    return hdfsBlocksDistribution;
}
Also used : HDFSBlocksDistribution(org.apache.hadoop.hbase.HDFSBlocksDistribution)

Example 12 with HDFSBlocksDistribution

use of org.apache.hadoop.hbase.HDFSBlocksDistribution in project hbase by apache.

the class TestRegionLocationFinder method testInternalGetTopBlockLocation.

@Test
public void testInternalGetTopBlockLocation() throws Exception {
    for (int i = 0; i < ServerNum; i++) {
        HRegionServer server = cluster.getRegionServer(i);
        for (Region region : server.getOnlineRegions(tableName)) {
            // get region's hdfs block distribution by region and RegionLocationFinder, 
            // they should have same result
            HDFSBlocksDistribution blocksDistribution1 = region.getHDFSBlocksDistribution();
            HDFSBlocksDistribution blocksDistribution2 = finder.getBlockDistribution(region.getRegionInfo());
            assertEquals(blocksDistribution1.getUniqueBlocksTotalWeight(), blocksDistribution2.getUniqueBlocksTotalWeight());
            if (blocksDistribution1.getUniqueBlocksTotalWeight() != 0) {
                assertEquals(blocksDistribution1.getTopHosts().get(0), blocksDistribution2.getTopHosts().get(0));
            }
        }
    }
}
Also used : Region(org.apache.hadoop.hbase.regionserver.Region) HDFSBlocksDistribution(org.apache.hadoop.hbase.HDFSBlocksDistribution) HRegionServer(org.apache.hadoop.hbase.regionserver.HRegionServer) Test(org.junit.Test)

Aggregations

HDFSBlocksDistribution (org.apache.hadoop.hbase.HDFSBlocksDistribution)12 Test (org.junit.Test)4 FileSystem (org.apache.hadoop.fs.FileSystem)3 Path (org.apache.hadoop.fs.Path)3 HBaseTestingUtility (org.apache.hadoop.hbase.HBaseTestingUtility)3 ArrayList (java.util.ArrayList)2 ExecutionException (java.util.concurrent.ExecutionException)2 Configuration (org.apache.hadoop.conf.Configuration)2 FileStatus (org.apache.hadoop.fs.FileStatus)2 LocatedFileStatus (org.apache.hadoop.fs.LocatedFileStatus)2 Table (org.apache.hadoop.hbase.client.Table)2 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)1 IOException (java.io.IOException)1 InterruptedIOException (java.io.InterruptedIOException)1 BlockLocation (org.apache.hadoop.fs.BlockLocation)1 Cell (org.apache.hadoop.hbase.Cell)1 DoNotRetryIOException (org.apache.hadoop.hbase.DoNotRetryIOException)1 HBaseConfiguration (org.apache.hadoop.hbase.HBaseConfiguration)1 HColumnDescriptor (org.apache.hadoop.hbase.HColumnDescriptor)1 HRegionInfo (org.apache.hadoop.hbase.HRegionInfo)1