Search in sources :

Example 6 with ShortCircuitReplicaInfo

use of org.apache.hadoop.hdfs.shortcircuit.ShortCircuitReplicaInfo in project hadoop by apache.

the class BlockReaderFactory method getBlockReaderLocal.

private BlockReader getBlockReaderLocal() throws InvalidToken {
    LOG.trace("{}: trying to construct a BlockReaderLocal for short-circuit " + " reads.", this);
    if (pathInfo == null) {
        pathInfo = clientContext.getDomainSocketFactory().getPathInfo(inetSocketAddress, conf.getShortCircuitConf());
    }
    if (!pathInfo.getPathState().getUsableForShortCircuit()) {
        PerformanceAdvisory.LOG.debug("{}: {} is not usable for short circuit; " + "giving up on BlockReaderLocal.", this, pathInfo);
        return null;
    }
    ShortCircuitCache cache = clientContext.getShortCircuitCache();
    ExtendedBlockId key = new ExtendedBlockId(block.getBlockId(), block.getBlockPoolId());
    ShortCircuitReplicaInfo info = cache.fetchOrCreate(key, this);
    InvalidToken exc = info.getInvalidTokenException();
    if (exc != null) {
        LOG.trace("{}: got InvalidToken exception while trying to construct " + "BlockReaderLocal via {}", this, pathInfo.getPath());
        throw exc;
    }
    if (info.getReplica() == null) {
        PerformanceAdvisory.LOG.debug("{}: failed to get " + "ShortCircuitReplica. Cannot construct " + "BlockReaderLocal via {}", this, pathInfo.getPath());
        return null;
    }
    return new BlockReaderLocal.Builder(conf.getShortCircuitConf()).setFilename(fileName).setBlock(block).setStartOffset(startOffset).setShortCircuitReplica(info.getReplica()).setVerifyChecksum(verifyChecksum).setCachingStrategy(cachingStrategy).setStorageType(storageType).setTracer(tracer).build();
}
Also used : ExtendedBlockId(org.apache.hadoop.hdfs.ExtendedBlockId) InvalidToken(org.apache.hadoop.security.token.SecretManager.InvalidToken) ShortCircuitCache(org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache) ShortCircuitReplicaInfo(org.apache.hadoop.hdfs.shortcircuit.ShortCircuitReplicaInfo)

Aggregations

ShortCircuitCache (org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache)6 ShortCircuitReplicaInfo (org.apache.hadoop.hdfs.shortcircuit.ShortCircuitReplicaInfo)6 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 Configuration (org.apache.hadoop.conf.Configuration)3 Path (org.apache.hadoop.fs.Path)3 DistributedFileSystem (org.apache.hadoop.hdfs.DistributedFileSystem)3 ExtendedBlockId (org.apache.hadoop.hdfs.ExtendedBlockId)3 MiniDFSCluster (org.apache.hadoop.hdfs.MiniDFSCluster)3 DatanodeInfoBuilder (org.apache.hadoop.hdfs.protocol.DatanodeInfo.DatanodeInfoBuilder)3 TemporarySocketDirectory (org.apache.hadoop.net.unix.TemporarySocketDirectory)3 Test (org.junit.Test)3 IOException (java.io.IOException)2 CountDownLatch (java.util.concurrent.CountDownLatch)2 BlockReader (org.apache.hadoop.hdfs.BlockReader)2 LocatedBlock (org.apache.hadoop.hdfs.protocol.LocatedBlock)2 InvalidToken (org.apache.hadoop.security.token.SecretManager.InvalidToken)2 BufferedOutputStream (java.io.BufferedOutputStream)1 DataInputStream (java.io.DataInputStream)1 DataOutputStream (java.io.DataOutputStream)1 FileInputStream (java.io.FileInputStream)1