Search in sources :

Example 31 with XceiverClientManager

use of org.apache.hadoop.hdds.scm.XceiverClientManager in project ozone by apache.

the class TestXceiverClientManager method testCaching.

@Test
public void testCaching() throws IOException {
    OzoneConfiguration conf = new OzoneConfiguration();
    String metaDir = GenericTestUtils.getTempPath(TestXceiverClientManager.class.getName() + UUID.randomUUID());
    conf.set(HDDS_METADATA_DIR_NAME, metaDir);
    XceiverClientManager clientManager = new XceiverClientManager(conf);
    ContainerWithPipeline container1 = storageContainerLocationClient.allocateContainer(SCMTestUtils.getReplicationType(conf), HddsProtos.ReplicationFactor.ONE, OzoneConsts.OZONE);
    XceiverClientSpi client1 = clientManager.acquireClient(container1.getPipeline());
    Assert.assertEquals(1, client1.getRefcount());
    ContainerWithPipeline container2 = storageContainerLocationClient.allocateContainer(SCMTestUtils.getReplicationType(conf), HddsProtos.ReplicationFactor.THREE, OzoneConsts.OZONE);
    XceiverClientSpi client2 = clientManager.acquireClient(container2.getPipeline());
    Assert.assertEquals(1, client2.getRefcount());
    XceiverClientSpi client3 = clientManager.acquireClient(container1.getPipeline());
    Assert.assertEquals(2, client3.getRefcount());
    Assert.assertEquals(2, client1.getRefcount());
    Assert.assertEquals(client1, client3);
    clientManager.releaseClient(client1, false);
    clientManager.releaseClient(client2, false);
    clientManager.releaseClient(client3, false);
}
Also used : OzoneConfiguration(org.apache.hadoop.hdds.conf.OzoneConfiguration) XceiverClientManager(org.apache.hadoop.hdds.scm.XceiverClientManager) XceiverClientSpi(org.apache.hadoop.hdds.scm.XceiverClientSpi) ContainerWithPipeline(org.apache.hadoop.hdds.scm.container.common.helpers.ContainerWithPipeline) Test(org.junit.Test)

Example 32 with XceiverClientManager

use of org.apache.hadoop.hdds.scm.XceiverClientManager in project ozone by apache.

the class TestXceiverClientManager method testFreeByRetryFailure.

@Test
public void testFreeByRetryFailure() throws IOException {
    OzoneConfiguration conf = new OzoneConfiguration();
    ScmClientConfig clientConfig = conf.getObject(ScmClientConfig.class);
    clientConfig.setMaxSize(1);
    XceiverClientManager clientManager = new XceiverClientManager(conf, clientConfig, null);
    Cache<String, XceiverClientSpi> cache = clientManager.getClientCache();
    // client is added in cache
    ContainerWithPipeline container1 = storageContainerLocationClient.allocateContainer(SCMTestUtils.getReplicationType(conf), SCMTestUtils.getReplicationFactor(conf), OzoneConsts.OZONE);
    XceiverClientSpi client1 = clientManager.acquireClient(container1.getPipeline());
    clientManager.acquireClient(container1.getPipeline());
    Assert.assertEquals(2, client1.getRefcount());
    // client should be invalidated in the cache
    clientManager.releaseClient(client1, true);
    Assert.assertEquals(1, client1.getRefcount());
    Assert.assertNull(cache.getIfPresent(container1.getContainerInfo().getPipelineID().getId().toString() + container1.getContainerInfo().getReplicationType()));
    // new client should be added in cache
    XceiverClientSpi client2 = clientManager.acquireClient(container1.getPipeline());
    Assert.assertNotEquals(client1, client2);
    Assert.assertEquals(1, client2.getRefcount());
    // on releasing the old client the entry in cache should not be invalidated
    clientManager.releaseClient(client1, true);
    Assert.assertEquals(0, client1.getRefcount());
    Assert.assertNotNull(cache.getIfPresent(container1.getContainerInfo().getPipelineID().getId().toString() + container1.getContainerInfo().getReplicationType()));
}
Also used : OzoneConfiguration(org.apache.hadoop.hdds.conf.OzoneConfiguration) ScmClientConfig(org.apache.hadoop.hdds.scm.XceiverClientManager.ScmClientConfig) XceiverClientManager(org.apache.hadoop.hdds.scm.XceiverClientManager) XceiverClientSpi(org.apache.hadoop.hdds.scm.XceiverClientSpi) ContainerWithPipeline(org.apache.hadoop.hdds.scm.container.common.helpers.ContainerWithPipeline) Test(org.junit.Test)

Aggregations

XceiverClientManager (org.apache.hadoop.hdds.scm.XceiverClientManager)32 XceiverClientSpi (org.apache.hadoop.hdds.scm.XceiverClientSpi)18 OzoneConfiguration (org.apache.hadoop.hdds.conf.OzoneConfiguration)17 ContainerWithPipeline (org.apache.hadoop.hdds.scm.container.common.helpers.ContainerWithPipeline)13 Pipeline (org.apache.hadoop.hdds.scm.pipeline.Pipeline)13 Test (org.junit.Test)12 BeforeClass (org.junit.BeforeClass)8 IOException (java.io.IOException)7 XceiverClientReply (org.apache.hadoop.hdds.scm.XceiverClientReply)7 ArrayList (java.util.ArrayList)5 OzoneClientConfig (org.apache.hadoop.hdds.scm.OzoneClientConfig)5 XceiverClientRatis (org.apache.hadoop.hdds.scm.XceiverClientRatis)5 BlockID (org.apache.hadoop.hdds.client.BlockID)4 DatanodeRatisServerConfig (org.apache.hadoop.hdds.conf.DatanodeRatisServerConfig)4 ContainerProtos (org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos)4 Random (java.util.Random)3 CompletableFuture (java.util.concurrent.CompletableFuture)3 DatanodeDetails (org.apache.hadoop.hdds.protocol.DatanodeDetails)3 ContainerCommandRequestProto (org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerCommandRequestProto)3 RatisClientConfig (org.apache.hadoop.hdds.ratis.conf.RatisClientConfig)3