Search in sources :

Example 16 with SimpleScopeType

use of org.apache.gobblin.broker.SimpleScopeType in project incubator-gobblin by apache.

the class SharedLimiterFactoryTest method testCountLimiter.

@Test
public void testCountLimiter() throws Exception {
    SharedResourcesBrokerImpl<SimpleScopeType> broker = getBrokerForConfigMap(ImmutableMap.of(JOINER.join(BrokerConstants.GOBBLIN_BROKER_CONFIG_PREFIX, SharedLimiterFactory.NAME, SharedLimiterFactory.LIMITER_CLASS_KEY), "CountBasedLimiter", JOINER.join(BrokerConstants.GOBBLIN_BROKER_CONFIG_PREFIX, SharedLimiterFactory.NAME, CountBasedLimiter.Factory.COUNT_KEY), "10"));
    SharedLimiterFactory<SimpleScopeType> factory = new SharedLimiterFactory<>();
    Assert.assertEquals(factory.getAutoScope(broker, broker.getConfigView(SimpleScopeType.LOCAL, new SharedLimiterKey("resource"), factory.getName())), SimpleScopeType.GLOBAL);
    Limiter limiter = ((ResourceInstance<Limiter>) factory.createResource(broker, broker.getConfigView(SimpleScopeType.GLOBAL, new SharedLimiterKey("resource"), factory.getName()))).getResource();
    Assert.assertTrue(limiter instanceof CountBasedLimiter);
    Assert.assertEquals(((CountBasedLimiter) limiter).getCountLimit(), 10);
}
Also used : CountBasedLimiter(org.apache.gobblin.util.limiter.CountBasedLimiter) ResourceInstance(org.apache.gobblin.broker.ResourceInstance) SimpleScopeType(org.apache.gobblin.broker.SimpleScopeType) Limiter(org.apache.gobblin.util.limiter.Limiter) CountBasedLimiter(org.apache.gobblin.util.limiter.CountBasedLimiter) MultiLimiter(org.apache.gobblin.util.limiter.MultiLimiter) NoopLimiter(org.apache.gobblin.util.limiter.NoopLimiter) Test(org.testng.annotations.Test)

Example 17 with SimpleScopeType

use of org.apache.gobblin.broker.SimpleScopeType in project incubator-gobblin by apache.

the class SharedLimiterFactoryTest method testEmptyConfig.

@Test
public void testEmptyConfig() throws Exception {
    SharedResourcesBrokerImpl<SimpleScopeType> broker = getBrokerForConfigMap(ImmutableMap.<String, String>of());
    SharedLimiterFactory<SimpleScopeType> factory = new SharedLimiterFactory<>();
    Assert.assertEquals(factory.getAutoScope(broker, broker.getConfigView(SimpleScopeType.LOCAL, new SharedLimiterKey("resource"), factory.getName())), SimpleScopeType.GLOBAL);
    Limiter limiter = ((ResourceInstance<Limiter>) factory.createResource(broker, broker.getConfigView(SimpleScopeType.GLOBAL, new SharedLimiterKey("resource"), factory.getName()))).getResource();
    Assert.assertTrue(limiter instanceof NoopLimiter);
}
Also used : ResourceInstance(org.apache.gobblin.broker.ResourceInstance) NoopLimiter(org.apache.gobblin.util.limiter.NoopLimiter) SimpleScopeType(org.apache.gobblin.broker.SimpleScopeType) Limiter(org.apache.gobblin.util.limiter.Limiter) CountBasedLimiter(org.apache.gobblin.util.limiter.CountBasedLimiter) MultiLimiter(org.apache.gobblin.util.limiter.MultiLimiter) NoopLimiter(org.apache.gobblin.util.limiter.NoopLimiter) Test(org.testng.annotations.Test)

Aggregations

SimpleScopeType (org.apache.gobblin.broker.SimpleScopeType)17 Test (org.testng.annotations.Test)15 SharedLimiterKey (org.apache.gobblin.util.limiter.broker.SharedLimiterKey)6 Config (com.typesafe.config.Config)4 Limiter (org.apache.gobblin.util.limiter.Limiter)4 MultiLimiter (org.apache.gobblin.util.limiter.MultiLimiter)4 NoopLimiter (org.apache.gobblin.util.limiter.NoopLimiter)4 URI (java.net.URI)3 ResourceInstance (org.apache.gobblin.broker.ResourceInstance)3 SharedRestClientKey (org.apache.gobblin.restli.SharedRestClientKey)3 CountBasedLimiter (org.apache.gobblin.util.limiter.CountBasedLimiter)3 Configuration (org.apache.hadoop.conf.Configuration)3 FileSystem (org.apache.hadoop.fs.FileSystem)3 LocalFileSystem (org.apache.hadoop.fs.LocalFileSystem)3 ImmutableMap (com.google.common.collect.ImmutableMap)2 Map (java.util.Map)2 SharedResourcesBroker (org.apache.gobblin.broker.iface.SharedResourcesBroker)2 MetricContext (org.apache.gobblin.metrics.MetricContext)2 EmbeddedRestliServer (org.apache.gobblin.restli.EmbeddedRestliServer)2 ThrottlingGuiceServletConfig (org.apache.gobblin.restli.throttling.ThrottlingGuiceServletConfig)2