Search in sources :

Example 21 with SettableFeature

use of org.apache.bookkeeper.feature.SettableFeature in project distributedlog by twitter.

the class TestRollLogSegments method testDisableRollingLogSegments.

@Test(timeout = 60000)
public void testDisableRollingLogSegments() throws Exception {
    String name = "distrlog-disable-rolling-log-segments";
    DistributedLogConfiguration confLocal = new DistributedLogConfiguration();
    confLocal.addConfiguration(conf);
    confLocal.setImmediateFlushEnabled(true);
    confLocal.setOutputBufferSize(0);
    confLocal.setLogSegmentRollingIntervalMinutes(0);
    confLocal.setMaxLogSegmentBytes(40);
    int numEntries = 100;
    BKDistributedLogManager dlm = (BKDistributedLogManager) createNewDLM(confLocal, name);
    BKAsyncLogWriter writer = dlm.startAsyncLogSegmentNonPartitioned();
    SettableFeature disableLogSegmentRolling = (SettableFeature) dlm.getFeatureProvider().getFeature(CoreFeatureKeys.DISABLE_LOGSEGMENT_ROLLING.name().toLowerCase());
    disableLogSegmentRolling.set(true);
    final CountDownLatch latch = new CountDownLatch(numEntries);
    // send requests in parallel
    for (int i = 1; i <= numEntries; i++) {
        final int entryId = i;
        writer.write(DLMTestUtil.getLogRecordInstance(entryId)).addEventListener(new FutureEventListener<DLSN>() {

            @Override
            public void onSuccess(DLSN value) {
                logger.info("Completed entry {} : {}.", entryId, value);
                latch.countDown();
            }

            @Override
            public void onFailure(Throwable cause) {
            // nope
            }
        });
    }
    latch.await();
    // make sure all ensure blocks were executed
    writer.closeAndComplete();
    List<LogSegmentMetadata> segments = dlm.getLogSegments();
    assertEquals(1, segments.size());
    dlm.close();
}
Also used : CountDownLatch(java.util.concurrent.CountDownLatch) SettableFeature(org.apache.bookkeeper.feature.SettableFeature) Test(org.junit.Test) FlakyTest(com.twitter.distributedlog.annotations.DistributedLogAnnotations.FlakyTest)

Example 22 with SettableFeature

use of org.apache.bookkeeper.feature.SettableFeature in project distributedlog by twitter.

the class TestConfigurationFeatureProvider method testConfigurationFeatureProvider.

@Test(timeout = 60000)
public void testConfigurationFeatureProvider() throws Exception {
    String rootScope = "dl";
    ConcurrentBaseConfiguration featureConf = new ConcurrentBaseConfiguration();
    ConcurrentMap<String, SettableFeature> features = new ConcurrentHashMap<String, SettableFeature>();
    ConfigurationFeatureProvider featureProvider = new ConfigurationFeatureProvider(rootScope, featureConf, features);
    String featureName1 = "feature1";
    String fullFeatureName1 = rootScope + "." + featureName1;
    int availability1 = 1234;
    featureConf.setProperty(fullFeatureName1, availability1);
    Feature feature1 = featureProvider.getFeature(featureName1);
    assertEquals(availability1, feature1.availability());
    assertTrue(features.containsKey(fullFeatureName1));
    assertTrue(feature1 == features.get(fullFeatureName1));
    String subScope = "subscope";
    String featureName2 = "feature2";
    String fullFeatureName2 = rootScope + "." + subScope + "." + featureName2;
    int availability2 = 4321;
    featureConf.setProperty(fullFeatureName2, availability2);
    Feature feature2 = featureProvider.scope(subScope).getFeature(featureName2);
    assertEquals(availability2, feature2.availability());
    assertTrue(features.containsKey(fullFeatureName2));
    assertTrue(feature2 == features.get(fullFeatureName2));
}
Also used : ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) SettableFeature(org.apache.bookkeeper.feature.SettableFeature) SettableFeature(org.apache.bookkeeper.feature.SettableFeature) Feature(org.apache.bookkeeper.feature.Feature) ConcurrentBaseConfiguration(com.twitter.distributedlog.config.ConcurrentBaseConfiguration) Test(org.junit.Test)

Example 23 with SettableFeature

use of org.apache.bookkeeper.feature.SettableFeature in project distributedlog by twitter.

the class TestWriteLimiter method testDisabledFeature.

@Test
public void testDisabledFeature() throws Exception {
    // Disable darkmode, but should still ignore limits because of the feature.
    SettableFeature feature = new SettableFeature("test", 10000);
    SimplePermitLimiter streamLimiter = createPermitLimiter(false, 1, feature);
    SimplePermitLimiter globalLimiter = createPermitLimiter(false, Integer.MAX_VALUE, feature);
    WriteLimiter limiter = new WriteLimiter("test", streamLimiter, globalLimiter);
    limiter.acquire();
    limiter.acquire();
    assertPermits(streamLimiter, 2, globalLimiter, 2);
    limiter.release();
    limiter.release();
    assertPermits(streamLimiter, 0, globalLimiter, 0);
}
Also used : SimplePermitLimiter(com.twitter.distributedlog.util.SimplePermitLimiter) SettableFeature(org.apache.bookkeeper.feature.SettableFeature) Test(org.junit.Test)

Example 24 with SettableFeature

use of org.apache.bookkeeper.feature.SettableFeature in project distributedlog by twitter.

the class TestWriteLimiter method testSetDisableFeatureAfterAcquireAndBeforeRelease.

@Test
public void testSetDisableFeatureAfterAcquireAndBeforeRelease() throws Exception {
    SettableFeature feature = new SettableFeature("test", 0);
    SimplePermitLimiter streamLimiter = createPermitLimiter(false, 2, feature);
    SimplePermitLimiter globalLimiter = createPermitLimiter(false, Integer.MAX_VALUE, feature);
    WriteLimiter limiter = new WriteLimiter("test", streamLimiter, globalLimiter);
    limiter.acquire();
    limiter.acquire();
    assertPermits(streamLimiter, 2, globalLimiter, 2);
    feature.set(10000);
    limiter.release();
    limiter.release();
    assertPermits(streamLimiter, 0, globalLimiter, 0);
}
Also used : SimplePermitLimiter(com.twitter.distributedlog.util.SimplePermitLimiter) SettableFeature(org.apache.bookkeeper.feature.SettableFeature) Test(org.junit.Test)

Example 25 with SettableFeature

use of org.apache.bookkeeper.feature.SettableFeature in project distributedlog by twitter.

the class TestWriteLimiter method testUnsetDisableFeatureAfterPermitsExceeded.

@Test
public void testUnsetDisableFeatureAfterPermitsExceeded() throws Exception {
    SettableFeature feature = new SettableFeature("test", 10000);
    SimplePermitLimiter streamLimiter = createPermitLimiter(false, 1, feature);
    SimplePermitLimiter globalLimiter = createPermitLimiter(false, Integer.MAX_VALUE, feature);
    WriteLimiter limiter = new WriteLimiter("test", streamLimiter, globalLimiter);
    limiter.acquire();
    limiter.acquire();
    limiter.acquire();
    limiter.acquire();
    assertPermits(streamLimiter, 4, globalLimiter, 4);
    feature.set(0);
    limiter.release();
    assertPermits(streamLimiter, 3, globalLimiter, 3);
    try {
        limiter.acquire();
        fail("should have thrown stream limit exception");
    } catch (OverCapacityException ex) {
    }
    assertPermits(streamLimiter, 3, globalLimiter, 3);
    limiter.release();
    limiter.release();
    limiter.release();
    assertPermits(streamLimiter, 0, globalLimiter, 0);
}
Also used : SimplePermitLimiter(com.twitter.distributedlog.util.SimplePermitLimiter) SettableFeature(org.apache.bookkeeper.feature.SettableFeature) OverCapacityException(com.twitter.distributedlog.exceptions.OverCapacityException) Test(org.junit.Test)

Aggregations

SettableFeature (org.apache.bookkeeper.feature.SettableFeature)27 Test (org.junit.Test)22 SettableFeatureProvider (org.apache.bookkeeper.feature.SettableFeatureProvider)6 SimplePermitLimiter (com.twitter.distributedlog.util.SimplePermitLimiter)5 CountDownLatch (java.util.concurrent.CountDownLatch)5 BookieSocketAddress (org.apache.bookkeeper.net.BookieSocketAddress)5 SimplePermitLimiter (org.apache.distributedlog.util.SimplePermitLimiter)5 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)4 DistributedLogConfiguration (com.twitter.distributedlog.DistributedLogConfiguration)3 ConcurrentConstConfiguration (com.twitter.distributedlog.config.ConcurrentConstConfiguration)3 DynamicDistributedLogConfiguration (com.twitter.distributedlog.config.DynamicDistributedLogConfiguration)3 HashSet (java.util.HashSet)3 BKNotEnoughBookiesException (org.apache.bookkeeper.client.BKException.BKNotEnoughBookiesException)3 ClientConfiguration (org.apache.bookkeeper.conf.ClientConfiguration)3 FeatureProvider (org.apache.bookkeeper.feature.FeatureProvider)3 OverCapacityException (com.twitter.distributedlog.exceptions.OverCapacityException)2 WriteOp (com.twitter.distributedlog.service.stream.WriteOp)2 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)2 Feature (org.apache.bookkeeper.feature.Feature)2 OverCapacityException (org.apache.distributedlog.exceptions.OverCapacityException)2