Examples with AccumuloServerContext org.apache.accumulo.server.AccumuloServerContext used on opensource projects

Search in sources :

Example 21 with AccumuloServerContext

use of org.apache.accumulo.server.AccumuloServerContext in project accumulo by apache.

the class HostRegexTableLoadBalancerTest method testSplitCurrentByRegexUsingOverlappingPools.

@Test
public void testSplitCurrentByRegexUsingOverlappingPools() {
    init(new AccumuloServerContext(instance, new TestServerConfigurationFactory(instance) {

        @Override
        public TableConfiguration getTableConfiguration(Table.ID tableId) {
            NamespaceConfiguration defaultConf = new NamespaceConfiguration(Namespace.ID.DEFAULT, this.instance, DefaultConfiguration.getInstance());
            return new TableConfiguration(instance, tableId, defaultConf) {

                HashMap<String, String> tableProperties = new HashMap<>();

                {
                    tableProperties.put(HostRegexTableLoadBalancer.HOST_BALANCER_PREFIX + FOO.getTableName(), "r.*");
                    tableProperties.put(HostRegexTableLoadBalancer.HOST_BALANCER_PREFIX + BAR.getTableName(), "r01.*|r02.*");
                }

                @Override
                public String get(Property property) {
                    return tableProperties.get(property.name());
                }

                @Override
                public void getProperties(Map<String, String> props, Predicate<String> filter) {
                    for (Entry<String, String> e : tableProperties.entrySet()) {
                        if (filter.test(e.getKey())) {
                            props.put(e.getKey(), e.getValue());
                        }
                    }
                }

                @Override
                public long getUpdateCount() {
                    return 0;
                }
            };
        }
    }));
    Map<String, SortedMap<TServerInstance, TabletServerStatus>> groups = this.splitCurrentByRegex(createCurrent(15));
    Assert.assertEquals(2, groups.size());
    Assert.assertTrue(groups.containsKey(FOO.getTableName()));
    SortedMap<TServerInstance, TabletServerStatus> fooHosts = groups.get(FOO.getTableName());
    Assert.assertEquals(15, fooHosts.size());
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.1:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.2:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.3:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.4:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.5:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.6:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.7:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.8:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.9:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.10:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.11:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.12:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.13:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.14:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.15:9997", 1)));
    Assert.assertTrue(groups.containsKey(BAR.getTableName()));
    SortedMap<TServerInstance, TabletServerStatus> barHosts = groups.get(BAR.getTableName());
    Assert.assertEquals(10, barHosts.size());
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.1:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.2:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.3:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.4:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.5:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.6:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.7:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.8:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.9:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.10:9997", 1)));
}
Also used : AccumuloServerContext(org.apache.accumulo.server.AccumuloServerContext) HashMap(java.util.HashMap) TServerInstance(org.apache.accumulo.server.master.state.TServerInstance) Predicate(java.util.function.Predicate) SortedMap(java.util.SortedMap) NamespaceConfiguration(org.apache.accumulo.server.conf.NamespaceConfiguration) Property(org.apache.accumulo.core.conf.Property) HashMap(java.util.HashMap) Map(java.util.Map) SortedMap(java.util.SortedMap) TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration) TabletServerStatus(org.apache.accumulo.core.master.thrift.TabletServerStatus) Test(org.junit.Test)

Example 22 with AccumuloServerContext

use of org.apache.accumulo.server.AccumuloServerContext in project accumulo by apache.

the class HostRegexTableLoadBalancerTest method testInit.

@Test
public void testInit() {
    init(new AccumuloServerContext(instance, factory));
    Assert.assertEquals("OOB check interval value is incorrect", 7000, this.getOobCheckMillis());
    Assert.assertEquals("Max migrations is incorrect", 4, this.getMaxMigrations());
    Assert.assertEquals("Max outstanding migrations is incorrect", 10, this.getMaxOutstandingMigrations());
    Assert.assertFalse(isIpBasedRegex());
    Map<String, Pattern> patterns = this.getPoolNameToRegexPattern();
    Assert.assertEquals(2, patterns.size());
    Assert.assertTrue(patterns.containsKey(FOO.getTableName()));
    Assert.assertEquals(Pattern.compile("r01.*").pattern(), patterns.get(FOO.getTableName()).pattern());
    Assert.assertTrue(patterns.containsKey(BAR.getTableName()));
    Assert.assertEquals(Pattern.compile("r02.*").pattern(), patterns.get(BAR.getTableName()).pattern());
    Map<Table.ID, String> tids = this.getTableIdToTableName();
    Assert.assertEquals(3, tids.size());
    Assert.assertTrue(tids.containsKey(FOO.getId()));
    Assert.assertEquals(FOO.getTableName(), tids.get(FOO.getId()));
    Assert.assertTrue(tids.containsKey(BAR.getId()));
    Assert.assertEquals(BAR.getTableName(), tids.get(BAR.getId()));
    Assert.assertTrue(tids.containsKey(BAZ.getId()));
    Assert.assertEquals(BAZ.getTableName(), tids.get(BAZ.getId()));
    Assert.assertEquals(false, this.isIpBasedRegex());
}
Also used : Pattern(java.util.regex.Pattern) AccumuloServerContext(org.apache.accumulo.server.AccumuloServerContext) Test(org.junit.Test)

Example 23 with AccumuloServerContext

use of org.apache.accumulo.server.AccumuloServerContext in project accumulo by apache.

the class HostRegexTableLoadBalancerTest method testOutOfBoundsTablets.

@Test
public void testOutOfBoundsTablets() {
    init(new AccumuloServerContext(instance, factory));
    // Wait to trigger the out of bounds check which will call our version of getOnlineTabletsForTable
    UtilWaitThread.sleep(11000);
    Set<KeyExtent> migrations = new HashSet<>();
    List<TabletMigration> migrationsOut = new ArrayList<>();
    this.balance(createCurrent(15), migrations, migrationsOut);
    Assert.assertEquals(2, migrationsOut.size());
}
Also used : AccumuloServerContext(org.apache.accumulo.server.AccumuloServerContext) TabletMigration(org.apache.accumulo.server.master.state.TabletMigration) ArrayList(java.util.ArrayList) TKeyExtent(org.apache.accumulo.core.data.thrift.TKeyExtent) KeyExtent(org.apache.accumulo.core.data.impl.KeyExtent) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 24 with AccumuloServerContext

use of org.apache.accumulo.server.AccumuloServerContext in project accumulo by apache.

the class HostRegexTableLoadBalancerTest method testSplitCurrentByRegexUsingIP.

@Test
public void testSplitCurrentByRegexUsingIP() {
    init(new AccumuloServerContext(instance, new TestServerConfigurationFactory(instance) {

        @Override
        public synchronized AccumuloConfiguration getSystemConfiguration() {
            HashMap<String, String> props = new HashMap<>();
            props.put(HostRegexTableLoadBalancer.HOST_BALANCER_OOB_CHECK_KEY, "30s");
            props.put(HostRegexTableLoadBalancer.HOST_BALANCER_REGEX_USING_IPS_KEY, "true");
            return new ConfigurationCopy(props);
        }

        @Override
        public TableConfiguration getTableConfiguration(Table.ID tableId) {
            NamespaceConfiguration defaultConf = new NamespaceConfiguration(Namespace.ID.DEFAULT, this.instance, DefaultConfiguration.getInstance());
            return new TableConfiguration(instance, tableId, defaultConf) {

                HashMap<String, String> tableProperties = new HashMap<>();

                {
                    tableProperties.put(HostRegexTableLoadBalancer.HOST_BALANCER_PREFIX + FOO.getTableName(), "192\\.168\\.0\\.[1-5]");
                    tableProperties.put(HostRegexTableLoadBalancer.HOST_BALANCER_PREFIX + BAR.getTableName(), "192\\.168\\.0\\.[6-9]|192\\.168\\.0\\.10");
                }

                @Override
                public String get(Property property) {
                    return tableProperties.get(property.name());
                }

                @Override
                public void getProperties(Map<String, String> props, Predicate<String> filter) {
                    for (Entry<String, String> e : tableProperties.entrySet()) {
                        if (filter.test(e.getKey())) {
                            props.put(e.getKey(), e.getValue());
                        }
                    }
                }

                @Override
                public long getUpdateCount() {
                    return 0;
                }
            };
        }
    }));
    Assert.assertTrue(isIpBasedRegex());
    Map<String, SortedMap<TServerInstance, TabletServerStatus>> groups = this.splitCurrentByRegex(createCurrent(15));
    Assert.assertEquals(3, groups.size());
    Assert.assertTrue(groups.containsKey(FOO.getTableName()));
    SortedMap<TServerInstance, TabletServerStatus> fooHosts = groups.get(FOO.getTableName());
    Assert.assertEquals(5, fooHosts.size());
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.1:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.2:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.3:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.4:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.5:9997", 1)));
    Assert.assertTrue(groups.containsKey(BAR.getTableName()));
    SortedMap<TServerInstance, TabletServerStatus> barHosts = groups.get(BAR.getTableName());
    Assert.assertEquals(5, barHosts.size());
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.6:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.7:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.8:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.9:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.10:9997", 1)));
    Assert.assertTrue(groups.containsKey(DEFAULT_POOL));
    SortedMap<TServerInstance, TabletServerStatus> defHosts = groups.get(DEFAULT_POOL);
    Assert.assertEquals(5, defHosts.size());
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.11:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.12:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.13:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.14:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.15:9997", 1)));
}
Also used : AccumuloServerContext(org.apache.accumulo.server.AccumuloServerContext) ConfigurationCopy(org.apache.accumulo.core.conf.ConfigurationCopy) HashMap(java.util.HashMap) TServerInstance(org.apache.accumulo.server.master.state.TServerInstance) Predicate(java.util.function.Predicate) SortedMap(java.util.SortedMap) NamespaceConfiguration(org.apache.accumulo.server.conf.NamespaceConfiguration) Property(org.apache.accumulo.core.conf.Property) HashMap(java.util.HashMap) Map(java.util.Map) SortedMap(java.util.SortedMap) TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration) TabletServerStatus(org.apache.accumulo.core.master.thrift.TabletServerStatus) Test(org.junit.Test)

Example 25 with AccumuloServerContext

use of org.apache.accumulo.server.AccumuloServerContext in project accumulo by apache.

the class HostRegexTableLoadBalancerTest method testSplitCurrentByRegexUsingHostname.

@Test
public void testSplitCurrentByRegexUsingHostname() {
    init(new AccumuloServerContext(instance, factory));
    Map<String, SortedMap<TServerInstance, TabletServerStatus>> groups = this.splitCurrentByRegex(createCurrent(15));
    Assert.assertEquals(3, groups.size());
    Assert.assertTrue(groups.containsKey(FOO.getTableName()));
    SortedMap<TServerInstance, TabletServerStatus> fooHosts = groups.get(FOO.getTableName());
    Assert.assertEquals(5, fooHosts.size());
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.1:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.2:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.3:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.4:9997", 1)));
    Assert.assertTrue(fooHosts.containsKey(new TServerInstance("192.168.0.5:9997", 1)));
    Assert.assertTrue(groups.containsKey(BAR.getTableName()));
    SortedMap<TServerInstance, TabletServerStatus> barHosts = groups.get(BAR.getTableName());
    Assert.assertEquals(5, barHosts.size());
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.6:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.7:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.8:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.9:9997", 1)));
    Assert.assertTrue(barHosts.containsKey(new TServerInstance("192.168.0.10:9997", 1)));
    Assert.assertTrue(groups.containsKey(DEFAULT_POOL));
    SortedMap<TServerInstance, TabletServerStatus> defHosts = groups.get(DEFAULT_POOL);
    Assert.assertEquals(5, defHosts.size());
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.11:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.12:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.13:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.14:9997", 1)));
    Assert.assertTrue(defHosts.containsKey(new TServerInstance("192.168.0.15:9997", 1)));
}
Also used : AccumuloServerContext(org.apache.accumulo.server.AccumuloServerContext) SortedMap(java.util.SortedMap) TServerInstance(org.apache.accumulo.server.master.state.TServerInstance) TabletServerStatus(org.apache.accumulo.core.master.thrift.TabletServerStatus) Test(org.junit.Test)

Aggregations

AccumuloServerContext (org.apache.accumulo.server.AccumuloServerContext)38 Test (org.junit.Test)20 ServerConfigurationFactory (org.apache.accumulo.server.conf.ServerConfigurationFactory)17 Instance (org.apache.accumulo.core.client.Instance)16 KeyExtent (org.apache.accumulo.core.data.impl.KeyExtent)12 TServerInstance (org.apache.accumulo.server.master.state.TServerInstance)12 HashMap (java.util.HashMap)9 TKeyExtent (org.apache.accumulo.core.data.thrift.TKeyExtent)8 HdfsZooInstance (org.apache.accumulo.server.client.HdfsZooInstance)8 ArrayList (java.util.ArrayList)7 VolumeManager (org.apache.accumulo.server.fs.VolumeManager)7 Connector (org.apache.accumulo.core.client.Connector)6 GCStatus (org.apache.accumulo.core.gc.thrift.GCStatus)6 Map (java.util.Map)5 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)5 GcCycleStats (org.apache.accumulo.core.gc.thrift.GcCycleStats)5 TabletServerStatus (org.apache.accumulo.core.master.thrift.TabletServerStatus)5 WalStateManager (org.apache.accumulo.server.log.WalStateManager)5 LiveTServerSet (org.apache.accumulo.server.master.LiveTServerSet)5 TabletMigration (org.apache.accumulo.server.master.state.TabletMigration)5
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial