Search in sources :

Example 11 with TableConfiguration

use of org.apache.accumulo.server.conf.TableConfiguration in project accumulo by apache.

the class LargestFirstMemoryManagerTest method testDeletedTable.

@Test
public void testDeletedTable() throws Exception {
    final String deletedTableId = "1";
    Function<Table.ID, Boolean> existenceCheck = tableId -> !deletedTableId.contentEquals(tableId.canonicalID());
    LargestFirstMemoryManagerWithExistenceCheck mgr = new LargestFirstMemoryManagerWithExistenceCheck(existenceCheck);
    ServerConfiguration config = new ServerConfiguration() {

        ServerConfigurationFactory delegate = new ServerConfigurationFactory(inst);

        @Override
        public AccumuloConfiguration getSystemConfiguration() {
            return DefaultConfiguration.getInstance();
        }

        @Override
        public TableConfiguration getTableConfiguration(Table.ID tableId) {
            return delegate.getTableConfiguration(tableId);
        }

        @Override
        public NamespaceConfiguration getNamespaceConfiguration(Namespace.ID namespaceId) {
            return delegate.getNamespaceConfiguration(namespaceId);
        }
    };
    mgr.init(config);
    MemoryManagementActions result;
    // one tablet is really big and the other is for a nonexistent table
    KeyExtent extent = new KeyExtent(Table.ID.of("2"), new Text("j"), null);
    result = mgr.getMemoryManagementActions(tablets(t(extent, ZERO, ONE_GIG, 0), t(k("j"), ZERO, ONE_GIG, 0)));
    assertEquals(1, result.tabletsToMinorCompact.size());
    assertEquals(extent, result.tabletsToMinorCompact.get(0));
}
Also used : Arrays(java.util.Arrays) Table(org.apache.accumulo.core.client.impl.Table) NamespaceConfiguration(org.apache.accumulo.server.conf.NamespaceConfiguration) TabletState(org.apache.accumulo.server.tabletserver.TabletState) Text(org.apache.hadoop.io.Text) Instance(org.apache.accumulo.core.client.Instance) Test(org.junit.Test) EasyMock(org.easymock.EasyMock) Function(java.util.function.Function) LargestFirstMemoryManager(org.apache.accumulo.server.tabletserver.LargestFirstMemoryManager) DefaultConfiguration(org.apache.accumulo.core.conf.DefaultConfiguration) KeyExtent(org.apache.accumulo.core.data.impl.KeyExtent) AccumuloConfiguration(org.apache.accumulo.core.conf.AccumuloConfiguration) SiteConfiguration(org.apache.accumulo.core.conf.SiteConfiguration) Namespace(org.apache.accumulo.core.client.impl.Namespace) List(java.util.List) ServerConfigurationFactory(org.apache.accumulo.server.conf.ServerConfigurationFactory) TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration) ServerConfiguration(org.apache.accumulo.server.conf.ServerConfiguration) Assert.assertEquals(org.junit.Assert.assertEquals) Property(org.apache.accumulo.core.conf.Property) MemoryManagementActions(org.apache.accumulo.server.tabletserver.MemoryManagementActions) Before(org.junit.Before) MemoryManagementActions(org.apache.accumulo.server.tabletserver.MemoryManagementActions) ServerConfiguration(org.apache.accumulo.server.conf.ServerConfiguration) ServerConfigurationFactory(org.apache.accumulo.server.conf.ServerConfigurationFactory) Text(org.apache.hadoop.io.Text) KeyExtent(org.apache.accumulo.core.data.impl.KeyExtent) Test(org.junit.Test)

Example 12 with TableConfiguration

use of org.apache.accumulo.server.conf.TableConfiguration in project accumulo by apache.

the class TableConfigurationUpdateIT method test.

@Test
public void test() throws Exception {
    Connector conn = getConnector();
    Instance inst = conn.getInstance();
    String table = getUniqueNames(1)[0];
    conn.tableOperations().create(table);
    final NamespaceConfiguration defaultConf = new NamespaceConfiguration(Namespace.ID.DEFAULT, inst, DefaultConfiguration.getInstance());
    // Cache invalidates 25% of the time
    int randomMax = 4;
    // Number of threads
    int numThreads = 2;
    // Number of iterations per thread
    int iterations = 100000;
    AccumuloConfiguration tableConf = new TableConfiguration(inst, org.apache.accumulo.core.client.impl.Table.ID.of(table), defaultConf);
    long start = System.currentTimeMillis();
    ExecutorService svc = Executors.newFixedThreadPool(numThreads);
    CountDownLatch countDown = new CountDownLatch(numThreads);
    ArrayList<Future<Exception>> futures = new ArrayList<>(numThreads);
    for (int i = 0; i < numThreads; i++) {
        futures.add(svc.submit(new TableConfRunner(randomMax, iterations, tableConf, countDown)));
    }
    svc.shutdown();
    Assert.assertTrue(svc.awaitTermination(60, TimeUnit.MINUTES));
    for (Future<Exception> fut : futures) {
        Exception e = fut.get();
        if (null != e) {
            Assert.fail("Thread failed with exception " + e);
        }
    }
    long end = System.currentTimeMillis();
    log.debug("{} with {} iterations and {} threads and cache invalidates {}% took {} second(s)", tableConf, iterations, numThreads, ((1. / randomMax) * 100.), (end - start) / 1000);
}
Also used : Connector(org.apache.accumulo.core.client.Connector) Instance(org.apache.accumulo.core.client.Instance) ArrayList(java.util.ArrayList) CountDownLatch(java.util.concurrent.CountDownLatch) ExecutorService(java.util.concurrent.ExecutorService) Future(java.util.concurrent.Future) NamespaceConfiguration(org.apache.accumulo.server.conf.NamespaceConfiguration) TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration) AccumuloConfiguration(org.apache.accumulo.core.conf.AccumuloConfiguration) Test(org.junit.Test)

Example 13 with TableConfiguration

use of org.apache.accumulo.server.conf.TableConfiguration in project accumulo by apache.

the class PerTableVolumeChooser method getVolumeChooserForTable.

private VolumeChooser getVolumeChooserForTable(VolumeChooserEnvironment env, ServerConfigurationFactory confFactory) {
    log.trace("Looking up property {} for table id: {}", TABLE_VOLUME_CHOOSER, env.getTableId());
    final TableConfiguration tableConf = confFactory.getTableConfiguration(env.getTableId());
    String clazz = tableConf.get(TABLE_VOLUME_CHOOSER);
    // fall back to global default scope, so setting only one default is necessary, rather than a separate default for TABLE scope than other scopes
    if (null == clazz || clazz.isEmpty()) {
        clazz = confFactory.getSystemConfiguration().get(DEFAULT_SCOPED_VOLUME_CHOOSER);
    }
    if (null == clazz || clazz.isEmpty()) {
        String msg = "Property " + TABLE_VOLUME_CHOOSER + " or " + DEFAULT_SCOPED_VOLUME_CHOOSER + " must be a valid " + VolumeChooser.class.getSimpleName() + " to use the " + getClass().getSimpleName();
        throw new VolumeChooserException(msg);
    }
    // can be null
    String context = getTableContext(tableConf);
    return createVolumeChooser(context, clazz, TABLE_VOLUME_CHOOSER, env.getTableId(), tableSpecificChooserCache);
}
Also used : TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration)

Example 14 with TableConfiguration

use of org.apache.accumulo.server.conf.TableConfiguration in project accumulo by apache.

the class PreferredVolumeChooser method getPreferredVolumesForTable.

private String[] getPreferredVolumesForTable(VolumeChooserEnvironment env, ServerConfigurationFactory confFactory, String[] options) {
    log.trace("Looking up property {} + for Table id: {}", TABLE_PREFERRED_VOLUMES, env.getTableId());
    final TableConfiguration tableConf = confFactory.getTableConfiguration(env.getTableId());
    String preferredVolumes = tableConf.get(TABLE_PREFERRED_VOLUMES);
    // fall back to global default scope, so setting only one default is necessary, rather than a separate default for TABLE scope than other scopes
    if (null == preferredVolumes || preferredVolumes.isEmpty()) {
        preferredVolumes = confFactory.getSystemConfiguration().get(DEFAULT_SCOPED_PREFERRED_VOLUMES);
    }
    // throw an error if volumes not specified or empty
    if (null == preferredVolumes || preferredVolumes.isEmpty()) {
        String msg = "Property " + TABLE_PREFERRED_VOLUMES + " or " + DEFAULT_SCOPED_PREFERRED_VOLUMES + " must be a subset of " + Arrays.toString(options) + " to use the " + getClass().getSimpleName();
        throw new VolumeChooserException(msg);
    }
    return parsePreferred(TABLE_PREFERRED_VOLUMES, preferredVolumes, options);
}
Also used : TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration)

Example 15 with TableConfiguration

use of org.apache.accumulo.server.conf.TableConfiguration in project accumulo by apache.

the class TabletServer method getMincEventDurability.

private Durability getMincEventDurability(KeyExtent extent) {
    TableConfiguration conf;
    if (extent.isMeta()) {
        conf = confFactory.getTableConfiguration(RootTable.ID);
    } else {
        conf = confFactory.getTableConfiguration(MetadataTable.ID);
    }
    Durability durability = DurabilityImpl.fromString(conf.get(Property.TABLE_DURABILITY));
    return durability;
}
Also used : Durability(org.apache.accumulo.core.client.Durability) TDurability(org.apache.accumulo.core.tabletserver.thrift.TDurability) TableConfiguration(org.apache.accumulo.server.conf.TableConfiguration)

Aggregations

TableConfiguration (org.apache.accumulo.server.conf.TableConfiguration)15 HashMap (java.util.HashMap)6 Test (org.junit.Test)6 ArrayList (java.util.ArrayList)5 Table (org.apache.accumulo.core.client.impl.Table)5 Property (org.apache.accumulo.core.conf.Property)5 KeyExtent (org.apache.accumulo.core.data.impl.KeyExtent)5 NamespaceConfiguration (org.apache.accumulo.server.conf.NamespaceConfiguration)5 List (java.util.List)3 Map (java.util.Map)3 SortedMap (java.util.SortedMap)3 Instance (org.apache.accumulo.core.client.Instance)3 AccumuloConfiguration (org.apache.accumulo.core.conf.AccumuloConfiguration)3 Key (org.apache.accumulo.core.data.Key)3 TabletServerStatus (org.apache.accumulo.core.master.thrift.TabletServerStatus)3 AccumuloServerContext (org.apache.accumulo.server.AccumuloServerContext)3 TServerInstance (org.apache.accumulo.server.master.state.TServerInstance)3 IOException (java.io.IOException)2 HashSet (java.util.HashSet)2 TreeMap (java.util.TreeMap)2