Search in sources :

Example 91 with CConfiguration

use of co.cask.cdap.common.conf.CConfiguration in project cdap by caskdata.

the class AbstractSystemMetadataWriterTest method setup.

@BeforeClass
public static void setup() throws IOException {
    CConfiguration cConf = CConfiguration.create();
    Injector injector = Guice.createInjector(new ConfigModule(cConf), new AuthorizationTestModule(), new AuthorizationEnforcementModule().getInMemoryModules(), new AuthenticationContextModules().getMasterModule(), Modules.override(new DataSetsModules().getInMemoryModules()).with(new AbstractModule() {

        @Override
        protected void configure() {
            // Need the distributed metadata store.
            bind(MetadataStore.class).to(DefaultMetadataStore.class);
        }
    }), new LocationRuntimeModule().getInMemoryModules(), new TransactionInMemoryModule(), new SystemDatasetRuntimeModule().getInMemoryModules(), new NamespaceClientRuntimeModule().getInMemoryModules());
    txManager = injector.getInstance(TransactionManager.class);
    txManager.startAndWait();
    store = injector.getInstance(MetadataStore.class);
}
Also used : NamespaceClientRuntimeModule(co.cask.cdap.common.namespace.guice.NamespaceClientRuntimeModule) TransactionInMemoryModule(org.apache.tephra.runtime.TransactionInMemoryModule) ConfigModule(co.cask.cdap.common.guice.ConfigModule) AuthenticationContextModules(co.cask.cdap.security.auth.context.AuthenticationContextModules) DataSetsModules(co.cask.cdap.data.runtime.DataSetsModules) LocationRuntimeModule(co.cask.cdap.common.guice.LocationRuntimeModule) CConfiguration(co.cask.cdap.common.conf.CConfiguration) AuthorizationTestModule(co.cask.cdap.security.authorization.AuthorizationTestModule) AbstractModule(com.google.inject.AbstractModule) DefaultMetadataStore(co.cask.cdap.data2.metadata.store.DefaultMetadataStore) MetadataStore(co.cask.cdap.data2.metadata.store.MetadataStore) Injector(com.google.inject.Injector) TransactionManager(org.apache.tephra.TransactionManager) SystemDatasetRuntimeModule(co.cask.cdap.data.runtime.SystemDatasetRuntimeModule) AuthorizationEnforcementModule(co.cask.cdap.security.authorization.AuthorizationEnforcementModule) BeforeClass(org.junit.BeforeClass)

Example 92 with CConfiguration

use of co.cask.cdap.common.conf.CConfiguration in project cdap by caskdata.

the class ContextManager method createContext.

// this method is called by the mappers/reducers of jobs launched by Hive.
private static Context createContext(Configuration conf) throws IOException {
    // Create context needs to happen only when running in as a MapReduce job.
    // In other cases, ContextManager will be initialized using saveContext method.
    CConfiguration cConf = ConfigurationUtil.get(conf, Constants.Explore.CCONF_KEY, CConfCodec.INSTANCE);
    Configuration hConf = ConfigurationUtil.get(conf, Constants.Explore.HCONF_KEY, HConfCodec.INSTANCE);
    Injector injector = createInjector(cConf, hConf);
    ZKClientService zkClientService = injector.getInstance(ZKClientService.class);
    zkClientService.startAndWait();
    DatasetFramework datasetFramework = injector.getInstance(DatasetFramework.class);
    StreamAdmin streamAdmin = injector.getInstance(StreamAdmin.class);
    SystemDatasetInstantiatorFactory datasetInstantiatorFactory = injector.getInstance(SystemDatasetInstantiatorFactory.class);
    AuthenticationContext authenticationContext = injector.getInstance(AuthenticationContext.class);
    AuthorizationEnforcer authorizationEnforcer = injector.getInstance(AuthorizationEnforcer.class);
    return new Context(datasetFramework, streamAdmin, zkClientService, datasetInstantiatorFactory, authenticationContext, authorizationEnforcer);
}
Also used : DatasetFramework(co.cask.cdap.data2.dataset2.DatasetFramework) AuthenticationContext(co.cask.cdap.security.spi.authentication.AuthenticationContext) StreamAdmin(co.cask.cdap.data2.transaction.stream.StreamAdmin) SystemDatasetInstantiatorFactory(co.cask.cdap.data.dataset.SystemDatasetInstantiatorFactory) AuthenticationContext(co.cask.cdap.security.spi.authentication.AuthenticationContext) CConfiguration(co.cask.cdap.common.conf.CConfiguration) Configuration(org.apache.hadoop.conf.Configuration) ZKClientService(org.apache.twill.zookeeper.ZKClientService) Injector(com.google.inject.Injector) AuthorizationEnforcer(co.cask.cdap.security.spi.authorization.AuthorizationEnforcer) CConfiguration(co.cask.cdap.common.conf.CConfiguration)

Example 93 with CConfiguration

use of co.cask.cdap.common.conf.CConfiguration in project cdap by caskdata.

the class MessageTableRegionObserver method initializePruneState.

private void initializePruneState(RegionCoprocessorEnvironment env) {
    CConfiguration conf = topicMetadataCache.getCConfiguration();
    if (conf != null) {
        pruneEnable = conf.getBoolean(TxConstants.TransactionPruning.PRUNE_ENABLE, TxConstants.TransactionPruning.DEFAULT_PRUNE_ENABLE);
        if (Boolean.TRUE.equals(pruneEnable)) {
            String pruneTable = conf.get(TxConstants.TransactionPruning.PRUNE_STATE_TABLE, TxConstants.TransactionPruning.DEFAULT_PRUNE_STATE_TABLE);
            long pruneFlushInterval = TimeUnit.SECONDS.toMillis(conf.getLong(TxConstants.TransactionPruning.PRUNE_FLUSH_INTERVAL, TxConstants.TransactionPruning.DEFAULT_PRUNE_FLUSH_INTERVAL));
            compactionState = new CompactionState(env, TableName.valueOf(pruneTable), pruneFlushInterval);
            if (LOG.isDebugEnabled()) {
                LOG.debug(String.format("Automatic invalid list pruning is enabled for table %s. Compaction state " + "will be recorded in table %s", env.getRegionInfo().getTable().getNameWithNamespaceInclAsString(), pruneTable));
            }
        }
    }
}
Also used : CompactionState(org.apache.tephra.hbase.txprune.CompactionState) CConfiguration(co.cask.cdap.common.conf.CConfiguration)

Example 94 with CConfiguration

use of co.cask.cdap.common.conf.CConfiguration in project cdap by caskdata.

the class HBaseQueueRegionObserver method reloadPruneState.

private void reloadPruneState(RegionCoprocessorEnvironment env) {
    if (pruneEnable == null) {
        // If prune enable has never been initialized, try to do so now
        initializePruneState(env);
    } else {
        CConfiguration conf = configCache.getCConf();
        if (conf != null) {
            boolean newPruneEnable = conf.getBoolean(TxConstants.TransactionPruning.PRUNE_ENABLE, TxConstants.TransactionPruning.DEFAULT_PRUNE_ENABLE);
            if (newPruneEnable != pruneEnable) {
                // pruning enable has been changed, resetting prune state
                if (LOG.isDebugEnabled()) {
                    LOG.debug(String.format("Transaction Invalid List pruning feature is set to %s now for region %s.", newPruneEnable, env.getRegionInfo().getRegionNameAsString()));
                }
                resetPruneState();
                initializePruneState(env);
            }
        }
    }
}
Also used : CConfiguration(co.cask.cdap.common.conf.CConfiguration)

Example 95 with CConfiguration

use of co.cask.cdap.common.conf.CConfiguration in project cdap by caskdata.

the class HBaseQueueRegionObserver method initializePruneState.

private void initializePruneState(RegionCoprocessorEnvironment env) {
    CConfiguration conf = configCache.getCConf();
    if (conf != null) {
        pruneEnable = conf.getBoolean(TxConstants.TransactionPruning.PRUNE_ENABLE, TxConstants.TransactionPruning.DEFAULT_PRUNE_ENABLE);
        if (Boolean.TRUE.equals(pruneEnable)) {
            String pruneTable = conf.get(TxConstants.TransactionPruning.PRUNE_STATE_TABLE, TxConstants.TransactionPruning.DEFAULT_PRUNE_STATE_TABLE);
            long pruneFlushInterval = TimeUnit.SECONDS.toMillis(conf.getLong(TxConstants.TransactionPruning.PRUNE_FLUSH_INTERVAL, TxConstants.TransactionPruning.DEFAULT_PRUNE_FLUSH_INTERVAL));
            compactionState = new CompactionState(env, TableName.valueOf(pruneTable), pruneFlushInterval);
            if (LOG.isDebugEnabled()) {
                LOG.debug(String.format("Automatic invalid list pruning is enabled for table %s. Compaction state " + "will be recorded in table %s", env.getRegionInfo().getTable().getNameWithNamespaceInclAsString(), pruneTable));
            }
        }
    }
}
Also used : CompactionState(org.apache.tephra.hbase.txprune.CompactionState) CConfiguration(co.cask.cdap.common.conf.CConfiguration)

Aggregations

CConfiguration (co.cask.cdap.common.conf.CConfiguration)180 Test (org.junit.Test)52 BeforeClass (org.junit.BeforeClass)46 ConfigModule (co.cask.cdap.common.guice.ConfigModule)40 Injector (com.google.inject.Injector)35 Configuration (org.apache.hadoop.conf.Configuration)32 AbstractModule (com.google.inject.AbstractModule)31 AuthorizationEnforcementModule (co.cask.cdap.security.authorization.AuthorizationEnforcementModule)28 DataSetsModules (co.cask.cdap.data.runtime.DataSetsModules)27 DiscoveryRuntimeModule (co.cask.cdap.common.guice.DiscoveryRuntimeModule)26 AuthenticationContextModules (co.cask.cdap.security.auth.context.AuthenticationContextModules)26 AuthorizationTestModule (co.cask.cdap.security.authorization.AuthorizationTestModule)25 TransactionManager (org.apache.tephra.TransactionManager)23 NonCustomLocationUnitTestModule (co.cask.cdap.common.guice.NonCustomLocationUnitTestModule)22 UnsupportedUGIProvider (co.cask.cdap.security.impersonation.UnsupportedUGIProvider)19 Location (org.apache.twill.filesystem.Location)18 DefaultOwnerAdmin (co.cask.cdap.security.impersonation.DefaultOwnerAdmin)17 SystemDatasetRuntimeModule (co.cask.cdap.data.runtime.SystemDatasetRuntimeModule)16 File (java.io.File)16 ZKClientModule (co.cask.cdap.common.guice.ZKClientModule)14