Search in sources :

Example 1 with TxMetricsCollector

use of org.apache.tephra.metrics.TxMetricsCollector in project phoenix by apache.

the class BaseTest method setupTxManager.

protected static void setupTxManager() throws SQLException, IOException {
    ConnectionInfo connInfo = ConnectionInfo.create(getUrl());
    zkClient = ZKClientServices.delegate(ZKClients.reWatchOnExpire(ZKClients.retryOnFailure(ZKClientService.Builder.of(connInfo.getZookeeperConnectionString()).setSessionTimeout(config.getInt(HConstants.ZK_SESSION_TIMEOUT, HConstants.DEFAULT_ZK_SESSION_TIMEOUT)).build(), RetryStrategies.exponentialDelay(500, 2000, TimeUnit.MILLISECONDS))));
    zkClient.startAndWait();
    DiscoveryService discovery = new ZKDiscoveryService(zkClient);
    txManager = new TransactionManager(config, new HDFSTransactionStateStorage(config, new SnapshotCodecProvider(config), new TxMetricsCollector()), new TxMetricsCollector());
    txService = new TransactionService(config, zkClient, discovery, Providers.of(txManager));
    txService.startAndWait();
}
Also used : ZKDiscoveryService(org.apache.twill.discovery.ZKDiscoveryService) HDFSTransactionStateStorage(org.apache.tephra.persist.HDFSTransactionStateStorage) TransactionService(org.apache.tephra.distributed.TransactionService) TransactionManager(org.apache.tephra.TransactionManager) TxMetricsCollector(org.apache.tephra.metrics.TxMetricsCollector) ConnectionInfo(org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.ConnectionInfo) DiscoveryService(org.apache.twill.discovery.DiscoveryService) ZKDiscoveryService(org.apache.twill.discovery.ZKDiscoveryService) SnapshotCodecProvider(org.apache.tephra.snapshot.SnapshotCodecProvider)

Example 2 with TxMetricsCollector

use of org.apache.tephra.metrics.TxMetricsCollector in project cdap by caskdata.

the class HBaseTableCoprocessorTestRun method setupBeforeClass.

@BeforeClass
public static void setupBeforeClass() throws Exception {
    hConf = HBASE_TEST_BASE.getConfiguration();
    hConf.set(HBaseTableUtil.CFG_HBASE_TABLE_COMPRESSION, HBaseTableUtil.CompressionType.NONE.name());
    cConf.set(Constants.CFG_LOCAL_DATA_DIR, TEMP_FOLDER.newFolder().getAbsolutePath());
    cConf.set(Constants.CFG_HDFS_NAMESPACE, cConf.get(Constants.CFG_LOCAL_DATA_DIR));
    cConf.set(Constants.CFG_HDFS_USER, System.getProperty("user.name"));
    // Reduce the metadata cache refresh frequency for unit tests
    cConf.set(Constants.MessagingSystem.COPROCESSOR_METADATA_CACHE_UPDATE_FREQUENCY_SECONDS, Integer.toString(METADATA_CACHE_EXPIRY));
    hBaseAdmin = HBASE_TEST_BASE.getHBaseAdmin();
    hBaseAdmin.getConfiguration().set(HBaseTableUtil.CFG_HBASE_TABLE_COMPRESSION, HBaseTableUtil.CompressionType.NONE.name());
    tableUtil = new HBaseTableUtilFactory(cConf).get();
    ddlExecutor = new HBaseDDLExecutorFactory(cConf, hConf).get();
    ddlExecutor.createNamespaceIfNotExists(tableUtil.getHBaseNamespace(NamespaceId.SYSTEM));
    LocationFactory locationFactory = getInjector().getInstance(LocationFactory.class);
    tableFactory = new HBaseTableFactory(cConf, hBaseAdmin.getConfiguration(), tableUtil, locationFactory);
    new ConfigurationWriter(hConf, cConf).write(ConfigurationReader.Type.DEFAULT, cConf);
    // write an initial transaction snapshot
    invalidList.addAll(ImmutableList.of(V[3], V[5], V[7]));
    TransactionSnapshot txSnapshot = TransactionSnapshot.copyFrom(System.currentTimeMillis(), V[6] - 1, V[7], invalidList, // this will set visibility upper bound to V[6]
    Maps.newTreeMap(ImmutableSortedMap.of(V[6], new TransactionManager.InProgressTx(V[6] - 1, Long.MAX_VALUE, TransactionManager.InProgressType.SHORT))), new HashMap<Long, TransactionManager.ChangeSet>(), new TreeMap<Long, TransactionManager.ChangeSet>());
    HDFSTransactionStateStorage tmpStorage = new HDFSTransactionStateStorage(hConf, new SnapshotCodecProvider(hConf), new TxMetricsCollector());
    tmpStorage.startAndWait();
    tmpStorage.writeSnapshot(txSnapshot);
    tmpStorage.stopAndWait();
}
Also used : ConfigurationWriter(co.cask.cdap.data2.util.hbase.ConfigurationWriter) TxMetricsCollector(org.apache.tephra.metrics.TxMetricsCollector) SnapshotCodecProvider(org.apache.tephra.snapshot.SnapshotCodecProvider) LocationFactory(org.apache.twill.filesystem.LocationFactory) HDFSTransactionStateStorage(org.apache.tephra.persist.HDFSTransactionStateStorage) TransactionSnapshot(org.apache.tephra.persist.TransactionSnapshot) TransactionManager(org.apache.tephra.TransactionManager) HBaseDDLExecutorFactory(co.cask.cdap.data2.util.hbase.HBaseDDLExecutorFactory) HBaseTableUtilFactory(co.cask.cdap.data2.util.hbase.HBaseTableUtilFactory) BeforeClass(org.junit.BeforeClass)

Example 3 with TxMetricsCollector

use of org.apache.tephra.metrics.TxMetricsCollector in project cdap by caskdata.

the class TransactionStateCache method tryInit.

/**
 * Try to initialize the Configuration and TransactionStateStorage instances.  Obtaining the Configuration may
 * fail until ReactorServiceMain has been started.
 */
private void tryInit() {
    try {
        Configuration conf = getSnapshotConfiguration();
        if (conf != null) {
            // Since this is only used for background loading of transaction snapshots, we use the no-op metrics collector,
            // as there are no relevant metrics to report
            this.storage = new HDFSTransactionStateStorage(conf, new SnapshotCodecProvider(conf), new TxMetricsCollector());
            this.storage.startAndWait();
            this.snapshotRefreshFrequency = conf.getLong(TxConstants.Manager.CFG_TX_SNAPSHOT_INTERVAL, TxConstants.Manager.DEFAULT_TX_SNAPSHOT_INTERVAL) * 1000;
            this.initialized = true;
        } else {
            LOG.info(prefixLog("Could not load configuration"));
        }
    } catch (Exception e) {
        LOG.info(prefixLog("Failed to initialize TransactionStateCache due to: "), e);
    }
}
Also used : HDFSTransactionStateStorage(org.apache.tephra.persist.HDFSTransactionStateStorage) Configuration(org.apache.hadoop.conf.Configuration) TxMetricsCollector(org.apache.tephra.metrics.TxMetricsCollector) SnapshotCodecProvider(org.apache.tephra.snapshot.SnapshotCodecProvider) IOException(java.io.IOException)

Example 4 with TxMetricsCollector

use of org.apache.tephra.metrics.TxMetricsCollector in project phoenix by apache.

the class TephraTransactionContext method setupTxManager.

@Override
public void setupTxManager(Configuration config, String url) throws SQLException {
    if (txService != null) {
        return;
    }
    ConnectionInfo connInfo = ConnectionInfo.create(url);
    zkClient = ZKClientServices.delegate(ZKClients.reWatchOnExpire(ZKClients.retryOnFailure(ZKClientService.Builder.of(connInfo.getZookeeperConnectionString()).setSessionTimeout(config.getInt(HConstants.ZK_SESSION_TIMEOUT, HConstants.DEFAULT_ZK_SESSION_TIMEOUT)).build(), RetryStrategies.exponentialDelay(500, 2000, TimeUnit.MILLISECONDS))));
    zkClient.startAndWait();
    DiscoveryService discovery = new ZKDiscoveryService(zkClient);
    txManager = new TransactionManager(config, new HDFSTransactionStateStorage(config, new SnapshotCodecProvider(config), new TxMetricsCollector()), new TxMetricsCollector());
    txService = new TransactionService(config, zkClient, discovery, Providers.of(txManager));
    txService.startAndWait();
}
Also used : ZKDiscoveryService(org.apache.twill.discovery.ZKDiscoveryService) HDFSTransactionStateStorage(org.apache.tephra.persist.HDFSTransactionStateStorage) TransactionService(org.apache.tephra.distributed.TransactionService) TransactionManager(org.apache.tephra.TransactionManager) TxMetricsCollector(org.apache.tephra.metrics.TxMetricsCollector) ConnectionInfo(org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.ConnectionInfo) ZKDiscoveryService(org.apache.twill.discovery.ZKDiscoveryService) DiscoveryService(org.apache.twill.discovery.DiscoveryService) SnapshotCodecProvider(org.apache.tephra.snapshot.SnapshotCodecProvider)

Aggregations

TxMetricsCollector (org.apache.tephra.metrics.TxMetricsCollector)4 HDFSTransactionStateStorage (org.apache.tephra.persist.HDFSTransactionStateStorage)4 SnapshotCodecProvider (org.apache.tephra.snapshot.SnapshotCodecProvider)4 TransactionManager (org.apache.tephra.TransactionManager)3 ConnectionInfo (org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.ConnectionInfo)2 TransactionService (org.apache.tephra.distributed.TransactionService)2 DiscoveryService (org.apache.twill.discovery.DiscoveryService)2 ZKDiscoveryService (org.apache.twill.discovery.ZKDiscoveryService)2 ConfigurationWriter (co.cask.cdap.data2.util.hbase.ConfigurationWriter)1 HBaseDDLExecutorFactory (co.cask.cdap.data2.util.hbase.HBaseDDLExecutorFactory)1 HBaseTableUtilFactory (co.cask.cdap.data2.util.hbase.HBaseTableUtilFactory)1 IOException (java.io.IOException)1 Configuration (org.apache.hadoop.conf.Configuration)1 TransactionSnapshot (org.apache.tephra.persist.TransactionSnapshot)1 LocationFactory (org.apache.twill.filesystem.LocationFactory)1 BeforeClass (org.junit.BeforeClass)1