Search in sources :

Example 1 with HbaseTemplate

use of com.alibaba.otter.canal.client.adapter.hbase.support.HbaseTemplate in project canal by alibaba.

the class HbaseAdapter method init.

@Override
public void init(OuterAdapterConfig configuration, Properties envProperties) {
    try {
        this.envProperties = envProperties;
        Map<String, MappingConfig> hbaseMappingTmp = MappingConfigLoader.load(envProperties);
        // 过滤不匹配的key的配置
        hbaseMappingTmp.forEach((key, mappingConfig) -> {
            if ((mappingConfig.getOuterAdapterKey() == null && configuration.getKey() == null) || (mappingConfig.getOuterAdapterKey() != null && mappingConfig.getOuterAdapterKey().equalsIgnoreCase(configuration.getKey()))) {
                hbaseMapping.put(key, mappingConfig);
            }
        });
        for (Map.Entry<String, MappingConfig> entry : hbaseMapping.entrySet()) {
            String configName = entry.getKey();
            MappingConfig mappingConfig = entry.getValue();
            String k;
            if (envProperties != null && !"tcp".equalsIgnoreCase(envProperties.getProperty("canal.conf.mode"))) {
                k = StringUtils.trimToEmpty(mappingConfig.getDestination()) + "-" + StringUtils.trimToEmpty(mappingConfig.getGroupId()) + "_" + mappingConfig.getHbaseMapping().getDatabase() + "-" + mappingConfig.getHbaseMapping().getTable();
            } else {
                k = StringUtils.trimToEmpty(mappingConfig.getDestination()) + "_" + mappingConfig.getHbaseMapping().getDatabase() + "-" + mappingConfig.getHbaseMapping().getTable();
            }
            Map<String, MappingConfig> configMap = mappingConfigCache.computeIfAbsent(k, k1 -> new ConcurrentHashMap<>());
            configMap.put(configName, mappingConfig);
        }
        Map<String, String> properties = configuration.getProperties();
        Configuration hbaseConfig = HBaseConfiguration.create();
        properties.forEach(hbaseConfig::set);
        hbaseTemplate = new HbaseTemplate(hbaseConfig);
        hbaseSyncService = new HbaseSyncService(hbaseTemplate);
        configMonitor = new HbaseConfigMonitor();
        configMonitor.init(this, envProperties);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) HBaseConfiguration(org.apache.hadoop.hbase.HBaseConfiguration) HbaseTemplate(com.alibaba.otter.canal.client.adapter.hbase.support.HbaseTemplate) HbaseConfigMonitor(com.alibaba.otter.canal.client.adapter.hbase.monitor.HbaseConfigMonitor) IOException(java.io.IOException) MappingConfig(com.alibaba.otter.canal.client.adapter.hbase.config.MappingConfig) HbaseSyncService(com.alibaba.otter.canal.client.adapter.hbase.service.HbaseSyncService) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap)

Example 2 with HbaseTemplate

use of com.alibaba.otter.canal.client.adapter.hbase.support.HbaseTemplate in project canal by alibaba.

the class HBaseConnectionTest method test01.

@Test
public void test01() {
    Configuration hbaseConfig = HBaseConfiguration.create();
    hbaseConfig.set("hbase.zookeeper.quorum", "127.0.0.1");
    hbaseConfig.set("hbase.zookeeper.property.clientPort", "2181");
    hbaseConfig.set("zookeeper.znode.parent", "/hbase");
    HbaseTemplate hbaseTemplate = new HbaseTemplate(hbaseConfig);
    System.out.println(hbaseTemplate.tableExists("ttt"));
}
Also used : HBaseConfiguration(org.apache.hadoop.hbase.HBaseConfiguration) Configuration(org.apache.hadoop.conf.Configuration) HbaseTemplate(com.alibaba.otter.canal.client.adapter.hbase.support.HbaseTemplate) Test(org.junit.Test)

Aggregations

HbaseTemplate (com.alibaba.otter.canal.client.adapter.hbase.support.HbaseTemplate)2 Configuration (org.apache.hadoop.conf.Configuration)2 HBaseConfiguration (org.apache.hadoop.hbase.HBaseConfiguration)2 MappingConfig (com.alibaba.otter.canal.client.adapter.hbase.config.MappingConfig)1 HbaseConfigMonitor (com.alibaba.otter.canal.client.adapter.hbase.monitor.HbaseConfigMonitor)1 HbaseSyncService (com.alibaba.otter.canal.client.adapter.hbase.service.HbaseSyncService)1 IOException (java.io.IOException)1 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)1 Test (org.junit.Test)1