Search in sources :

Example 26 with ESSyncConfig

use of com.alibaba.otter.canal.client.adapter.es.core.config.ESSyncConfig in project canal by alibaba.

the class ConfigLoadTest method testLoad.

@Test
public void testLoad() {
    Map<String, ESSyncConfig> configMap = ESSyncConfigLoader.load(null);
    ESSyncConfig config = configMap.get("mytest_user.yml");
    config.validate();
    Assert.assertNotNull(config);
    Assert.assertEquals("defaultDS", config.getDataSourceKey());
    ESSyncConfig.ESMapping esMapping = config.getEsMapping();
    Assert.assertEquals("mytest_user", esMapping.get_index());
    Assert.assertEquals("_doc", esMapping.get_type());
    Assert.assertEquals("id", esMapping.get_id());
    Assert.assertNotNull(esMapping.getSql());
// Map<String, List<ESSyncConfig>> dbTableEsSyncConfig =
// ESSyncConfigLoader.getDbTableEsSyncConfig();
// Assert.assertFalse(dbTableEsSyncConfig.isEmpty());
}
Also used : ESSyncConfig(com.alibaba.otter.canal.client.adapter.es.core.config.ESSyncConfig) Test(org.junit.Test)

Example 27 with ESSyncConfig

use of com.alibaba.otter.canal.client.adapter.es.core.config.ESSyncConfig in project canal by alibaba.

the class LabelSyncJoinSubTest method test02.

/**
 * 子查询从表更新
 */
@Test
public void test02() {
    DataSource ds = DatasourceConfig.DATA_SOURCES.get("defaultDS");
    Common.sqlExe(ds, "update label set label='aa' where id=1");
    Dml dml = new Dml();
    dml.setDestination("example");
    dml.setTs(new Date().getTime());
    dml.setType("UPDATE");
    dml.setDatabase("mytest");
    dml.setTable("label");
    List<Map<String, Object>> dataList = new ArrayList<>();
    Map<String, Object> data = new LinkedHashMap<>();
    dataList.add(data);
    data.put("id", 1L);
    data.put("user_id", 1L);
    data.put("label", "aa");
    dml.setData(dataList);
    List<Map<String, Object>> oldList = new ArrayList<>();
    Map<String, Object> old = new LinkedHashMap<>();
    oldList.add(old);
    old.put("label", "a");
    dml.setOld(oldList);
    String database = dml.getDatabase();
    String table = dml.getTable();
    Map<String, ESSyncConfig> esSyncConfigs = esAdapter.getDbTableEsSyncConfig().get(database + "-" + table);
    esAdapter.getEsSyncService().sync(esSyncConfigs.values(), dml);
    GetResponse response = esAdapter.getEsConnection().getTransportClient().prepareGet("mytest_user", "_doc", "1").get();
    Assert.assertEquals("b;aa", response.getSource().get("_labels"));
}
Also used : Dml(com.alibaba.otter.canal.client.adapter.support.Dml) ArrayList(java.util.ArrayList) GetResponse(org.elasticsearch.action.get.GetResponse) Date(java.util.Date) DataSource(javax.sql.DataSource) LinkedHashMap(java.util.LinkedHashMap) ESSyncConfig(com.alibaba.otter.canal.client.adapter.es.core.config.ESSyncConfig) LinkedHashMap(java.util.LinkedHashMap) Map(java.util.Map) Test(org.junit.Test)

Aggregations

ESSyncConfig (com.alibaba.otter.canal.client.adapter.es.core.config.ESSyncConfig)27 LinkedHashMap (java.util.LinkedHashMap)21 Map (java.util.Map)21 ArrayList (java.util.ArrayList)19 Dml (com.alibaba.otter.canal.client.adapter.support.Dml)18 DataSource (javax.sql.DataSource)18 Test (org.junit.Test)18 Date (java.util.Date)17 GetResponse (org.elasticsearch.action.get.GetResponse)17 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)4 SearchResponse (org.elasticsearch.action.search.SearchResponse)4 SchemaItem (com.alibaba.otter.canal.client.adapter.es.core.config.SchemaItem)3 ESTemplate (com.alibaba.otter.canal.client.adapter.es.core.support.ESTemplate)3 DatasourceConfig (com.alibaba.otter.canal.client.adapter.support.DatasourceConfig)3 EtlResult (com.alibaba.otter.canal.client.adapter.support.EtlResult)3 List (java.util.List)3 StringUtils (org.apache.commons.lang.StringUtils)3 ESMapping (com.alibaba.otter.canal.client.adapter.es.core.config.ESSyncConfig.ESMapping)2 ColumnItem (com.alibaba.otter.canal.client.adapter.es.core.config.SchemaItem.ColumnItem)2 FieldItem (com.alibaba.otter.canal.client.adapter.es.core.config.SchemaItem.FieldItem)2