Search in sources :

Example 16 with DbBatch

use of com.alibaba.otter.shared.etl.model.DbBatch in project otter by alibaba.

the class FreedomExtractorTest method test_oracle.

@Test
public void test_oracle() {
    final Pipeline pipeline = new Pipeline();
    pipeline.setId(100L);
    int start = RandomUtils.nextInt();
    int count = 10;
    List<DataMediaPair> pairs = getDataMediaPairForOracle(start, count);
    pipeline.setPairs(pairs);
    new NonStrictExpectations() {

        {
            configClientService.findPipeline(100L);
            returns(pipeline);
        }
    };
    // 构造数据
    RowBatch rowBatch = new RowBatch();
    rowBatch.setIdentity(identity);
    for (int tableId = start; tableId < start + count; tableId++) {
        for (int i = start; i < start + count; i++) {
            EventData eventData = getEventData(tableId, i);
            eventData.setSchemaName("retl");
            eventData.setTableName("retl_buffer");
            rowBatch.merge(eventData);
        }
    }
    DbBatch dbBatch = new DbBatch(rowBatch);
    freedomExtractor.extract(dbBatch);
    want.collection(dbBatch.getRowBatch().getDatas()).sizeEq(count * count);
}
Also used : DataMediaPair(com.alibaba.otter.shared.common.model.config.data.DataMediaPair) RowBatch(com.alibaba.otter.shared.etl.model.RowBatch) EventData(com.alibaba.otter.shared.etl.model.EventData) DbBatch(com.alibaba.otter.shared.etl.model.DbBatch) Pipeline(com.alibaba.otter.shared.common.model.config.pipeline.Pipeline) Test(org.testng.annotations.Test) BaseDbTest(com.alibaba.otter.node.etl.BaseDbTest)

Aggregations

DbBatch (com.alibaba.otter.shared.etl.model.DbBatch)16 RowBatch (com.alibaba.otter.shared.etl.model.RowBatch)12 Test (org.testng.annotations.Test)9 EventData (com.alibaba.otter.shared.etl.model.EventData)7 Identity (com.alibaba.otter.shared.etl.model.Identity)7 Pipeline (com.alibaba.otter.shared.common.model.config.pipeline.Pipeline)6 BaseOtterTest (com.alibaba.otter.node.etl.BaseOtterTest)5 PipeKey (com.alibaba.otter.node.etl.common.pipe.PipeKey)5 BaseDbTest (com.alibaba.otter.node.etl.BaseDbTest)4 AggregationItem (com.alibaba.otter.node.etl.common.jmx.StageAggregation.AggregationItem)4 EtlEventData (com.alibaba.otter.shared.arbitrate.model.EtlEventData)4 DataMediaPair (com.alibaba.otter.shared.common.model.config.data.DataMediaPair)4 FileBatch (com.alibaba.otter.shared.etl.model.FileBatch)4 MemoryPipeKey (com.alibaba.otter.node.etl.common.pipe.impl.memory.MemoryPipeKey)3 RpcPipeKey (com.alibaba.otter.node.etl.common.pipe.impl.rpc.RpcPipeKey)3 SetlFuture (com.alibaba.otter.node.etl.extract.SetlFuture)3 File (java.io.File)3 List (java.util.List)3 NodeCommmunicationClient (com.alibaba.otter.node.common.communication.NodeCommmunicationClient)2 PipeException (com.alibaba.otter.node.etl.common.pipe.exception.PipeException)2