Search in sources :

Example 11 with Data

use of org.jumpmind.symmetric.model.Data in project symmetric-ds by JumpMind.

the class DataGapRouteReaderTest method testTransactionalChannelMaxDataToRoute.

@Test
public void testTransactionalChannelMaxDataToRoute() throws Exception {
    nodeChannel.setBatchAlgorithm(TransactionalBatchAlgorithm.NAME);
    nodeChannel.setMaxDataToRoute(3);
    when(parameterService.getInt(ParameterConstants.ROUTING_PEEK_AHEAD_WINDOW)).thenReturn(100);
    List<DataGap> dataGaps = new ArrayList<DataGap>();
    dataGaps.add(new DataGap(0, Long.MAX_VALUE));
    List<Data> data = new ArrayList<Data>();
    data.add(new Data(1, null, null, null, TABLE1, null, null, null, TRAN1, null));
    data.add(new Data(2, null, null, null, TABLE1, null, null, null, TRAN1, null));
    data.add(new Data(3, null, null, null, TABLE1, null, null, null, TRAN2, null));
    data.add(new Data(4, null, null, null, TABLE1, null, null, null, TRAN2, null));
    data.add(new Data(5, null, null, null, TABLE1, null, null, null, TRAN1, null));
    data.add(new Data(6, null, null, null, TABLE1, null, null, null, TRAN1, null));
    ISqlRowMapper<Data> mapper = any();
    when(sqlTemplate.queryForCursor((String) any(), mapper, (Object[]) any(), (int[]) any())).thenReturn(new ListReadCursor(data));
    DataGapRouteReader dataGapRouteReader = buildReader(50, dataGaps);
    dataGapRouteReader.execute();
    BlockingQueue<Data> queue = dataGapRouteReader.getDataQueue();
    assertEquals(5, queue.size());
    Iterator<Data> iter = queue.iterator();
    int index = 0;
    long[] ids = { 1, 2, 5, 6, -1 };
    while (iter.hasNext()) {
        Data d = iter.next();
        assertEquals(ids[index], d.getDataId());
        index++;
    }
}
Also used : ArrayList(java.util.ArrayList) Data(org.jumpmind.symmetric.model.Data) TransformedData(org.jumpmind.symmetric.io.data.transform.TransformedData) DataGap(org.jumpmind.symmetric.model.DataGap) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 12 with Data

use of org.jumpmind.symmetric.model.Data in project symmetric-ds by JumpMind.

the class XmlPublisherDataRouterTest method testPublishUpdateAsXml.

@Test
public void testPublishUpdateAsXml() {
    Data data = new Data();
    data.setDataEventType(DataEventType.UPDATE);
    data.setRowData("2,updated data");
    data.setTriggerHistory(new TriggerHistory(TABLE_NAME, "ID", "ID,DATA"));
    data.setTableName(TABLE_NAME);
    router.routeToNodes(context, new DataMetaData(data, table, null, null), null, false, false, null);
    router.contextCommitted(context);
    assertEquals(UPDATE_XML.trim(), output.toString().trim());
}
Also used : TriggerHistory(org.jumpmind.symmetric.model.TriggerHistory) Data(org.jumpmind.symmetric.model.Data) DataMetaData(org.jumpmind.symmetric.model.DataMetaData) DataMetaData(org.jumpmind.symmetric.model.DataMetaData) Test(org.junit.Test)

Example 13 with Data

use of org.jumpmind.symmetric.model.Data in project symmetric-ds by JumpMind.

the class XmlPublisherDataRouterTest method testPublishInsertAsXml.

@Test
public void testPublishInsertAsXml() {
    Data data = new Data();
    data.setDataEventType(DataEventType.INSERT);
    data.setRowData("1,new inserted data");
    data.setTriggerHistory(new TriggerHistory(TABLE_NAME, "ID", "ID,DATA"));
    data.setTableName(TABLE_NAME);
    router.routeToNodes(context, new DataMetaData(data, table, null, null), null, false, false, null);
    router.contextCommitted(context);
    assertEquals(INSERT_XML.trim(), output.toString().trim());
}
Also used : TriggerHistory(org.jumpmind.symmetric.model.TriggerHistory) Data(org.jumpmind.symmetric.model.Data) DataMetaData(org.jumpmind.symmetric.model.DataMetaData) DataMetaData(org.jumpmind.symmetric.model.DataMetaData) Test(org.junit.Test)

Example 14 with Data

use of org.jumpmind.symmetric.model.Data in project symmetric-ds by JumpMind.

the class XmlPublisherDataRouterTest method testPublishDeleteAsXml.

@Test
public void testPublishDeleteAsXml() {
    Data data = new Data();
    data.setDataEventType(DataEventType.DELETE);
    data.setOldData("3,old deleted data");
    data.setTriggerHistory(new TriggerHistory(TABLE_NAME, "ID", "ID,DATA"));
    data.setTableName(TABLE_NAME);
    router.routeToNodes(context, new DataMetaData(data, table, null, null), null, false, false, null);
    router.contextCommitted(context);
    assertEquals(DELETE_XML.trim(), output.toString().trim());
}
Also used : TriggerHistory(org.jumpmind.symmetric.model.TriggerHistory) Data(org.jumpmind.symmetric.model.Data) DataMetaData(org.jumpmind.symmetric.model.DataMetaData) DataMetaData(org.jumpmind.symmetric.model.DataMetaData) Test(org.junit.Test)

Example 15 with Data

use of org.jumpmind.symmetric.model.Data in project symmetric-ds by JumpMind.

the class DataGapRouteReader method fillPeekAheadQueue.

protected boolean fillPeekAheadQueue(List<Data> peekAheadQueue, int peekAheadCount, ISqlReadCursor<Data> cursor) throws SQLException {
    boolean moreData = true;
    int dataCount = 0;
    long ts = System.currentTimeMillis();
    Data data = null;
    boolean isFirstRead = context.getStartDataId() == 0;
    while (reading && dataCount < peekAheadCount) {
        data = cursor.next();
        if (data != null) {
            if (process(data)) {
                peekAheadQueue.add(data);
                peekAheadSizeInBytes += data.getSizeInBytes();
                dataCount++;
                context.incrementStat(System.currentTimeMillis() - ts, ChannelRouterContext.STAT_READ_DATA_MS);
            } else {
                context.incrementStat(System.currentTimeMillis() - ts, ChannelRouterContext.STAT_REREAD_DATA_MS);
            }
            if (isFirstRead) {
                context.setStartDataId(data.getDataId());
                isFirstRead = false;
            }
            context.setEndDataId(data.getDataId());
            ts = System.currentTimeMillis();
        } else {
            moreData = false;
            break;
        }
    }
    context.incrementDataReadCount(dataCount);
    context.incrementPeekAheadFillCount(1);
    int size = peekAheadQueue.size();
    if (context.getMaxPeekAheadQueueSize() < size) {
        context.setMaxPeekAheadQueueSize(size);
    }
    return moreData && reading;
}
Also used : Data(org.jumpmind.symmetric.model.Data)

Aggregations

Data (org.jumpmind.symmetric.model.Data)42 TriggerHistory (org.jumpmind.symmetric.model.TriggerHistory)19 DataMetaData (org.jumpmind.symmetric.model.DataMetaData)18 Test (org.junit.Test)18 CsvData (org.jumpmind.symmetric.io.data.CsvData)16 Table (org.jumpmind.db.model.Table)14 ArrayList (java.util.ArrayList)12 Node (org.jumpmind.symmetric.model.Node)12 HashSet (java.util.HashSet)11 NodeChannel (org.jumpmind.symmetric.model.NodeChannel)10 Router (org.jumpmind.symmetric.model.Router)9 DataGap (org.jumpmind.symmetric.model.DataGap)7 TransformedData (org.jumpmind.symmetric.io.data.transform.TransformedData)6 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)6 Trigger (org.jumpmind.symmetric.model.Trigger)5 TriggerRouter (org.jumpmind.symmetric.model.TriggerRouter)5 File (java.io.File)3 ISqlTransaction (org.jumpmind.db.sql.ISqlTransaction)3 TransformTableNodeGroupLink (org.jumpmind.symmetric.service.impl.TransformService.TransformTableNodeGroupLink)3 Row (org.jumpmind.db.sql.Row)2