Search in sources :

Example 6 with OTSRange

use of com.alibaba.datax.plugin.reader.otsreader.model.OTSRange in project DataX by alibaba.

the class RangeSplit method rangeSplitByPoint.

public static List<OTSRange> rangeSplitByPoint(TableMeta meta, RowPrimaryKey beginPK, RowPrimaryKey endPK, List<PrimaryKeyValue> splits) {
    List<OTSRange> results = new ArrayList<OTSRange>();
    int pkCount = meta.getPrimaryKey().size();
    String partName = Common.getPartitionKey(meta).getName();
    PrimaryKeyValue begin = beginPK.getPrimaryKey().get(partName);
    PrimaryKeyValue end = endPK.getPrimaryKey().get(partName);
    List<PrimaryKeyValue> newSplits = getSplitPoint(begin, end, splits);
    if (newSplits.isEmpty()) {
        return results;
    }
    for (int i = 0; i < newSplits.size() - 1; i++) {
        OTSRange item = new OTSRange(ParamChecker.checkInputPrimaryKeyAndGet(meta, getCompletePK(pkCount, newSplits.get(i))), ParamChecker.checkInputPrimaryKeyAndGet(meta, getCompletePK(pkCount, newSplits.get(i + 1))));
        results.add(item);
    }
    // replace first and last
    OTSRange first = results.get(0);
    OTSRange last = results.get(results.size() - 1);
    first.setBegin(beginPK);
    last.setEnd(endPK);
    return results;
}
Also used : OTSRange(com.alibaba.datax.plugin.reader.otsreader.model.OTSRange) ArrayList(java.util.ArrayList) PrimaryKeyValue(com.aliyun.openservices.ots.model.PrimaryKeyValue)

Aggregations

OTSRange (com.alibaba.datax.plugin.reader.otsreader.model.OTSRange)6 RowPrimaryKey (com.aliyun.openservices.ots.model.RowPrimaryKey)4 ArrayList (java.util.ArrayList)4 Direction (com.aliyun.openservices.ots.model.Direction)2 PrimaryKeyValue (com.aliyun.openservices.ots.model.PrimaryKeyValue)2 Configuration (com.alibaba.datax.common.util.Configuration)1 GetRangeCallable (com.alibaba.datax.plugin.reader.otsreader.callable.GetRangeCallable)1 OTSConf (com.alibaba.datax.plugin.reader.otsreader.model.OTSConf)1 OTSPrimaryKeyColumn (com.alibaba.datax.plugin.reader.otsreader.model.OTSPrimaryKeyColumn)1 DefaultNoRetry (com.alibaba.datax.plugin.reader.otsreader.utils.DefaultNoRetry)1 OTSClientAsync (com.aliyun.openservices.ots.OTSClientAsync)1 OTSServiceConfiguration (com.aliyun.openservices.ots.OTSServiceConfiguration)1 GetRangeResult (com.aliyun.openservices.ots.model.GetRangeResult)1