Examples with UnsafeFixedWidthAggregationMap io.mycat.memory.unsafe.map.UnsafeFixedWidthAggregationMap used on opensource projects

Search in sources :

Example 1 with UnsafeFixedWidthAggregationMap

use of io.mycat.memory.unsafe.map.UnsafeFixedWidthAggregationMap in project Mycat_plus by coderczp.

the class UnsafeRowGrouper method aggregateRow.

private void aggregateRow(UnsafeRow key, UnsafeRow toRow, UnsafeRow newRow) throws UnsupportedEncodingException {
    if (mergCols == null) {
        return;
    }
    for (MergeCol merg : mergCols) {
        if (merg.mergeType != MergeCol.MERGE_AVG && merg.colMeta != null) {
            byte[] result = null;
            byte[] left = null;
            byte[] right = null;
            int type = merg.colMeta.colType;
            int index = merg.colMeta.colIndex;
            switch(type) {
                case ColMeta.COL_TYPE_INT:
                case ColMeta.COL_TYPE_LONG:
                case ColMeta.COL_TYPE_INT24:
                    left = BytesTools.int2Bytes(toRow.getInt(index));
                    right = BytesTools.int2Bytes(newRow.getInt(index));
                    break;
                case ColMeta.COL_TYPE_SHORT:
                    left = BytesTools.short2Bytes(toRow.getShort(index));
                    right = BytesTools.short2Bytes(newRow.getShort(index));
                    break;
                case ColMeta.COL_TYPE_LONGLONG:
                    left = BytesTools.long2Bytes(toRow.getLong(index));
                    right = BytesTools.long2Bytes(newRow.getLong(index));
                    break;
                case ColMeta.COL_TYPE_FLOAT:
                    left = BytesTools.float2Bytes(toRow.getFloat(index));
                    right = BytesTools.float2Bytes(newRow.getFloat(index));
                    break;
                case ColMeta.COL_TYPE_DOUBLE:
                    left = BytesTools.double2Bytes(toRow.getDouble(index));
                    right = BytesTools.double2Bytes(newRow.getDouble(index));
                    break;
                case ColMeta.COL_TYPE_NEWDECIMAL:
                    // left = BytesTools.double2Bytes(toRow.getDouble(index));
                    // right = BytesTools.double2Bytes(newRow.getDouble(index));
                    int scale = merg.colMeta.decimals;
                    BigDecimal decimalLeft = toRow.getDecimal(index, scale);
                    BigDecimal decimalRight = newRow.getDecimal(index, scale);
                    left = decimalLeft == null ? null : decimalLeft.toString().getBytes();
                    right = decimalRight == null ? null : decimalRight.toString().getBytes();
                    break;
                case ColMeta.COL_TYPE_DATE:
                case ColMeta.COL_TYPE_TIMSTAMP:
                case ColMeta.COL_TYPE_TIME:
                case ColMeta.COL_TYPE_YEAR:
                case ColMeta.COL_TYPE_DATETIME:
                case ColMeta.COL_TYPE_NEWDATE:
                case ColMeta.COL_TYPE_BIT:
                case ColMeta.COL_TYPE_VAR_STRING:
                case ColMeta.COL_TYPE_STRING:
                case ColMeta.COL_TYPE_ENUM:
                case ColMeta.COL_TYPE_SET:
                    left = toRow.getBinary(index);
                    right = newRow.getBinary(index);
                    break;
                default:
                    break;
            }
            result = mertFields(left, right, type, merg.mergeType);
            if (result != null) {
                switch(type) {
                    case ColMeta.COL_TYPE_BIT:
                        toRow.setByte(index, result[0]);
                    case ColMeta.COL_TYPE_INT:
                    case ColMeta.COL_TYPE_LONG:
                    case ColMeta.COL_TYPE_INT24:
                        toRow.setInt(index, BytesTools.getInt(result));
                        break;
                    case ColMeta.COL_TYPE_SHORT:
                        toRow.setShort(index, BytesTools.getShort(result));
                        break;
                    case ColMeta.COL_TYPE_LONGLONG:
                        toRow.setLong(index, BytesTools.getLong(result));
                        break;
                    case ColMeta.COL_TYPE_FLOAT:
                        toRow.setFloat(index, BytesTools.getFloat(result));
                        break;
                    case ColMeta.COL_TYPE_DOUBLE:
                        toRow.setDouble(index, BytesTools.getDouble(result));
                        break;
                    case ColMeta.COL_TYPE_NEWDECIMAL:
                        // toRow.setDouble(index,BytesTools.getDouble(result));
                        toRow.updateDecimal(index, new BigDecimal(new String(result)));
                        break;
                    /**
                     *TODO UnsafeFixedWidthAggregationMap 中存放
                     * UnsafeRow时，非数值类型的列不可更改其值，
                     * 为了统一处理聚合函数这块
                     * 做max或者min聚合时候，目前解决方法
                     * 先free原来 UnsafeFixedWidthAggregationMap对象。
                     * 然后重新创建一个UnsafeFixedWidthAggregationMap对象
                     * 然后存放最新的max或者min值作为下次比较。
                     */
                    case ColMeta.COL_TYPE_DATE:
                    case ColMeta.COL_TYPE_TIMSTAMP:
                    case ColMeta.COL_TYPE_TIME:
                    case ColMeta.COL_TYPE_YEAR:
                    case ColMeta.COL_TYPE_DATETIME:
                    case ColMeta.COL_TYPE_NEWDATE:
                    case ColMeta.COL_TYPE_VAR_STRING:
                    case ColMeta.COL_TYPE_STRING:
                    case ColMeta.COL_TYPE_ENUM:
                    case ColMeta.COL_TYPE_SET:
                        aggregationMap.free();
                        DataNodeMemoryManager dataNodeMemoryManager = new DataNodeMemoryManager(memoryManager, Thread.currentThread().getId());
                        aggregationMap = new UnsafeFixedWidthAggregationMap(emptyAggregationBuffer, aggBufferSchema, groupKeySchema, dataNodeMemoryManager, 1024, conf.getSizeAsBytes("mycat.buffer.pageSize", "32k"), false);
                        UnsafeRow unsafeRow = new UnsafeRow(toRow.numFields());
                        bufferHolder = new BufferHolder(unsafeRow, 0);
                        unsafeRowWriter = new UnsafeRowWriter(bufferHolder, toRow.numFields());
                        bufferHolder.reset();
                        for (int i = 0; i < toRow.numFields(); i++) {
                            if (!toRow.isNullAt(i) && i != index) {
                                unsafeRowWriter.write(i, toRow.getBinary(i));
                            } else if (!toRow.isNullAt(i) && i == index) {
                                unsafeRowWriter.write(i, result);
                            } else if (toRow.isNullAt(i)) {
                                unsafeRow.setNullAt(i);
                            }
                        }
                        unsafeRow.setTotalSize(bufferHolder.totalSize());
                        aggregationMap.put(key, unsafeRow);
                        break;
                    default:
                        break;
                }
            }
        }
    }
}
Also used : UnsafeFixedWidthAggregationMap(io.mycat.memory.unsafe.map.UnsafeFixedWidthAggregationMap) DataNodeMemoryManager(io.mycat.memory.unsafe.memory.mm.DataNodeMemoryManager) UnsafeRowWriter(io.mycat.memory.unsafe.row.UnsafeRowWriter) UnsafeRow(io.mycat.memory.unsafe.row.UnsafeRow) BigDecimal(java.math.BigDecimal) BufferHolder(io.mycat.memory.unsafe.row.BufferHolder)

Example 2 with UnsafeFixedWidthAggregationMap

use of io.mycat.memory.unsafe.map.UnsafeFixedWidthAggregationMap in project Mycat-Server by MyCATApache.

the class UnsafeRowGrouper method aggregateRow.

private void aggregateRow(UnsafeRow key, UnsafeRow toRow, UnsafeRow newRow) throws UnsupportedEncodingException {
    if (mergCols == null) {
        return;
    }
    for (MergeCol merg : mergCols) {
        if (merg.mergeType != MergeCol.MERGE_AVG && merg.colMeta != null) {
            byte[] result = null;
            byte[] left = null;
            byte[] right = null;
            int type = merg.colMeta.colType;
            int index = merg.colMeta.colIndex;
            switch(type) {
                case ColMeta.COL_TYPE_INT:
                case ColMeta.COL_TYPE_LONG:
                case ColMeta.COL_TYPE_INT24:
                    left = BytesTools.int2Bytes(toRow.getInt(index));
                    right = BytesTools.int2Bytes(newRow.getInt(index));
                    break;
                case ColMeta.COL_TYPE_SHORT:
                    left = BytesTools.short2Bytes(toRow.getShort(index));
                    right = BytesTools.short2Bytes(newRow.getShort(index));
                    break;
                case ColMeta.COL_TYPE_LONGLONG:
                    left = BytesTools.long2Bytes(toRow.getLong(index));
                    right = BytesTools.long2Bytes(newRow.getLong(index));
                    break;
                case ColMeta.COL_TYPE_FLOAT:
                    left = BytesTools.float2Bytes(toRow.getFloat(index));
                    right = BytesTools.float2Bytes(newRow.getFloat(index));
                    break;
                case ColMeta.COL_TYPE_DOUBLE:
                    left = BytesTools.double2Bytes(toRow.getDouble(index));
                    right = BytesTools.double2Bytes(newRow.getDouble(index));
                    break;
                case ColMeta.COL_TYPE_NEWDECIMAL:
                    // left = BytesTools.double2Bytes(toRow.getDouble(index));
                    // right = BytesTools.double2Bytes(newRow.getDouble(index));
                    int scale = merg.colMeta.decimals;
                    BigDecimal decimalLeft = toRow.getDecimal(index, scale);
                    BigDecimal decimalRight = newRow.getDecimal(index, scale);
                    left = decimalLeft == null ? null : decimalLeft.toString().getBytes();
                    right = decimalRight == null ? null : decimalRight.toString().getBytes();
                    break;
                case ColMeta.COL_TYPE_DATE:
                case ColMeta.COL_TYPE_TIMSTAMP:
                case ColMeta.COL_TYPE_TIME:
                case ColMeta.COL_TYPE_YEAR:
                case ColMeta.COL_TYPE_DATETIME:
                case ColMeta.COL_TYPE_NEWDATE:
                case ColMeta.COL_TYPE_BIT:
                case ColMeta.COL_TYPE_VAR_STRING:
                case ColMeta.COL_TYPE_STRING:
                case ColMeta.COL_TYPE_ENUM:
                case ColMeta.COL_TYPE_SET:
                    left = toRow.getBinary(index);
                    right = newRow.getBinary(index);
                    break;
                default:
                    break;
            }
            result = mertFields(left, right, type, merg.mergeType);
            if (result != null) {
                switch(type) {
                    case ColMeta.COL_TYPE_BIT:
                        toRow.setByte(index, result[0]);
                    case ColMeta.COL_TYPE_INT:
                    case ColMeta.COL_TYPE_LONG:
                    case ColMeta.COL_TYPE_INT24:
                        toRow.setInt(index, BytesTools.getInt(result));
                        break;
                    case ColMeta.COL_TYPE_SHORT:
                        toRow.setShort(index, BytesTools.getShort(result));
                        break;
                    case ColMeta.COL_TYPE_LONGLONG:
                        toRow.setLong(index, BytesTools.getLong(result));
                        break;
                    case ColMeta.COL_TYPE_FLOAT:
                        toRow.setFloat(index, BytesTools.getFloat(result));
                        break;
                    case ColMeta.COL_TYPE_DOUBLE:
                        toRow.setDouble(index, BytesTools.getDouble(result));
                        break;
                    case ColMeta.COL_TYPE_NEWDECIMAL:
                        // toRow.setDouble(index,BytesTools.getDouble(result));
                        toRow.updateDecimal(index, new BigDecimal(new String(result)));
                        break;
                    /**
                     *TODO UnsafeFixedWidthAggregationMap 中存放
                     * UnsafeRow时，非数值类型的列不可更改其值，
                     * 为了统一处理聚合函数这块
                     * 做max或者min聚合时候，目前解决方法
                     * 先free原来 UnsafeFixedWidthAggregationMap对象。
                     * 然后重新创建一个UnsafeFixedWidthAggregationMap对象
                     * 然后存放最新的max或者min值作为下次比较。
                     */
                    case ColMeta.COL_TYPE_DATE:
                    case ColMeta.COL_TYPE_TIMSTAMP:
                    case ColMeta.COL_TYPE_TIME:
                    case ColMeta.COL_TYPE_YEAR:
                    case ColMeta.COL_TYPE_DATETIME:
                    case ColMeta.COL_TYPE_NEWDATE:
                    case ColMeta.COL_TYPE_VAR_STRING:
                    case ColMeta.COL_TYPE_STRING:
                    case ColMeta.COL_TYPE_ENUM:
                    case ColMeta.COL_TYPE_SET:
                        aggregationMap.free();
                        DataNodeMemoryManager dataNodeMemoryManager = new DataNodeMemoryManager(memoryManager, Thread.currentThread().getId());
                        aggregationMap = new UnsafeFixedWidthAggregationMap(emptyAggregationBuffer, aggBufferSchema, groupKeySchema, dataNodeMemoryManager, 1024, conf.getSizeAsBytes("mycat.buffer.pageSize", "32k"), false);
                        UnsafeRow unsafeRow = new UnsafeRow(toRow.numFields());
                        bufferHolder = new BufferHolder(unsafeRow, 0);
                        unsafeRowWriter = new UnsafeRowWriter(bufferHolder, toRow.numFields());
                        bufferHolder.reset();
                        for (int i = 0; i < toRow.numFields(); i++) {
                            if (!toRow.isNullAt(i) && i != index) {
                                unsafeRowWriter.write(i, toRow.getBinary(i));
                            } else if (!toRow.isNullAt(i) && i == index) {
                                unsafeRowWriter.write(i, result);
                            } else if (toRow.isNullAt(i)) {
                                unsafeRow.setNullAt(i);
                            }
                        }
                        unsafeRow.setTotalSize(bufferHolder.totalSize());
                        aggregationMap.put(key, unsafeRow);
                        break;
                    default:
                        break;
                }
            }
        }
    }
}
Also used : UnsafeFixedWidthAggregationMap(io.mycat.memory.unsafe.map.UnsafeFixedWidthAggregationMap) DataNodeMemoryManager(io.mycat.memory.unsafe.memory.mm.DataNodeMemoryManager) UnsafeRowWriter(io.mycat.memory.unsafe.row.UnsafeRowWriter) UnsafeRow(io.mycat.memory.unsafe.row.UnsafeRow) BigDecimal(java.math.BigDecimal) BufferHolder(io.mycat.memory.unsafe.row.BufferHolder)

Aggregations

UnsafeFixedWidthAggregationMap (io.mycat.memory.unsafe.map.UnsafeFixedWidthAggregationMap)2 DataNodeMemoryManager (io.mycat.memory.unsafe.memory.mm.DataNodeMemoryManager)2 BufferHolder (io.mycat.memory.unsafe.row.BufferHolder)2 UnsafeRow (io.mycat.memory.unsafe.row.UnsafeRow)2 UnsafeRowWriter (io.mycat.memory.unsafe.row.UnsafeRowWriter)2 BigDecimal (java.math.BigDecimal)2
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial