Examples with IntContainer com.carrotsearch.hppc.IntContainer used on opensource projects

Example 6 with IntContainer

use of com.carrotsearch.hppc.IntContainer in project graphhopper by graphhopper.

the class PrepareContractionHierarchies method contractNodesUsingHeuristicNodeOrdering.

private void contractNodesUsingHeuristicNodeOrdering() {
    StopWatch sw = new StopWatch().start();
    logger.info("Building initial queue of nodes to be contracted: {} nodes, {}", nodes, getMemInfo());
    // note that we update the priorities before preparing the node contractor. this does not make much sense,
    // but has always been like that and changing it would possibly require retuning the contraction parameters
    updatePrioritiesOfRemainingNodes();
    logger.info("Finished building queue, took: {}s, {}", sw.stop().getSeconds(), getMemInfo());
    final int initSize = sortedNodes.size();
    int level = 0;
    checkCounter = 0;
    final long logSize = params.getLogMessagesPercentage() == 0 ? Long.MAX_VALUE : Math.round(Math.max(10, initSize * (params.getLogMessagesPercentage() / 100d)));
    // specifies after how many contracted nodes the queue of remaining nodes is rebuilt. this takes time but the
    // more often we do this the more up-to-date the node priorities will be
    // todo: instead of using a fixed interval size maybe try adjusting it depending on the number of remaining
    // nodes ?
    final long periodicUpdatesCount = params.getPeriodicUpdatesPercentage() == 0 ? Long.MAX_VALUE : Math.round(Math.max(10, initSize * (params.getPeriodicUpdatesPercentage() / 100d)));
    int updateCounter = 0;
    // enable lazy updates for last x percentage of nodes. lazy updates make preparation slower but potentially
    // keep node priorities more up to date, possibly resulting in a better preparation.
    final long lastNodesLazyUpdates = Math.round(initSize * (params.getLastNodesLazyUpdatePercentage() / 100d));
    // according to paper "Polynomial-time Construction of Contraction Hierarchies for Multi-criteria Objectives" by Funke and Storandt
    // we don't need to wait for all nodes to be contracted
    final long nodesToAvoidContract = Math.round(initSize * ((100 - params.getNodesContractedPercentage()) / 100d));
    // Recompute priority of (the given percentage of) uncontracted neighbors. Doing neighbor updates takes additional
    // time during preparation but keeps node priorities more up to date. this potentially improves query time and
    // reduces number of shortcuts.
    final boolean neighborUpdate = (params.getNeighborUpdatePercentage() != 0);
    while (!sortedNodes.isEmpty()) {
        stopIfInterrupted();
        // periodically update priorities of ALL nodes
        if (checkCounter > 0 && checkCounter % periodicUpdatesCount == 0) {
            updatePrioritiesOfRemainingNodes();
            updateCounter++;
            if (sortedNodes.isEmpty())
                throw new IllegalStateException("Cannot prepare as no unprepared nodes where found. Called preparation twice?");
        }
        if (checkCounter % logSize == 0) {
            logHeuristicStats(updateCounter);
        }
        checkCounter++;
        int polledNode = sortedNodes.poll();
        if (!sortedNodes.isEmpty() && sortedNodes.size() < lastNodesLazyUpdates) {
            lazyUpdateSW.start();
            float priority = calculatePriority(polledNode);
            if (priority > sortedNodes.peekValue()) {
                // current node got more important => insert as new value and contract it later
                sortedNodes.push(polledNode, priority);
                lazyUpdateSW.stop();
                continue;
            }
            lazyUpdateSW.stop();
        }
        // contract node v!
        IntContainer neighbors = contractNode(polledNode, level);
        level++;
        if (sortedNodes.size() < nodesToAvoidContract)
            // skipped nodes are already set to maxLevel
            break;
        int neighborCount = 0;
        // there might be multiple edges going to the same neighbor nodes -> only calculate priority once per node
        for (IntCursor neighbor : neighbors) {
            if (neighborUpdate && (params.getMaxNeighborUpdates() < 0 || neighborCount < params.getMaxNeighborUpdates()) && rand.nextInt(100) < params.getNeighborUpdatePercentage()) {
                neighborCount++;
                neighborUpdateSW.start();
                float priority = calculatePriority(neighbor.value);
                sortedNodes.update(neighbor.value, priority);
                neighborUpdateSW.stop();
            }
        }
    }
    nodeContractor.finishContraction();
    logHeuristicStats(updateCounter);
    logger.info("new shortcuts: " + nf(nodeContractor.getAddedShortcutsCount()) + ", initSize:" + nf(initSize) + ", " + chConfig.getWeighting() + ", periodic:" + params.getPeriodicUpdatesPercentage() + ", lazy:" + params.getLastNodesLazyUpdatePercentage() + ", neighbor:" + params.getNeighborUpdatePercentage() + ", " + getTimesAsString() + ", lazy-overhead: " + (int) (100 * ((checkCounter / (double) initSize) - 1)) + "%" + ", " + Helper.getMemInfo());
    // Preparation works only once so we can release temporary data.
    // The preparation object itself has to be intact to create the algorithm.
    _close();
}

Example 7 with IntContainer

use of com.carrotsearch.hppc.IntContainer in project crate by crate.

the class NodeFetchResponseTest method setUpStreamBucketsAndStreamer.

@Before
public void setUpStreamBucketsAndStreamer() throws Exception {
    streamers = new IntObjectHashMap<>(1);
    streamers.put(1, new Streamer[] { DataTypes.BOOLEAN.streamer() });
    IntObjectHashMap<IntContainer> toFetch = new IntObjectHashMap<>();
    IntHashSet docIds = new IntHashSet(3);
    toFetch.put(1, docIds);
    StreamBucket.Builder builder = new StreamBucket.Builder(streamers.get(1), RamAccounting.NO_ACCOUNTING);
    builder.add(new RowN(new Object[] { true }));
    fetched = new IntObjectHashMap<>(1);
    fetched.put(1, builder.build());
}