Examples with ClusterCapacity org.apache.hyracks.api.job.resource.ClusterCapacity used on opensource projects

Example 6 with ClusterCapacity

use of org.apache.hyracks.api.job.resource.ClusterCapacity in project asterixdb by apache.

the class ResourceUtils method getRequiredCompacity.

/**
     * Calculates the required cluster capacity from a given query plan, the computation locations,
     * the operator memory budgets, and frame size.
     *
     * @param plan,
     *            a given query plan.
     * @param computationLocations,
     *            the partitions for computation.
     * @param sortFrameLimit,
     *            the frame limit for one sorter partition.
     * @param groupFrameLimit,
     *            the frame limit for one group-by partition.
     * @param joinFrameLimit
     *            the frame limit for one joiner partition.
     * @param frameSize
     *            the frame size used in query execution.
     * @return the required cluster capacity for executing the query.
     * @throws AlgebricksException
     *             if the query plan is malformed.
     */
public static IClusterCapacity getRequiredCompacity(ILogicalPlan plan, AlgebricksAbsolutePartitionConstraint computationLocations, int sortFrameLimit, int groupFrameLimit, int joinFrameLimit, int frameSize) throws AlgebricksException {
    // Creates a cluster capacity visitor.
    IClusterCapacity clusterCapacity = new ClusterCapacity();
    RequiredCapacityVisitor visitor = new RequiredCapacityVisitor(computationLocations.getLocations().length, sortFrameLimit, groupFrameLimit, joinFrameLimit, frameSize, clusterCapacity);
    // There could be only one root operator for a top-level query plan.
    ILogicalOperator rootOp = plan.getRoots().get(0).getValue();
    rootOp.accept(visitor, null);
    return clusterCapacity;
}

Example 7 with ClusterCapacity

use of org.apache.hyracks.api.job.resource.ClusterCapacity in project asterixdb by apache.

the class RequiredCapacityVisitorTest method testUnPartitionedGroupBy.

@Test
public void testUnPartitionedGroupBy() throws AlgebricksException {
    IClusterCapacity clusterCapacity = new ClusterCapacity();
    RequiredCapacityVisitor visitor = makeComputationCapacityVisitor(PARALLELISM, clusterCapacity);
    // Constructs a parallel group-by query plan.
    GroupByOperator globalGby = makeGroupByOperator(AbstractLogicalOperator.ExecutionMode.UNPARTITIONED);
    ExchangeOperator exchange = new ExchangeOperator();
    exchange.setPhysicalOperator(new OneToOneExchangePOperator());
    exchange.setExecutionMode(AbstractLogicalOperator.ExecutionMode.UNPARTITIONED);
    GroupByOperator localGby = makeGroupByOperator(AbstractLogicalOperator.ExecutionMode.UNPARTITIONED);
    globalGby.getInputs().add(new MutableObject<>(exchange));
    exchange.getInputs().add(new MutableObject<>(localGby));
    // Verifies the calculated cluster capacity requirement for the test quer plan.
    globalGby.accept(visitor, null);
    Assert.assertTrue(clusterCapacity.getAggregatedCores() == 1);
    Assert.assertTrue(clusterCapacity.getAggregatedMemoryByteSize() == 2 * MEMORY_BUDGET + FRAME_SIZE);
}