Search in sources :

Example 1 with DistributedSemaphore

use of org.apache.drill.exec.coord.DistributedSemaphore in project drill by apache.

the class Foreman method acquireQuerySemaphore.

private void acquireQuerySemaphore(double totalCost) throws ForemanSetupException {
    final OptionManager optionManager = queryContext.getOptions();
    final long queueThreshold = optionManager.getOption(ExecConstants.QUEUE_THRESHOLD_SIZE);
    final long queueTimeout = optionManager.getOption(ExecConstants.QUEUE_TIMEOUT);
    final String queueName;
    try {
        final ClusterCoordinator clusterCoordinator = drillbitContext.getClusterCoordinator();
        final DistributedSemaphore distributedSemaphore;
        // get the appropriate semaphore
        if (totalCost > queueThreshold) {
            final int largeQueue = (int) optionManager.getOption(ExecConstants.LARGE_QUEUE_SIZE);
            distributedSemaphore = clusterCoordinator.getSemaphore("query.large", largeQueue);
            queueName = "large";
        } else {
            final int smallQueue = (int) optionManager.getOption(ExecConstants.SMALL_QUEUE_SIZE);
            distributedSemaphore = clusterCoordinator.getSemaphore("query.small", smallQueue);
            queueName = "small";
        }
        lease = distributedSemaphore.acquire(queueTimeout, TimeUnit.MILLISECONDS);
    } catch (final Exception e) {
        throw new ForemanSetupException("Unable to acquire slot for query.", e);
    }
    if (lease == null) {
        throw UserException.resourceError().message("Unable to acquire queue resources for query within timeout.  Timeout for %s queue was set at %d seconds.", queueName, queueTimeout / 1000).build(logger);
    }
}
Also used : DistributedSemaphore(org.apache.drill.exec.coord.DistributedSemaphore) ClusterCoordinator(org.apache.drill.exec.coord.ClusterCoordinator) OptionManager(org.apache.drill.exec.server.options.OptionManager) DrillbitEndpoint(org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint) UserException(org.apache.drill.common.exceptions.UserException) RpcException(org.apache.drill.exec.rpc.RpcException) InvalidProtocolBufferException(com.google.protobuf.InvalidProtocolBufferException) OptimizerException(org.apache.drill.exec.exception.OptimizerException) OutOfMemoryException(org.apache.drill.exec.exception.OutOfMemoryException) ExecutionSetupException(org.apache.drill.common.exceptions.ExecutionSetupException) IOException(java.io.IOException)

Aggregations

InvalidProtocolBufferException (com.google.protobuf.InvalidProtocolBufferException)1 IOException (java.io.IOException)1 ExecutionSetupException (org.apache.drill.common.exceptions.ExecutionSetupException)1 UserException (org.apache.drill.common.exceptions.UserException)1 ClusterCoordinator (org.apache.drill.exec.coord.ClusterCoordinator)1 DistributedSemaphore (org.apache.drill.exec.coord.DistributedSemaphore)1 OptimizerException (org.apache.drill.exec.exception.OptimizerException)1 OutOfMemoryException (org.apache.drill.exec.exception.OutOfMemoryException)1 DrillbitEndpoint (org.apache.drill.exec.proto.CoordinationProtos.DrillbitEndpoint)1 RpcException (org.apache.drill.exec.rpc.RpcException)1 OptionManager (org.apache.drill.exec.server.options.OptionManager)1