Search in sources :

Example 11 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class ZkEphemeralStore method putIfAbsent.

@Override
public V putIfAbsent(final String key, final V value) {
    try {
        final InstanceSerializer<V> serializer = config.getSerializer();
        final byte[] bytes = serializer.serialize(value);
        final byte[] data = getClient().putIfAbsent(key, bytes);
        if (data == null) {
            return null;
        }
        return serializer.deserialize(data);
    } catch (final IOException e) {
        throw new DrillRuntimeException(String.format("unable to serialize value of type %s", value.getClass()), e);
    }
}
Also used : IOException(java.io.IOException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException)

Example 12 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class ZookeeperClient method putIfAbsent.

/**
   * Puts the given byte sequence into the given path if path is does not exist.
   *
   * @param path  target path
   * @param data  data to store
   * @return null if path was created, else data stored for the given path
   */
public byte[] putIfAbsent(final String path, final byte[] data) {
    Preconditions.checkNotNull(path, "path is required");
    Preconditions.checkNotNull(data, "data is required");
    final String target = PathUtils.join(root, path);
    try {
        try {
            curator.create().withMode(mode).forPath(target, data);
            getCache().rebuildNode(target);
            return null;
        } catch (NodeExistsException e) {
        // do nothing
        }
        return curator.getData().forPath(target);
    } catch (final Exception e) {
        throw new DrillRuntimeException("unable to put ", e);
    }
}
Also used : NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) KeeperException(org.apache.zookeeper.KeeperException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException)

Example 13 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class ZookeeperClient method put.

/**
   * Puts the given byte sequence into the given path.
   *
   * If path does not exists, this call creates it.
   *
   * If version holder is not null and path already exists, passes given version for comparison.
   * Zookeeper maintains stat structure that holds version number which increases each time znode data change is performed.
   * If we pass version that doesn't match the actual version of the data,
   * the update will fail {@link org.apache.zookeeper.KeeperException.BadVersionException}.
   * We catch such exception and re-throw it as {@link VersionMismatchException}.
   * Link to documentation - https://zookeeper.apache.org/doc/r3.2.2/zookeeperProgrammers.html#sc_zkDataModel_znodes
   *
   * @param path  target path
   * @param data  data to store
   * @param version version holder
   */
public void put(final String path, final byte[] data, DataChangeVersion version) {
    Preconditions.checkNotNull(path, "path is required");
    Preconditions.checkNotNull(data, "data is required");
    final String target = PathUtils.join(root, path);
    try {
        // we make a consistent read to ensure this call won't fail upon consecutive calls on the same path
        // before cache is updated
        boolean hasNode = hasPath(path, true);
        if (!hasNode) {
            try {
                curator.create().withMode(mode).forPath(target, data);
            } catch (NodeExistsException e) {
                // Handle race conditions since Drill is distributed and other
                // drillbits may have just created the node. This assumes that we do want to
                // override the new node. Makes sense here, because if the node had existed,
                // we'd have updated it.
                hasNode = true;
            }
        }
        if (hasNode) {
            if (version != null) {
                try {
                    curator.setData().withVersion(version.getVersion()).forPath(target, data);
                } catch (final KeeperException.BadVersionException e) {
                    throw new VersionMismatchException("Unable to put data. Version mismatch is detected.", version.getVersion(), e);
                }
            } else {
                curator.setData().forPath(target, data);
            }
        }
        getCache().rebuildNode(target);
    } catch (final VersionMismatchException e) {
        throw e;
    } catch (final Exception e) {
        throw new DrillRuntimeException("unable to put ", e);
    }
}
Also used : NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) KeeperException(org.apache.zookeeper.KeeperException) KeeperException(org.apache.zookeeper.KeeperException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException)

Example 14 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class CreateFunctionHandler method initRemoteRegistration.

/**
   * Instantiates remote registration. First gets remote function registry with version.
   * Version is used to ensure that we update the same registry we validated against.
   * Then validates against list of remote jars.
   * If validation is successful, first copies jars to registry area and starts updating remote function registry.
   * If during update {@link VersionMismatchException} was detected,
   * attempts to repeat remote registration process till retry attempts exceeds the limit.
   * If retry attempts number hits 0, throws exception that failed to update remote function registry.
   * In case of any error, if jars have been already copied to registry area, they will be deleted.
   *
   * @param functions list of functions present in jar
   * @param jarManager helper class for copying jars to registry area
   * @param remoteRegistry remote function registry
   * @throws IOException in case of problems with copying jars to registry area
   */
private void initRemoteRegistration(List<String> functions, JarManager jarManager, RemoteFunctionRegistry remoteRegistry) throws IOException {
    int retryAttempts = remoteRegistry.getRetryAttempts();
    boolean copyJars = true;
    try {
        while (retryAttempts >= 0) {
            DataChangeVersion version = new DataChangeVersion();
            List<Jar> remoteJars = remoteRegistry.getRegistry(version).getJarList();
            validateAgainstRemoteRegistry(remoteJars, jarManager.getBinaryName(), functions);
            if (copyJars) {
                jarManager.copyToRegistryArea();
                copyJars = false;
            }
            List<Jar> jars = Lists.newArrayList(remoteJars);
            jars.add(Jar.newBuilder().setName(jarManager.getBinaryName()).addAllFunctionSignature(functions).build());
            Registry updatedRegistry = Registry.newBuilder().addAllJar(jars).build();
            try {
                remoteRegistry.updateRegistry(updatedRegistry, version);
                return;
            } catch (VersionMismatchException ex) {
                logger.debug("Failed to update function registry during registration, version mismatch was detected.", ex);
                retryAttempts--;
            }
        }
        throw new DrillRuntimeException("Failed to update remote function registry. Exceeded retry attempts limit.");
    } catch (Exception e) {
        if (!copyJars) {
            jarManager.deleteQuietlyFromRegistryArea();
        }
        throw e;
    }
}
Also used : Jar(org.apache.drill.exec.proto.UserBitShared.Jar) FunctionImplementationRegistry(org.apache.drill.exec.expr.fn.FunctionImplementationRegistry) RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) Registry(org.apache.drill.exec.proto.UserBitShared.Registry) DataChangeVersion(org.apache.drill.exec.store.sys.store.DataChangeVersion) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) UserException(org.apache.drill.common.exceptions.UserException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) FunctionValidationException(org.apache.drill.exec.exception.FunctionValidationException) JarValidationException(org.apache.drill.exec.exception.JarValidationException) IOException(java.io.IOException) ForemanSetupException(org.apache.drill.exec.work.foreman.ForemanSetupException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException)

Example 15 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class DropFunctionHandler method unregister.

/**
   * Gets remote function registry with version.
   * Version is used to ensure that we update the same registry we removed jars from.
   * Looks for a jar to be deleted, if founds one,
   * attempts to update remote registry with list of jars, that excludes jar to be deleted.
   * If during update {@link VersionMismatchException} was detected,
   * attempts to repeat unregistration process till retry attempts exceeds the limit.
   * If retry attempts number hits 0, throws exception that failed to update remote function registry.
   *
   * @param jarName jar name
   * @param remoteFunctionRegistry remote function registry
   * @return jar that was unregistered, null otherwise
   */
private Jar unregister(String jarName, RemoteFunctionRegistry remoteFunctionRegistry) {
    int retryAttempts = remoteFunctionRegistry.getRetryAttempts();
    while (retryAttempts >= 0) {
        DataChangeVersion version = new DataChangeVersion();
        Registry registry = remoteFunctionRegistry.getRegistry(version);
        Jar jarToBeDeleted = null;
        List<Jar> jars = Lists.newArrayList();
        for (Jar j : registry.getJarList()) {
            if (j.getName().equals(jarName)) {
                jarToBeDeleted = j;
            } else {
                jars.add(j);
            }
        }
        if (jarToBeDeleted == null) {
            return null;
        }
        Registry updatedRegistry = Registry.newBuilder().addAllJar(jars).build();
        try {
            remoteFunctionRegistry.updateRegistry(updatedRegistry, version);
            return jarToBeDeleted;
        } catch (VersionMismatchException ex) {
            logger.debug("Failed to update function registry during unregistration, version mismatch was detected.", ex);
            retryAttempts--;
        }
    }
    throw new DrillRuntimeException("Failed to update remote function registry. Exceeded retry attempts limit.");
}
Also used : Jar(org.apache.drill.exec.proto.UserBitShared.Jar) Registry(org.apache.drill.exec.proto.UserBitShared.Registry) RemoteFunctionRegistry(org.apache.drill.exec.expr.fn.registry.RemoteFunctionRegistry) DataChangeVersion(org.apache.drill.exec.store.sys.store.DataChangeVersion) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException)

Aggregations

DrillRuntimeException (org.apache.drill.common.exceptions.DrillRuntimeException)69 IOException (java.io.IOException)32 VersionMismatchException (org.apache.drill.exec.exception.VersionMismatchException)9 Stopwatch (com.google.common.base.Stopwatch)7 ExecutionSetupException (org.apache.drill.common.exceptions.ExecutionSetupException)7 UserException (org.apache.drill.common.exceptions.UserException)6 KeeperException (org.apache.zookeeper.KeeperException)6 NodeExistsException (org.apache.zookeeper.KeeperException.NodeExistsException)6 NoSuchElementException (java.util.NoSuchElementException)5 Path (org.apache.hadoop.fs.Path)5 Bson (org.bson.conversions.Bson)4 Registry (org.apache.drill.exec.proto.UserBitShared.Registry)3 ValueVector (org.apache.drill.exec.vector.ValueVector)3 Admin (org.apache.hadoop.hbase.client.Admin)3 Document (org.bson.Document)3 File (java.io.File)2 UnsupportedEncodingException (java.io.UnsupportedEncodingException)2 URI (java.net.URI)2 RexNode (org.apache.calcite.rex.RexNode)2 LogicalExpression (org.apache.drill.common.expression.LogicalExpression)2