Search in sources :

Example 6 with NodeExistsException

use of org.apache.zookeeper.KeeperException.NodeExistsException in project lucene-solr by apache.

the class SolrZkClient method makePath.

/**
   * Creates the path in ZooKeeper, creating each node as necessary.
   *
   * e.g. If <code>path=/solr/group/node</code> and none of the nodes, solr,
   * group, node exist, each will be created.
   * 
   * skipPathParts will force the call to fail if the first skipPathParts do not exist already.
   *
   * Note: retryOnConnLoss is only respected for the final node - nodes
   * before that are always retried on connection loss.
   */
public void makePath(String path, byte[] data, CreateMode createMode, Watcher watcher, boolean failOnExists, boolean retryOnConnLoss, int skipPathParts) throws KeeperException, InterruptedException {
    log.debug("makePath: {}", path);
    boolean retry = true;
    if (path.startsWith("/")) {
        path = path.substring(1, path.length());
    }
    String[] paths = path.split("/");
    StringBuilder sbPath = new StringBuilder();
    for (int i = 0; i < paths.length; i++) {
        String pathPiece = paths[i];
        sbPath.append("/" + pathPiece);
        if (i < skipPathParts) {
            continue;
        }
        byte[] bytes = null;
        final String currentPath = sbPath.toString();
        Object exists = exists(currentPath, watcher, retryOnConnLoss);
        if (exists == null || ((i == paths.length - 1) && failOnExists)) {
            CreateMode mode = CreateMode.PERSISTENT;
            if (i == paths.length - 1) {
                mode = createMode;
                bytes = data;
                if (!retryOnConnLoss)
                    retry = false;
            }
            try {
                if (retry) {
                    final CreateMode finalMode = mode;
                    final byte[] finalBytes = bytes;
                    zkCmdExecutor.retryOperation(new ZkOperation() {

                        @Override
                        public Object execute() throws KeeperException, InterruptedException {
                            keeper.create(currentPath, finalBytes, zkACLProvider.getACLsToAdd(currentPath), finalMode);
                            return null;
                        }
                    });
                } else {
                    keeper.create(currentPath, bytes, zkACLProvider.getACLsToAdd(currentPath), mode);
                }
            } catch (NodeExistsException e) {
                if (!failOnExists) {
                    // TODO: version ? for now, don't worry about race
                    setData(currentPath, data, -1, retryOnConnLoss);
                    // set new watch
                    exists(currentPath, watcher, retryOnConnLoss);
                    return;
                }
                // ignore unless it's the last node in the path
                if (i == paths.length - 1) {
                    throw e;
                }
            }
            if (i == paths.length - 1) {
                // set new watch
                exists(currentPath, watcher, retryOnConnLoss);
            }
        } else if (i == paths.length - 1) {
            // TODO: version ? for now, don't worry about race
            setData(currentPath, data, -1, retryOnConnLoss);
            // set new watch
            exists(currentPath, watcher, retryOnConnLoss);
        }
    }
}
Also used : NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) CreateMode(org.apache.zookeeper.CreateMode) KeeperException(org.apache.zookeeper.KeeperException)

Example 7 with NodeExistsException

use of org.apache.zookeeper.KeeperException.NodeExistsException in project zookeeper by apache.

the class DataTree method createNode.

/**
     * Add a new node to the DataTree.
     * @param path
     * 			  Path for the new node.
     * @param data
     *            Data to store in the node.
     * @param acl
     *            Node acls
     * @param ephemeralOwner
     *            the session id that owns this node. -1 indicates this is not
     *            an ephemeral node.
     * @param zxid
     *            Transaction ID
     * @param time
     * @param outputStat
     * 			  A Stat object to store Stat output results into.
     * @throws NodeExistsException 
     * @throws NoNodeException 
     * @throws KeeperException
     */
public void createNode(final String path, byte[] data, List<ACL> acl, long ephemeralOwner, int parentCVersion, long zxid, long time, Stat outputStat) throws KeeperException.NoNodeException, KeeperException.NodeExistsException {
    int lastSlash = path.lastIndexOf('/');
    String parentName = path.substring(0, lastSlash);
    String childName = path.substring(lastSlash + 1);
    StatPersisted stat = new StatPersisted();
    stat.setCtime(time);
    stat.setMtime(time);
    stat.setCzxid(zxid);
    stat.setMzxid(zxid);
    stat.setPzxid(zxid);
    stat.setVersion(0);
    stat.setAversion(0);
    stat.setEphemeralOwner(ephemeralOwner);
    DataNode parent = nodes.get(parentName);
    if (parent == null) {
        throw new KeeperException.NoNodeException();
    }
    synchronized (parent) {
        Set<String> children = parent.getChildren();
        if (children.contains(childName)) {
            throw new KeeperException.NodeExistsException();
        }
        if (parentCVersion == -1) {
            parentCVersion = parent.stat.getCversion();
            parentCVersion++;
        }
        parent.stat.setCversion(parentCVersion);
        parent.stat.setPzxid(zxid);
        Long longval = aclCache.convertAcls(acl);
        DataNode child = new DataNode(data, longval, stat);
        parent.addChild(childName);
        nodes.put(path, child);
        EphemeralType ephemeralType = EphemeralType.get(ephemeralOwner);
        if (ephemeralType == EphemeralType.CONTAINER) {
            containers.add(path);
        } else if (ephemeralType == EphemeralType.TTL) {
            ttls.add(path);
        } else if (ephemeralOwner != 0) {
            HashSet<String> list = ephemerals.get(ephemeralOwner);
            if (list == null) {
                list = new HashSet<String>();
                ephemerals.put(ephemeralOwner, list);
            }
            synchronized (list) {
                list.add(path);
            }
        }
        if (outputStat != null) {
            child.copyStat(outputStat);
        }
    }
    // now check if its one of the zookeeper node child
    if (parentName.startsWith(quotaZookeeper)) {
        // now check if its the limit node
        if (Quotas.limitNode.equals(childName)) {
            // this is the limit node
            // get the parent and add it to the trie
            pTrie.addPath(parentName.substring(quotaZookeeper.length()));
        }
        if (Quotas.statNode.equals(childName)) {
            updateQuotaForPath(parentName.substring(quotaZookeeper.length()));
        }
    }
    // also check to update the quotas for this node
    String lastPrefix = getMaxPrefixWithQuota(path);
    if (lastPrefix != null) {
        // ok we have some match and need to update
        updateCount(lastPrefix, 1);
        updateBytes(lastPrefix, data == null ? 0 : data.length);
    }
    dataWatches.triggerWatch(path, Event.EventType.NodeCreated);
    childWatches.triggerWatch(parentName.equals("") ? "/" : parentName, Event.EventType.NodeChildrenChanged);
}
Also used : NoNodeException(org.apache.zookeeper.KeeperException.NoNodeException) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) StatPersisted(org.apache.zookeeper.data.StatPersisted)

Example 8 with NodeExistsException

use of org.apache.zookeeper.KeeperException.NodeExistsException in project helios by spotify.

the class ZooKeeperMasterModel method addJob.

/**
   * Adds a job into the configuration.
   */
@Override
public void addJob(final Job job) throws JobExistsException {
    log.info("adding job: {}", job);
    final JobId id = job.getId();
    final UUID operationId = UUID.randomUUID();
    final String creationPath = Paths.configJobCreation(id, operationId);
    final ZooKeeperClient client = provider.get("addJob");
    try {
        try {
            client.ensurePath(Paths.historyJob(id));
            client.transaction(create(Paths.configJob(id), job), create(Paths.configJobRefShort(id), id), create(Paths.configJobHosts(id)), create(creationPath), // change down the tree. Effectively, make it that version == cVersion.
            set(Paths.configJobs(), UUID.randomUUID().toString().getBytes()));
        } catch (final NodeExistsException e) {
            if (client.exists(creationPath) != null) {
                // The job was created, we're done here
                return;
            }
            throw new JobExistsException(id.toString());
        }
    } catch (NoNodeException e) {
        throw new HeliosRuntimeException("adding job " + job + " failed due to missing ZK path: " + e.getPath(), e);
    } catch (final KeeperException e) {
        throw new HeliosRuntimeException("adding job " + job + " failed", e);
    }
}
Also used : NoNodeException(org.apache.zookeeper.KeeperException.NoNodeException) ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) UUID(java.util.UUID) JobId(com.spotify.helios.common.descriptors.JobId) KeeperException(org.apache.zookeeper.KeeperException)

Example 9 with NodeExistsException

use of org.apache.zookeeper.KeeperException.NodeExistsException in project helios by spotify.

the class ZooKeeperMasterModel method addDeploymentGroup.

/**
   * Create a deployment group.
   *
   * <p>If successful, the following ZK nodes will be created:
   * <ul>
   *   <li>/config/deployment-groups/[group-name]</li>
   *   <li>/status/deployment-groups/[group-name]</li>
   *   <li>/status/deployment-groups/[group-name]/hosts</li>
   * </ul>
   * These nodes are guaranteed to exist until the DG is removed.
   *
   * <p>If the operation fails no ZK nodes will be created. If any of the nodes above already exist
   * the operation will fail.
   *
   * @throws DeploymentGroupExistsException If a DG with the same name already exists.
   */
@Override
public void addDeploymentGroup(final DeploymentGroup deploymentGroup) throws DeploymentGroupExistsException {
    log.info("adding deployment-group: {}", deploymentGroup);
    final ZooKeeperClient client = provider.get("addDeploymentGroup");
    try {
        try {
            client.ensurePath(Paths.configDeploymentGroups());
            client.ensurePath(Paths.statusDeploymentGroups());
            client.transaction(create(Paths.configDeploymentGroup(deploymentGroup.getName()), deploymentGroup), create(Paths.statusDeploymentGroup(deploymentGroup.getName())), create(Paths.statusDeploymentGroupHosts(deploymentGroup.getName()), Json.asBytesUnchecked(emptyList())), create(Paths.statusDeploymentGroupRemovedHosts(deploymentGroup.getName()), Json.asBytesUnchecked(emptyList())));
        } catch (final NodeExistsException e) {
            throw new DeploymentGroupExistsException(deploymentGroup.getName());
        }
    } catch (final KeeperException e) {
        throw new HeliosRuntimeException("adding deployment-group " + deploymentGroup + " failed", e);
    }
}
Also used : ZooKeeperClient(com.spotify.helios.servicescommon.coordination.ZooKeeperClient) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) HeliosRuntimeException(com.spotify.helios.common.HeliosRuntimeException) KeeperException(org.apache.zookeeper.KeeperException)

Example 10 with NodeExistsException

use of org.apache.zookeeper.KeeperException.NodeExistsException in project helios by spotify.

the class TaskHistoryWriter method run.

@Override
public void run() {
    while (true) {
        final TaskStatusEvent item = getNext();
        if (item == null) {
            return;
        }
        final JobId jobId = item.getStatus().getJob().getId();
        final String historyPath = Paths.historyJobHostEventsTimestamp(jobId, hostname, item.getTimestamp());
        try {
            log.debug("writing queued item to zookeeper {} {}", item.getStatus().getJob().getId(), item.getTimestamp());
            client.ensurePath(historyPath, true);
            client.createAndSetData(historyPath, item.getStatus().toJsonBytes());
            // See if too many
            final List<String> events = client.getChildren(Paths.historyJobHostEvents(jobId, hostname));
            if (events.size() > MAX_NUMBER_STATUS_EVENTS_TO_RETAIN) {
                trimStatusEvents(events, jobId);
            }
        } catch (NodeExistsException e) {
            // Ahh, the two generals problem...  We handle by doing nothing since the thing
            // we wanted in, is in.
            log.debug("item we wanted in is already there");
        } catch (ConnectionLossException e) {
            log.warn("Connection lost while putting item into zookeeper, will retry");
            putBack(item);
            break;
        } catch (KeeperException e) {
            log.error("Error putting item into zookeeper, will retry", e);
            putBack(item);
            break;
        }
    }
}
Also used : TaskStatusEvent(com.spotify.helios.common.descriptors.TaskStatusEvent) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException) ConnectionLossException(org.apache.zookeeper.KeeperException.ConnectionLossException) JobId(com.spotify.helios.common.descriptors.JobId) KeeperException(org.apache.zookeeper.KeeperException)

Aggregations

NodeExistsException (org.apache.zookeeper.KeeperException.NodeExistsException)15 KeeperException (org.apache.zookeeper.KeeperException)8 NoNodeException (org.apache.zookeeper.KeeperException.NoNodeException)4 HeliosRuntimeException (com.spotify.helios.common.HeliosRuntimeException)3 JobId (com.spotify.helios.common.descriptors.JobId)3 ZooKeeperClient (com.spotify.helios.servicescommon.coordination.ZooKeeperClient)2 UUID (java.util.UUID)2 DrillRuntimeException (org.apache.drill.common.exceptions.DrillRuntimeException)2 VersionMismatchException (org.apache.drill.exec.exception.VersionMismatchException)2 CreateMode (org.apache.zookeeper.CreateMode)2 JCommander (com.beust.jcommander.JCommander)1 Job (com.spotify.helios.common.descriptors.Job)1 RolloutTask (com.spotify.helios.common.descriptors.RolloutTask)1 Task (com.spotify.helios.common.descriptors.Task)1 TaskStatusEvent (com.spotify.helios.common.descriptors.TaskStatusEvent)1 ZooKeeperOperation (com.spotify.helios.servicescommon.coordination.ZooKeeperOperation)1 ClusterData (com.yahoo.pulsar.common.policies.data.ClusterData)1 ZooKeeperClientFactory (com.yahoo.pulsar.zookeeper.ZooKeeperClientFactory)1 ZookeeperClientFactoryImpl (com.yahoo.pulsar.zookeeper.ZookeeperClientFactoryImpl)1 IOException (java.io.IOException)1