Search in sources :

Example 1 with WriteBatch

use of org.iq80.leveldb.WriteBatch in project hadoop by apache.

the class NMLeveldbStateStoreService method cleanupKeysWithPrefix.

private void cleanupKeysWithPrefix(String prefix) throws IOException {
    WriteBatch batch = null;
    LeveldbIterator iter = null;
    try {
        iter = new LeveldbIterator(db);
        try {
            batch = db.createWriteBatch();
            iter.seek(bytes(prefix));
            while (iter.hasNext()) {
                byte[] key = iter.next().getKey();
                String keyStr = asString(key);
                if (!keyStr.startsWith(prefix)) {
                    break;
                }
                batch.delete(key);
                if (LOG.isDebugEnabled()) {
                    LOG.debug("cleanup " + keyStr + " from leveldb");
                }
            }
            db.write(batch);
        } catch (DBException e) {
            throw new IOException(e);
        } finally {
            if (batch != null) {
                batch.close();
            }
        }
    } catch (DBException e) {
        throw new IOException(e);
    } finally {
        if (iter != null) {
            iter.close();
        }
    }
}
Also used : DBException(org.iq80.leveldb.DBException) LeveldbIterator(org.apache.hadoop.yarn.server.utils.LeveldbIterator) JniDBFactory.asString(org.fusesource.leveldbjni.JniDBFactory.asString) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch)

Example 2 with WriteBatch

use of org.iq80.leveldb.WriteBatch in project hadoop by apache.

the class NMLeveldbStateStoreService method storeContainerResourceChanged.

@Override
public void storeContainerResourceChanged(ContainerId containerId, int containerVersion, Resource capability) throws IOException {
    if (LOG.isDebugEnabled()) {
        LOG.debug("storeContainerResourceChanged: containerId=" + containerId + ", capability=" + capability);
    }
    String keyResChng = CONTAINERS_KEY_PREFIX + containerId.toString() + CONTAINER_RESOURCE_CHANGED_KEY_SUFFIX;
    String keyVersion = CONTAINERS_KEY_PREFIX + containerId.toString() + CONTAINER_VERSION_KEY_SUFFIX;
    try {
        WriteBatch batch = db.createWriteBatch();
        try {
            // New value will overwrite old values for the same key
            batch.put(bytes(keyResChng), ((ResourcePBImpl) capability).getProto().toByteArray());
            batch.put(bytes(keyVersion), bytes(Integer.toString(containerVersion)));
            db.write(batch);
        } finally {
            batch.close();
        }
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) JniDBFactory.asString(org.fusesource.leveldbjni.JniDBFactory.asString) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch) ResourcePBImpl(org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl)

Example 3 with WriteBatch

use of org.iq80.leveldb.WriteBatch in project hadoop by apache.

the class NMLeveldbStateStoreService method finishResourceLocalization.

@Override
public void finishResourceLocalization(String user, ApplicationId appId, LocalizedResourceProto proto) throws IOException {
    String localPath = proto.getLocalPath();
    String startedKey = getResourceStartedKey(user, appId, localPath);
    String completedKey = getResourceCompletedKey(user, appId, localPath);
    if (LOG.isDebugEnabled()) {
        LOG.debug("Storing localized resource to " + completedKey);
    }
    try {
        WriteBatch batch = db.createWriteBatch();
        try {
            batch.delete(bytes(startedKey));
            batch.put(bytes(completedKey), proto.toByteArray());
            db.write(batch);
        } finally {
            batch.close();
        }
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) JniDBFactory.asString(org.fusesource.leveldbjni.JniDBFactory.asString) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch)

Example 4 with WriteBatch

use of org.iq80.leveldb.WriteBatch in project hadoop by apache.

the class NMLeveldbStateStoreService method storeContainer.

@Override
public void storeContainer(ContainerId containerId, int containerVersion, StartContainerRequest startRequest) throws IOException {
    String idStr = containerId.toString();
    if (LOG.isDebugEnabled()) {
        LOG.debug("storeContainer: containerId= " + idStr + ", startRequest= " + startRequest);
    }
    String keyRequest = CONTAINERS_KEY_PREFIX + idStr + CONTAINER_REQUEST_KEY_SUFFIX;
    String keyVersion = getContainerVersionKey(idStr);
    try {
        WriteBatch batch = db.createWriteBatch();
        try {
            batch.put(bytes(keyRequest), ((StartContainerRequestPBImpl) startRequest).getProto().toByteArray());
            if (containerVersion != 0) {
                batch.put(bytes(keyVersion), bytes(Integer.toString(containerVersion)));
            }
            db.write(batch);
        } finally {
            batch.close();
        }
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) StartContainerRequestPBImpl(org.apache.hadoop.yarn.api.protocolrecords.impl.pb.StartContainerRequestPBImpl) JniDBFactory.asString(org.fusesource.leveldbjni.JniDBFactory.asString) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch)

Example 5 with WriteBatch

use of org.iq80.leveldb.WriteBatch in project hadoop by apache.

the class NMLeveldbStateStoreService method removeLocalizedResource.

@Override
public void removeLocalizedResource(String user, ApplicationId appId, Path localPath) throws IOException {
    String localPathStr = localPath.toString();
    String startedKey = getResourceStartedKey(user, appId, localPathStr);
    String completedKey = getResourceCompletedKey(user, appId, localPathStr);
    if (LOG.isDebugEnabled()) {
        LOG.debug("Removing local resource at " + localPathStr);
    }
    try {
        WriteBatch batch = db.createWriteBatch();
        try {
            batch.delete(bytes(startedKey));
            batch.delete(bytes(completedKey));
            db.write(batch);
        } finally {
            batch.close();
        }
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) JniDBFactory.asString(org.fusesource.leveldbjni.JniDBFactory.asString) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch)

Aggregations

WriteBatch (org.iq80.leveldb.WriteBatch)32 IOException (java.io.IOException)22 DBException (org.iq80.leveldb.DBException)18 JniDBFactory.asString (org.fusesource.leveldbjni.JniDBFactory.asString)12 Map (java.util.Map)10 DB (org.iq80.leveldb.DB)8 DBIterator (org.iq80.leveldb.DBIterator)5 ImmutableMap (com.google.common.collect.ImmutableMap)4 ImmutableSortedMap (com.google.common.collect.ImmutableSortedMap)4 NavigableMap (java.util.NavigableMap)4 RawMessageTableEntry (co.cask.cdap.messaging.store.RawMessageTableEntry)3 RollingWriteBatch (org.apache.hadoop.yarn.server.timeline.RollingLevelDB.RollingWriteBatch)3 ImmutableMessageTableEntry (co.cask.cdap.messaging.store.ImmutableMessageTableEntry)2 RawPayloadTableEntry (co.cask.cdap.messaging.store.RawPayloadTableEntry)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 DataOutputStream (java.io.DataOutputStream)2 HashMap (java.util.HashMap)2 Row (co.cask.cdap.api.dataset.table.Row)1 Scanner (co.cask.cdap.api.dataset.table.Scanner)1 AbstractMessageTable (co.cask.cdap.messaging.store.AbstractMessageTable)1