Search in sources :

Example 31 with DBException

use of org.iq80.leveldb.DBException in project hadoop by apache.

the class LeveldbTimelineStateStore method removeToken.

@Override
public void removeToken(TimelineDelegationTokenIdentifier tokenId) throws IOException {
    try {
        byte[] key = createTokenEntryKey(tokenId.getSequenceNumber());
        db.delete(key);
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) IOException(java.io.IOException)

Example 32 with DBException

use of org.iq80.leveldb.DBException in project hadoop by apache.

the class LeveldbTimelineStateStore method startStorage.

@Override
protected void startStorage() throws IOException {
    Options options = new Options();
    Path dbPath = new Path(getConfig().get(YarnConfiguration.TIMELINE_SERVICE_LEVELDB_STATE_STORE_PATH), DB_NAME);
    FileSystem localFS = null;
    try {
        localFS = FileSystem.getLocal(getConfig());
        if (!localFS.exists(dbPath)) {
            if (!localFS.mkdirs(dbPath)) {
                throw new IOException("Couldn't create directory for leveldb " + "timeline store " + dbPath);
            }
            localFS.setPermission(dbPath, LEVELDB_DIR_UMASK);
        }
    } finally {
        IOUtils.cleanup(LOG, localFS);
    }
    JniDBFactory factory = new JniDBFactory();
    try {
        options.createIfMissing(false);
        db = factory.open(new File(dbPath.toString()), options);
        LOG.info("Loading the existing database at th path: " + dbPath.toString());
        checkVersion();
    } catch (NativeDB.DBException e) {
        if (e.isNotFound() || e.getMessage().contains(" does not exist ")) {
            try {
                options.createIfMissing(true);
                db = factory.open(new File(dbPath.toString()), options);
                LOG.info("Creating a new database at th path: " + dbPath.toString());
                storeVersion(CURRENT_VERSION_INFO);
            } catch (DBException ex) {
                throw new IOException(ex);
            }
        } else {
            throw new IOException(e);
        }
    } catch (DBException e) {
        throw new IOException(e);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) Options(org.iq80.leveldb.Options) DBException(org.iq80.leveldb.DBException) FileSystem(org.apache.hadoop.fs.FileSystem) IOException(java.io.IOException) JniDBFactory(org.fusesource.leveldbjni.JniDBFactory) NativeDB(org.fusesource.leveldbjni.internal.NativeDB) File(java.io.File)

Example 33 with DBException

use of org.iq80.leveldb.DBException in project hadoop by apache.

the class TestLeveldbTimelineStore method deleteNextEntity.

private boolean deleteNextEntity(String entityType, byte[] ts) throws IOException, InterruptedException {
    LeveldbIterator iterator = null;
    LeveldbIterator pfIterator = null;
    try {
        iterator = ((LeveldbTimelineStore) store).getDbIterator(false);
        pfIterator = ((LeveldbTimelineStore) store).getDbIterator(false);
        return ((LeveldbTimelineStore) store).deleteNextEntity(entityType, ts, iterator, pfIterator, false);
    } catch (DBException e) {
        throw new IOException(e);
    } finally {
        IOUtils.cleanup(null, iterator, pfIterator);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) LeveldbIterator(org.apache.hadoop.yarn.server.utils.LeveldbIterator) IOException(java.io.IOException)

Example 34 with DBException

use of org.iq80.leveldb.DBException in project cdap by caskdata.

the class LevelDBMessageTable method rollback.

@Override
protected void rollback(byte[] startKey, byte[] stopKey, byte[] txWritePtr) throws IOException {
    WriteBatch writeBatch = levelDB.createWriteBatch();
    try (CloseableIterator<Map.Entry<byte[], byte[]>> rowIterator = new DBScanIterator(levelDB, startKey, stopKey)) {
        while (rowIterator.hasNext()) {
            Map.Entry<byte[], byte[]> rowValue = rowIterator.next();
            byte[] value = rowValue.getValue();
            Map<String, byte[]> columns = decodeValue(value);
            writeBatch.put(rowValue.getKey(), encodeValue(txWritePtr, columns.get(PAYLOAD_COL)));
        }
    }
    try {
        levelDB.write(writeBatch, WRITE_OPTIONS);
    } catch (DBException ex) {
        throw new IOException(ex);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) ImmutableMessageTableEntry(co.cask.cdap.messaging.store.ImmutableMessageTableEntry) RawMessageTableEntry(co.cask.cdap.messaging.store.RawMessageTableEntry) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch) HashMap(java.util.HashMap) Map(java.util.Map)

Example 35 with DBException

use of org.iq80.leveldb.DBException in project cdap by caskdata.

the class LevelDBMessageTable method persist.

@Override
protected void persist(Iterator<RawMessageTableEntry> entries) throws IOException {
    try (WriteBatch writeBatch = levelDB.createWriteBatch()) {
        while (entries.hasNext()) {
            RawMessageTableEntry entry = entries.next();
            byte[] rowKey = entry.getKey();
            // LevelDB doesn't make copies, and since we reuse RawMessageTableEntry object, we need to create copies.
            writeBatch.put(Arrays.copyOf(rowKey, rowKey.length), encodeValue(entry.getTxPtr(), entry.getPayload()));
        }
        levelDB.write(writeBatch, WRITE_OPTIONS);
    } catch (DBException ex) {
        throw new IOException(ex);
    }
}
Also used : DBException(org.iq80.leveldb.DBException) RawMessageTableEntry(co.cask.cdap.messaging.store.RawMessageTableEntry) IOException(java.io.IOException) WriteBatch(org.iq80.leveldb.WriteBatch)

Aggregations

IOException (java.io.IOException)70 DBException (org.iq80.leveldb.DBException)70 JniDBFactory.asString (org.fusesource.leveldbjni.JniDBFactory.asString)39 LeveldbIterator (org.apache.hadoop.yarn.server.utils.LeveldbIterator)19 WriteBatch (org.iq80.leveldb.WriteBatch)18 Path (org.apache.hadoop.fs.Path)8 ByteString (com.google.protobuf.ByteString)6 File (java.io.File)6 Map (java.util.Map)6 NativeDB (org.fusesource.leveldbjni.internal.NativeDB)6 Options (org.iq80.leveldb.Options)6 ByteArrayOutputStream (java.io.ByteArrayOutputStream)5 DataOutputStream (java.io.DataOutputStream)5 TopicMetadata (co.cask.cdap.messaging.TopicMetadata)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)4 HashMap (java.util.HashMap)4 TopicNotFoundException (co.cask.cdap.api.messaging.TopicNotFoundException)3 RawMessageTableEntry (co.cask.cdap.messaging.store.RawMessageTableEntry)3 TreeMap (java.util.TreeMap)3 ImmutableMessageTableEntry (co.cask.cdap.messaging.store.ImmutableMessageTableEntry)2