Search in sources :

Example 66 with HoodieException

use of org.apache.hudi.exception.HoodieException in project hudi by apache.

the class RocksDBDAO method dropColumnFamily.

/**
 * Note : Does not delete from underlying DB. Just closes the handle.
 *
 * @param columnFamilyName Column Family Name
 */
public void dropColumnFamily(String columnFamilyName) {
    ValidationUtils.checkArgument(!closed);
    managedDescriptorMap.computeIfPresent(columnFamilyName, (colFamilyName, descriptor) -> {
        ColumnFamilyHandle handle = managedHandlesMap.get(colFamilyName);
        try {
            getRocksDB().dropColumnFamily(handle);
            handle.close();
        } catch (RocksDBException e) {
            throw new HoodieException(e);
        }
        managedHandlesMap.remove(columnFamilyName);
        return null;
    });
}
Also used : RocksDBException(org.rocksdb.RocksDBException) HoodieException(org.apache.hudi.exception.HoodieException) ColumnFamilyHandle(org.rocksdb.ColumnFamilyHandle)

Example 67 with HoodieException

use of org.apache.hudi.exception.HoodieException in project hudi by apache.

the class RocksDBDAO method writeBatch.

/**
 * Perform a batch write operation.
 */
public void writeBatch(BatchHandler handler) {
    try (WriteBatch batch = new WriteBatch()) {
        handler.apply(batch);
        getRocksDB().write(new WriteOptions(), batch);
    } catch (RocksDBException re) {
        throw new HoodieException(re);
    }
}
Also used : WriteOptions(org.rocksdb.WriteOptions) RocksDBException(org.rocksdb.RocksDBException) HoodieException(org.apache.hudi.exception.HoodieException) WriteBatch(org.rocksdb.WriteBatch)

Example 68 with HoodieException

use of org.apache.hudi.exception.HoodieException in project hudi by apache.

the class RocksDBDAO method putInBatch.

/**
 * Helper to add put operation in batch.
 *
 * @param batch Batch Handle
 * @param columnFamilyName Column Family
 * @param key Key
 * @param value Payload
 * @param <T> Type of payload
 */
public <T extends Serializable> void putInBatch(WriteBatch batch, String columnFamilyName, String key, T value) {
    try {
        byte[] payload = serializePayload(value);
        batch.put(managedHandlesMap.get(columnFamilyName), key.getBytes(), payload);
    } catch (Exception e) {
        throw new HoodieException(e);
    }
}
Also used : HoodieException(org.apache.hudi.exception.HoodieException) HoodieException(org.apache.hudi.exception.HoodieException) RocksDBException(org.rocksdb.RocksDBException) IOException(java.io.IOException) HoodieIOException(org.apache.hudi.exception.HoodieIOException)

Example 69 with HoodieException

use of org.apache.hudi.exception.HoodieException in project hudi by apache.

the class BoundedInMemoryExecutor method execute.

/**
 * Main API to run both production and consumption.
 */
public E execute() {
    try {
        startProducers();
        Future<E> future = startConsumer();
        // Wait for consumer to be done
        return future.get();
    } catch (InterruptedException ie) {
        shutdownNow();
        Thread.currentThread().interrupt();
        throw new HoodieException(ie);
    } catch (Exception e) {
        throw new HoodieException(e);
    }
}
Also used : HoodieException(org.apache.hudi.exception.HoodieException) HoodieException(org.apache.hudi.exception.HoodieException)

Example 70 with HoodieException

use of org.apache.hudi.exception.HoodieException in project hudi by apache.

the class ReflectionUtils method getClass.

public static Class<?> getClass(String clazzName) {
    synchronized (CLAZZ_CACHE) {
        if (!CLAZZ_CACHE.containsKey(clazzName)) {
            try {
                Class<?> clazz = Class.forName(clazzName);
                CLAZZ_CACHE.put(clazzName, clazz);
            } catch (ClassNotFoundException e) {
                throw new HoodieException("Unable to load class", e);
            }
        }
    }
    return CLAZZ_CACHE.get(clazzName);
}
Also used : HoodieException(org.apache.hudi.exception.HoodieException)

Aggregations

HoodieException (org.apache.hudi.exception.HoodieException)171 IOException (java.io.IOException)87 Path (org.apache.hadoop.fs.Path)45 Schema (org.apache.avro.Schema)35 HoodieIOException (org.apache.hudi.exception.HoodieIOException)35 List (java.util.List)30 ArrayList (java.util.ArrayList)27 HoodieTableMetaClient (org.apache.hudi.common.table.HoodieTableMetaClient)23 Collectors (java.util.stream.Collectors)21 HoodieInstant (org.apache.hudi.common.table.timeline.HoodieInstant)19 Option (org.apache.hudi.common.util.Option)19 HoodieTimeline (org.apache.hudi.common.table.timeline.HoodieTimeline)18 Map (java.util.Map)16 HoodieRecord (org.apache.hudi.common.model.HoodieRecord)16 GenericRecord (org.apache.avro.generic.GenericRecord)15 Arrays (java.util.Arrays)14 HoodieLogFile (org.apache.hudi.common.model.HoodieLogFile)14 Logger (org.apache.log4j.Logger)14 FileStatus (org.apache.hadoop.fs.FileStatus)13 HoodieCommitMetadata (org.apache.hudi.common.model.HoodieCommitMetadata)13