Search in sources :

Example 6 with ProcessorStateException

use of org.apache.kafka.streams.errors.ProcessorStateException in project kafka by apache.

the class RocksDBStore method putAll.

@Override
public void putAll(List<KeyValue<K, V>> entries) {
    try (WriteBatch batch = new WriteBatch()) {
        for (KeyValue<K, V> entry : entries) {
            final byte[] rawKey = serdes.rawKey(entry.key);
            if (entry.value == null) {
                db.delete(rawKey);
            } else {
                final byte[] value = serdes.rawValue(entry.value);
                batch.put(rawKey, value);
            }
        }
        db.write(wOptions, batch);
    } catch (RocksDBException e) {
        throw new ProcessorStateException("Error while batch writing to store " + this.name, e);
    }
}
Also used : RocksDBException(org.rocksdb.RocksDBException) WriteBatch(org.rocksdb.WriteBatch) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException)

Example 7 with ProcessorStateException

use of org.apache.kafka.streams.errors.ProcessorStateException in project kafka by apache.

the class AbstractTask method initializeOffsetLimits.

protected void initializeOffsetLimits() {
    for (TopicPartition partition : partitions) {
        try {
            // TODO: batch API?
            OffsetAndMetadata metadata = consumer.committed(partition);
            stateMgr.putOffsetLimit(partition, metadata != null ? metadata.offset() : 0L);
        } catch (AuthorizationException e) {
            throw new ProcessorStateException(String.format("task [%s] AuthorizationException when initializing offsets for %s", id, partition), e);
        } catch (WakeupException e) {
            throw e;
        } catch (KafkaException e) {
            throw new ProcessorStateException(String.format("task [%s] Failed to initialize offsets for %s", id, partition), e);
        }
    }
}
Also used : AuthorizationException(org.apache.kafka.common.errors.AuthorizationException) TopicPartition(org.apache.kafka.common.TopicPartition) OffsetAndMetadata(org.apache.kafka.clients.consumer.OffsetAndMetadata) KafkaException(org.apache.kafka.common.KafkaException) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException) WakeupException(org.apache.kafka.common.errors.WakeupException)

Example 8 with ProcessorStateException

use of org.apache.kafka.streams.errors.ProcessorStateException in project kafka by apache.

the class ProcessorStateManager method flush.

@Override
public void flush(final InternalProcessorContext context) {
    if (!this.stores.isEmpty()) {
        log.debug("{} Flushing all stores registered in the state manager", logPrefix);
        for (StateStore store : this.stores.values()) {
            try {
                log.trace("{} Flushing store={}", logPrefix, store.name());
                store.flush();
            } catch (Exception e) {
                throw new ProcessorStateException(String.format("%s Failed to flush state store %s", logPrefix, store.name()), e);
            }
        }
    }
}
Also used : StateStore(org.apache.kafka.streams.processor.StateStore) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException) ProcessorStateException(org.apache.kafka.streams.errors.ProcessorStateException) IOException(java.io.IOException) StreamsException(org.apache.kafka.streams.errors.StreamsException) LockException(org.apache.kafka.streams.errors.LockException)

Aggregations

ProcessorStateException (org.apache.kafka.streams.errors.ProcessorStateException)8 IOException (java.io.IOException)3 LockException (org.apache.kafka.streams.errors.LockException)3 StreamsException (org.apache.kafka.streams.errors.StreamsException)3 StateStore (org.apache.kafka.streams.processor.StateStore)2 RocksDBException (org.rocksdb.RocksDBException)2 File (java.io.File)1 ArrayList (java.util.ArrayList)1 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)1 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)1 KafkaException (org.apache.kafka.common.KafkaException)1 TopicPartition (org.apache.kafka.common.TopicPartition)1 AuthorizationException (org.apache.kafka.common.errors.AuthorizationException)1 WakeupException (org.apache.kafka.common.errors.WakeupException)1 InvalidStateStoreException (org.apache.kafka.streams.errors.InvalidStateStoreException)1 StateRestoreCallback (org.apache.kafka.streams.processor.StateRestoreCallback)1 OffsetCheckpoint (org.apache.kafka.streams.state.internals.OffsetCheckpoint)1 NoOpReadOnlyStore (org.apache.kafka.test.NoOpReadOnlyStore)1 Test (org.junit.Test)1 WriteBatch (org.rocksdb.WriteBatch)1