Search in sources :

Example 1 with ChangelogSSPIterator

use of org.apache.samza.system.ChangelogSSPIterator in project samza by apache.

the class TransactionalStateTaskRestoreManager method restore.

@Override
public CompletableFuture<Void> restore() {
    return CompletableFuture.runAsync(() -> {
        Map<String, RestoreOffsets> storesToRestore = storeActions.storesToRestore;
        for (Map.Entry<String, RestoreOffsets> entry : storesToRestore.entrySet()) {
            String storeName = entry.getKey();
            String endOffset = entry.getValue().endingOffset;
            SystemStream systemStream = storeChangelogs.get(storeName);
            SystemAdmin systemAdmin = systemAdmins.getSystemAdmin(systemStream.getSystem());
            SystemConsumer systemConsumer = storeConsumers.get(storeName);
            SystemStreamPartition changelogSSP = new SystemStreamPartition(systemStream, taskModel.getChangelogPartition());
            ChangelogSSPIterator changelogSSPIterator = new ChangelogSSPIterator(systemConsumer, changelogSSP, endOffset, systemAdmin, true, currentChangelogOffsets.get(changelogSSP).getNewestOffset());
            StorageEngine taskStore = storeEngines.get(storeName);
            LOG.info("Restoring store: {} for task: {}", storeName, taskModel.getTaskName());
            try {
                taskStore.restore(changelogSSPIterator);
            } catch (InterruptedException e) {
                String msg = String.format("Interrupted while restoring store: %s for task: %s", storeName, taskModel.getTaskName().getTaskName());
                // wrap in unchecked exception to throw from lambda
                throw new SamzaException(msg, e);
            }
        }
    }, restoreExecutor);
}
Also used : SystemConsumer(org.apache.samza.system.SystemConsumer) SystemStream(org.apache.samza.system.SystemStream) ChangelogSSPIterator(org.apache.samza.system.ChangelogSSPIterator) SamzaException(org.apache.samza.SamzaException) SystemAdmin(org.apache.samza.system.SystemAdmin) HashMap(java.util.HashMap) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) SystemStreamPartition(org.apache.samza.system.SystemStreamPartition)

Example 2 with ChangelogSSPIterator

use of org.apache.samza.system.ChangelogSSPIterator in project samza by apache.

the class NonTransactionalStateTaskRestoreManager method restore.

/**
 * Restore each store in taskStoresToRestore sequentially
 */
@Override
public CompletableFuture<Void> restore() {
    return CompletableFuture.runAsync(() -> {
        for (String storeName : taskStoresToRestore) {
            LOG.info("Restoring store: {} for task: {}", storeName, taskModel.getTaskName());
            SystemConsumer systemConsumer = storeConsumers.get(storeName);
            SystemStream systemStream = storeChangelogs.get(storeName);
            SystemAdmin systemAdmin = systemAdmins.getSystemAdmin(systemStream.getSystem());
            ChangelogSSPIterator changelogSSPIterator = new ChangelogSSPIterator(systemConsumer, new SystemStreamPartition(systemStream, taskModel.getChangelogPartition()), null, systemAdmin, false);
            try {
                taskStores.get(storeName).restore(changelogSSPIterator);
            } catch (InterruptedException e) {
                String msg = String.format("Interrupted while restoring store: %s for task: %s", storeName, taskModel.getTaskName().getTaskName());
                // wrap in unchecked exception to throw from lambda
                throw new SamzaException(msg, e);
            }
        }
    }, restoreExecutor);
}
Also used : SystemConsumer(org.apache.samza.system.SystemConsumer) SystemStream(org.apache.samza.system.SystemStream) SystemAdmin(org.apache.samza.system.SystemAdmin) ChangelogSSPIterator(org.apache.samza.system.ChangelogSSPIterator) SamzaException(org.apache.samza.SamzaException) SystemStreamPartition(org.apache.samza.system.SystemStreamPartition)

Aggregations

SamzaException (org.apache.samza.SamzaException)2 ChangelogSSPIterator (org.apache.samza.system.ChangelogSSPIterator)2 SystemAdmin (org.apache.samza.system.SystemAdmin)2 SystemConsumer (org.apache.samza.system.SystemConsumer)2 SystemStream (org.apache.samza.system.SystemStream)2 SystemStreamPartition (org.apache.samza.system.SystemStreamPartition)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1