Search in sources :

Example 51 with RetryPolicy

use of net.jodah.failsafe.RetryPolicy in project open-kilda by telstra.

the class BaseResourceAllocationAction method allocateFlowResources.

@SneakyThrows
protected FlowResources allocateFlowResources(Flow flow, PathId forwardPathId, PathId reversePathId) throws ResourceAllocationException {
    RetryPolicy<FlowResources> resourceAllocationRetryPolicy = transactionManager.<FlowResources>getDefaultRetryPolicy().handle(ResourceAllocationException.class).handle(ConstraintViolationException.class).onRetry(e -> log.warn("Failure in resource allocation. Retrying #{}...", e.getAttemptCount(), e.getLastFailure())).onRetriesExceeded(e -> log.warn("Failure in resource allocation. No more retries", e.getFailure())).withMaxRetries(resourceAllocationRetriesLimit);
    FlowResources flowResources = transactionManager.doInTransaction(resourceAllocationRetryPolicy, () -> resourcesManager.allocateFlowResources(flow, forwardPathId, reversePathId));
    log.debug("Resources have been allocated: {}", flowResources);
    return flowResources;
}
Also used : FlowPath(org.openkilda.model.FlowPath) HistoryMapper(org.openkilda.wfm.share.mappers.HistoryMapper) SneakyThrows(lombok.SneakyThrows) ResourceAllocationException(org.openkilda.wfm.share.flow.resources.ResourceAllocationException) FlowPathStatus(org.openkilda.model.FlowPathStatus) UnroutableFlowException(org.openkilda.pce.exception.UnroutableFlowException) FlowPathSwappingFsm(org.openkilda.wfm.topology.flowhs.fsm.common.FlowPathSwappingFsm) FailsafeException(net.jodah.failsafe.FailsafeException) Flow(org.openkilda.model.Flow) Duration(java.time.Duration) Map(java.util.Map) FlowPathPair(org.openkilda.wfm.topology.flow.model.FlowPathPair) DumpType(org.openkilda.wfm.share.history.model.FlowDumpData.DumpType) Path(org.openkilda.pce.Path) SwitchProperties(org.openkilda.model.SwitchProperties) IslEndpoints(org.openkilda.persistence.repositories.IslRepository.IslEndpoints) Predicate(java.util.function.Predicate) Collection(java.util.Collection) RetryPolicy(net.jodah.failsafe.RetryPolicy) PersistenceException(org.openkilda.persistence.exceptions.PersistenceException) FlowResources(org.openkilda.wfm.share.flow.resources.FlowResources) String.format(java.lang.String.format) Objects(java.util.Objects) SwitchPropertiesRepository(org.openkilda.persistence.repositories.SwitchPropertiesRepository) Slf4j(lombok.extern.slf4j.Slf4j) List(java.util.List) FlowDumpData(org.openkilda.wfm.share.history.model.FlowDumpData) PathComputer(org.openkilda.pce.PathComputer) Optional(java.util.Optional) ConstraintViolationException(org.openkilda.persistence.exceptions.ConstraintViolationException) SwitchRepository(org.openkilda.persistence.repositories.SwitchRepository) FlowSegmentCookieBuilder(org.openkilda.model.cookie.FlowSegmentCookie.FlowSegmentCookieBuilder) Message(org.openkilda.messaging.Message) PathSegment(org.openkilda.model.PathSegment) KildaConfigurationRepository(org.openkilda.persistence.repositories.KildaConfigurationRepository) HashMap(java.util.HashMap) Supplier(java.util.function.Supplier) RecoverableException(org.openkilda.pce.exception.RecoverableException) ArrayList(java.util.ArrayList) FlowPathDirection(org.openkilda.model.FlowPathDirection) PathSegmentRepository(org.openkilda.persistence.repositories.PathSegmentRepository) IslRepository(org.openkilda.persistence.repositories.IslRepository) Suppliers(com.google.common.base.Suppliers) FlowSegmentCookie(org.openkilda.model.cookie.FlowSegmentCookie) PersistenceManager(org.openkilda.persistence.PersistenceManager) PathId(org.openkilda.model.PathId) DetectConnectedDevices(org.openkilda.model.DetectConnectedDevices) FlowOperationsDashboardLogger(org.openkilda.wfm.share.logger.FlowOperationsDashboardLogger) ErrorType(org.openkilda.messaging.error.ErrorType) IslStatus(org.openkilda.model.IslStatus) Failsafe(net.jodah.failsafe.Failsafe) RepositoryFactory(org.openkilda.persistence.repositories.RepositoryFactory) FlowPathBuilder(org.openkilda.wfm.topology.flowhs.service.FlowPathBuilder) SwitchId(org.openkilda.model.SwitchId) FlowResourcesManager(org.openkilda.wfm.share.flow.resources.FlowResourcesManager) GetPathsResult(org.openkilda.pce.GetPathsResult) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Isl(org.openkilda.model.Isl) LazyMap(org.apache.commons.collections4.map.LazyMap) FlowResources(org.openkilda.wfm.share.flow.resources.FlowResources) ConstraintViolationException(org.openkilda.persistence.exceptions.ConstraintViolationException) SneakyThrows(lombok.SneakyThrows)

Example 52 with RetryPolicy

use of net.jodah.failsafe.RetryPolicy in project open-kilda by telstra.

the class BaseResourceAllocationAction method allocatePathPair.

@SneakyThrows
protected GetPathsResult allocatePathPair(Flow flow, PathId newForwardPathId, PathId newReversePathId, boolean forceToIgnoreBandwidth, List<PathId> pathsToReuseBandwidth, FlowPathPair oldPaths, boolean allowOldPaths, String sharedBandwidthGroupId, Predicate<GetPathsResult> whetherCreatePathSegments) throws RecoverableException, UnroutableFlowException, ResourceAllocationException {
    // Lazy initialisable map with reused bandwidth...
    Supplier<Map<IslEndpoints, Long>> reuseBandwidthPerIsl = Suppliers.memoize(() -> {
        Map<IslEndpoints, Long> result = new HashMap<>();
        if (pathsToReuseBandwidth != null && !pathsToReuseBandwidth.isEmpty()) {
            pathsToReuseBandwidth.stream().map(pathId -> flow.getPath(pathId).orElse(flowPathRepository.findById(pathId).orElse(null))).filter(Objects::nonNull).flatMap(path -> path.getSegments().stream()).forEach(segment -> {
                IslEndpoints isl = new IslEndpoints(segment.getSrcSwitchId().toString(), segment.getSrcPort(), segment.getDestSwitchId().toString(), segment.getDestPort());
                result.put(isl, result.getOrDefault(isl, 0L) + segment.getBandwidth());
            });
        }
        return result;
    });
    RetryPolicy<GetPathsResult> pathAllocationRetryPolicy = new RetryPolicy<GetPathsResult>().handle(RecoverableException.class).handle(ResourceAllocationException.class).handle(UnroutableFlowException.class).handle(PersistenceException.class).onRetry(e -> log.warn("Failure in path allocation. Retrying #{}...", e.getAttemptCount(), e.getLastFailure())).onRetriesExceeded(e -> log.warn("Failure in path allocation. No more retries", e.getFailure())).withMaxRetries(pathAllocationRetriesLimit);
    if (pathAllocationRetryDelay > 0) {
        pathAllocationRetryPolicy.withDelay(Duration.ofMillis(pathAllocationRetryDelay));
    }
    try {
        return Failsafe.with(pathAllocationRetryPolicy).get(() -> {
            GetPathsResult potentialPath;
            if (forceToIgnoreBandwidth) {
                boolean originalIgnoreBandwidth = flow.isIgnoreBandwidth();
                flow.setIgnoreBandwidth(true);
                potentialPath = pathComputer.getPath(flow);
                flow.setIgnoreBandwidth(originalIgnoreBandwidth);
            } else {
                potentialPath = pathComputer.getPath(flow, pathsToReuseBandwidth);
            }
            boolean newPathFound = isNotSamePath(potentialPath, oldPaths);
            if (allowOldPaths || newPathFound) {
                if (!newPathFound) {
                    log.debug("Found the same path for flow {}. Proceed with recreating it", flow.getFlowId());
                }
                if (whetherCreatePathSegments.test(potentialPath)) {
                    boolean ignoreBandwidth = forceToIgnoreBandwidth || flow.isIgnoreBandwidth();
                    List<PathSegment> forwardSegments = flowPathBuilder.buildPathSegments(newForwardPathId, potentialPath.getForward(), flow.getBandwidth(), ignoreBandwidth, sharedBandwidthGroupId);
                    List<PathSegment> reverseSegments = flowPathBuilder.buildPathSegments(newReversePathId, potentialPath.getReverse(), flow.getBandwidth(), ignoreBandwidth, sharedBandwidthGroupId);
                    transactionManager.doInTransaction(() -> {
                        createPathSegments(forwardSegments, reuseBandwidthPerIsl);
                        createPathSegments(reverseSegments, reuseBandwidthPerIsl);
                    });
                }
                return potentialPath;
            }
            return null;
        });
    } catch (FailsafeException ex) {
        throw ex.getCause();
    }
}
Also used : FlowPath(org.openkilda.model.FlowPath) HistoryMapper(org.openkilda.wfm.share.mappers.HistoryMapper) SneakyThrows(lombok.SneakyThrows) ResourceAllocationException(org.openkilda.wfm.share.flow.resources.ResourceAllocationException) FlowPathStatus(org.openkilda.model.FlowPathStatus) UnroutableFlowException(org.openkilda.pce.exception.UnroutableFlowException) FlowPathSwappingFsm(org.openkilda.wfm.topology.flowhs.fsm.common.FlowPathSwappingFsm) FailsafeException(net.jodah.failsafe.FailsafeException) Flow(org.openkilda.model.Flow) Duration(java.time.Duration) Map(java.util.Map) FlowPathPair(org.openkilda.wfm.topology.flow.model.FlowPathPair) DumpType(org.openkilda.wfm.share.history.model.FlowDumpData.DumpType) Path(org.openkilda.pce.Path) SwitchProperties(org.openkilda.model.SwitchProperties) IslEndpoints(org.openkilda.persistence.repositories.IslRepository.IslEndpoints) Predicate(java.util.function.Predicate) Collection(java.util.Collection) RetryPolicy(net.jodah.failsafe.RetryPolicy) PersistenceException(org.openkilda.persistence.exceptions.PersistenceException) FlowResources(org.openkilda.wfm.share.flow.resources.FlowResources) String.format(java.lang.String.format) Objects(java.util.Objects) SwitchPropertiesRepository(org.openkilda.persistence.repositories.SwitchPropertiesRepository) Slf4j(lombok.extern.slf4j.Slf4j) List(java.util.List) FlowDumpData(org.openkilda.wfm.share.history.model.FlowDumpData) PathComputer(org.openkilda.pce.PathComputer) Optional(java.util.Optional) ConstraintViolationException(org.openkilda.persistence.exceptions.ConstraintViolationException) SwitchRepository(org.openkilda.persistence.repositories.SwitchRepository) FlowSegmentCookieBuilder(org.openkilda.model.cookie.FlowSegmentCookie.FlowSegmentCookieBuilder) Message(org.openkilda.messaging.Message) PathSegment(org.openkilda.model.PathSegment) KildaConfigurationRepository(org.openkilda.persistence.repositories.KildaConfigurationRepository) HashMap(java.util.HashMap) Supplier(java.util.function.Supplier) RecoverableException(org.openkilda.pce.exception.RecoverableException) ArrayList(java.util.ArrayList) FlowPathDirection(org.openkilda.model.FlowPathDirection) PathSegmentRepository(org.openkilda.persistence.repositories.PathSegmentRepository) IslRepository(org.openkilda.persistence.repositories.IslRepository) Suppliers(com.google.common.base.Suppliers) FlowSegmentCookie(org.openkilda.model.cookie.FlowSegmentCookie) PersistenceManager(org.openkilda.persistence.PersistenceManager) PathId(org.openkilda.model.PathId) DetectConnectedDevices(org.openkilda.model.DetectConnectedDevices) FlowOperationsDashboardLogger(org.openkilda.wfm.share.logger.FlowOperationsDashboardLogger) ErrorType(org.openkilda.messaging.error.ErrorType) IslStatus(org.openkilda.model.IslStatus) Failsafe(net.jodah.failsafe.Failsafe) RepositoryFactory(org.openkilda.persistence.repositories.RepositoryFactory) FlowPathBuilder(org.openkilda.wfm.topology.flowhs.service.FlowPathBuilder) SwitchId(org.openkilda.model.SwitchId) FlowResourcesManager(org.openkilda.wfm.share.flow.resources.FlowResourcesManager) GetPathsResult(org.openkilda.pce.GetPathsResult) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Isl(org.openkilda.model.Isl) LazyMap(org.apache.commons.collections4.map.LazyMap) HashMap(java.util.HashMap) FailsafeException(net.jodah.failsafe.FailsafeException) PathSegment(org.openkilda.model.PathSegment) GetPathsResult(org.openkilda.pce.GetPathsResult) IslEndpoints(org.openkilda.persistence.repositories.IslRepository.IslEndpoints) Objects(java.util.Objects) PersistenceException(org.openkilda.persistence.exceptions.PersistenceException) ResourceAllocationException(org.openkilda.wfm.share.flow.resources.ResourceAllocationException) Map(java.util.Map) HashMap(java.util.HashMap) LazyMap(org.apache.commons.collections4.map.LazyMap) SneakyThrows(lombok.SneakyThrows)

Example 53 with RetryPolicy

use of net.jodah.failsafe.RetryPolicy in project samza by apache.

the class FailsafeAdapter method valueOf.

/**
 * Convert the {@link TableRetryPolicy} to failsafe {@link RetryPolicy}.
 * @return this policy instance
 */
static RetryPolicy valueOf(TableRetryPolicy policy) {
    // max retries default changed to 2 in v2.0. switching back to infinite retries by default for back compat.
    RetryPolicy failSafePolicy = new RetryPolicy().withMaxRetries(-1);
    switch(policy.getBackoffType()) {
        case NONE:
            break;
        case FIXED:
            failSafePolicy.withDelay(policy.getSleepTime());
            break;
        case RANDOM:
            failSafePolicy.withDelay(policy.getRandomMin().toMillis(), policy.getRandomMax().toMillis(), ChronoUnit.MILLIS);
            break;
        case EXPONENTIAL:
            failSafePolicy.withBackoff(policy.getSleepTime().toMillis(), policy.getExponentialMaxSleep().toMillis(), ChronoUnit.MILLIS, policy.getExponentialFactor());
            break;
        default:
            throw new SamzaException("Unknown retry policy type.");
    }
    if (policy.getMaxDuration() != null) {
        failSafePolicy.withMaxDuration(policy.getMaxDuration());
    }
    if (policy.getMaxAttempts() != null) {
        failSafePolicy.withMaxRetries(policy.getMaxAttempts());
    }
    if (policy.getJitter() != null && policy.getBackoffType() != TableRetryPolicy.BackoffType.RANDOM) {
        failSafePolicy.withJitter(policy.getJitter());
    }
    failSafePolicy.abortOn(policy.getRetryPredicate().negate());
    return failSafePolicy;
}
Also used : RetryPolicy(net.jodah.failsafe.RetryPolicy) SamzaException(org.apache.samza.SamzaException)

Example 54 with RetryPolicy

use of net.jodah.failsafe.RetryPolicy in project samza by apache.

the class FutureUtil method executeAsyncWithRetries.

public static <T> CompletableFuture<T> executeAsyncWithRetries(String opName, Supplier<? extends CompletionStage<T>> action, Predicate<? extends Throwable> abortRetries, ExecutorService executor) {
    Duration maxDuration = Duration.ofMinutes(10);
    RetryPolicy<Object> retryPolicy = new RetryPolicy<>().withBackoff(100, 312500, ChronoUnit.MILLIS, // 100 ms, 500 ms, 2500 ms, 12.5 s, 1.05 min, 5.20 min, 5.20 min
    5).withMaxDuration(maxDuration).abortOn(// stop retrying if predicate returns true
    abortRetries).onRetry(e -> LOG.warn("Action: {} attempt: {} completed with error {} ms after start. Retrying up to {} ms.", opName, e.getAttemptCount(), e.getElapsedTime().toMillis(), maxDuration.toMillis(), e.getLastFailure()));
    return Failsafe.with(retryPolicy).with(executor).getStageAsync(action::get);
}
Also used : Duration(java.time.Duration) RetryPolicy(net.jodah.failsafe.RetryPolicy)

Example 55 with RetryPolicy

use of net.jodah.failsafe.RetryPolicy in project samza by apache.

the class TestTableRetryPolicy method testExponentialRetry.

@Test
public void testExponentialRetry() {
    TableRetryPolicy retryPolicy = new TableRetryPolicy();
    retryPolicy.withExponentialBackoff(Duration.ofMillis(1000), Duration.ofMillis(2000), 1.5);
    retryPolicy.withJitter(Duration.ofMillis(100));
    Assert.assertEquals(TableRetryPolicy.BackoffType.EXPONENTIAL, retryPolicy.getBackoffType());
    RetryPolicy fsRetry = FailsafeAdapter.valueOf(retryPolicy);
    Assert.assertEquals(1000, fsRetry.getDelay().toMillis());
    Assert.assertEquals(2000, fsRetry.getMaxDelay().toMillis());
    Assert.assertEquals(1.5, fsRetry.getDelayFactor(), 0.001);
    Assert.assertEquals(100, fsRetry.getJitter().toMillis());
    Assert.assertEquals("{\"sleepTime\":{\"seconds\":1,\"nanos\":0},\"exponentialFactor\":1.5," + "\"exponentialMaxSleep\":{\"seconds\":2,\"nanos\":0},\"jitter\":{\"seconds\":0,\"nanos\":100000000}," + "\"backoffType\":\"EXPONENTIAL\",\"retryPredicate\":{}}", retryPolicy.toConfig(null, null).get("TableRetryPolicy"));
}
Also used : RetryPolicy(net.jodah.failsafe.RetryPolicy) Test(org.junit.Test)

Aggregations

RetryPolicy (net.jodah.failsafe.RetryPolicy)56 Failsafe (net.jodah.failsafe.Failsafe)16 IOException (java.io.IOException)8 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)8 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)8 TimeUnit (java.util.concurrent.TimeUnit)6 ArrayList (java.util.ArrayList)5 List (java.util.List)5 Executors (java.util.concurrent.Executors)5 Test (org.junit.Test)5 Logger (org.slf4j.Logger)5 LoggerFactory (org.slf4j.LoggerFactory)5 Test (org.testng.annotations.Test)5 ConnectException (java.net.ConnectException)4 Duration (java.time.Duration)4 Map (java.util.Map)4 FailsafeException (net.jodah.failsafe.FailsafeException)4 Assert.assertEquals (org.testng.Assert.assertEquals)4 Collection (java.util.Collection)3 HashMap (java.util.HashMap)3