Search in sources :

Example 1 with SingularityDisasterDataPoints

use of com.hubspot.singularity.SingularityDisasterDataPoints in project Singularity by HubSpot.

the class SingularityDisasterDetectionPoller method runActionOnPoll.

@Override
public void runActionOnPoll() {
    LOG.trace("Starting disaster detection");
    clearExpiredDisabledActions();
    List<SingularityDisasterType> previouslyActiveDisasters = disasterManager.getActiveDisasters();
    List<SingularityDisasterDataPoint> dataPoints = disasterManager.getDisasterStats().getDataPoints();
    SingularityDisasterDataPoint newStats = collectDisasterStats();
    dataPoints.add(0, newStats);
    if (dataPoints.size() > disasterConfiguration.getStatsHistorySize()) {
        dataPoints.remove(dataPoints.size() - 1);
    }
    LOG.debug("Collected new disaster detection dataPoints: {}", newStats);
    List<SingularityDisasterType> newActiveDisasters = checkDataPoints(dataPoints);
    if (!newActiveDisasters.isEmpty()) {
        LOG.warn("Detected new active disasters: {}", newActiveDisasters);
    }
    disasterManager.updateActiveDisasters(previouslyActiveDisasters, newActiveDisasters);
    disasterManager.saveDisasterStats(new SingularityDisasterDataPoints(dataPoints));
    if (!newActiveDisasters.isEmpty()) {
        if (!disasterManager.isAutomatedDisabledActionsDisabled()) {
            disasterManager.addDisabledActionsForDisasters(newActiveDisasters);
        }
        if (!previouslyActiveDisasters.containsAll(newActiveDisasters)) {
            queueDisasterEmail(dataPoints, newActiveDisasters);
        }
    } else {
        disasterManager.clearSystemGeneratedDisabledActions();
    }
}
Also used : SingularityDisasterDataPoints(com.hubspot.singularity.SingularityDisasterDataPoints) SingularityDisasterType(com.hubspot.singularity.SingularityDisasterType) SingularityDisasterDataPoint(com.hubspot.singularity.SingularityDisasterDataPoint)

Example 2 with SingularityDisasterDataPoints

use of com.hubspot.singularity.SingularityDisasterDataPoints in project Singularity by HubSpot.

the class DisasterManager method getDisasterStats.

public SingularityDisasterDataPoints getDisasterStats() {
    SingularityDisasterDataPoints stats = getData(DISASTER_STATS_PATH, disasterStatsTranscoder).or(SingularityDisasterDataPoints.empty());
    Collections.sort(stats.getDataPoints());
    return stats;
}
Also used : SingularityDisasterDataPoints(com.hubspot.singularity.SingularityDisasterDataPoints)

Aggregations

SingularityDisasterDataPoints (com.hubspot.singularity.SingularityDisasterDataPoints)2 SingularityDisasterDataPoint (com.hubspot.singularity.SingularityDisasterDataPoint)1 SingularityDisasterType (com.hubspot.singularity.SingularityDisasterType)1