Search in sources :

Example 21 with MeasurementsTable

use of com.microsoft.dhalion.core.MeasurementsTable in project heron by twitter.

the class SkewDetector method detect.

/**
 * Detects components experiencing skew on a specific metric
 *
 * @return At most two symptoms corresponding to each affected component -- one for positive skew
 * and one for negative skew
 */
@Override
public Collection<Symptom> detect(Collection<Measurement> measurements) {
    Collection<Symptom> result = new ArrayList<>();
    MeasurementsTable metrics = MeasurementsTable.of(measurements).type(metricName);
    Instant now = context.checkpoint();
    for (String component : metrics.uniqueComponents()) {
        Set<String> addresses = new HashSet<>();
        Set<String> positiveAddresses = new HashSet<>();
        Set<String> negativeAddresses = new HashSet<>();
        double componentMax = getMaxOfAverage(metrics.component(component));
        double componentMin = getMinOfAverage(metrics.component(component));
        if (componentMax > skewRatio * componentMin) {
            // there is skew
            addresses.add(component);
            result.add(new Symptom(symptomType.text(), now, addresses));
            for (String instance : metrics.component(component).uniqueInstances()) {
                if (metrics.instance(instance).mean() >= 0.90 * componentMax) {
                    positiveAddresses.add(instance);
                }
                if (metrics.instance(instance).mean() <= 1.10 * componentMin) {
                    negativeAddresses.add(instance);
                }
            }
            if (!positiveAddresses.isEmpty()) {
                result.add(new Symptom("POSITIVE " + symptomType.text(), now, positiveAddresses));
            }
            if (!negativeAddresses.isEmpty()) {
                result.add(new Symptom("NEGATIVE " + symptomType.text(), now, negativeAddresses));
            }
        }
    }
    return result;
}
Also used : MeasurementsTable(com.microsoft.dhalion.core.MeasurementsTable) Instant(java.time.Instant) ArrayList(java.util.ArrayList) Symptom(com.microsoft.dhalion.core.Symptom) HashSet(java.util.HashSet)

Aggregations

MeasurementsTable (com.microsoft.dhalion.core.MeasurementsTable)21 Measurement (com.microsoft.dhalion.core.Measurement)14 Test (org.junit.Test)12 ArrayList (java.util.ArrayList)10 Instant (java.time.Instant)8 Symptom (com.microsoft.dhalion.core.Symptom)4 TopologyManager (org.apache.heron.proto.tmanager.TopologyManager)4 MetricsProvider (com.microsoft.dhalion.api.MetricsProvider)3 PhysicalPlanProvider (org.apache.heron.healthmgr.common.PhysicalPlanProvider)3 Diagnosis (com.microsoft.dhalion.core.Diagnosis)2 SymptomsTable (com.microsoft.dhalion.core.SymptomsTable)2 PoliciesExecutor (com.microsoft.dhalion.policy.PoliciesExecutor)2 Duration (java.time.Duration)2 HashSet (java.util.HashSet)2 PackingPlanProvider (org.apache.heron.healthmgr.common.PackingPlanProvider)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 ExecutionContext (com.microsoft.dhalion.policy.PoliciesExecutor.ExecutionContext)1 HealthManagerMetrics (org.apache.heron.healthmgr.HealthManagerMetrics)1 HealthPolicyConfig (org.apache.heron.healthmgr.HealthPolicyConfig)1