Search in sources :

Example 16 with Cluster

use of com.sequenceiq.periscope.domain.Cluster in project cloudbreak by hortonworks.

the class ClusterCreationEvaluator method ambariHealthCheck.

private void ambariHealthCheck(PeriscopeUser user, AmbariStack ambariStack) {
    String host = ambariStack.getAmbari().getHost();
    try {
        AmbariClient client = ambariClientProvider.createAmbariClient(new Cluster(user, ambariStack));
        String healthCheckResult = client.healthCheck();
        if (!"RUNNING".equals(healthCheckResult)) {
            throw new AmbariHealtCheckException(String.format("Ambari on host '%s' is not in 'RUNNING' state.", host));
        }
    } catch (Exception ex) {
        throw new AmbariHealtCheckException(String.format("Health check failed on host '%s':", host), ex);
    }
}
Also used : Cluster(com.sequenceiq.periscope.domain.Cluster) TlsConfigurationException(com.sequenceiq.periscope.service.security.TlsConfigurationException) AmbariClient(com.sequenceiq.ambari.client.AmbariClient)

Example 17 with Cluster

use of com.sequenceiq.periscope.domain.Cluster in project cloudbreak by hortonworks.

the class CronTimeEvaluator method run.

@Override
public void run() {
    Cluster cluster = clusterService.find(clusterId);
    MDCBuilder.buildMdcContext(cluster);
    for (TimeAlert alert : alertRepository.findAllByCluster(clusterId)) {
        if (isTrigger(alert) && isPolicyAttached(alert)) {
            LOGGER.info("Time alert '{}' triggers the '{}' scaling policy", alert.getName(), alert.getScalingPolicy().getName());
            publishEvent(new ScalingEvent(alert));
            break;
        }
    }
}
Also used : TimeAlert(com.sequenceiq.periscope.domain.TimeAlert) ScalingEvent(com.sequenceiq.periscope.monitor.event.ScalingEvent) Cluster(com.sequenceiq.periscope.domain.Cluster)

Example 18 with Cluster

use of com.sequenceiq.periscope.domain.Cluster in project cloudbreak by hortonworks.

the class PrometheusEvaluator method run.

@Override
public void run() {
    try {
        Cluster cluster = clusterService.find(clusterId);
        MDCBuilder.buildMdcContext(cluster);
        TlsConfiguration tlsConfig = tlsSecurityService.getConfiguration(cluster);
        Client client = RestClientUtil.createClient(tlsConfig.getServerCert(), tlsConfig.getClientCert(), tlsConfig.getClientKey(), true, PrometheusEvaluator.class);
        String prometheusAddress = String.format("https://%s:%s/prometheus", cluster.getAmbari().getHost(), cluster.getPort());
        WebTarget target = client.target(prometheusAddress);
        for (PrometheusAlert alert : alertRepository.findAllByCluster(clusterId)) {
            String alertName = alert.getName();
            LOGGER.info("Checking Prometheus based alert: '{}'", alertName);
            String query = URLEncoder.encode(String.format("ALERTS{alertname=\"%s\"}[%dm]", alert.getName(), alert.getPeriod()), "UTF-8");
            Response response = target.path("/api/v1/query").queryParam("query", query).request().header("Accept", MediaType.APPLICATION_JSON_VALUE).get();
            PrometheusResponse prometheusResponse = JaxRSUtil.response(response, PrometheusResponse.class);
            boolean triggerScale = false;
            switch(alert.getAlertState()) {
                case OK:
                    triggerScale = prometheusResponse.getData().getResult().isEmpty();
                    break;
                case CRITICAL:
                    for (Result alertResult : prometheusResponse.getData().getResult()) {
                        if ("firing".equals(alertResult.getMetric().getAlertstate())) {
                            List<Object> lastSample = alertResult.getValues().get(alertResult.getValues().size() - 1);
                            Object alertValue = lastSample.get(1);
                            if (alertValue instanceof String) {
                                if ("0".equals(alertValue)) {
                                    break;
                                }
                                triggerScale = true;
                            }
                        }
                    }
                    break;
                default:
                    triggerScale = false;
                    break;
            }
            if (triggerScale && isPolicyAttached(alert)) {
                publishEvent(new ScalingEvent(alert));
            }
        }
    } catch (Exception e) {
        LOGGER.error("Failed to retrieve alerts from Prometheus", e);
        publishEvent(new UpdateFailedEvent(clusterId));
    }
}
Also used : UpdateFailedEvent(com.sequenceiq.periscope.monitor.event.UpdateFailedEvent) PrometheusResponse(com.sequenceiq.periscope.model.PrometheusResponse) Cluster(com.sequenceiq.periscope.domain.Cluster) Result(com.sequenceiq.periscope.model.PrometheusResponse.Result) PrometheusResponse(com.sequenceiq.periscope.model.PrometheusResponse) Response(javax.ws.rs.core.Response) PrometheusAlert(com.sequenceiq.periscope.domain.PrometheusAlert) ScalingEvent(com.sequenceiq.periscope.monitor.event.ScalingEvent) TlsConfiguration(com.sequenceiq.periscope.model.TlsConfiguration) WebTarget(javax.ws.rs.client.WebTarget) Client(javax.ws.rs.client.Client)

Example 19 with Cluster

use of com.sequenceiq.periscope.domain.Cluster in project cloudbreak by hortonworks.

the class ClusterRequestConverter method convert.

@Override
public Cluster convert(AutoscaleClusterRequest source) {
    Cluster cluster = new Cluster();
    cluster.setStackId(source.getStackId());
    cluster.setAutoscalingEnabled(source.enableAutoscaling());
    List<MetricAlertRequest> metricAlertResponses = source.getMetricAlerts();
    if (metricAlertResponses != null && !metricAlertResponses.isEmpty()) {
        Set<MetricAlert> alerts = metricAlertResponses.stream().map(metricAlertJson -> {
            MetricAlert alert = metricAlertRequestConverter.convert(metricAlertJson);
            alert.setCluster(cluster);
            return alert;
        }).collect(Collectors.toSet());
        cluster.setMetricAlerts(alerts);
    }
    List<TimeAlertRequest> timeAlertRequests = source.getTimeAlerts();
    if (timeAlertRequests != null && !timeAlertRequests.isEmpty()) {
        Set<TimeAlert> alerts = timeAlertRequests.stream().map(timeAlertJson -> {
            TimeAlert alert = timeAlertRequestConverter.convert(timeAlertJson);
            alert.setCluster(cluster);
            return alert;
        }).collect(Collectors.toSet());
        cluster.setTimeAlerts(alerts);
    }
    List<PrometheusAlertRequest> prometheusAlertRequests = source.getPrometheusAlerts();
    if (prometheusAlertRequests != null && !prometheusAlertRequests.isEmpty()) {
        Set<PrometheusAlert> alerts = prometheusAlertRequests.stream().map(prometheusAlertJson -> {
            PrometheusAlert alert = prometheusAlertRequestConverter.convert(prometheusAlertJson);
            alert.setCluster(cluster);
            return alert;
        }).collect(Collectors.toSet());
        cluster.setPrometheusAlerts(alerts);
    }
    ScalingConfigurationRequest scalingConfiguration = source.getScalingConfiguration();
    if (scalingConfiguration != null) {
        cluster.setMinSize(scalingConfiguration.getMinSize());
        cluster.setMaxSize(scalingConfiguration.getMaxSize());
        cluster.setCoolDown(scalingConfiguration.getCoolDown());
    }
    return cluster;
}
Also used : TimeAlertRequest(com.sequenceiq.periscope.api.model.TimeAlertRequest) Cluster(com.sequenceiq.periscope.domain.Cluster) ScalingConfigurationRequest(com.sequenceiq.periscope.api.model.ScalingConfigurationRequest) PrometheusAlertRequest(com.sequenceiq.periscope.api.model.PrometheusAlertRequest) Set(java.util.Set) Collectors(java.util.stream.Collectors) Inject(javax.inject.Inject) List(java.util.List) Component(org.springframework.stereotype.Component) MetricAlertRequest(com.sequenceiq.periscope.api.model.MetricAlertRequest) MetricAlert(com.sequenceiq.periscope.domain.MetricAlert) TimeAlert(com.sequenceiq.periscope.domain.TimeAlert) PrometheusAlert(com.sequenceiq.periscope.domain.PrometheusAlert) AutoscaleClusterRequest(com.sequenceiq.periscope.api.model.AutoscaleClusterRequest) Cluster(com.sequenceiq.periscope.domain.Cluster) MetricAlertRequest(com.sequenceiq.periscope.api.model.MetricAlertRequest) ScalingConfigurationRequest(com.sequenceiq.periscope.api.model.ScalingConfigurationRequest) TimeAlertRequest(com.sequenceiq.periscope.api.model.TimeAlertRequest) PrometheusAlertRequest(com.sequenceiq.periscope.api.model.PrometheusAlertRequest) TimeAlert(com.sequenceiq.periscope.domain.TimeAlert) PrometheusAlert(com.sequenceiq.periscope.domain.PrometheusAlert) MetricAlert(com.sequenceiq.periscope.domain.MetricAlert)

Example 20 with Cluster

use of com.sequenceiq.periscope.domain.Cluster in project cloudbreak by hortonworks.

the class UpdateFailedHandler method reportAmbariServerFailure.

private void reportAmbariServerFailure(Cluster cluster, StackResponse stackResponse, CloudbreakClient cbClient) {
    Optional<InstanceMetaDataJson> pgw = stackResponse.getInstanceGroups().stream().flatMap(ig -> ig.getMetadata().stream()).filter(im -> im.getInstanceType() == InstanceMetadataType.GATEWAY_PRIMARY && im.getInstanceStatus() != InstanceStatus.TERMINATED).findFirst();
    if (pgw.isPresent()) {
        FailureReport failureReport = new FailureReport();
        failureReport.setFailedNodes(Collections.singletonList(pgw.get().getDiscoveryFQDN()));
        try {
            cbClient.clusterEndpoint().failureReport(cluster.getStackId(), failureReport);
        } catch (Exception e) {
            LOGGER.warn("Exception during failure report", e);
        }
    }
}
Also used : Logger(org.slf4j.Logger) Cluster(com.sequenceiq.periscope.domain.Cluster) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) LoggerFactory(org.slf4j.LoggerFactory) CloudbreakClient(com.sequenceiq.cloudbreak.client.CloudbreakClient) ApplicationListener(org.springframework.context.ApplicationListener) UpdateFailedEvent(com.sequenceiq.periscope.monitor.event.UpdateFailedEvent) HashSet(java.util.HashSet) Inject(javax.inject.Inject) InstanceMetadataType(com.sequenceiq.cloudbreak.api.model.InstanceMetadataType) FailureReport(com.sequenceiq.cloudbreak.api.model.FailureReport) Component(org.springframework.stereotype.Component) Map(java.util.Map) InstanceStatus(com.sequenceiq.cloudbreak.api.model.InstanceStatus) MDCBuilder(com.sequenceiq.periscope.log.MDCBuilder) StackResponse(com.sequenceiq.cloudbreak.api.model.StackResponse) ClusterState(com.sequenceiq.periscope.api.model.ClusterState) Optional(java.util.Optional) InstanceMetaDataJson(com.sequenceiq.cloudbreak.api.model.InstanceMetaDataJson) Collections(java.util.Collections) CloudbreakClientConfiguration(com.sequenceiq.periscope.service.configuration.CloudbreakClientConfiguration) ClusterService(com.sequenceiq.periscope.service.ClusterService) FailureReport(com.sequenceiq.cloudbreak.api.model.FailureReport) InstanceMetaDataJson(com.sequenceiq.cloudbreak.api.model.InstanceMetaDataJson)

Aggregations

Cluster (com.sequenceiq.periscope.domain.Cluster)34 Map (java.util.Map)6 PrometheusAlert (com.sequenceiq.periscope.domain.PrometheusAlert)5 AmbariClient (com.sequenceiq.ambari.client.AmbariClient)4 CloudbreakClient (com.sequenceiq.cloudbreak.client.CloudbreakClient)4 MetricAlert (com.sequenceiq.periscope.domain.MetricAlert)4 PeriscopeUser (com.sequenceiq.periscope.domain.PeriscopeUser)4 UpdateFailedEvent (com.sequenceiq.periscope.monitor.event.UpdateFailedEvent)4 TimeAlert (com.sequenceiq.periscope.domain.TimeAlert)3 ScalingEvent (com.sequenceiq.periscope.monitor.event.ScalingEvent)3 Collections (java.util.Collections)3 List (java.util.List)3 Set (java.util.Set)3 Inject (javax.inject.Inject)3 Logger (org.slf4j.Logger)3 LoggerFactory (org.slf4j.LoggerFactory)3 Component (org.springframework.stereotype.Component)3 AutoscaleStackResponse (com.sequenceiq.cloudbreak.api.model.AutoscaleStackResponse)2 FailureReport (com.sequenceiq.cloudbreak.api.model.FailureReport)2 StackResponse (com.sequenceiq.cloudbreak.api.model.StackResponse)2