Search in sources :

Example 6 with MetricArray

use of org.apache.gobblin.rest.MetricArray in project incubator-gobblin by apache.

the class DatabaseJobHistoryStoreV101 method addMetricsToTasks.

private void addMetricsToTasks(Connection connection, JobExecutionQuery query, Filter tableFilter, Map<String, Map<String, TaskExecutionInfo>> taskExecutionInfos) throws SQLException {
    if (query.isIncludeTaskMetrics() && taskExecutionInfos.size() > 0) {
        int index = 1;
        String template = String.format(TASK_METRIC_QUERY_STATEMENT_TEMPLATE, getInPredicate(taskExecutionInfos.size()));
        if (tableFilter.isPresent()) {
            template += " AND t." + tableFilter;
        }
        try (PreparedStatement taskMetricQueryStatement = connection.prepareStatement(template)) {
            for (String jobId : taskExecutionInfos.keySet()) {
                taskMetricQueryStatement.setString(index++, jobId);
            }
            if (tableFilter.isPresent()) {
                tableFilter.addParameters(taskMetricQueryStatement, index);
            }
            try (ResultSet taskMetricRs = taskMetricQueryStatement.executeQuery()) {
                while (taskMetricRs.next()) {
                    String jobId = taskMetricRs.getString("job_id");
                    String taskId = taskMetricRs.getString("task_id");
                    TaskExecutionInfo taskExecutionInfo = taskExecutionInfos.get(jobId).get(taskId);
                    MetricArray metricsArray = taskExecutionInfo.getMetrics(GetMode.NULL);
                    if (metricsArray == null) {
                        metricsArray = new MetricArray();
                        taskExecutionInfo.setMetrics(metricsArray);
                    }
                    metricsArray.add(resultSetToMetric(taskMetricRs));
                }
            }
        }
    }
}
Also used : TaskExecutionInfo(org.apache.gobblin.rest.TaskExecutionInfo) ResultSet(java.sql.ResultSet) PreparedStatement(java.sql.PreparedStatement) MetricArray(org.apache.gobblin.rest.MetricArray)

Aggregations

MetricArray (org.apache.gobblin.rest.MetricArray)6 StringMap (com.linkedin.data.template.StringMap)4 JobExecutionInfo (org.apache.gobblin.rest.JobExecutionInfo)4 TaskExecutionInfo (org.apache.gobblin.rest.TaskExecutionInfo)4 PreparedStatement (java.sql.PreparedStatement)3 ResultSet (java.sql.ResultSet)3 Map (java.util.Map)3 Metric (org.apache.gobblin.rest.Metric)3 TaskExecutionInfoArray (org.apache.gobblin.rest.TaskExecutionInfoArray)3 Counter (com.codahale.metrics.Counter)2 Gauge (com.codahale.metrics.Gauge)2 Meter (com.codahale.metrics.Meter)2 Table (org.apache.gobblin.rest.Table)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 AbstractMap (java.util.AbstractMap)1 JobMetrics (org.apache.gobblin.runtime.util.JobMetrics)1 TaskMetrics (org.apache.gobblin.runtime.util.TaskMetrics)1 Extract (org.apache.gobblin.source.workunit.Extract)1