Search in sources :

Example 1 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class CheckpointStatsHandlerTest method testArchiver.

@Test
public void testArchiver() throws IOException {
    JsonArchivist archivist = new CheckpointStatsDetailsHandler.CheckpointStatsDetailsJsonArchivist();
    TestCheckpointStats testCheckpointStats = createTestCheckpointStats();
    when(testCheckpointStats.graph.getJobID()).thenReturn(new JobID());
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(testCheckpointStats.graph);
    Assert.assertEquals(3, archives.size());
    ObjectMapper mapper = new ObjectMapper();
    Iterator<ArchivedJson> iterator = archives.iterator();
    ArchivedJson archive1 = iterator.next();
    Assert.assertEquals("/jobs/" + testCheckpointStats.graph.getJobID() + "/checkpoints/details/" + testCheckpointStats.inProgress.getCheckpointId(), archive1.getPath());
    compareInProgressCheckpoint(testCheckpointStats.inProgress, mapper.readTree(archive1.getJson()));
    ArchivedJson archive2 = iterator.next();
    Assert.assertEquals("/jobs/" + testCheckpointStats.graph.getJobID() + "/checkpoints/details/" + testCheckpointStats.completedSavepoint.getCheckpointId(), archive2.getPath());
    compareCompletedSavepoint(testCheckpointStats.completedSavepoint, mapper.readTree(archive2.getJson()));
    ArchivedJson archive3 = iterator.next();
    Assert.assertEquals("/jobs/" + testCheckpointStats.graph.getJobID() + "/checkpoints/details/" + testCheckpointStats.failed.getCheckpointId(), archive3.getPath());
    compareFailedCheckpoint(testCheckpointStats.failed, mapper.readTree(archive3.getJson()));
}
Also used : JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) JobID(org.apache.flink.api.common.JobID) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test)

Example 2 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class JobPlanHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new JobPlanHandler.JobPlanJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/plan", archive.getPath());
    Assert.assertEquals(originalJob.getJsonPlan(), archive.getJson());
}
Also used : JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) Test(org.junit.Test)

Example 3 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class JobVertexAccumulatorsHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new JobVertexAccumulatorsHandler.JobVertexAccumulatorsJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    AccessExecutionJobVertex originalTask = ArchivedJobGenerationUtils.getTestTask();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/vertices/" + originalTask.getJobVertexId() + "/accumulators", archive.getPath());
    compareAccumulators(originalTask, archive.getJson());
}
Also used : AccessExecutionJobVertex(org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex) JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) Test(org.junit.Test)

Example 4 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class CurrentJobsOverviewHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new CurrentJobsOverviewHandler.CurrentJobsOverviewJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    JobDetails expectedDetails = WebMonitorUtils.createDetailsForJob(originalJob);
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/joboverview", archive.getPath());
    JsonNode result = ArchivedJobGenerationUtils.mapper.readTree(archive.getJson());
    ArrayNode running = (ArrayNode) result.get("running");
    Assert.assertEquals(0, running.size());
    ArrayNode finished = (ArrayNode) result.get("finished");
    Assert.assertEquals(1, finished.size());
    compareJobOverview(expectedDetails, finished.get(0).toString());
}
Also used : JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) JsonNode(com.fasterxml.jackson.databind.JsonNode) ArrayNode(com.fasterxml.jackson.databind.node.ArrayNode) JobDetails(org.apache.flink.runtime.messages.webmonitor.JobDetails) Test(org.junit.Test)

Example 5 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class SubtaskExecutionAttemptAccumulatorsHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new SubtaskExecutionAttemptAccumulatorsHandler.SubtaskExecutionAttemptAccumulatorsJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    AccessExecutionJobVertex originalTask = ArchivedJobGenerationUtils.getTestTask();
    AccessExecution originalAttempt = ArchivedJobGenerationUtils.getTestAttempt();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/vertices/" + originalTask.getJobVertexId() + "/subtasks/" + originalAttempt.getParallelSubtaskIndex() + "/attempts/" + originalAttempt.getAttemptNumber() + "/accumulators", archive.getPath());
    compareAttemptAccumulators(originalAttempt, archive.getJson());
}
Also used : AccessExecutionJobVertex(org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex) JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) AccessExecution(org.apache.flink.runtime.executiongraph.AccessExecution) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) Test(org.junit.Test)

Aggregations

ArchivedJson (org.apache.flink.runtime.webmonitor.history.ArchivedJson)35 JsonArchivist (org.apache.flink.runtime.webmonitor.history.JsonArchivist)18 Test (org.junit.Test)17 AccessExecutionGraph (org.apache.flink.runtime.executiongraph.AccessExecutionGraph)16 ResponseBody (org.apache.flink.runtime.rest.messages.ResponseBody)15 AccessExecutionJobVertex (org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex)12 ArrayList (java.util.ArrayList)10 AccessExecution (org.apache.flink.runtime.executiongraph.AccessExecution)5 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 JobID (org.apache.flink.api.common.JobID)4 CheckpointStatsHistory (org.apache.flink.runtime.checkpoint.CheckpointStatsHistory)4 CheckpointStatsSnapshot (org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot)4 JsonNode (com.fasterxml.jackson.databind.JsonNode)3 AbstractCheckpointStats (org.apache.flink.runtime.checkpoint.AbstractCheckpointStats)3 AccessExecutionVertex (org.apache.flink.runtime.executiongraph.AccessExecutionVertex)3 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 IOException (java.io.IOException)2 TaskStateStats (org.apache.flink.runtime.checkpoint.TaskStateStats)2 RestHandlerException (org.apache.flink.runtime.rest.handler.RestHandlerException)2 ErrorResponseBody (org.apache.flink.runtime.rest.messages.ErrorResponseBody)2