Search in sources :

Example 11 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class JobDetailsHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new JobDetailsHandler.JobDetailsJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(2, archives.size());
    Iterator<ArchivedJson> iterator = archives.iterator();
    ArchivedJson archive1 = iterator.next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID(), archive1.getPath());
    compareJobDetails(originalJob, archive1.getJson());
    ArchivedJson archive2 = iterator.next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/vertices", archive2.getPath());
    compareJobDetails(originalJob, archive2.getJson());
}
Also used : JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) Test(org.junit.Test)

Example 12 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class JobExceptionsHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new JobExceptionsHandler.JobExceptionsJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/exceptions", archive.getPath());
    compareExceptions(originalJob, archive.getJson());
}
Also used : JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) Test(org.junit.Test)

Example 13 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class CheckpointStatsDetailsHandlerTest method testArchiver.

@Test
public void testArchiver() throws IOException {
    JsonArchivist archivist = new CheckpointStatsDetailsHandler.CheckpointStatsDetailsJsonArchivist();
    CompletedCheckpointStats completedCheckpoint = createCompletedCheckpoint();
    FailedCheckpointStats failedCheckpoint = createFailedCheckpoint();
    List<AbstractCheckpointStats> checkpoints = new ArrayList<>();
    checkpoints.add(failedCheckpoint);
    checkpoints.add(completedCheckpoint);
    CheckpointStatsHistory history = mock(CheckpointStatsHistory.class);
    when(history.getCheckpoints()).thenReturn(checkpoints);
    CheckpointStatsSnapshot snapshot = mock(CheckpointStatsSnapshot.class);
    when(snapshot.getHistory()).thenReturn(history);
    AccessExecutionGraph graph = mock(AccessExecutionGraph.class);
    when(graph.getCheckpointStatsSnapshot()).thenReturn(snapshot);
    when(graph.getJobID()).thenReturn(new JobID());
    ObjectMapper mapper = new ObjectMapper();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(graph);
    Assert.assertEquals(2, archives.size());
    Iterator<ArchivedJson> iterator = archives.iterator();
    ArchivedJson archive1 = iterator.next();
    Assert.assertEquals("/jobs/" + graph.getJobID() + "/checkpoints/details/" + failedCheckpoint.getCheckpointId(), archive1.getPath());
    compareFailedCheckpoint(failedCheckpoint, mapper.readTree(archive1.getJson()));
    ArchivedJson archive2 = iterator.next();
    Assert.assertEquals("/jobs/" + graph.getJobID() + "/checkpoints/details/" + completedCheckpoint.getCheckpointId(), archive2.getPath());
    compareCompletedCheckpoint(completedCheckpoint, mapper.readTree(archive2.getJson()));
}
Also used : AbstractCheckpointStats(org.apache.flink.runtime.checkpoint.AbstractCheckpointStats) ArrayList(java.util.ArrayList) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) CheckpointStatsHistory(org.apache.flink.runtime.checkpoint.CheckpointStatsHistory) CompletedCheckpointStats(org.apache.flink.runtime.checkpoint.CompletedCheckpointStats) FailedCheckpointStats(org.apache.flink.runtime.checkpoint.FailedCheckpointStats) CheckpointStatsSnapshot(org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot) JobID(org.apache.flink.api.common.JobID) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test)

Example 14 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class JobVertexDetailsHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new JobVertexDetailsHandler.JobVertexDetailsJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    AccessExecutionJobVertex originalTask = ArchivedJobGenerationUtils.getTestTask();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/vertices/" + originalTask.getJobVertexId(), archive.getPath());
    compareVertexDetails(originalTask, archive.getJson());
}
Also used : AccessExecutionJobVertex(org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex) JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) Test(org.junit.Test)

Example 15 with ArchivedJson

use of org.apache.flink.runtime.webmonitor.history.ArchivedJson in project flink by apache.

the class JobVertexTaskManagersHandlerTest method testArchiver.

@Test
public void testArchiver() throws Exception {
    JsonArchivist archivist = new JobVertexTaskManagersHandler.JobVertexTaskManagersJsonArchivist();
    AccessExecutionGraph originalJob = ArchivedJobGenerationUtils.getTestJob();
    AccessExecutionJobVertex originalTask = ArchivedJobGenerationUtils.getTestTask();
    AccessExecutionVertex originalSubtask = ArchivedJobGenerationUtils.getTestSubtask();
    Collection<ArchivedJson> archives = archivist.archiveJsonWithPath(originalJob);
    Assert.assertEquals(1, archives.size());
    ArchivedJson archive = archives.iterator().next();
    Assert.assertEquals("/jobs/" + originalJob.getJobID() + "/vertices/" + originalTask.getJobVertexId() + "/taskmanagers", archive.getPath());
    compareVertexTaskManagers(originalTask, originalSubtask, archive.getJson());
}
Also used : AccessExecutionJobVertex(org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex) JsonArchivist(org.apache.flink.runtime.webmonitor.history.JsonArchivist) ArchivedJson(org.apache.flink.runtime.webmonitor.history.ArchivedJson) AccessExecutionGraph(org.apache.flink.runtime.executiongraph.AccessExecutionGraph) AccessExecutionVertex(org.apache.flink.runtime.executiongraph.AccessExecutionVertex) Test(org.junit.Test)

Aggregations

ArchivedJson (org.apache.flink.runtime.webmonitor.history.ArchivedJson)17 JsonArchivist (org.apache.flink.runtime.webmonitor.history.JsonArchivist)17 Test (org.junit.Test)17 AccessExecutionGraph (org.apache.flink.runtime.executiongraph.AccessExecutionGraph)16 AccessExecutionJobVertex (org.apache.flink.runtime.executiongraph.AccessExecutionJobVertex)7 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 JobID (org.apache.flink.api.common.JobID)4 JsonNode (com.fasterxml.jackson.databind.JsonNode)3 AccessExecution (org.apache.flink.runtime.executiongraph.AccessExecution)3 CheckpointStatsHistory (org.apache.flink.runtime.checkpoint.CheckpointStatsHistory)2 CheckpointStatsSnapshot (org.apache.flink.runtime.checkpoint.CheckpointStatsSnapshot)2 ArrayNode (com.fasterxml.jackson.databind.node.ArrayNode)1 ArrayList (java.util.ArrayList)1 AbstractCheckpointStats (org.apache.flink.runtime.checkpoint.AbstractCheckpointStats)1 CompletedCheckpointStats (org.apache.flink.runtime.checkpoint.CompletedCheckpointStats)1 FailedCheckpointStats (org.apache.flink.runtime.checkpoint.FailedCheckpointStats)1 PendingCheckpointStats (org.apache.flink.runtime.checkpoint.PendingCheckpointStats)1 TaskStateStats (org.apache.flink.runtime.checkpoint.TaskStateStats)1 AccessExecutionVertex (org.apache.flink.runtime.executiongraph.AccessExecutionVertex)1 ExternalizedCheckpointSettings (org.apache.flink.runtime.jobgraph.tasks.ExternalizedCheckpointSettings)1