Search in sources :

Example 96 with JobID

use of org.apache.flink.api.common.JobID in project flink by apache.

the class MiniClusterITCase method getSimpleJob.

private static JobGraph getSimpleJob() throws IOException {
    JobVertex task = new JobVertex("Test task");
    task.setParallelism(1);
    task.setMaxParallelism(1);
    task.setInvokableClass(NoOpInvokable.class);
    JobGraph jg = new JobGraph(new JobID(), "Test Job", task);
    jg.setAllowQueuedScheduling(true);
    jg.setScheduleMode(ScheduleMode.EAGER);
    ExecutionConfig executionConfig = new ExecutionConfig();
    executionConfig.setRestartStrategy(RestartStrategies.fixedDelayRestart(Integer.MAX_VALUE, 1000));
    jg.setExecutionConfig(executionConfig);
    return jg;
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) JobVertex(org.apache.flink.runtime.jobgraph.JobVertex) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) JobID(org.apache.flink.api.common.JobID)

Example 97 with JobID

use of org.apache.flink.api.common.JobID in project flink by apache.

the class JobManagerGroupTest method addAndRemoveJobs.

// ------------------------------------------------------------------------
//  adding and removing jobs
// ------------------------------------------------------------------------
@Test
public void addAndRemoveJobs() {
    MetricRegistry registry = new MetricRegistry(MetricRegistryConfiguration.defaultMetricRegistryConfiguration());
    final JobManagerMetricGroup group = new JobManagerMetricGroup(registry, "localhost");
    final JobID jid1 = new JobID();
    final JobID jid2 = new JobID();
    final String jobName1 = "testjob";
    final String jobName2 = "anotherJob";
    JobManagerJobMetricGroup jmJobGroup11 = group.addJob(new JobGraph(jid1, jobName1));
    JobManagerJobMetricGroup jmJobGroup12 = group.addJob(new JobGraph(jid1, jobName1));
    JobManagerJobMetricGroup jmJobGroup21 = group.addJob(new JobGraph(jid2, jobName2));
    assertEquals(jmJobGroup11, jmJobGroup12);
    assertEquals(2, group.numRegisteredJobMetricGroups());
    group.removeJob(jid1);
    assertTrue(jmJobGroup11.isClosed());
    assertEquals(1, group.numRegisteredJobMetricGroups());
    group.removeJob(jid2);
    assertTrue(jmJobGroup21.isClosed());
    assertEquals(0, group.numRegisteredJobMetricGroups());
    registry.shutdown();
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) MetricRegistry(org.apache.flink.runtime.metrics.MetricRegistry) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 98 with JobID

use of org.apache.flink.api.common.JobID in project flink by apache.

the class JobManagerGroupTest method testCloseClosesAll.

@Test
public void testCloseClosesAll() {
    MetricRegistry registry = new MetricRegistry(MetricRegistryConfiguration.defaultMetricRegistryConfiguration());
    final JobManagerMetricGroup group = new JobManagerMetricGroup(registry, "localhost");
    final JobID jid1 = new JobID();
    final JobID jid2 = new JobID();
    final String jobName1 = "testjob";
    final String jobName2 = "anotherJob";
    JobManagerJobMetricGroup jmJobGroup11 = group.addJob(new JobGraph(jid1, jobName1));
    JobManagerJobMetricGroup jmJobGroup21 = group.addJob(new JobGraph(jid2, jobName2));
    group.close();
    assertTrue(jmJobGroup11.isClosed());
    assertTrue(jmJobGroup21.isClosed());
    registry.shutdown();
}
Also used : JobGraph(org.apache.flink.runtime.jobgraph.JobGraph) MetricRegistry(org.apache.flink.runtime.metrics.MetricRegistry) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 99 with JobID

use of org.apache.flink.api.common.JobID in project flink by apache.

the class JobManagerJobGroupTest method testGenerateScopeCustom.

@Test
public void testGenerateScopeCustom() {
    Configuration cfg = new Configuration();
    cfg.setString(ConfigConstants.METRICS_SCOPE_NAMING_JM, "abc");
    cfg.setString(ConfigConstants.METRICS_SCOPE_NAMING_JM_JOB, "some-constant.<job_name>");
    MetricRegistry registry = new MetricRegistry(MetricRegistryConfiguration.fromConfiguration(cfg));
    JobID jid = new JobID();
    JobManagerMetricGroup tmGroup = new JobManagerMetricGroup(registry, "theHostName");
    JobMetricGroup jmGroup = new JobManagerJobMetricGroup(registry, tmGroup, jid, "myJobName");
    assertArrayEquals(new String[] { "some-constant", "myJobName" }, jmGroup.getScopeComponents());
    assertEquals("some-constant.myJobName.name", jmGroup.getMetricIdentifier("name"));
    registry.shutdown();
}
Also used : MetricRegistryConfiguration(org.apache.flink.runtime.metrics.MetricRegistryConfiguration) Configuration(org.apache.flink.configuration.Configuration) MetricRegistry(org.apache.flink.runtime.metrics.MetricRegistry) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 100 with JobID

use of org.apache.flink.api.common.JobID in project flink by apache.

the class JobManagerJobGroupTest method testCreateQueryServiceMetricInfo.

@Test
public void testCreateQueryServiceMetricInfo() {
    JobID jid = new JobID();
    MetricRegistry registry = new MetricRegistry(MetricRegistryConfiguration.defaultMetricRegistryConfiguration());
    JobManagerMetricGroup jm = new JobManagerMetricGroup(registry, "host");
    JobManagerJobMetricGroup jmj = new JobManagerJobMetricGroup(registry, jm, jid, "jobname");
    QueryScopeInfo.JobQueryScopeInfo info = jmj.createQueryServiceMetricInfo(new DummyCharacterFilter());
    assertEquals("", info.scope);
    assertEquals(jid.toString(), info.jobID);
}
Also used : QueryScopeInfo(org.apache.flink.runtime.metrics.dump.QueryScopeInfo) MetricRegistry(org.apache.flink.runtime.metrics.MetricRegistry) DummyCharacterFilter(org.apache.flink.runtime.metrics.util.DummyCharacterFilter) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Aggregations

JobID (org.apache.flink.api.common.JobID)335 Test (org.junit.Test)274 JobVertexID (org.apache.flink.runtime.jobgraph.JobVertexID)88 IOException (java.io.IOException)74 Configuration (org.apache.flink.configuration.Configuration)72 ExecutionAttemptID (org.apache.flink.runtime.executiongraph.ExecutionAttemptID)61 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)48 ActorGateway (org.apache.flink.runtime.instance.ActorGateway)47 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)44 ExecutionVertex (org.apache.flink.runtime.executiongraph.ExecutionVertex)42 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)38 ArrayList (java.util.ArrayList)37 MetricRegistry (org.apache.flink.runtime.metrics.MetricRegistry)32 KeyGroupRange (org.apache.flink.runtime.state.KeyGroupRange)31 HashMap (java.util.HashMap)29 AllocationID (org.apache.flink.runtime.clusterframework.types.AllocationID)29 FiniteDuration (scala.concurrent.duration.FiniteDuration)28 IntermediateDataSetID (org.apache.flink.runtime.jobgraph.IntermediateDataSetID)24 File (java.io.File)23 UUID (java.util.UUID)23