Search in sources :

Example 1 with BSPJobID

use of org.apache.hama.bsp.BSPJobID in project whirr by apache.

the class HamaServiceTest method test.

@Test(timeout = TestConstants.ITEST_TIMEOUT)
public void test() throws Exception {
    HamaConfiguration jobConf = controller.getConfiguration();
    jobConf.set("hadoop.rpc.socket.factory.class.default", "org.apache.hadoop.net.StandardSocketFactory");
    BSPJob bsp = new BSPJob(jobConf, new BSPJobID());
    LOG.info("Job conf: " + bsp.getConf().get("hadoop.rpc.socket.factory.class.default") + ", " + bsp.getJobID().toString());
    bsp.setJarByClass(MyEstimator.class);
    bsp.setBspClass(MyEstimator.class);
    bsp.setInputFormat(NullInputFormat.class);
    bsp.setOutputKeyClass(Text.class);
    bsp.setOutputValueClass(DoubleWritable.class);
    bsp.setOutputFormat(TextOutputFormat.class);
    bsp.set("bsp.working.dir", "/tmp");
    FileOutputFormat.setOutputPath(bsp, TMP_OUTPUT);
    LOG.info("Client configuration start ..");
    HamaConfiguration clientConf = controller.getConfiguration();
    BSPJobClient jobClient = new BSPJobClient(clientConf);
    ClusterStatus cluster = jobClient.getClusterStatus(true);
    assertNotNull(cluster);
    assertTrue(cluster.getGroomServers() > 0);
    assertTrue(cluster.getMaxTasks() > 1);
    bsp.setNumBspTask(cluster.getMaxTasks());
    LOG.info("Client conf: " + clientConf.get("hadoop.rpc.socket.factory.class.default"));
    RunningJob rJob = jobClient.submitJob(bsp);
    rJob.waitForCompletion();
    LOG.info("finished");
}
Also used : BSPJobClient(org.apache.hama.bsp.BSPJobClient) BSPJobID(org.apache.hama.bsp.BSPJobID) BSPJob(org.apache.hama.bsp.BSPJob) RunningJob(org.apache.hama.bsp.RunningJob) HamaConfiguration(org.apache.hama.HamaConfiguration) ClusterStatus(org.apache.hama.bsp.ClusterStatus) Test(org.junit.Test)

Aggregations

HamaConfiguration (org.apache.hama.HamaConfiguration)1 BSPJob (org.apache.hama.bsp.BSPJob)1 BSPJobClient (org.apache.hama.bsp.BSPJobClient)1 BSPJobID (org.apache.hama.bsp.BSPJobID)1 ClusterStatus (org.apache.hama.bsp.ClusterStatus)1 RunningJob (org.apache.hama.bsp.RunningJob)1 Test (org.junit.Test)1