Search in sources :

Example 1 with CheckPointDataInput

use of org.goldenorb.io.input.checkpoint.CheckPointDataInput in project goldenorb by jzachr.

the class CheckPointDataTest method testCheckpointInput.

/**
 * Tests the CheckPointDataInput class by reading several different types of Writables from the checkpoint.
 * Asserts that Writables that were written in are of the same value and type when reading in from HDFS.
 *
 * @throws Exception
 */
@Test
public void testCheckpointInput() throws Exception {
    int superStep = 0;
    int partition = 0;
    OrbConfiguration orbConf = new OrbConfiguration();
    orbConf.set("fs.default.name", "hdfs://localhost:" + cluster.getNameNodePort());
    orbConf.setJobNumber("0");
    orbConf.setFileOutputPath("test");
    CheckPointDataInput checkpointInput = new CheckPointDataInput(orbConf, superStep, partition);
    // Data is read on a FIFO basis
    IntWritable intInput = new IntWritable();
    intInput.readFields(checkpointInput);
    LongWritable longInput = new LongWritable();
    longInput.readFields(checkpointInput);
    Text textInput = new Text();
    textInput.readFields(checkpointInput);
    FloatWritable floatInput = new FloatWritable();
    floatInput.readFields(checkpointInput);
    checkpointInput.close();
    assertThat(checkpointInput, notNullValue());
    assertEquals(intInput.get(), 4);
    assertEquals(longInput.get(), 9223372036854775807L);
    assertEquals(textInput.toString(), "test");
    assertTrue(floatInput.get() == 3.14159F);
}
Also used : FloatWritable(org.apache.hadoop.io.FloatWritable) OrbConfiguration(org.goldenorb.conf.OrbConfiguration) Text(org.apache.hadoop.io.Text) LongWritable(org.apache.hadoop.io.LongWritable) CheckPointDataInput(org.goldenorb.io.input.checkpoint.CheckPointDataInput) IntWritable(org.apache.hadoop.io.IntWritable)

Aggregations

FloatWritable (org.apache.hadoop.io.FloatWritable)1 IntWritable (org.apache.hadoop.io.IntWritable)1 LongWritable (org.apache.hadoop.io.LongWritable)1 Text (org.apache.hadoop.io.Text)1 OrbConfiguration (org.goldenorb.conf.OrbConfiguration)1 CheckPointDataInput (org.goldenorb.io.input.checkpoint.CheckPointDataInput)1