use of org.apache.beam.sdk.io.hadoop.SerializableConfiguration in project beam by apache.
the class HadoopInputFormatIOTest method testReadBuildsCorrectlyIfWithConfigurationIsCalledMoreThanOneTime.
/**
* This test validates {@link HadoopInputFormatIO.Read Read} object creation if
* {@link HadoopInputFormatIO.Read#withConfiguration() withConfiguration()} is called more than
* once.
* @throws InterruptedException
* @throws IOException
*/
@Test
public void testReadBuildsCorrectlyIfWithConfigurationIsCalledMoreThanOneTime() throws IOException, InterruptedException {
SerializableConfiguration diffConf = loadTestConfiguration(EmployeeInputFormat.class, Employee.class, Text.class);
HadoopInputFormatIO.Read<String, String> read = HadoopInputFormatIO.<String, String>read().withConfiguration(serConf.get()).withKeyTranslation(myKeyTranslate).withConfiguration(diffConf.get());
assertEquals(diffConf.get(), read.getConfiguration().get());
assertEquals(myKeyTranslate, read.getKeyTranslationFunction());
assertEquals(null, read.getValueTranslationFunction());
assertEquals(myKeyTranslate.getOutputTypeDescriptor(), read.getKeyTypeDescriptor());
assertEquals(diffConf.get().getClass("value.class", Object.class), read.getValueTypeDescriptor().getRawType());
}
use of org.apache.beam.sdk.io.hadoop.SerializableConfiguration in project beam by apache.
the class HadoopFormatIOReadTest method loadTestConfiguration.
private static SerializableConfiguration loadTestConfiguration(Class<?> inputFormatClassName, Class<?> keyClass, Class<?> valueClass) {
Configuration conf = new Configuration();
conf.setClass("mapreduce.job.inputformat.class", inputFormatClassName, InputFormat.class);
conf.setClass("key.class", keyClass, Object.class);
conf.setClass("value.class", valueClass, Object.class);
return new SerializableConfiguration(conf);
}
Aggregations