Search in sources :

Example 1 with PlanKeyValuePair

use of org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair in project tez by apache.

the class TestTezClientUtils method testConfSerializationForAm.

@Test(timeout = 5000)
public void testConfSerializationForAm() {
    Configuration conf = new Configuration(false);
    String val1 = "fixedProperty";
    String val2 = "parametrizedProperty/${user.name}";
    String expVal2 = "parametrizedProperty/" + System.getProperty("user.name");
    conf.set("property1", val1);
    conf.set("property2", val2);
    Map<String, String> expected = new HashMap<String, String>();
    expected.put("property1", val1);
    expected.put("property2", expVal2);
    ConfigurationProto confProto = TezClientUtils.createFinalConfProtoForApp(conf, null);
    for (PlanKeyValuePair kvPair : confProto.getConfKeyValuesList()) {
        String v = expected.remove(kvPair.getKey());
        assertEquals(v, kvPair.getValue());
    }
    assertTrue(expected.isEmpty());
}
Also used : PlanKeyValuePair(org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair) Configuration(org.apache.hadoop.conf.Configuration) TezConfiguration(org.apache.tez.dag.api.TezConfiguration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) HashMap(java.util.HashMap) ConfigurationProto(org.apache.tez.dag.api.records.DAGProtos.ConfigurationProto) Test(org.junit.Test)

Example 2 with PlanKeyValuePair

use of org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair in project tez by apache.

the class TestTezClientUtils method testConfigurationAllowAll.

@Test(timeout = 5000)
public void testConfigurationAllowAll() {
    Configuration srcConf = new Configuration(false);
    Map<String, String> confMap = new HashMap<String, String>();
    confMap.put("foo.property", "2000");
    confMap.put("tez.property", "tezProperty");
    confMap.put("yarn.property", "yarnProperty");
    for (Map.Entry<String, String> entry : confMap.entrySet()) {
        srcConf.set(entry.getKey(), entry.getValue());
    }
    ConfigurationProto confProto = TezClientUtils.createFinalConfProtoForApp(srcConf, null);
    for (PlanKeyValuePair kvPair : confProto.getConfKeyValuesList()) {
        String val = confMap.remove(kvPair.getKey());
        assertNotNull(val);
        assertEquals(val, kvPair.getValue());
    }
    assertTrue(confMap.isEmpty());
}
Also used : PlanKeyValuePair(org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair) Configuration(org.apache.hadoop.conf.Configuration) TezConfiguration(org.apache.tez.dag.api.TezConfiguration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) HashMap(java.util.HashMap) Map(java.util.Map) HashMap(java.util.HashMap) ConfigurationProto(org.apache.tez.dag.api.records.DAGProtos.ConfigurationProto) Test(org.junit.Test)

Example 3 with PlanKeyValuePair

use of org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair in project tez by apache.

the class DagTypeConverters method convertConfFromProto.

public static Map<String, String> convertConfFromProto(ConfigurationProto confProto) {
    List<PlanKeyValuePair> settingList = confProto.getConfKeyValuesList();
    Map<String, String> map = new HashMap<String, String>();
    for (PlanKeyValuePair setting : settingList) {
        map.put(setting.getKey(), setting.getValue());
    }
    return map;
}
Also used : PlanKeyValuePair(org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair) HashMap(java.util.HashMap) ByteString(com.google.protobuf.ByteString)

Aggregations

HashMap (java.util.HashMap)3 PlanKeyValuePair (org.apache.tez.dag.api.records.DAGProtos.PlanKeyValuePair)3 Configuration (org.apache.hadoop.conf.Configuration)2 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)2 TezConfiguration (org.apache.tez.dag.api.TezConfiguration)2 ConfigurationProto (org.apache.tez.dag.api.records.DAGProtos.ConfigurationProto)2 Test (org.junit.Test)2 ByteString (com.google.protobuf.ByteString)1 Map (java.util.Map)1