Examples with ProcessorDescriptor org.apache.tez.dag.api.ProcessorDescriptor used on opensource projects

Example 21 with ProcessorDescriptor

use of org.apache.tez.dag.api.ProcessorDescriptor in project hive by apache.

the class DagUtils method createPreWarmVertex.

/**
 * @param numContainers number of containers to pre-warm
 * @param localResources additional resources to pre-warm with
 * @return prewarm vertex to run
 */
public PreWarmVertex createPreWarmVertex(TezConfiguration conf, int numContainers, Map<String, LocalResource> localResources) throws IOException, TezException {
    ProcessorDescriptor prewarmProcDescriptor = ProcessorDescriptor.create(HivePreWarmProcessor.class.getName());
    prewarmProcDescriptor.setUserPayload(TezUtils.createUserPayloadFromConf(conf));
    PreWarmVertex prewarmVertex = PreWarmVertex.create("prewarm", prewarmProcDescriptor, numContainers, getContainerResource(conf));
    Map<String, LocalResource> combinedResources = new HashMap<String, LocalResource>();
    if (localResources != null) {
        combinedResources.putAll(localResources);
    }
    prewarmVertex.addTaskLocalFiles(localResources);
    prewarmVertex.setTaskLaunchCmdOpts(getContainerJavaOpts(conf));
    prewarmVertex.setTaskEnvironment(getContainerEnvironment(conf, false));
    return prewarmVertex;
}

Example 22 with ProcessorDescriptor

use of org.apache.tez.dag.api.ProcessorDescriptor in project hive by apache.

the class Converters method convertProcessorDescriptorFromProto.

private static ProcessorDescriptor convertProcessorDescriptorFromProto(EntityDescriptorProto proto) {
    String className = proto.getClassName();
    UserPayload payload = convertPayloadFromProto(proto);
    ProcessorDescriptor pd = ProcessorDescriptor.create(className);
    setUserPayload(pd, payload);
    return pd;
}