Examples with UserPayload org.apache.tez.dag.api.UserPayload used on opensource projects

Example 6 with UserPayload

use of org.apache.tez.dag.api.UserPayload in project hive by apache.

the class Converters method convertToProto.

private static EntityDescriptorProto convertToProto(EntityDescriptor<?> descriptor) {
    EntityDescriptorProto.Builder builder = EntityDescriptorProto.newBuilder();
    builder.setClassName(descriptor.getClassName());
    UserPayload userPayload = descriptor.getUserPayload();
    if (userPayload != null) {
        UserPayloadProto.Builder payloadBuilder = UserPayloadProto.newBuilder();
        if (userPayload.hasPayload()) {
            payloadBuilder.setUserPayload(ByteString.copyFrom(userPayload.getPayload()));
            payloadBuilder.setVersion(userPayload.getVersion());
        }
        builder.setUserPayload(payloadBuilder.build());
    }
    if (descriptor.getHistoryText() != null) {
        try {
            builder.setHistoryText(TezCommonUtils.compressByteArrayToByteString(descriptor.getHistoryText().getBytes("UTF-8")));
        } catch (IOException e) {
            throw new TezUncheckedException(e);
        }
    }
    return builder.build();
}

Example 7 with UserPayload

use of org.apache.tez.dag.api.UserPayload in project hive by apache.

the class Converters method convertInputDescriptorFromProto.

private static InputDescriptor convertInputDescriptorFromProto(EntityDescriptorProto proto) {
    String className = proto.getClassName();
    UserPayload payload = convertPayloadFromProto(proto);
    InputDescriptor id = InputDescriptor.create(className);
    setUserPayload(id, payload);
    return id;
}

Example 8 with UserPayload

use of org.apache.tez.dag.api.UserPayload in project hive by apache.

the class LlapTaskCommunicator method extractQueryId.

private String extractQueryId(TaskSpec taskSpec) throws IOException {
    UserPayload processorPayload = taskSpec.getProcessorDescriptor().getUserPayload();
    Configuration conf = TezUtils.createConfFromUserPayload(processorPayload);
    return HiveConf.getVar(conf, HiveConf.ConfVars.HIVEQUERYID);
}

Example 9 with UserPayload

use of org.apache.tez.dag.api.UserPayload in project hive by apache.

the class Converters method convertProcessorDescriptorFromProto.

private static ProcessorDescriptor convertProcessorDescriptorFromProto(EntityDescriptorProto proto) {
    String className = proto.getClassName();
    UserPayload payload = convertPayloadFromProto(proto);
    ProcessorDescriptor pd = ProcessorDescriptor.create(className);
    setUserPayload(pd, payload);
    return pd;
}

Example 10 with UserPayload

use of org.apache.tez.dag.api.UserPayload in project hive by apache.

the class DagUtils method setupAutoReducerParallelism.

private void setupAutoReducerParallelism(TezEdgeProperty edgeProp, Vertex v) throws IOException {
    if (edgeProp.isAutoReduce()) {
        Configuration pluginConf = new Configuration(false);
        VertexManagerPluginDescriptor desc = VertexManagerPluginDescriptor.create(ShuffleVertexManager.class.getName());
        pluginConf.setBoolean(ShuffleVertexManager.TEZ_SHUFFLE_VERTEX_MANAGER_ENABLE_AUTO_PARALLEL, true);
        pluginConf.setInt(ShuffleVertexManager.TEZ_SHUFFLE_VERTEX_MANAGER_MIN_TASK_PARALLELISM, edgeProp.getMinReducer());
        pluginConf.setLong(ShuffleVertexManager.TEZ_SHUFFLE_VERTEX_MANAGER_DESIRED_TASK_INPUT_SIZE, edgeProp.getInputSizePerReducer());
        UserPayload payload = TezUtils.createUserPayloadFromConf(pluginConf);
        desc.setUserPayload(payload);
        v.setVertexManagerPlugin(desc);
    }
}