Search in sources :

Example 16 with ParamAction

use of com.dtstack.taier.pluginapi.pojo.ParamAction in project Taier by DTStack.

the class PerJobClientFactory method getClusterClient.

@Override
public ClusterClient getClusterClient(JobIdentifier jobIdentifier) {
    String applicationId = jobIdentifier.getApplicationId();
    String jobId = jobIdentifier.getJobId();
    ClusterClient clusterClient = null;
    try {
        clusterClient = KerberosUtils.login(flinkConfig, () -> {
            try {
                return perJobClientCache.get(applicationId, () -> {
                    ParamAction action = new ParamAction();
                    action.setJobId(jobId);
                    action.setName("jobId-" + jobId);
                    action.setTaskType(EJobType.SQL.getType());
                    action.setComputeType(ComputeType.STREAM.getType());
                    action.setTenantId(-1L);
                    String taskParams = "flinkTaskRunMode=per_job";
                    action.setTaskParams(taskParams);
                    JobClient jobClient = new JobClient(action);
                    try (YarnClusterDescriptor perJobYarnClusterDescriptor = this.createPerJobClusterDescriptor(jobClient)) {
                        return perJobYarnClusterDescriptor.retrieve(ConverterUtils.toApplicationId(applicationId)).getClusterClient();
                    }
                });
            } catch (ExecutionException e) {
                throw new PluginDefineException(e);
            }
        }, flinkClientBuilder.getYarnConf());
    } catch (Exception e) {
        LOG.error("job[{}] get perJobClient exception:{}", jobId, e);
        throw new PluginDefineException(e);
    }
    return clusterClient;
}
Also used : ParamAction(com.dtstack.taier.pluginapi.pojo.ParamAction) ClusterClient(org.apache.flink.client.program.ClusterClient) PluginDefineException(com.dtstack.taier.pluginapi.exception.PluginDefineException) YarnClusterDescriptor(org.apache.flink.yarn.YarnClusterDescriptor) ExecutionException(java.util.concurrent.ExecutionException) JobClient(com.dtstack.taier.pluginapi.JobClient) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) PluginDefineException(com.dtstack.taier.pluginapi.exception.PluginDefineException)

Aggregations

ParamAction (com.dtstack.taier.pluginapi.pojo.ParamAction)16 JobClient (com.dtstack.taier.pluginapi.JobClient)11 ScheduleEngineJobCache (com.dtstack.taier.dao.domain.ScheduleEngineJobCache)6 ScheduleJob (com.dtstack.taier.dao.domain.ScheduleJob)3 JobIdentifier (com.dtstack.taier.pluginapi.JobIdentifier)3 ClusterResource (com.dtstack.taier.pluginapi.pojo.ClusterResource)3 IOException (java.io.IOException)3 Map (java.util.Map)3 Properties (java.util.Properties)3 JSONObject (com.alibaba.fastjson.JSONObject)2 RdosDefineException (com.dtstack.taier.common.exception.RdosDefineException)2 PluginDefineException (com.dtstack.taier.pluginapi.exception.PluginDefineException)2 JobResult (com.dtstack.taier.pluginapi.pojo.JobResult)2 EJobCacheStage (com.dtstack.taier.common.enums.EJobCacheStage)1 EnvironmentContext (com.dtstack.taier.common.env.EnvironmentContext)1 JobClientComparator (com.dtstack.taier.common.queue.comparator.JobClientComparator)1 ConsoleJobInfoVO (com.dtstack.taier.develop.vo.console.ConsoleJobInfoVO)1 ConsoleJobVO (com.dtstack.taier.develop.vo.console.ConsoleJobVO)1 CustomThreadFactory (com.dtstack.taier.pluginapi.CustomThreadFactory)1 JarFileInfo (com.dtstack.taier.pluginapi.JarFileInfo)1