Search in sources :

Example 1 with Sql

use of com.flink.platform.common.job.Sql in project flink-platform-backend by itinycheng.

the class Sql112Application method main.

public static void main(String[] args) throws Exception {
    // deSer sql context
    Path sqlContextPath = Paths.get(args[0]);
    SqlContext sqlContext = JsonUtil.toBean(sqlContextPath, SqlContext.class);
    // step 1: create and configure environment
    TableEnvironment tEnv = ExecutionEnvs.createExecutionEnv(sqlContext.getExecMode());
    Map<String, String> configMap = ConfigLoader.loadDefault(sqlContext.getExecMode());
    configMap.putAll(sqlContext.getConfigs());
    configMap.putAll(sqlContext.getSqls().stream().filter(sql -> SqlType.SET.equals(sql.getType())).map(Sql::getOperands).collect(toMap(operands -> operands[0], operands -> operands[1])));
    configMap.forEach((key, value) -> Configurations.setConfig(tEnv, key, value));
    // step 2: add catalog
    List<Catalog> catalogs = sqlContext.getCatalogs();
    Catalogs.registerCatalogsToTableEnv(tEnv, catalogs);
    // step 3: create temporary system function
    List<Function> functions = sqlContext.getFunctions();
    Functions.registerFunctionsToTableEnv(tEnv, functions);
    // step 4: exec sql
    List<Sql> executableSqls = sqlContext.getSqls().stream().filter(sql -> !SqlType.SET.equals(sql.getType())).collect(toList());
    ExecuteSqls.execSqls(tEnv, executableSqls);
}
Also used : Path(java.nio.file.Path) Functions(com.flink.platform.core.helper.Functions) TableEnvironment(org.apache.flink.table.api.TableEnvironment) Configurations(com.flink.platform.core.helper.Configurations) Sql(com.flink.platform.common.job.Sql) JsonUtil(com.flink.platform.common.util.JsonUtil) List(java.util.List) Collectors.toList(java.util.stream.Collectors.toList) ConfigLoader(com.flink.platform.core.common.ConfigLoader) Collectors.toMap(java.util.stream.Collectors.toMap) Catalog(com.flink.platform.common.job.Catalog) Catalogs(com.flink.platform.core.helper.Catalogs) Paths(java.nio.file.Paths) Map(java.util.Map) Function(com.flink.platform.common.job.Function) SqlContext(com.flink.platform.common.job.SqlContext) ExecuteSqls(com.flink.platform.core.helper.ExecuteSqls) SqlType(com.flink.platform.common.enums.SqlType) ExecutionEnvs(com.flink.platform.core.helper.ExecutionEnvs) Path(java.nio.file.Path) Function(com.flink.platform.common.job.Function) SqlContext(com.flink.platform.common.job.SqlContext) TableEnvironment(org.apache.flink.table.api.TableEnvironment) Catalog(com.flink.platform.common.job.Catalog) Sql(com.flink.platform.common.job.Sql)

Example 2 with Sql

use of com.flink.platform.common.job.Sql in project flink-platform-backend by itinycheng.

the class SqlContextHelper method toSqls.

public List<Sql> toSqls(String subject) {
    subject = subject.trim();
    if (!subject.endsWith(SEMICOLON)) {
        subject = subject + SEMICOLON;
    }
    List<Sql> sqlList = new ArrayList<>();
    Matcher matcher = SQL_PATTERN.matcher(subject);
    while (matcher.find()) {
        String statement = matcher.group();
        sqlList.add(SqlType.parse(statement));
    }
    if (sqlList.size() == 0) {
        throw new JobCommandGenException(String.format("no sql found or parsing failed, subject: %s", subject));
    }
    return sqlList;
}
Also used : Matcher(java.util.regex.Matcher) ArrayList(java.util.ArrayList) JobCommandGenException(com.flink.platform.common.exception.JobCommandGenException) Sql(com.flink.platform.common.job.Sql)

Example 3 with Sql

use of com.flink.platform.common.job.Sql in project flink-platform-backend by itinycheng.

the class JsonUtilTest method testEnum.

@Test
public void testEnum() {
    SqlContext sqlContext = new SqlContext();
    sqlContext.setId("a");
    Sql sql = new Sql();
    sql.setType(SqlType.INSERT_INTO);
    sql.setOperands(new String[] { "insert into table" });
    sqlContext.setSqls(Collections.singletonList(sql));
    sqlContext.setExecMode(ExecutionMode.BATCH);
    String jsonString = JsonUtil.toJsonString(sqlContext);
    System.out.println(jsonString);
    SqlContext sqlContext1 = JsonUtil.toBean(jsonString, SqlContext.class);
    System.out.println(sqlContext1);
}
Also used : SqlContext(com.flink.platform.common.job.SqlContext) Sql(com.flink.platform.common.job.Sql) Test(org.junit.Test)

Aggregations

Sql (com.flink.platform.common.job.Sql)3 SqlContext (com.flink.platform.common.job.SqlContext)2 SqlType (com.flink.platform.common.enums.SqlType)1 JobCommandGenException (com.flink.platform.common.exception.JobCommandGenException)1 Catalog (com.flink.platform.common.job.Catalog)1 Function (com.flink.platform.common.job.Function)1 JsonUtil (com.flink.platform.common.util.JsonUtil)1 ConfigLoader (com.flink.platform.core.common.ConfigLoader)1 Catalogs (com.flink.platform.core.helper.Catalogs)1 Configurations (com.flink.platform.core.helper.Configurations)1 ExecuteSqls (com.flink.platform.core.helper.ExecuteSqls)1 ExecutionEnvs (com.flink.platform.core.helper.ExecutionEnvs)1 Functions (com.flink.platform.core.helper.Functions)1 Path (java.nio.file.Path)1 Paths (java.nio.file.Paths)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 Map (java.util.Map)1 Matcher (java.util.regex.Matcher)1 Collectors.toList (java.util.stream.Collectors.toList)1