Search in sources :

Example 1 with SqlNode

use of org.apache.calcite.sql.SqlNode in project storm by apache.

the class QueryPlanner method validateAndConvert.

private RelNode validateAndConvert(SqlNode sqlNode) throws ValidationException, RelConversionException {
    SqlNode validated = validateNode(sqlNode);
    RelNode relNode = convertToRelNode(validated);
    return convertToStormRel(relNode);
}
Also used : RelNode(org.apache.calcite.rel.RelNode) SqlNode(org.apache.calcite.sql.SqlNode)

Example 2 with SqlNode

use of org.apache.calcite.sql.SqlNode in project storm by apache.

the class QueryPlanner method validateNode.

private SqlNode validateNode(SqlNode sqlNode) throws ValidationException {
    SqlNode validatedSqlNode = planner.validate(sqlNode);
    validatedSqlNode.accept(new UnsupportedOperatorsVisitor());
    return validatedSqlNode;
}
Also used : UnsupportedOperatorsVisitor(org.apache.storm.sql.planner.UnsupportedOperatorsVisitor) SqlNode(org.apache.calcite.sql.SqlNode)

Example 3 with SqlNode

use of org.apache.calcite.sql.SqlNode in project storm by apache.

the class TestCompilerUtils method sqlOverDummyTable.

public static CalciteState sqlOverDummyTable(String sql) throws RelConversionException, ValidationException, SqlParseException {
    SchemaPlus schema = Frameworks.createRootSchema(true);
    JavaTypeFactory typeFactory = new JavaTypeFactoryImpl(RelDataTypeSystem.DEFAULT);
    StreamableTable streamableTable = new CompilerUtil.TableBuilderInfo(typeFactory).field("ID", SqlTypeName.INTEGER).field("NAME", typeFactory.createType(String.class)).field("ADDR", typeFactory.createType(String.class)).build();
    Table table = streamableTable.stream();
    schema.add("FOO", table);
    schema.add("BAR", table);
    schema.add("MYPLUS", ScalarFunctionImpl.create(MyPlus.class, "eval"));
    List<SqlOperatorTable> sqlOperatorTables = new ArrayList<>();
    sqlOperatorTables.add(SqlStdOperatorTable.instance());
    sqlOperatorTables.add(new CalciteCatalogReader(CalciteSchema.from(schema), false, Collections.<String>emptyList(), typeFactory));
    SqlOperatorTable chainedSqlOperatorTable = new ChainedSqlOperatorTable(sqlOperatorTables);
    FrameworkConfig config = Frameworks.newConfigBuilder().defaultSchema(schema).operatorTable(chainedSqlOperatorTable).build();
    Planner planner = Frameworks.getPlanner(config);
    SqlNode parse = planner.parse(sql);
    SqlNode validate = planner.validate(parse);
    RelNode tree = planner.convert(validate);
    System.out.println(RelOptUtil.toString(tree, SqlExplainLevel.ALL_ATTRIBUTES));
    return new CalciteState(schema, tree);
}
Also used : SqlOperatorTable(org.apache.calcite.sql.SqlOperatorTable) Table(org.apache.calcite.schema.Table) ChainedSqlOperatorTable(org.apache.calcite.sql.util.ChainedSqlOperatorTable) SqlStdOperatorTable(org.apache.calcite.sql.fun.SqlStdOperatorTable) StreamableTable(org.apache.calcite.schema.StreamableTable) SchemaPlus(org.apache.calcite.schema.SchemaPlus) ArrayList(java.util.ArrayList) StreamableTable(org.apache.calcite.schema.StreamableTable) ChainedSqlOperatorTable(org.apache.calcite.sql.util.ChainedSqlOperatorTable) CalciteCatalogReader(org.apache.calcite.prepare.CalciteCatalogReader) RelNode(org.apache.calcite.rel.RelNode) JavaTypeFactoryImpl(org.apache.calcite.jdbc.JavaTypeFactoryImpl) SqlOperatorTable(org.apache.calcite.sql.SqlOperatorTable) ChainedSqlOperatorTable(org.apache.calcite.sql.util.ChainedSqlOperatorTable) JavaTypeFactory(org.apache.calcite.adapter.java.JavaTypeFactory) Planner(org.apache.calcite.tools.Planner) FrameworkConfig(org.apache.calcite.tools.FrameworkConfig) SqlNode(org.apache.calcite.sql.SqlNode)

Example 4 with SqlNode

use of org.apache.calcite.sql.SqlNode in project storm by apache.

the class StormSqlImpl method explain.

@Override
public void explain(Iterable<String> statements) throws Exception {
    Map<String, ISqlTridentDataSource> dataSources = new HashMap<>();
    for (String sql : statements) {
        StormParser parser = new StormParser(sql);
        SqlNode node = parser.impl().parseSqlStmtEof();
        System.out.println("===========================================================");
        System.out.println("query>");
        System.out.println(sql);
        System.out.println("-----------------------------------------------------------");
        if (node instanceof SqlCreateTable) {
            handleCreateTableForTrident((SqlCreateTable) node, dataSources);
            System.out.println("No plan presented on DDL");
        } else if (node instanceof SqlCreateFunction) {
            handleCreateFunction((SqlCreateFunction) node);
            System.out.println("No plan presented on DDL");
        } else {
            FrameworkConfig config = buildFrameWorkConfig();
            Planner planner = Frameworks.getPlanner(config);
            SqlNode parse = planner.parse(sql);
            SqlNode validate = planner.validate(parse);
            RelNode tree = planner.convert(validate);
            String plan = StormRelUtils.explain(tree, SqlExplainLevel.ALL_ATTRIBUTES);
            System.out.println("plan>");
            System.out.println(plan);
        }
        System.out.println("===========================================================");
    }
}
Also used : RelNode(org.apache.calcite.rel.RelNode) HashMap(java.util.HashMap) SqlCreateFunction(org.apache.storm.sql.parser.SqlCreateFunction) ISqlTridentDataSource(org.apache.storm.sql.runtime.ISqlTridentDataSource) Planner(org.apache.calcite.tools.Planner) QueryPlanner(org.apache.storm.sql.planner.trident.QueryPlanner) FrameworkConfig(org.apache.calcite.tools.FrameworkConfig) StormParser(org.apache.storm.sql.parser.StormParser) SqlCreateTable(org.apache.storm.sql.parser.SqlCreateTable) SqlNode(org.apache.calcite.sql.SqlNode)

Example 5 with SqlNode

use of org.apache.calcite.sql.SqlNode in project storm by apache.

the class StormSqlImpl method submit.

@Override
public void submit(String name, Iterable<String> statements, Map<String, ?> stormConf, SubmitOptions opts, StormSubmitter.ProgressListener progressListener, String asUser) throws Exception {
    Map<String, ISqlTridentDataSource> dataSources = new HashMap<>();
    for (String sql : statements) {
        StormParser parser = new StormParser(sql);
        SqlNode node = parser.impl().parseSqlStmtEof();
        if (node instanceof SqlCreateTable) {
            handleCreateTableForTrident((SqlCreateTable) node, dataSources);
        } else if (node instanceof SqlCreateFunction) {
            handleCreateFunction((SqlCreateFunction) node);
        } else {
            QueryPlanner planner = new QueryPlanner(schema);
            AbstractTridentProcessor processor = planner.compile(dataSources, sql);
            TridentTopology topo = processor.build();
            Path jarPath = null;
            try {
                // QueryPlanner on Trident mode configures the topology with compiled classes,
                // so we need to add new classes into topology jar
                // Topology will be serialized and sent to Nimbus, and deserialized and executed in workers.
                jarPath = Files.createTempFile("storm-sql", ".jar");
                System.setProperty("storm.jar", jarPath.toString());
                packageTopology(jarPath, processor);
                StormSubmitter.submitTopologyAs(name, stormConf, topo.build(), opts, progressListener, asUser);
            } finally {
                if (jarPath != null) {
                    Files.delete(jarPath);
                }
            }
        }
    }
}
Also used : Path(java.nio.file.Path) HashMap(java.util.HashMap) TridentTopology(org.apache.storm.trident.TridentTopology) SqlCreateFunction(org.apache.storm.sql.parser.SqlCreateFunction) ISqlTridentDataSource(org.apache.storm.sql.runtime.ISqlTridentDataSource) StormParser(org.apache.storm.sql.parser.StormParser) SqlCreateTable(org.apache.storm.sql.parser.SqlCreateTable) QueryPlanner(org.apache.storm.sql.planner.trident.QueryPlanner) SqlNode(org.apache.calcite.sql.SqlNode)

Aggregations

SqlNode (org.apache.calcite.sql.SqlNode)23 RelNode (org.apache.calcite.rel.RelNode)6 SqlIdentifier (org.apache.calcite.sql.SqlIdentifier)5 FrameworkConfig (org.apache.calcite.tools.FrameworkConfig)5 Planner (org.apache.calcite.tools.Planner)5 SchemaPlus (org.apache.calcite.schema.SchemaPlus)4 SqlSelect (org.apache.calcite.sql.SqlSelect)4 HashMap (java.util.HashMap)3 SqlNodeList (org.apache.calcite.sql.SqlNodeList)3 RelConversionException (org.apache.calcite.tools.RelConversionException)3 SqlCreateFunction (org.apache.storm.sql.parser.SqlCreateFunction)3 SqlCreateTable (org.apache.storm.sql.parser.SqlCreateTable)3 StormParser (org.apache.storm.sql.parser.StormParser)3 QueryPlanner (org.apache.storm.sql.planner.trident.QueryPlanner)3 ISqlTridentDataSource (org.apache.storm.sql.runtime.ISqlTridentDataSource)3 ArrayList (java.util.ArrayList)2 JavaTypeFactory (org.apache.calcite.adapter.java.JavaTypeFactory)2 JavaTypeFactoryImpl (org.apache.calcite.jdbc.JavaTypeFactoryImpl)2 CalciteCatalogReader (org.apache.calcite.prepare.CalciteCatalogReader)2 StreamableTable (org.apache.calcite.schema.StreamableTable)2