Search in sources :

Example 76 with CommonTree

use of org.antlr.runtime.tree.CommonTree in project SQLWindowing by hbutani.

the class ParseUtils method parseSelect.

public static SelectSpec parseSelect(String selectExprStr) throws WindowingException {
    Windowing2Lexer lexer;
    CommonTokenStream tokens;
    Windowing2Parser parser = null;
    CommonTree t;
    CommonTreeNodeStream nodes;
    QSpecBuilder2 qSpecBldr = null;
    String err;
    try {
        lexer = new Windowing2Lexer(new ANTLRStringStream(selectExprStr));
        tokens = new CommonTokenStream(lexer);
        parser = new Windowing2Parser(tokens);
        parser.setTreeAdaptor(TranslateUtils.adaptor);
        t = (CommonTree) parser.select().getTree();
        err = parser.getWindowingParseErrors();
        if (err != null) {
            throw new WindowingException(err);
        }
    } catch (WindowingException we) {
        throw we;
    } catch (Throwable te) {
        err = parser.getWindowingParseErrors();
        if (err != null) {
            throw new WindowingException(err);
        }
        throw new WindowingException("Parse Error:" + te.toString(), te);
    }
    TranslateUtils.unescapeStringLiterals((ASTNode) t);
    try {
        nodes = new CommonTreeNodeStream(t);
        nodes.setTokenStream(tokens);
        qSpecBldr = new QSpecBuilder2(nodes);
        SelectSpec selectSpec = qSpecBldr.select();
        err = qSpecBldr.getWindowingParseErrors();
        if (err != null) {
            throw new WindowingException(err);
        }
        return selectSpec;
    } catch (WindowingException we) {
        throw we;
    } catch (Throwable te) {
        err = qSpecBldr.getWindowingParseErrors();
        if (err != null) {
            throw new WindowingException(err);
        }
        throw new WindowingException("Parse Error:" + te.toString(), te);
    }
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) CommonTokenStream(org.antlr.runtime.CommonTokenStream) CommonTree(org.antlr.runtime.tree.CommonTree) WindowingException(com.sap.hadoop.windowing.WindowingException) SelectSpec(com.sap.hadoop.windowing.query2.specification.SelectSpec) CommonTreeNodeStream(org.antlr.runtime.tree.CommonTreeNodeStream)

Example 77 with CommonTree

use of org.antlr.runtime.tree.CommonTree in project SQLWindowing by hbutani.

the class WindowFunctionSpec method toString.

public String toString() {
    StringBuilder buf = new StringBuilder();
    buf.append(name).append("(");
    if (isStar) {
        buf.append("*");
    } else {
        if (isDistinct) {
            buf.append("distinct ");
        }
        if (args != null) {
            boolean first = true;
            for (CommonTree arg : args) {
                if (first)
                    first = false;
                else
                    buf.append(", ");
                buf.append(arg.toStringTree());
            }
        }
    }
    buf.append(")");
    if (windowSpec != null) {
        buf.append(" ").append(windowSpec.toString());
    }
    if (alias != null) {
        buf.append(" as ").append(alias);
    }
    return buf.toString();
}
Also used : CommonTree(org.antlr.runtime.tree.CommonTree)

Example 78 with CommonTree

use of org.antlr.runtime.tree.CommonTree in project binnavi by google.

the class DebuggerMemoryExpressionParser method parse.

/**
   * Parses a single memory expression string.
   *
   * @param memoryExpression The memory expression string to parse.
   *
   * @return The parsed memory expression tree.
   *
   * @throws RecognitionException Thrown if parsing failed.
   */
public static MemoryExpressionElement parse(final String memoryExpression) throws RecognitionException {
    final CharStream charStream = new ANTLRStringStream(memoryExpression);
    final MemoryExpressionLexer lexer = new MemoryExpressionLexer(charStream);
    final CommonTokenStream tokens = new CommonTokenStream();
    tokens.setTokenSource(lexer);
    final MemoryExpressionParser parser = new MemoryExpressionParser(tokens) {

        @Override
        public void recover(final IntStream input, final RecognitionException exception) {
        // Nothing to do
        }

        @Override
        public Object recoverFromMismatchedToken(final IntStream input, final int ttype, final org.antlr.runtime.BitSet follow) throws RecognitionException {
            throw new MismatchedTokenException(ttype, input);
        }

        @Override
        public void reportError(final RecognitionException exception) {
        // Nothing to do
        }
    };
    parser.setTreeAdaptor(adaptor);
    final MemoryExpressionParser.prog_return parserResult = parser.prog();
    final CommonTree ast = (CommonTree) parserResult.getTree();
    return convert(ast);
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) MemoryExpressionParser(com.google.security.zynamics.binnavi.parsers.gotomem.MemoryExpressionParser) CommonTokenStream(org.antlr.runtime.CommonTokenStream) MemoryExpressionLexer(com.google.security.zynamics.binnavi.parsers.gotomem.MemoryExpressionLexer) CommonTree(org.antlr.runtime.tree.CommonTree) MismatchedTokenException(org.antlr.runtime.MismatchedTokenException) IntStream(org.antlr.runtime.IntStream) CharStream(org.antlr.runtime.CharStream) RecognitionException(org.antlr.runtime.RecognitionException)

Example 79 with CommonTree

use of org.antlr.runtime.tree.CommonTree in project smali by JesusFreke.

the class SmaliTestUtils method compileSmali.

public static ClassDef compileSmali(String smaliText, int apiLevel) throws RecognitionException, IOException {
    CommonTokenStream tokens;
    LexerErrorInterface lexer;
    DexBuilder dexBuilder = new DexBuilder(Opcodes.forApi(apiLevel));
    Reader reader = new StringReader(smaliText);
    lexer = new smaliFlexLexer(reader);
    tokens = new CommonTokenStream((TokenSource) lexer);
    smaliParser parser = new smaliParser(tokens);
    parser.setVerboseErrors(true);
    parser.setAllowOdex(false);
    parser.setApiLevel(apiLevel);
    smaliParser.smali_file_return result = parser.smali_file();
    if (parser.getNumberOfSyntaxErrors() > 0 || lexer.getNumberOfSyntaxErrors() > 0) {
        throw new RuntimeException("Error occured while compiling text");
    }
    CommonTree t = result.getTree();
    CommonTreeNodeStream treeStream = new CommonTreeNodeStream(t);
    treeStream.setTokenStream(tokens);
    smaliTreeWalker dexGen = new smaliTreeWalker(treeStream);
    dexGen.setApiLevel(apiLevel);
    dexGen.setVerboseErrors(true);
    dexGen.setDexBuilder(dexBuilder);
    dexGen.smali_file();
    if (dexGen.getNumberOfSyntaxErrors() > 0) {
        throw new RuntimeException("Error occured while compiling text");
    }
    MemoryDataStore dataStore = new MemoryDataStore();
    dexBuilder.writeTo(dataStore);
    DexBackedDexFile dexFile = new DexBackedDexFile(Opcodes.forApi(apiLevel), dataStore.getData());
    return Iterables.getFirst(dexFile.getClasses(), null);
}
Also used : CommonTokenStream(org.antlr.runtime.CommonTokenStream) TokenSource(org.antlr.runtime.TokenSource) DexBackedDexFile(org.jf.dexlib2.dexbacked.DexBackedDexFile) CommonTree(org.antlr.runtime.tree.CommonTree) MemoryDataStore(org.jf.dexlib2.writer.io.MemoryDataStore) Reader(java.io.Reader) StringReader(java.io.StringReader) DexBuilder(org.jf.dexlib2.writer.builder.DexBuilder) StringReader(java.io.StringReader) CommonTreeNodeStream(org.antlr.runtime.tree.CommonTreeNodeStream)

Example 80 with CommonTree

use of org.antlr.runtime.tree.CommonTree in project freud by LMAX-Exchange.

the class JavaParser method arrayInitializer.

// $ANTLR start "arrayInitializer"
// /home/langera/dev/freud/trunk/src/grammar/Java.g:608:1: arrayInitializer : LCURLY ( variableInitializer ( COMMA variableInitializer )* ( COMMA )? )? RCURLY -> ^( ARRAY_INITIALIZER[$LCURLY, \"ARRAY_INITIALIZER\"] ( variableInitializer )* ) ;
public final JavaParser.arrayInitializer_return arrayInitializer() throws RecognitionException {
    JavaParser.arrayInitializer_return retval = new JavaParser.arrayInitializer_return();
    retval.start = input.LT(1);
    int arrayInitializer_StartIndex = input.index();
    CommonTree root_0 = null;
    Token LCURLY143 = null;
    Token COMMA145 = null;
    Token COMMA147 = null;
    Token RCURLY148 = null;
    JavaParser.variableInitializer_return variableInitializer144 = null;
    JavaParser.variableInitializer_return variableInitializer146 = null;
    CommonTree LCURLY143_tree = null;
    CommonTree COMMA145_tree = null;
    CommonTree COMMA147_tree = null;
    CommonTree RCURLY148_tree = null;
    RewriteRuleTokenStream stream_LCURLY = new RewriteRuleTokenStream(adaptor, "token LCURLY");
    RewriteRuleTokenStream stream_COMMA = new RewriteRuleTokenStream(adaptor, "token COMMA");
    RewriteRuleTokenStream stream_RCURLY = new RewriteRuleTokenStream(adaptor, "token RCURLY");
    RewriteRuleSubtreeStream stream_variableInitializer = new RewriteRuleSubtreeStream(adaptor, "rule variableInitializer");
    try {
        if (state.backtracking > 0 && alreadyParsedRule(input, 35)) {
            return retval;
        }
        // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:5: ( LCURLY ( variableInitializer ( COMMA variableInitializer )* ( COMMA )? )? RCURLY -> ^( ARRAY_INITIALIZER[$LCURLY, \"ARRAY_INITIALIZER\"] ( variableInitializer )* ) )
        // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:9: LCURLY ( variableInitializer ( COMMA variableInitializer )* ( COMMA )? )? RCURLY
        {
            LCURLY143 = (Token) match(input, LCURLY, FOLLOW_LCURLY_in_arrayInitializer6527);
            if (state.failed) {
                return retval;
            }
            if (state.backtracking == 0) {
                stream_LCURLY.add(LCURLY143);
            }
            // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:16: ( variableInitializer ( COMMA variableInitializer )* ( COMMA )? )?
            int alt53 = 2;
            int LA53_0 = input.LA(1);
            if ((LA53_0 == DEC || LA53_0 == INC || LA53_0 == LCURLY || LA53_0 == LESS_THAN || LA53_0 == LOGICAL_NOT || (LA53_0 >= LPAREN && LA53_0 <= MINUS) || LA53_0 == NOT || LA53_0 == PLUS || LA53_0 == BOOLEAN || LA53_0 == BYTE || LA53_0 == CHAR || LA53_0 == DOUBLE || LA53_0 == FALSE || LA53_0 == FLOAT || (LA53_0 >= INT && LA53_0 <= LONG) || (LA53_0 >= NEW && LA53_0 <= NULL) || LA53_0 == SHORT || LA53_0 == SUPER || LA53_0 == THIS || LA53_0 == TRUE || LA53_0 == VOID || (LA53_0 >= IDENT && LA53_0 <= STRING_LITERAL))) {
                alt53 = 1;
            }
            switch(alt53) {
                case 1:
                    // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:17: variableInitializer ( COMMA variableInitializer )* ( COMMA )?
                    {
                        pushFollow(FOLLOW_variableInitializer_in_arrayInitializer6530);
                        variableInitializer144 = variableInitializer();
                        state._fsp--;
                        if (state.failed) {
                            return retval;
                        }
                        if (state.backtracking == 0) {
                            stream_variableInitializer.add(variableInitializer144.getTree());
                        }
                        // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:37: ( COMMA variableInitializer )*
                        loop51: do {
                            int alt51 = 2;
                            int LA51_0 = input.LA(1);
                            if ((LA51_0 == COMMA)) {
                                int LA51_1 = input.LA(2);
                                if ((LA51_1 == DEC || LA51_1 == INC || LA51_1 == LCURLY || LA51_1 == LESS_THAN || LA51_1 == LOGICAL_NOT || (LA51_1 >= LPAREN && LA51_1 <= MINUS) || LA51_1 == NOT || LA51_1 == PLUS || LA51_1 == BOOLEAN || LA51_1 == BYTE || LA51_1 == CHAR || LA51_1 == DOUBLE || LA51_1 == FALSE || LA51_1 == FLOAT || (LA51_1 >= INT && LA51_1 <= LONG) || (LA51_1 >= NEW && LA51_1 <= NULL) || LA51_1 == SHORT || LA51_1 == SUPER || LA51_1 == THIS || LA51_1 == TRUE || LA51_1 == VOID || (LA51_1 >= IDENT && LA51_1 <= STRING_LITERAL))) {
                                    alt51 = 1;
                                }
                            }
                            switch(alt51) {
                                case 1:
                                    // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:38: COMMA variableInitializer
                                    {
                                        COMMA145 = (Token) match(input, COMMA, FOLLOW_COMMA_in_arrayInitializer6533);
                                        if (state.failed) {
                                            return retval;
                                        }
                                        if (state.backtracking == 0) {
                                            stream_COMMA.add(COMMA145);
                                        }
                                        pushFollow(FOLLOW_variableInitializer_in_arrayInitializer6535);
                                        variableInitializer146 = variableInitializer();
                                        state._fsp--;
                                        if (state.failed) {
                                            return retval;
                                        }
                                        if (state.backtracking == 0) {
                                            stream_variableInitializer.add(variableInitializer146.getTree());
                                        }
                                    }
                                    break;
                                default:
                                    break loop51;
                            }
                        } while (true);
                        // /home/langera/dev/freud/trunk/src/grammar/Java.g:609:66: ( COMMA )?
                        int alt52 = 2;
                        int LA52_0 = input.LA(1);
                        if ((LA52_0 == COMMA)) {
                            alt52 = 1;
                        }
                        switch(alt52) {
                            case 1:
                                // /home/langera/dev/freud/trunk/src/grammar/Java.g:0:0: COMMA
                                {
                                    COMMA147 = (Token) match(input, COMMA, FOLLOW_COMMA_in_arrayInitializer6539);
                                    if (state.failed) {
                                        return retval;
                                    }
                                    if (state.backtracking == 0) {
                                        stream_COMMA.add(COMMA147);
                                    }
                                }
                                break;
                        }
                    }
                    break;
            }
            RCURLY148 = (Token) match(input, RCURLY, FOLLOW_RCURLY_in_arrayInitializer6544);
            if (state.failed) {
                return retval;
            }
            if (state.backtracking == 0) {
                stream_RCURLY.add(RCURLY148);
            }
            // wildcard labels:
            if (state.backtracking == 0) {
                retval.tree = root_0;
                RewriteRuleSubtreeStream stream_retval = new RewriteRuleSubtreeStream(adaptor, "rule retval", retval != null ? retval.tree : null);
                root_0 = (CommonTree) adaptor.nil();
                // 610:9: -> ^( ARRAY_INITIALIZER[$LCURLY, \"ARRAY_INITIALIZER\"] ( variableInitializer )* )
                {
                    // /home/langera/dev/freud/trunk/src/grammar/Java.g:610:13: ^( ARRAY_INITIALIZER[$LCURLY, \"ARRAY_INITIALIZER\"] ( variableInitializer )* )
                    {
                        CommonTree root_1 = (CommonTree) adaptor.nil();
                        root_1 = (CommonTree) adaptor.becomeRoot((CommonTree) adaptor.create(ARRAY_INITIALIZER, LCURLY143, "ARRAY_INITIALIZER"), root_1);
                        // /home/langera/dev/freud/trunk/src/grammar/Java.g:610:63: ( variableInitializer )*
                        while (stream_variableInitializer.hasNext()) {
                            adaptor.addChild(root_1, stream_variableInitializer.nextTree());
                        }
                        stream_variableInitializer.reset();
                        adaptor.addChild(root_0, root_1);
                    }
                }
                retval.tree = root_0;
            }
        }
        retval.stop = input.LT(-1);
        if (state.backtracking == 0) {
            retval.tree = (CommonTree) adaptor.rulePostProcessing(root_0);
            adaptor.setTokenBoundaries(retval.tree, retval.start, retval.stop);
        }
    } catch (RecognitionException re) {
        reportError(re);
        recover(input, re);
        retval.tree = (CommonTree) adaptor.errorNode(input, retval.start, input.LT(-1), re);
    } finally {
        if (state.backtracking > 0) {
            memoize(input, 35, arrayInitializer_StartIndex);
        }
    }
    return retval;
}
Also used : RewriteRuleTokenStream(org.antlr.runtime.tree.RewriteRuleTokenStream) CommonTree(org.antlr.runtime.tree.CommonTree) RewriteRuleSubtreeStream(org.antlr.runtime.tree.RewriteRuleSubtreeStream) Token(org.antlr.runtime.Token) RecognitionException(org.antlr.runtime.RecognitionException)

Aggregations

CommonTree (org.antlr.runtime.tree.CommonTree)155 RecognitionException (org.antlr.runtime.RecognitionException)132 Token (org.antlr.runtime.Token)106 RewriteRuleSubtreeStream (org.antlr.runtime.tree.RewriteRuleSubtreeStream)81 RewriteRuleTokenStream (org.antlr.runtime.tree.RewriteRuleTokenStream)62 NoViableAltException (org.antlr.runtime.NoViableAltException)33 RewriteEarlyExitException (org.antlr.runtime.tree.RewriteEarlyExitException)15 CommonTokenStream (org.antlr.runtime.CommonTokenStream)12 CommonTreeNodeStream (org.antlr.runtime.tree.CommonTreeNodeStream)8 ANTLRStringStream (org.antlr.runtime.ANTLRStringStream)6 EarlyExitException (org.antlr.runtime.EarlyExitException)4 MismatchedSetException (org.antlr.runtime.MismatchedSetException)4 WindowingException (com.sap.hadoop.windowing.WindowingException)3 FileInputStream (java.io.FileInputStream)3 InputStreamReader (java.io.InputStreamReader)3 CharStream (org.antlr.runtime.CharStream)3 Tree (org.antlr.runtime.tree.Tree)3 ArrayList (java.util.ArrayList)2 HashSet (java.util.HashSet)2 TokenSource (org.antlr.runtime.TokenSource)2