Examples with JavaLanguageLexer com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageLexer used on opensource projects

Example 1 with JavaLanguageLexer

use of com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageLexer in project checkstyle by checkstyle.

the class JavaParser method parse.

/**
 * Static helper method to parses a Java source file.
 *
 * @param contents contains the contents of the file
 * @return the root of the AST
 * @throws CheckstyleException if the contents is not a valid Java source
 */
public static DetailAST parse(FileContents contents) throws CheckstyleException {
    final String fullText = contents.getText().getFullText().toString();
    final CharStream codePointCharStream = CharStreams.fromString(fullText);
    final JavaLanguageLexer lexer = new JavaLanguageLexer(codePointCharStream, true);
    lexer.setCommentListener(contents);
    lexer.removeErrorListeners();
    final CommonTokenStream tokenStream = new CommonTokenStream(lexer);
    final JavaLanguageParser parser = new JavaLanguageParser(tokenStream, JavaLanguageParser.CLEAR_DFA_LIMIT);
    parser.setErrorHandler(new CheckstyleParserErrorStrategy());
    parser.removeErrorListeners();
    parser.addErrorListener(new CheckstyleErrorListener());
    final JavaLanguageParser.CompilationUnitContext compilationUnit;
    try {
        compilationUnit = parser.compilationUnit();
    } catch (IllegalStateException ex) {
        final String exceptionMsg = String.format(Locale.ROOT, "%s occurred while parsing file %s.", ex.getClass().getSimpleName(), contents.getFileName());
        throw new CheckstyleException(exceptionMsg, ex);
    }
    return new JavaAstVisitor(tokenStream).visit(compilationUnit);
}

Example 2 with JavaLanguageLexer

use of com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageLexer in project checkstyle by checkstyle.

the class JavaAstVisitorTest method testNoStackOverflowOnDeepStringConcat.

/**
 * This test exists to kill surviving mutation from pitest removing expression AST building
 * optimization in {@link JavaAstVisitor#visitBinOp(JavaLanguageParser.BinOpContext)}.
 * We do not use {@link JavaParser#parse(FileContents)} here due to DFA clearing hack.
 *
 * <p>
 * Reason: we have iterative expression AST building to avoid stackoverflow
 * in {@link JavaAstVisitor#visitBinOp(JavaLanguageParser.BinOpContext)}. In actual
 * generated parser, we avoid stackoverflow thanks to the left recursive expression
 * rule (eliminating unnecessary recursive calls to hierarchical expression production rules).
 * However, ANTLR's ParserATNSimulator has no such optimization. So, the number of recursive
 * calls to ParserATNSimulator#closure when calling ParserATNSimulator#clearDFA causes a
 * StackOverflow error. We avoid this by using the single argument constructor (thus not
 * forcing DFA clearing) in this test.
 * </p>
 *
 * @throws Exception if input file does not exist
 */
@Test
public void testNoStackOverflowOnDeepStringConcat() throws Exception {
    final File file = new File(getPath("InputJavaAstVisitorNoStackOverflowOnDeepStringConcat.java"));
    final FileText fileText = new FileText(file, StandardCharsets.UTF_8.name());
    final FileContents contents = new FileContents(fileText);
    final String fullText = contents.getText().getFullText().toString();
    final CharStream codePointCharStream = CharStreams.fromString(fullText);
    final JavaLanguageLexer lexer = new JavaLanguageLexer(codePointCharStream, true);
    lexer.setCommentListener(contents);
    final CommonTokenStream tokenStream = new CommonTokenStream(lexer);
    final JavaLanguageParser parser = new JavaLanguageParser(tokenStream);
    final JavaLanguageParser.CompilationUnitContext compilationUnit = parser.compilationUnit();
    // We restrict execution to use limited resources here, so that we can
    // kill surviving pitest mutation from removal of nested binary operation
    // optimization in JavaAstVisitor#visitBinOp. Limited resources (small stack size)
    // ensure that we throw a StackOverflowError if optimization is removed.
    final DetailAST root = TestUtil.getResultWithLimitedResources(() -> new JavaAstVisitor(tokenStream).visit(compilationUnit));
    assertWithMessage("File parsing and AST building should complete successfully.").that(root).isNotNull();
}