Examples with Lexer org.eclipse.xtext.parser.antlr.Lexer used on opensource projects

Example 11 with Lexer

use of org.eclipse.xtext.parser.antlr.Lexer in project xtext-xtend by eclipse.

the class AbstractSmokeTest method testSkipTokensInBetween.

@Test
public void testSkipTokensInBetween() throws Exception {
    for (String string : smokeTestModels) {
        List<CommonToken> tokenList = Lists.newArrayList();
        {
            Lexer lexer = lexerProvider.get();
            lexer.setCharStream(new ANTLRStringStream(string));
            Token token = lexer.nextToken();
            while (token != Token.EOF_TOKEN) {
                tokenList.add((CommonToken) token);
                token = lexer.nextToken();
            }
        }
        for (CommonToken token : tokenList) {
            int start = token.getStartIndex();
            int length = token.getText().length();
            logProgress(token);
            doParseAndCheckForSmoke(string.substring(0, start) + string.substring(start + length));
        }
    }
}

Example 12 with Lexer

use of org.eclipse.xtext.parser.antlr.Lexer in project xtext-xtend by eclipse.

the class PresentationDamagerTest method createDocument.

@Override
public Document createDocument(final String before) throws Exception {
    XtextDocument _xblockexpression = null;
    {
        XtendDocumentTokenSource _xtendDocumentTokenSource = new XtendDocumentTokenSource();
        final Procedure1<XtendDocumentTokenSource> _function = (XtendDocumentTokenSource it) -> {
            AntlrTokenDefProvider _antlrTokenDefProvider = new AntlrTokenDefProvider();
            final Procedure1<AntlrTokenDefProvider> _function_1 = (AntlrTokenDefProvider it_1) -> {
                XtendAntlrTokenFileProvider _xtendAntlrTokenFileProvider = new XtendAntlrTokenFileProvider();
                it_1.setAntlrTokenFileProvider(_xtendAntlrTokenFileProvider);
            };
            AntlrTokenDefProvider _doubleArrow = ObjectExtensions.<AntlrTokenDefProvider>operator_doubleArrow(_antlrTokenDefProvider, _function_1);
            it.setTokenDefProvider(_doubleArrow);
            final Provider<Lexer> _function_2 = () -> {
                return this.createLexer();
            };
            it.setLexer(_function_2);
            it.setFlexerFactory(this.flexerFactory);
        };
        final XtendDocumentTokenSource source = ObjectExtensions.<XtendDocumentTokenSource>operator_doubleArrow(_xtendDocumentTokenSource, _function);
        final XtextDocument document = new XtextDocument(source, null, this.outdatedStateManager, this.operationCanceledManager);
        document.set(before);
        _xblockexpression = document;
    }
    return _xblockexpression;
}

Example 13 with Lexer

use of org.eclipse.xtext.parser.antlr.Lexer in project xtext-core by eclipse.

the class AbstractLexerBasedConverter method getTokenSource.

protected TokenSource getTokenSource(String escapedValue) {
    Lexer result = getLexer();
    if (result == null)
        return null;
    result.setCharStream(new ANTLRStringStream(escapedValue));
    return result;
}