Search in sources :

Example 1 with WtNode

use of org.sweble.wikitext.parser.nodes.WtNode in project wikivoyage-listings by baturin.

the class ArticleParser method parsePage.

/**
 * Parse single Wikivoyage page, look for listings, put them into list of POIs
 * @param article Name of Wikivoyage article
 * @param text Wikivoyage page as string
 */
public List<Listing> parsePage(String article, String text) {
    log.debug("Start: parse article '" + article + "'");
    LinkedList<Listing> pois = new LinkedList<>();
    try {
        ParserConfig config = new SimpleParserConfig();
        WikitextPreprocessor p = new WikitextPreprocessor(config);
        WtNode node = p.parseArticle(text, "");
        processNode(article, node, pois);
    } catch (Exception e) {
        System.err.println("Failure");
        e.printStackTrace();
    }
    log.debug("End: parse article '" + article + "'");
    return pois;
}
Also used : WtNode(org.sweble.wikitext.parser.nodes.WtNode) Listing(org.wikivoyage.listings.entity.Listing) WikitextPreprocessor(org.sweble.wikitext.parser.WikitextPreprocessor) SimpleParserConfig(org.sweble.wikitext.parser.utils.SimpleParserConfig) LinkedList(java.util.LinkedList) SimpleParserConfig(org.sweble.wikitext.parser.utils.SimpleParserConfig) ParserConfig(org.sweble.wikitext.parser.ParserConfig) StringConversionException(org.sweble.wikitext.parser.utils.StringConversionException)

Example 2 with WtNode

use of org.sweble.wikitext.parser.nodes.WtNode in project wikivoyage-listings by baturin.

the class TemplateNode method parseArguments.

/**
 * Parse template arguments into a dictionary
 */
private void parseArguments() {
    namedArguments = new LinkedHashMap<>();
    positionalArguments = new LinkedList<>();
    for (WtNode templateArgumentsChildNode : node.getArgs()) {
        if (templateArgumentsChildNode instanceof WtTemplateArgument) {
            WtTemplateArgument templateArgument = (WtTemplateArgument) templateArgumentsChildNode;
            String name = convertWtNodeToString(templateArgument.getName()).trim();
            String value = convertWtNodeToString(templateArgument.getValue()).trim();
            if (name.equals("")) {
                positionalArguments.add(value);
            } else {
                namedArguments.put(name, value);
            }
        }
    }
}
Also used : WtNode(org.sweble.wikitext.parser.nodes.WtNode) WtTemplateArgument(org.sweble.wikitext.parser.nodes.WtTemplateArgument)

Example 3 with WtNode

use of org.sweble.wikitext.parser.nodes.WtNode in project wikivoyage-listings by baturin.

the class TemplateNode method convertWtNodeToString.

/**
 * Text conversion of WtNode object to string.
 */
private String convertWtNodeToString(WtNode node) {
    if (node instanceof WtTemplate) {
        TemplateNode templateNode = new TemplateNode(languageCode, (WtTemplate) node, templateConverters);
        for (TemplateToStringConverter parser : templateConverters) {
            if (templateNode.getNameLowercase().equals(parser.getTemplateName())) {
                return parser.convertToString(templateNode);
            }
        }
        log.debug("Template '" + templateNode.getName() + "' was not parsed");
        UnrecognizeTemplateCounter.getInstance().addUnrecognizedTemplate(languageCode, templateNode.getName());
        return WtRtDataPrinter.print(node);
    } else if (node instanceof WtXmlComment) {
        // HTML comments inside listings are ignored
        return "";
    } else if (node instanceof AstStringNode) {
        return ((AstStringNode) node).getContent().replaceAll("\\[\\[([^|\\]]*?\\||)([^|\\]]*?)\\]\\]", "$2");
    } else {
        String s = "";
        for (WtNode childNode : node) {
            s += convertWtNodeToString(childNode);
        }
        return s;
    }
}
Also used : WtNode(org.sweble.wikitext.parser.nodes.WtNode) WtXmlComment(org.sweble.wikitext.parser.nodes.WtXmlComment) WtTemplate(org.sweble.wikitext.parser.nodes.WtTemplate) AstStringNode(de.fau.cs.osr.ptk.common.ast.AstStringNode)

Aggregations

WtNode (org.sweble.wikitext.parser.nodes.WtNode)3 AstStringNode (de.fau.cs.osr.ptk.common.ast.AstStringNode)1 LinkedList (java.util.LinkedList)1 ParserConfig (org.sweble.wikitext.parser.ParserConfig)1 WikitextPreprocessor (org.sweble.wikitext.parser.WikitextPreprocessor)1 WtTemplate (org.sweble.wikitext.parser.nodes.WtTemplate)1 WtTemplateArgument (org.sweble.wikitext.parser.nodes.WtTemplateArgument)1 WtXmlComment (org.sweble.wikitext.parser.nodes.WtXmlComment)1 SimpleParserConfig (org.sweble.wikitext.parser.utils.SimpleParserConfig)1 StringConversionException (org.sweble.wikitext.parser.utils.StringConversionException)1 Listing (org.wikivoyage.listings.entity.Listing)1