Search in sources :

Example 1 with HTMLParserListener

use of com.gargoylesoftware.htmlunit.html.HTMLParserListener in project yacy_grid_loader by yacy.

the class HtmlUnitLoader method initClient.

public static void initClient(String userAgent) {
    userAgentDefault = userAgent;
    if (webClient != null) {
        webClient.getCache().clear();
        webClient.close();
    }
    webClient = new WebClient(getBrowser(userAgent));
    WebClientOptions options = webClient.getOptions();
    options.setJavaScriptEnabled(true);
    options.setCssEnabled(false);
    options.setPopupBlockerEnabled(true);
    options.setRedirectEnabled(true);
    options.setThrowExceptionOnScriptError(false);
    options.setDownloadImages(false);
    options.setGeolocationEnabled(false);
    options.setPrintContentOnFailingStatusCode(false);
    // this might be a bit large, is regulated with throttling and client cache clear in short memory status
    webClient.getCache().setMaxSize(10000);
    webClient.setIncorrectnessListener(new IncorrectnessListener() {

        @Override
        public void notify(String arg0, Object arg1) {
        }
    });
    webClient.setCssErrorHandler(new ErrorHandler() {

        @Override
        public void warning(CSSParseException exception) throws CSSException {
        }

        @Override
        public void fatalError(CSSParseException exception) throws CSSException {
        }

        @Override
        public void error(CSSParseException exception) throws CSSException {
        }
    });
    webClient.setJavaScriptErrorListener(new JavaScriptErrorListener() {

        @Override
        public void timeoutError(HtmlPage arg0, long arg1, long arg2) {
        }

        @Override
        public void scriptException(HtmlPage arg0, ScriptException arg1) {
        }

        @Override
        public void malformedScriptURL(HtmlPage arg0, String arg1, MalformedURLException arg2) {
        }

        @Override
        public void loadScriptError(HtmlPage arg0, URL arg1, Exception arg2) {
        }
    });
    webClient.setHTMLParserListener(new HTMLParserListener() {

        @Override
        public void error(String message, URL url, String html, int line, int column, String key) {
        }

        @Override
        public void warning(String message, URL url, String html, int line, int column, String key) {
        }
    });
}
Also used : HTMLParserListener(com.gargoylesoftware.htmlunit.html.HTMLParserListener) ErrorHandler(org.w3c.css.sac.ErrorHandler) WebClientOptions(com.gargoylesoftware.htmlunit.WebClientOptions) MalformedURLException(java.net.MalformedURLException) HtmlPage(com.gargoylesoftware.htmlunit.html.HtmlPage) WebClient(com.gargoylesoftware.htmlunit.WebClient) IncorrectnessListener(com.gargoylesoftware.htmlunit.IncorrectnessListener) URL(java.net.URL) MalformedURLException(java.net.MalformedURLException) CSSException(org.w3c.css.sac.CSSException) CSSParseException(org.w3c.css.sac.CSSParseException) IOException(java.io.IOException) ScriptException(com.gargoylesoftware.htmlunit.ScriptException) JavaScriptErrorListener(com.gargoylesoftware.htmlunit.javascript.JavaScriptErrorListener) ScriptException(com.gargoylesoftware.htmlunit.ScriptException) CSSParseException(org.w3c.css.sac.CSSParseException) CSSException(org.w3c.css.sac.CSSException)

Aggregations

IncorrectnessListener (com.gargoylesoftware.htmlunit.IncorrectnessListener)1 ScriptException (com.gargoylesoftware.htmlunit.ScriptException)1 WebClient (com.gargoylesoftware.htmlunit.WebClient)1 WebClientOptions (com.gargoylesoftware.htmlunit.WebClientOptions)1 HTMLParserListener (com.gargoylesoftware.htmlunit.html.HTMLParserListener)1 HtmlPage (com.gargoylesoftware.htmlunit.html.HtmlPage)1 JavaScriptErrorListener (com.gargoylesoftware.htmlunit.javascript.JavaScriptErrorListener)1 IOException (java.io.IOException)1 MalformedURLException (java.net.MalformedURLException)1 URL (java.net.URL)1 CSSException (org.w3c.css.sac.CSSException)1 CSSParseException (org.w3c.css.sac.CSSParseException)1 ErrorHandler (org.w3c.css.sac.ErrorHandler)1