Search in sources :

Example 1 with NicelyResynchronizingAjaxController

use of com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController in project saga by timurstrekalov.

the class WebClientFactory method newInstance.

// New version allows for creation of web drivers without the need for a config object
// which may not be readily available if the driver is created by third-party code.
public static WebClient newInstance(final BrowserVersion version) {
    final WebClient client = new WebClient(version) {

        @Override
        public WebResponse loadWebResponse(final WebRequest webRequest) throws IOException {
            return new WebResponseProxy(super.loadWebResponse(webRequest));
        }
    };
    client.setIncorrectnessListener(quietIncorrectnessListener);
    client.setJavaScriptErrorListener(loggingJsErrorListener);
    client.setHTMLParserListener(quietHtmlParserListener);
    client.setCssErrorHandler(quietCssErrorHandler);
    client.getOptions().setJavaScriptEnabled(true);
    client.setAjaxController(new NicelyResynchronizingAjaxController());
    client.getOptions().setThrowExceptionOnScriptError(false);
    client.getOptions().setThrowExceptionOnFailingStatusCode(false);
    client.getOptions().setPrintContentOnFailingStatusCode(false);
    client.setWebConnection(new HttpWebConnection(client) {

        @Override
        protected WebResponse newWebResponseInstance(final WebResponseData responseData, final long loadTime, final WebRequest request) {
            return new WebResponseProxy(super.newWebResponseInstance(responseData, loadTime, request));
        }
    });
    return client;
}
Also used : WebResponse(com.gargoylesoftware.htmlunit.WebResponse) WebRequest(com.gargoylesoftware.htmlunit.WebRequest) NicelyResynchronizingAjaxController(com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController) HttpWebConnection(com.gargoylesoftware.htmlunit.HttpWebConnection) WebResponseData(com.gargoylesoftware.htmlunit.WebResponseData) WebClient(com.gargoylesoftware.htmlunit.WebClient)

Example 2 with NicelyResynchronizingAjaxController

use of com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController in project yyl_example by Relucent.

the class HtmlUnitTest2 method main.

public static void main(String[] args) throws Exception {
    WebClient webClient = new WebClient(BrowserVersion.FIREFOX_68);
    webClient.setCssErrorHandler(new SilentCssErrorHandler());
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());
    webClient.getOptions().setCssEnabled(true);
    webClient.getOptions().setRedirectEnabled(false);
    webClient.getOptions().setAppletEnabled(false);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setPopupBlockerEnabled(true);
    webClient.getOptions().setTimeout(10000);
    // JS运行错误时,是否抛出异常
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    HtmlPage page = webClient.getPage("http://huaban.com/favorite/home/");
    System.out.println(page.asXml());
    webClient.close();
}
Also used : HtmlPage(com.gargoylesoftware.htmlunit.html.HtmlPage) NicelyResynchronizingAjaxController(com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController) SilentCssErrorHandler(com.gargoylesoftware.htmlunit.SilentCssErrorHandler) WebClient(com.gargoylesoftware.htmlunit.WebClient)

Example 3 with NicelyResynchronizingAjaxController

use of com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController in project yyl_example by Relucent.

the class HtmlUnitTest1 method main.

public static void main(String[] args) {
    // 得到浏览器对象
    WebClient webClient = null;
    try {
        webClient = new WebClient();
        webClient.waitForBackgroundJavaScript(60 * 1000);
        WebClientOptions options = webClient.getOptions();
        // 不加载css
        options.setCssEnabled(true);
        // 启用JS解释器,默认为true
        options.setJavaScriptEnabled(false);
        options.setUseInsecureSSL(true);
        // options.setRedirectEnabled(true);
        // 当出现HttpError时,是否抛出异常
        options.setThrowExceptionOnFailingStatusCode(false);
        // JS运行错误时,是否抛出异常
        options.setThrowExceptionOnScriptError(false);
        // 设置AJAX异步处理控制器(启用AJAX支持)
        webClient.setAjaxController(new NicelyResynchronizingAjaxController());
        // 拿到网页
        HtmlPage htmlpage = webClient.getPage("http://news.baidu.com/advanced_news.html");
        // 根据名字得到表单(表单的名字叫“f”)
        HtmlForm form = htmlpage.getFormByName("f");
        System.out.println(form);
        // 获取“百度一下”这个按钮
        HtmlSubmitInput button = form.getInputByValue("百度一下");
        System.out.println(button);
        // 得到搜索框
        HtmlTextInput textField = form.getInputByName("q1");
        System.out.println(textField);
        // 在搜索框内填入“HtmlUnit”
        textField.setValueAttribute("HtmlUnit");
        // 点击按钮
        HtmlPage nextPage = button.click();
        // 下一个页面
        System.out.println(nextPage);
        // 获得页面结果
        String result = nextPage.asXml();
        System.out.println(result);
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        if (webClient != null) {
            webClient.close();
        }
    }
}
Also used : WebClientOptions(com.gargoylesoftware.htmlunit.WebClientOptions) HtmlForm(com.gargoylesoftware.htmlunit.html.HtmlForm) HtmlTextInput(com.gargoylesoftware.htmlunit.html.HtmlTextInput) HtmlPage(com.gargoylesoftware.htmlunit.html.HtmlPage) HtmlSubmitInput(com.gargoylesoftware.htmlunit.html.HtmlSubmitInput) NicelyResynchronizingAjaxController(com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController) WebClient(com.gargoylesoftware.htmlunit.WebClient)

Example 4 with NicelyResynchronizingAjaxController

use of com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController in project nymph by Onnt.

the class HtmlUnit method getPage.

@SuppressWarnings("resource")
public String getPage(String path) throws FailingHttpStatusCodeException, MalformedURLException, IOException {
    final WebClient webClient = new WebClient(BrowserVersion.CHROME);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.getOptions().setJavaScriptEnabled(true);
    webClient.getOptions().setActiveXNative(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.waitForBackgroundJavaScript(600 * 1000);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());
    final HtmlPage page = webClient.getPage(path);
    webClient.waitForBackgroundJavaScript(1000 * 3);
    webClient.setJavaScriptTimeout(0);
    webClient.getOptions().setJavaScriptEnabled(true);
    return page.asXml();
}
Also used : HtmlPage(com.gargoylesoftware.htmlunit.html.HtmlPage) NicelyResynchronizingAjaxController(com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController) WebClient(com.gargoylesoftware.htmlunit.WebClient)

Example 5 with NicelyResynchronizingAjaxController

use of com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController in project archiva by apache.

the class AbstractRepositoryServletTestCase method newClient.

protected static WebClient newClient() {
    final WebClient webClient = new WebClient();
    webClient.getOptions().setJavaScriptEnabled(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setAppletEnabled(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    webClient.setAjaxController(new NicelyResynchronizingAjaxController());
    return webClient;
}
Also used : NicelyResynchronizingAjaxController(com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController) WebClient(com.gargoylesoftware.htmlunit.WebClient)

Aggregations

NicelyResynchronizingAjaxController (com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController)7 WebClient (com.gargoylesoftware.htmlunit.WebClient)7 HtmlPage (com.gargoylesoftware.htmlunit.html.HtmlPage)4 SilentCssErrorHandler (com.gargoylesoftware.htmlunit.SilentCssErrorHandler)2 ProxyBean (com.cdeledu.crawler.common.bean.ProxyBean)1 BrowserVersion (com.gargoylesoftware.htmlunit.BrowserVersion)1 HttpWebConnection (com.gargoylesoftware.htmlunit.HttpWebConnection)1 WebClientOptions (com.gargoylesoftware.htmlunit.WebClientOptions)1 WebRequest (com.gargoylesoftware.htmlunit.WebRequest)1 WebResponse (com.gargoylesoftware.htmlunit.WebResponse)1 WebResponseData (com.gargoylesoftware.htmlunit.WebResponseData)1 WebWindow (com.gargoylesoftware.htmlunit.WebWindow)1 DomElement (com.gargoylesoftware.htmlunit.html.DomElement)1 DomNode (com.gargoylesoftware.htmlunit.html.DomNode)1 HtmlForm (com.gargoylesoftware.htmlunit.html.HtmlForm)1 HtmlInput (com.gargoylesoftware.htmlunit.html.HtmlInput)1 HtmlSubmitInput (com.gargoylesoftware.htmlunit.html.HtmlSubmitInput)1 HtmlTextInput (com.gargoylesoftware.htmlunit.html.HtmlTextInput)1