Examples with VSCrawlerRoutePlanner com.virjar.vscrawler.core.net.proxy.VSCrawlerRoutePlanner used on opensource projects

Example 1 with VSCrawlerRoutePlanner

use of com.virjar.vscrawler.core.net.proxy.VSCrawlerRoutePlanner in project vscrawler by virjar.

the class CrawlerSession method decorateRoutePlanner.

private void decorateRoutePlanner(CrawlerHttpClient crawlerHttpClient) {
    HttpRoutePlanner routePlanner = crawlerHttpClient.getRoutePlanner();
    if (!(routePlanner instanceof ProxyBindRoutPlanner)) {
        log.warn("自定义了代理发生器,vscrawler的代理功能将不会生效");
        return;
    }
    VSCrawlerRoutePlanner vsCrawlerRoutePlanner = new VSCrawlerRoutePlanner((ProxyBindRoutPlanner) routePlanner, ipPool, proxyPlanner, this);
    crawlerHttpClient.setRoutePlanner(vsCrawlerRoutePlanner);
}