Examples with Crawler org.asqatasun.crawler.Crawler used on opensource projects

Example 1 with Crawler

use of org.asqatasun.crawler.Crawler in project Asqatasun by Asqatasun.

the class CrawlerServiceImpl method crawlGroupOfPages.

@Override
public WebResource crawlGroupOfPages(Audit audit, String siteUrl, List<String> urlList) {
    Crawler crawler = getCrawlerInstance((Set<Parameter>) audit.getParameterSet(), true);
    crawler.setSiteURL(siteUrl, urlList);
    return crawl(crawler, audit, true);
}

Example 2 with Crawler

use of org.asqatasun.crawler.Crawler in project Asqatasun by Asqatasun.

the class CrawlerServiceImpl method crawlPage.

@Override
public WebResource crawlPage(Audit audit, String pageUrl) {
    Crawler crawler = getCrawlerInstance((Set<Parameter>) audit.getParameterSet(), true);
    crawler.setPageURL(pageUrl);
    return crawl(crawler, audit, true);
}

Example 3 with Crawler

use of org.asqatasun.crawler.Crawler in project Asqatasun by Asqatasun.

the class CrawlerServiceImpl method crawlSite.

/**
     * Calls the crawler component process then updates the site.
     *
     * @param siteUrl the site to crawl
     * @param audit the current audit
     *
     * @return returns the site after modification
     */
@Override
public WebResource crawlSite(Audit audit, String siteUrl) {
    Crawler crawler = getCrawlerInstance((Set<Parameter>) audit.getParameterSet(), true);
    crawler.setSiteURL(siteUrl);
    return crawl(crawler, audit, true);
}