Search in sources :

Example 66 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method sendHeadRequest.

@Test
public void sendHeadRequest() throws IOException {
    String url = "http://direct.infohound.net/tools/parse-xml.xml";
    Connection con = Jsoup.connect(url).method(Connection.Method.HEAD);
    final Connection.Response response = con.execute();
    assertEquals("text/xml", response.header("Content-Type"));
    // head ought to have no body
    assertEquals("", response.body());
    Document doc = response.parse();
    assertEquals("", doc.text());
}
Also used : Connection(org.jsoup.Connection) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 67 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method ignores500tExceptionIfSoConfigured.

@Test
public void ignores500tExceptionIfSoConfigured() throws IOException {
    Connection con = Jsoup.connect("http://direct.infohound.net/tools/500.pl").ignoreHttpErrors(true);
    Connection.Response res = con.execute();
    Document doc = res.parse();
    assertEquals(500, res.statusCode());
    assertEquals("Application Error", res.statusMessage());
    assertEquals("Woops", doc.select("h1").first().text());
}
Also used : Connection(org.jsoup.Connection) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 68 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method fetchToW3c.

@Test
public void fetchToW3c() throws IOException {
    String url = "https://jsoup.org";
    Document doc = Jsoup.connect(url).get();
    W3CDom dom = new W3CDom();
    org.w3c.dom.Document wDoc = dom.fromJsoup(doc);
    assertEquals(url, wDoc.getDocumentURI());
    String html = dom.asString(wDoc);
    assertTrue(html.contains("jsoup"));
}
Also used : W3CDom(org.jsoup.helper.W3CDom) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 69 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method baseHrefCorrectAfterHttpEquiv.

@Test
public void baseHrefCorrectAfterHttpEquiv() throws IOException {
    // https://github.com/jhy/jsoup/issues/440
    Connection.Response res = Jsoup.connect("http://direct.infohound.net/tools/charset-base.html").execute();
    Document doc = res.parse();
    assertEquals("http://example.com/foo.jpg", doc.select("img").first().absUrl("src"));
}
Also used : Connection(org.jsoup.Connection) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 70 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method doesGet.

@Test
public void doesGet() throws IOException {
    Connection con = Jsoup.connect(echoURL + "?what=the").userAgent("Mozilla").referrer("http://example.com").data("what", "about & me?");
    Document doc = con.get();
    assertEquals("what=the&what=about+%26+me%3F", ihVal("QUERY_STRING", doc));
    assertEquals("the, about & me?", ihVal("what", doc));
    assertEquals("Mozilla", ihVal("HTTP_USER_AGENT", doc));
    assertEquals("http://example.com", ihVal("HTTP_REFERER", doc));
}
Also used : Connection(org.jsoup.Connection) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Aggregations

Document (org.jsoup.nodes.Document)405 Test (org.junit.Test)194 Element (org.jsoup.nodes.Element)164 IOException (java.io.IOException)102 File (java.io.File)81 Elements (org.jsoup.select.Elements)78 ElementHandlerImpl (org.asqatasun.ruleimplementation.ElementHandlerImpl)51 ArrayList (java.util.ArrayList)41 Connection (org.jsoup.Connection)38 URL (java.net.URL)25 HashMap (java.util.HashMap)17 InputStream (java.io.InputStream)14 List (java.util.List)10 MalformedURLException (java.net.MalformedURLException)8 Logger (org.slf4j.Logger)8 Matcher (java.util.regex.Matcher)7 Jsoup (org.jsoup.Jsoup)7 LoggerFactory (org.slf4j.LoggerFactory)7 Pattern (java.util.regex.Pattern)6 HttpGet (org.apache.http.client.methods.HttpGet)6