Search in sources :

Example 76 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method ignoresContentTypeIfSoConfigured.

@Test
public void ignoresContentTypeIfSoConfigured() throws IOException {
    Document doc = Jsoup.connect("https://jsoup.org/rez/osi_logo.png").ignoreContentType(true).get();
    // this will cause an ugly parse tree
    assertEquals("", doc.title());
}
Also used : Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 77 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method sendsRequestBody.

@Test
public void sendsRequestBody() throws IOException {
    final String body = "{key:value}";
    Document doc = Jsoup.connect(echoURL).requestBody(body).header("Content-Type", "text/plain").userAgent(browserUa).post();
    assertEquals("POST", ihVal("REQUEST_METHOD", doc));
    assertEquals("text/plain", ihVal("CONTENT_TYPE", doc));
    assertEquals(body, doc.select("th:contains(POSTDATA) ~ td").text());
}
Also used : Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 78 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method handlesUnescapedRedirects.

@Test
public void handlesUnescapedRedirects() throws IOException {
    // URL locations should be url safe (ascii) but are often not, so we should try to guess
    // in this case the location header is utf-8, but defined in spec as iso8859, so detect, convert, encode
    String url = "http://direct.infohound.net/tools/302-utf.pl";
    String urlEscaped = "http://direct.infohound.net/tools/test%F0%9F%92%A9.html";
    Connection.Response res = Jsoup.connect(url).execute();
    Document doc = res.parse();
    assertEquals(doc.body().text(), "💩!");
    assertEquals(doc.location(), urlEscaped);
    Connection.Response res2 = Jsoup.connect(url).followRedirects(false).execute();
    assertEquals("/tools/test💩.html", res2.header("Location"));
// if we didn't notice it was utf8, would look like: Location: /tools/test💩.html
}
Also used : Connection(org.jsoup.Connection) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 79 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method sendsRequestBodyWithUrlParams.

@Test
public void sendsRequestBodyWithUrlParams() throws IOException {
    final String body = "{key:value}";
    Document doc = Jsoup.connect(echoURL).requestBody(body).data("uname", "Jsoup", "uname", "Jonathan", "百", "度一下").header("Content-Type", // todo - if user sets content-type, we should append postcharset
    "text/plain").userAgent(browserUa).post();
    assertEquals("POST", ihVal("REQUEST_METHOD", doc));
    assertEquals("uname=Jsoup&uname=Jonathan&%E7%99%BE=%E5%BA%A6%E4%B8%80%E4%B8%8B", ihVal("QUERY_STRING", doc));
    assertEquals(body, ihVal("POSTDATA", doc));
}
Also used : Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Example 80 with Document

use of org.jsoup.nodes.Document in project jsoup by jhy.

the class UrlConnectTest method inWildUtfRedirect.

@Test
public void inWildUtfRedirect() throws IOException {
    Connection.Response res = Jsoup.connect("http://brabantn.ws/Q4F").execute();
    Document doc = res.parse();
    assertEquals("http://www.omroepbrabant.nl/?news/2474781303/Gestrande+ree+in+Oss+niet+verdoofd,+maar+doodgeschoten+%E2%80%98Dit+kan+gewoon+niet,+bizar%E2%80%99+[VIDEO].aspx", doc.location());
}
Also used : Connection(org.jsoup.Connection) Document(org.jsoup.nodes.Document) Test(org.junit.Test)

Aggregations

Document (org.jsoup.nodes.Document)405 Test (org.junit.Test)194 Element (org.jsoup.nodes.Element)164 IOException (java.io.IOException)102 File (java.io.File)81 Elements (org.jsoup.select.Elements)78 ElementHandlerImpl (org.asqatasun.ruleimplementation.ElementHandlerImpl)51 ArrayList (java.util.ArrayList)41 Connection (org.jsoup.Connection)38 URL (java.net.URL)25 HashMap (java.util.HashMap)17 InputStream (java.io.InputStream)14 List (java.util.List)10 MalformedURLException (java.net.MalformedURLException)8 Logger (org.slf4j.Logger)8 Matcher (java.util.regex.Matcher)7 Jsoup (org.jsoup.Jsoup)7 LoggerFactory (org.slf4j.LoggerFactory)7 Pattern (java.util.regex.Pattern)6 HttpGet (org.apache.http.client.methods.HttpGet)6