Search in sources :

Example 1 with ContentExchange

use of org.eclipse.jetty.client.ContentExchange in project learning-spark by databricks.

the class BasicMapPartitions method main.

public static void main(String[] args) throws Exception {
    String master;
    if (args.length > 0) {
        master = args[0];
    } else {
        master = "local";
    }
    JavaSparkContext sc = new JavaSparkContext(master, "basicmappartitions", System.getenv("SPARK_HOME"), System.getenv("JARS"));
    JavaRDD<String> rdd = sc.parallelize(Arrays.asList("KK6JKQ", "Ve3UoW", "kk6jlk", "W6BB"));
    JavaRDD<String> result = rdd.mapPartitions(new FlatMapFunction<Iterator<String>, String>() {

        public Iterable<String> call(Iterator<String> input) {
            ArrayList<String> content = new ArrayList<String>();
            ArrayList<ContentExchange> cea = new ArrayList<ContentExchange>();
            HttpClient client = new HttpClient();
            try {
                client.start();
                while (input.hasNext()) {
                    ContentExchange exchange = new ContentExchange(true);
                    exchange.setURL("http://qrzcq.com/call/" + input.next());
                    client.send(exchange);
                    cea.add(exchange);
                }
                for (ContentExchange exchange : cea) {
                    exchange.waitForDone();
                    content.add(exchange.getResponseContent());
                }
            } catch (Exception e) {
            }
            return content;
        }
    });
    System.out.println(StringUtils.join(result.collect(), ","));
}
Also used : ArrayList(java.util.ArrayList) HttpClient(org.eclipse.jetty.client.HttpClient) Iterator(java.util.Iterator) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) ContentExchange(org.eclipse.jetty.client.ContentExchange)

Example 2 with ContentExchange

use of org.eclipse.jetty.client.ContentExchange in project elastic-job by dangdangdotcom.

the class RestfulServerTest method assertCallFailure.

@Test
public void assertCallFailure() throws Exception {
    ContentExchange actual = sentRequest("{\"string\":\"test\",\"integer\":\"invalid_number\"}");
    Assert.assertThat(actual.getResponseStatus(), Is.is(500));
    Assert.assertThat(actual.getResponseContent(), StringStartsWith.startsWith("java.lang.NumberFormatException"));
    Mockito.verify(caller).call("test");
}
Also used : ContentExchange(org.eclipse.jetty.client.ContentExchange) Test(org.junit.Test)

Example 3 with ContentExchange

use of org.eclipse.jetty.client.ContentExchange in project elastic-job by dangdangdotcom.

the class RestfulServerTest method sentRequest.

private static ContentExchange sentRequest(final String content) throws Exception {
    HttpClient httpClient = new HttpClient();
    try {
        httpClient.start();
        ContentExchange result = new ContentExchange();
        result.setMethod("POST");
        result.setRequestContentType(MediaType.APPLICATION_JSON);
        result.setRequestContent(new ByteArrayBuffer(content.getBytes("UTF-8")));
        httpClient.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);
        result.setURL(URL);
        httpClient.send(result);
        result.waitForDone();
        return result;
    } finally {
        httpClient.stop();
    }
}
Also used : HttpClient(org.eclipse.jetty.client.HttpClient) ContentExchange(org.eclipse.jetty.client.ContentExchange) ByteArrayBuffer(org.eclipse.jetty.io.ByteArrayBuffer)

Example 4 with ContentExchange

use of org.eclipse.jetty.client.ContentExchange in project elastic-job by dangdangdotcom.

the class RestfulServerTest method assertCallSuccess.

@Test
public void assertCallSuccess() throws Exception {
    ContentExchange actual = sentRequest("{\"string\":\"test\",\"integer\":1}");
    Assert.assertThat(actual.getResponseStatus(), Is.is(200));
    Assert.assertThat(actual.getResponseContent(), Is.is("{\"string\":\"test_processed\",\"integer\":\"1_processed\"}"));
    Mockito.verify(caller).call("test");
    Mockito.verify(caller).call(1);
}
Also used : ContentExchange(org.eclipse.jetty.client.ContentExchange) Test(org.junit.Test)

Example 5 with ContentExchange

use of org.eclipse.jetty.client.ContentExchange in project elastic-job by dangdangdotcom.

the class RestfulTestsUtil method sentGetRequest.

public static String sentGetRequest(final String url) throws Exception {
    HttpClient httpClient = new HttpClient();
    try {
        httpClient.start();
        ContentExchange contentExchange = new ContentExchange();
        contentExchange.setMethod("GET");
        contentExchange.setRequestContentType(MediaType.APPLICATION_JSON);
        httpClient.setConnectorType(HttpClient.CONNECTOR_SELECT_CHANNEL);
        contentExchange.setURL(url);
        httpClient.send(contentExchange);
        contentExchange.waitForDone();
        return contentExchange.getResponseContent();
    } finally {
        httpClient.stop();
    }
}
Also used : HttpClient(org.eclipse.jetty.client.HttpClient) ContentExchange(org.eclipse.jetty.client.ContentExchange)

Aggregations

ContentExchange (org.eclipse.jetty.client.ContentExchange)9 HttpClient (org.eclipse.jetty.client.HttpClient)5 Tuple2 (scala.Tuple2)3 ArrayList (java.util.ArrayList)2 Iterator (java.util.Iterator)2 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)2 Test (org.junit.Test)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 FileNotFoundException (java.io.FileNotFoundException)1 JavaDoubleRDD (org.apache.spark.api.java.JavaDoubleRDD)1 PairFunction (org.apache.spark.api.java.function.PairFunction)1 StatCounter (org.apache.spark.util.StatCounter)1 ByteArrayBuffer (org.eclipse.jetty.io.ByteArrayBuffer)1