Search in sources :

Example 1 with Client

use of com.sun.jersey.api.client.Client in project hadoop by apache.

the class LogsCLI method getAMContainerInfoForRMWebService.

protected List<JSONObject> getAMContainerInfoForRMWebService(Configuration conf, String appId) throws ClientHandlerException, UniformInterfaceException, JSONException {
    Client webServiceClient = Client.create();
    String webAppAddress = WebAppUtils.getRMWebAppURLWithScheme(conf);
    WebResource webResource = webServiceClient.resource(webAppAddress);
    ClientResponse response = webResource.path("ws").path("v1").path("cluster").path("apps").path(appId).path("appattempts").accept(MediaType.APPLICATION_JSON).get(ClientResponse.class);
    JSONObject json = response.getEntity(JSONObject.class).getJSONObject("appAttempts");
    JSONArray requests = json.getJSONArray("appAttempt");
    List<JSONObject> amContainersList = new ArrayList<JSONObject>();
    for (int i = 0; i < requests.length(); i++) {
        amContainersList.add(requests.getJSONObject(i));
    }
    return amContainersList;
}
Also used : ClientResponse(com.sun.jersey.api.client.ClientResponse) JSONObject(org.codehaus.jettison.json.JSONObject) JSONArray(org.codehaus.jettison.json.JSONArray) ArrayList(java.util.ArrayList) WebResource(com.sun.jersey.api.client.WebResource) YarnClient(org.apache.hadoop.yarn.client.api.YarnClient) Client(com.sun.jersey.api.client.Client)

Example 2 with Client

use of com.sun.jersey.api.client.Client in project hadoop by apache.

the class LogsCLI method getContainerLogFiles.

private List<Pair<PerContainerLogFileInfo, String>> getContainerLogFiles(Configuration conf, String containerIdStr, String nodeHttpAddress) throws IOException {
    List<Pair<PerContainerLogFileInfo, String>> logFileInfos = new ArrayList<>();
    Client webServiceClient = Client.create();
    try {
        WebResource webResource = webServiceClient.resource(WebAppUtils.getHttpSchemePrefix(conf) + nodeHttpAddress);
        ClientResponse response = webResource.path("ws").path("v1").path("node").path("containers").path(containerIdStr).path("logs").accept(MediaType.APPLICATION_JSON).get(ClientResponse.class);
        if (response.getStatusInfo().getStatusCode() == ClientResponse.Status.OK.getStatusCode()) {
            try {
                JSONArray array = new JSONArray();
                JSONObject json = response.getEntity(JSONObject.class);
                Object logsInfoObj = json.get("containerLogsInfo");
                if (logsInfoObj instanceof JSONObject) {
                    array.put((JSONObject) logsInfoObj);
                } else if (logsInfoObj instanceof JSONArray) {
                    JSONArray logsArray = (JSONArray) logsInfoObj;
                    for (int i = 0; i < logsArray.length(); i++) {
                        array.put(logsArray.getJSONObject(i));
                    }
                }
                for (int i = 0; i < array.length(); i++) {
                    JSONObject log = array.getJSONObject(i);
                    String aggregateType = log.has("logAggregationType") ? log.getString("logAggregationType") : "N/A";
                    Object ob = log.get("containerLogInfo");
                    if (ob instanceof JSONArray) {
                        JSONArray obArray = (JSONArray) ob;
                        for (int j = 0; j < obArray.length(); j++) {
                            logFileInfos.add(new Pair<PerContainerLogFileInfo, String>(generatePerContainerLogFileInfoFromJSON(obArray.getJSONObject(j)), aggregateType));
                        }
                    } else if (ob instanceof JSONObject) {
                        logFileInfos.add(new Pair<PerContainerLogFileInfo, String>(generatePerContainerLogFileInfoFromJSON((JSONObject) ob), aggregateType));
                    }
                }
            } catch (Exception e) {
                System.err.println("Unable to parse json from webservice. Error:");
                System.err.println(e.getMessage());
                throw new IOException(e);
            }
        }
    } catch (ClientHandlerException | UniformInterfaceException ex) {
        System.err.println("Unable to fetch log files list");
        throw new IOException(ex);
    }
    return logFileInfos;
}
Also used : ClientResponse(com.sun.jersey.api.client.ClientResponse) ClientHandlerException(com.sun.jersey.api.client.ClientHandlerException) ArrayList(java.util.ArrayList) JSONArray(org.codehaus.jettison.json.JSONArray) WebResource(com.sun.jersey.api.client.WebResource) IOException(java.io.IOException) PerContainerLogFileInfo(org.apache.hadoop.yarn.logaggregation.PerContainerLogFileInfo) ParseException(org.apache.commons.cli.ParseException) UniformInterfaceException(com.sun.jersey.api.client.UniformInterfaceException) YarnException(org.apache.hadoop.yarn.exceptions.YarnException) IOException(java.io.IOException) JSONException(org.codehaus.jettison.json.JSONException) ClientHandlerException(com.sun.jersey.api.client.ClientHandlerException) UniformInterfaceException(com.sun.jersey.api.client.UniformInterfaceException) JSONObject(org.codehaus.jettison.json.JSONObject) JSONObject(org.codehaus.jettison.json.JSONObject) YarnClient(org.apache.hadoop.yarn.client.api.YarnClient) Client(com.sun.jersey.api.client.Client) Pair(org.apache.commons.math3.util.Pair)

Example 3 with Client

use of com.sun.jersey.api.client.Client in project hadoop by apache.

the class TestTimelineClientForATS1_5 method createTimelineClient.

private TimelineClientImpl createTimelineClient(YarnConfiguration conf) {
    TimelineClientImpl client = new TimelineClientImpl() {

        @Override
        protected TimelineWriter createTimelineWriter(Configuration conf, UserGroupInformation authUgi, Client client, URI resURI) throws IOException {
            TimelineWriter timelineWriter = new FileSystemTimelineWriter(conf, authUgi, client, resURI) {

                public ClientResponse doPostingObject(Object object, String path) {
                    ClientResponse response = mock(ClientResponse.class);
                    when(response.getStatusInfo()).thenReturn(ClientResponse.Status.OK);
                    return response;
                }
            };
            spyTimelineWriter = spy(timelineWriter);
            return spyTimelineWriter;
        }
    };
    client.init(conf);
    client.start();
    return client;
}
Also used : ClientResponse(com.sun.jersey.api.client.ClientResponse) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) Configuration(org.apache.hadoop.conf.Configuration) Client(com.sun.jersey.api.client.Client) URI(java.net.URI) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation)

Example 4 with Client

use of com.sun.jersey.api.client.Client in project hadoop by apache.

the class TestRMHA method checkActiveRMWebServices.

// Do some sanity testing of the web-services after fail-over.
private void checkActiveRMWebServices() throws JSONException {
    // Validate web-service
    Client webServiceClient = Client.create(new DefaultClientConfig());
    InetSocketAddress rmWebappAddr = NetUtils.getConnectAddress(rm.getWebapp().getListenerAddress());
    String webappURL = "http://" + rmWebappAddr.getHostName() + ":" + rmWebappAddr.getPort();
    WebResource webResource = webServiceClient.resource(webappURL);
    String path = app.getApplicationId().toString();
    ClientResponse response = webResource.path("ws").path("v1").path("cluster").path("apps").path(path).accept(MediaType.APPLICATION_JSON).get(ClientResponse.class);
    assertEquals(MediaType.APPLICATION_JSON_TYPE + "; " + JettyUtils.UTF_8, response.getType().toString());
    JSONObject json = response.getEntity(JSONObject.class);
    assertEquals("incorrect number of elements", 1, json.length());
    JSONObject appJson = json.getJSONObject("app");
    assertEquals("ACCEPTED", appJson.getString("state"));
// Other stuff is verified in the regular web-services related tests
}
Also used : ClientResponse(com.sun.jersey.api.client.ClientResponse) DefaultClientConfig(com.sun.jersey.api.client.config.DefaultClientConfig) JSONObject(org.codehaus.jettison.json.JSONObject) InetSocketAddress(java.net.InetSocketAddress) WebResource(com.sun.jersey.api.client.WebResource) Client(com.sun.jersey.api.client.Client)

Example 5 with Client

use of com.sun.jersey.api.client.Client in project hadoop by apache.

the class TestTimelineReaderWebServicesHBaseStorage method testGetFlows.

@Test
public void testGetFlows() throws Exception {
    Client client = createClient();
    try {
        URI uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows");
        ClientResponse resp = getResponse(client, uri);
        Set<FlowActivityEntity> entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(2, entities.size());
        for (FlowActivityEntity entity : entities) {
            assertTrue((entity.getId().endsWith("@flow_name") && entity.getFlowRuns().size() == 2) || (entity.getId().endsWith("@flow_name2") && entity.getFlowRuns().size() == 1));
        }
        // Query without specifying cluster ID.
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/flows/");
        resp = getResponse(client, uri);
        entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(2, entities.size());
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?limit=1");
        resp = getResponse(client, uri);
        entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(1, entities.size());
        long firstFlowActivity = HBaseTimelineStorageUtils.getTopOfTheDayTimestamp(1425016501000L);
        DateFormat fmt = TimelineReaderWebServices.DATE_FORMAT.get();
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=" + fmt.format(firstFlowActivity) + "-" + fmt.format(dayTs));
        resp = getResponse(client, uri);
        entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(2, entities.size());
        for (FlowActivityEntity entity : entities) {
            assertTrue((entity.getId().endsWith("@flow_name") && entity.getFlowRuns().size() == 2) || (entity.getId().endsWith("@flow_name2") && entity.getFlowRuns().size() == 1));
        }
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=" + fmt.format(dayTs + (4 * 86400000L)));
        resp = getResponse(client, uri);
        entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(0, entities.size());
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=-" + fmt.format(dayTs));
        resp = getResponse(client, uri);
        entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(2, entities.size());
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=" + fmt.format(firstFlowActivity) + "-");
        resp = getResponse(client, uri);
        entities = resp.getEntity(new GenericType<Set<FlowActivityEntity>>() {
        });
        assertNotNull(entities);
        assertEquals(2, entities.size());
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=20150711:20150714");
        verifyHttpResponse(client, uri, Status.BAD_REQUEST);
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=20150714-20150711");
        verifyHttpResponse(client, uri, Status.BAD_REQUEST);
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=2015071129-20150712");
        verifyHttpResponse(client, uri, Status.BAD_REQUEST);
        uri = URI.create("http://localhost:" + serverPort + "/ws/v2/" + "timeline/clusters/cluster1/flows?daterange=20150711-2015071243");
        verifyHttpResponse(client, uri, Status.BAD_REQUEST);
    } finally {
        client.destroy();
    }
}
Also used : ClientResponse(com.sun.jersey.api.client.ClientResponse) GenericType(com.sun.jersey.api.client.GenericType) Set(java.util.Set) HashSet(java.util.HashSet) DateFormat(java.text.DateFormat) FlowActivityEntity(org.apache.hadoop.yarn.api.records.timelineservice.FlowActivityEntity) Client(com.sun.jersey.api.client.Client) URI(java.net.URI) Test(org.junit.Test)

Aggregations

Client (com.sun.jersey.api.client.Client)85 ClientResponse (com.sun.jersey.api.client.ClientResponse)60 Test (org.junit.Test)59 URI (java.net.URI)51 TimelineEntity (org.apache.hadoop.yarn.api.records.timelineservice.TimelineEntity)36 Set (java.util.Set)30 HashSet (java.util.HashSet)19 WebResource (com.sun.jersey.api.client.WebResource)18 GenericType (com.sun.jersey.api.client.GenericType)17 DefaultClientConfig (com.sun.jersey.api.client.config.DefaultClientConfig)9 TimelineMetric (org.apache.hadoop.yarn.api.records.timelineservice.TimelineMetric)9 JSONObject (org.codehaus.jettison.json.JSONObject)7 ClientConfig (com.sun.jersey.api.client.config.ClientConfig)6 URLConnectionClientHandler (com.sun.jersey.client.urlconnection.URLConnectionClientHandler)4 ArrayList (java.util.ArrayList)4 FlowRunEntity (org.apache.hadoop.yarn.api.records.timelineservice.FlowRunEntity)4 YarnClient (org.apache.hadoop.yarn.client.api.YarnClient)4 ClientHandlerException (com.sun.jersey.api.client.ClientHandlerException)3 HTTPBasicAuthFilter (com.sun.jersey.api.client.filter.HTTPBasicAuthFilter)3 FlowActivityEntity (org.apache.hadoop.yarn.api.records.timelineservice.FlowActivityEntity)3