Search in sources :

Example 1 with QueryStringDecoder

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.QueryStringDecoder in project hadoop by apache.

the class WebHdfsHandler method channelRead0.

@Override
public void channelRead0(final ChannelHandlerContext ctx, final HttpRequest req) throws Exception {
    Preconditions.checkArgument(req.getUri().startsWith(WEBHDFS_PREFIX));
    QueryStringDecoder queryString = new QueryStringDecoder(req.getUri());
    params = new ParameterParser(queryString, conf);
    DataNodeUGIProvider ugiProvider = new DataNodeUGIProvider(params);
    ugi = ugiProvider.ugi();
    path = params.path();
    injectToken();
    ugi.doAs(new PrivilegedExceptionAction<Void>() {

        @Override
        public Void run() throws Exception {
            try {
                handle(ctx, req);
            } finally {
                String host = null;
                try {
                    host = ((InetSocketAddress) ctx.channel().remoteAddress()).getAddress().getHostAddress();
                } catch (Exception e) {
                    LOG.warn("Error retrieving hostname: ", e);
                    host = "unknown";
                }
                REQLOG.info(host + " " + req.getMethod() + " " + req.getUri() + " " + getResponseCode());
            }
            return null;
        }
    });
}
Also used : QueryStringDecoder(io.netty.handler.codec.http.QueryStringDecoder) URISyntaxException(java.net.URISyntaxException) IOException(java.io.IOException)

Example 2 with QueryStringDecoder

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.QueryStringDecoder in project hadoop by apache.

the class TestDataNodeUGIProvider method testUGICacheSecure.

@Test
public void testUGICacheSecure() throws Exception {
    // fake turning on security so api thinks it should use tokens
    SecurityUtil.setAuthenticationMethod(KERBEROS, conf);
    UserGroupInformation.setConfiguration(conf);
    UserGroupInformation ugi = UserGroupInformation.createRemoteUser("test-user");
    ugi.setAuthenticationMethod(KERBEROS);
    ugi = UserGroupInformation.createProxyUser("test-proxy-user", ugi);
    UserGroupInformation.setLoginUser(ugi);
    List<Token<DelegationTokenIdentifier>> tokens = Lists.newArrayList();
    getWebHdfsFileSystem(ugi, conf, tokens);
    String uri1 = WebHdfsFileSystem.PATH_PREFIX + PATH + "?op=OPEN" + Param.toSortedString("&", new NamenodeAddressParam("127.0.0.1:1010"), new OffsetParam((long) OFFSET), new LengthParam((long) LENGTH), new DelegationParam(tokens.get(0).encodeToUrlString()));
    String uri2 = WebHdfsFileSystem.PATH_PREFIX + PATH + "?op=OPEN" + Param.toSortedString("&", new NamenodeAddressParam("127.0.0.1:1010"), new OffsetParam((long) OFFSET), new LengthParam((long) LENGTH), new DelegationParam(tokens.get(1).encodeToUrlString()));
    DataNodeUGIProvider ugiProvider1 = new DataNodeUGIProvider(new ParameterParser(new QueryStringDecoder(URI.create(uri1)), conf));
    UserGroupInformation ugi11 = ugiProvider1.ugi();
    UserGroupInformation ugi12 = ugiProvider1.ugi();
    Assert.assertEquals("With UGI cache, two UGIs returned by the same token should be same", ugi11, ugi12);
    DataNodeUGIProvider ugiProvider2 = new DataNodeUGIProvider(new ParameterParser(new QueryStringDecoder(URI.create(uri2)), conf));
    UserGroupInformation url21 = ugiProvider2.ugi();
    UserGroupInformation url22 = ugiProvider2.ugi();
    Assert.assertEquals("With UGI cache, two UGIs returned by the same token should be same", url21, url22);
    Assert.assertNotEquals("With UGI cache, two UGIs for the different token should not be same", ugi11, url22);
    ugiProvider2.clearCache();
    awaitCacheEmptyDueToExpiration();
    ugi12 = ugiProvider1.ugi();
    url22 = ugiProvider2.ugi();
    String msg = "With cache eviction, two UGIs returned" + " by the same token should not be same";
    Assert.assertNotEquals(msg, ugi11, ugi12);
    Assert.assertNotEquals(msg, url21, url22);
    Assert.assertNotEquals("With UGI cache, two UGIs for the different token should not be same", ugi11, url22);
}
Also used : QueryStringDecoder(io.netty.handler.codec.http.QueryStringDecoder) NamenodeAddressParam(org.apache.hadoop.hdfs.web.resources.NamenodeAddressParam) OffsetParam(org.apache.hadoop.hdfs.web.resources.OffsetParam) LengthParam(org.apache.hadoop.hdfs.web.resources.LengthParam) Token(org.apache.hadoop.security.token.Token) DelegationParam(org.apache.hadoop.hdfs.web.resources.DelegationParam) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation) Test(org.junit.Test)

Example 3 with QueryStringDecoder

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.QueryStringDecoder in project hadoop by apache.

the class TestDataNodeUGIProvider method testUGICacheInSecure.

@Test
public void testUGICacheInSecure() throws Exception {
    String uri1 = WebHdfsFileSystem.PATH_PREFIX + PATH + "?op=OPEN" + Param.toSortedString("&", new OffsetParam((long) OFFSET), new LengthParam((long) LENGTH), new UserParam("root"));
    String uri2 = WebHdfsFileSystem.PATH_PREFIX + PATH + "?op=OPEN" + Param.toSortedString("&", new OffsetParam((long) OFFSET), new LengthParam((long) LENGTH), new UserParam("hdfs"));
    DataNodeUGIProvider ugiProvider1 = new DataNodeUGIProvider(new ParameterParser(new QueryStringDecoder(URI.create(uri1)), conf));
    UserGroupInformation ugi11 = ugiProvider1.ugi();
    UserGroupInformation ugi12 = ugiProvider1.ugi();
    Assert.assertEquals("With UGI cache, two UGIs for the same user should be same", ugi11, ugi12);
    DataNodeUGIProvider ugiProvider2 = new DataNodeUGIProvider(new ParameterParser(new QueryStringDecoder(URI.create(uri2)), conf));
    UserGroupInformation url21 = ugiProvider2.ugi();
    UserGroupInformation url22 = ugiProvider2.ugi();
    Assert.assertEquals("With UGI cache, two UGIs for the same user should be same", url21, url22);
    Assert.assertNotEquals("With UGI cache, two UGIs for the different user should not be same", ugi11, url22);
    awaitCacheEmptyDueToExpiration();
    ugi12 = ugiProvider1.ugi();
    url22 = ugiProvider2.ugi();
    String msg = "With cache eviction, two UGIs returned by" + " the same user should not be same";
    Assert.assertNotEquals(msg, ugi11, ugi12);
    Assert.assertNotEquals(msg, url21, url22);
    Assert.assertNotEquals("With UGI cache, two UGIs for the different user should not be same", ugi11, url22);
}
Also used : QueryStringDecoder(io.netty.handler.codec.http.QueryStringDecoder) UserParam(org.apache.hadoop.hdfs.web.resources.UserParam) OffsetParam(org.apache.hadoop.hdfs.web.resources.OffsetParam) LengthParam(org.apache.hadoop.hdfs.web.resources.LengthParam) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation) Test(org.junit.Test)

Example 4 with QueryStringDecoder

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.QueryStringDecoder in project hadoop by apache.

the class TestParameterParser method testDeserializeHAToken.

@Test
public void testDeserializeHAToken() throws IOException {
    Configuration conf = DFSTestUtil.newHAConfiguration(LOGICAL_NAME);
    final Token<DelegationTokenIdentifier> token = new Token<DelegationTokenIdentifier>();
    QueryStringDecoder decoder = new QueryStringDecoder(WebHdfsHandler.WEBHDFS_PREFIX + "/?" + NamenodeAddressParam.NAME + "=" + LOGICAL_NAME + "&" + DelegationParam.NAME + "=" + token.encodeToUrlString());
    ParameterParser testParser = new ParameterParser(decoder, conf);
    final Token<DelegationTokenIdentifier> tok2 = testParser.delegationToken();
    Assert.assertTrue(HAUtilClient.isTokenForLogicalUri(tok2));
}
Also used : QueryStringDecoder(io.netty.handler.codec.http.QueryStringDecoder) Configuration(org.apache.hadoop.conf.Configuration) DelegationTokenIdentifier(org.apache.hadoop.hdfs.security.token.delegation.DelegationTokenIdentifier) Token(org.apache.hadoop.security.token.Token) Test(org.junit.Test)

Example 5 with QueryStringDecoder

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.QueryStringDecoder in project hadoop by apache.

the class TestParameterParser method testDecodePath.

@Test
public void testDecodePath() {
    final String ESCAPED_PATH = "/test%25+1%26%3Dtest?op=OPEN&foo=bar";
    final String EXPECTED_PATH = "/test%+1&=test";
    Configuration conf = new Configuration();
    QueryStringDecoder decoder = new QueryStringDecoder(WebHdfsHandler.WEBHDFS_PREFIX + ESCAPED_PATH);
    ParameterParser testParser = new ParameterParser(decoder, conf);
    Assert.assertEquals(EXPECTED_PATH, testParser.path());
}
Also used : QueryStringDecoder(io.netty.handler.codec.http.QueryStringDecoder) Configuration(org.apache.hadoop.conf.Configuration) Test(org.junit.Test)

Aggregations

QueryStringDecoder (io.netty.handler.codec.http.QueryStringDecoder)69 List (java.util.List)29 Test (org.junit.Test)15 FullHttpRequest (io.netty.handler.codec.http.FullHttpRequest)14 Map (java.util.Map)10 HashMap (java.util.HashMap)9 IOException (java.io.IOException)8 URI (java.net.URI)7 ByteBuf (io.netty.buffer.ByteBuf)6 HttpContent (io.netty.handler.codec.http.HttpContent)6 LastHttpContent (io.netty.handler.codec.http.LastHttpContent)6 HttpPostRequestDecoder (io.netty.handler.codec.http.multipart.HttpPostRequestDecoder)6 ArrayList (java.util.ArrayList)6 HttpMethod (io.netty.handler.codec.http.HttpMethod)5 HttpRequest (io.netty.handler.codec.http.HttpRequest)5 DeviceSession (org.traccar.DeviceSession)5 Position (org.traccar.model.Position)5 DefaultHttpResponse (io.netty.handler.codec.http.DefaultHttpResponse)4 Channel (io.netty.channel.Channel)3 HttpHeaders (io.netty.handler.codec.http.HttpHeaders)3