Search in sources :

Example 1 with CommonNettyHttpServiceBuilder

use of io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder in project cdap by caskdata.

the class TetheringServerHandlerTest method setUp.

@Before
public void setUp() throws Exception {
    // Define all StructuredTable before starting any services that need StructuredTable
    StoreDefinition.createAllTables(injector.getInstance(StructuredTableAdmin.class));
    cConf.setBoolean(Constants.Tethering.TETHERING_SERVER_ENABLED, true);
    cConf.setInt(Constants.Tethering.CONNECTION_TIMEOUT_SECONDS, 1);
    List<Permission> tetheringPermissions = Arrays.asList(InstancePermission.TETHER);
    InMemoryAccessController inMemoryAccessController = new InMemoryAccessController();
    inMemoryAccessController.grant(Authorizable.fromEntityId(InstanceId.SELF), MASTER_PRINCIPAL, Collections.unmodifiableSet(new HashSet<>(tetheringPermissions)));
    ContextAccessEnforcer contextAccessEnforcer = new DefaultContextAccessEnforcer(new AuthenticationTestContext(), inMemoryAccessController);
    AuthenticationTestContext.actAsPrincipal(MASTER_PRINCIPAL);
    service = new CommonNettyHttpServiceBuilder(CConfiguration.create(), getClass().getSimpleName()).setHttpHandlers(new TetheringServerHandler(cConf, tetheringStore, messagingService, contextAccessEnforcer), new TetheringHandler(cConf, tetheringStore, messagingService)).build();
    service.start();
    config = ClientConfig.builder().setConnectionConfig(ConnectionConfig.builder().setHostname(service.getBindAddress().getHostName()).setPort(service.getBindAddress().getPort()).setSSLEnabled(false).build()).build();
}
Also used : CommonNettyHttpServiceBuilder(io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder) StructuredTableAdmin(io.cdap.cdap.spi.data.StructuredTableAdmin) InMemoryAccessController(io.cdap.cdap.security.authorization.InMemoryAccessController) InstancePermission(io.cdap.cdap.proto.security.InstancePermission) Permission(io.cdap.cdap.proto.security.Permission) DefaultContextAccessEnforcer(io.cdap.cdap.security.authorization.DefaultContextAccessEnforcer) AuthenticationTestContext(io.cdap.cdap.security.auth.context.AuthenticationTestContext) DefaultContextAccessEnforcer(io.cdap.cdap.security.authorization.DefaultContextAccessEnforcer) ContextAccessEnforcer(io.cdap.cdap.security.spi.authorization.ContextAccessEnforcer) HashSet(java.util.HashSet) Before(org.junit.Before)

Example 2 with CommonNettyHttpServiceBuilder

use of io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder in project cdap by caskdata.

the class RemoteConfiguratorTest method init.

@BeforeClass
public static void init() throws Exception {
    cConf = CConfiguration.create();
    cConf.set(Constants.CFG_LOCAL_DATA_DIR, TEMP_FOLDER.newFolder().getAbsolutePath());
    cConf.setInt(Constants.TaskWorker.CONTAINER_KILL_AFTER_REQUEST_COUNT, 0);
    InMemoryDiscoveryService discoveryService = new InMemoryDiscoveryService();
    MasterEnvironments.setMasterEnvironment(new TestMasterEnvironment(discoveryService));
    NamespaceAdmin namespaceAdmin = new InMemoryNamespaceAdmin();
    namespaceAdmin.create(NamespaceMeta.SYSTEM);
    namespaceAdmin.create(NamespaceMeta.DEFAULT);
    remoteClientFactory = new RemoteClientFactory(discoveryService, new DefaultInternalAuthenticator(new AuthenticationTestContext()));
    httpService = new CommonNettyHttpServiceBuilder(cConf, "test").setHttpHandlers(new TaskWorkerHttpHandlerInternal(cConf, className -> {
    }, new NoOpMetricsCollectionService()), new ArtifactHttpHandlerInternal(new TestArtifactRepository(cConf), namespaceAdmin), new ArtifactLocalizerHttpHandlerInternal(new ArtifactLocalizer(cConf, remoteClientFactory, ((namespaceId, retryStrategy) -> {
        return new NoOpArtifactManager();
    })))).setPort(cConf.getInt(Constants.ArtifactLocalizer.PORT)).setChannelPipelineModifier(new ChannelPipelineModifier() {

        @Override
        public void modify(ChannelPipeline pipeline) {
            pipeline.addAfter("compressor", "decompressor", new HttpContentDecompressor());
        }
    }).build();
    httpService.start();
    discoveryService.register(URIScheme.createDiscoverable(Constants.Service.TASK_WORKER, httpService));
    discoveryService.register(URIScheme.createDiscoverable(Constants.Service.APP_FABRIC_HTTP, httpService));
    metricsCollectionService = new NoOpMetricsCollectionService();
}
Also used : ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) AccessException(io.cdap.cdap.api.security.AccessException) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) ArtifactClasses(io.cdap.cdap.api.artifact.ArtifactClasses) ArtifactLocalizerHttpHandlerInternal(io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizerHttpHandlerInternal) GsonBuilder(com.google.gson.GsonBuilder) ArtifactMeta(io.cdap.cdap.internal.app.runtime.artifact.ArtifactMeta) MasterEnvironments(io.cdap.cdap.master.environment.MasterEnvironments) AppJarHelper(io.cdap.cdap.common.test.AppJarHelper) Gson(com.google.gson.Gson) TaskWorkerHttpHandlerInternal(io.cdap.cdap.internal.app.worker.TaskWorkerHttpHandlerInternal) MasterEnvironmentRunnableContext(io.cdap.cdap.master.spi.environment.MasterEnvironmentRunnableContext) NettyHttpService(io.cdap.http.NettyHttpService) After(org.junit.After) Map(java.util.Map) ConfigTestApp(io.cdap.cdap.ConfigTestApp) ArtifactInfo(io.cdap.cdap.api.artifact.ArtifactInfo) CloseableClassLoader(io.cdap.cdap.api.artifact.CloseableClassLoader) ClassRule(org.junit.ClassRule) NoOpMetricsCollectionService(io.cdap.cdap.common.metrics.NoOpMetricsCollectionService) ArtifactManager(io.cdap.cdap.api.artifact.ArtifactManager) AuthenticationTestContext(io.cdap.cdap.security.auth.context.AuthenticationTestContext) AfterClass(org.junit.AfterClass) ArtifactDescriptor(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor) ArtifactRepositoryReader(io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepositoryReader) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) ProgramType(io.cdap.cdap.api.app.ProgramType) MasterEnvironment(io.cdap.cdap.master.spi.environment.MasterEnvironment) ChannelPipeline(io.netty.channel.ChannelPipeline) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) Id(io.cdap.cdap.common.id.Id) ApplicationSpecificationAdapter(io.cdap.cdap.internal.app.ApplicationSpecificationAdapter) CommonNettyHttpServiceBuilder(io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder) ArtifactLocalizer(io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizer) NamespaceMeta(io.cdap.cdap.proto.NamespaceMeta) List(java.util.List) Constants(io.cdap.cdap.common.conf.Constants) Configurator(io.cdap.cdap.app.deploy.Configurator) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) NotFoundException(io.cdap.cdap.common.NotFoundException) RemoteClientFactory(io.cdap.cdap.common.internal.remote.RemoteClientFactory) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) InMemoryDiscoveryService(org.apache.twill.discovery.InMemoryDiscoveryService) BeforeClass(org.junit.BeforeClass) InMemoryNamespaceAdmin(io.cdap.cdap.common.namespace.InMemoryNamespaceAdmin) Location(org.apache.twill.filesystem.Location) ChannelPipelineModifier(io.cdap.http.ChannelPipelineModifier) HashMap(java.util.HashMap) Supplier(java.util.function.Supplier) DiscoveryServiceClient(org.apache.twill.discovery.DiscoveryServiceClient) ArtifactHttpHandlerInternal(io.cdap.cdap.gateway.handlers.ArtifactHttpHandlerInternal) ConfiguratorTask(io.cdap.cdap.internal.app.worker.ConfiguratorTask) TwillRunnerService(org.apache.twill.api.TwillRunnerService) ArtifactId(io.cdap.cdap.proto.id.ArtifactId) ArtifactDetail(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDetail) Nullable(javax.annotation.Nullable) DiscoveryService(org.apache.twill.discovery.DiscoveryService) URIScheme(io.cdap.cdap.common.discovery.URIScheme) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) Test(org.junit.Test) IOException(java.io.IOException) LocationFactory(org.apache.twill.filesystem.LocationFactory) MetricsCollectionService(io.cdap.cdap.api.metrics.MetricsCollectionService) DefaultInternalAuthenticator(io.cdap.cdap.common.internal.remote.DefaultInternalAuthenticator) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) NamespaceAdmin(io.cdap.cdap.common.namespace.NamespaceAdmin) ArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepository) HttpContentDecompressor(io.netty.handler.codec.http.HttpContentDecompressor) MasterEnvironmentRunnable(io.cdap.cdap.master.spi.environment.MasterEnvironmentRunnable) AllProgramsApp(io.cdap.cdap.AllProgramsApp) Assert(org.junit.Assert) Collections(java.util.Collections) TemporaryFolder(org.junit.rules.TemporaryFolder) RemoteClientFactory(io.cdap.cdap.common.internal.remote.RemoteClientFactory) CommonNettyHttpServiceBuilder(io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder) InMemoryNamespaceAdmin(io.cdap.cdap.common.namespace.InMemoryNamespaceAdmin) NamespaceAdmin(io.cdap.cdap.common.namespace.NamespaceAdmin) AuthenticationTestContext(io.cdap.cdap.security.auth.context.AuthenticationTestContext) ArtifactLocalizer(io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizer) TaskWorkerHttpHandlerInternal(io.cdap.cdap.internal.app.worker.TaskWorkerHttpHandlerInternal) NoOpMetricsCollectionService(io.cdap.cdap.common.metrics.NoOpMetricsCollectionService) ChannelPipeline(io.netty.channel.ChannelPipeline) HttpContentDecompressor(io.netty.handler.codec.http.HttpContentDecompressor) DefaultInternalAuthenticator(io.cdap.cdap.common.internal.remote.DefaultInternalAuthenticator) InMemoryNamespaceAdmin(io.cdap.cdap.common.namespace.InMemoryNamespaceAdmin) ArtifactHttpHandlerInternal(io.cdap.cdap.gateway.handlers.ArtifactHttpHandlerInternal) InMemoryDiscoveryService(org.apache.twill.discovery.InMemoryDiscoveryService) ChannelPipelineModifier(io.cdap.http.ChannelPipelineModifier) ArtifactLocalizerHttpHandlerInternal(io.cdap.cdap.internal.app.worker.sidecar.ArtifactLocalizerHttpHandlerInternal) BeforeClass(org.junit.BeforeClass)

Example 3 with CommonNettyHttpServiceBuilder

use of io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder in project cdap by caskdata.

the class AuthorizationHandlerTest method setUp.

@Before
public void setUp() throws Exception {
    CConfiguration conf = CConfiguration.create();
    conf.setBoolean(Constants.Security.Authorization.ENABLED, true);
    conf.setBoolean(Constants.Security.ENABLED, true);
    properties.setProperty("superusers", admin.getName());
    final InMemoryAccessController auth = new InMemoryAccessController();
    auth.initialize(FACTORY.create(properties));
    service = new CommonNettyHttpServiceBuilder(conf, getClass().getSimpleName()).setHttpHandlers(new AuthorizationHandler(auth, new AccessControllerInstantiator(conf, FACTORY) {

        @Override
        public AccessController get() {
            return auth;
        }
    }, conf, new MasterAuthenticationContext())).setChannelPipelineModifier(new ChannelPipelineModifier() {

        @Override
        public void modify(ChannelPipeline pipeline) {
            pipeline.addBefore("dispatcher", "usernamesetter", new TestUserNameSetter());
        }
    }).build();
    service.start();
    client = new AuthorizationClient(ClientConfig.builder().setConnectionConfig(ConnectionConfig.builder().setHostname(service.getBindAddress().getHostName()).setPort(service.getBindAddress().getPort()).setSSLEnabled(false).build()).build());
    System.setProperty(USERNAME_PROPERTY, admin.getName());
}
Also used : MasterAuthenticationContext(io.cdap.cdap.security.auth.context.MasterAuthenticationContext) InMemoryAccessController(io.cdap.cdap.security.authorization.InMemoryAccessController) AccessController(io.cdap.cdap.security.spi.authorization.AccessController) CommonNettyHttpServiceBuilder(io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder) InMemoryAccessController(io.cdap.cdap.security.authorization.InMemoryAccessController) AccessControllerInstantiator(io.cdap.cdap.security.authorization.AccessControllerInstantiator) AuthorizationClient(io.cdap.cdap.client.AuthorizationClient) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) ChannelPipelineModifier(io.cdap.http.ChannelPipelineModifier) ChannelPipeline(io.netty.channel.ChannelPipeline) Before(org.junit.Before)

Example 4 with CommonNettyHttpServiceBuilder

use of io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder in project cdap by caskdata.

the class RemoteClientAuthenticatorTest method setup.

@BeforeClass
public static void setup() throws Exception {
    mockRemoteAuthenticatorProvider = new MockRemoteAuthenticatorProvider();
    // Setup Guice injector.
    injector = Guice.createInjector(new ConfigModule(), new InMemoryDiscoveryModule(), new PrivateModule() {

        @Override
        protected void configure() {
            bind(RemoteAuthenticator.class).toProvider(mockRemoteAuthenticatorProvider);
            expose(RemoteAuthenticator.class);
        }
    }, new AuthenticationContextModules().getNoOpModule());
    CConfiguration cConf = injector.getInstance(CConfiguration.class);
    DiscoveryService discoveryService = injector.getInstance(DiscoveryService.class);
    // Setup test HTTP handler and register the service.
    testHttpHandler = new TestHttpHandler();
    httpService = new CommonNettyHttpServiceBuilder(cConf, TEST_SERVICE).setHttpHandlers(testHttpHandler).build();
    httpService.start();
    discoveryService.register(new Discoverable(TEST_SERVICE, httpService.getBindAddress()));
}
Also used : InMemoryDiscoveryModule(io.cdap.cdap.common.guice.InMemoryDiscoveryModule) Discoverable(org.apache.twill.discovery.Discoverable) NoOpRemoteAuthenticator(io.cdap.cdap.common.internal.remote.NoOpRemoteAuthenticator) RemoteAuthenticator(io.cdap.cdap.security.spi.authenticator.RemoteAuthenticator) CommonNettyHttpServiceBuilder(io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder) ConfigModule(io.cdap.cdap.common.guice.ConfigModule) DiscoveryService(org.apache.twill.discovery.DiscoveryService) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) PrivateModule(com.google.inject.PrivateModule) BeforeClass(org.junit.BeforeClass)

Example 5 with CommonNettyHttpServiceBuilder

use of io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder in project cdap by caskdata.

the class SecureStoreTest method beforeClass.

@BeforeClass
public static void beforeClass() throws Exception {
    CConfiguration cConf = CConfiguration.create();
    cConf.set(Constants.CFG_LOCAL_DATA_DIR, TEMP_FOLDER.newFolder().getAbsolutePath());
    cConf.set(Constants.Security.Store.PROVIDER, "file");
    SConfiguration sConf = SConfiguration.create();
    sConf.set(Constants.Security.Store.FILE_PASSWORD, "secret");
    Injector injector = Guice.createInjector(new ConfigModule(cConf, new Configuration(), sConf), new SecureStoreServerModule(), new AuthorizationTestModule(), new AuthenticationContextModules().getNoOpModule(), new AbstractModule() {

        @Override
        protected void configure() {
            bind(AccessEnforcer.class).to(NoOpAccessController.class);
            bind(NamespaceAdmin.class).to(InMemoryNamespaceAdmin.class).in(Scopes.SINGLETON);
            bind(NamespaceQueryAdmin.class).to(NamespaceAdmin.class);
        }
    });
    injector.getInstance(NamespaceAdmin.class).create(NamespaceMeta.DEFAULT);
    httpServer = new CommonNettyHttpServiceBuilder(injector.getInstance(CConfiguration.class), "SecureStore").setHttpHandlers(Collections.singleton(injector.getInstance(SecureStoreHandler.class))).build();
    httpServer.start();
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) SConfiguration(io.cdap.cdap.common.conf.SConfiguration) CommonNettyHttpServiceBuilder(io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder) ConfigModule(io.cdap.cdap.common.guice.ConfigModule) AuthenticationContextModules(io.cdap.cdap.security.auth.context.AuthenticationContextModules) InMemoryNamespaceAdmin(io.cdap.cdap.common.namespace.InMemoryNamespaceAdmin) NamespaceAdmin(io.cdap.cdap.common.namespace.NamespaceAdmin) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) AuthorizationTestModule(io.cdap.cdap.security.authorization.AuthorizationTestModule) AbstractModule(com.google.inject.AbstractModule) Injector(com.google.inject.Injector) SConfiguration(io.cdap.cdap.common.conf.SConfiguration) NoOpAccessController(io.cdap.cdap.security.spi.authorization.NoOpAccessController) SecureStoreServerModule(io.cdap.cdap.security.guice.SecureStoreServerModule) BeforeClass(org.junit.BeforeClass)

Aggregations

CommonNettyHttpServiceBuilder (io.cdap.cdap.common.http.CommonNettyHttpServiceBuilder)11 CConfiguration (io.cdap.cdap.common.conf.CConfiguration)6 AuthenticationTestContext (io.cdap.cdap.security.auth.context.AuthenticationTestContext)4 InMemoryAccessController (io.cdap.cdap.security.authorization.InMemoryAccessController)4 NettyHttpService (io.cdap.http.NettyHttpService)4 HttpsEnabler (io.cdap.cdap.common.security.HttpsEnabler)3 BeforeClass (org.junit.BeforeClass)3 MetricsCollectionService (io.cdap.cdap.api.metrics.MetricsCollectionService)2 AccessException (io.cdap.cdap.api.security.AccessException)2 AuthorizationClient (io.cdap.cdap.client.AuthorizationClient)2 Constants (io.cdap.cdap.common.conf.Constants)2 SConfiguration (io.cdap.cdap.common.conf.SConfiguration)2 URIScheme (io.cdap.cdap.common.discovery.URIScheme)2 ConfigModule (io.cdap.cdap.common.guice.ConfigModule)2 DefaultInternalAuthenticator (io.cdap.cdap.common.internal.remote.DefaultInternalAuthenticator)2 InMemoryNamespaceAdmin (io.cdap.cdap.common.namespace.InMemoryNamespaceAdmin)2 NamespaceAdmin (io.cdap.cdap.common.namespace.NamespaceAdmin)2 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)2 ChannelPipelineModifier (io.cdap.http.ChannelPipelineModifier)2 ChannelPipeline (io.netty.channel.ChannelPipeline)2