Search in sources :

Example 6 with ConfigResponse

use of io.cdap.cdap.app.deploy.ConfigResponse in project cdap by caskdata.

the class RemoteConfiguratorTest method testRemoteConfigurator.

@Test
public void testRemoteConfigurator() throws Exception {
    LocationFactory locationFactory = new LocalLocationFactory(TEMP_FOLDER.newFolder());
    Location appJar = AppJarHelper.createDeploymentJar(locationFactory, AllProgramsApp.class);
    ArtifactId artifactId = NamespaceId.DEFAULT.artifact(AllProgramsApp.class.getSimpleName(), "1.0.0");
    artifacts.put(artifactId, new ArtifactDetail(new ArtifactDescriptor(artifactId.getNamespace(), artifactId.toApiArtifactId(), appJar), new ArtifactMeta(ArtifactClasses.builder().build())));
    AppDeploymentInfo info = new AppDeploymentInfo(artifactId, appJar, NamespaceId.DEFAULT, new ApplicationClass(AllProgramsApp.class.getName(), "", null), null, null, null);
    Configurator configurator = new RemoteConfigurator(cConf, metricsCollectionService, info, remoteClientFactory);
    // Extract response from the configurator.
    ListenableFuture<ConfigResponse> result = configurator.config();
    ConfigResponse response = result.get(10, TimeUnit.SECONDS);
    Assert.assertNotNull(response);
    AppSpecInfo appSpecInfo = response.getAppSpecInfo();
    if (appSpecInfo == null) {
        throw new IllegalStateException("Failed to deploy application");
    }
    ApplicationSpecification specification = appSpecInfo.getAppSpec();
    Assert.assertNotNull(specification);
    // Simple checks.
    Assert.assertEquals(AllProgramsApp.NAME, specification.getName());
    ApplicationSpecification expectedSpec = Specifications.from(new AllProgramsApp());
    for (ProgramType programType : ProgramType.values()) {
        Assert.assertEquals(expectedSpec.getProgramsByType(programType), specification.getProgramsByType(programType));
    }
    Assert.assertEquals(expectedSpec.getDatasets(), specification.getDatasets());
}
Also used : ArtifactMeta(io.cdap.cdap.internal.app.runtime.artifact.ArtifactMeta) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) ArtifactId(io.cdap.cdap.proto.id.ArtifactId) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) Configurator(io.cdap.cdap.app.deploy.Configurator) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AllProgramsApp(io.cdap.cdap.AllProgramsApp) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) LocationFactory(org.apache.twill.filesystem.LocationFactory) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) ArtifactDescriptor(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor) ProgramType(io.cdap.cdap.api.app.ProgramType) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) Location(org.apache.twill.filesystem.Location) ArtifactDetail(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDetail) Test(org.junit.Test)

Example 7 with ConfigResponse

use of io.cdap.cdap.app.deploy.ConfigResponse in project cdap by cdapio.

the class DefaultRuntimeJob method regenerateAppSpec.

@Nullable
private ApplicationSpecification regenerateAppSpec(Map<String, String> systemArguments, Map<String, String> userArguments, ProgramId programId, ApplicationSpecification existingAppSpec, ProgramDescriptor programDescriptor, ConfiguratorFactory configuratorFactory) throws InterruptedException, ExecutionException, TimeoutException {
    String appClassName = systemArguments.get(ProgramOptionConstants.APPLICATION_CLASS);
    Location programJarLocation = Locations.toLocation(new File(systemArguments.get(ProgramOptionConstants.PROGRAM_JAR)));
    AppDeploymentInfo deploymentInfo = new AppDeploymentInfo(programDescriptor.getArtifactId(), programJarLocation, programId.getNamespaceId(), appClassName, programId.getApplication(), programId.getVersion(), existingAppSpec.getConfiguration(), null, false, new AppDeploymentRuntimeInfo(existingAppSpec, userArguments, systemArguments));
    Configurator configurator = configuratorFactory.create(deploymentInfo);
    ListenableFuture<ConfigResponse> future = configurator.config();
    ConfigResponse response = future.get(120, TimeUnit.SECONDS);
    if (response.getExitCode() == 0) {
        AppSpecInfo appSpecInfo = response.getAppSpecInfo();
        if (appSpecInfo != null && appSpecInfo.getAppSpec() != null) {
            return appSpecInfo.getAppSpec();
        }
    }
    return null;
}
Also used : AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) InMemoryConfigurator(io.cdap.cdap.internal.app.deploy.InMemoryConfigurator) Configurator(io.cdap.cdap.app.deploy.Configurator) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) File(java.io.File) Location(org.apache.twill.filesystem.Location) AppDeploymentRuntimeInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentRuntimeInfo) Nullable(javax.annotation.Nullable)

Example 8 with ConfigResponse

use of io.cdap.cdap.app.deploy.ConfigResponse in project cdap by cdapio.

the class ConfiguratorTest method testInMemoryConfigurator.

@Test
public void testInMemoryConfigurator() throws Exception {
    LocationFactory locationFactory = new LocalLocationFactory(TMP_FOLDER.newFolder());
    Location appJar = AppJarHelper.createDeploymentJar(locationFactory, AllProgramsApp.class);
    Id.Artifact artifactId = Id.Artifact.from(Id.Namespace.DEFAULT, AllProgramsApp.class.getSimpleName(), "1.0.0");
    CConfiguration cConf = CConfiguration.create();
    ArtifactRepository baseArtifactRepo = new DefaultArtifactRepository(conf, null, null, null, new DummyProgramRunnerFactory(), new DefaultImpersonator(cConf, null));
    ArtifactRepository artifactRepo = new AuthorizationArtifactRepository(baseArtifactRepo, authEnforcer, authenticationContext);
    PluginFinder pluginFinder = new LocalPluginFinder(artifactRepo);
    AppDeploymentInfo appDeploymentInfo = new AppDeploymentInfo(artifactId.toEntityId(), appJar, NamespaceId.DEFAULT, new ApplicationClass(AllProgramsApp.class.getName(), "", null), null, null, null);
    // Create a configurator that is testable. Provide it a application.
    Configurator configurator = new InMemoryConfigurator(conf, pluginFinder, new DefaultImpersonator(cConf, null), artifactRepo, null, appDeploymentInfo);
    // Extract response from the configurator.
    ListenableFuture<ConfigResponse> result = configurator.config();
    ConfigResponse response = result.get(10, TimeUnit.SECONDS);
    Assert.assertNotNull(response);
    AppSpecInfo appSpecInfo = response.getAppSpecInfo();
    if (appSpecInfo == null) {
        throw new IllegalStateException("Failed to deploy application");
    }
    ApplicationSpecification specification = appSpecInfo.getAppSpec();
    Assert.assertNotNull(specification);
    // Simple checks.
    Assert.assertEquals(AllProgramsApp.NAME, specification.getName());
    ApplicationSpecification expectedSpec = Specifications.from(new AllProgramsApp());
    for (ProgramType programType : ProgramType.values()) {
        Assert.assertEquals(expectedSpec.getProgramsByType(programType), specification.getProgramsByType(programType));
    }
    Assert.assertEquals(expectedSpec.getDatasets(), specification.getDatasets());
}
Also used : ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) AuthorizationArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository) DummyProgramRunnerFactory(io.cdap.cdap.app.runtime.DummyProgramRunnerFactory) LocalPluginFinder(io.cdap.cdap.internal.app.runtime.artifact.LocalPluginFinder) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) Configurator(io.cdap.cdap.app.deploy.Configurator) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AllProgramsApp(io.cdap.cdap.AllProgramsApp) AuthorizationArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) ArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepository) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) DefaultImpersonator(io.cdap.cdap.security.impersonation.DefaultImpersonator) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) LocationFactory(org.apache.twill.filesystem.LocationFactory) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) PluginFinder(io.cdap.cdap.internal.app.runtime.artifact.PluginFinder) LocalPluginFinder(io.cdap.cdap.internal.app.runtime.artifact.LocalPluginFinder) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) Id(io.cdap.cdap.common.id.Id) ProgramType(io.cdap.cdap.api.app.ProgramType) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) Location(org.apache.twill.filesystem.Location) Test(org.junit.Test)

Example 9 with ConfigResponse

use of io.cdap.cdap.app.deploy.ConfigResponse in project cdap by cdapio.

the class ConfiguratorTest method testAppWithConfig.

@Test
public void testAppWithConfig() throws Exception {
    LocationFactory locationFactory = new LocalLocationFactory(TMP_FOLDER.newFolder());
    Location appJar = AppJarHelper.createDeploymentJar(locationFactory, ConfigTestApp.class);
    Id.Artifact artifactId = Id.Artifact.from(Id.Namespace.DEFAULT, ConfigTestApp.class.getSimpleName(), "1.0.0");
    CConfiguration cConf = CConfiguration.create();
    ArtifactRepository baseArtifactRepo = new DefaultArtifactRepository(conf, null, null, null, new DummyProgramRunnerFactory(), new DefaultImpersonator(cConf, null));
    ArtifactRepository artifactRepo = new AuthorizationArtifactRepository(baseArtifactRepo, authEnforcer, authenticationContext);
    PluginFinder pluginFinder = new LocalPluginFinder(artifactRepo);
    ConfigTestApp.ConfigClass config = new ConfigTestApp.ConfigClass("myTable");
    AppDeploymentInfo appDeploymentInfo = new AppDeploymentInfo(artifactId.toEntityId(), appJar, NamespaceId.DEFAULT, new ApplicationClass(ConfigTestApp.class.getName(), "", null), null, null, new Gson().toJson(config));
    // Create a configurator that is testable. Provide it an application.
    Configurator configurator = new InMemoryConfigurator(conf, pluginFinder, new DefaultImpersonator(cConf, null), artifactRepo, null, appDeploymentInfo);
    ListenableFuture<ConfigResponse> result = configurator.config();
    ConfigResponse response = result.get(10, TimeUnit.SECONDS);
    Assert.assertNotNull(response);
    AppSpecInfo appSpecInfo = response.getAppSpecInfo();
    if (appSpecInfo == null) {
        throw new IllegalStateException("Failed to deploy application");
    }
    ApplicationSpecification specification = appSpecInfo.getAppSpec();
    Assert.assertNotNull(specification);
    Assert.assertEquals(1, specification.getDatasets().size());
    Assert.assertTrue(specification.getDatasets().containsKey("myTable"));
    // Create a deployment info without the app configuration
    appDeploymentInfo = new AppDeploymentInfo(artifactId.toEntityId(), appJar, NamespaceId.DEFAULT, new ApplicationClass(ConfigTestApp.class.getName(), "", null), null, null, null);
    Configurator configuratorWithoutConfig = new InMemoryConfigurator(conf, pluginFinder, new DefaultImpersonator(cConf, null), artifactRepo, null, appDeploymentInfo);
    result = configuratorWithoutConfig.config();
    response = result.get(10, TimeUnit.SECONDS);
    Assert.assertNotNull(response);
    appSpecInfo = response.getAppSpecInfo();
    if (appSpecInfo == null) {
        throw new IllegalStateException("Failed to deploy application");
    }
    specification = appSpecInfo.getAppSpec();
    Assert.assertNotNull(specification);
    Assert.assertEquals(1, specification.getDatasets().size());
    Assert.assertTrue(specification.getDatasets().containsKey(ConfigTestApp.DEFAULT_TABLE));
    Assert.assertNotNull(specification.getProgramSchedules().get(ConfigTestApp.SCHEDULE_NAME));
    ProgramStatusTrigger trigger = (ProgramStatusTrigger) specification.getProgramSchedules().get(ConfigTestApp.SCHEDULE_NAME).getTrigger();
    Assert.assertEquals(trigger.getProgramId().getProgram(), ConfigTestApp.WORKFLOW_NAME);
}
Also used : ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) LocalPluginFinder(io.cdap.cdap.internal.app.runtime.artifact.LocalPluginFinder) Configurator(io.cdap.cdap.app.deploy.Configurator) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) Gson(com.google.gson.Gson) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) PluginFinder(io.cdap.cdap.internal.app.runtime.artifact.PluginFinder) LocalPluginFinder(io.cdap.cdap.internal.app.runtime.artifact.LocalPluginFinder) ProgramStatusTrigger(io.cdap.cdap.internal.app.runtime.schedule.trigger.ProgramStatusTrigger) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) ConfigTestApp(io.cdap.cdap.ConfigTestApp) AuthorizationArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository) DummyProgramRunnerFactory(io.cdap.cdap.app.runtime.DummyProgramRunnerFactory) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AuthorizationArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) ArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepository) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) DefaultImpersonator(io.cdap.cdap.security.impersonation.DefaultImpersonator) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) LocationFactory(org.apache.twill.filesystem.LocationFactory) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) Id(io.cdap.cdap.common.id.Id) Location(org.apache.twill.filesystem.Location) Test(org.junit.Test)

Example 10 with ConfigResponse

use of io.cdap.cdap.app.deploy.ConfigResponse in project cdap by cdapio.

the class InMemoryConfigurator method config.

/**
 * Executes the <code>Application.configure</code> within the same JVM.
 * <p>
 * This method could be dangerous and should be used only in standalone mode.
 * </p>
 *
 * @return A instance of {@link ListenableFuture}.
 */
@Override
public ListenableFuture<ConfigResponse> config() {
    // Create the classloader
    EntityImpersonator classLoaderImpersonator = new EntityImpersonator(artifactId.toEntityId(), impersonator);
    try (CloseableClassLoader classLoader = artifactRepository.createArtifactClassLoader(new ArtifactDescriptor(artifactId.getNamespace().getId(), artifactId.toArtifactId(), artifactLocation), classLoaderImpersonator)) {
        SettableFuture<ConfigResponse> result = SettableFuture.create();
        Object appMain = classLoader.loadClass(appClassName).newInstance();
        if (!(appMain instanceof Application)) {
            throw new IllegalStateException(String.format("Application main class is of invalid type: %s", appMain.getClass().getName()));
        }
        Application<?> app = (Application<?>) appMain;
        ConfigResponse response = createResponse(app, classLoader);
        result.set(response);
        return result;
    } catch (Throwable t) {
        return Futures.immediateFailedFuture(t);
    }
}
Also used : ArtifactDescriptor(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor) EntityImpersonator(io.cdap.cdap.security.impersonation.EntityImpersonator) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) CloseableClassLoader(io.cdap.cdap.api.artifact.CloseableClassLoader) Application(io.cdap.cdap.api.app.Application)

Aggregations

ConfigResponse (io.cdap.cdap.app.deploy.ConfigResponse)18 Configurator (io.cdap.cdap.app.deploy.Configurator)14 AppDeploymentInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo)14 AppSpecInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo)14 ApplicationClass (io.cdap.cdap.api.artifact.ApplicationClass)10 ApplicationSpecification (io.cdap.cdap.api.app.ApplicationSpecification)8 Location (org.apache.twill.filesystem.Location)8 AppDeploymentRuntimeInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentRuntimeInfo)6 Nullable (javax.annotation.Nullable)6 LocalLocationFactory (org.apache.twill.filesystem.LocalLocationFactory)6 LocationFactory (org.apache.twill.filesystem.LocationFactory)6 Test (org.junit.Test)6 AllProgramsApp (io.cdap.cdap.AllProgramsApp)4 ProgramType (io.cdap.cdap.api.app.ProgramType)4 DummyProgramRunnerFactory (io.cdap.cdap.app.runtime.DummyProgramRunnerFactory)4 CConfiguration (io.cdap.cdap.common.conf.CConfiguration)4 Id (io.cdap.cdap.common.id.Id)4 ArtifactDescriptor (io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor)4 ArtifactRepository (io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepository)4 AuthorizationArtifactRepository (io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository)4