Search in sources :

Example 1 with ProgramType

use of io.cdap.cdap.api.app.ProgramType in project cdap by caskdata.

the class RemoteConfiguratorTest method testRemoteConfigurator.

@Test
public void testRemoteConfigurator() throws Exception {
    LocationFactory locationFactory = new LocalLocationFactory(TEMP_FOLDER.newFolder());
    Location appJar = AppJarHelper.createDeploymentJar(locationFactory, AllProgramsApp.class);
    ArtifactId artifactId = NamespaceId.DEFAULT.artifact(AllProgramsApp.class.getSimpleName(), "1.0.0");
    artifacts.put(artifactId, new ArtifactDetail(new ArtifactDescriptor(artifactId.getNamespace(), artifactId.toApiArtifactId(), appJar), new ArtifactMeta(ArtifactClasses.builder().build())));
    AppDeploymentInfo info = new AppDeploymentInfo(artifactId, appJar, NamespaceId.DEFAULT, new ApplicationClass(AllProgramsApp.class.getName(), "", null), null, null, null);
    Configurator configurator = new RemoteConfigurator(cConf, metricsCollectionService, info, remoteClientFactory);
    // Extract response from the configurator.
    ListenableFuture<ConfigResponse> result = configurator.config();
    ConfigResponse response = result.get(10, TimeUnit.SECONDS);
    Assert.assertNotNull(response);
    AppSpecInfo appSpecInfo = response.getAppSpecInfo();
    if (appSpecInfo == null) {
        throw new IllegalStateException("Failed to deploy application");
    }
    ApplicationSpecification specification = appSpecInfo.getAppSpec();
    Assert.assertNotNull(specification);
    // Simple checks.
    Assert.assertEquals(AllProgramsApp.NAME, specification.getName());
    ApplicationSpecification expectedSpec = Specifications.from(new AllProgramsApp());
    for (ProgramType programType : ProgramType.values()) {
        Assert.assertEquals(expectedSpec.getProgramsByType(programType), specification.getProgramsByType(programType));
    }
    Assert.assertEquals(expectedSpec.getDatasets(), specification.getDatasets());
}
Also used : ArtifactMeta(io.cdap.cdap.internal.app.runtime.artifact.ArtifactMeta) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) ArtifactId(io.cdap.cdap.proto.id.ArtifactId) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) Configurator(io.cdap.cdap.app.deploy.Configurator) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AllProgramsApp(io.cdap.cdap.AllProgramsApp) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) LocationFactory(org.apache.twill.filesystem.LocationFactory) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) ArtifactDescriptor(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDescriptor) ProgramType(io.cdap.cdap.api.app.ProgramType) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) Location(org.apache.twill.filesystem.Location) ArtifactDetail(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDetail) Test(org.junit.Test)

Example 2 with ProgramType

use of io.cdap.cdap.api.app.ProgramType in project cdap by caskdata.

the class ApplicationSpecificationAdapterTest method testGetProgramIds.

@Test
public void testGetProgramIds() throws IOException {
    ApplicationSpecification appSpec = Specifications.from(new AllProgramsApp());
    ApplicationId appId = NamespaceId.DEFAULT.app(appSpec.getName());
    Set<ProgramId> expectedIds = new HashSet<>();
    for (ProgramType programType : ProgramType.values()) {
        appSpec.getProgramsByType(programType).stream().map(name -> appId.program(io.cdap.cdap.proto.ProgramType.valueOf(programType.name()), name)).forEach(expectedIds::add);
    }
    String json = ApplicationSpecificationAdapter.create().toJson(appSpec);
    try (JsonReader reader = new JsonReader(new StringReader(json))) {
        Set<ProgramId> programIds = ApplicationSpecificationAdapter.getProgramIds(appId, reader);
        Assert.assertEquals(expectedIds, programIds);
    }
}
Also used : NamespaceId(io.cdap.cdap.proto.id.NamespaceId) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) ProgramId(io.cdap.cdap.proto.id.ProgramId) Set(java.util.Set) ProgramType(io.cdap.cdap.api.app.ProgramType) Test(org.junit.Test) IOException(java.io.IOException) JsonReader(com.google.gson.stream.JsonReader) HashSet(java.util.HashSet) StringReader(java.io.StringReader) AllProgramsApp(io.cdap.cdap.AllProgramsApp) Assert(org.junit.Assert) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) Specifications(io.cdap.cdap.internal.app.deploy.Specifications) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) StringReader(java.io.StringReader) JsonReader(com.google.gson.stream.JsonReader) AllProgramsApp(io.cdap.cdap.AllProgramsApp) ProgramType(io.cdap.cdap.api.app.ProgramType) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) ProgramId(io.cdap.cdap.proto.id.ProgramId) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 3 with ProgramType

use of io.cdap.cdap.api.app.ProgramType in project cdap by caskdata.

the class ConfiguratorTest method testInMemoryConfigurator.

@Test
public void testInMemoryConfigurator() throws Exception {
    LocationFactory locationFactory = new LocalLocationFactory(TMP_FOLDER.newFolder());
    Location appJar = AppJarHelper.createDeploymentJar(locationFactory, AllProgramsApp.class);
    Id.Artifact artifactId = Id.Artifact.from(Id.Namespace.DEFAULT, AllProgramsApp.class.getSimpleName(), "1.0.0");
    CConfiguration cConf = CConfiguration.create();
    ArtifactRepository baseArtifactRepo = new DefaultArtifactRepository(conf, null, null, null, new DummyProgramRunnerFactory(), new DefaultImpersonator(cConf, null));
    ArtifactRepository artifactRepo = new AuthorizationArtifactRepository(baseArtifactRepo, authEnforcer, authenticationContext);
    PluginFinder pluginFinder = new LocalPluginFinder(artifactRepo);
    AppDeploymentInfo appDeploymentInfo = new AppDeploymentInfo(artifactId.toEntityId(), appJar, NamespaceId.DEFAULT, new ApplicationClass(AllProgramsApp.class.getName(), "", null), null, null, null);
    // Create a configurator that is testable. Provide it a application.
    Configurator configurator = new InMemoryConfigurator(conf, pluginFinder, new DefaultImpersonator(cConf, null), artifactRepo, null, appDeploymentInfo);
    // Extract response from the configurator.
    ListenableFuture<ConfigResponse> result = configurator.config();
    ConfigResponse response = result.get(10, TimeUnit.SECONDS);
    Assert.assertNotNull(response);
    AppSpecInfo appSpecInfo = response.getAppSpecInfo();
    if (appSpecInfo == null) {
        throw new IllegalStateException("Failed to deploy application");
    }
    ApplicationSpecification specification = appSpecInfo.getAppSpec();
    Assert.assertNotNull(specification);
    // Simple checks.
    Assert.assertEquals(AllProgramsApp.NAME, specification.getName());
    ApplicationSpecification expectedSpec = Specifications.from(new AllProgramsApp());
    for (ProgramType programType : ProgramType.values()) {
        Assert.assertEquals(expectedSpec.getProgramsByType(programType), specification.getProgramsByType(programType));
    }
    Assert.assertEquals(expectedSpec.getDatasets(), specification.getDatasets());
}
Also used : ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) AuthorizationArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository) DummyProgramRunnerFactory(io.cdap.cdap.app.runtime.DummyProgramRunnerFactory) LocalPluginFinder(io.cdap.cdap.internal.app.runtime.artifact.LocalPluginFinder) AppSpecInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo) Configurator(io.cdap.cdap.app.deploy.Configurator) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) AllProgramsApp(io.cdap.cdap.AllProgramsApp) AuthorizationArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.AuthorizationArtifactRepository) DefaultArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.DefaultArtifactRepository) ArtifactRepository(io.cdap.cdap.internal.app.runtime.artifact.ArtifactRepository) ConfigResponse(io.cdap.cdap.app.deploy.ConfigResponse) CConfiguration(io.cdap.cdap.common.conf.CConfiguration) DefaultImpersonator(io.cdap.cdap.security.impersonation.DefaultImpersonator) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) LocationFactory(org.apache.twill.filesystem.LocationFactory) AppDeploymentInfo(io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo) PluginFinder(io.cdap.cdap.internal.app.runtime.artifact.PluginFinder) LocalPluginFinder(io.cdap.cdap.internal.app.runtime.artifact.LocalPluginFinder) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) Id(io.cdap.cdap.common.id.Id) ProgramType(io.cdap.cdap.api.app.ProgramType) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) Location(org.apache.twill.filesystem.Location) Test(org.junit.Test)

Example 4 with ProgramType

use of io.cdap.cdap.api.app.ProgramType in project cdap by caskdata.

the class AuditPublishTest method testPublish.

@Test
public void testPublish() throws Exception {
    String appName = AllProgramsApp.NAME;
    ApplicationId appId = NamespaceId.DEFAULT.app(appName);
    ApplicationSpecification spec = Specifications.from(new AllProgramsApp());
    // Define expected values
    Set<EntityId> expectedMetadataChangeEntities = new HashSet<>();
    // Metadata change on the artifact and app
    expectedMetadataChangeEntities.add(NamespaceId.DEFAULT.artifact(AllProgramsApp.class.getSimpleName(), "1"));
    expectedMetadataChangeEntities.add(appId);
    // All programs would have metadata change
    for (ProgramType programType : ProgramType.values()) {
        for (String programName : spec.getProgramsByType(programType)) {
            io.cdap.cdap.proto.ProgramType internalProgramType = io.cdap.cdap.proto.ProgramType.valueOf(programType.name());
            expectedMetadataChangeEntities.add(appId.program(internalProgramType, programName));
        }
    }
    // All dataset would have metadata change as well as creation
    Set<EntityId> expectedCreateEntities = new HashSet<>();
    for (String dataset : spec.getDatasets().keySet()) {
        expectedCreateEntities.add(NamespaceId.DEFAULT.dataset(dataset));
    }
    expectedMetadataChangeEntities.addAll(expectedCreateEntities);
    // TODO (CDAP-14733): Scheduler doesn't publish CREATE audit events. Once it does, we must expect them here, too.
    for (String schedule : spec.getProgramSchedules().keySet()) {
        expectedMetadataChangeEntities.add(appId.schedule(schedule));
    }
    Multimap<AuditType, EntityId> expectedAuditEntities = HashMultimap.create();
    expectedAuditEntities.putAll(AuditType.METADATA_CHANGE, expectedMetadataChangeEntities);
    expectedAuditEntities.putAll(AuditType.CREATE, expectedCreateEntities);
    // Deploy application
    AppFabricTestHelper.deployApplication(Id.Namespace.DEFAULT, AllProgramsApp.class, null, cConf);
    // Verify audit messages
    Tasks.waitFor(expectedAuditEntities, () -> {
        List<AuditMessage> publishedMessages = fetchAuditMessages();
        Multimap<AuditType, EntityId> actualAuditEntities = HashMultimap.create();
        for (AuditMessage message : publishedMessages) {
            EntityId entityId = EntityId.fromMetadataEntity(message.getEntity());
            if (entityId instanceof NamespacedEntityId) {
                if (((NamespacedEntityId) entityId).getNamespace().equals(NamespaceId.SYSTEM.getNamespace())) {
                    // Ignore system audit messages
                    continue;
                }
            }
            if (entityId.getEntityType() == EntityType.ARTIFACT && entityId instanceof ArtifactId) {
                ArtifactId artifactId = (ArtifactId) entityId;
                // Version is dynamic for deploys in test cases
                entityId = Ids.namespace(artifactId.getNamespace()).artifact(artifactId.getArtifact(), "1");
            }
            actualAuditEntities.put(message.getType(), entityId);
        }
        return actualAuditEntities;
    }, 5, TimeUnit.SECONDS);
}
Also used : ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) AuditMessage(io.cdap.cdap.proto.audit.AuditMessage) ArtifactId(io.cdap.cdap.proto.id.ArtifactId) AuditType(io.cdap.cdap.proto.audit.AuditType) AllProgramsApp(io.cdap.cdap.AllProgramsApp) NamespacedEntityId(io.cdap.cdap.proto.id.NamespacedEntityId) EntityId(io.cdap.cdap.proto.id.EntityId) NamespacedEntityId(io.cdap.cdap.proto.id.NamespacedEntityId) ProgramType(io.cdap.cdap.api.app.ProgramType) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

AllProgramsApp (io.cdap.cdap.AllProgramsApp)4 ApplicationSpecification (io.cdap.cdap.api.app.ApplicationSpecification)4 ProgramType (io.cdap.cdap.api.app.ProgramType)4 Test (org.junit.Test)4 ApplicationClass (io.cdap.cdap.api.artifact.ApplicationClass)2 ConfigResponse (io.cdap.cdap.app.deploy.ConfigResponse)2 Configurator (io.cdap.cdap.app.deploy.Configurator)2 AppDeploymentInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo)2 AppSpecInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppSpecInfo)2 ApplicationId (io.cdap.cdap.proto.id.ApplicationId)2 ArtifactId (io.cdap.cdap.proto.id.ArtifactId)2 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)2 HashSet (java.util.HashSet)2 LocalLocationFactory (org.apache.twill.filesystem.LocalLocationFactory)2 Location (org.apache.twill.filesystem.Location)2 LocationFactory (org.apache.twill.filesystem.LocationFactory)2 JsonReader (com.google.gson.stream.JsonReader)1 DummyProgramRunnerFactory (io.cdap.cdap.app.runtime.DummyProgramRunnerFactory)1 CConfiguration (io.cdap.cdap.common.conf.CConfiguration)1 Id (io.cdap.cdap.common.id.Id)1