Search in sources :

Example 6 with ArtifactSummary

use of io.cdap.cdap.api.artifact.ArtifactSummary in project cdap by caskdata.

the class DefaultArtifactRepository method getApplicationClasses.

@Override
public List<ApplicationClassInfo> getApplicationClasses(NamespaceId namespace, String className) throws IOException {
    List<ApplicationClassInfo> infos = Lists.newArrayList();
    for (Map.Entry<ArtifactDescriptor, ApplicationClass> entry : artifactStore.getApplicationClasses(namespace, className).entrySet()) {
        ArtifactSummary artifactSummary = ArtifactSummary.from(entry.getKey().getArtifactId());
        ApplicationClass appClass = entry.getValue();
        infos.add(new ApplicationClassInfo(artifactSummary, appClass.getClassName(), appClass.getConfigSchema()));
    }
    return Collections.unmodifiableList(infos);
}
Also used : ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ApplicationClassInfo(io.cdap.cdap.proto.artifact.ApplicationClassInfo) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) Map(java.util.Map) SortedMap(java.util.SortedMap) HashMap(java.util.HashMap)

Example 7 with ArtifactSummary

use of io.cdap.cdap.api.artifact.ArtifactSummary in project cdap by caskdata.

the class ClientUpgradeContext method getPluginArtifact.

@Nullable
@Override
public ArtifactSelectorConfig getPluginArtifact(String pluginType, String pluginName) {
    try {
        List<PluginInfo> plugins = artifactClient.getPluginInfo(getArtifactId(), pluginType, pluginName, ArtifactScope.SYSTEM);
        if (plugins.isEmpty()) {
            return null;
        }
        // doesn't really matter which one we choose, as all of them should be valid.
        // choosing the last one because that tends to be the one with the highest version.
        // order is not guaranteed though.
        ArtifactSummary chosenArtifact = plugins.get(plugins.size() - 1).getArtifact();
        return new ArtifactSelectorConfig(chosenArtifact.getScope().name(), chosenArtifact.getName(), chosenArtifact.getVersion());
    } catch (Exception e) {
        LOG.warn("Unable to find an artifact for plugin of type {} and name {}. " + "Plugin artifact section will be left empty.", pluginType, pluginName);
        return null;
    }
}
Also used : ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ArtifactSelectorConfig(io.cdap.cdap.etl.proto.ArtifactSelectorConfig) PluginInfo(io.cdap.cdap.proto.artifact.PluginInfo) Nullable(javax.annotation.Nullable)

Example 8 with ArtifactSummary

use of io.cdap.cdap.api.artifact.ArtifactSummary in project cdap by caskdata.

the class ScheduledRunTimeTest method testGetNextRun.

@Test
public void testGetNextRun() throws Exception {
    ApplicationId appId = NamespaceId.DEFAULT.app("test");
    deploy(appId, new AppRequest<>(new ArtifactSummary(ARTIFACT_ID.getName(), ARTIFACT_ID.getVersion().getVersion())));
    String scheduleName = "schedule1";
    // Add a schedule. Use a constraint to make it not going to be executed
    ProgramId programId = appId.workflow(WorkflowApp.FunWorkflow.NAME);
    Constraint constraint = new DelayConstraint(1, TimeUnit.HOURS);
    ScheduleProgramInfo scheduleProgramInfo = new ScheduleProgramInfo(programId.getType().getSchedulableType(), programId.getProgram());
    addSchedule(appId.getNamespace(), appId.getApplication(), appId.getVersion(), scheduleName, new ScheduleDetail(scheduleName, null, scheduleProgramInfo, null, new TimeTrigger("0 0 * * * "), Collections.singletonList(constraint), null));
    long now = System.currentTimeMillis();
    HttpResponse response = enableSchedule(programId.getNamespace(), programId.getApplication(), programId.getVersion(), scheduleName);
    Assert.assertEquals(200, response.getResponseCode());
    // Get the next run time
    List<ScheduledRuntime> scheduledRunTimes = getScheduledRunTimes(programId, true);
    Assert.assertEquals(1, scheduledRunTimes.size());
    long nextTime = scheduledRunTimes.get(0).getTime();
    Assert.assertTrue(nextTime >= now);
}
Also used : TimeTrigger(io.cdap.cdap.internal.app.runtime.schedule.trigger.TimeTrigger) Constraint(io.cdap.cdap.internal.schedule.constraint.Constraint) DelayConstraint(io.cdap.cdap.internal.app.runtime.schedule.constraint.DelayConstraint) HttpResponse(io.cdap.common.http.HttpResponse) ProgramId(io.cdap.cdap.proto.id.ProgramId) DelayConstraint(io.cdap.cdap.internal.app.runtime.schedule.constraint.DelayConstraint) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ScheduleDetail(io.cdap.cdap.proto.ScheduleDetail) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) ScheduleProgramInfo(io.cdap.cdap.api.workflow.ScheduleProgramInfo) ScheduledRuntime(io.cdap.cdap.proto.ScheduledRuntime) Test(org.junit.Test)

Example 9 with ArtifactSummary

use of io.cdap.cdap.api.artifact.ArtifactSummary in project cdap by caskdata.

the class ScheduledRunTimeTest method testBatchGetNextRun.

@Test
public void testBatchGetNextRun() throws Exception {
    // deploys 5 apps and create schedules for each of them
    long now = System.currentTimeMillis();
    List<ProgramId> programIds = new ArrayList<>();
    // Use a constraint to make it not going to be executed
    Constraint constraint = new DelayConstraint(1, TimeUnit.HOURS);
    for (int i = 0; i < 5; i++) {
        ApplicationId appId = NamespaceId.DEFAULT.app("test" + i);
        deploy(appId, new AppRequest<>(new ArtifactSummary(ARTIFACT_ID.getName(), ARTIFACT_ID.getVersion().getVersion())));
        String scheduleName = "schedule" + i;
        // Add a schedule
        ProgramId programId = appId.workflow(WorkflowApp.FunWorkflow.NAME);
        programIds.add(programId);
        ScheduleProgramInfo scheduleProgramInfo = new ScheduleProgramInfo(programId.getType().getSchedulableType(), programId.getProgram());
        addSchedule(appId.getNamespace(), appId.getApplication(), appId.getVersion(), scheduleName, new ScheduleDetail(scheduleName, null, scheduleProgramInfo, null, new TimeTrigger("0 0 * * * "), Collections.singletonList(constraint), null));
        HttpResponse response = enableSchedule(programId.getNamespace(), programId.getApplication(), programId.getVersion(), scheduleName);
        Assert.assertEquals(200, response.getResponseCode());
    }
    // Add programs that the app or the program doesn't exist
    programIds.add(NamespaceId.DEFAULT.app("not-exist").workflow("not-exist"));
    programIds.add(NamespaceId.DEFAULT.app("test1").workflow("not-exist"));
    List<BatchProgramSchedule> schedules = getScheduledRunTimes(NamespaceId.DEFAULT.getNamespace(), programIds, true);
    Assert.assertEquals(programIds.size(), schedules.size());
    // For the first 5 programs, they should have a next run
    for (int i = 0; i < 5; i++) {
        BatchProgramSchedule schedule = schedules.get(i);
        Assert.assertEquals(200, schedule.getStatusCode());
        List<ScheduledRuntime> nextRuns = schedule.getSchedules();
        Assert.assertNotNull(nextRuns);
        Assert.assertEquals(1, nextRuns.size());
        long nextTime = nextRuns.get(0).getTime();
        Assert.assertTrue(nextTime >= now);
    }
    // The last two should be a not found
    Assert.assertEquals(404, schedules.get(5).getStatusCode());
    Assert.assertEquals(404, schedules.get(6).getStatusCode());
}
Also used : BatchProgramSchedule(io.cdap.cdap.proto.BatchProgramSchedule) TimeTrigger(io.cdap.cdap.internal.app.runtime.schedule.trigger.TimeTrigger) Constraint(io.cdap.cdap.internal.schedule.constraint.Constraint) DelayConstraint(io.cdap.cdap.internal.app.runtime.schedule.constraint.DelayConstraint) ArrayList(java.util.ArrayList) HttpResponse(io.cdap.common.http.HttpResponse) ProgramId(io.cdap.cdap.proto.id.ProgramId) Constraint(io.cdap.cdap.internal.schedule.constraint.Constraint) DelayConstraint(io.cdap.cdap.internal.app.runtime.schedule.constraint.DelayConstraint) DelayConstraint(io.cdap.cdap.internal.app.runtime.schedule.constraint.DelayConstraint) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ScheduleDetail(io.cdap.cdap.proto.ScheduleDetail) ApplicationId(io.cdap.cdap.proto.id.ApplicationId) ScheduleProgramInfo(io.cdap.cdap.api.workflow.ScheduleProgramInfo) ScheduledRuntime(io.cdap.cdap.proto.ScheduledRuntime) Test(org.junit.Test)

Example 10 with ArtifactSummary

use of io.cdap.cdap.api.artifact.ArtifactSummary in project cdap by caskdata.

the class DataPipelineServiceTest method testValidatePipelineBadPipelineArtifact.

@Test
public void testValidatePipelineBadPipelineArtifact() throws IOException {
    ETLBatchConfig config = ETLBatchConfig.builder().addStage(new ETLStage("src", MockSource.getPlugin("dummy1"))).addStage(new ETLStage("sink", MockSink.getPlugin("dummy2"))).addConnection("src", "sink").build();
    ArtifactSummary badArtifact = new ArtifactSummary("ghost", "1.0.0");
    AppRequest<ETLBatchConfig> appRequest = new AppRequest<>(badArtifact, config);
    URL validatePipelineURL = serviceURI.resolve(String.format("v1/contexts/%s/validations/pipeline", NamespaceId.DEFAULT.getNamespace())).toURL();
    HttpRequest request = HttpRequest.builder(HttpMethod.POST, validatePipelineURL).withBody(GSON.toJson(appRequest)).build();
    HttpResponse response = HttpRequests.execute(request, new DefaultHttpRequestConfig(false));
    Assert.assertEquals(400, response.getResponseCode());
}
Also used : ETLBatchConfig(io.cdap.cdap.etl.proto.v2.ETLBatchConfig) HttpRequest(io.cdap.common.http.HttpRequest) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ETLStage(io.cdap.cdap.etl.proto.v2.ETLStage) DefaultHttpRequestConfig(io.cdap.cdap.common.http.DefaultHttpRequestConfig) HttpResponse(io.cdap.common.http.HttpResponse) URL(java.net.URL) AppRequest(io.cdap.cdap.proto.artifact.AppRequest) Test(org.junit.Test)

Aggregations

ArtifactSummary (io.cdap.cdap.api.artifact.ArtifactSummary)152 ApplicationId (io.cdap.cdap.proto.id.ApplicationId)86 AppRequest (io.cdap.cdap.proto.artifact.AppRequest)80 Test (org.junit.Test)70 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)48 ArtifactId (io.cdap.cdap.proto.id.ArtifactId)44 ProgramId (io.cdap.cdap.proto.id.ProgramId)44 Id (io.cdap.cdap.common.id.Id)36 ProfileId (io.cdap.cdap.proto.id.ProfileId)26 HttpResponse (io.cdap.common.http.HttpResponse)26 IOException (java.io.IOException)22 URL (java.net.URL)22 JsonObject (com.google.gson.JsonObject)18 NotFoundException (io.cdap.cdap.common.NotFoundException)18 ArtifactVersion (io.cdap.cdap.api.artifact.ArtifactVersion)16 ArtifactNotFoundException (io.cdap.cdap.common.ArtifactNotFoundException)16 File (java.io.File)16 Map (java.util.Map)16 ArtifactId (io.cdap.cdap.api.artifact.ArtifactId)14 KerberosPrincipalId (io.cdap.cdap.proto.id.KerberosPrincipalId)14