Search in sources :

Example 16 with ArtifactId

use of io.cdap.cdap.api.artifact.ArtifactId in project cdap by caskdata.

the class DirectRuntimeRequestValidatorTest method testFetcher.

@Test
public void testFetcher() throws BadRequestException {
    ArtifactId artifactId = new ArtifactId("test", new ArtifactVersion("1.0"), ArtifactScope.USER);
    ProgramRunId programRunId = NamespaceId.DEFAULT.app("app").spark("spark").run(RunIds.generate());
    ProgramRunStatus programRunStatus = ProgramRunStatus.RUNNING;
    RunRecordDetail runRecord = RunRecordDetail.builder().setProgramRunId(programRunId).setStartTime(System.currentTimeMillis()).setArtifactId(artifactId).setStatus(programRunStatus).setSystemArgs(ImmutableMap.of(SystemArguments.PROFILE_NAME, "default", SystemArguments.PROFILE_PROVISIONER, "native")).setProfileId(NamespaceId.DEFAULT.profile("native")).setSourceId(new byte[MessageId.RAW_ID_SIZE]).build();
    MockProgramRunRecordFetcher runRecordFetcher = new MockProgramRunRecordFetcher().setRunRecord(runRecord);
    RuntimeRequestValidator validator = new DirectRuntimeRequestValidator(cConf, txRunner, runRecordFetcher, accessEnforcer, authenticationContext);
    // The first call should be hitting the run record fetching to fetch the run record.
    ProgramRunInfo programRunInfo = validator.getProgramRunStatus(programRunId, new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/"));
    Assert.assertEquals(programRunStatus, programRunInfo.getProgramRunStatus());
    // The second call will hit the runtime store, so it shouldn't matter what the run record fetch returns
    runRecordFetcher.setRunRecord(null);
    programRunInfo = validator.getProgramRunStatus(programRunId, new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, "/"));
    Assert.assertEquals(programRunStatus, programRunInfo.getProgramRunStatus());
}
Also used : ArtifactVersion(io.cdap.cdap.api.artifact.ArtifactVersion) ProgramRunStatus(io.cdap.cdap.proto.ProgramRunStatus) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) DefaultHttpRequest(io.netty.handler.codec.http.DefaultHttpRequest) RunRecordDetail(io.cdap.cdap.internal.app.store.RunRecordDetail) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) Test(org.junit.Test)

Example 17 with ArtifactId

use of io.cdap.cdap.api.artifact.ArtifactId in project cdap by caskdata.

the class MapReduceContextConfigTest method testManyMacrosInAppSpec.

@Test
public void testManyMacrosInAppSpec() {
    Configuration hConf = new Configuration();
    MapReduceContextConfig cfg = new MapReduceContextConfig(hConf);
    StringBuilder appCfg = new StringBuilder();
    for (int i = 0; i < 100; i++) {
        appCfg.append("${").append(i).append("}");
        hConf.setInt(String.valueOf(i), i);
    }
    ApplicationSpecification appSpec = new DefaultApplicationSpecification("name", ProjectInfo.getVersion().toString(), "desc", appCfg.toString(), new ArtifactId("artifact", new ArtifactVersion("1.0.0"), ArtifactScope.USER), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap());
    cfg.setApplicationSpecification(appSpec);
    Assert.assertEquals(appSpec.getConfiguration(), cfg.getApplicationSpecification().getConfiguration());
}
Also used : DefaultApplicationSpecification(io.cdap.cdap.internal.app.DefaultApplicationSpecification) ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) ArtifactVersion(io.cdap.cdap.api.artifact.ArtifactVersion) Configuration(org.apache.hadoop.conf.Configuration) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) DefaultApplicationSpecification(io.cdap.cdap.internal.app.DefaultApplicationSpecification) Test(org.junit.Test)

Example 18 with ArtifactId

use of io.cdap.cdap.api.artifact.ArtifactId in project cdap by caskdata.

the class TrackedPluginSelector method select.

@Nullable
@Override
public Map.Entry<ArtifactId, PluginClass> select(SortedMap<ArtifactId, PluginClass> plugins) {
    ArtifactId latestArtifact = plugins.tailMap(plugins.lastKey()).entrySet().iterator().next().getKey();
    suggestion = new ArtifactSelectorConfig(latestArtifact.getScope().name(), latestArtifact.getName(), latestArtifact.getVersion().getVersion());
    Map.Entry<ArtifactId, PluginClass> selected = delegate.select(plugins);
    selectedArtifact = selected == null ? null : selected.getKey();
    return selected;
}
Also used : ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) ArtifactSelectorConfig(io.cdap.cdap.etl.proto.ArtifactSelectorConfig) PluginClass(io.cdap.cdap.api.plugin.PluginClass) Map(java.util.Map) SortedMap(java.util.SortedMap) Nullable(javax.annotation.Nullable)

Example 19 with ArtifactId

use of io.cdap.cdap.api.artifact.ArtifactId in project cdap by caskdata.

the class CapabilityApplier method shouldDeployApp.

// Returns true if capability applier should try to deploy this application. 2 conditions when it returns true:
// 1. Either the application is not deployed before.
// 2. If application is deployed before then the app artifact of the deployed application is not the latest one
// available.
private boolean shouldDeployApp(ApplicationId applicationId, SystemApplication application) throws Exception {
    ApplicationDetail currAppDetail;
    try {
        currAppDetail = applicationLifecycleService.getAppDetail(applicationId);
    } catch (ApplicationNotFoundException exception) {
        return true;
    }
    // Compare if the app artifact version of currently deployed application with highest version of app artifact
    // available. If it's not same, capability applier should redeploy application.
    ArtifactSummary summary = application.getArtifact();
    NamespaceId artifactNamespace = ArtifactScope.SYSTEM.equals(summary.getScope()) ? NamespaceId.SYSTEM : applicationId.getParent();
    ArtifactRange range = new ArtifactRange(artifactNamespace.getNamespace(), summary.getName(), ArtifactVersionRange.parse(summary.getVersion()));
    // this method will not throw ArtifactNotFoundException, if no artifacts in the range, we are expecting an empty
    // collection returned.
    List<ArtifactDetail> artifactDetail = artifactRepository.getArtifactDetails(range, 1, ArtifactSortOrder.DESC);
    if (artifactDetail.isEmpty()) {
        throw new ArtifactNotFoundException(range.getNamespace(), range.getName());
    }
    ArtifactId latestArtifactId = artifactDetail.get(0).getDescriptor().getArtifactId();
    // same artifact. If same means no need to deploy the application again.
    return !currAppDetail.getArtifact().getVersion().equals(latestArtifactId.getVersion().getVersion());
}
Also used : ApplicationDetail(io.cdap.cdap.proto.ApplicationDetail) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) ApplicationNotFoundException(io.cdap.cdap.common.ApplicationNotFoundException) ArtifactRange(io.cdap.cdap.api.artifact.ArtifactRange) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) ArtifactDetail(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDetail)

Example 20 with ArtifactId

use of io.cdap.cdap.api.artifact.ArtifactId in project cdap by caskdata.

the class ProvisioningServiceTest method createTaskInfo.

private TaskFields createTaskInfo(ProvisionerInfo provisionerInfo) {
    ProgramRunId programRunId = NamespaceId.DEFAULT.app("app").workflow("wf").run(RunIds.generate());
    Map<String, String> systemArgs = new HashMap<>();
    Map<String, String> userArgs = new HashMap<>();
    Profile profile = new Profile(ProfileId.NATIVE.getProfile(), "label", "desc", provisionerInfo);
    SystemArguments.addProfileArgs(systemArgs, profile);
    systemArgs.put(Constants.APP_CDAP_VERSION, APP_CDAP_VERSION);
    ProgramOptions programOptions = new SimpleProgramOptions(programRunId.getParent(), new BasicArguments(systemArgs), new BasicArguments(userArgs));
    ArtifactId artifactId = NamespaceId.DEFAULT.artifact("testArtifact", "1.0").toApiArtifactId();
    ApplicationSpecification appSpec = new DefaultApplicationSpecification("name", "1.0.0", APP_CDAP_VERSION, "desc", null, artifactId, Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap(), Collections.emptyMap());
    ProgramDescriptor programDescriptor = new ProgramDescriptor(programRunId.getParent(), appSpec);
    return new TaskFields(programDescriptor, programOptions, programRunId);
}
Also used : ApplicationSpecification(io.cdap.cdap.api.app.ApplicationSpecification) DefaultApplicationSpecification(io.cdap.cdap.internal.app.DefaultApplicationSpecification) ArtifactId(io.cdap.cdap.api.artifact.ArtifactId) HashMap(java.util.HashMap) Profile(io.cdap.cdap.proto.profile.Profile) SimpleProgramOptions(io.cdap.cdap.internal.app.runtime.SimpleProgramOptions) ProgramOptions(io.cdap.cdap.app.runtime.ProgramOptions) ProgramRunId(io.cdap.cdap.proto.id.ProgramRunId) SimpleProgramOptions(io.cdap.cdap.internal.app.runtime.SimpleProgramOptions) BasicArguments(io.cdap.cdap.internal.app.runtime.BasicArguments) DefaultApplicationSpecification(io.cdap.cdap.internal.app.DefaultApplicationSpecification) ProgramDescriptor(io.cdap.cdap.app.program.ProgramDescriptor)

Aggregations

ArtifactId (io.cdap.cdap.api.artifact.ArtifactId)150 Test (org.junit.Test)86 ProgramId (io.cdap.cdap.proto.id.ProgramId)64 ArtifactVersion (io.cdap.cdap.api.artifact.ArtifactVersion)50 ProgramRunId (io.cdap.cdap.proto.id.ProgramRunId)50 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)38 RunId (org.apache.twill.api.RunId)36 ApplicationSpecification (io.cdap.cdap.api.app.ApplicationSpecification)28 File (java.io.File)28 HashMap (java.util.HashMap)28 ApplicationId (io.cdap.cdap.proto.id.ApplicationId)26 Id (io.cdap.cdap.common.id.Id)20 HttpResponse (io.cdap.common.http.HttpResponse)18 Map (java.util.Map)18 Gson (com.google.gson.Gson)16 ProgramDescriptor (io.cdap.cdap.app.program.ProgramDescriptor)16 Location (org.apache.twill.filesystem.Location)16 ApplicationClass (io.cdap.cdap.api.artifact.ApplicationClass)14 PluginClass (io.cdap.cdap.api.plugin.PluginClass)14 AppDeploymentInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo)14