Search in sources :

Example 6 with ArtifactNotFoundException

use of io.cdap.cdap.common.ArtifactNotFoundException in project cdap by caskdata.

the class ApplicationLifecycleService method deployApp.

/**
 * Deploy an application using the specified artifact and configuration. When an app is deployed, the Application
 * class is instantiated and configure() is called in order to generate an {@link ApplicationSpecification}.
 * Programs, datasets, and streams are created based on the specification before the spec is persisted in the
 * {@link Store}. This method can create a new application as well as update an existing one.
 *
 * @param namespace the namespace to deploy the app to
 * @param appName the name of the app. If null, the name will be set based on the application spec
 * @param summary the artifact summary of the app
 * @param configStr the configuration to send to the application when generating the application specification
 * @param programTerminator a program terminator that will stop programs that are removed when updating an app.
 *                          For example, if an update removes a flow, the terminator defines how to stop that flow.
 * @param ownerPrincipal the kerberos principal of the application owner
 * @param updateSchedules specifies if schedules of the workflow have to be updated,
 *                        if null value specified by the property "app.deploy.update.schedules" will be used.
 * @param isPreview whether the app deployment is for preview
 * @param userProps the user properties for the app deployment, this is basically used for preview deployment
 * @return information about the deployed application
 * @throws InvalidArtifactException if the artifact does not contain any application classes
 * @throws IOException if there was an IO error reading artifact detail from the meta store
 * @throws ArtifactNotFoundException if the specified artifact does not exist
 * @throws Exception if there was an exception during the deployment pipeline. This exception will often wrap
 *                   the actual exception
 */
public ApplicationWithPrograms deployApp(NamespaceId namespace, @Nullable String appName, @Nullable String appVersion, ArtifactSummary summary, @Nullable String configStr, ProgramTerminator programTerminator, @Nullable KerberosPrincipalId ownerPrincipal, @Nullable Boolean updateSchedules, boolean isPreview, Map<String, String> userProps) throws Exception {
    NamespaceId artifactNamespace = ArtifactScope.SYSTEM.equals(summary.getScope()) ? NamespaceId.SYSTEM : namespace;
    ArtifactRange range = new ArtifactRange(artifactNamespace.getNamespace(), summary.getName(), ArtifactVersionRange.parse(summary.getVersion()));
    // this method will not throw ArtifactNotFoundException, if no artifacts in the range, we are expecting an empty
    // collection returned.
    List<ArtifactDetail> artifactDetail = artifactRepository.getArtifactDetails(range, 1, ArtifactSortOrder.DESC);
    if (artifactDetail.isEmpty()) {
        throw new ArtifactNotFoundException(range.getNamespace(), range.getName());
    }
    return deployApp(namespace, appName, appVersion, configStr, programTerminator, artifactDetail.iterator().next(), ownerPrincipal, updateSchedules == null ? appUpdateSchedules : updateSchedules, isPreview, userProps);
}
Also used : ArtifactRange(io.cdap.cdap.api.artifact.ArtifactRange) NamespaceId(io.cdap.cdap.proto.id.NamespaceId) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) ArtifactDetail(io.cdap.cdap.internal.app.runtime.artifact.ArtifactDetail)

Example 7 with ArtifactNotFoundException

use of io.cdap.cdap.common.ArtifactNotFoundException in project cdap by caskdata.

the class RemoteArtifactManager method getArtifactLocation.

@Override
protected Location getArtifactLocation(ArtifactInfo artifactInfo, @Nullable String artifactNamespace) throws IOException, UnauthorizedException {
    String namespace;
    if (ArtifactScope.SYSTEM.equals(artifactInfo.getScope())) {
        namespace = NamespaceId.SYSTEM.getNamespace();
    } else if (artifactNamespace != null) {
        namespace = artifactNamespace;
    } else {
        namespace = namespaceId.getNamespace();
    }
    // If ArtifactLocalizerClient is set, use it to get the cached location of artifact.
    if (artifactLocalizerClient != null) {
        ArtifactId artifactId = new ArtifactId(namespace, artifactInfo.getName(), artifactInfo.getVersion());
        try {
            // artifact ClassLoader only.
            return Locations.toLocation(artifactLocalizerClient.getUnpackedArtifactLocation(artifactId));
        } catch (ArtifactNotFoundException e) {
            throw new IOException(String.format("Artifact %s is not found", artifactId), e);
        }
    }
    HttpRequest.Builder requestBuilder = remoteClientInternal.requestBuilder(HttpMethod.GET, String.format("namespaces/%s/artifacts/%s/versions/%s/location", namespace, artifactInfo.getName(), artifactInfo.getVersion()));
    HttpResponse httpResponse = remoteClientInternal.execute(requestBuilder.build());
    if (httpResponse.getResponseCode() == HttpResponseStatus.NOT_FOUND.code()) {
        throw new IOException("Could not get artifact detail, endpoint not found");
    }
    if (httpResponse.getResponseCode() != 200) {
        throw new IOException(String.format("Exception while getting artifacts list %s", httpResponse.getResponseBodyAsString()));
    }
    String path = httpResponse.getResponseBodyAsString();
    Location location = Locations.getLocationFromAbsolutePath(locationFactory, path);
    if (!location.exists()) {
        throw new IOException(String.format("Artifact location does not exist %s for artifact %s version %s", path, artifactInfo.getName(), artifactInfo.getVersion()));
    }
    return location;
}
Also used : HttpRequest(io.cdap.common.http.HttpRequest) ArtifactId(io.cdap.cdap.proto.id.ArtifactId) HttpResponse(io.cdap.common.http.HttpResponse) IOException(java.io.IOException) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) Location(org.apache.twill.filesystem.Location)

Example 8 with ArtifactNotFoundException

use of io.cdap.cdap.common.ArtifactNotFoundException in project cdap by caskdata.

the class ArtifactClientTestRun method testArtifacts.

@Test
public void testArtifacts() throws Exception {
    // add 2 versions of an artifact with an application
    ArtifactId myapp1Id = NamespaceId.DEFAULT.artifact("myapp", "1.0.0");
    ArtifactId myapp2Id = NamespaceId.DEFAULT.artifact("myapp", "2.0.0");
    LocalLocationFactory locationFactory = new LocalLocationFactory(TMP_FOLDER.newFolder());
    Manifest manifest = new Manifest();
    manifest.getMainAttributes().put(ManifestFields.BUNDLE_VERSION, "2.0.0");
    final Location appJarLoc = AppJarHelper.createDeploymentJar(locationFactory, MyApp.class, manifest);
    ContentProvider<InputStream> contentProvider = appJarLoc::getInputStream;
    artifactClient.add(myapp1Id.getParent(), myapp1Id.getArtifact(), contentProvider, myapp1Id.getVersion());
    // add some properties
    Map<String, String> myapp1Properties = ImmutableMap.of("k1", "v1");
    artifactClient.writeProperties(myapp1Id, myapp1Properties);
    // let it derive version from jar manifest, which has bundle-version at 2.0.0
    artifactClient.add(myapp2Id.getParent(), myapp2Id.getArtifact(), contentProvider, null, null);
    // add some properties
    Map<String, String> myapp2Properties = ImmutableMap.of("k1", "v1", "k2", "v2");
    artifactClient.writeProperties(myapp2Id, myapp2Properties);
    // add an artifact that contains a plugin, but only extends myapp-2.0.0
    ArtifactId pluginId = NamespaceId.DEFAULT.artifact("myapp-plugins", "2.0.0");
    manifest = new Manifest();
    manifest.getMainAttributes().put(ManifestFields.EXPORT_PACKAGE, Plugin1.class.getPackage().getName());
    final Location pluginJarLoc = PluginJarHelper.createPluginJar(locationFactory, manifest, Plugin1.class);
    contentProvider = pluginJarLoc::getInputStream;
    Set<ArtifactRange> parents = Sets.newHashSet(new ArtifactRange(myapp2Id.getParent().getNamespace(), myapp2Id.getArtifact(), new ArtifactVersion(myapp2Id.getVersion()), new ArtifactVersion("3.0.0")));
    Set<PluginClass> additionalPlugins = Sets.newHashSet(PluginClass.builder().setName("mysql").setType("jdbc").setDescription("").setClassName(Plugin1.class.getName()).build());
    artifactClient.add(pluginId.getParent(), pluginId.getArtifact(), contentProvider, pluginId.getVersion(), parents, additionalPlugins);
    ArtifactSummary myapp1Summary = new ArtifactSummary(myapp1Id.getArtifact(), myapp1Id.getVersion());
    ArtifactSummary myapp2Summary = new ArtifactSummary(myapp2Id.getArtifact(), myapp2Id.getVersion());
    ArtifactSummary pluginArtifactSummary = new ArtifactSummary(pluginId.getArtifact(), pluginId.getVersion());
    Set<ArtifactSummary> artifacts = Sets.newHashSet(artifactClient.list(NamespaceId.DEFAULT));
    Assert.assertEquals(Sets.newHashSet(myapp1Summary, myapp2Summary, pluginArtifactSummary), artifacts);
    // list all artifacts named 'myapp'
    Assert.assertEquals(Sets.newHashSet(myapp1Summary, myapp2Summary), Sets.newHashSet(artifactClient.listVersions(NamespaceId.DEFAULT, myapp1Id.getArtifact())));
    // list all artifacts named 'myapp-plugins'
    Assert.assertEquals(Sets.newHashSet(pluginArtifactSummary), Sets.newHashSet(artifactClient.listVersions(NamespaceId.DEFAULT, pluginId.getArtifact())));
    // artifacts should be in user scope
    try {
        artifactClient.listVersions(NamespaceId.DEFAULT, pluginId.getArtifact(), ArtifactScope.SYSTEM);
        Assert.fail();
    } catch (ArtifactNotFoundException e) {
    // expected
    }
    // get info about specific artifacts
    Schema myAppConfigSchema = new ReflectionSchemaGenerator(false).generate(MyApp.Conf.class);
    ArtifactClasses myAppClasses = ArtifactClasses.builder().addApp(new ApplicationClass(MyApp.class.getName(), "", myAppConfigSchema)).build();
    // test get myapp-1.0.0
    ArtifactInfo myapp1Info = new ArtifactInfo(myapp1Id.getArtifact(), myapp1Id.getVersion(), ArtifactScope.USER, myAppClasses, myapp1Properties);
    Assert.assertEquals(myapp1Info, artifactClient.getArtifactInfo(myapp1Id));
    // test get myapp-2.0.0
    ArtifactInfo myapp2Info = new ArtifactInfo(myapp2Id.getArtifact(), myapp2Id.getVersion(), ArtifactScope.USER, myAppClasses, myapp2Properties);
    Assert.assertEquals(myapp2Info, artifactClient.getArtifactInfo(myapp2Id));
    // test overwriting properties
    myapp2Properties = ImmutableMap.of("k1", "v3", "k5", "v5");
    artifactClient.writeProperties(myapp2Id, myapp2Properties);
    Assert.assertEquals(myapp2Properties, artifactClient.getArtifactInfo(myapp2Id).getProperties());
    // test deleting property
    artifactClient.deleteProperty(myapp2Id, "k1");
    Assert.assertEquals(ImmutableMap.of("k5", "v5"), artifactClient.getArtifactInfo(myapp2Id).getProperties());
    // test writing property
    artifactClient.writeProperty(myapp2Id, "k5", "v4");
    Assert.assertEquals(ImmutableMap.of("k5", "v4"), artifactClient.getArtifactInfo(myapp2Id).getProperties());
    // test deleting properties
    artifactClient.deleteProperties(myapp2Id);
    Assert.assertTrue(artifactClient.getArtifactInfo(myapp2Id).getProperties().isEmpty());
    // test get myapp-plugins-2.0.0
    Map<String, PluginPropertyField> props = ImmutableMap.of("x", new PluginPropertyField("x", "", "int", true, false));
    ArtifactClasses pluginClasses = ArtifactClasses.builder().addPlugin(PluginClass.builder().setName("plugin1").setType("callable").setDescription("p1 description").setClassName(Plugin1.class.getName()).setConfigFieldName("conf").setProperties(props).build()).addPlugins(additionalPlugins).build();
    ArtifactInfo pluginArtifactInfo = new ArtifactInfo(pluginId.getArtifact(), pluginId.getVersion(), ArtifactScope.USER, pluginClasses, ImmutableMap.<String, String>of());
    Assert.assertEquals(pluginArtifactInfo, artifactClient.getArtifactInfo(pluginId));
    // test get all app classes in namespace
    Set<ApplicationClassSummary> expectedSummaries = ImmutableSet.of(new ApplicationClassSummary(myapp1Summary, MyApp.class.getName()), new ApplicationClassSummary(myapp2Summary, MyApp.class.getName()));
    Set<ApplicationClassSummary> appClassSummaries = Sets.newHashSet(artifactClient.getApplicationClasses(NamespaceId.DEFAULT));
    Assert.assertEquals(expectedSummaries, appClassSummaries);
    // test get all app classes in namespace with name MyApp.class.getName()
    Set<ApplicationClassInfo> appClassInfos = Sets.newHashSet(artifactClient.getApplicationClasses(NamespaceId.DEFAULT, MyApp.class.getName()));
    Set<ApplicationClassInfo> expectedInfos = ImmutableSet.of(new ApplicationClassInfo(myapp1Summary, MyApp.class.getName(), myAppConfigSchema), new ApplicationClassInfo(myapp2Summary, MyApp.class.getName(), myAppConfigSchema));
    Assert.assertEquals(expectedInfos, appClassInfos);
    // test get plugin types for myapp-1.0.0. should be empty, since plugins only extends versions [2.0.0 - 3.0.0)
    Assert.assertTrue(artifactClient.getPluginTypes(myapp1Id).isEmpty());
    // test get plugin types for myapp-2.0.0
    Assert.assertEquals(Lists.newArrayList("callable", "jdbc"), artifactClient.getPluginTypes(myapp2Id));
    // test get plugins of type callable for myapp-2.0.0
    PluginSummary pluginSummary = new PluginSummary("plugin1", "callable", null, Plugin1.class.getName(), pluginArtifactSummary, "p1 description");
    Assert.assertEquals(Sets.newHashSet(pluginSummary), Sets.newHashSet(artifactClient.getPluginSummaries(myapp2Id, "callable")));
    // no plugins of type "runnable"
    Assert.assertTrue(artifactClient.getPluginSummaries(myapp2Id, "runnable").isEmpty());
    // test get plugin details for plugin1 for myapp-2.0.0
    PluginInfo pluginInfo = new PluginInfo("plugin1", "callable", null, Plugin1.class.getName(), "conf", pluginArtifactSummary, props, "p1 description");
    Assert.assertEquals(Sets.newHashSet(pluginInfo), Sets.newHashSet(artifactClient.getPluginInfo(myapp2Id, "callable", "plugin1")));
}
Also used : MyApp(io.cdap.cdap.client.artifact.MyApp) ArtifactId(io.cdap.cdap.proto.id.ArtifactId) Schema(io.cdap.cdap.api.data.schema.Schema) ArtifactRange(io.cdap.cdap.api.artifact.ArtifactRange) PluginPropertyField(io.cdap.cdap.api.plugin.PluginPropertyField) ArtifactVersion(io.cdap.cdap.api.artifact.ArtifactVersion) ArtifactClasses(io.cdap.cdap.api.artifact.ArtifactClasses) ApplicationClassInfo(io.cdap.cdap.proto.artifact.ApplicationClassInfo) PluginInfo(io.cdap.cdap.proto.artifact.PluginInfo) PluginSummary(io.cdap.cdap.proto.artifact.PluginSummary) LocalLocationFactory(org.apache.twill.filesystem.LocalLocationFactory) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) Plugin1(io.cdap.cdap.client.artifact.plugin.Plugin1) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) ApplicationClassSummary(io.cdap.cdap.proto.artifact.ApplicationClassSummary) Manifest(java.util.jar.Manifest) ReflectionSchemaGenerator(io.cdap.cdap.internal.io.ReflectionSchemaGenerator) ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ArtifactInfo(io.cdap.cdap.api.artifact.ArtifactInfo) PluginClass(io.cdap.cdap.api.plugin.PluginClass) Location(org.apache.twill.filesystem.Location) Test(org.junit.Test)

Example 9 with ArtifactNotFoundException

use of io.cdap.cdap.common.ArtifactNotFoundException in project cdap by caskdata.

the class ArtifactClient method writeProperty.

/**
 * Write a property for an artifact. If the property already exists, it will be overwritten. If the property
 * does not exist, it will be added.
 *
 * @param artifactId the artifact to write the property to
 * @param key the property key to write
 * @param value the property value to write
 * @throws BadRequestException if the request is invalid. For example, if the artifact name or version is invalid
 * @throws UnauthenticatedException if the request is not authorized successfully in the gateway server
 * @throws ArtifactNotFoundException if the artifact does not exist
 * @throws IOException if a network error occurred
 */
public void writeProperty(ArtifactId artifactId, String key, String value) throws IOException, UnauthenticatedException, ArtifactNotFoundException, BadRequestException, UnauthorizedException {
    String path = String.format("artifacts/%s/versions/%s/properties/%s", artifactId.getArtifact(), artifactId.getVersion(), key);
    URL url = config.resolveNamespacedURLV3(artifactId.getParent(), path);
    HttpRequest.Builder requestBuilder = HttpRequest.put(url);
    HttpRequest request = requestBuilder.withBody(value).build();
    HttpResponse response = restClient.execute(request, config.getAccessToken(), HttpURLConnection.HTTP_BAD_REQUEST, HttpURLConnection.HTTP_NOT_FOUND);
    int responseCode = response.getResponseCode();
    if (responseCode == HttpURLConnection.HTTP_NOT_FOUND) {
        throw new ArtifactNotFoundException(artifactId);
    } else if (responseCode == HttpURLConnection.HTTP_BAD_REQUEST) {
        throw new BadRequestException(response.getResponseBodyAsString());
    }
}
Also used : HttpRequest(io.cdap.common.http.HttpRequest) HttpResponse(io.cdap.common.http.HttpResponse) BadRequestException(io.cdap.cdap.common.BadRequestException) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) URL(java.net.URL)

Example 10 with ArtifactNotFoundException

use of io.cdap.cdap.common.ArtifactNotFoundException in project cdap by caskdata.

the class ArtifactClient method deleteProperty.

/**
 * Delete a property for an artifact. If the property does not exist, this will be a no-op.
 *
 * @param artifactId the artifact to delete a property from
 * @param key the property to delete
 * @throws BadRequestException if the request is invalid. For example, if the artifact name or version is invalid
 * @throws UnauthenticatedException if the request is not authorized successfully in the gateway server
 * @throws ArtifactNotFoundException if the artifact does not exist
 * @throws IOException if a network error occurred
 */
public void deleteProperty(ArtifactId artifactId, String key) throws IOException, UnauthenticatedException, ArtifactNotFoundException, BadRequestException, UnauthorizedException {
    String path = String.format("artifacts/%s/versions/%s/properties/%s", artifactId.getArtifact(), artifactId.getVersion(), key);
    URL url = config.resolveNamespacedURLV3(artifactId.getParent(), path);
    HttpRequest.Builder requestBuilder = HttpRequest.delete(url);
    HttpRequest request = requestBuilder.build();
    HttpResponse response = restClient.execute(request, config.getAccessToken(), HttpURLConnection.HTTP_BAD_REQUEST, HttpURLConnection.HTTP_NOT_FOUND);
    int responseCode = response.getResponseCode();
    if (responseCode == HttpURLConnection.HTTP_NOT_FOUND) {
        throw new ArtifactNotFoundException(artifactId);
    } else if (responseCode == HttpURLConnection.HTTP_BAD_REQUEST) {
        throw new BadRequestException(response.getResponseBodyAsString());
    }
}
Also used : HttpRequest(io.cdap.common.http.HttpRequest) HttpResponse(io.cdap.common.http.HttpResponse) BadRequestException(io.cdap.cdap.common.BadRequestException) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException) URL(java.net.URL)

Aggregations

ArtifactNotFoundException (io.cdap.cdap.common.ArtifactNotFoundException)62 HttpResponse (io.cdap.common.http.HttpResponse)24 URL (java.net.URL)20 IOException (java.io.IOException)18 ArtifactRange (io.cdap.cdap.api.artifact.ArtifactRange)16 ArtifactId (io.cdap.cdap.proto.id.ArtifactId)16 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)16 ArtifactVersion (io.cdap.cdap.api.artifact.ArtifactVersion)14 HttpRequest (io.cdap.common.http.HttpRequest)14 ArtifactSummary (io.cdap.cdap.api.artifact.ArtifactSummary)12 PluginClass (io.cdap.cdap.api.plugin.PluginClass)12 StructuredRow (io.cdap.cdap.spi.data.StructuredRow)12 Location (org.apache.twill.filesystem.Location)12 BadRequestException (io.cdap.cdap.common.BadRequestException)10 File (java.io.File)10 Test (org.junit.Test)10 StructuredTable (io.cdap.cdap.spi.data.StructuredTable)9 ApplicationClass (io.cdap.cdap.api.artifact.ApplicationClass)6 ApplicationNotFoundException (io.cdap.cdap.common.ApplicationNotFoundException)6 Id (io.cdap.cdap.common.id.Id)6