Search in sources :

Example 1 with ArtifactClasses

use of co.cask.cdap.api.artifact.ArtifactClasses in project cdap by caskdata.

the class ArtifactRepository method addArtifact.

/**
   * Inspects and builds plugin and application information for the given artifact, adding an additional set of
   * plugin classes to the plugins found through inspection. This method is used when all plugin classes
   * cannot be derived by inspecting the artifact but need to be explicitly set. This is true for 3rd party plugins
   * like jdbc drivers.
   *
   * @param artifactId the id of the artifact to inspect and store
   * @param artifactFile the artifact to inspect and store
   * @param parentArtifacts artifacts the given artifact extends.
   *                        If null, the given artifact does not extend another artifact
   * @param additionalPlugins the set of additional plugin classes to add to the plugins found through inspection.
   *                          If null, no additional plugin classes will be added
   * @param properties properties for the artifact
   * @throws IOException if there was an exception reading from the artifact store
   * @throws ArtifactRangeNotFoundException if none of the parent artifacts could be found
   * @throws UnauthorizedException if the user is not authorized to add an artifact in the specified namespace. To add
   *                               an artifact, a user must have {@link Action#WRITE} on the namespace in which
   *                               the artifact is being added. If authorization is successful, and
   *                               the artifact is added successfully, then the user gets all {@link Action privileges}
   *                               on the added artifact.
   */
@VisibleForTesting
public ArtifactDetail addArtifact(final Id.Artifact artifactId, final File artifactFile, @Nullable Set<ArtifactRange> parentArtifacts, @Nullable Set<PluginClass> additionalPlugins, Map<String, String> properties) throws Exception {
    if (additionalPlugins != null) {
        validatePluginSet(additionalPlugins);
    }
    parentArtifacts = parentArtifacts == null ? Collections.<ArtifactRange>emptySet() : parentArtifacts;
    CloseableClassLoader parentClassLoader = null;
    EntityImpersonator entityImpersonator = new EntityImpersonator(artifactId.toEntityId(), impersonator);
    if (!parentArtifacts.isEmpty()) {
        validateParentSet(artifactId, parentArtifacts);
        parentClassLoader = createParentClassLoader(artifactId, parentArtifacts, entityImpersonator);
    }
    try {
        ArtifactClasses artifactClasses = inspectArtifact(artifactId, artifactFile, additionalPlugins, parentClassLoader);
        ArtifactMeta meta = new ArtifactMeta(artifactClasses, parentArtifacts, properties);
        ArtifactDetail artifactDetail = artifactStore.write(artifactId, meta, Files.newInputStreamSupplier(artifactFile), entityImpersonator);
        ArtifactDescriptor descriptor = artifactDetail.getDescriptor();
        // info hides some fields that are available in detail, such as the location of the artifact
        ArtifactInfo artifactInfo = new ArtifactInfo(descriptor.getArtifactId(), artifactDetail.getMeta().getClasses(), artifactDetail.getMeta().getProperties());
        // add system metadata for artifacts
        writeSystemMetadata(artifactId.toEntityId(), artifactInfo);
        return artifactDetail;
    } finally {
        Closeables.closeQuietly(parentClassLoader);
    }
}
Also used : ArtifactInfo(co.cask.cdap.api.artifact.ArtifactInfo) ArtifactClasses(co.cask.cdap.api.artifact.ArtifactClasses) EntityImpersonator(co.cask.cdap.security.impersonation.EntityImpersonator) ArtifactRange(co.cask.cdap.api.artifact.ArtifactRange) CloseableClassLoader(co.cask.cdap.api.artifact.CloseableClassLoader) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 2 with ArtifactClasses

use of co.cask.cdap.api.artifact.ArtifactClasses in project cdap by caskdata.

the class ArtifactSystemMetadataWriter method getSystemPropertiesToAdd.

@Override
protected Map<String, String> getSystemPropertiesToAdd() {
    ImmutableMap.Builder<String, String> properties = ImmutableMap.builder();
    properties.put(ENTITY_NAME_KEY, artifactInfo.getName());
    ArtifactClasses classes = artifactInfo.getClasses();
    for (PluginClass pluginClass : classes.getPlugins()) {
        addPlugin(pluginClass, artifactInfo.getVersion(), properties);
    }
    if (!existing) {
        properties.put(CREATION_TIME_KEY, String.valueOf(System.currentTimeMillis()));
    }
    return properties.build();
}
Also used : ArtifactClasses(co.cask.cdap.api.artifact.ArtifactClasses) PluginClass(co.cask.cdap.api.plugin.PluginClass) ImmutableMap(com.google.common.collect.ImmutableMap)

Example 3 with ArtifactClasses

use of co.cask.cdap.api.artifact.ArtifactClasses in project cdap by caskdata.

the class ArtifactStore method writeMeta.

// write a new artifact snapshot and clean up the old snapshot data
private void writeMeta(Table table, Id.Artifact artifactId, ArtifactData data) throws IOException {
    ArtifactCell artifactCell = new ArtifactCell(artifactId);
    table.put(artifactCell.rowkey, artifactCell.column, Bytes.toBytes(GSON.toJson(data)));
    // column for plugin meta and app meta. {artifact-name}:{artifact-version}
    // does not need to contain namespace because namespace is in the rowkey
    byte[] artifactColumn = new ArtifactColumn(artifactId).getColumn();
    ArtifactClasses classes = data.meta.getClasses();
    Location artifactLocation = Locations.getLocationFromAbsolutePath(locationFactory, data.getLocationPath());
    // write pluginClass metadata
    for (PluginClass pluginClass : classes.getPlugins()) {
        // write metadata for each artifact this plugin extends
        for (ArtifactRange artifactRange : data.meta.getUsableBy()) {
            // p:{namespace}:{type}:{name}
            PluginKey pluginKey = new PluginKey(artifactRange.getNamespace(), artifactRange.getName(), pluginClass.getType(), pluginClass.getName());
            byte[] pluginDataBytes = Bytes.toBytes(GSON.toJson(new PluginData(pluginClass, artifactLocation, artifactRange)));
            table.put(pluginKey.getRowKey(), artifactColumn, pluginDataBytes);
        }
        // by any other artifact in the same namespace.
        if (data.meta.getUsableBy().isEmpty()) {
            // Write a special entry for plugin that doesn't have parent, which means any artifact can use it
            UniversalPluginKey pluginKey = new UniversalPluginKey(artifactId.getNamespace().getId(), pluginClass.getType(), pluginClass.getName());
            byte[] pluginDataBytes = Bytes.toBytes(GSON.toJson(new PluginData(pluginClass, artifactLocation, null)));
            table.put(pluginKey.getRowKey(), artifactColumn, pluginDataBytes);
        }
    }
    // write appClass metadata
    for (ApplicationClass appClass : classes.getApps()) {
        // a:{namespace}:{classname}
        AppClassKey appClassKey = new AppClassKey(artifactId.getNamespace().toEntityId(), appClass.getClassName());
        byte[] appDataBytes = Bytes.toBytes(GSON.toJson(new AppData(appClass, artifactLocation)));
        table.put(appClassKey.getRowKey(), artifactColumn, appDataBytes);
    }
}
Also used : ArtifactRange(co.cask.cdap.api.artifact.ArtifactRange) ApplicationClass(co.cask.cdap.api.artifact.ApplicationClass) ArtifactClasses(co.cask.cdap.api.artifact.ArtifactClasses) PluginClass(co.cask.cdap.api.plugin.PluginClass) Location(org.apache.twill.filesystem.Location)

Example 4 with ArtifactClasses

use of co.cask.cdap.api.artifact.ArtifactClasses in project cdap by caskdata.

the class DefaultArtifactRepository method inspectArtifact.

private ArtifactClasses inspectArtifact(Id.Artifact artifactId, File artifactFile, @Nullable Set<PluginClass> additionalPlugins, @Nullable ClassLoader parentClassLoader) throws IOException, InvalidArtifactException {
    ArtifactClasses artifactClasses = artifactInspector.inspectArtifact(artifactId, artifactFile, parentClassLoader);
    validatePluginSet(artifactClasses.getPlugins());
    if (additionalPlugins == null || additionalPlugins.isEmpty()) {
        return artifactClasses;
    } else {
        return ArtifactClasses.builder().addApps(artifactClasses.getApps()).addPlugins(artifactClasses.getPlugins()).addPlugins(additionalPlugins).build();
    }
}
Also used : ArtifactClasses(co.cask.cdap.api.artifact.ArtifactClasses)

Example 5 with ArtifactClasses

use of co.cask.cdap.api.artifact.ArtifactClasses in project cdap by caskdata.

the class ArtifactInspectorTest method inspectAppsAndPlugins.

@Test
public void inspectAppsAndPlugins() throws Exception {
    Manifest manifest = new Manifest();
    manifest.getMainAttributes().put(ManifestFields.EXPORT_PACKAGE, InspectionApp.class.getPackage().getName());
    File appFile = createJar(InspectionApp.class, new File(TMP_FOLDER.newFolder(), "InspectionApp-1.0.0.jar"), manifest);
    Id.Artifact artifactId = Id.Artifact.from(Id.Namespace.DEFAULT, "InspectionApp", "1.0.0");
    Location artifactLocation = Locations.toLocation(appFile);
    try (CloseableClassLoader artifactClassLoader = classLoaderFactory.createClassLoader(ImmutableList.of(artifactLocation).iterator(), new EntityImpersonator(artifactId.toEntityId(), new DefaultImpersonator(CConfiguration.create(), null)))) {
        ArtifactClasses classes = artifactInspector.inspectArtifact(artifactId, appFile, artifactClassLoader);
        // check app classes
        Set<ApplicationClass> expectedApps = ImmutableSet.of(new ApplicationClass(InspectionApp.class.getName(), "", new ReflectionSchemaGenerator(false).generate(InspectionApp.AConfig.class)));
        Assert.assertEquals(expectedApps, classes.getApps());
        // check plugin classes
        PluginClass expectedPlugin = new PluginClass(InspectionApp.PLUGIN_TYPE, InspectionApp.PLUGIN_NAME, InspectionApp.PLUGIN_DESCRIPTION, InspectionApp.AppPlugin.class.getName(), "pluginConf", ImmutableMap.of("y", new PluginPropertyField("y", "", "double", true, true), "isSomething", new PluginPropertyField("isSomething", "", "boolean", true, false)));
        Assert.assertEquals(ImmutableSet.of(expectedPlugin), classes.getPlugins());
    }
}
Also used : EntityImpersonator(co.cask.cdap.security.impersonation.EntityImpersonator) ApplicationClass(co.cask.cdap.api.artifact.ApplicationClass) CloseableClassLoader(co.cask.cdap.api.artifact.CloseableClassLoader) Manifest(java.util.jar.Manifest) ReflectionSchemaGenerator(co.cask.cdap.internal.io.ReflectionSchemaGenerator) DefaultImpersonator(co.cask.cdap.security.impersonation.DefaultImpersonator) PluginPropertyField(co.cask.cdap.api.plugin.PluginPropertyField) ArtifactClasses(co.cask.cdap.api.artifact.ArtifactClasses) Id(co.cask.cdap.common.id.Id) PluginClass(co.cask.cdap.api.plugin.PluginClass) File(java.io.File) Location(org.apache.twill.filesystem.Location) InspectionApp(co.cask.cdap.internal.app.runtime.artifact.app.inspection.InspectionApp) Test(org.junit.Test)

Aggregations

ArtifactClasses (co.cask.cdap.api.artifact.ArtifactClasses)9 CloseableClassLoader (co.cask.cdap.api.artifact.CloseableClassLoader)4 PluginClass (co.cask.cdap.api.plugin.PluginClass)4 Location (org.apache.twill.filesystem.Location)4 ApplicationClass (co.cask.cdap.api.artifact.ApplicationClass)3 ArtifactInfo (co.cask.cdap.api.artifact.ArtifactInfo)3 ArtifactRange (co.cask.cdap.api.artifact.ArtifactRange)3 EntityImpersonator (co.cask.cdap.security.impersonation.EntityImpersonator)3 PluginPropertyField (co.cask.cdap.api.plugin.PluginPropertyField)2 ReflectionSchemaGenerator (co.cask.cdap.internal.io.ReflectionSchemaGenerator)2 File (java.io.File)2 IOException (java.io.IOException)2 Manifest (java.util.jar.Manifest)2 Test (org.junit.Test)2 ArtifactSummary (co.cask.cdap.api.artifact.ArtifactSummary)1 ArtifactVersion (co.cask.cdap.api.artifact.ArtifactVersion)1 Schema (co.cask.cdap.api.data.schema.Schema)1 MyApp (co.cask.cdap.client.artifact.MyApp)1 Plugin1 (co.cask.cdap.client.artifact.plugin.Plugin1)1 ArtifactNotFoundException (co.cask.cdap.common.ArtifactNotFoundException)1