Search in sources :

Example 1 with ApplicationClass

use of io.cdap.cdap.api.artifact.ApplicationClass in project cdap by caskdata.

the class ArtifactStore method getApplicationClasses.

/**
 * Get all application classes that belong to the specified namespace.
 * Results are returned as a sorted map from artifact to application classes in that artifact.
 * Map entries are sorted by the artifact.
 *
 * @param namespace the namespace from which to get application classes
 * @return an unmodifiable map of artifact to a list of all application classes in that artifact.
 *         The map will never be null. If there are no application classes, an empty map will be returned.
 */
public SortedMap<ArtifactDescriptor, List<ApplicationClass>> getApplicationClasses(NamespaceId namespace) {
    return TransactionRunners.run(transactionRunner, context -> {
        SortedMap<ArtifactDescriptor, List<ApplicationClass>> result = Maps.newTreeMap();
        StructuredTable table = getTable(context, StoreDefinition.ArtifactStore.APP_DATA_TABLE);
        Collection<Field<?>> keys = Collections.singleton(Fields.stringField(StoreDefinition.ArtifactStore.NAMESPACE_FIELD, namespace.getNamespace()));
        try (CloseableIterator<StructuredRow> iterator = table.scan(Range.singleton(keys), Integer.MAX_VALUE)) {
            while (iterator.hasNext()) {
                StructuredRow row = iterator.next();
                Map.Entry<ArtifactDescriptor, ApplicationClass> entry = extractApplicationClass(row);
                List<ApplicationClass> existingAppClasses = result.computeIfAbsent(entry.getKey(), k -> new ArrayList<>());
                existingAppClasses.add(entry.getValue());
            }
        }
        return Collections.unmodifiableSortedMap(result);
    });
}
Also used : StructuredTable(io.cdap.cdap.spi.data.StructuredTable) StructuredRow(io.cdap.cdap.spi.data.StructuredRow) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) Field(io.cdap.cdap.spi.data.table.field.Field) List(java.util.List) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) Map(java.util.Map) SortedMap(java.util.SortedMap) AbstractMap(java.util.AbstractMap) TreeMap(java.util.TreeMap)

Example 2 with ApplicationClass

use of io.cdap.cdap.api.artifact.ApplicationClass in project cdap by caskdata.

the class ArtifactStore method getApplicationClasses.

/**
 * Get all application classes that belong to the specified namespace of the specified classname.
 * Results are returned as a sorted map from artifact to application classes in that artifact.
 * Map entries are sorted by the artifact.
 *
 * @param namespace the namespace from which to get application classes
 * @param className the classname of application classes to get
 * @return an unmodifiable map of artifact the application classes in that artifact.
 *         The map will never be null. If there are no application classes, an empty map will be returned.
 */
public SortedMap<ArtifactDescriptor, ApplicationClass> getApplicationClasses(final NamespaceId namespace, final String className) {
    return TransactionRunners.run(transactionRunner, context -> {
        StructuredTable table = getTable(context, StoreDefinition.ArtifactStore.APP_DATA_TABLE);
        SortedMap<ArtifactDescriptor, ApplicationClass> result = Maps.newTreeMap();
        Collection<Field<?>> keys = ImmutableList.of(Fields.stringField(StoreDefinition.ArtifactStore.NAMESPACE_FIELD, namespace.getNamespace()), Fields.stringField(StoreDefinition.ArtifactStore.CLASS_NAME_FIELD, className));
        try (CloseableIterator<StructuredRow> iterator = table.scan(Range.singleton(keys), Integer.MAX_VALUE)) {
            while (iterator.hasNext()) {
                StructuredRow row = iterator.next();
                Map.Entry<ArtifactDescriptor, ApplicationClass> entry = extractApplicationClass(row);
                result.put(entry.getKey(), entry.getValue());
            }
        }
        return Collections.unmodifiableSortedMap(result);
    });
}
Also used : Field(io.cdap.cdap.spi.data.table.field.Field) StructuredTable(io.cdap.cdap.spi.data.StructuredTable) StructuredRow(io.cdap.cdap.spi.data.StructuredRow) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) Map(java.util.Map) SortedMap(java.util.SortedMap) AbstractMap(java.util.AbstractMap) TreeMap(java.util.TreeMap)

Example 3 with ApplicationClass

use of io.cdap.cdap.api.artifact.ApplicationClass in project cdap by caskdata.

the class DefaultArtifactRepository method getApplicationClasses.

@Override
public List<ApplicationClassInfo> getApplicationClasses(NamespaceId namespace, String className) throws IOException {
    List<ApplicationClassInfo> infos = Lists.newArrayList();
    for (Map.Entry<ArtifactDescriptor, ApplicationClass> entry : artifactStore.getApplicationClasses(namespace, className).entrySet()) {
        ArtifactSummary artifactSummary = ArtifactSummary.from(entry.getKey().getArtifactId());
        ApplicationClass appClass = entry.getValue();
        infos.add(new ApplicationClassInfo(artifactSummary, appClass.getClassName(), appClass.getConfigSchema()));
    }
    return Collections.unmodifiableList(infos);
}
Also used : ArtifactSummary(io.cdap.cdap.api.artifact.ArtifactSummary) ApplicationClassInfo(io.cdap.cdap.proto.artifact.ApplicationClassInfo) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) Map(java.util.Map) SortedMap(java.util.SortedMap) HashMap(java.util.HashMap)

Example 4 with ApplicationClass

use of io.cdap.cdap.api.artifact.ApplicationClass in project cdap by caskdata.

the class ArtifactStore method deleteMeta.

private void deleteMeta(StructuredTableContext context, Id.Artifact artifactId, ArtifactData oldMeta) throws IOException {
    // delete old artifact data
    StructuredTable artifactTable = getTable(context, StoreDefinition.ArtifactStore.ARTIFACT_DATA_TABLE);
    ArtifactCell artifactCell = new ArtifactCell(artifactId);
    artifactTable.delete(artifactCell.keys);
    // delete old appclass metadata
    StructuredTable appClassTable = getTable(context, StoreDefinition.ArtifactStore.APP_DATA_TABLE);
    for (ApplicationClass appClass : oldMeta.meta.getClasses().getApps()) {
        AppClassKey appClassKey = new AppClassKey(artifactId.getNamespace().toEntityId(), appClass.getClassName());
        deleteRangeFromTable(appClassTable, Range.singleton(appClassKey.keys));
    }
    // delete old plugins, we loop twice to only access to one table at a time to prevent deadlock
    StructuredTable pluginDataTable = getTable(context, StoreDefinition.ArtifactStore.PLUGIN_DATA_TABLE);
    for (PluginClass pluginClass : oldMeta.meta.getClasses().getPlugins()) {
        // delete metadata for each artifact this plugin extends
        for (ArtifactRange artifactRange : oldMeta.meta.getUsableBy()) {
            // these four fields are prefixes of the plugin table primary keys
            PluginKeyPrefix pluginKey = new PluginKeyPrefix(artifactRange.getNamespace(), artifactRange.getName(), pluginClass.getType(), pluginClass.getName());
            pluginDataTable.delete(concatFields(pluginKey.keys, artifactCell.keys));
        }
    }
    // Delete the universal plugin row
    StructuredTable uniPluginTable = getTable(context, StoreDefinition.ArtifactStore.UNIV_PLUGIN_DATA_TABLE);
    for (PluginClass pluginClass : oldMeta.meta.getClasses().getPlugins()) {
        if (oldMeta.meta.getUsableBy().isEmpty()) {
            UniversalPluginKeyPrefix pluginKey = new UniversalPluginKeyPrefix(artifactId.getNamespace().getId(), pluginClass.getType(), pluginClass.getName());
            uniPluginTable.delete(concatFields(pluginKey.keys, artifactCell.keys));
        }
    }
    // delete the old jar file
    try {
        new EntityImpersonator(artifactId.toEntityId(), impersonator).impersonate(() -> {
            Locations.getLocationFromAbsolutePath(locationFactory, oldMeta.getLocationPath()).delete();
            return null;
        });
    } catch (IOException ioe) {
        throw ioe;
    } catch (Exception e) {
        // this should not happen
        throw Throwables.propagate(e);
    }
}
Also used : StructuredTable(io.cdap.cdap.spi.data.StructuredTable) EntityImpersonator(io.cdap.cdap.security.impersonation.EntityImpersonator) ArtifactRange(io.cdap.cdap.api.artifact.ArtifactRange) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) IOException(java.io.IOException) PluginClass(io.cdap.cdap.api.plugin.PluginClass) ArtifactAlreadyExistsException(io.cdap.cdap.common.ArtifactAlreadyExistsException) TransactionException(io.cdap.cdap.spi.data.transaction.TransactionException) PluginNotExistsException(io.cdap.cdap.internal.app.runtime.plugin.PluginNotExistsException) IOException(java.io.IOException) TableNotFoundException(io.cdap.cdap.spi.data.TableNotFoundException) ArtifactNotFoundException(io.cdap.cdap.common.ArtifactNotFoundException)

Example 5 with ApplicationClass

use of io.cdap.cdap.api.artifact.ApplicationClass in project cdap by caskdata.

the class ArtifactStore method writeMeta.

// write a new artifact snapshot and clean up the old snapshot data
private void writeMeta(StructuredTableContext context, Id.Artifact artifactId, ArtifactData data) throws IOException {
    // write to artifact data table
    StructuredTable artifactDataTable = getTable(context, StoreDefinition.ArtifactStore.ARTIFACT_DATA_TABLE);
    ArtifactCell artifactCell = new ArtifactCell(artifactId);
    artifactDataTable.upsert(concatFields(artifactCell.keys, Collections.singleton(Fields.stringField(StoreDefinition.ArtifactStore.ARTIFACT_DATA_FIELD, GSON.toJson(data)))));
    ArtifactClasses classes = data.meta.getClasses();
    Location artifactLocation = Locations.getLocationFromAbsolutePath(locationFactory, data.getLocationPath());
    // write appClass metadata
    StructuredTable appTable = getTable(context, StoreDefinition.ArtifactStore.APP_DATA_TABLE);
    ArtifactCell artifactkeys = new ArtifactCell(artifactId);
    for (ApplicationClass appClass : classes.getApps()) {
        // a:{namespace}:{classname}
        AppClassKey appClassKey = new AppClassKey(artifactId.getNamespace().toEntityId(), appClass.getClassName());
        Field<String> appDataField = Fields.stringField(StoreDefinition.ArtifactStore.APP_DATA_FIELD, GSON.toJson(new AppData(appClass, artifactLocation)));
        appTable.upsert(concatFields(appClassKey.keys, artifactkeys.keys, Collections.singleton(appDataField)));
    }
    // write pluginClass metadata, we loop twice to only access to one table at a time to prevent deadlock
    StructuredTable pluginTable = getTable(context, StoreDefinition.ArtifactStore.PLUGIN_DATA_TABLE);
    for (PluginClass pluginClass : classes.getPlugins()) {
        // write metadata for each artifact this plugin extends
        for (ArtifactRange artifactRange : data.meta.getUsableBy()) {
            PluginKeyPrefix pluginKey = new PluginKeyPrefix(artifactRange.getNamespace(), artifactRange.getName(), pluginClass.getType(), pluginClass.getName());
            Field<String> pluginDataField = Fields.stringField(StoreDefinition.ArtifactStore.PLUGIN_DATA_FIELD, GSON.toJson(new PluginData(pluginClass, artifactLocation, artifactRange)));
            pluginTable.upsert(concatFields(pluginKey.keys, artifactkeys.keys, Collections.singleton(pluginDataField)));
        }
    }
    // write universal plugin class metadata
    StructuredTable uniPluginTable = getTable(context, StoreDefinition.ArtifactStore.UNIV_PLUGIN_DATA_TABLE);
    for (PluginClass pluginClass : classes.getPlugins()) {
        // by any other artifact in the same namespace.
        if (data.meta.getUsableBy().isEmpty()) {
            // Write a special entry for plugin that doesn't have parent, which means any artifact can use it
            UniversalPluginKeyPrefix pluginKey = new UniversalPluginKeyPrefix(artifactId.getNamespace().getId(), pluginClass.getType(), pluginClass.getName());
            Field<String> pluginDataField = Fields.stringField(StoreDefinition.ArtifactStore.PLUGIN_DATA_FIELD, GSON.toJson(new PluginData(pluginClass, artifactLocation, null)));
            uniPluginTable.upsert(concatFields(pluginKey.keys, artifactkeys.keys, Collections.singleton(pluginDataField)));
        }
    }
}
Also used : StructuredTable(io.cdap.cdap.spi.data.StructuredTable) ArtifactRange(io.cdap.cdap.api.artifact.ArtifactRange) ApplicationClass(io.cdap.cdap.api.artifact.ApplicationClass) ArtifactClasses(io.cdap.cdap.api.artifact.ArtifactClasses) PluginClass(io.cdap.cdap.api.plugin.PluginClass) Location(org.apache.twill.filesystem.Location)

Aggregations

ApplicationClass (io.cdap.cdap.api.artifact.ApplicationClass)32 Test (org.junit.Test)20 Location (org.apache.twill.filesystem.Location)15 AppDeploymentInfo (io.cdap.cdap.internal.app.deploy.pipeline.AppDeploymentInfo)14 ArtifactId (io.cdap.cdap.proto.id.ArtifactId)10 ArtifactVersion (io.cdap.cdap.api.artifact.ArtifactVersion)8 NamespaceId (io.cdap.cdap.proto.id.NamespaceId)8 LocalLocationFactory (org.apache.twill.filesystem.LocalLocationFactory)8 PluginClass (io.cdap.cdap.api.plugin.PluginClass)7 Id (io.cdap.cdap.common.id.Id)7 LocationFactory (org.apache.twill.filesystem.LocationFactory)7 AllProgramsApp (io.cdap.cdap.AllProgramsApp)6 ArtifactId (io.cdap.cdap.api.artifact.ArtifactId)6 Configurator (io.cdap.cdap.app.deploy.Configurator)6 ConfigTestApp (io.cdap.cdap.ConfigTestApp)5 ApplicationSpecification (io.cdap.cdap.api.app.ApplicationSpecification)5 ArtifactRange (io.cdap.cdap.api.artifact.ArtifactRange)5 ArtifactNotFoundException (io.cdap.cdap.common.ArtifactNotFoundException)5 ReflectionSchemaGenerator (io.cdap.cdap.internal.io.ReflectionSchemaGenerator)5 Gson (com.google.gson.Gson)4