Search in sources :

Example 66 with DatasetId

use of co.cask.cdap.proto.id.DatasetId in project cdap by caskdata.

the class LineageAdmin method getRollupRelations.

private Multimap<RelationKey, Relation> getRollupRelations(Multimap<RelationKey, Relation> relations, Map<ProgramRunId, RunRecordMeta> runRecordMap, Map<String, ProgramRunId> workflowIdMap) throws NotFoundException {
    Multimap<RelationKey, Relation> relationsNew = HashMultimap.create();
    for (Map.Entry<RelationKey, Collection<Relation>> entry : relations.asMap().entrySet()) {
        for (Relation relation : entry.getValue()) {
            ProgramRunId workflowProgramRunId = getWorkflowProgramRunid(relation, runRecordMap, workflowIdMap);
            if (workflowProgramRunId == null) {
                relationsNew.put(entry.getKey(), relation);
            } else {
                ProgramId workflowProgramId = new ProgramId(workflowProgramRunId.getNamespace(), workflowProgramRunId.getApplication(), workflowProgramRunId.getType(), workflowProgramRunId.getProgram());
                Relation workflowRelation;
                NamespacedEntityId data = relation.getData();
                if (data instanceof DatasetId) {
                    workflowRelation = new Relation((DatasetId) data, workflowProgramId, relation.getAccess(), RunIds.fromString(workflowProgramRunId.getRun()));
                } else {
                    workflowRelation = new Relation((StreamId) data, workflowProgramId, relation.getAccess(), RunIds.fromString(workflowProgramRunId.getRun()));
                }
                relationsNew.put(entry.getKey(), workflowRelation);
            }
        }
    }
    return relationsNew;
}
Also used : Relation(co.cask.cdap.data2.metadata.lineage.Relation) NamespacedEntityId(co.cask.cdap.proto.id.NamespacedEntityId) StreamId(co.cask.cdap.proto.id.StreamId) Collection(java.util.Collection) ProgramRunId(co.cask.cdap.proto.id.ProgramRunId) ProgramId(co.cask.cdap.proto.id.ProgramId) HashMap(java.util.HashMap) Map(java.util.Map) DatasetId(co.cask.cdap.proto.id.DatasetId)

Example 67 with DatasetId

use of co.cask.cdap.proto.id.DatasetId in project cdap by caskdata.

the class MetadataHttpHandler method removeDatasetProperty.

@DELETE
@Path("/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties/{property}")
public void removeDatasetProperty(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespaceId, @PathParam("dataset-id") String datasetId, @PathParam("property") String property) throws NotFoundException {
    DatasetId dataset = new DatasetId(namespaceId, datasetId);
    metadataAdmin.removeProperties(dataset, property);
    responder.sendString(HttpResponseStatus.OK, String.format("Metadata property %s for dataset %s deleted successfully.", property, dataset));
}
Also used : DatasetId(co.cask.cdap.proto.id.DatasetId) Path(javax.ws.rs.Path) DELETE(javax.ws.rs.DELETE)

Example 68 with DatasetId

use of co.cask.cdap.proto.id.DatasetId in project cdap by caskdata.

the class MetadataHttpHandler method addDatasetProperties.

@POST
@Path("/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties")
@AuditPolicy(AuditDetail.REQUEST_BODY)
public void addDatasetProperties(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespaceId, @PathParam("dataset-id") String datasetId) throws BadRequestException, NotFoundException {
    DatasetId dataset = new DatasetId(namespaceId, datasetId);
    metadataAdmin.addProperties(dataset, readMetadata(request));
    responder.sendString(HttpResponseStatus.OK, "Metadata added successfully to " + dataset);
}
Also used : DatasetId(co.cask.cdap.proto.id.DatasetId) Path(javax.ws.rs.Path) AuditPolicy(co.cask.cdap.common.security.AuditPolicy) POST(javax.ws.rs.POST)

Example 69 with DatasetId

use of co.cask.cdap.proto.id.DatasetId in project cdap by caskdata.

the class MetadataHttpHandler method addDatasetTags.

@POST
@Path("/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/tags")
@AuditPolicy(AuditDetail.REQUEST_BODY)
public void addDatasetTags(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespaceId, @PathParam("dataset-id") String datasetId) throws BadRequestException, NotFoundException {
    DatasetId dataset = new DatasetId(namespaceId, datasetId);
    metadataAdmin.addTags(dataset, readArray(request));
    responder.sendString(HttpResponseStatus.OK, String.format("Added tags to dataset %s successfully.", dataset));
}
Also used : DatasetId(co.cask.cdap.proto.id.DatasetId) Path(javax.ws.rs.Path) AuditPolicy(co.cask.cdap.common.security.AuditPolicy) POST(javax.ws.rs.POST)

Example 70 with DatasetId

use of co.cask.cdap.proto.id.DatasetId in project cdap by caskdata.

the class MetadataHttpHandler method removeDatasetProperties.

@DELETE
@Path("/namespaces/{namespace-id}/datasets/{dataset-id}/metadata/properties")
public void removeDatasetProperties(HttpRequest request, HttpResponder responder, @PathParam("namespace-id") String namespaceId, @PathParam("dataset-id") String datasetId) throws NotFoundException {
    DatasetId dataset = new DatasetId(namespaceId, datasetId);
    metadataAdmin.removeProperties(dataset);
    responder.sendString(HttpResponseStatus.OK, String.format("Metadata properties for dataset %s deleted successfully.", dataset));
}
Also used : DatasetId(co.cask.cdap.proto.id.DatasetId) Path(javax.ws.rs.Path) DELETE(javax.ws.rs.DELETE)

Aggregations

DatasetId (co.cask.cdap.proto.id.DatasetId)180 Test (org.junit.Test)96 ProgramId (co.cask.cdap.proto.id.ProgramId)34 StreamId (co.cask.cdap.proto.id.StreamId)34 Path (javax.ws.rs.Path)34 TransactionExecutor (org.apache.tephra.TransactionExecutor)31 NamespaceId (co.cask.cdap.proto.id.NamespaceId)25 ApplicationId (co.cask.cdap.proto.id.ApplicationId)23 IOException (java.io.IOException)19 POST (javax.ws.rs.POST)17 TransactionFailureException (org.apache.tephra.TransactionFailureException)17 DatasetSpecification (co.cask.cdap.api.dataset.DatasetSpecification)16 QueryResult (co.cask.cdap.proto.QueryResult)16 DatasetManagementException (co.cask.cdap.api.dataset.DatasetManagementException)15 ColumnDesc (co.cask.cdap.proto.ColumnDesc)14 Map (java.util.Map)13 NoSuchElementException (java.util.NoSuchElementException)13 Table (co.cask.cdap.api.dataset.table.Table)12 NamespaceMeta (co.cask.cdap.proto.NamespaceMeta)11 ProgramRunId (co.cask.cdap.proto.id.ProgramRunId)11