Search in sources :

Example 1 with Action

use of ddf.catalog.core.versioning.MetacardVersion.Action in project ddf by codice.

the class Historian method version.

/**
     * Versions updated {@link Metacard}s and {@link ContentItem}s.
     *
     * @param streamUpdateRequest   Needed to pass {@link ddf.catalog.core.versioning.MetacardVersion#SKIP_VERSIONING}
     *                              flag into downstream update
     * @param updateStorageResponse Versions this response's updated items
     * @return the update response originally passed in
     * @throws UnsupportedQueryException
     * @throws SourceUnavailableException
     * @throws IngestException
     */
public UpdateStorageResponse version(UpdateStorageRequest streamUpdateRequest, UpdateStorageResponse updateStorageResponse, UpdateResponse updateResponse) throws UnsupportedQueryException, SourceUnavailableException, IngestException {
    if (doSkip(updateStorageResponse)) {
        return updateStorageResponse;
    }
    setSkipFlag(streamUpdateRequest);
    setSkipFlag(updateStorageResponse);
    List<Metacard> updatedMetacards = updateStorageResponse.getUpdatedContentItems().stream().filter(ci -> StringUtils.isBlank(ci.getQualifier())).map(ContentItem::getMetacard).filter(Objects::nonNull).filter(isNotVersionNorDeleted).collect(Collectors.toList());
    Map<String, Metacard> originalMetacards = query(forIds(updatedMetacards.stream().map(Metacard::getId).collect(Collectors.toList())));
    Collection<ReadStorageRequest> ids = getReadStorageRequests(updatedMetacards);
    Map<String, List<ContentItem>> content = getContent(ids);
    Function<String, Action> getAction = (id) -> content.containsKey(id) ? Action.VERSIONED_CONTENT : Action.VERSIONED;
    Map<String, Metacard> versionMetacards = getVersionMetacards(originalMetacards.values(), getAction, (Subject) updateResponse.getProperties().get(SecurityConstants.SECURITY_SUBJECT));
    CreateStorageResponse createStorageResponse = versionContentItems(content, versionMetacards);
    if (createStorageResponse == null) {
        LOGGER.debug("Could not version content items.");
        return updateStorageResponse;
    }
    setResourceUriForContent(/*mutable*/
    versionMetacards, createStorageResponse);
    storeVersionMetacards(versionMetacards);
    return updateStorageResponse;
}
Also used : UnsupportedQueryException(ddf.catalog.source.UnsupportedQueryException) LoggerFactory(org.slf4j.LoggerFactory) StringUtils(org.apache.commons.lang3.StringUtils) ReadStorageRequest(ddf.catalog.content.operation.ReadStorageRequest) UpdateStorageRequest(ddf.catalog.content.operation.UpdateStorageRequest) DeletedMetacardImpl(ddf.catalog.core.versioning.impl.DeletedMetacardImpl) Map(java.util.Map) Action(ddf.catalog.core.versioning.MetacardVersion.Action) SubjectUtils(ddf.security.SubjectUtils) Bundle(org.osgi.framework.Bundle) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) MetacardVersionImpl(ddf.catalog.core.versioning.impl.MetacardVersionImpl) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Collectors(java.util.stream.Collectors) BundleContext(org.osgi.framework.BundleContext) MetacardType(ddf.catalog.data.MetacardType) Objects(java.util.Objects) StorageException(ddf.catalog.content.StorageException) List(java.util.List) Operation(ddf.catalog.operation.Operation) CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Optional(java.util.Optional) UpdateResponse(ddf.catalog.operation.UpdateResponse) QueryRequestImpl(ddf.catalog.operation.impl.QueryRequestImpl) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl) FilterBuilder(ddf.catalog.filter.FilterBuilder) SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) HashMap(java.util.HashMap) Callable(java.util.concurrent.Callable) DeleteResponse(ddf.catalog.operation.DeleteResponse) Function(java.util.function.Function) Update(ddf.catalog.operation.Update) ArrayList(java.util.ArrayList) Lists(com.google.common.collect.Lists) SKIP_VERSIONING(ddf.catalog.core.versioning.MetacardVersion.SKIP_VERSIONING) ContentItem(ddf.catalog.content.data.ContentItem) CreateResponse(ddf.catalog.operation.CreateResponse) Metacard(ddf.catalog.data.Metacard) SecurityConstants(ddf.security.SecurityConstants) StorageProvider(ddf.catalog.content.StorageProvider) ByteSource(com.google.common.io.ByteSource) Result(ddf.catalog.data.Result) Hashtable(java.util.Hashtable) Nullable(javax.annotation.Nullable) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) QueryImpl(ddf.catalog.operation.impl.QueryImpl) Logger(org.slf4j.Logger) Security(org.codice.ddf.security.common.Security) IngestException(ddf.catalog.source.IngestException) IOException(java.io.IOException) Subject(ddf.security.Subject) TimeUnit(java.util.concurrent.TimeUnit) StorageRequest(ddf.catalog.content.operation.StorageRequest) ReadStorageRequestImpl(ddf.catalog.content.operation.impl.ReadStorageRequestImpl) SourceResponse(ddf.catalog.operation.SourceResponse) CatalogProvider(ddf.catalog.source.CatalogProvider) Filter(org.opengis.filter.Filter) Collections(java.util.Collections) FrameworkUtil(org.osgi.framework.FrameworkUtil) InputStream(java.io.InputStream) UuidGenerator(org.codice.ddf.platform.util.uuidgenerator.UuidGenerator) ReadStorageResponse(ddf.catalog.content.operation.ReadStorageResponse) Action(ddf.catalog.core.versioning.MetacardVersion.Action) CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Metacard(ddf.catalog.data.Metacard) ReadStorageRequest(ddf.catalog.content.operation.ReadStorageRequest) Objects(java.util.Objects) List(java.util.List) ArrayList(java.util.ArrayList)

Example 2 with Action

use of ddf.catalog.core.versioning.MetacardVersion.Action in project ddf by codice.

the class Historian method version.

/**
     * Versions deleted {@link Metacard}s.
     *
     * @param deleteResponse Versions this responses deleted metacards
     */
public DeleteResponse version(DeleteResponse deleteResponse) throws SourceUnavailableException, IngestException {
    if (doSkip(deleteResponse)) {
        return deleteResponse;
    }
    setSkipFlag(deleteResponse);
    List<Metacard> deletedMetacards = deleteResponse.getDeletedMetacards().stream().filter(isNotVersionNorDeleted).collect(Collectors.toList());
    // [ContentItem.getId: content items]
    Map<String, List<ContentItem>> contentItems = getContent(getReadStorageRequests(deletedMetacards));
    Function<String, Action> getAction = (id) -> contentItems.containsKey(id) ? Action.DELETED_CONTENT : Action.DELETED;
    // [MetacardVersion.VERSION_OF_ID: versioned metacard]
    Map<String, Metacard> versionedMap = getVersionMetacards(deletedMetacards, getAction, (Subject) deleteResponse.getRequest().getProperties().get(SecurityConstants.SECURITY_SUBJECT));
    CreateStorageResponse createStorageResponse = versionContentItems(contentItems, versionedMap);
    if (createStorageResponse != null) {
        setResourceUriForContent(/*Mutable*/
        versionedMap, createStorageResponse);
    }
    executeAsSystem(() -> catalogProvider().create(new CreateRequestImpl(new ArrayList<>(versionedMap.values()))));
    String emailAddress = SubjectUtils.getEmailAddress((Subject) deleteResponse.getProperties().get(SecurityConstants.SECURITY_SUBJECT));
    List<Metacard> deletionMetacards = versionedMap.entrySet().stream().map(s -> new DeletedMetacardImpl(uuidGenerator.generateUuid(), s.getKey(), emailAddress, s.getValue().getId(), MetacardVersionImpl.toMetacard(s.getValue(), metacardTypes))).collect(Collectors.toList());
    executeAsSystem(() -> catalogProvider().create(new CreateRequestImpl(deletionMetacards, new HashMap<>())));
    return deleteResponse;
}
Also used : UnsupportedQueryException(ddf.catalog.source.UnsupportedQueryException) LoggerFactory(org.slf4j.LoggerFactory) StringUtils(org.apache.commons.lang3.StringUtils) ReadStorageRequest(ddf.catalog.content.operation.ReadStorageRequest) UpdateStorageRequest(ddf.catalog.content.operation.UpdateStorageRequest) DeletedMetacardImpl(ddf.catalog.core.versioning.impl.DeletedMetacardImpl) Map(java.util.Map) Action(ddf.catalog.core.versioning.MetacardVersion.Action) SubjectUtils(ddf.security.SubjectUtils) Bundle(org.osgi.framework.Bundle) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) MetacardVersionImpl(ddf.catalog.core.versioning.impl.MetacardVersionImpl) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Collectors(java.util.stream.Collectors) BundleContext(org.osgi.framework.BundleContext) MetacardType(ddf.catalog.data.MetacardType) Objects(java.util.Objects) StorageException(ddf.catalog.content.StorageException) List(java.util.List) Operation(ddf.catalog.operation.Operation) CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Optional(java.util.Optional) UpdateResponse(ddf.catalog.operation.UpdateResponse) QueryRequestImpl(ddf.catalog.operation.impl.QueryRequestImpl) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl) FilterBuilder(ddf.catalog.filter.FilterBuilder) SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) HashMap(java.util.HashMap) Callable(java.util.concurrent.Callable) DeleteResponse(ddf.catalog.operation.DeleteResponse) Function(java.util.function.Function) Update(ddf.catalog.operation.Update) ArrayList(java.util.ArrayList) Lists(com.google.common.collect.Lists) SKIP_VERSIONING(ddf.catalog.core.versioning.MetacardVersion.SKIP_VERSIONING) ContentItem(ddf.catalog.content.data.ContentItem) CreateResponse(ddf.catalog.operation.CreateResponse) Metacard(ddf.catalog.data.Metacard) SecurityConstants(ddf.security.SecurityConstants) StorageProvider(ddf.catalog.content.StorageProvider) ByteSource(com.google.common.io.ByteSource) Result(ddf.catalog.data.Result) Hashtable(java.util.Hashtable) Nullable(javax.annotation.Nullable) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) QueryImpl(ddf.catalog.operation.impl.QueryImpl) Logger(org.slf4j.Logger) Security(org.codice.ddf.security.common.Security) IngestException(ddf.catalog.source.IngestException) IOException(java.io.IOException) Subject(ddf.security.Subject) TimeUnit(java.util.concurrent.TimeUnit) StorageRequest(ddf.catalog.content.operation.StorageRequest) ReadStorageRequestImpl(ddf.catalog.content.operation.impl.ReadStorageRequestImpl) SourceResponse(ddf.catalog.operation.SourceResponse) CatalogProvider(ddf.catalog.source.CatalogProvider) Filter(org.opengis.filter.Filter) Collections(java.util.Collections) FrameworkUtil(org.osgi.framework.FrameworkUtil) InputStream(java.io.InputStream) UuidGenerator(org.codice.ddf.platform.util.uuidgenerator.UuidGenerator) ReadStorageResponse(ddf.catalog.content.operation.ReadStorageResponse) Action(ddf.catalog.core.versioning.MetacardVersion.Action) CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Metacard(ddf.catalog.data.Metacard) DeletedMetacardImpl(ddf.catalog.core.versioning.impl.DeletedMetacardImpl) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) List(java.util.List) ArrayList(java.util.ArrayList)

Example 3 with Action

use of ddf.catalog.core.versioning.MetacardVersion.Action in project ddf by codice.

the class MetacardApplication method revertMetacard.

private void revertMetacard(Metacard versionMetacard, String id, boolean alreadyCreated) throws SourceUnavailableException, IngestException, FederationException, UnsupportedQueryException {
    LOGGER.trace("Reverting metacard [{}] to version [{}]", id, versionMetacard.getId());
    Metacard revertMetacard = MetacardVersionImpl.toMetacard(versionMetacard, types);
    Action action = Action.fromKey((String) versionMetacard.getAttribute(MetacardVersion.ACTION).getValue());
    if (DELETE_ACTIONS.contains(action)) {
        attemptDeleteDeletedMetacard(id);
        if (!alreadyCreated) {
            catalogFramework.create(new CreateRequestImpl(revertMetacard));
        }
    } else {
        tryUpdate(4, () -> {
            catalogFramework.update(new UpdateRequestImpl(id, revertMetacard));
            return true;
        });
    }
}
Also used : DeletedMetacard(ddf.catalog.core.versioning.DeletedMetacard) Metacard(ddf.catalog.data.Metacard) Action(ddf.catalog.core.versioning.MetacardVersion.Action) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) UpdateRequestImpl(ddf.catalog.operation.impl.UpdateRequestImpl)

Example 4 with Action

use of ddf.catalog.core.versioning.MetacardVersion.Action in project ddf by codice.

the class MetacardApplication method revertContentandMetacard.

private void revertContentandMetacard(Metacard latestContent, Metacard versionMetacard, String id) throws SourceUnavailableException, IngestException, ResourceNotFoundException, IOException, ResourceNotSupportedException, FederationException, UnsupportedQueryException {
    LOGGER.trace("Reverting content and metacard for metacard [{}]. \nLatest content: [{}] \nVersion metacard: [{}]", id, latestContent.getId(), versionMetacard.getId());
    Map<String, Serializable> properties = new HashMap<>();
    properties.put("no-default-tags", true);
    ResourceResponse latestResource = catalogFramework.getLocalResource(new ResourceRequestById(latestContent.getId(), properties));
    ContentItemImpl contentItem = new ContentItemImpl(id, new ByteSourceWrapper(() -> latestResource.getResource().getInputStream()), latestResource.getResource().getMimeTypeValue(), latestResource.getResource().getName(), latestResource.getResource().getSize(), MetacardVersionImpl.toMetacard(versionMetacard, types));
    // Try to delete the "deleted metacard" marker first.
    boolean alreadyCreated = false;
    Action action = Action.fromKey((String) versionMetacard.getAttribute(MetacardVersion.ACTION).getValue());
    if (DELETE_ACTIONS.contains(action)) {
        alreadyCreated = true;
        catalogFramework.create(new CreateStorageRequestImpl(Collections.singletonList(contentItem), id, new HashMap<>()));
    } else {
        // Currently we can't guarantee the metacard will exist yet because of the 1 second
        // soft commit in solr. this busy wait loop should be fixed when alternate solution
        // is found.
        tryUpdate(4, () -> {
            catalogFramework.update(new UpdateStorageRequestImpl(Collections.singletonList(contentItem), id, new HashMap<>()));
            return true;
        });
    }
    LOGGER.trace("Successfully reverted metacard content for [{}]", id);
    revertMetacard(versionMetacard, id, alreadyCreated);
}
Also used : Serializable(java.io.Serializable) Action(ddf.catalog.core.versioning.MetacardVersion.Action) ResourceResponse(ddf.catalog.operation.ResourceResponse) UpdateStorageRequestImpl(ddf.catalog.content.operation.impl.UpdateStorageRequestImpl) HashMap(java.util.HashMap) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) ResourceRequestById(ddf.catalog.operation.impl.ResourceRequestById) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl)

Aggregations

Action (ddf.catalog.core.versioning.MetacardVersion.Action)4 ContentItemImpl (ddf.catalog.content.data.impl.ContentItemImpl)3 CreateStorageRequestImpl (ddf.catalog.content.operation.impl.CreateStorageRequestImpl)3 Metacard (ddf.catalog.data.Metacard)3 CreateRequestImpl (ddf.catalog.operation.impl.CreateRequestImpl)3 HashMap (java.util.HashMap)3 Lists (com.google.common.collect.Lists)2 ByteSource (com.google.common.io.ByteSource)2 StorageException (ddf.catalog.content.StorageException)2 StorageProvider (ddf.catalog.content.StorageProvider)2 ContentItem (ddf.catalog.content.data.ContentItem)2 CreateStorageResponse (ddf.catalog.content.operation.CreateStorageResponse)2 ReadStorageRequest (ddf.catalog.content.operation.ReadStorageRequest)2 ReadStorageResponse (ddf.catalog.content.operation.ReadStorageResponse)2 StorageRequest (ddf.catalog.content.operation.StorageRequest)2 UpdateStorageRequest (ddf.catalog.content.operation.UpdateStorageRequest)2 UpdateStorageResponse (ddf.catalog.content.operation.UpdateStorageResponse)2 ReadStorageRequestImpl (ddf.catalog.content.operation.impl.ReadStorageRequestImpl)2 SKIP_VERSIONING (ddf.catalog.core.versioning.MetacardVersion.SKIP_VERSIONING)2 DeletedMetacardImpl (ddf.catalog.core.versioning.impl.DeletedMetacardImpl)2