Search in sources :

Example 1 with UpdateStorageResponse

use of ddf.catalog.content.operation.UpdateStorageResponse in project ddf by codice.

the class FileSystemStorageProvider method update.

@Override
public UpdateStorageResponse update(UpdateStorageRequest updateRequest) throws StorageException {
    LOGGER.trace("ENTERING: update");
    List<ContentItem> contentItems = updateRequest.getContentItems();
    List<ContentItem> updatedItems = new ArrayList<>(updateRequest.getContentItems().size());
    for (ContentItem contentItem : contentItems) {
        try {
            if (!ContentItemValidator.validate(contentItem)) {
                LOGGER.warn("Item is not valid: {}", contentItem);
                continue;
            }
            ContentItem updateItem = contentItem;
            if (StringUtils.isBlank(contentItem.getFilename()) || StringUtils.equals(contentItem.getFilename(), ContentItem.DEFAULT_FILE_NAME)) {
                ContentItem existingItem = readContent(new URI(contentItem.getUri()));
                updateItem = new ContentItemDecorator(contentItem, existingItem);
            }
            Path contentIdDir = getTempContentItemDir(updateRequest.getId(), new URI(updateItem.getUri()));
            updatedItems.add(generateContentFile(updateItem, contentIdDir, (String) updateRequest.getPropertyValue(Constants.STORE_REFERENCE_KEY)));
        } catch (IOException | URISyntaxException | IllegalArgumentException e) {
            throw new StorageException(e);
        }
    }
    for (ContentItem contentItem : updatedItems) {
        if (contentItem.getMetacard().getResourceURI() == null && StringUtils.isBlank(contentItem.getQualifier())) {
            contentItem.getMetacard().setAttribute(new AttributeImpl(Metacard.RESOURCE_URI, contentItem.getUri()));
            try {
                contentItem.getMetacard().setAttribute(new AttributeImpl(Metacard.RESOURCE_SIZE, contentItem.getSize()));
            } catch (IOException e) {
                LOGGER.info("Could not set size of content item [{}] on metacard [{}]", contentItem.getId(), contentItem.getMetacard().getId(), e);
            }
        }
    }
    UpdateStorageResponse response = new UpdateStorageResponseImpl(updateRequest, updatedItems);
    updateMap.put(updateRequest.getId(), updatedItems.stream().map(ContentItem::getUri).collect(Collectors.toSet()));
    LOGGER.trace("EXITING: update");
    return response;
}
Also used : Path(java.nio.file.Path) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) ArrayList(java.util.ArrayList) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) URI(java.net.URI) UpdateStorageResponseImpl(ddf.catalog.content.operation.impl.UpdateStorageResponseImpl) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) StorageException(ddf.catalog.content.StorageException) ContentItem(ddf.catalog.content.data.ContentItem)

Example 2 with UpdateStorageResponse

use of ddf.catalog.content.operation.UpdateStorageResponse in project ddf by codice.

the class FileSystemStorageProviderTest method testCreateWithQualifierAndOneInvalidItem.

@Test
public void testCreateWithQualifierAndOneInvalidItem() throws Exception {
    String uuid = UUID.randomUUID().toString().replaceAll("-", "");
    ByteSource byteSource = new ByteSource() {

        @Override
        public InputStream openStream() throws IOException {
            return IOUtils.toInputStream("This data is my data, this data is your data.");
        }
    };
    ContentItem contentItem = new ContentItemImpl(uuid, null, byteSource, "application/text", "datadatadata", byteSource.size(), mock(Metacard.class));
    String invalidUuid = "wow-this-isn't-a-valid-uuid-right?!@#%";
    ContentItem badContentItem = new ContentItemImpl(invalidUuid, null, byteSource, "application/text", "datadatadata", byteSource.size(), mock(Metacard.class));
    CreateStorageRequest createRequest = new CreateStorageRequestImpl(Lists.newArrayList(contentItem, badContentItem), null);
    CreateStorageResponse createResponse = provider.create(createRequest);
    assertThat(createResponse.getCreatedContentItems().size(), is(1));
    assertThat(createResponse.getCreatedContentItems().get(0).getId(), is(uuid));
    ContentItem updateContentItem = new ContentItemImpl(invalidUuid, null, byteSource, "application/text", "datadatadata", byteSource.size(), mock(Metacard.class));
    UpdateStorageRequest updateRequest = new UpdateStorageRequestImpl(Lists.newArrayList(updateContentItem), null);
    UpdateStorageResponse updateResponse = provider.update(updateRequest);
    assertThat(updateResponse.getUpdatedContentItems().size(), is(0));
}
Also used : CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Metacard(ddf.catalog.data.Metacard) UpdateStorageRequestImpl(ddf.catalog.content.operation.impl.UpdateStorageRequestImpl) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) UpdateStorageRequest(ddf.catalog.content.operation.UpdateStorageRequest) ByteSource(com.google.common.io.ByteSource) Matchers.isEmptyString(org.hamcrest.Matchers.isEmptyString) ContentItem(ddf.catalog.content.data.ContentItem) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl) CreateStorageRequest(ddf.catalog.content.operation.CreateStorageRequest) Test(org.junit.Test)

Example 3 with UpdateStorageResponse

use of ddf.catalog.content.operation.UpdateStorageResponse in project ddf by codice.

the class HistorianTest method testUpdateStorageResponseSkipProperty.

@Test
public void testUpdateStorageResponseSkipProperty() throws SourceUnavailableException, IngestException, UnsupportedQueryException {
    Map<String, Serializable> properties = new HashMap<>();
    properties.put(MetacardVersion.SKIP_VERSIONING, true);
    UpdateStorageResponse updateStorageResponse = mock(UpdateStorageResponse.class);
    when(updateStorageResponse.getProperties()).thenReturn(properties);
    historian.version(mock(UpdateStorageRequest.class), updateStorageResponse, mock(UpdateResponse.class));
    verifyZeroInteractions(catalogProvider);
}
Also used : UpdateResponse(ddf.catalog.operation.UpdateResponse) Serializable(java.io.Serializable) HashMap(java.util.HashMap) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) UpdateStorageRequest(ddf.catalog.content.operation.UpdateStorageRequest) Test(org.junit.Test)

Example 4 with UpdateStorageResponse

use of ddf.catalog.content.operation.UpdateStorageResponse in project ddf by codice.

the class Historian method version.

/**
     * Versions updated {@link Metacard}s and {@link ContentItem}s.
     *
     * @param streamUpdateRequest   Needed to pass {@link ddf.catalog.core.versioning.MetacardVersion#SKIP_VERSIONING}
     *                              flag into downstream update
     * @param updateStorageResponse Versions this response's updated items
     * @return the update response originally passed in
     * @throws UnsupportedQueryException
     * @throws SourceUnavailableException
     * @throws IngestException
     */
public UpdateStorageResponse version(UpdateStorageRequest streamUpdateRequest, UpdateStorageResponse updateStorageResponse, UpdateResponse updateResponse) throws UnsupportedQueryException, SourceUnavailableException, IngestException {
    if (doSkip(updateStorageResponse)) {
        return updateStorageResponse;
    }
    setSkipFlag(streamUpdateRequest);
    setSkipFlag(updateStorageResponse);
    List<Metacard> updatedMetacards = updateStorageResponse.getUpdatedContentItems().stream().filter(ci -> StringUtils.isBlank(ci.getQualifier())).map(ContentItem::getMetacard).filter(Objects::nonNull).filter(isNotVersionNorDeleted).collect(Collectors.toList());
    Map<String, Metacard> originalMetacards = query(forIds(updatedMetacards.stream().map(Metacard::getId).collect(Collectors.toList())));
    Collection<ReadStorageRequest> ids = getReadStorageRequests(updatedMetacards);
    Map<String, List<ContentItem>> content = getContent(ids);
    Function<String, Action> getAction = (id) -> content.containsKey(id) ? Action.VERSIONED_CONTENT : Action.VERSIONED;
    Map<String, Metacard> versionMetacards = getVersionMetacards(originalMetacards.values(), getAction, (Subject) updateResponse.getProperties().get(SecurityConstants.SECURITY_SUBJECT));
    CreateStorageResponse createStorageResponse = versionContentItems(content, versionMetacards);
    if (createStorageResponse == null) {
        LOGGER.debug("Could not version content items.");
        return updateStorageResponse;
    }
    setResourceUriForContent(/*mutable*/
    versionMetacards, createStorageResponse);
    storeVersionMetacards(versionMetacards);
    return updateStorageResponse;
}
Also used : UnsupportedQueryException(ddf.catalog.source.UnsupportedQueryException) LoggerFactory(org.slf4j.LoggerFactory) StringUtils(org.apache.commons.lang3.StringUtils) ReadStorageRequest(ddf.catalog.content.operation.ReadStorageRequest) UpdateStorageRequest(ddf.catalog.content.operation.UpdateStorageRequest) DeletedMetacardImpl(ddf.catalog.core.versioning.impl.DeletedMetacardImpl) Map(java.util.Map) Action(ddf.catalog.core.versioning.MetacardVersion.Action) SubjectUtils(ddf.security.SubjectUtils) Bundle(org.osgi.framework.Bundle) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) MetacardVersionImpl(ddf.catalog.core.versioning.impl.MetacardVersionImpl) Predicate(java.util.function.Predicate) Collection(java.util.Collection) Collectors(java.util.stream.Collectors) BundleContext(org.osgi.framework.BundleContext) MetacardType(ddf.catalog.data.MetacardType) Objects(java.util.Objects) StorageException(ddf.catalog.content.StorageException) List(java.util.List) Operation(ddf.catalog.operation.Operation) CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Optional(java.util.Optional) UpdateResponse(ddf.catalog.operation.UpdateResponse) QueryRequestImpl(ddf.catalog.operation.impl.QueryRequestImpl) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl) FilterBuilder(ddf.catalog.filter.FilterBuilder) SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) HashMap(java.util.HashMap) Callable(java.util.concurrent.Callable) DeleteResponse(ddf.catalog.operation.DeleteResponse) Function(java.util.function.Function) Update(ddf.catalog.operation.Update) ArrayList(java.util.ArrayList) Lists(com.google.common.collect.Lists) SKIP_VERSIONING(ddf.catalog.core.versioning.MetacardVersion.SKIP_VERSIONING) ContentItem(ddf.catalog.content.data.ContentItem) CreateResponse(ddf.catalog.operation.CreateResponse) Metacard(ddf.catalog.data.Metacard) SecurityConstants(ddf.security.SecurityConstants) StorageProvider(ddf.catalog.content.StorageProvider) ByteSource(com.google.common.io.ByteSource) Result(ddf.catalog.data.Result) Hashtable(java.util.Hashtable) Nullable(javax.annotation.Nullable) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) QueryImpl(ddf.catalog.operation.impl.QueryImpl) Logger(org.slf4j.Logger) Security(org.codice.ddf.security.common.Security) IngestException(ddf.catalog.source.IngestException) IOException(java.io.IOException) Subject(ddf.security.Subject) TimeUnit(java.util.concurrent.TimeUnit) StorageRequest(ddf.catalog.content.operation.StorageRequest) ReadStorageRequestImpl(ddf.catalog.content.operation.impl.ReadStorageRequestImpl) SourceResponse(ddf.catalog.operation.SourceResponse) CatalogProvider(ddf.catalog.source.CatalogProvider) Filter(org.opengis.filter.Filter) Collections(java.util.Collections) FrameworkUtil(org.osgi.framework.FrameworkUtil) InputStream(java.io.InputStream) UuidGenerator(org.codice.ddf.platform.util.uuidgenerator.UuidGenerator) ReadStorageResponse(ddf.catalog.content.operation.ReadStorageResponse) Action(ddf.catalog.core.versioning.MetacardVersion.Action) CreateStorageResponse(ddf.catalog.content.operation.CreateStorageResponse) Metacard(ddf.catalog.data.Metacard) ReadStorageRequest(ddf.catalog.content.operation.ReadStorageRequest) Objects(java.util.Objects) List(java.util.List) ArrayList(java.util.ArrayList)

Example 5 with UpdateStorageResponse

use of ddf.catalog.content.operation.UpdateStorageResponse in project ddf by codice.

the class HistorianTest method testReadStorageException.

@Test
public void testReadStorageException() throws StorageException, SourceUnavailableException, IngestException, UnsupportedQueryException, URISyntaxException {
    StorageProvider exceptionStorageProvider = mock(StorageProvider.class);
    when(exceptionStorageProvider.read(any())).thenThrow(StorageException.class);
    historian.setStorageProviders(Collections.singletonList(exceptionStorageProvider));
    Metacard metacard = getMetacardUpdatePair().get(0);
    // Parameters for historian
    UpdateStorageRequest storageRequest = mock(UpdateStorageRequest.class);
    UpdateStorageResponse storageResponse = mock(UpdateStorageResponse.class);
    UpdateResponse updateResponse = mock(UpdateResponse.class);
    // create the update request
    updateMetacard(storageRequest, storageResponse, metacard);
    mockQuery(metacard);
    historian.version(storageRequest, storageResponse, updateResponse);
    // Verify that it wasn't updated
    verify(catalogProvider, times(0)).create(anyObject());
}
Also used : UpdateResponse(ddf.catalog.operation.UpdateResponse) DeletedMetacard(ddf.catalog.core.versioning.DeletedMetacard) Metacard(ddf.catalog.data.Metacard) UpdateStorageResponse(ddf.catalog.content.operation.UpdateStorageResponse) UpdateStorageRequest(ddf.catalog.content.operation.UpdateStorageRequest) StorageProvider(ddf.catalog.content.StorageProvider) Test(org.junit.Test)

Aggregations

UpdateStorageResponse (ddf.catalog.content.operation.UpdateStorageResponse)16 UpdateStorageRequest (ddf.catalog.content.operation.UpdateStorageRequest)14 Metacard (ddf.catalog.data.Metacard)13 Test (org.junit.Test)12 UpdateResponse (ddf.catalog.operation.UpdateResponse)11 ContentItem (ddf.catalog.content.data.ContentItem)9 DeletedMetacard (ddf.catalog.core.versioning.DeletedMetacard)8 StorageProvider (ddf.catalog.content.StorageProvider)4 CreateStorageResponse (ddf.catalog.content.operation.CreateStorageResponse)4 ReadStorageRequest (ddf.catalog.content.operation.ReadStorageRequest)4 HashMap (java.util.HashMap)4 StorageException (ddf.catalog.content.StorageException)3 ReadStorageResponse (ddf.catalog.content.operation.ReadStorageResponse)3 StorageRequest (ddf.catalog.content.operation.StorageRequest)3 UpdateStorageRequestImpl (ddf.catalog.content.operation.impl.UpdateStorageRequestImpl)3 AttributeImpl (ddf.catalog.data.impl.AttributeImpl)3 ArrayList (java.util.ArrayList)3 ByteSource (com.google.common.io.ByteSource)2 ContentItemImpl (ddf.catalog.content.data.impl.ContentItemImpl)2 DeleteStorageRequest (ddf.catalog.content.operation.DeleteStorageRequest)2