Search in sources :

Example 31 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class CreateOperations method doRemoteCreate.

private CreateResponse doRemoteCreate(CreateRequest createRequest) {
    HashSet<ProcessingDetails> exceptions = new HashSet<>();
    Map<String, Serializable> properties = new HashMap<>();
    List<CatalogStore> stores = opsCatStoreSupport.getCatalogStoresForRequest(createRequest, exceptions);
    for (CatalogStore store : stores) {
        try {
            if (!store.isAvailable()) {
                exceptions.add(new ProcessingDetailsImpl(store.getId(), null, "CatalogStore is not available"));
            } else {
                CreateResponse response = store.create(createRequest);
                properties.put(store.getId(), new ArrayList<>(response.getCreatedMetacards()));
            }
        } catch (IngestException e) {
            INGEST_LOGGER.error("Error creating metacards for CatalogStore {}", store.getId(), e);
            exceptions.add(new ProcessingDetailsImpl(store.getId(), e));
        }
    }
    return new CreateResponseImpl(createRequest, properties, createRequest.getMetacards(), exceptions);
}
Also used : Serializable(java.io.Serializable) HashMap(java.util.HashMap) CreateResponse(ddf.catalog.operation.CreateResponse) ProcessingDetailsImpl(ddf.catalog.operation.impl.ProcessingDetailsImpl) ProcessingDetails(ddf.catalog.operation.ProcessingDetails) CatalogStore(ddf.catalog.source.CatalogStore) InternalIngestException(ddf.catalog.source.InternalIngestException) IngestException(ddf.catalog.source.IngestException) HashSet(java.util.HashSet) CreateResponseImpl(ddf.catalog.operation.impl.CreateResponseImpl)

Example 32 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class CatalogFrameworkImplTest method testProviderUnavailableCreate.

// End testing CatalogFramework
// Test negative use-cases (expected errors)
/**
     * Tests that the framework properly throws a catalog exception when the local provider is not
     * available for create.
     *
     * @throws SourceUnavailableException
     */
@Test(expected = SourceUnavailableException.class)
public void testProviderUnavailableCreate() throws SourceUnavailableException {
    MockEventProcessor eventAdmin = new MockEventProcessor();
    MockMemoryProvider provider = new MockMemoryProvider("Provider", "Provider", "v1.0", "DDF", new HashSet<ContentType>(), false, null);
    CatalogFramework framework = createDummyCatalogFramework(provider, storageProvider, eventAdmin, false);
    List<Metacard> metacards = new ArrayList<Metacard>();
    MetacardImpl newCard = new MetacardImpl();
    newCard.setId(null);
    metacards.add(newCard);
    CreateRequest create = new CreateRequestImpl(metacards);
    // expected to throw exception due to catalog provider being unavailable
    try {
        framework.create(create);
    } catch (IngestException e) {
        fail();
    }
}
Also used : Metacard(ddf.catalog.data.Metacard) ContentType(ddf.catalog.data.ContentType) CreateRequest(ddf.catalog.operation.CreateRequest) CatalogFramework(ddf.catalog.CatalogFramework) ArrayList(java.util.ArrayList) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) IngestException(ddf.catalog.source.IngestException) MetacardImpl(ddf.catalog.data.impl.MetacardImpl) Test(org.junit.Test)

Example 33 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class MetacardApplication method patchMetacards.

protected UpdateResponse patchMetacards(List<MetacardChanges> metacardChanges) throws SourceUnavailableException, IngestException, FederationException, UnsupportedQueryException {
    Set<String> changedIds = metacardChanges.stream().flatMap(mc -> mc.getIds().stream()).collect(Collectors.toSet());
    Map<String, Result> results = util.getMetacards(changedIds, "*");
    for (MetacardChanges changeset : metacardChanges) {
        for (AttributeChange attributeChange : changeset.getAttributes()) {
            for (String id : changeset.getIds()) {
                List<String> values = attributeChange.getValues();
                Metacard result = results.get(id).getMetacard();
                Function<Serializable, Serializable> mapFunc = Function.identity();
                if (isChangeTypeDate(attributeChange, result)) {
                    mapFunc = mapFunc.andThen(util::parseDate);
                }
                result.setAttribute(new AttributeImpl(attributeChange.getAttribute(), values.stream().filter(Objects::nonNull).map(mapFunc).collect(Collectors.toList())));
            }
        }
    }
    List<Metacard> changedMetacards = results.values().stream().map(Result::getMetacard).collect(Collectors.toList());
    return catalogFramework.update(new UpdateRequestImpl(changedMetacards.stream().map(Metacard::getId).collect(Collectors.toList()).toArray(new String[0]), changedMetacards));
}
Also used : CONTENT_TYPE(javax.ws.rs.core.HttpHeaders.CONTENT_TYPE) Spark.delete(spark.Spark.delete) Spark.patch(spark.Spark.patch) Date(java.util.Date) UpdateRequestImpl(ddf.catalog.operation.impl.UpdateRequestImpl) UnsupportedQueryException(ddf.catalog.source.UnsupportedQueryException) Spark.exception(spark.Spark.exception) LoggerFactory(org.slf4j.LoggerFactory) UpdateStorageRequestImpl(ddf.catalog.content.operation.impl.UpdateStorageRequestImpl) SparkApplication(spark.servlet.SparkApplication) AttributeType(ddf.catalog.data.AttributeType) ExperimentalEnumerationExtractor(org.codice.ddf.catalog.ui.metacard.enumerations.ExperimentalEnumerationExtractor) MetacardVersion(ddf.catalog.core.versioning.MetacardVersion) HistoryResponse(org.codice.ddf.catalog.ui.metacard.history.HistoryResponse) Map(java.util.Map) Action(ddf.catalog.core.versioning.MetacardVersion.Action) DeleteRequestImpl(ddf.catalog.operation.impl.DeleteRequestImpl) Spark.put(spark.Spark.put) SubjectUtils(ddf.security.SubjectUtils) APPLICATION_JSON(javax.ws.rs.core.MediaType.APPLICATION_JSON) MetacardVersionImpl(ddf.catalog.core.versioning.impl.MetacardVersionImpl) ImmutableSet(com.google.common.collect.ImmutableSet) AttributeDescriptor(ddf.catalog.data.AttributeDescriptor) StringUtils.isEmpty(org.apache.commons.lang.StringUtils.isEmpty) ImmutableMap(com.google.common.collect.ImmutableMap) WorkspaceAttributes(org.codice.ddf.catalog.ui.metacard.workspace.WorkspaceAttributes) ResourceRequestById(ddf.catalog.operation.impl.ResourceRequestById) Spark.after(spark.Spark.after) DeletedMetacard(ddf.catalog.core.versioning.DeletedMetacard) Set(java.util.Set) Instant(java.time.Instant) Collectors(java.util.stream.Collectors) NotFoundException(javax.ws.rs.NotFoundException) MetacardType(ddf.catalog.data.MetacardType) Serializable(java.io.Serializable) ResourceNotFoundException(ddf.catalog.resource.ResourceNotFoundException) Objects(java.util.Objects) QueryResponse(ddf.catalog.operation.QueryResponse) List(java.util.List) Validator(org.codice.ddf.catalog.ui.metacard.validation.Validator) SubscriptionsPersistentStore(org.codice.ddf.catalog.ui.query.monitor.api.SubscriptionsPersistentStore) Optional(java.util.Optional) MetacardChanges(org.codice.ddf.catalog.ui.metacard.edit.MetacardChanges) UpdateResponse(ddf.catalog.operation.UpdateResponse) TEXT_PLAIN(javax.ws.rs.core.MediaType.TEXT_PLAIN) SecurityUtils(org.apache.shiro.SecurityUtils) ResourceResponse(ddf.catalog.operation.ResourceResponse) QueryRequestImpl(ddf.catalog.operation.impl.QueryRequestImpl) Spark.get(spark.Spark.get) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl) FilterBuilder(ddf.catalog.filter.FilterBuilder) SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) CatalogFramework(ddf.catalog.CatalogFramework) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) Spark.post(spark.Spark.post) HashMap(java.util.HashMap) Callable(java.util.concurrent.Callable) DeleteResponse(ddf.catalog.operation.DeleteResponse) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) SortBy(org.opengis.filter.sort.SortBy) EndpointUtil(org.codice.ddf.catalog.ui.util.EndpointUtil) ContentItem(ddf.catalog.content.data.ContentItem) Metacard(ddf.catalog.data.Metacard) AttributeChange(org.codice.ddf.catalog.ui.metacard.edit.AttributeChange) ByteSource(com.google.common.io.ByteSource) Result(ddf.catalog.data.Result) WorkspaceTransformer(org.codice.ddf.catalog.ui.metacard.workspace.WorkspaceTransformer) Associated(org.codice.ddf.catalog.ui.metacard.associations.Associated) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) QueryImpl(ddf.catalog.operation.impl.QueryImpl) Logger(org.slf4j.Logger) Security(org.codice.ddf.security.common.Security) JsonFactory(org.boon.json.JsonFactory) IngestException(ddf.catalog.source.IngestException) IOException(java.io.IOException) Subject(ddf.security.Subject) FederationException(ddf.catalog.federation.FederationException) TimeUnit(java.util.concurrent.TimeUnit) ResourceNotSupportedException(ddf.catalog.resource.ResourceNotSupportedException) ExecutionException(org.apache.shiro.subject.ExecutionException) Filter(org.opengis.filter.Filter) Comparator(java.util.Comparator) Collections(java.util.Collections) InputStream(java.io.InputStream) Serializable(java.io.Serializable) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) MetacardChanges(org.codice.ddf.catalog.ui.metacard.edit.MetacardChanges) Result(ddf.catalog.data.Result) DeletedMetacard(ddf.catalog.core.versioning.DeletedMetacard) Metacard(ddf.catalog.data.Metacard) AttributeChange(org.codice.ddf.catalog.ui.metacard.edit.AttributeChange) Objects(java.util.Objects) UpdateRequestImpl(ddf.catalog.operation.impl.UpdateRequestImpl)

Example 34 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class ExceptionsTest method testIngestException.

@Test
public void testIngestException() {
    IngestException ie = new IngestException();
    assertNotNull(ie);
    ie = new IngestException(msg);
    assertEquals(ie.getMessage(), msg);
    ie = new IngestException(testCause);
    assertEquals(ie.getCause(), testCause);
    ie = new IngestException(msg, testCause);
    assertEquals(ie.getMessage(), msg);
    assertEquals(ie.getCause(), testCause);
}
Also used : IngestException(ddf.catalog.source.IngestException) Test(org.junit.Test)

Example 35 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class SolrCatalogProvider method deleteListOfMetacards.

private void deleteListOfMetacards(List<Metacard> deletedMetacards, List<? extends Serializable> identifiers, String attributeName) throws IngestException {
    String fieldName = attributeName + SchemaFields.TEXT_SUFFIX;
    SolrDocumentList docs = getSolrDocumentList(identifiers, fieldName);
    createListOfDeletedMetacards(deletedMetacards, docs);
    try {
        // the assumption is if something was deleted, it should be gone
        // right away, such as expired data, etc.
        // so we force the commit
        client.deleteByIds(fieldName, identifiers, true);
    } catch (SolrServerException | IOException e) {
        LOGGER.info("Failed to delete metacards by ID(s).", e);
        throw new IngestException(COULD_NOT_COMPLETE_DELETE_REQUEST_MESSAGE);
    }
}
Also used : SolrServerException(org.apache.solr.client.solrj.SolrServerException) IngestException(ddf.catalog.source.IngestException) SolrDocumentList(org.apache.solr.common.SolrDocumentList) IOException(java.io.IOException)

Aggregations

IngestException (ddf.catalog.source.IngestException)56 Metacard (ddf.catalog.data.Metacard)33 ArrayList (java.util.ArrayList)32 HashMap (java.util.HashMap)21 SourceUnavailableException (ddf.catalog.source.SourceUnavailableException)20 CreateResponse (ddf.catalog.operation.CreateResponse)19 CreateRequestImpl (ddf.catalog.operation.impl.CreateRequestImpl)17 IOException (java.io.IOException)17 UnsupportedQueryException (ddf.catalog.source.UnsupportedQueryException)15 Test (org.junit.Test)15 QueryRequestImpl (ddf.catalog.operation.impl.QueryRequestImpl)13 InternalIngestException (ddf.catalog.source.InternalIngestException)13 QueryResponse (ddf.catalog.operation.QueryResponse)11 Serializable (java.io.Serializable)11 List (java.util.List)11 Map (java.util.Map)11 FederationException (ddf.catalog.federation.FederationException)10 CreateRequest (ddf.catalog.operation.CreateRequest)10 DeleteResponse (ddf.catalog.operation.DeleteResponse)10 QueryImpl (ddf.catalog.operation.impl.QueryImpl)10