Search in sources :

Example 16 with ProcessingDetailsImpl

use of ddf.catalog.operation.impl.ProcessingDetailsImpl in project ddf by codice.

the class SortedFederationStrategyTest method testResponseWithProcessingDetails.

@Test
public void testResponseWithProcessingDetails() throws UnsupportedQueryException {
    Source mockSource = mock(Source.class);
    String testSource = "test source";
    when(mockSource.getId()).thenReturn(testSource);
    SourceResponse mockResponseWithProcessingDetails = mock(SourceResponse.class);
    ProcessingDetails processingDetailsForNullPointer = new ProcessingDetailsImpl(testSource, null, "Look! A null pointer!");
    ProcessingDetails processingDetailsForUnsupportedQuery = new ProcessingDetailsImpl(testSource, null, "We do not support this query.");
    Set<ProcessingDetails> processingDetails = new HashSet<>();
    processingDetails.add(processingDetailsForNullPointer);
    processingDetails.add(processingDetailsForUnsupportedQuery);
    doReturn(processingDetails).when(mockResponseWithProcessingDetails).getProcessingDetails();
    QueryRequest fedQueryRequest = new QueryRequestImpl(mockQuery, properties);
    when(mockSource.query(any(QueryRequest.class))).thenReturn(mockResponseWithProcessingDetails);
    SourceResponse federatedResponse = strategy.federate(Collections.singletonList(mockSource), fedQueryRequest);
    assertThat(federatedResponse.getProcessingDetails(), containsInAnyOrder(processingDetailsForNullPointer, processingDetailsForUnsupportedQuery));
}
Also used : ProcessingDetails(ddf.catalog.operation.ProcessingDetails) SourceResponse(ddf.catalog.operation.SourceResponse) QueryRequest(ddf.catalog.operation.QueryRequest) QueryRequestImpl(ddf.catalog.operation.impl.QueryRequestImpl) Source(ddf.catalog.source.Source) ProcessingDetailsImpl(ddf.catalog.operation.impl.ProcessingDetailsImpl) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 17 with ProcessingDetailsImpl

use of ddf.catalog.operation.impl.ProcessingDetailsImpl in project ddf by codice.

the class UpdateOperations method doRemoteUpdate.

private UpdateResponse doRemoteUpdate(UpdateRequest updateRequest) {
    HashSet<ProcessingDetails> exceptions = new HashSet<>();
    Map<String, Serializable> properties = new HashMap<>();
    List<CatalogStore> stores = opsCatStoreSupport.getCatalogStoresForRequest(updateRequest, exceptions);
    List<Update> updates = new ArrayList<>();
    for (CatalogStore store : stores) {
        try {
            if (!store.isAvailable()) {
                exceptions.add(new ProcessingDetailsImpl(store.getId(), null, "CatalogStore is not available"));
            } else {
                UpdateResponse response = store.update(updateRequest);
                properties.put(store.getId(), new ArrayList<>(response.getUpdatedMetacards()));
                updates = response.getUpdatedMetacards();
            }
        } catch (IngestException e) {
            INGEST_LOGGER.error("Error updating metacards for CatalogStore {}", store.getId(), e);
            exceptions.add(new ProcessingDetailsImpl(store.getId(), e));
        }
    }
    return new UpdateResponseImpl(updateRequest, properties, updates, exceptions);
}
Also used : Serializable(java.io.Serializable) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Update(ddf.catalog.operation.Update) ProcessingDetailsImpl(ddf.catalog.operation.impl.ProcessingDetailsImpl) ProcessingDetails(ddf.catalog.operation.ProcessingDetails) CatalogStore(ddf.catalog.source.CatalogStore) UpdateResponse(ddf.catalog.operation.UpdateResponse) UpdateResponseImpl(ddf.catalog.operation.impl.UpdateResponseImpl) InternalIngestException(ddf.catalog.source.InternalIngestException) IngestException(ddf.catalog.source.IngestException) HashSet(java.util.HashSet)

Example 18 with ProcessingDetailsImpl

use of ddf.catalog.operation.impl.ProcessingDetailsImpl in project ddf by codice.

the class AbstractCswStore method create.

@Override
public CreateResponse create(CreateRequest createRequest) throws IngestException {
    Map<String, Serializable> properties = new HashMap<>();
    validateOperation();
    Subject subject = (Subject) createRequest.getPropertyValue(SecurityConstants.SECURITY_SUBJECT);
    Csw csw = factory.getClientForSubject(subject);
    CswTransactionRequest transactionRequest = getTransactionRequest();
    List<Metacard> metacards = createRequest.getMetacards();
    List<String> metacardIds = metacards.stream().map(Metacard::getId).collect(Collectors.toList());
    List<Metacard> createdMetacards = new ArrayList<>();
    List<Filter> createdMetacardFilters;
    HashSet<ProcessingDetails> errors = new HashSet<>();
    String insertTypeName = schemaTransformerManager.getTransformerIdForSchema(cswSourceConfiguration.getOutputSchema());
    if (insertTypeName == null) {
        throw new IngestException("Could not find transformer for output schema " + cswSourceConfiguration.getOutputSchema());
    }
    transactionRequest.getInsertActions().add(new InsertActionImpl(insertTypeName, null, metacards));
    try {
        TransactionResponseType response = csw.transaction(transactionRequest);
        Set<String> processedIds = new HashSet<>();
        // dive down into the response to get the created ID's. We need these so we can query
        // the source again to get the created metacards and put them in the result
        createdMetacardFilters = response.getInsertResult().stream().map(InsertResultType::getBriefRecord).flatMap(Collection::stream).map(BriefRecordType::getIdentifier).flatMap(Collection::stream).map(JAXBElement::getValue).map(SimpleLiteral::getContent).flatMap(Collection::stream).map(id -> {
            processedIds.add(id);
            return filterBuilder.attribute(Core.ID).is().equalTo().text(id);
        }).collect(Collectors.toList());
        metacardIds.removeAll(processedIds);
        errors.addAll(metacardIds.stream().map(id -> new ProcessingDetailsImpl(id, null, "Failed to create metacard")).collect(Collectors.toList()));
    } catch (CswException e) {
        throw new IngestException("Csw Transaction Failed : ", e);
    }
    try {
        createdMetacards = transactionQuery(createdMetacardFilters, subject);
    } catch (UnsupportedQueryException e) {
        errors.add(new ProcessingDetailsImpl(this.getId(), e, "Failed to retrieve newly created metacards"));
    }
    return new CreateResponseImpl(createRequest, properties, createdMetacards, errors);
}
Also used : Serializable(java.io.Serializable) HashMap(java.util.HashMap) UnsupportedQueryException(ddf.catalog.source.UnsupportedQueryException) ArrayList(java.util.ArrayList) CswException(org.codice.ddf.spatial.ogc.csw.catalog.common.CswException) ProcessingDetailsImpl(ddf.catalog.operation.impl.ProcessingDetailsImpl) TransactionResponseType(net.opengis.cat.csw.v_2_0_2.TransactionResponseType) ProcessingDetails(ddf.catalog.operation.ProcessingDetails) IngestException(ddf.catalog.source.IngestException) SimpleLiteral(net.opengis.cat.csw.v_2_0_2.dc.elements.SimpleLiteral) HashSet(java.util.HashSet) Csw(org.codice.ddf.spatial.ogc.csw.catalog.common.Csw) Subject(ddf.security.Subject) Metacard(ddf.catalog.data.Metacard) InsertActionImpl(org.codice.ddf.spatial.ogc.csw.catalog.common.transaction.InsertActionImpl) Filter(org.opengis.filter.Filter) CswTransactionRequest(org.codice.ddf.spatial.ogc.csw.catalog.common.transaction.CswTransactionRequest) Collection(java.util.Collection) CreateResponseImpl(ddf.catalog.operation.impl.CreateResponseImpl)

Example 19 with ProcessingDetailsImpl

use of ddf.catalog.operation.impl.ProcessingDetailsImpl in project ddf by codice.

the class QueryResultCachePlugin method cloneResponse.

private SourceResponse cloneResponse(SourceResponse sourceResponse) {
    List<Result> clonedResults = sourceResponse.getResults().stream().map(Result::getMetacard).map(m -> new MetacardImpl(m, m.getMetacardType())).map(ResultImpl::new).collect(Collectors.toList());
    Set<ProcessingDetails> processingDetails = new HashSet<>();
    if (clonedResults.size() > 0) {
        String sourceId = clonedResults.get(0).getMetacard().getSourceId();
        processingDetails = sourceResponse.getProcessingDetails().stream().map(sourceDetails -> new ProcessingDetailsImpl(sourceDetails, sourceId)).collect(Collectors.toSet());
    }
    return new QueryResponseImpl(sourceResponse.getRequest(), clonedResults, true, sourceResponse.getHits(), sourceResponse.getProperties(), processingDetails);
}
Also used : ProcessingDetails(ddf.catalog.operation.ProcessingDetails) QueryResponseImpl(ddf.catalog.operation.impl.QueryResponseImpl) MetacardImpl(ddf.catalog.data.impl.MetacardImpl) ProcessingDetailsImpl(ddf.catalog.operation.impl.ProcessingDetailsImpl) Result(ddf.catalog.data.Result) HashSet(java.util.HashSet)

Example 20 with ProcessingDetailsImpl

use of ddf.catalog.operation.impl.ProcessingDetailsImpl in project ddf by codice.

the class SolrCacheSource method query.

@Override
public SourceResponse query(QueryRequest request) throws UnsupportedQueryException {
    if (request.isEnterprise()) {
        LOGGER.debug("Ignoring enterprise query to the cache.");
        return new QueryResponseImpl(request, new ArrayList<>(), true, 0);
    }
    final QueryResponseImpl queryResponse = new QueryResponseImpl(request);
    try {
        SourceResponse result = cache.query(cacheQueryFactory.getQueryRequestWithSourcesFilter(request));
        queryResponse.setHits(result.getHits());
        queryResponse.setProperties(result.getProperties());
        queryResponse.addResults(result.getResults(), true);
    } catch (UnsupportedQueryException e) {
        queryResponse.getProcessingDetails().add(new ProcessingDetailsImpl(getId(), e));
        queryResponse.closeResultQueue();
    }
    return queryResponse;
}
Also used : QueryResponseImpl(ddf.catalog.operation.impl.QueryResponseImpl) SourceResponse(ddf.catalog.operation.SourceResponse) UnsupportedQueryException(ddf.catalog.source.UnsupportedQueryException) ProcessingDetailsImpl(ddf.catalog.operation.impl.ProcessingDetailsImpl)

Aggregations

ProcessingDetailsImpl (ddf.catalog.operation.impl.ProcessingDetailsImpl)37 Test (org.junit.Test)19 UnsupportedQueryException (ddf.catalog.source.UnsupportedQueryException)18 ProcessingDetails (ddf.catalog.operation.ProcessingDetails)13 QueryResponseImpl (ddf.catalog.operation.impl.QueryResponseImpl)8 IngestException (ddf.catalog.source.IngestException)8 HashSet (java.util.HashSet)8 QueryRequest (ddf.catalog.operation.QueryRequest)7 SourceResponse (ddf.catalog.operation.SourceResponse)6 Serializable (java.io.Serializable)6 HashMap (java.util.HashMap)6 Tag (io.micrometer.core.instrument.Tag)5 ArrayList (java.util.ArrayList)5 Metacard (ddf.catalog.data.Metacard)4 QueryResponse (ddf.catalog.operation.QueryResponse)4 QueryRequestImpl (ddf.catalog.operation.impl.QueryRequestImpl)4 CatalogStore (ddf.catalog.source.CatalogStore)3 SourceUnavailableException (ddf.catalog.source.SourceUnavailableException)3 ExecutionException (java.util.concurrent.ExecutionException)3 TimeoutException (java.util.concurrent.TimeoutException)3