Search in sources :

Example 16 with PluginExecutionException

use of ddf.catalog.plugin.PluginExecutionException in project ddf by codice.

the class InMemoryProcessingFramework method submitUpdate.

@Override
public void submitUpdate(ProcessRequest<ProcessUpdateItem> input) {
    if (postProcessPlugins == null || postProcessPlugins.isEmpty()) {
        LOGGER.debug("postProcessPlugins is empty. Not starting post process thread");
    } else {
        threadPool.submit(() -> {
            ProcessRequest<ProcessUpdateItem> request = input;
            for (PostProcessPlugin plugin : postProcessPlugins) {
                try {
                    request = plugin.processUpdate(request);
                } catch (PluginExecutionException e) {
                    LOGGER.debug("Unable to process update request through plugin: {}", plugin.getClass().getCanonicalName(), e);
                }
            }
            storeProcessRequest(request);
            closeInputStream(request);
        });
    }
}
Also used : PostProcessPlugin(org.codice.ddf.catalog.async.plugin.api.internal.PostProcessPlugin) ProcessUpdateItem(org.codice.ddf.catalog.async.data.api.internal.ProcessUpdateItem) PluginExecutionException(ddf.catalog.plugin.PluginExecutionException)

Example 17 with PluginExecutionException

use of ddf.catalog.plugin.PluginExecutionException in project ddf by codice.

the class CachingFederationStrategy method sourceFederate.

private QueryResponse sourceFederate(List<Source> sources, final QueryRequest queryRequest) {
    if (LOGGER.isDebugEnabled()) {
        for (Source source : sources) {
            if (source != null) {
                LOGGER.debug("source to query: {}", source.getId());
            }
        }
    }
    Query originalQuery = queryRequest.getQuery();
    int offset = originalQuery.getStartIndex();
    final int pageSize = originalQuery.getPageSize();
    // limit offset to max value
    if (offset > this.maxStartIndex) {
        offset = this.maxStartIndex;
    }
    final QueryResponseImpl queryResponseQueue = new QueryResponseImpl(queryRequest, null);
    Map<Future<SourceResponse>, QueryRequest> futures = new HashMap<>();
    Query modifiedQuery = getModifiedQuery(originalQuery, sources.size(), offset, pageSize);
    QueryRequest modifiedQueryRequest = new QueryRequestImpl(modifiedQuery, queryRequest.isEnterprise(), queryRequest.getSourceIds(), queryRequest.getProperties());
    CompletionService<SourceResponse> queryCompletion = new ExecutorCompletionService<>(queryExecutorService);
    // Do NOT call source.isAvailable() when checking sources
    for (final Source source : sources) {
        if (source != null) {
            if (!futuresContainsSource(source, futures)) {
                LOGGER.debug("running query on source: {}", source.getId());
                QueryRequest sourceQueryRequest = new QueryRequestImpl(modifiedQuery, queryRequest.isEnterprise(), Collections.singleton(source.getId()), new HashMap<>(queryRequest.getProperties()));
                try {
                    for (PreFederatedQueryPlugin service : preQuery) {
                        try {
                            sourceQueryRequest = service.process(source, sourceQueryRequest);
                        } catch (PluginExecutionException e) {
                            LOGGER.info("Error executing PreFederatedQueryPlugin", e);
                        }
                    }
                } catch (StopProcessingException e) {
                    LOGGER.info("Plugin stopped processing", e);
                }
                if (source instanceof CatalogProvider && SystemInfo.getSiteName().equals(source.getId())) {
                    // TODO RAP 12 Jul 16: DDF-2294 - Extract into a new PreFederatedQueryPlugin
                    sourceQueryRequest = validationQueryFactory.getQueryRequestWithValidationFilter(sourceQueryRequest, showErrors, showWarnings);
                }
                futures.put(queryCompletion.submit(new CallableSourceResponse(source, sourceQueryRequest)), sourceQueryRequest);
            } else {
                LOGGER.info("Duplicate source found with name {}. Ignoring second one.", source.getId());
            }
        }
    }
    QueryResponseImpl offsetResults = null;
    // OffsetResultHandler does.
    if (offset > 1 && sources.size() > 1) {
        offsetResults = new QueryResponseImpl(queryRequest, null);
        queryExecutorService.submit(new OffsetResultHandler(queryResponseQueue, offsetResults, pageSize, offset));
    }
    queryExecutorService.submit(sortedQueryMonitorFactory.createMonitor(queryCompletion, futures, queryResponseQueue, modifiedQueryRequest, postQuery));
    QueryResponse queryResponse;
    if (offset > 1 && sources.size() > 1) {
        queryResponse = offsetResults;
        LOGGER.debug("returning offsetResults");
    } else {
        queryResponse = queryResponseQueue;
        LOGGER.debug("returning returnResults: {}", queryResponse);
    }
    LOGGER.debug("returning Query Results: {}", queryResponse);
    return queryResponse;
}
Also used : Query(ddf.catalog.operation.Query) QueryRequest(ddf.catalog.operation.QueryRequest) SourceResponse(ddf.catalog.operation.SourceResponse) PreFederatedQueryPlugin(ddf.catalog.plugin.PreFederatedQueryPlugin) HashMap(java.util.HashMap) ExecutorCompletionService(java.util.concurrent.ExecutorCompletionService) StopProcessingException(ddf.catalog.plugin.StopProcessingException) Source(ddf.catalog.source.Source) QueryResponseImpl(ddf.catalog.operation.impl.QueryResponseImpl) CatalogProvider(ddf.catalog.source.CatalogProvider) QueryRequestImpl(ddf.catalog.operation.impl.QueryRequestImpl) QueryResponse(ddf.catalog.operation.QueryResponse) Future(java.util.concurrent.Future) PluginExecutionException(ddf.catalog.plugin.PluginExecutionException)

Example 18 with PluginExecutionException

use of ddf.catalog.plugin.PluginExecutionException in project ddf by codice.

the class IdentificationPlugin method process.

/**
     * For registry metacards verifies the update should take place by checking that the update
     * metacard is at least as up to date as the existing one. Also updates the tags, identifiers,
     * and transient attributes of the updated metacard.
     *
     * @param input the {@link UpdateRequest} to process
     * @return
     * @throws PluginExecutionException
     * @throws StopProcessingException
     */
@Override
public UpdateRequest process(UpdateRequest input) throws PluginExecutionException, StopProcessingException {
    if (!Requests.isLocal(input)) {
        return input;
    }
    OperationTransaction operationTransaction = (OperationTransaction) input.getProperties().get(Constants.OPERATION_TRANSACTION_KEY);
    List<Metacard> previousMetacards = operationTransaction.getPreviousStateMetacards();
    Map<String, Metacard> previousMetacardsMap = previousMetacards.stream().filter(e -> RegistryUtility.isRegistryMetacard(e) || RegistryUtility.isInternalRegistryMetacard(e)).collect(Collectors.toMap(RegistryUtility::getRegistryId, Function.identity()));
    List<Map.Entry<Serializable, Metacard>> entriesToRemove = new ArrayList<>();
    List<Map.Entry<Serializable, Metacard>> registryUpdates = input.getUpdates().stream().filter(e -> RegistryUtility.isRegistryMetacard(e.getValue())).collect(Collectors.toList());
    for (Map.Entry<Serializable, Metacard> entry : registryUpdates) {
        Metacard updateMetacard = entry.getValue();
        Metacard existingMetacard = previousMetacardsMap.get(RegistryUtility.getRegistryId(updateMetacard));
        if (existingMetacard == null) {
            continue;
        }
        if (updateMetacard.getMetadata() != null && !updateMetacard.getModifiedDate().before(existingMetacard.getModifiedDate())) {
            updateMetacard.setAttribute(new AttributeImpl(Metacard.ID, existingMetacard.getId()));
            copyTransientAttributes(updateMetacard, existingMetacard);
            updateTags(updateMetacard);
            if (isInternal(updateMetacard)) {
                updateMetacard.setAttribute(existingMetacard.getAttribute(RegistryObjectMetacardType.REMOTE_METACARD_ID));
                updateMetacard.setAttribute(existingMetacard.getAttribute(RegistryObjectMetacardType.REMOTE_REGISTRY_ID));
            }
            updateIdentifiers(updateMetacard, false);
        } else {
            entriesToRemove.add(entry);
        }
    }
    input.getUpdates().removeAll(entriesToRemove);
    return input;
}
Also used : PreIngestPlugin(ddf.catalog.plugin.PreIngestPlugin) CreateRequest(ddf.catalog.operation.CreateRequest) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) OperationTransaction(ddf.catalog.operation.OperationTransaction) Function(java.util.function.Function) ArrayList(java.util.ArrayList) PluginExecutionException(ddf.catalog.plugin.PluginExecutionException) MetacardMarshaller(org.codice.ddf.registry.schemabindings.helper.MetacardMarshaller) Requests(ddf.catalog.util.impl.Requests) Constants(ddf.catalog.Constants) Metacard(ddf.catalog.data.Metacard) Map(java.util.Map) UpdateRequest(ddf.catalog.operation.UpdateRequest) RegistryConstants(org.codice.ddf.registry.common.RegistryConstants) RegistryPackageType(oasis.names.tc.ebxml_regrep.xsd.rim._3.RegistryPackageType) RegistryUtility(org.codice.ddf.registry.common.metacard.RegistryUtility) StopProcessingException(ddf.catalog.plugin.StopProcessingException) ParserException(org.codice.ddf.parser.ParserException) Collectors(java.util.stream.Collectors) Serializable(java.io.Serializable) DeleteRequest(ddf.catalog.operation.DeleteRequest) List(java.util.List) Attribute(ddf.catalog.data.Attribute) ExternalIdentifierType(oasis.names.tc.ebxml_regrep.xsd.rim._3.ExternalIdentifierType) RegistryObjectMetacardType(org.codice.ddf.registry.common.metacard.RegistryObjectMetacardType) UuidGenerator(org.codice.ddf.platform.util.uuidgenerator.UuidGenerator) Serializable(java.io.Serializable) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) ArrayList(java.util.ArrayList) OperationTransaction(ddf.catalog.operation.OperationTransaction) Metacard(ddf.catalog.data.Metacard) Map(java.util.Map)

Aggregations

PluginExecutionException (ddf.catalog.plugin.PluginExecutionException)18 Metacard (ddf.catalog.data.Metacard)7 QueryRequest (ddf.catalog.operation.QueryRequest)5 QueryRequestImpl (ddf.catalog.operation.impl.QueryRequestImpl)5 StopProcessingException (ddf.catalog.plugin.StopProcessingException)4 IOException (java.io.IOException)4 Query (ddf.catalog.operation.Query)3 QueryResponse (ddf.catalog.operation.QueryResponse)3 PreFederatedQueryPlugin (ddf.catalog.plugin.PreFederatedQueryPlugin)3 Source (ddf.catalog.source.Source)3 HashMap (java.util.HashMap)3 BinaryContent (ddf.catalog.data.BinaryContent)2 Result (ddf.catalog.data.Result)2 AttributeImpl (ddf.catalog.data.impl.AttributeImpl)2 CopyFilterDelegate (ddf.catalog.filter.delegate.CopyFilterDelegate)2 SourceResponse (ddf.catalog.operation.SourceResponse)2 Update (ddf.catalog.operation.Update)2 UpdateRequest (ddf.catalog.operation.UpdateRequest)2 QueryImpl (ddf.catalog.operation.impl.QueryImpl)2 QueryResponseImpl (ddf.catalog.operation.impl.QueryResponseImpl)2