Examples with IngestException ddf.catalog.source.IngestException used on opensource projects

Example 46 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class ExceptionsTest method testIngestException.

@Test
public void testIngestException() {
    IngestException ie = new IngestException();
    assertNotNull(ie);
    ie = new IngestException(msg);
    assertEquals(ie.getMessage(), msg);
    ie = new IngestException(testCause);
    assertEquals(ie.getCause(), testCause);
    ie = new IngestException(msg, testCause);
    assertEquals(ie.getMessage(), msg);
    assertEquals(ie.getCause(), testCause);
}

Example 47 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class SolrCatalogProvider method deleteListOfMetacards.

private void deleteListOfMetacards(List<Metacard> deletedMetacards, List<? extends Serializable> identifiers, String attributeName) throws IngestException {
    String fieldName = attributeName + SchemaFields.TEXT_SUFFIX;
    SolrDocumentList docs = getSolrDocumentList(identifiers, fieldName);
    createListOfDeletedMetacards(deletedMetacards, docs);
    try {
        // the assumption is if something was deleted, it should be gone
        // right away, such as expired data, etc.
        // so we force the commit
        client.deleteByIds(fieldName, identifiers, true);
    } catch (SolrServerException | IOException e) {
        LOGGER.info("Failed to delete metacards by ID(s).", e);
        throw new IngestException(COULD_NOT_COMPLETE_DELETE_REQUEST_MESSAGE);
    }
}

Example 48 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class SolrCatalogProvider method getSolrDocumentList.

private SolrDocumentList getSolrDocumentList(List<? extends Serializable> identifierPaged, String fieldName) throws IngestException {
    SolrQuery query = new SolrQuery(client.getIdentifierQuery(fieldName, identifierPaged));
    query.setRows(identifierPaged.size());
    QueryResponse solrResponse;
    try {
        solrResponse = solr.query(query, METHOD.POST);
    } catch (SolrServerException | IOException e) {
        LOGGER.info("Failed to get list of Solr documents for delete.", e);
        throw new IngestException(COULD_NOT_COMPLETE_DELETE_REQUEST_MESSAGE);
    }
    return solrResponse.getResults();
}

Example 49 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class ExportCommand method doDelete.

private void doDelete(List<ExportItem> exportedItems, List<ExportItem> exportedContentItems) {
    Instant start;
    console.println("Starting delete");
    start = Instant.now();
    for (ExportItem exportedContentItem : exportedContentItems) {
        try {
            DeleteStorageRequestImpl deleteRequest = new DeleteStorageRequestImpl(Collections.singletonList(new IdAndUriMetacard(exportedContentItem.getId(), exportedContentItem.getResourceUri())), exportedContentItem.getId(), Collections.emptyMap());
            storageProvider.delete(deleteRequest);
            storageProvider.commit(deleteRequest);
        } catch (StorageException e) {
            printErrorMessage("Could not delete content for metacard: " + exportedContentItem.toString());
        }
    }
    for (ExportItem exported : exportedItems) {
        try {
            catalogProvider.delete(new DeleteRequestImpl(exported.getId()));
        } catch (IngestException e) {
            printErrorMessage("Could not delete metacard: " + exported.toString());
        }
    }
    // delete items from cache
    try {
        getCacheProxy().removeById(exportedItems.stream().map(ExportItem::getId).collect(Collectors.toList()).toArray(new String[exportedItems.size()]));
    } catch (Exception e) {
        LOGGER.warn("Could not delete all exported items from cache (Results will eventually expire)", e);
    }
    console.println("Metacards and Content deleted in: " + getFormattedDuration(start));
    console.println("Number of metacards deleted: " + exportedItems.size());
    console.println("Number of content deleted: " + exportedContentItems.size());
}

Example 50 with IngestException

use of ddf.catalog.source.IngestException in project ddf by codice.

the class IngestCommand method addFileToQueue.

private void addFileToQueue(ArrayBlockingQueue<Metacard> metacardQueue, long start, File file) {
    if (file.isHidden()) {
        fileCount.incrementAndGet();
        ignoreCount.incrementAndGet();
        return;
    }
    String extension = "." + FilenameUtils.getExtension(file.getName());
    if (ignoreList != null && (ignoreList.contains(extension) || ignoreList.contains(file.getName()))) {
        ignoreCount.incrementAndGet();
        printProgressAndFlush(start, fileCount.get(), ingestCount.get() + ignoreCount.get());
        return;
    }
    Metacard result = null;
    try {
        result = readMetacard(file);
    } catch (IngestException e) {
        logIngestException(e, file);
        if (failedIngestDirectory != null) {
            moveToFailedIngestDirectory(file);
        }
        printErrorMessage(String.format("Failed to ingest file [%s].", file.getAbsolutePath()));
        INGEST_LOGGER.warn("Failed to ingest file [{}].", file.getAbsolutePath());
    }
    if (result != null) {
        putMetacardOnQueue(metacardQueue, result);
    }
}