Search in sources :

Example 56 with CreateRequestImpl

use of ddf.catalog.operation.impl.CreateRequestImpl in project ddf by codice.

the class CatalogComponentFrameworkTest method testCreateWithSingleMetacard.

@Test
public /**
 * Operation: CREATE Body contains: Metacard
 */
void testCreateWithSingleMetacard() throws Exception {
    resetMocks();
    // Setup expectations to verify
    final MockEndpoint mockVerifierEndpoint = getMockEndpoint("mock:result");
    mockVerifierEndpoint.expectedMessageCount(1);
    final List<Metacard> metacards = new ArrayList<Metacard>();
    metacards.add(metacard1);
    // Mock catalog framework
    final CreateRequest createRequest = new CreateRequestImpl(metacards);
    final CreateResponse createResponse = new CreateResponseImpl(createRequest, new HashMap(), metacards);
    when(catalogFramework.create(any(CreateRequest.class))).thenReturn(createResponse);
    // Exercise the route with a CREATE operation
    template.sendBodyAndHeader("direct:sampleInput", metacard1, "Operation", "CREATE");
    // Verify that the number of metacards in the exchange after the records
    // is identical to the input
    assertListSize(mockVerifierEndpoint.getExchanges(), 1);
    final Exchange exchange = mockVerifierEndpoint.getExchanges().get(0);
    final List<Metacard> cardsCreated = (List<Metacard>) exchange.getIn().getBody();
    assertListSize(cardsCreated, 1);
    mockVerifierEndpoint.assertIsSatisfied();
}
Also used : Exchange(org.apache.camel.Exchange) Metacard(ddf.catalog.data.Metacard) MockEndpoint(org.apache.camel.component.mock.MockEndpoint) HashMap(java.util.HashMap) CreateRequest(ddf.catalog.operation.CreateRequest) CreateResponse(ddf.catalog.operation.CreateResponse) ArrayList(java.util.ArrayList) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) ArrayList(java.util.ArrayList) List(java.util.List) CreateResponseImpl(ddf.catalog.operation.impl.CreateResponseImpl) Test(org.junit.Test)

Example 57 with CreateRequestImpl

use of ddf.catalog.operation.impl.CreateRequestImpl in project ddf by codice.

the class FrameworkProducer method create.

/**
 * Creates metacard(s) in the catalog using the Catalog Framework.
 *
 * @param exchange The {@link org.apache.camel.Exchange} can contain a {@link
 *     org.apache.camel.Message} with a body of type {@link java.util.List} of Metacard or a
 *     single Metacard.
 * @throws ddf.catalog.source.SourceUnavailableException
 * @throws ddf.catalog.source.IngestException
 * @throws ddf.camel.component.catalog.framework.FrameworkProducerException
 */
private void create(final Exchange exchange) throws SourceUnavailableException, IngestException, FrameworkProducerException {
    CreateResponse createResponse = null;
    // read in data
    final List<Metacard> metacardsToBeCreated = readBodyDataAsMetacards(exchange);
    if (!validateList(metacardsToBeCreated, Metacard.class)) {
        processCatalogResponse(createResponse, exchange);
        throw new FrameworkProducerException("Validation of Metacard list failed");
    }
    LOGGER.debug("Validation of Metacard list passed...");
    final CreateRequest createRequest = new CreateRequestImpl(metacardsToBeCreated);
    int expectedNumberOfCreatedMetacards = metacardsToBeCreated.size();
    if (expectedNumberOfCreatedMetacards < 1) {
        LOGGER.debug("Empty list of Metacards...nothing to process");
        processCatalogResponse(createResponse, exchange);
        return;
    }
    LOGGER.debug("Making CREATE call to Catalog Framework...");
    createResponse = catalogFramework.create(createRequest);
    if (createResponse == null) {
        LOGGER.debug("CreateResponse is null from catalog framework");
        processCatalogResponse(createResponse, exchange);
        return;
    }
    final List<Metacard> createdMetacards = createResponse.getCreatedMetacards();
    if (createdMetacards == null) {
        LOGGER.debug("CreateResponse returned null metacards list");
        processCatalogResponse(createResponse, exchange);
        return;
    }
    final int numberOfCreatedMetacards = createdMetacards.size();
    if (numberOfCreatedMetacards != expectedNumberOfCreatedMetacards) {
        LOGGER.debug("Expected {} metacards created but only {} were successfully created", expectedNumberOfCreatedMetacards, numberOfCreatedMetacards);
        processCatalogResponse(createResponse, exchange);
        return;
    }
    LOGGER.debug("Created {} metacards", numberOfCreatedMetacards);
    processCatalogResponse(createResponse, exchange);
}
Also used : Metacard(ddf.catalog.data.Metacard) CreateResponse(ddf.catalog.operation.CreateResponse) CreateRequest(ddf.catalog.operation.CreateRequest) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) Endpoint(org.apache.camel.Endpoint)

Example 58 with CreateRequestImpl

use of ddf.catalog.operation.impl.CreateRequestImpl in project ddf by codice.

the class ImportCommand method executeWithSubject.

@Override
protected final Object executeWithSubject() throws Exception {
    int metacards = 0;
    int content = 0;
    int derivedContent = 0;
    File file = initImportFile(importFile);
    InputTransformer transformer = getServiceByFilter(InputTransformer.class, String.format("(%s=%s)", "id", DEFAULT_TRANSFORMER_ID)).orElseThrow(() -> new CatalogCommandRuntimeException("Could not get " + DEFAULT_TRANSFORMER_ID + " input transformer"));
    if (unsafe) {
        if (!force) {
            String input = session.readLine("This will import data with no check to see if data is modified/corrupt. Do you wish to continue? (y/N) ", null);
            if (!input.matches("^[yY][eE]?[sS]?$")) {
                console.println("ABORTED IMPORT.");
                return null;
            }
        }
        securityLogger.audit("Skipping validation check of imported data. There are no " + "guarantees of integrity or authenticity of the imported data." + "File being imported: {}", importFile);
    } else {
        if (StringUtils.isBlank(signatureFile)) {
            String message = "A signature file must be provided with import data";
            console.println(message);
            throw new CatalogCommandRuntimeException(message);
        }
        String alias = AccessController.doPrivileged((PrivilegedAction<String>) () -> System.getProperty("org.codice.ddf.system.hostname"));
        try (FileInputStream fileIs = new FileInputStream(file);
            FileInputStream sigFileIs = new FileInputStream(signatureFile)) {
            if (verifier == null) {
                verifier = new DigitalSignature(security);
            }
            if (!verifier.verifyDigitalSignature(fileIs, sigFileIs, alias)) {
                throw new CatalogCommandRuntimeException("The provided data could not be verified");
            }
        }
    }
    securityLogger.audit("Called catalog:import command on the file: {}", importFile);
    console.println("Importing file");
    Instant start = Instant.now();
    try (InputStream fis = new FileInputStream(file);
        ZipInputStream zipInputStream = new ZipInputStream(fis)) {
        ZipEntry entry = zipInputStream.getNextEntry();
        while (entry != null) {
            String filename = entry.getName();
            if (filename.startsWith("META-INF")) {
                entry = zipInputStream.getNextEntry();
                continue;
            }
            String[] pathParts = filename.split("\\" + File.separator);
            if (pathParts.length < 5) {
                console.println("Entry is not valid! " + filename);
                entry = zipInputStream.getNextEntry();
                continue;
            }
            String id = pathParts[ID];
            String type = pathParts[TYPE];
            switch(type) {
                case "metacard":
                    {
                        String metacardName = pathParts[NAME];
                        Metacard metacard = null;
                        try {
                            metacard = transformer.transform(new UncloseableBufferedInputStreamWrapper(zipInputStream), id);
                        } catch (IOException | CatalogTransformerException e) {
                            LOGGER.debug("Could not transform metacard: {}", LogSanitizer.sanitize(id));
                            entry = zipInputStream.getNextEntry();
                            continue;
                        }
                        metacard = applyInjectors(metacard, attributeInjectors);
                        catalogProvider.create(new CreateRequestImpl(metacard));
                        metacards++;
                        break;
                    }
                case "content":
                    {
                        content++;
                        String contentFilename = pathParts[NAME];
                        ContentItem contentItem = new ContentItemImpl(id, new ZipEntryByteSource(new UncloseableBufferedInputStreamWrapper(zipInputStream)), null, contentFilename, entry.getSize(), null);
                        CreateStorageRequestImpl createStorageRequest = new CreateStorageRequestImpl(Collections.singletonList(contentItem), id, new HashMap<>());
                        storageProvider.create(createStorageRequest);
                        storageProvider.commit(createStorageRequest);
                        break;
                    }
                case "derived":
                    {
                        derivedContent++;
                        String qualifier = pathParts[NAME];
                        String derivedContentName = pathParts[DERIVED_NAME];
                        ContentItem contentItem = new ContentItemImpl(id, qualifier, new ZipEntryByteSource(new UncloseableBufferedInputStreamWrapper(zipInputStream)), null, derivedContentName, entry.getSize(), null);
                        CreateStorageRequestImpl createStorageRequest = new CreateStorageRequestImpl(Collections.singletonList(contentItem), id, new HashMap<>());
                        storageProvider.create(createStorageRequest);
                        storageProvider.commit(createStorageRequest);
                        break;
                    }
                default:
                    {
                        LOGGER.debug("Cannot interpret type of {}", LogSanitizer.sanitize(type));
                    }
            }
            entry = zipInputStream.getNextEntry();
        }
    } catch (Exception e) {
        printErrorMessage(String.format("Exception while importing metacards (%s)%nFor more information set the log level to INFO (log:set INFO org.codice.ddf.commands.catalog) ", e.getMessage()));
        LOGGER.info("Exception while importing metacards", e);
        throw e;
    }
    console.println("File imported successfully. Imported in: " + getFormattedDuration(start));
    console.println("Number of metacards imported: " + metacards);
    console.println("Number of content imported: " + content);
    console.println("Number of derived content imported: " + derivedContent);
    return null;
}
Also used : HashMap(java.util.HashMap) BufferedInputStream(java.io.BufferedInputStream) ZipInputStream(java.util.zip.ZipInputStream) FileInputStream(java.io.FileInputStream) InputStream(java.io.InputStream) Instant(java.time.Instant) ZipEntry(java.util.zip.ZipEntry) DigitalSignature(org.codice.ddf.commands.util.DigitalSignature) CatalogCommandRuntimeException(org.codice.ddf.commands.util.CatalogCommandRuntimeException) InputTransformer(ddf.catalog.transform.InputTransformer) FileInputStream(java.io.FileInputStream) CatalogCommandRuntimeException(org.codice.ddf.commands.util.CatalogCommandRuntimeException) IOException(java.io.IOException) CatalogTransformerException(ddf.catalog.transform.CatalogTransformerException) ZipInputStream(java.util.zip.ZipInputStream) Metacard(ddf.catalog.data.Metacard) CreateStorageRequestImpl(ddf.catalog.content.operation.impl.CreateStorageRequestImpl) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) File(java.io.File) ContentItem(ddf.catalog.content.data.ContentItem) ContentItemImpl(ddf.catalog.content.data.impl.ContentItemImpl)

Example 59 with CreateRequestImpl

use of ddf.catalog.operation.impl.CreateRequestImpl in project ddf by codice.

the class OperationPluginTest method noPropertiesRequestTest.

@Test
public void noPropertiesRequestTest() throws Exception {
    CreateRequestImpl request = new CreateRequestImpl(new ArrayList<>(), null);
    plugin.processPreCreate(request);
}
Also used : CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) Test(org.junit.Test)

Example 60 with CreateRequestImpl

use of ddf.catalog.operation.impl.CreateRequestImpl in project ddf by codice.

the class OperationPluginTest method noSubjectRequestTest.

@Test(expected = StopProcessingException.class)
public void noSubjectRequestTest() throws Exception {
    Map<String, Serializable> properties = new HashMap<>();
    HashMap<String, Set<String>> perms = new HashMap<>();
    Set<String> roles = new HashSet<>();
    roles.add("A");
    perms.put("Roles", roles);
    properties.put(PolicyPlugin.OPERATION_SECURITY, perms);
    CreateRequestImpl request = new CreateRequestImpl(new ArrayList<>(), properties);
    plugin.processPreCreate(request);
}
Also used : Serializable(java.io.Serializable) HashSet(java.util.HashSet) Set(java.util.Set) HashMap(java.util.HashMap) CreateRequestImpl(ddf.catalog.operation.impl.CreateRequestImpl) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

CreateRequestImpl (ddf.catalog.operation.impl.CreateRequestImpl)80 Test (org.junit.Test)60 Metacard (ddf.catalog.data.Metacard)53 CreateResponse (ddf.catalog.operation.CreateResponse)44 CreateRequest (ddf.catalog.operation.CreateRequest)42 ArrayList (java.util.ArrayList)30 HashMap (java.util.HashMap)29 MetacardImpl (ddf.catalog.data.impl.MetacardImpl)26 Serializable (java.io.Serializable)22 CreateResponseImpl (ddf.catalog.operation.impl.CreateResponseImpl)19 IngestException (ddf.catalog.source.IngestException)19 QueryRequestImpl (ddf.catalog.operation.impl.QueryRequestImpl)17 SourceUnavailableException (ddf.catalog.source.SourceUnavailableException)17 InputStream (java.io.InputStream)17 HashSet (java.util.HashSet)16 QueryImpl (ddf.catalog.operation.impl.QueryImpl)15 Subject (ddf.security.Subject)15 QueryRequest (ddf.catalog.operation.QueryRequest)13 QueryResponse (ddf.catalog.operation.QueryResponse)13 UnsupportedQueryException (ddf.catalog.source.UnsupportedQueryException)12