Search in sources :

Example 51 with CatalogTransformerException

use of ddf.catalog.transform.CatalogTransformerException in project ddf by codice.

the class TestRestEndpoint method getMockInputTransformer.

protected InputTransformer getMockInputTransformer() {
    InputTransformer inputTransformer = mock(InputTransformer.class);
    Metacard generatedMetacard = getSimpleMetacard();
    try {
        when(inputTransformer.transform(isA(InputStream.class))).thenReturn(generatedMetacard);
        when(inputTransformer.transform(isA(InputStream.class), isA(String.class))).thenReturn(generatedMetacard);
    } catch (IOException e) {
        LOGGER.debug("Exception occurred during test", e);
    } catch (CatalogTransformerException e) {
        LOGGER.debug("Exception occurred during test", e);
    }
    return inputTransformer;
}
Also used : Metacard(ddf.catalog.data.Metacard) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) CatalogTransformerException(ddf.catalog.transform.CatalogTransformerException) Matchers.anyString(org.mockito.Matchers.anyString) IOException(java.io.IOException) InputTransformer(ddf.catalog.transform.InputTransformer)

Example 52 with CatalogTransformerException

use of ddf.catalog.transform.CatalogTransformerException in project ddf by codice.

the class PptxInputTransformer method transformLogic.

/**
     * This is a three step process. First, create a FileBackedOutputStream because we need to
     * consume the stream twice. Once for the injected inputTransformer and once for Apache POI.
     * Next, extract the metadata with the injected input transformer. And last, use Apache POI
     * to create the thumbnail.
     *
     * @param input
     * @param id
     * @return
     * @throws IOException
     * @throws CatalogTransformerException
     */
private Metacard transformLogic(InputStream input, String id) throws IOException, CatalogTransformerException {
    try (TemporaryFileBackedOutputStream fileBackedOutputStream = new TemporaryFileBackedOutputStream()) {
        try {
            int c = IOUtils.copy(input, fileBackedOutputStream);
            LOGGER.debug("copied {} bytes from input stream to file backed output stream", c);
        } catch (IOException e) {
            throw new CatalogTransformerException("Could not copy bytes of content message.", e);
        }
        Metacard metacard = extractInitialMetadata(fileBackedOutputStream.asByteSource().openStream());
        extractThumbnail(metacard, fileBackedOutputStream.asByteSource().openStream());
        if (!usePptTitleAsTitle) {
            // Allow the metacard title to be set as the filename
            metacard.setAttribute(new AttributeImpl(Core.TITLE, (Serializable) null));
        }
        return metacard;
    }
}
Also used : Metacard(ddf.catalog.data.Metacard) Serializable(java.io.Serializable) TemporaryFileBackedOutputStream(org.codice.ddf.platform.util.TemporaryFileBackedOutputStream) AttributeImpl(ddf.catalog.data.impl.AttributeImpl) CatalogTransformerException(ddf.catalog.transform.CatalogTransformerException) IOException(java.io.IOException)

Example 53 with CatalogTransformerException

use of ddf.catalog.transform.CatalogTransformerException in project ddf by codice.

the class PdfInputTransformer method transformWithExtractors.

private Metacard transformWithExtractors(InputStream input, String id) throws IOException, CatalogTransformerException {
    try (TemporaryFileBackedOutputStream fbos = new TemporaryFileBackedOutputStream()) {
        try {
            IOUtils.copy(input, fbos);
        } catch (IOException e) {
            throw new CatalogTransformerException("Could not copy bytes of content message.", e);
        }
        String plainText = null;
        try (InputStream isCopy = fbos.asByteSource().openStream()) {
            Parser parser = new AutoDetectParser();
            ContentHandler contentHandler = new ToTextContentHandler();
            TikaMetadataExtractor tikaMetadataExtractor = new TikaMetadataExtractor(parser, contentHandler);
            tikaMetadataExtractor.parseMetadata(isCopy, new ParseContext());
            plainText = contentHandler.toString();
        } catch (CatalogTransformerException e) {
            LOGGER.warn("Cannot extract metadata from pdf", e);
        }
        try (InputStream isCopy = fbos.asByteSource().openStream();
            PDDocument pdfDocument = pdDocumentGenerator.apply(isCopy)) {
            return transformPdf(id, pdfDocument, plainText);
        } catch (InvalidPasswordException e) {
            LOGGER.debug("Cannot transform encrypted pdf", e);
            return initializeMetacard(id);
        }
    }
}
Also used : TemporaryFileBackedOutputStream(org.codice.ddf.platform.util.TemporaryFileBackedOutputStream) InputStream(java.io.InputStream) CatalogTransformerException(ddf.catalog.transform.CatalogTransformerException) IOException(java.io.IOException) ContentHandler(org.xml.sax.ContentHandler) ToTextContentHandler(org.apache.tika.sax.ToTextContentHandler) Parser(org.apache.tika.parser.Parser) AutoDetectParser(org.apache.tika.parser.AutoDetectParser) ToTextContentHandler(org.apache.tika.sax.ToTextContentHandler) TikaMetadataExtractor(ddf.catalog.transformer.common.tika.TikaMetadataExtractor) PDDocument(org.apache.pdfbox.pdmodel.PDDocument) ParseContext(org.apache.tika.parser.ParseContext) AutoDetectParser(org.apache.tika.parser.AutoDetectParser) InvalidPasswordException(org.apache.pdfbox.pdmodel.encryption.InvalidPasswordException)

Example 54 with CatalogTransformerException

use of ddf.catalog.transform.CatalogTransformerException in project ddf by codice.

the class TestOpenSearchSource method getMockInputTransformer.

protected InputTransformer getMockInputTransformer() {
    InputTransformer inputTransformer = mock(InputTransformer.class);
    Metacard generatedMetacard = getSimpleMetacard();
    try {
        when(inputTransformer.transform(isA(InputStream.class))).thenReturn(generatedMetacard);
        when(inputTransformer.transform(isA(InputStream.class), isA(String.class))).thenReturn(generatedMetacard);
    } catch (IOException e) {
        fail();
    } catch (CatalogTransformerException e) {
        fail();
    }
    return inputTransformer;
}
Also used : Metacard(ddf.catalog.data.Metacard) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) CatalogTransformerException(ddf.catalog.transform.CatalogTransformerException) Matchers.containsString(org.hamcrest.Matchers.containsString) IOException(java.io.IOException) InputTransformer(ddf.catalog.transform.InputTransformer)

Example 55 with CatalogTransformerException

use of ddf.catalog.transform.CatalogTransformerException in project ddf by codice.

the class TestCswQueryResponseTransformer method testMarshalAcknowledgementWithFailedTransforms.

@Test
public void testMarshalAcknowledgementWithFailedTransforms() throws WebApplicationException, IOException, JAXBException, CatalogTransformerException {
    GetRecordsType query = new GetRecordsType();
    query.setResultType(ResultType.RESULTS);
    query.setMaxRecords(BigInteger.valueOf(6));
    query.setStartPosition(BigInteger.valueOf(0));
    SourceResponse sourceResponse = createSourceResponse(query, 6);
    Map<String, Serializable> args = new HashMap<>();
    args.put(CswConstants.RESULT_TYPE_PARAMETER, ResultType.RESULTS);
    args.put(CswConstants.GET_RECORDS, query);
    PrintWriter printWriter = getSimplePrintWriter();
    MetacardTransformer mockMetacardTransformer = mock(MetacardTransformer.class);
    final AtomicLong atomicLong = new AtomicLong(0);
    when(mockMetacardTransformer.transform(any(Metacard.class), anyMap())).then(invocationOnMock -> {
        if (atomicLong.incrementAndGet() == 2) {
            throw new CatalogTransformerException("");
        }
        Metacard metacard = (Metacard) invocationOnMock.getArguments()[0];
        BinaryContentImpl bci = new BinaryContentImpl(IOUtils.toInputStream(metacard.getId() + ","), new MimeType("application/xml"));
        return bci;
    });
    when(mockPrintWriterProvider.build((Class<Metacard>) notNull())).thenReturn(printWriter);
    when(mockTransformerManager.getTransformerBySchema(anyString())).thenReturn(mockMetacardTransformer);
    CswQueryResponseTransformer cswQueryResponseTransformer = new CswQueryResponseTransformer(mockTransformerManager, mockPrintWriterProvider);
    cswQueryResponseTransformer.init();
    BinaryContent content = cswQueryResponseTransformer.transform(sourceResponse, args);
    cswQueryResponseTransformer.destroy();
    String xml = new String(content.getByteArray());
    assertThat(xml, containsString(CswQueryResponseTransformer.NUMBER_OF_RECORDS_MATCHED_ATTRIBUTE + " 6"));
    assertThat(xml, containsString(CswQueryResponseTransformer.NUMBER_OF_RECORDS_RETURNED_ATTRIBUTE + " 5"));
    assertThat(xml, containsString(CswQueryResponseTransformer.NEXT_RECORD_ATTRIBUTE + " 0"));
}
Also used : Serializable(java.io.Serializable) MetacardTransformer(ddf.catalog.transform.MetacardTransformer) SourceResponse(ddf.catalog.operation.SourceResponse) HashMap(java.util.HashMap) GetRecordsType(net.opengis.cat.csw.v_2_0_2.GetRecordsType) CatalogTransformerException(ddf.catalog.transform.CatalogTransformerException) Matchers.containsString(org.hamcrest.Matchers.containsString) Matchers.anyString(org.mockito.Matchers.anyString) BinaryContentImpl(ddf.catalog.data.impl.BinaryContentImpl) BinaryContent(ddf.catalog.data.BinaryContent) MimeType(javax.activation.MimeType) AtomicLong(java.util.concurrent.atomic.AtomicLong) Metacard(ddf.catalog.data.Metacard) PrintWriter(ddf.catalog.transformer.api.PrintWriter) Test(org.junit.Test)

Aggregations

CatalogTransformerException (ddf.catalog.transform.CatalogTransformerException)112 IOException (java.io.IOException)53 Metacard (ddf.catalog.data.Metacard)44 InputStream (java.io.InputStream)40 ByteArrayInputStream (java.io.ByteArrayInputStream)29 BinaryContent (ddf.catalog.data.BinaryContent)25 InputTransformer (ddf.catalog.transform.InputTransformer)21 Serializable (java.io.Serializable)21 HashMap (java.util.HashMap)21 Result (ddf.catalog.data.Result)16 BinaryContentImpl (ddf.catalog.data.impl.BinaryContentImpl)15 TemporaryFileBackedOutputStream (org.codice.ddf.platform.util.TemporaryFileBackedOutputStream)14 ArrayList (java.util.ArrayList)13 MetacardImpl (ddf.catalog.data.impl.MetacardImpl)12 Test (org.junit.Test)12 AttributeImpl (ddf.catalog.data.impl.AttributeImpl)10 MimeType (javax.activation.MimeType)10 SourceResponse (ddf.catalog.operation.SourceResponse)9 MetacardTransformer (ddf.catalog.transform.MetacardTransformer)8 List (java.util.List)8