Search in sources :

Example 1 with Base64OutputStream

use of org.apache.commons.codec.binary.Base64OutputStream in project nifi by apache.

the class Base64EncodeContent method onTrigger.

@Override
public void onTrigger(final ProcessContext context, final ProcessSession session) {
    FlowFile flowFile = session.get();
    if (flowFile == null) {
        return;
    }
    final ComponentLog logger = getLogger();
    boolean encode = context.getProperty(MODE).getValue().equalsIgnoreCase(ENCODE_MODE);
    try {
        final StopWatch stopWatch = new StopWatch(true);
        if (encode) {
            flowFile = session.write(flowFile, new StreamCallback() {

                @Override
                public void process(InputStream in, OutputStream out) throws IOException {
                    try (Base64OutputStream bos = new Base64OutputStream(out)) {
                        int len = -1;
                        byte[] buf = new byte[8192];
                        while ((len = in.read(buf)) > 0) {
                            bos.write(buf, 0, len);
                        }
                        bos.flush();
                    }
                }
            });
        } else {
            flowFile = session.write(flowFile, new StreamCallback() {

                @Override
                public void process(InputStream in, OutputStream out) throws IOException {
                    try (Base64InputStream bis = new Base64InputStream(new ValidatingBase64InputStream(in))) {
                        int len = -1;
                        byte[] buf = new byte[8192];
                        while ((len = bis.read(buf)) > 0) {
                            out.write(buf, 0, len);
                        }
                        out.flush();
                    }
                }
            });
        }
        logger.info("Successfully {} {}", new Object[] { encode ? "encoded" : "decoded", flowFile });
        session.getProvenanceReporter().modifyContent(flowFile, stopWatch.getElapsed(TimeUnit.MILLISECONDS));
        session.transfer(flowFile, REL_SUCCESS);
    } catch (ProcessException e) {
        logger.error("Failed to {} {} due to {}", new Object[] { encode ? "encode" : "decode", flowFile, e });
        session.transfer(flowFile, REL_FAILURE);
    }
}
Also used : FlowFile(org.apache.nifi.flowfile.FlowFile) ValidatingBase64InputStream(org.apache.nifi.processors.standard.util.ValidatingBase64InputStream) Base64InputStream(org.apache.commons.codec.binary.Base64InputStream) InputStream(java.io.InputStream) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) OutputStream(java.io.OutputStream) ValidatingBase64InputStream(org.apache.nifi.processors.standard.util.ValidatingBase64InputStream) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) ComponentLog(org.apache.nifi.logging.ComponentLog) StreamCallback(org.apache.nifi.processor.io.StreamCallback) StopWatch(org.apache.nifi.util.StopWatch) ProcessException(org.apache.nifi.processor.exception.ProcessException) ValidatingBase64InputStream(org.apache.nifi.processors.standard.util.ValidatingBase64InputStream) Base64InputStream(org.apache.commons.codec.binary.Base64InputStream)

Example 2 with Base64OutputStream

use of org.apache.commons.codec.binary.Base64OutputStream in project xwiki-platform by xwiki.

the class XMLWriter method writeBase64.

/**
 * Writes the <code>{@link Element}</code>, including its <code>{@link
 * org.dom4j.Attribute}</code>s, using the <code>{@link InputStream}</code> encoded in Base64 for its content.
 *
 * @param element <code>{@link Element}</code> to output.
 * @param is <code>{@link InputStream}</code> that will be fully read and encoded in Base64 into the element
 *            content.
 * @throws IOException a problem occurs during reading or writing.
 */
public void writeBase64(Element element, InputStream is) throws IOException {
    writeOpen(element);
    flush();
    try (Base64OutputStream base64 = new Base64OutputStream(new CloseShieldOutputStream(this.out))) {
        IOUtils.copy(is, base64);
    }
    writeClose(element);
}
Also used : Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) CloseShieldOutputStream(org.apache.commons.io.output.CloseShieldOutputStream)

Example 3 with Base64OutputStream

use of org.apache.commons.codec.binary.Base64OutputStream in project incubator-gobblin by apache.

the class EncodingBenchmark method write1KRecordsBase64Only.

@Benchmark
public byte[] write1KRecordsBase64Only(EncodingBenchmarkState state) throws IOException {
    ByteArrayOutputStream sink = new ByteArrayOutputStream();
    OutputStream os = new Base64OutputStream(sink);
    os.write(state.OneKBytes);
    os.close();
    return sink.toByteArray();
}
Also used : OutputStream(java.io.OutputStream) ByteArrayOutputStream(java.io.ByteArrayOutputStream) CipherOutputStream(javax.crypto.CipherOutputStream) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) Benchmark(org.openjdk.jmh.annotations.Benchmark)

Example 4 with Base64OutputStream

use of org.apache.commons.codec.binary.Base64OutputStream in project santuario-java by apache.

the class TransformBase64Decode method transform.

@Override
public void transform(XMLSecEvent xmlSecEvent) throws XMLStreamException {
    int eventType = xmlSecEvent.getEventType();
    switch(eventType) {
        case XMLStreamConstants.CHARACTERS:
            if (getOutputStream() != null) {
                // encoding shouldn't matter here, because the data is Base64 encoded and is therefore in the ASCII range.
                try {
                    getOutputStream().write(xmlSecEvent.asCharacters().getData().getBytes());
                } catch (IOException e) {
                    throw new XMLStreamException(e);
                }
            } else {
                // we have a child transformer
                if (childOutputMethod == null) {
                    final XMLSecurityConstants.TransformMethod preferredChildTransformMethod = getTransformer().getPreferredTransformMethod(XMLSecurityConstants.TransformMethod.XMLSecEvent);
                    switch(preferredChildTransformMethod) {
                        case XMLSecEvent:
                            {
                                childOutputMethod = new ChildOutputMethod() {

                                    private UnsyncByteArrayOutputStream byteArrayOutputStream;

                                    private Base64OutputStream base64OutputStream;

                                    @Override
                                    public void transform(Object object) throws XMLStreamException {
                                        if (base64OutputStream == null) {
                                            byteArrayOutputStream = new UnsyncByteArrayOutputStream();
                                            base64OutputStream = new Base64OutputStream(byteArrayOutputStream, false);
                                        }
                                        try {
                                            base64OutputStream.write((byte[]) object);
                                        } catch (IOException e) {
                                            throw new XMLStreamException(e);
                                        }
                                    }

                                    @Override
                                    public void doFinal() throws XMLStreamException {
                                        try {
                                            base64OutputStream.close();
                                        } catch (IOException e) {
                                            throw new XMLStreamException(e);
                                        }
                                        try (InputStream is = new UnsyncByteArrayInputStream(byteArrayOutputStream.toByteArray())) {
                                            XMLEventReaderInputProcessor xmlEventReaderInputProcessor = new XMLEventReaderInputProcessor(null, getXmlInputFactory().createXMLStreamReader(is));
                                            XMLSecEvent xmlSecEvent;
                                            do {
                                                xmlSecEvent = xmlEventReaderInputProcessor.processNextEvent(null);
                                                getTransformer().transform(xmlSecEvent);
                                            } while (xmlSecEvent.getEventType() != XMLStreamConstants.END_DOCUMENT);
                                        } catch (XMLSecurityException | IOException e) {
                                            throw new XMLStreamException(e);
                                        }
                                        getTransformer().doFinal();
                                    }
                                };
                                break;
                            }
                        case InputStream:
                            {
                                childOutputMethod = new ChildOutputMethod() {

                                    private UnsyncByteArrayOutputStream byteArrayOutputStream;

                                    private Base64OutputStream base64OutputStream;

                                    @Override
                                    public void transform(Object object) throws XMLStreamException {
                                        if (base64OutputStream == null) {
                                            byteArrayOutputStream = new UnsyncByteArrayOutputStream();
                                            base64OutputStream = new Base64OutputStream(byteArrayOutputStream, false);
                                        }
                                        try {
                                            base64OutputStream.write((byte[]) object);
                                        } catch (IOException e) {
                                            throw new XMLStreamException(e);
                                        }
                                    }

                                    @Override
                                    public void doFinal() throws XMLStreamException {
                                        try {
                                            base64OutputStream.close();
                                        } catch (IOException e) {
                                            throw new XMLStreamException(e);
                                        }
                                        try (InputStream is = new UnsyncByteArrayInputStream(byteArrayOutputStream.toByteArray())) {
                                            getTransformer().transform(is);
                                            getTransformer().doFinal();
                                        } catch (IOException ex) {
                                            throw new XMLStreamException(ex);
                                        }
                                    }
                                };
                                break;
                            }
                    }
                }
                childOutputMethod.transform(xmlSecEvent.asCharacters().getData().getBytes());
            }
            break;
    }
}
Also used : XMLSecurityConstants(org.apache.xml.security.stax.ext.XMLSecurityConstants) UnsyncByteArrayOutputStream(org.apache.xml.security.utils.UnsyncByteArrayOutputStream) UnsyncByteArrayInputStream(org.apache.xml.security.utils.UnsyncByteArrayInputStream) Base64InputStream(org.apache.commons.codec.binary.Base64InputStream) XMLEventReaderInputProcessor(org.apache.xml.security.stax.impl.processor.input.XMLEventReaderInputProcessor) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) XMLSecurityException(org.apache.xml.security.exceptions.XMLSecurityException) XMLSecEvent(org.apache.xml.security.stax.ext.stax.XMLSecEvent) XMLStreamException(javax.xml.stream.XMLStreamException) UnsyncByteArrayInputStream(org.apache.xml.security.utils.UnsyncByteArrayInputStream)

Example 5 with Base64OutputStream

use of org.apache.commons.codec.binary.Base64OutputStream in project azure-tools-for-java by Microsoft.

the class JobUtils method uploadFileToHDFSBase.

public static String uploadFileToHDFSBase(IClusterDetail selectedClusterDetail, String buildJarPath, @Nullable Observer<SimpleImmutableEntry<MessageInfoType, String>> legacyLogSubject, @Nullable Observer<SparkLogLine> newLogSubject) throws HDIException {
    ctrlInfo(legacyLogSubject, newLogSubject, String.format("Get target jar from %s.", buildJarPath));
    final File srcJarFile = new File(buildJarPath);
    final URI destUri = URI.create(String.format("/SparkSubmission/%s/%s", getFormatPathByDate(), srcJarFile.getName()));
    final String username = selectedClusterDetail.getHttpUserName();
    final String password = selectedClusterDetail.getHttpPassword();
    final String sessionName = "Helper session to upload " + destUri.toString();
    final URI livyUri = selectedClusterDetail instanceof LivyCluster ? URI.create(((LivyCluster) selectedClusterDetail).getLivyConnectionUrl()) : URI.create(selectedClusterDetail.getConnectionUrl());
    ctrlInfo(legacyLogSubject, newLogSubject, "Create Spark helper interactive session...");
    try {
        return Observable.using(() -> new SparkSession(sessionName, livyUri, username, password), SparkSession::create, SparkSession::close).map(sparkSession -> {
            sparkSession.getCtrlSubject().subscribe(logLine -> ctrlInfo(legacyLogSubject, newLogSubject, logLine.getRawLog()), err -> ctrlError(legacyLogSubject, newLogSubject, err), () -> {
            });
            ClusterFileBase64BufferedOutputStream clusterFileBase64Out = new ClusterFileBase64BufferedOutputStream(sparkSession, destUri);
            Base64OutputStream base64Enc = new Base64OutputStream(clusterFileBase64Out, true);
            InputStream inFile;
            try {
                inFile = new BufferedInputStream(new FileInputStream(srcJarFile));
                ctrlInfo(legacyLogSubject, newLogSubject, String.format("Uploading %s...", srcJarFile));
                IOUtils.copy(inFile, base64Enc);
                inFile.close();
                base64Enc.close();
            } catch (FileNotFoundException fnfEx) {
                throw propagate(new HDIException(String.format("Source file %s not found.", srcJarFile), fnfEx));
            } catch (IOException ioEx) {
                throw propagate(new HDIException(String.format("Failed to upload file %s.", destUri), ioEx));
            }
            ctrlInfo(legacyLogSubject, newLogSubject, String.format("Uploaded to %s.", destUri));
            return destUri.toString();
        }).toBlocking().single();
    } catch (final NoSuchElementException ignored) {
        // The cause exception will be thrown inside
        throw new HDIException("Failed to upload file to HDFS (Should Not Reach).");
    }
}
Also used : ClusterFileBase64BufferedOutputStream(com.microsoft.azure.hdinsight.sdk.io.spark.ClusterFileBase64BufferedOutputStream) SparkSession(com.microsoft.azure.hdinsight.sdk.common.livy.interactive.SparkSession) LivyCluster(com.microsoft.azure.hdinsight.sdk.cluster.LivyCluster) HDIException(com.microsoft.azure.hdinsight.sdk.common.HDIException) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream) URI(java.net.URI)

Aggregations

Base64OutputStream (org.apache.commons.codec.binary.Base64OutputStream)14 ByteArrayOutputStream (java.io.ByteArrayOutputStream)6 InputStream (java.io.InputStream)4 OutputStream (java.io.OutputStream)4 GZIPOutputStream (java.util.zip.GZIPOutputStream)4 Base64InputStream (org.apache.commons.codec.binary.Base64InputStream)4 CloseShieldOutputStream (org.apache.commons.io.output.CloseShieldOutputStream)2 LivyCluster (com.microsoft.azure.hdinsight.sdk.cluster.LivyCluster)1 HDIException (com.microsoft.azure.hdinsight.sdk.common.HDIException)1 SparkSession (com.microsoft.azure.hdinsight.sdk.common.livy.interactive.SparkSession)1 ClusterFileBase64BufferedOutputStream (com.microsoft.azure.hdinsight.sdk.io.spark.ClusterFileBase64BufferedOutputStream)1 Ack (io.socket.client.Ack)1 BufferedOutputStream (java.io.BufferedOutputStream)1 Closeable (java.io.Closeable)1 DataOutputStream (java.io.DataOutputStream)1 File (java.io.File)1 FileInputStream (java.io.FileInputStream)1 FileOutputStream (java.io.FileOutputStream)1 FilterOutputStream (java.io.FilterOutputStream)1 IOException (java.io.IOException)1