Search in sources :

Example 16 with ByteArrayOutputStream

use of org.apache.commons.io.output.ByteArrayOutputStream in project zeppelin by apache.

the class PigInterpreter method interpret.

@Override
public InterpreterResult interpret(String cmd, InterpreterContext contextInterpreter) {
    // remember the origial stdout, because we will redirect stdout to capture
    // the pig dump output.
    PrintStream originalStdOut = System.out;
    ByteArrayOutputStream bytesOutput = new ByteArrayOutputStream();
    File tmpFile = null;
    try {
        pigServer.setJobName(createJobName(cmd, contextInterpreter));
        tmpFile = PigUtils.createTempPigScript(cmd);
        System.setOut(new PrintStream(bytesOutput));
        // each thread should its own ScriptState & PigStats
        ScriptState.start(pigServer.getPigContext().getExecutionEngine().instantiateScriptState());
        // reset PigStats, otherwise you may get the PigStats of last job in the same thread
        // because PigStats is ThreadLocal variable
        PigStats.start(pigServer.getPigContext().getExecutionEngine().instantiatePigStats());
        PigScriptListener scriptListener = new PigScriptListener();
        ScriptState.get().registerListener(scriptListener);
        listenerMap.put(contextInterpreter.getParagraphId(), scriptListener);
        pigServer.registerScript(tmpFile.getAbsolutePath());
    } catch (IOException e) {
        if (e instanceof FrontendException) {
            FrontendException fe = (FrontendException) e;
            if (!fe.getMessage().contains("Backend error :")) {
                // If the error message contains "Backend error :", that means the exception is from
                // backend.
                LOGGER.error("Fail to run pig script.", e);
                return new InterpreterResult(Code.ERROR, ExceptionUtils.getStackTrace(e));
            }
        }
        PigStats stats = PigStats.get();
        if (stats != null) {
            String errorMsg = PigUtils.extactJobStats(stats);
            if (errorMsg != null) {
                LOGGER.error("Fail to run pig script, " + errorMsg);
                return new InterpreterResult(Code.ERROR, errorMsg);
            }
        }
        LOGGER.error("Fail to run pig script.", e);
        return new InterpreterResult(Code.ERROR, ExceptionUtils.getStackTrace(e));
    } finally {
        System.setOut(originalStdOut);
        listenerMap.remove(contextInterpreter.getParagraphId());
        if (tmpFile != null) {
            tmpFile.delete();
        }
    }
    StringBuilder outputBuilder = new StringBuilder();
    PigStats stats = PigStats.get();
    if (stats != null && includeJobStats) {
        String jobStats = PigUtils.extactJobStats(stats);
        if (jobStats != null) {
            outputBuilder.append(jobStats);
        }
    }
    outputBuilder.append(bytesOutput.toString());
    return new InterpreterResult(Code.SUCCESS, outputBuilder.toString());
}
Also used : PrintStream(java.io.PrintStream) InterpreterResult(org.apache.zeppelin.interpreter.InterpreterResult) ByteArrayOutputStream(org.apache.commons.io.output.ByteArrayOutputStream) IOException(java.io.IOException) File(java.io.File) FrontendException(org.apache.pig.impl.logicalLayer.FrontendException)

Example 17 with ByteArrayOutputStream

use of org.apache.commons.io.output.ByteArrayOutputStream in project hive by apache.

the class KryoSerializer method serialize.

public static byte[] serialize(Object object) {
    ByteArrayOutputStream stream = new ByteArrayOutputStream();
    Output output = new Output(stream);
    Kryo kryo = SerializationUtilities.borrowKryo();
    kryo.setClassLoader(Thread.currentThread().getContextClassLoader());
    try {
        kryo.writeObject(output, object);
    } finally {
        SerializationUtilities.releaseKryo(kryo);
    }
    // close() also calls flush()
    output.close();
    return stream.toByteArray();
}
Also used : Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(org.apache.commons.io.output.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo)

Example 18 with ByteArrayOutputStream

use of org.apache.commons.io.output.ByteArrayOutputStream in project hudson-2.x by hudson.

the class ConsoleNote method encodeToBytes.

private ByteArrayOutputStream encodeToBytes() throws IOException {
    ByteArrayOutputStream buf = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(new GZIPOutputStream(buf));
    oos.writeObject(this);
    oos.close();
    ByteArrayOutputStream buf2 = new ByteArrayOutputStream();
    DataOutputStream dos = new DataOutputStream(new Base64OutputStream(buf2, true, -1, null));
    buf2.write(PREAMBLE);
    dos.writeInt(buf.size());
    buf.writeTo(dos);
    dos.close();
    buf2.write(POSTAMBLE);
    return buf2;
}
Also used : GZIPOutputStream(java.util.zip.GZIPOutputStream) DataOutputStream(java.io.DataOutputStream) ByteArrayOutputStream(org.apache.commons.io.output.ByteArrayOutputStream) ObjectOutputStream(java.io.ObjectOutputStream) Base64OutputStream(org.apache.commons.codec.binary.Base64OutputStream)

Example 19 with ByteArrayOutputStream

use of org.apache.commons.io.output.ByteArrayOutputStream in project hudson-2.x by hudson.

the class WindowsInstallerLink method doDoInstall.

/**
     * Performs installation.
     */
public void doDoInstall(StaplerRequest req, StaplerResponse rsp, @QueryParameter("dir") String _dir) throws IOException, ServletException {
    if (installationDir != null) {
        // installation already complete
        sendError("Installation is already complete", req, rsp);
        return;
    }
    if (!DotNet.isInstalled(2, 0)) {
        sendError(".NET Framework 2.0 or later is required for this feature", req, rsp);
        return;
    }
    Hudson.getInstance().checkPermission(Hudson.ADMINISTER);
    File dir = new File(_dir).getAbsoluteFile();
    dir.mkdirs();
    if (!dir.exists()) {
        sendError("Failed to create installation directory: " + dir, req, rsp);
        return;
    }
    try {
        // copy files over there
        copy(req, rsp, dir, getClass().getResource("/windows-service/hudson.exe"), "hudson.exe");
        copy(req, rsp, dir, getClass().getResource("/windows-service/hudson.xml"), "hudson.xml");
        if (!hudsonWar.getCanonicalFile().equals(new File(dir, "hudson.war").getCanonicalFile()))
            copy(req, rsp, dir, hudsonWar.toURI().toURL(), "hudson.war");
        // install as a service
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        StreamTaskListener task = new StreamTaskListener(baos);
        task.getLogger().println("Installing a service");
        int r = WindowsSlaveInstaller.runElevated(new File(dir, "hudson.exe"), "install", task, dir);
        if (r != 0) {
            sendError(baos.toString(), req, rsp);
            return;
        }
        // installation was successful
        installationDir = dir;
        rsp.sendRedirect(".");
    } catch (AbortException e) {
    // this exception is used as a signal to terminate processing. the error should have been already reported
    } catch (InterruptedException e) {
        throw new ServletException(e);
    }
}
Also used : ServletException(javax.servlet.ServletException) StreamTaskListener(hudson.util.StreamTaskListener) ByteArrayOutputStream(org.apache.commons.io.output.ByteArrayOutputStream) File(java.io.File) AbortException(hudson.AbortException)

Example 20 with ByteArrayOutputStream

use of org.apache.commons.io.output.ByteArrayOutputStream in project hudson-2.x by hudson.

the class ZFSInstaller method init.

@Extension
public static AdministrativeMonitor init() {
    String migrationTarget = System.getProperty(ZFSInstaller.class.getName() + ".migrate");
    if (migrationTarget != null) {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        StreamTaskListener listener = new StreamTaskListener(new ForkOutputStream(System.out, out));
        try {
            if (migrate(listener, migrationTarget)) {
                // completed successfully
                return new MigrationCompleteNotice();
            }
        } catch (Exception e) {
            // if we let any exception from here, it will prevent Hudson from starting.
            e.printStackTrace(listener.error("Migration failed"));
        }
        // migration failed
        return new MigrationFailedNotice(out);
    }
    // install the monitor if applicable
    ZFSInstaller zi = new ZFSInstaller();
    if (zi.isActivated())
        return zi;
    return null;
}
Also used : ForkOutputStream(hudson.util.ForkOutputStream) StreamTaskListener(hudson.util.StreamTaskListener) ByteArrayOutputStream(org.apache.commons.io.output.ByteArrayOutputStream) ServletException(javax.servlet.ServletException) ZFSException(org.jvnet.solaris.libzfs.ZFSException) IOException(java.io.IOException) Extension(hudson.Extension)

Aggregations

ByteArrayOutputStream (org.apache.commons.io.output.ByteArrayOutputStream)92 Test (org.junit.Test)36 DataOutputStream (java.io.DataOutputStream)15 IOException (java.io.IOException)14 HashSet (java.util.HashSet)13 ByteArrayInputStream (java.io.ByteArrayInputStream)12 ArrayList (java.util.ArrayList)12 Configuration (org.apache.hadoop.conf.Configuration)12 PrintStream (java.io.PrintStream)10 SparkConf (org.apache.spark.SparkConf)10 Edge (uk.gov.gchq.gaffer.data.element.Edge)10 Element (uk.gov.gchq.gaffer.data.element.Element)10 Entity (uk.gov.gchq.gaffer.data.element.Entity)10 Graph (uk.gov.gchq.gaffer.graph.Graph)10 User (uk.gov.gchq.gaffer.user.User)10 File (java.io.File)9 HashMap (java.util.HashMap)8 InputStream (java.io.InputStream)7 OutputStream (java.io.OutputStream)6 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)6