Search in sources :

Example 31 with RuntimeDataException

use of org.apache.asterix.common.exceptions.RuntimeDataException in project asterixdb by apache.

the class FeedIntakeOperatorNodePushable method start.

@Override
protected void start() throws HyracksDataException, InterruptedException {
    try {
        writer.open();
        Thread.currentThread().setName("Intake Thread");
        FeedAdapter adapter = (FeedAdapter) adapterFactory.createAdapter(ctx, partition);
        adapterRuntimeManager = new AdapterRuntimeManager(ctx, runtimeId.getEntityId(), adapter, writer, partition);
        IFrame message = new VSizeFrame(ctx);
        TaskUtil.putInSharedMap(HyracksConstants.KEY_MESSAGE, message, ctx);
        /*
             * Set null feed message. Feed pipeline carries with it a message with each frame
             * Initially, the message is set to a null message that can be changed by feed adapters.
             * One use case is adapters which consume data sources that allow restartability. Such adapters
             * can propagate progress information through the ingestion pipeline to storage nodes
             */
        message.getBuffer().put(MessagingFrameTupleAppender.NULL_FEED_MESSAGE);
        message.getBuffer().flip();
        adapterRuntimeManager.start();
        synchronized (adapterRuntimeManager) {
            while (!adapterRuntimeManager.isDone()) {
                adapterRuntimeManager.wait();
            }
        }
        if (adapterRuntimeManager.isFailed()) {
            throw new RuntimeDataException(ErrorCode.OPERATORS_FEED_INTAKE_OPERATOR_NODE_PUSHABLE_FAIL_AT_INGESTION);
        }
    } catch (Exception e) {
        /*
             * An Interrupted Exception is thrown if the Intake job cannot progress further due to failure of another
             * node involved in the Hyracks job. As the Intake job involves only the intake operator, the exception is
             * indicative of a failure at the sibling intake operator location. The surviving intake partitions must
             * continue to live and receive data from the external source.
             */
        writer.fail();
        throw e;
    } finally {
        writer.close();
    }
}
Also used : IFrame(org.apache.hyracks.api.comm.IFrame) FeedAdapter(org.apache.asterix.external.dataset.adapter.FeedAdapter) AdapterRuntimeManager(org.apache.asterix.external.feed.runtime.AdapterRuntimeManager) VSizeFrame(org.apache.hyracks.api.comm.VSizeFrame) RuntimeDataException(org.apache.asterix.common.exceptions.RuntimeDataException) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) RuntimeDataException(org.apache.asterix.common.exceptions.RuntimeDataException)

Example 32 with RuntimeDataException

use of org.apache.asterix.common.exceptions.RuntimeDataException in project asterixdb by apache.

the class RateControlledTupleForwarder method addTuple.

@Override
public void addTuple(ArrayTupleBuilder tb) throws HyracksDataException {
    if (delayConfigured) {
        try {
            Thread.sleep(interTupleInterval);
        } catch (InterruptedException e) {
            throw new HyracksDataException(e);
        }
    }
    boolean success = appender.append(tb.getFieldEndOffsets(), tb.getByteArray(), 0, tb.getSize());
    if (!success) {
        FrameUtils.flushFrame(frame.getBuffer(), writer);
        appender.reset(frame, true);
        success = appender.append(tb.getFieldEndOffsets(), tb.getByteArray(), 0, tb.getSize());
        if (!success) {
            throw new RuntimeDataException(ErrorCode.DATAFLOW_ILLEGAL_STATE);
        }
    }
}
Also used : HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) RuntimeDataException(org.apache.asterix.common.exceptions.RuntimeDataException)

Example 33 with RuntimeDataException

use of org.apache.asterix.common.exceptions.RuntimeDataException in project asterixdb by apache.

the class FileSystemWatcher method init.

public synchronized void init() throws HyracksDataException {
    try {
        dirs.clear();
        for (Path path : paths) {
            LocalFileSystemUtils.traverse(files, path.toFile(), expression, dirs);
            it = files.iterator();
            if (isFeed) {
                keys.clear();
                if (watcher != null) {
                    try {
                        watcher.close();
                    } catch (IOException e) {
                        LOGGER.warn("Failed to close watcher service", e);
                    }
                }
                watcher = FileSystems.getDefault().newWatchService();
                for (Path dirPath : dirs) {
                    register(dirPath);
                }
                resume();
            } else {
                if (files.isEmpty()) {
                    throw new RuntimeDataException(ErrorCode.UTIL_FILE_SYSTEM_WATCHER_NO_FILES_FOUND, path.toString());
                }
            }
        }
    } catch (IOException e) {
        throw new HyracksDataException(e);
    }
}
Also used : Path(java.nio.file.Path) IOException(java.io.IOException) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) RuntimeDataException(org.apache.asterix.common.exceptions.RuntimeDataException)

Aggregations

RuntimeDataException (org.apache.asterix.common.exceptions.RuntimeDataException)33 HyracksDataException (org.apache.hyracks.api.exceptions.HyracksDataException)15 IOException (java.io.IOException)12 DataOutput (java.io.DataOutput)7 IScalarEvaluator (org.apache.hyracks.algebricks.runtime.base.IScalarEvaluator)7 IHyracksTaskContext (org.apache.hyracks.api.context.IHyracksTaskContext)7 IPointable (org.apache.hyracks.data.std.api.IPointable)7 VoidPointable (org.apache.hyracks.data.std.primitive.VoidPointable)7 ArrayBackedValueStorage (org.apache.hyracks.data.std.util.ArrayBackedValueStorage)7 IFrameTupleReference (org.apache.hyracks.dataflow.common.data.accessors.IFrameTupleReference)7 ATypeTag (org.apache.asterix.om.types.ATypeTag)6 IAType (org.apache.asterix.om.types.IAType)6 IScalarEvaluatorFactory (org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory)6 AsterixException (org.apache.asterix.common.exceptions.AsterixException)5 TypeMismatchException (org.apache.asterix.runtime.exceptions.TypeMismatchException)4 AUnionType (org.apache.asterix.om.types.AUnionType)3 Path (java.nio.file.Path)2 ArrayList (java.util.ArrayList)2 RecordBuilder (org.apache.asterix.builders.RecordBuilder)2 IInputStreamFactory (org.apache.asterix.external.api.IInputStreamFactory)2