Search in sources :

Example 1 with HyracksDataException

use of org.apache.hyracks.api.exceptions.HyracksDataException in project asterixdb by apache.

the class StoragePathUtil method createFile.

/**
     * Create a file
     * Note: this method is not thread safe. It is the responsibility of the caller to ensure no path conflict when
     * creating files simultaneously
     *
     * @param name
     * @param count
     * @return
     * @throws HyracksDataException
     */
public static File createFile(String name, int count) throws HyracksDataException {
    try {
        String fileName = name + "_" + count;
        File file = new File(fileName);
        if (file.getParentFile() != null) {
            file.getParentFile().mkdirs();
        }
        if (!file.exists()) {
            boolean success = file.createNewFile();
            if (!success) {
                throw new HyracksDataException("Unable to create spill file " + fileName);
            } else {
                if (LOGGER.isEnabledFor(Level.INFO)) {
                    LOGGER.info("Created spill file " + file.getAbsolutePath());
                }
            }
        } else {
            throw new HyracksDataException("spill file " + fileName + " already exists");
        }
        return file;
    } catch (Exception e) {
        throw new HyracksDataException(e);
    }
}
Also used : File(java.io.File) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Example 2 with HyracksDataException

use of org.apache.hyracks.api.exceptions.HyracksDataException in project asterixdb by apache.

the class PrimaryIndexLogMarkerCallback method lsnFromImmutableMemoryComponents.

private long lsnFromImmutableMemoryComponents() {
    List<ILSMMemoryComponent> memComponents = index.getMemoryComponents();
    int numOtherMemComponents = memComponents.size() - 1;
    int next = index.getCurrentMemoryComponentIndex();
    long lsn = ComponentMetadataUtil.NOT_FOUND;
    for (int i = 0; i < numOtherMemComponents; i++) {
        next = next - 1;
        if (next < 0) {
            next = memComponents.size() - 1;
        }
        ILSMMemoryComponent c = index.getMemoryComponents().get(next);
        if (c.isReadable()) {
            try {
                lsn = ComponentMetadataUtil.getLong(c.getMetadata(), ComponentMetadataUtil.MARKER_LSN_KEY, ComponentMetadataUtil.NOT_FOUND);
            } catch (HyracksDataException e) {
                // Should never happen since this is a memory component
                throw new IllegalStateException(e);
            }
            if (lsn != ComponentMetadataUtil.NOT_FOUND) {
                return lsn;
            }
        }
    }
    return lsn;
}
Also used : ILSMMemoryComponent(org.apache.hyracks.storage.am.lsm.common.api.ILSMMemoryComponent) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Example 3 with HyracksDataException

use of org.apache.hyracks.api.exceptions.HyracksDataException in project asterixdb by apache.

the class ConcurrentFramePoolUnitTest method testLargerThanBudgetRequests.

@org.junit.Test
public void testLargerThanBudgetRequests() {
    HyracksDataException hde = null;
    try {
        ConcurrentFramePool fmm = new ConcurrentFramePool("TestNode", DEFAULT_FRAME_SIZE * 16, DEFAULT_FRAME_SIZE);
        fmm.get(32 * DEFAULT_FRAME_SIZE);
    } catch (HyracksDataException e) {
        hde = e;
    } catch (Throwable th) {
        th.printStackTrace();
        Assert.fail(th.getMessage());
    }
    Assert.assertNotNull(hde);
    Assert.assertNull(cause);
}
Also used : ConcurrentFramePool(org.apache.asterix.common.memory.ConcurrentFramePool) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Example 4 with HyracksDataException

use of org.apache.hyracks.api.exceptions.HyracksDataException in project asterixdb by apache.

the class RecordColumnarIndexer method reset.

@Override
public void reset(IIndexingDatasource reader) throws HyracksDataException {
    try {
        //TODO: Make this more generic. right now, it works because we only index hdfs files.
        @SuppressWarnings("unchecked") HDFSRecordReader<?, Writable> hdfsReader = (HDFSRecordReader<?, Writable>) reader;
        fileNumber.setValue(hdfsReader.getSnapshot().get(hdfsReader.getCurrentSplitIndex()).getFileNumber());
        recordReader = hdfsReader.getReader();
        offset.setValue(recordReader.getPos());
        nextOffset = offset.getLongValue();
        rowNumber.setValue(0);
    } catch (IOException e) {
        throw new HyracksDataException(e);
    }
}
Also used : Writable(org.apache.hadoop.io.Writable) IOException(java.io.IOException) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) HDFSRecordReader(org.apache.asterix.external.input.record.reader.hdfs.HDFSRecordReader)

Example 5 with HyracksDataException

use of org.apache.hyracks.api.exceptions.HyracksDataException in project asterixdb by apache.

the class HDFSInputStream method getRecordReader.

@SuppressWarnings("unchecked")
private RecordReader<Object, Text> getRecordReader(int splitIndex) throws IOException {
    reader = (RecordReader<Object, Text>) inputFormat.getRecordReader(inputSplits[splitIndex], conf, Reporter.NULL);
    if (key == null) {
        key = reader.createKey();
        value = reader.createValue();
    }
    if (indexer != null) {
        try {
            indexer.reset(this);
        } catch (Exception e) {
            throw new HyracksDataException(e);
        }
    }
    return reader;
}
Also used : Text(org.apache.hadoop.io.Text) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException) IOException(java.io.IOException) AsterixException(org.apache.asterix.common.exceptions.AsterixException) HyracksDataException(org.apache.hyracks.api.exceptions.HyracksDataException)

Aggregations

HyracksDataException (org.apache.hyracks.api.exceptions.HyracksDataException)566 IOException (java.io.IOException)209 DataOutput (java.io.DataOutput)96 ArrayBackedValueStorage (org.apache.hyracks.data.std.util.ArrayBackedValueStorage)78 ITupleReference (org.apache.hyracks.dataflow.common.data.accessors.ITupleReference)75 IPointable (org.apache.hyracks.data.std.api.IPointable)73 IFrameTupleReference (org.apache.hyracks.dataflow.common.data.accessors.IFrameTupleReference)70 TypeMismatchException (org.apache.asterix.runtime.exceptions.TypeMismatchException)67 IScalarEvaluator (org.apache.hyracks.algebricks.runtime.base.IScalarEvaluator)67 VoidPointable (org.apache.hyracks.data.std.primitive.VoidPointable)64 IScalarEvaluatorFactory (org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory)61 IHyracksTaskContext (org.apache.hyracks.api.context.IHyracksTaskContext)60 ArrayList (java.util.ArrayList)57 ACIDException (org.apache.asterix.common.exceptions.ACIDException)56 ISerializerDeserializer (org.apache.hyracks.api.dataflow.value.ISerializerDeserializer)56 ATypeTag (org.apache.asterix.om.types.ATypeTag)39 AsterixException (org.apache.asterix.common.exceptions.AsterixException)36 ArrayTupleBuilder (org.apache.hyracks.dataflow.common.comm.io.ArrayTupleBuilder)32 ByteBuffer (java.nio.ByteBuffer)30 MetadataEntityValueExtractor (org.apache.asterix.metadata.valueextractors.MetadataEntityValueExtractor)27