Search in sources :

Example 51 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class AvroRecordReader method processPrimitive.

private void processPrimitive(final Object value, final Schema.Type type, final String fieldName, final MapOrListWriterImpl writer) {
    if (value == null) {
        return;
    }
    switch(type) {
        case STRING:
            byte[] binary = null;
            final int length;
            if (value instanceof Utf8) {
                binary = ((Utf8) value).getBytes();
                length = ((Utf8) value).getByteLength();
            } else {
                binary = value.toString().getBytes(Charsets.UTF_8);
                length = binary.length;
            }
            ensure(length);
            buffer.setBytes(0, binary);
            writer.varChar(fieldName).writeVarChar(0, length, buffer);
            break;
        case INT:
            writer.integer(fieldName).writeInt((Integer) value);
            break;
        case LONG:
            writer.bigInt(fieldName).writeBigInt((Long) value);
            break;
        case FLOAT:
            writer.float4(fieldName).writeFloat4((Float) value);
            break;
        case DOUBLE:
            writer.float8(fieldName).writeFloat8((Double) value);
            break;
        case BOOLEAN:
            writer.bit(fieldName).writeBit((Boolean) value ? 1 : 0);
            break;
        case BYTES:
            final ByteBuffer buf = (ByteBuffer) value;
            length = buf.remaining();
            ensure(length);
            buffer.setBytes(0, buf);
            writer.binary(fieldName).writeVarBinary(0, length, buffer);
            break;
        case NULL:
            // Nothing to do for null type
            break;
        case ENUM:
            final String symbol = value.toString();
            final byte[] b;
            try {
                b = symbol.getBytes("UTF-8");
            } catch (UnsupportedEncodingException e) {
                throw new DrillRuntimeException("Unable to read enum value for field: " + fieldName, e);
            }
            ensure(b.length);
            buffer.setBytes(0, b);
            writer.varChar(fieldName).writeVarChar(0, b.length, buffer);
            break;
        default:
            throw new DrillRuntimeException("Unhandled Avro type: " + type.toString());
    }
}
Also used : Utf8(org.apache.avro.util.Utf8) UnsupportedEncodingException(java.io.UnsupportedEncodingException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) ByteBuffer(java.nio.ByteBuffer)

Example 52 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class AvroRecordReader method next.

@Override
public int next() {
    final Stopwatch watch = Stopwatch.createStarted();
    if (reader == null) {
        throw new IllegalStateException("Avro reader is not open.");
    }
    if (!reader.hasNext()) {
        return 0;
    }
    int recordCount = 0;
    writer.allocate();
    writer.reset();
    try {
        for (GenericContainer container = null; recordCount < DEFAULT_BATCH_SIZE && reader.hasNext() && !reader.pastSync(end); recordCount++) {
            writer.setPosition(recordCount);
            container = reader.next(container);
            processRecord(container, container.getSchema());
        }
        writer.setValueCount(recordCount);
    } catch (IOException e) {
        throw new DrillRuntimeException(e);
    }
    logger.debug("Read {} records in {} ms", recordCount, watch.elapsed(TimeUnit.MILLISECONDS));
    return recordCount;
}
Also used : Stopwatch(com.google.common.base.Stopwatch) IOException(java.io.IOException) GenericContainer(org.apache.avro.generic.GenericContainer) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException)

Example 53 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class BsonRecordReader method write.

public void write(ComplexWriter writer, BsonReader reader) throws IOException {
    reader.readStartDocument();
    BsonType readBsonType = reader.getCurrentBsonType();
    switch(readBsonType) {
        case DOCUMENT:
            writeToListOrMap(reader, new MapOrListWriterImpl(writer.rootAsMap()), false, null);
            break;
        default:
            throw new DrillRuntimeException("Root object must be DOCUMENT type. Found: " + readBsonType);
    }
}
Also used : BsonType(org.bson.BsonType) MapOrListWriterImpl(org.apache.drill.exec.vector.complex.impl.MapOrListWriterImpl) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException)

Example 54 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class FileSelection method createFromDirectories.

public static FileSelection createFromDirectories(final List<String> dirPaths, final FileSelection selection, final String cacheFileRoot) {
    Stopwatch timer = Stopwatch.createStarted();
    final String root = selection.getSelectionRoot();
    if (Strings.isNullOrEmpty(root)) {
        throw new DrillRuntimeException("Selection root is null or empty" + root);
    }
    if (dirPaths == null || dirPaths.isEmpty()) {
        throw new DrillRuntimeException("List of directories is null or empty");
    }
    List<String> dirs = Lists.newArrayList();
    if (selection.hadWildcard()) {
        // for wildcard the directory list should have already been expanded
        for (FileStatus status : selection.getFileStatuses()) {
            dirs.add(status.getPath().toString());
        }
    } else {
        for (String s : dirPaths) {
            dirs.add(s);
        }
    }
    final Path rootPath = handleWildCard(root);
    // final URI uri = dirPaths.get(0).toUri();
    final URI uri = selection.getFileStatuses().get(0).getPath().toUri();
    final Path path = new Path(uri.getScheme(), uri.getAuthority(), rootPath.toUri().getPath());
    FileSelection fileSel = new FileSelection(null, dirs, path.toString(), cacheFileRoot, false);
    fileSel.setHadWildcard(selection.hadWildcard());
    logger.info("FileSelection.createFromDirectories() took {} ms ", timer.elapsed(TimeUnit.MILLISECONDS));
    return fileSel;
}
Also used : Path(org.apache.hadoop.fs.Path) FileStatus(org.apache.hadoop.fs.FileStatus) Stopwatch(com.google.common.base.Stopwatch) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) URI(java.net.URI)

Example 55 with DrillRuntimeException

use of org.apache.drill.common.exceptions.DrillRuntimeException in project drill by apache.

the class ZookeeperClient method delete.

/**
   * Deletes the given node residing at the given path
   *
   * @param path  target path to delete
   */
public void delete(final String path) {
    Preconditions.checkNotNull(path, "path is required");
    final String target = PathUtils.join(root, path);
    try {
        curator.delete().forPath(target);
        getCache().rebuildNode(target);
    } catch (final Exception e) {
        throw new DrillRuntimeException(String.format("unable to delete node at %s", target), e);
    }
}
Also used : DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) KeeperException(org.apache.zookeeper.KeeperException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) VersionMismatchException(org.apache.drill.exec.exception.VersionMismatchException) NodeExistsException(org.apache.zookeeper.KeeperException.NodeExistsException)

Aggregations

DrillRuntimeException (org.apache.drill.common.exceptions.DrillRuntimeException)69 IOException (java.io.IOException)32 VersionMismatchException (org.apache.drill.exec.exception.VersionMismatchException)9 Stopwatch (com.google.common.base.Stopwatch)7 ExecutionSetupException (org.apache.drill.common.exceptions.ExecutionSetupException)7 UserException (org.apache.drill.common.exceptions.UserException)6 KeeperException (org.apache.zookeeper.KeeperException)6 NodeExistsException (org.apache.zookeeper.KeeperException.NodeExistsException)6 NoSuchElementException (java.util.NoSuchElementException)5 Path (org.apache.hadoop.fs.Path)5 Bson (org.bson.conversions.Bson)4 Registry (org.apache.drill.exec.proto.UserBitShared.Registry)3 ValueVector (org.apache.drill.exec.vector.ValueVector)3 Admin (org.apache.hadoop.hbase.client.Admin)3 Document (org.bson.Document)3 File (java.io.File)2 UnsupportedEncodingException (java.io.UnsupportedEncodingException)2 URI (java.net.URI)2 RexNode (org.apache.calcite.rex.RexNode)2 LogicalExpression (org.apache.drill.common.expression.LogicalExpression)2