Search in sources :

Example 26 with ExecException

use of org.apache.pig.backend.executionengine.ExecException in project pigeon by aseldawy.

the class YMin method exec.

@Override
public Double exec(Tuple input) throws IOException {
    Geometry geom = null;
    try {
        Object v = input.get(0);
        geom = geometryParser.parseGeom(v);
        Coordinate[] coords = geom.getEnvelope().getCoordinates();
        if (coords.length == 0)
            throw new ExecException("YMin cannot work on empty geometires");
        if (coords.length == 1)
            return coords[0].y;
        if (coords.length == 2)
            return Math.min(coords[0].y, coords[1].y);
        return Math.min(coords[0].y, coords[2].y);
    } catch (ExecException ee) {
        throw new GeoException(geom, ee);
    }
}
Also used : Geometry(org.locationtech.jts.geom.Geometry) Coordinate(org.locationtech.jts.geom.Coordinate) ExecException(org.apache.pig.backend.executionengine.ExecException)

Example 27 with ExecException

use of org.apache.pig.backend.executionengine.ExecException in project sketches-pig by DataSketches.

the class VarOptCommonImpl method createDataBagFromSketch.

// Produces a DataBag containing the samples from the input sketch
static DataBag createDataBagFromSketch(final VarOptItemsSketch<Tuple> sketch) {
    final DataBag output = BAG_FACTORY.newDefaultBag();
    final VarOptItemsSamples<Tuple> samples = sketch.getSketchSamples();
    try {
        // create (weight, item) tuples to add to output bag
        for (final VarOptItemsSamples<Tuple>.WeightedSample ws : samples) {
            final Tuple weightedSample = TUPLE_FACTORY.newTuple(2);
            weightedSample.set(0, ws.getWeight());
            weightedSample.set(1, ws.getItem());
            output.add(weightedSample);
        }
    } catch (final ExecException e) {
        throw new RuntimeException("Pig error: " + e.getMessage(), e);
    }
    return output;
}
Also used : DataBag(org.apache.pig.data.DataBag) ExecException(org.apache.pig.backend.executionengine.ExecException) VarOptItemsSamples(org.apache.datasketches.sampling.VarOptItemsSamples) Tuple(org.apache.pig.data.Tuple)

Example 28 with ExecException

use of org.apache.pig.backend.executionengine.ExecException in project sketches-pig by DataSketches.

the class ReservoirUnion method accumulate.

// We could overload exec() for easy cases, but we still need to compare the incoming
// reservoir's k vs max k and possibly downsample.
@Override
public void accumulate(final Tuple inputTuple) throws IOException {
    if (inputTuple == null || inputTuple.size() < 1 || inputTuple.isNull(0)) {
        return;
    }
    final DataBag reservoirs = (DataBag) inputTuple.get(0);
    if (this.union_ == null) {
        this.union_ = ReservoirItemsUnion.newInstance(this.maxK_);
    }
    try {
        for (Tuple t : reservoirs) {
            // if t == null or t.size() < 3, we'll throw an exception
            final long n = (long) t.get(0);
            final int k = (int) t.get(1);
            final DataBag sampleBag = (DataBag) t.get(2);
            final ArrayList<Tuple> samples = ReservoirSampling.dataBagToArrayList(sampleBag);
            this.union_.update(n, k, samples);
        }
    } catch (final IndexOutOfBoundsException e) {
        throw new ExecException("Cannot update union with given reservoir", e);
    }
}
Also used : DataBag(org.apache.pig.data.DataBag) ExecException(org.apache.pig.backend.executionengine.ExecException) Tuple(org.apache.pig.data.Tuple)

Example 29 with ExecException

use of org.apache.pig.backend.executionengine.ExecException in project sketches-pig by DataSketches.

the class ReservoirSamplingTest method generateDataBag.

static DataBag generateDataBag(final long numItems, final int startIdx) {
    final DataBag output = BagFactory.getInstance().newDefaultBag();
    try {
        for (int i = 0; i < numItems; ++i) {
            final Tuple t = TupleFactory.getInstance().newTuple(2);
            final int val = startIdx + i;
            t.set(0, val);
            t.set(1, Integer.toString(-val));
            output.add(t);
        }
    } catch (final ExecException e) {
        fail(e.getMessage());
    }
    return output;
}
Also used : DataBag(org.apache.pig.data.DataBag) ExecException(org.apache.pig.backend.executionengine.ExecException) Tuple(org.apache.pig.data.Tuple)

Example 30 with ExecException

use of org.apache.pig.backend.executionengine.ExecException in project elasticsearch-hadoop by elastic.

the class PigValueWriter method isPopulatedMixedValueMap.

/**
 * Checks to see if the given field is a schema-less Map that has values.
 * @return true if Map has no schema but has values (mixed schema map). false if not a Map or if Map is just empty.
 */
private boolean isPopulatedMixedValueMap(ResourceFieldSchema schema, int field, Tuple object) {
    if (schema.getType() != DataType.MAP) {
        // Can't be a mixed value map if it's not a map at all.
        return false;
    }
    try {
        Object fieldValue = object.get(field);
        Map<?, ?> map = (Map<?, ?>) fieldValue;
        return schema.getSchema() == null && !(map == null || map.isEmpty());
    } catch (ExecException e) {
        throw new EsHadoopIllegalStateException(e);
    }
}
Also used : EsHadoopIllegalStateException(org.elasticsearch.hadoop.EsHadoopIllegalStateException) ExecException(org.apache.pig.backend.executionengine.ExecException) Map(java.util.Map)

Aggregations

ExecException (org.apache.pig.backend.executionengine.ExecException)57 Tuple (org.apache.pig.data.Tuple)32 DataBag (org.apache.pig.data.DataBag)17 OGCGeometry (com.esri.core.geometry.ogc.OGCGeometry)13 DataByteArray (org.apache.pig.data.DataByteArray)11 IOException (java.io.IOException)9 Geometry (org.locationtech.jts.geom.Geometry)9 Test (org.testng.annotations.Test)6 Coordinate (org.locationtech.jts.geom.Coordinate)4 Field (com.twitter.elephantbird.thrift.TStructDescriptor.Field)3 FieldDescriptor (com.google.protobuf.Descriptors.FieldDescriptor)2 Builder (com.google.protobuf.Message.Builder)2 VarOptItemsSamples (com.yahoo.sketches.sampling.VarOptItemsSamples)2 ArrayList (java.util.ArrayList)2 List (java.util.List)2 VarOptItemsSamples (org.apache.datasketches.sampling.VarOptItemsSamples)2 Line (com.esri.core.geometry.Line)1 MultiPath (com.esri.core.geometry.MultiPath)1 Point (com.esri.core.geometry.Point)1 Polygon (com.esri.core.geometry.Polygon)1