Search in sources :

Example 31 with FrontendException

use of org.apache.pig.impl.logicalLayer.FrontendException in project sketches-pig by DataSketches.

the class VarOptSampling method outputSchema.

@Override
public Schema outputSchema(final Schema input) {
    try {
        if (input == null || input.size() == 0) {
            throw new IllegalArgumentException("Degenerate input schema to VarOptSampling");
        }
        // first element must be a bag, weightIdx_ element of tuples must be a float or double
        if (input.getField(0).type != DataType.BAG) {
            throw new IllegalArgumentException("VarOpt input must be a data bag: " + input.toString());
        }
        // record has a tuple in field 0
        final Schema record = input.getField(0).schema;
        final Schema fields = record.getField(0).schema;
        if (fields.getField(weightIdx_).type != DataType.DOUBLE && fields.getField(weightIdx_).type != DataType.FLOAT) {
            throw new IllegalArgumentException("weightIndex item of VarOpt tuple must be a " + "weight (double/float), found " + fields.getField(0).type + ": " + fields.toString());
        }
        final Schema weightedSampleSchema = new Schema();
        weightedSampleSchema.add(new Schema.FieldSchema(WEIGHT_ALIAS, DataType.DOUBLE));
        weightedSampleSchema.add(new Schema.FieldSchema(RECORD_ALIAS, record, DataType.TUPLE));
        return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), record), weightedSampleSchema, DataType.BAG));
    } catch (final FrontendException e) {
        throw new RuntimeException(e);
    }
}
Also used : Schema(org.apache.pig.impl.logicalLayer.schema.Schema) FrontendException(org.apache.pig.impl.logicalLayer.FrontendException)

Example 32 with FrontendException

use of org.apache.pig.impl.logicalLayer.FrontendException in project sketches-pig by DataSketches.

the class DataToSketch method outputSchema.

@Override
public Schema outputSchema(final Schema input) {
    if (input != null) {
        try {
            final Schema tupleSchema = new Schema();
            tupleSchema.add(new Schema.FieldSchema("Sketch", DataType.BYTEARRAY));
            return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input), tupleSchema, DataType.TUPLE));
        } catch (final FrontendException e) {
        // fall through
        }
    }
    return null;
}
Also used : Schema(org.apache.pig.impl.logicalLayer.schema.Schema) FrontendException(org.apache.pig.impl.logicalLayer.FrontendException)

Example 33 with FrontendException

use of org.apache.pig.impl.logicalLayer.FrontendException in project sketches-pig by DataSketches.

the class ErrorBounds method outputSchema.

/**
 * The output is a Sketch Result Tuple Schema.
 */
@Override
public Schema outputSchema(final Schema input) {
    if (input != null) {
        try {
            final Schema tupleSchema = new Schema();
            tupleSchema.add(new Schema.FieldSchema("Estimate", DataType.DOUBLE));
            tupleSchema.add(new Schema.FieldSchema("UpperBound", DataType.DOUBLE));
            tupleSchema.add(new Schema.FieldSchema("LowerBound", DataType.DOUBLE));
            return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input), tupleSchema, DataType.TUPLE));
        } catch (final FrontendException e) {
        // fall through
        }
    }
    return null;
}
Also used : Schema(org.apache.pig.impl.logicalLayer.schema.Schema) FrontendException(org.apache.pig.impl.logicalLayer.FrontendException)

Example 34 with FrontendException

use of org.apache.pig.impl.logicalLayer.FrontendException in project sketches-pig by DataSketches.

the class FrequentStringsSketchToEstimates method outputSchema.

@Override
public Schema outputSchema(final Schema inputSchema) {
    final Schema tupleSchema = new Schema();
    tupleSchema.add(new Schema.FieldSchema("item", DataType.CHARARRAY));
    tupleSchema.add(new Schema.FieldSchema("estimate", DataType.LONG));
    tupleSchema.add(new Schema.FieldSchema("lower_bound", DataType.LONG));
    tupleSchema.add(new Schema.FieldSchema("upper_bound", DataType.LONG));
    try {
        final Schema bagSchema = new Schema(new Schema.FieldSchema("item_tuple", tupleSchema, DataType.TUPLE));
        return new Schema(new Schema.FieldSchema("bag_of_item_tuples", bagSchema, DataType.BAG));
    } catch (final FrontendException e) {
        throw new RuntimeException(e);
    }
}
Also used : Schema(org.apache.pig.impl.logicalLayer.schema.Schema) FrontendException(org.apache.pig.impl.logicalLayer.FrontendException)

Example 35 with FrontendException

use of org.apache.pig.impl.logicalLayer.FrontendException in project sketches-pig by DataSketches.

the class DataToItemsSketch method outputSchema.

@Override
public Schema outputSchema(final Schema input) {
    if (input == null) {
        return null;
    }
    try {
        final Schema tupleSchema = new Schema();
        tupleSchema.add(new Schema.FieldSchema("Sketch", DataType.BYTEARRAY));
        return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input), tupleSchema, DataType.TUPLE));
    } catch (final FrontendException e) {
        throw new RuntimeException(e);
    }
}
Also used : Schema(org.apache.pig.impl.logicalLayer.schema.Schema) FrontendException(org.apache.pig.impl.logicalLayer.FrontendException)

Aggregations

FrontendException (org.apache.pig.impl.logicalLayer.FrontendException)36 Schema (org.apache.pig.impl.logicalLayer.schema.Schema)27 FieldSchema (org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema)11 ArrayList (java.util.ArrayList)6 HCatFieldSchema (org.apache.hive.hcatalog.data.schema.HCatFieldSchema)5 HCatSchema (org.apache.hive.hcatalog.data.schema.HCatSchema)5 ResourceSchema (org.apache.pig.ResourceSchema)4 IOException (java.io.IOException)3 OriginalType (org.apache.parquet.schema.OriginalType)3 PigServer (org.apache.pig.PigServer)3 DataType (org.apache.pig.data.DataType)3 Tuple (org.apache.pig.data.Tuple)3 File (java.io.File)2 List (java.util.List)2 HCatException (org.apache.hive.hcatalog.common.HCatException)2 PigSchemaConverter.parsePigSchema (org.apache.parquet.pig.PigSchemaConverter.parsePigSchema)2 GroupType (org.apache.parquet.schema.GroupType)2 MessageType (org.apache.parquet.schema.MessageType)2 PrimitiveType (org.apache.parquet.schema.PrimitiveType)2 Type (org.apache.parquet.schema.Type)2