Search in sources :

Example 1 with ResourceSchema

use of org.apache.pig.ResourceSchema in project phoenix by apache.

the class PhoenixPigSchemaUtilTest method testSchema.

@Test
public void testSchema() throws SQLException, IOException {
    final Configuration configuration = mock(Configuration.class);
    when(configuration.get(PhoenixConfigurationUtil.SCHEMA_TYPE)).thenReturn(SchemaType.TABLE.name());
    final ResourceSchema actual = PhoenixPigSchemaUtil.getResourceSchema(configuration, new Dependencies() {

        List<ColumnInfo> getSelectColumnMetadataList(Configuration configuration) throws SQLException {
            return Lists.newArrayList(ID_COLUMN, NAME_COLUMN);
        }
    });
    // expected schema.
    final ResourceFieldSchema[] fields = new ResourceFieldSchema[2];
    fields[0] = new ResourceFieldSchema().setName("ID").setType(DataType.LONG);
    fields[1] = new ResourceFieldSchema().setName("NAME").setType(DataType.CHARARRAY);
    final ResourceSchema expected = new ResourceSchema().setFields(fields);
    assertEquals(expected.toString(), actual.toString());
}
Also used : ResourceSchema(org.apache.pig.ResourceSchema) Configuration(org.apache.hadoop.conf.Configuration) SQLException(java.sql.SQLException) ResourceFieldSchema(org.apache.pig.ResourceSchema.ResourceFieldSchema) List(java.util.List) Dependencies(org.apache.phoenix.pig.util.PhoenixPigSchemaUtil.Dependencies) Test(org.junit.Test)

Example 2 with ResourceSchema

use of org.apache.pig.ResourceSchema in project elephant-bird by twitter.

the class ResourceSchemaUtil method createResourceFieldSchema.

/**
 * Creates a new ResourceFieldSchema which reflects data from an input RequiredField.
 *
 * @param field
 * @return new ResourceFieldSchema which reflects {@code field}.
 * @throws IOException
 */
public static ResourceFieldSchema createResourceFieldSchema(RequiredField field) throws IOException {
    ResourceFieldSchema schema = new ResourceFieldSchema().setName(field.getAlias()).setType(field.getType());
    List<RequiredField> subFields = field.getSubFields();
    if (subFields != null && !subFields.isEmpty()) {
        ResourceFieldSchema[] subSchemaFields = new ResourceFieldSchema[subFields.size()];
        int i = 0;
        for (RequiredField subField : subFields) {
            subSchemaFields[i++] = createResourceFieldSchema(subField);
        }
        ResourceSchema subSchema = new ResourceSchema();
        subSchema.setFields(subSchemaFields);
        schema.setSchema(subSchema);
    }
    return schema;
}
Also used : ResourceSchema(org.apache.pig.ResourceSchema) RequiredField(org.apache.pig.LoadPushDown.RequiredField) ResourceFieldSchema(org.apache.pig.ResourceSchema.ResourceFieldSchema)

Example 3 with ResourceSchema

use of org.apache.pig.ResourceSchema in project elephant-bird by twitter.

the class ThriftWritableConverter method checkStoreSchema.

@Override
public void checkStoreSchema(ResourceFieldSchema schema) throws IOException {
    Preconditions.checkNotNull(schema, "Schema is null");
    Preconditions.checkArgument(DataType.TUPLE == schema.getType(), "Expected schema type '%s' but found type '%s'", DataType.findTypeName(DataType.TUPLE), DataType.findTypeName(schema.getType()));
    ResourceSchema childSchema = schema.getSchema();
    Preconditions.checkNotNull(childSchema, "Child schema is null");
    Schema actualSchema = Schema.getPigSchema(childSchema);
    Preconditions.checkArgument(Schema.equals(expectedSchema, actualSchema, false, true), "Expected store schema '%s' but found schema '%s'", expectedSchema, actualSchema);
}
Also used : ResourceSchema(org.apache.pig.ResourceSchema) ResourceSchema(org.apache.pig.ResourceSchema) ResourceFieldSchema(org.apache.pig.ResourceSchema.ResourceFieldSchema) Schema(org.apache.pig.impl.logicalLayer.schema.Schema) FieldSchema(org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema)

Example 4 with ResourceSchema

use of org.apache.pig.ResourceSchema in project elephant-bird by twitter.

the class TestPigToProtobuf method testConvertInvalidTypeMap.

@Test(expected = IllegalArgumentException.class)
public void testConvertInvalidTypeMap() throws Descriptors.DescriptorValidationException {
    Schema schema = new Schema();
    schema.add(new Schema.FieldSchema("map", DataType.MAP));
    PigToProtobuf.schemaToProtoDescriptor(new ResourceSchema(schema));
}
Also used : ResourceSchema(org.apache.pig.ResourceSchema) ResourceSchema(org.apache.pig.ResourceSchema) Schema(org.apache.pig.impl.logicalLayer.schema.Schema) Test(org.junit.Test)

Example 5 with ResourceSchema

use of org.apache.pig.ResourceSchema in project elephant-bird by twitter.

the class TestPigToProtobuf method testConvertInvalidTypeTuple.

@Test(expected = IllegalArgumentException.class)
public void testConvertInvalidTypeTuple() throws Descriptors.DescriptorValidationException {
    Schema schema = new Schema();
    schema.add(new Schema.FieldSchema("tuple", DataType.TUPLE));
    PigToProtobuf.schemaToProtoDescriptor(new ResourceSchema(schema));
}
Also used : ResourceSchema(org.apache.pig.ResourceSchema) ResourceSchema(org.apache.pig.ResourceSchema) Schema(org.apache.pig.impl.logicalLayer.schema.Schema) Test(org.junit.Test)

Aggregations

ResourceSchema (org.apache.pig.ResourceSchema)35 ResourceFieldSchema (org.apache.pig.ResourceSchema.ResourceFieldSchema)20 Schema (org.apache.pig.impl.logicalLayer.schema.Schema)11 Test (org.junit.Test)10 IOException (java.io.IOException)9 ArrayList (java.util.ArrayList)6 HCatFieldSchema (org.apache.hive.hcatalog.data.schema.HCatFieldSchema)6 Properties (java.util.Properties)5 FieldSchema (org.apache.pig.impl.logicalLayer.schema.Schema.FieldSchema)4 UDFContext (org.apache.pig.impl.util.UDFContext)4 HashMap (java.util.HashMap)3 List (java.util.List)3 Map (java.util.Map)3 HCatSchema (org.apache.hive.hcatalog.data.schema.HCatSchema)3 Tuple (org.apache.pig.data.Tuple)3 Descriptors (com.google.protobuf.Descriptors)2 BasicDBObjectBuilder (com.mongodb.BasicDBObjectBuilder)2 SQLException (java.sql.SQLException)2 LinkedHashMap (java.util.LinkedHashMap)2 Field (org.apache.avro.Schema.Field)2