Examples with FieldType org.apache.beam.sdk.schemas.Schema.FieldType used on opensource projects

Example 56 with FieldType

use of org.apache.beam.sdk.schemas.Schema.FieldType in project beam by apache.

the class ProtoSchemaTranslator method beamFieldTypeFromProtoField.

private static FieldType beamFieldTypeFromProtoField(Descriptors.FieldDescriptor protoFieldDescriptor) {
    FieldType fieldType = null;
    if (protoFieldDescriptor.isMapField()) {
        FieldDescriptor keyFieldDescriptor = protoFieldDescriptor.getMessageType().findFieldByName("key");
        FieldDescriptor valueFieldDescriptor = protoFieldDescriptor.getMessageType().findFieldByName("value");
        fieldType = FieldType.map(beamFieldTypeFromProtoField(keyFieldDescriptor).withNullable(false), beamFieldTypeFromProtoField(valueFieldDescriptor).withNullable(false));
    } else if (protoFieldDescriptor.isRepeated()) {
        fieldType = FieldType.array(beamFieldTypeFromSingularProtoField(protoFieldDescriptor).withNullable(false));
    } else {
        fieldType = beamFieldTypeFromSingularProtoField(protoFieldDescriptor);
    }
    return fieldType;
}

Example 57 with FieldType

use of org.apache.beam.sdk.schemas.Schema.FieldType in project beam by apache.

the class BigQueryUtils method convertAvroArray.

private static Object convertAvroArray(FieldType beamField, Object value, BigQueryUtils.ConversionOptions options) {
    // Check whether the type of array element is equal.
    List<Object> values = (List<Object>) value;
    List<Object> ret = new ArrayList<>();
    FieldType collectionElement = beamField.getCollectionElementType();
    for (Object v : values) {
        ret.add(convertAvroFormat(collectionElement, v, options));
    }
    return ret;
}

Example 58 with FieldType

use of org.apache.beam.sdk.schemas.Schema.FieldType in project beam by apache.

the class BigQueryUtils method convertAvroRecordToMap.

private static Object convertAvroRecordToMap(FieldType beamField, Object value, BigQueryUtils.ConversionOptions options) {
    List<GenericData.Record> records = (List<GenericData.Record>) value;
    ImmutableMap.Builder<Object, Object> ret = ImmutableMap.builder();
    FieldType keyElement = beamField.getMapKeyType();
    FieldType valueElement = beamField.getMapValueType();
    for (GenericData.Record record : records) {
        ret.put(convertAvroFormat(keyElement, record.get(0), options), convertAvroFormat(valueElement, record.get(1), options));
    }
    return ret.build();
}