Search in sources :

Example 1 with Entry

use of com.google.cloud.datacatalog.v1beta1.Entry in project beam by apache.

the class DataCatalogTableProvider method setSchemaIfNotPresent.

@Internal
public boolean setSchemaIfNotPresent(String resource, Schema schema) {
    com.google.cloud.datacatalog.v1beta1.Schema dcSchema = SchemaUtils.toDataCatalog(schema);
    Entry entry = dataCatalog.lookupEntry(LookupEntryRequest.newBuilder().setSqlResource(resource).build());
    if (entry.getSchema().getColumnsCount() == 0) {
        dataCatalog.updateEntry(UpdateEntryRequest.newBuilder().setEntry(entry.toBuilder().setSchema(dcSchema).build()).setUpdateMask(FieldMask.newBuilder().addPaths("schema").build()).build());
        return true;
    } else {
        LOG.info(String.format("Not updating schema for '%s' since it already has one.", resource));
        return false;
    }
}
Also used : Entry(com.google.cloud.datacatalog.v1beta1.Entry) Internal(org.apache.beam.sdk.annotations.Internal)

Example 2 with Entry

use of com.google.cloud.datacatalog.v1beta1.Entry in project beam by apache.

the class GcsTableFactory method tableBuilder.

/**
 * Creates a Beam SQL table description from a GCS fileset entry.
 */
@Override
public Optional<Table.Builder> tableBuilder(Entry entry) {
    if (!entry.hasGcsFilesetSpec()) {
        return Optional.empty();
    }
    GcsFilesetSpec gcsFilesetSpec = entry.getGcsFilesetSpec();
    List<String> filePatterns = gcsFilesetSpec.getFilePatternsList();
    // We support exactly one 'file_patterns' field and nothing else at the moment
    if (filePatterns.size() != 1) {
        throw new UnsupportedOperationException("Unable to parse GCS entry '" + entry.getName() + "'");
    }
    String filePattern = filePatterns.get(0);
    if (!filePattern.startsWith("gs://")) {
        throw new UnsupportedOperationException("Unsupported file pattern. " + "Only file patterns with 'gs://' schema are supported at the moment.");
    }
    return Optional.of(Table.builder().type("text").location(filePattern).properties(new JSONObject()).comment(""));
}
Also used : GcsFilesetSpec(com.google.cloud.datacatalog.v1beta1.GcsFilesetSpec) JSONObject(com.alibaba.fastjson.JSONObject)

Example 3 with Entry

use of com.google.cloud.datacatalog.v1beta1.Entry in project xtext-core by eclipse.

the class ActionTestLanguage3SemanticSequencer method sequence.

@Override
public void sequence(ISerializationContext context, EObject semanticObject) {
    EPackage epackage = semanticObject.eClass().getEPackage();
    ParserRule rule = context.getParserRule();
    Action action = context.getAssignedAction();
    Set<Parameter> parameters = context.getEnabledBooleanParameters();
    if (epackage == ActionLang3Package.eINSTANCE)
        switch(semanticObject.eClass().getClassifierID()) {
            case ActionLang3Package.ENTRY:
                sequence_ProductionRule2(context, (Entry) semanticObject);
                return;
            case ActionLang3Package.P1:
                sequence_ProductionRule1_P2_2(context, (P1) semanticObject);
                return;
            case ActionLang3Package.P2:
                sequence_ProductionRule1_P3_4(context, (P2) semanticObject);
                return;
            case ActionLang3Package.P3:
                sequence_ProductionRule1(context, (P3) semanticObject);
                return;
        }
    if (errorAcceptor != null)
        errorAcceptor.accept(diagnosticProvider.createInvalidContextOrTypeDiagnostic(semanticObject, context));
}
Also used : ParserRule(org.eclipse.xtext.ParserRule) P1(org.eclipse.xtext.testlanguages.actionLang3.P1) Action(org.eclipse.xtext.Action) Entry(org.eclipse.xtext.testlanguages.actionLang3.Entry) P2(org.eclipse.xtext.testlanguages.actionLang3.P2) P3(org.eclipse.xtext.testlanguages.actionLang3.P3) Parameter(org.eclipse.xtext.Parameter) EPackage(org.eclipse.emf.ecore.EPackage)

Aggregations

JSONObject (com.alibaba.fastjson.JSONObject)1 Entry (com.google.cloud.datacatalog.v1beta1.Entry)1 GcsFilesetSpec (com.google.cloud.datacatalog.v1beta1.GcsFilesetSpec)1 Internal (org.apache.beam.sdk.annotations.Internal)1 EPackage (org.eclipse.emf.ecore.EPackage)1 Action (org.eclipse.xtext.Action)1 Parameter (org.eclipse.xtext.Parameter)1 ParserRule (org.eclipse.xtext.ParserRule)1 Entry (org.eclipse.xtext.testlanguages.actionLang3.Entry)1 P1 (org.eclipse.xtext.testlanguages.actionLang3.P1)1 P2 (org.eclipse.xtext.testlanguages.actionLang3.P2)1 P3 (org.eclipse.xtext.testlanguages.actionLang3.P3)1