Search in sources :

Example 6 with GetSchema

use of uk.gov.gchq.gaffer.store.operation.GetSchema in project Gaffer by gchq.

the class Store method validateSchemas.

public void validateSchemas() {
    final ValidationResult validationResult = new ValidationResult();
    if (null == schema) {
        validationResult.addError("Schema is missing");
    } else {
        validationResult.add(schema.validate());
        getSchemaElements().forEach((key, value) -> value.getProperties().forEach(propertyName -> {
            final Class propertyClass = value.getPropertyClass(propertyName);
            final Serialiser serialisation = value.getPropertyTypeDef(propertyName).getSerialiser();
            if (null == serialisation) {
                validationResult.addError(String.format("Could not find a serialiser for property '%s' in the group '%s'.", propertyName, key));
            } else if (!serialisation.canHandle(propertyClass)) {
                validationResult.addError(String.format("Schema serialiser (%s) for property '%s' in the group '%s' cannot handle property found in the schema", serialisation.getClass().getName(), propertyName, key));
            }
        }));
        validateSchema(validationResult, getSchema().getVertexSerialiser());
        getSchema().getTypes().forEach((k, v) -> validateSchema(validationResult, v.getSerialiser()));
    }
    if (!validationResult.isValid()) {
        throw new SchemaException("Schema is not valid. " + validationResult.getErrorString());
    }
}
Also used : GetExportsHandler(uk.gov.gchq.gaffer.store.operation.handler.export.GetExportsHandler) SetVariable(uk.gov.gchq.gaffer.operation.impl.SetVariable) Collections.unmodifiableList(java.util.Collections.unmodifiableList) SchemaException(uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException) SerialisationException(uk.gov.gchq.gaffer.exception.SerialisationException) GenerateElements(uk.gov.gchq.gaffer.operation.impl.generate.GenerateElements) NamedOperation(uk.gov.gchq.gaffer.named.operation.NamedOperation) Max(uk.gov.gchq.gaffer.operation.impl.compare.Max) Element(uk.gov.gchq.gaffer.data.element.Element) GetJobResultsHandler(uk.gov.gchq.gaffer.store.operation.handler.job.GetJobResultsHandler) DiscardOutputHandler(uk.gov.gchq.gaffer.store.operation.handler.DiscardOutputHandler) SortHandler(uk.gov.gchq.gaffer.store.operation.handler.compare.SortHandler) SchemaOptimiser(uk.gov.gchq.gaffer.store.schema.SchemaOptimiser) ValidateOperationChainHandler(uk.gov.gchq.gaffer.store.operation.handler.ValidateOperationChainHandler) ToArrayHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToArrayHandler) SetVariableHandler(uk.gov.gchq.gaffer.store.operation.handler.SetVariableHandler) Map(java.util.Map) If(uk.gov.gchq.gaffer.operation.impl.If) Count(uk.gov.gchq.gaffer.operation.impl.Count) DeleteNamedViewHandler(uk.gov.gchq.gaffer.store.operation.handler.named.DeleteNamedViewHandler) ToEntitySeedsHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToEntitySeedsHandler) ToSingletonList(uk.gov.gchq.gaffer.operation.impl.output.ToSingletonList) MaxHandler(uk.gov.gchq.gaffer.store.operation.handler.compare.MaxHandler) ToArray(uk.gov.gchq.gaffer.operation.impl.output.ToArray) OperationDeclaration(uk.gov.gchq.gaffer.store.operation.declaration.OperationDeclaration) AddSchemaToLibraryHandler(uk.gov.gchq.gaffer.store.operation.handler.AddSchemaToLibraryHandler) ToEntitySeeds(uk.gov.gchq.gaffer.operation.impl.output.ToEntitySeeds) ValidationResult(uk.gov.gchq.koryphe.ValidationResult) ForEachHandler(uk.gov.gchq.gaffer.store.operation.handler.ForEachHandler) LimitHandler(uk.gov.gchq.gaffer.store.operation.handler.LimitHandler) AddNamedViewHandler(uk.gov.gchq.gaffer.store.operation.handler.named.AddNamedViewHandler) ExportToGafferResultCache(uk.gov.gchq.gaffer.operation.impl.export.resultcache.ExportToGafferResultCache) Set(java.util.Set) ToListHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToListHandler) ExportToSet(uk.gov.gchq.gaffer.operation.impl.export.set.ExportToSet) ReflectionUtil(uk.gov.gchq.koryphe.util.ReflectionUtil) OperationChainDAO(uk.gov.gchq.gaffer.operation.OperationChainDAO) CountGroupsHandler(uk.gov.gchq.gaffer.store.operation.handler.CountGroupsHandler) AddNamedOperationHandler(uk.gov.gchq.gaffer.store.operation.handler.named.AddNamedOperationHandler) ForEach(uk.gov.gchq.gaffer.operation.impl.ForEach) AddElements(uk.gov.gchq.gaffer.operation.impl.add.AddElements) OutputOperationHandler(uk.gov.gchq.gaffer.store.operation.handler.OutputOperationHandler) DiscardOutput(uk.gov.gchq.gaffer.operation.impl.DiscardOutput) IfHandler(uk.gov.gchq.gaffer.store.operation.handler.IfHandler) JobTracker(uk.gov.gchq.gaffer.jobtracker.JobTracker) GetAllNamedViews(uk.gov.gchq.gaffer.named.view.GetAllNamedViews) GetElements(uk.gov.gchq.gaffer.operation.impl.get.GetElements) FilterHandler(uk.gov.gchq.gaffer.store.operation.handler.function.FilterHandler) GetVariables(uk.gov.gchq.gaffer.operation.impl.GetVariables) ArrayList(java.util.ArrayList) ValidateOperationChain(uk.gov.gchq.gaffer.operation.impl.ValidateOperationChain) LinkedHashMap(java.util.LinkedHashMap) GetVariable(uk.gov.gchq.gaffer.operation.impl.GetVariable) Transform(uk.gov.gchq.gaffer.operation.impl.function.Transform) ToStreamHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToStreamHandler) GetAllNamedOperations(uk.gov.gchq.gaffer.named.operation.GetAllNamedOperations) GetAllJobDetails(uk.gov.gchq.gaffer.operation.impl.job.GetAllJobDetails) ToVertices(uk.gov.gchq.gaffer.operation.impl.output.ToVertices) NoGraphLibrary(uk.gov.gchq.gaffer.store.library.NoGraphLibrary) DeleteNamedOperation(uk.gov.gchq.gaffer.named.operation.DeleteNamedOperation) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) Validate(uk.gov.gchq.gaffer.operation.impl.Validate) ViewValidator(uk.gov.gchq.gaffer.store.schema.ViewValidator) StreamSupport(java.util.stream.StreamSupport) Sort(uk.gov.gchq.gaffer.operation.impl.compare.Sort) Aggregate(uk.gov.gchq.gaffer.operation.impl.function.Aggregate) LinkedHashSet(java.util.LinkedHashSet) CancelScheduledJobHandler(uk.gov.gchq.gaffer.store.operation.handler.job.CancelScheduledJobHandler) Properties(java.util.Properties) OperationUtil(uk.gov.gchq.gaffer.store.operation.OperationUtil) ToMap(uk.gov.gchq.gaffer.operation.impl.output.ToMap) GetWalks(uk.gov.gchq.gaffer.operation.impl.GetWalks) ToSet(uk.gov.gchq.gaffer.operation.impl.output.ToSet) AddStorePropertiesToLibraryHandler(uk.gov.gchq.gaffer.store.operation.handler.AddStorePropertiesToLibraryHandler) IdentifierType(uk.gov.gchq.gaffer.data.element.IdentifierType) ToSingletonListHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToSingletonListHandler) MinHandler(uk.gov.gchq.gaffer.store.operation.handler.compare.MinHandler) Operation(uk.gov.gchq.gaffer.operation.Operation) Schema(uk.gov.gchq.gaffer.store.schema.Schema) GenerateObjects(uk.gov.gchq.gaffer.operation.impl.generate.GenerateObjects) OperationChainHandler(uk.gov.gchq.gaffer.store.operation.handler.OperationChainHandler) AddStorePropertiesToLibrary(uk.gov.gchq.gaffer.store.operation.add.AddStorePropertiesToLibrary) OperationHandler(uk.gov.gchq.gaffer.store.operation.handler.OperationHandler) JoinHandler(uk.gov.gchq.gaffer.store.operation.handler.join.JoinHandler) GetTraitsHandler(uk.gov.gchq.gaffer.store.operation.handler.GetTraitsHandler) GenerateElementsHandler(uk.gov.gchq.gaffer.store.operation.handler.generate.GenerateElementsHandler) Job(uk.gov.gchq.gaffer.jobtracker.Job) GetJobDetailsHandler(uk.gov.gchq.gaffer.store.operation.handler.job.GetJobDetailsHandler) LoggerFactory(org.slf4j.LoggerFactory) ToCsvHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToCsvHandler) ToCsv(uk.gov.gchq.gaffer.operation.impl.output.ToCsv) WhileHandler(uk.gov.gchq.gaffer.store.operation.handler.WhileHandler) GetVariableHandler(uk.gov.gchq.gaffer.store.operation.handler.GetVariableHandler) OperationDeclarations(uk.gov.gchq.gaffer.store.operation.declaration.OperationDeclarations) Filter(uk.gov.gchq.gaffer.operation.impl.function.Filter) CloseableIterable(uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable) GetAllNamedViewsHandler(uk.gov.gchq.gaffer.store.operation.handler.named.GetAllNamedViewsHandler) OperationChain(uk.gov.gchq.gaffer.operation.OperationChain) ToStream(uk.gov.gchq.gaffer.operation.impl.output.ToStream) MapHandler(uk.gov.gchq.gaffer.store.operation.handler.MapHandler) GetAllNamedOperationsHandler(uk.gov.gchq.gaffer.store.operation.handler.named.GetAllNamedOperationsHandler) CacheServiceLoader(uk.gov.gchq.gaffer.cache.CacheServiceLoader) TypeDefinition(uk.gov.gchq.gaffer.store.schema.TypeDefinition) AddNamedOperation(uk.gov.gchq.gaffer.named.operation.AddNamedOperation) GetSetExport(uk.gov.gchq.gaffer.operation.impl.export.set.GetSetExport) GetTraits(uk.gov.gchq.gaffer.store.operation.GetTraits) GenerateObjectsHandler(uk.gov.gchq.gaffer.store.operation.handler.generate.GenerateObjectsHandler) GetJobDetails(uk.gov.gchq.gaffer.operation.impl.job.GetJobDetails) CountGroups(uk.gov.gchq.gaffer.operation.impl.CountGroups) ToSetHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToSetHandler) GetExports(uk.gov.gchq.gaffer.operation.impl.export.GetExports) Join(uk.gov.gchq.gaffer.operation.impl.join.Join) List(java.util.List) AddSchemaToLibrary(uk.gov.gchq.gaffer.store.operation.add.AddSchemaToLibrary) ToVerticesHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToVerticesHandler) ReduceHandler(uk.gov.gchq.gaffer.store.operation.handler.ReduceHandler) CloseableUtil(uk.gov.gchq.gaffer.commonutil.CloseableUtil) OperationException(uk.gov.gchq.gaffer.operation.OperationException) GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) CountHandler(uk.gov.gchq.gaffer.store.operation.handler.CountHandler) DeleteNamedOperationHandler(uk.gov.gchq.gaffer.store.operation.handler.named.DeleteNamedOperationHandler) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) SchemaElementDefinition(uk.gov.gchq.gaffer.store.schema.SchemaElementDefinition) ToList(uk.gov.gchq.gaffer.operation.impl.output.ToList) Operations(uk.gov.gchq.gaffer.operation.Operations) JSONSerialiser(uk.gov.gchq.gaffer.jsonserialisation.JSONSerialiser) GetSetExportHandler(uk.gov.gchq.gaffer.store.operation.handler.export.set.GetSetExportHandler) AggregateHandler(uk.gov.gchq.gaffer.store.operation.handler.function.AggregateHandler) Serialiser(uk.gov.gchq.gaffer.serialisation.Serialiser) GetAllJobDetailsHandler(uk.gov.gchq.gaffer.store.operation.handler.job.GetAllJobDetailsHandler) User(uk.gov.gchq.gaffer.user.User) CancelScheduledJob(uk.gov.gchq.gaffer.operation.impl.job.CancelScheduledJob) GetWalksHandler(uk.gov.gchq.gaffer.store.operation.handler.GetWalksHandler) DeleteNamedView(uk.gov.gchq.gaffer.named.view.DeleteNamedView) Limit(uk.gov.gchq.gaffer.operation.impl.Limit) HashMap(java.util.HashMap) GetJobResults(uk.gov.gchq.gaffer.operation.impl.job.GetJobResults) ExportToSetHandler(uk.gov.gchq.gaffer.store.operation.handler.export.set.ExportToSetHandler) Input(uk.gov.gchq.gaffer.operation.io.Input) NamedOperationHandler(uk.gov.gchq.gaffer.store.operation.handler.named.NamedOperationHandler) OperationChainOptimiser(uk.gov.gchq.gaffer.store.optimiser.OperationChainOptimiser) GraphLibrary(uk.gov.gchq.gaffer.store.library.GraphLibrary) ToMapHandler(uk.gov.gchq.gaffer.store.operation.handler.output.ToMapHandler) Output(uk.gov.gchq.gaffer.operation.io.Output) While(uk.gov.gchq.gaffer.operation.impl.While) GetAllElements(uk.gov.gchq.gaffer.operation.impl.get.GetAllElements) EntityId(uk.gov.gchq.gaffer.data.element.id.EntityId) Logger(org.slf4j.Logger) JobDetail(uk.gov.gchq.gaffer.jobtracker.JobDetail) JobStatus(uk.gov.gchq.gaffer.jobtracker.JobStatus) TransformHandler(uk.gov.gchq.gaffer.store.operation.handler.function.TransformHandler) ValidateHandler(uk.gov.gchq.gaffer.store.operation.handler.ValidateHandler) ExecutorService(uk.gov.gchq.gaffer.commonutil.ExecutorService) GetAdjacentIds(uk.gov.gchq.gaffer.operation.impl.get.GetAdjacentIds) OperationChainValidator(uk.gov.gchq.gaffer.store.operation.OperationChainValidator) AddNamedView(uk.gov.gchq.gaffer.named.view.AddNamedView) Reduce(uk.gov.gchq.gaffer.operation.impl.Reduce) GetSchemaHandler(uk.gov.gchq.gaffer.store.operation.handler.GetSchemaHandler) GetVariablesHandler(uk.gov.gchq.gaffer.store.operation.handler.GetVariablesHandler) Min(uk.gov.gchq.gaffer.operation.impl.compare.Min) SchemaException(uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException) JSONSerialiser(uk.gov.gchq.gaffer.jsonserialisation.JSONSerialiser) Serialiser(uk.gov.gchq.gaffer.serialisation.Serialiser) ValidationResult(uk.gov.gchq.koryphe.ValidationResult)

Example 7 with GetSchema

use of uk.gov.gchq.gaffer.store.operation.GetSchema in project Gaffer by gchq.

the class FederatedGraphStorage method getSchema.

public Schema getSchema(final Map<String, String> config, final User user) {
    if (null == user) {
        // no user then return an empty schema
        return new Schema();
    }
    final List<String> graphIds = FederatedStoreUtil.getGraphIds(config);
    final Stream<Graph> graphs = getStream(user, graphIds);
    final Builder schemaBuilder = new Builder();
    try {
        graphs.forEach(g -> schemaBuilder.merge(g.getSchema()));
    } catch (final SchemaException e) {
        final List<String> resultGraphIds = getStream(user, graphIds).map(Graph::getGraphId).collect(Collectors.toList());
        throw new SchemaException(String.format(UNABLE_TO_MERGE_THE_SCHEMAS_FOR_ALL_OF_YOUR_FEDERATED_GRAPHS, resultGraphIds, KEY_OPERATION_OPTIONS_GRAPH_IDS), e);
    }
    return schemaBuilder.build();
}
Also used : SchemaException(uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException) Graph(uk.gov.gchq.gaffer.graph.Graph) Schema(uk.gov.gchq.gaffer.store.schema.Schema) GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) Builder(uk.gov.gchq.gaffer.store.schema.Schema.Builder) List(java.util.List)

Example 8 with GetSchema

use of uk.gov.gchq.gaffer.store.operation.GetSchema in project Gaffer by gchq.

the class FederatedGraphStorage method getSchema.

public Schema getSchema(final GetSchema operation, final Context context) {
    if (null == context || null == context.getUser()) {
        // no user then return an empty schema
        return new Schema();
    }
    if (null == operation) {
        return getSchema((Map<String, String>) null, context);
    }
    final List<String> graphIds = FederatedStoreUtil.getGraphIds(operation.getOptions());
    final Stream<Graph> graphs = getStream(context.getUser(), graphIds);
    final Builder schemaBuilder = new Builder();
    try {
        if (operation.isCompact()) {
            final GetSchema getSchema = new GetSchema.Builder().compact(true).build();
            graphs.forEach(g -> {
                try {
                    schemaBuilder.merge(g.execute(getSchema, context));
                } catch (final OperationException e) {
                    throw new RuntimeException("Unable to fetch schema from graph " + g.getGraphId(), e);
                }
            });
        } else {
            graphs.forEach(g -> schemaBuilder.merge(g.getSchema()));
        }
    } catch (final SchemaException e) {
        final List<String> resultGraphIds = getStream(context.getUser(), graphIds).map(Graph::getGraphId).collect(Collectors.toList());
        throw new SchemaException("Unable to merge the schemas for all of your federated graphs: " + resultGraphIds + ". You can limit which graphs to query for using the operation option: " + KEY_OPERATION_OPTIONS_GRAPH_IDS, e);
    }
    return schemaBuilder.build();
}
Also used : SchemaException(uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException) Graph(uk.gov.gchq.gaffer.graph.Graph) Schema(uk.gov.gchq.gaffer.store.schema.Schema) GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) Builder(uk.gov.gchq.gaffer.store.schema.Schema.Builder) List(java.util.List) GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) CacheOperationException(uk.gov.gchq.gaffer.cache.exception.CacheOperationException) OperationException(uk.gov.gchq.gaffer.operation.OperationException)

Example 9 with GetSchema

use of uk.gov.gchq.gaffer.store.operation.GetSchema in project Gaffer by gchq.

the class FederatedGetSchemaHandlerTest method shouldReturnSchemaOnlyForEnabledGraphs.

@Test
public void shouldReturnSchemaOnlyForEnabledGraphs() throws OperationException {
    library.addProperties(ACC_PROP_ID, PROPERTIES);
    fStore.setGraphLibrary(library);
    final Schema edgeSchema1 = new Schema.Builder().edge("edge", new SchemaEdgeDefinition.Builder().source("string").destination("string").property("prop1", "string").directed(DIRECTED_EITHER).build()).vertexSerialiser(new StringSerialiser()).type(DIRECTED_EITHER, Boolean.class).merge(STRING_SCHEMA).build();
    library.addSchema("edgeSchema1", edgeSchema1);
    final Schema edgeSchema2 = new Schema.Builder().edge("edge", new SchemaEdgeDefinition.Builder().source("string").destination("string").property("prop2", "string").directed(DIRECTED_EITHER).build()).vertexSerialiser(new StringSerialiser()).type(DIRECTED_EITHER, Boolean.class).merge(STRING_SCHEMA).build();
    library.addSchema("edgeSchema2", edgeSchema2);
    fStore.execute(Operation.asOperationChain(new AddGraph.Builder().graphId("schemaEnabled").parentPropertiesId(ACC_PROP_ID).parentSchemaIds(Lists.newArrayList("edgeSchema1")).disabledByDefault(false).build()), context);
    fStore.execute(Operation.asOperationChain(new AddGraph.Builder().graphId("schemaDisabled").parentPropertiesId(ACC_PROP_ID).parentSchemaIds(Lists.newArrayList("edgeSchema2")).disabledByDefault(true).build()), context);
    final GetSchema operation = new GetSchema.Builder().compact(true).build();
    // When
    final Schema result = handler.doOperation(operation, context, fStore);
    // Then
    assertNotNull(result);
    JsonAssert.assertEquals(edgeSchema1.toJson(true), result.toJson(true));
}
Also used : AddGraph(uk.gov.gchq.gaffer.federatedstore.operation.AddGraph) StringSerialiser(uk.gov.gchq.gaffer.serialisation.implementation.StringSerialiser) Schema(uk.gov.gchq.gaffer.store.schema.Schema) GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) Test(org.junit.jupiter.api.Test)

Example 10 with GetSchema

use of uk.gov.gchq.gaffer.store.operation.GetSchema in project Gaffer by gchq.

the class FederatedGetSchemaHandlerTest method shouldThrowExceptionForANullOperation.

@Test
public void shouldThrowExceptionForANullOperation() throws OperationException {
    library.addProperties(ACC_PROP_ID, PROPERTIES);
    fStore.setGraphLibrary(library);
    final GetSchema operation = null;
    try {
        handler.doOperation(operation, context, fStore);
    } catch (final OperationException e) {
        assertTrue(e.getMessage().contains("Operation cannot be null"));
    }
}
Also used : GetSchema(uk.gov.gchq.gaffer.store.operation.GetSchema) OperationException(uk.gov.gchq.gaffer.operation.OperationException) Test(org.junit.jupiter.api.Test)

Aggregations

GetSchema (uk.gov.gchq.gaffer.store.operation.GetSchema)10 Schema (uk.gov.gchq.gaffer.store.schema.Schema)7 Test (org.junit.jupiter.api.Test)5 List (java.util.List)3 SchemaException (uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException)3 OperationException (uk.gov.gchq.gaffer.operation.OperationException)3 Graph (uk.gov.gchq.gaffer.graph.Graph)2 Builder (uk.gov.gchq.gaffer.store.schema.Schema.Builder)2 SuppressFBWarnings (edu.umd.cs.findbugs.annotations.SuppressFBWarnings)1 ArrayList (java.util.ArrayList)1 Collections.unmodifiableList (java.util.Collections.unmodifiableList)1 HashMap (java.util.HashMap)1 LinkedHashMap (java.util.LinkedHashMap)1 LinkedHashSet (java.util.LinkedHashSet)1 Map (java.util.Map)1 Properties (java.util.Properties)1 Set (java.util.Set)1 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)1 StreamSupport (java.util.stream.StreamSupport)1 Logger (org.slf4j.Logger)1