Search in sources :

Example 1 with IsTrue

use of uk.gov.gchq.koryphe.impl.predicate.IsTrue in project gaffer-doc by gchq.

the class ExportToOtherGraphExample method simpleExportUsingGraphFromGraphLibrary.

public void simpleExportUsingGraphFromGraphLibrary() {
    // ---------------------------------------------------------
    // Setup the graphLibrary with an export graph
    final GraphLibrary graphLibrary = new FileGraphLibrary("target/graphLibrary");
    final AccumuloProperties exportStoreProperties = new AccumuloProperties();
    // set other store property config here.
    final Schema exportSchema = new Schema.Builder().edge("edge", new SchemaEdgeDefinition.Builder().source("int").destination("int").directed("true").property("count", "int").aggregate(false).build()).type("int", Integer.class).type("true", new TypeDefinition.Builder().clazz(Boolean.class).validateFunctions(new IsTrue()).build()).build();
    graphLibrary.addOrUpdate("exportGraphId", exportSchema, exportStoreProperties);
    final Graph graph = new Graph.Builder().config(StreamUtil.openStream(getClass(), "graphConfigWithLibrary.json")).addSchemas(StreamUtil.openStreams(getClass(), "operations/schema")).storeProperties(StreamUtil.openStream(getClass(), "mockaccumulostore.properties")).build();
    final OperationChain<Iterable<? extends Element>> opChain = new OperationChain.Builder().first(new GetAllElements.Builder().view(new View.Builder().edge("edge").build()).build()).then(new ExportToOtherGraph.Builder().graphId("exportGraphId").build()).build();
    // ---------------------------------------------------------
    showExample(opChain, "This example will export all Edges with group 'edge' to another existing graph 'exportGraphId' using a GraphLibrary." + "We demonstrate here that if we use a GraphLibrary, we can register a graph ID and reference it from the export operation. " + "This means the user does not have to proxy all the schema and store properties when they configure the export operation, they can just provide the ID.");
}
Also used : AccumuloProperties(uk.gov.gchq.gaffer.accumulostore.AccumuloProperties) Schema(uk.gov.gchq.gaffer.store.schema.Schema) Element(uk.gov.gchq.gaffer.data.element.Element) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) TypeDefinition(uk.gov.gchq.gaffer.store.schema.TypeDefinition) ExportToOtherGraph(uk.gov.gchq.gaffer.operation.export.graph.ExportToOtherGraph) Graph(uk.gov.gchq.gaffer.graph.Graph) FileGraphLibrary(uk.gov.gchq.gaffer.store.library.FileGraphLibrary) GraphLibrary(uk.gov.gchq.gaffer.store.library.GraphLibrary) FileGraphLibrary(uk.gov.gchq.gaffer.store.library.FileGraphLibrary) IsTrue(uk.gov.gchq.koryphe.impl.predicate.IsTrue) OperationChain(uk.gov.gchq.gaffer.operation.OperationChain) GetAllElements(uk.gov.gchq.gaffer.operation.impl.get.GetAllElements)

Example 2 with IsTrue

use of uk.gov.gchq.koryphe.impl.predicate.IsTrue in project gaffer-doc by gchq.

the class ExportToOtherGraphExample method exportToNewGraphBasedOnConfigFromGraphLibrary.

public void exportToNewGraphBasedOnConfigFromGraphLibrary() {
    // ---------------------------------------------------------
    // Setup the graphLibrary with a schema and store properties for exporting
    final GraphLibrary graphLibrary = new FileGraphLibrary("target/graphLibrary");
    final AccumuloProperties exportStoreProperties = new AccumuloProperties();
    // set other store property config here.
    graphLibrary.addProperties("exportStorePropertiesId", exportStoreProperties);
    final Schema exportSchema = new Schema.Builder().edge("edge", new SchemaEdgeDefinition.Builder().source("int").destination("int").directed("true").property("count", "int").aggregate(false).build()).type("int", Integer.class).type("true", new TypeDefinition.Builder().clazz(Boolean.class).validateFunctions(new IsTrue()).build()).build();
    graphLibrary.addSchema("exportSchemaId", exportSchema);
    final Graph graph = new Graph.Builder().config(StreamUtil.openStream(getClass(), "graphConfigWithLibrary.json")).addSchemas(StreamUtil.openStreams(getClass(), "operations/schema")).storeProperties(StreamUtil.openStream(getClass(), "mockaccumulostore.properties")).build();
    final OperationChain<Iterable<? extends Element>> opChain = new OperationChain.Builder().first(new GetAllElements.Builder().view(new View.Builder().edge("edge").build()).build()).then(new ExportToOtherGraph.Builder().graphId("newGraphId").parentSchemaIds("exportSchemaId").parentStorePropertiesId("exportStorePropertiesId").build()).build();
    // ---------------------------------------------------------
    showExample(opChain, "Similar to the previous example, this example will export all Edges with group 'edge' to another graph using a GraphLibrary. " + "But in this example we show that you can export to a new graph with id newGraphId by choosing any combination of schema and store properties registered in the GraphLibrary. " + "This is useful as a system administrator could register various different store properties, of different Accumulo/HBase clusters and a user could them just select which one to use by referring to the relevant store properties ID.");
}
Also used : AccumuloProperties(uk.gov.gchq.gaffer.accumulostore.AccumuloProperties) Schema(uk.gov.gchq.gaffer.store.schema.Schema) Element(uk.gov.gchq.gaffer.data.element.Element) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) TypeDefinition(uk.gov.gchq.gaffer.store.schema.TypeDefinition) ExportToOtherGraph(uk.gov.gchq.gaffer.operation.export.graph.ExportToOtherGraph) Graph(uk.gov.gchq.gaffer.graph.Graph) FileGraphLibrary(uk.gov.gchq.gaffer.store.library.FileGraphLibrary) GraphLibrary(uk.gov.gchq.gaffer.store.library.GraphLibrary) FileGraphLibrary(uk.gov.gchq.gaffer.store.library.FileGraphLibrary) IsTrue(uk.gov.gchq.koryphe.impl.predicate.IsTrue) OperationChain(uk.gov.gchq.gaffer.operation.OperationChain) GetAllElements(uk.gov.gchq.gaffer.operation.impl.get.GetAllElements)

Example 3 with IsTrue

use of uk.gov.gchq.koryphe.impl.predicate.IsTrue in project gaffer-doc by gchq.

the class TheBasics method run.

@Override
public CloseableIterable<? extends Element> run() throws OperationException, IOException {
    // [generate] Create some edges from the simple data file using our Road Use generator class
    // ---------------------------------------------------------
    final List<Element> elements = new ArrayList<>();
    final RoadUseElementGenerator dataGenerator = new RoadUseElementGenerator();
    for (final String line : IOUtils.readLines(StreamUtil.openStream(getClass(), dataPath))) {
        elements.add(dataGenerator._apply(line));
    }
    // ---------------------------------------------------------
    print("Elements generated from the data file.");
    for (final Element element : elements) {
        print("GENERATED_EDGES", element.toString());
    }
    print("");
    Graph graph;
    // [graph from files] Create a graph using config, schema and store properties files
    // ---------------------------------------------------------
    graph = new Graph.Builder().config(StreamUtil.openStream(getClass(), graphConfigPath)).addSchemas(StreamUtil.openStreams(getClass(), schemaPath)).storeProperties(StreamUtil.openStream(getClass(), storePropertiesPath)).build();
    // ---------------------------------------------------------
    // [graph] Create a graph using config, schema and store properties from java
    // ---------------------------------------------------------
    final GraphConfig config = new GraphConfig.Builder().graphId(getClass().getSimpleName()).build();
    final Schema schema = new Schema.Builder().edge("RoadUse", new SchemaEdgeDefinition.Builder().description("A directed edge representing vehicles moving from junction A to junction B.").source("junction").destination("junction").directed("true").property("count", "count.long").build()).type("junction", new TypeDefinition.Builder().description("A road junction represented by a String.").clazz(String.class).build()).type("count.long", new TypeDefinition.Builder().description("A long count that must be greater than or equal to 0.").clazz(Long.class).validateFunctions(new IsMoreThan(0L, true)).aggregateFunction(new Sum()).build()).type("true", new TypeDefinition.Builder().description("A simple boolean that must always be true.").clazz(Boolean.class).validateFunctions(new IsTrue()).build()).build();
    final AccumuloProperties properties = new AccumuloProperties();
    properties.setStoreClass(SingleUseMockAccumuloStore.class);
    properties.setInstance("instance1");
    properties.setZookeepers("zookeeper1");
    properties.setUser("user01");
    properties.setPassword("password");
    graph = new Graph.Builder().config(config).addSchema(schema).storeProperties(properties).build();
    // ---------------------------------------------------------
    // [user] Create a user
    // ---------------------------------------------------------
    final User user = new User("user01");
    // ---------------------------------------------------------
    // [add] Add the edges to the graph
    // ---------------------------------------------------------
    final AddElements addElements = new AddElements.Builder().input(elements).build();
    graph.execute(addElements, user);
    // ---------------------------------------------------------
    print("The elements have been added.");
    // [get] Get all the edges that contain the vertex "10"
    // ---------------------------------------------------------
    final GetElements query = new GetElements.Builder().input(new EntitySeed("10")).view(new View.Builder().edge("RoadUse").build()).build();
    final CloseableIterable<? extends Element> results = graph.execute(query, user);
    // ---------------------------------------------------------
    print("\nAll edges containing the vertex 10. The counts have been aggregated.");
    for (final Element e : results) {
        print("GET_ELEMENTS_RESULT", e.toString());
    }
    return results;
}
Also used : AddElements(uk.gov.gchq.gaffer.operation.impl.add.AddElements) User(uk.gov.gchq.gaffer.user.User) AccumuloProperties(uk.gov.gchq.gaffer.accumulostore.AccumuloProperties) Element(uk.gov.gchq.gaffer.data.element.Element) Schema(uk.gov.gchq.gaffer.store.schema.Schema) ArrayList(java.util.ArrayList) GetElements(uk.gov.gchq.gaffer.operation.impl.get.GetElements) TypeDefinition(uk.gov.gchq.gaffer.store.schema.TypeDefinition) GraphConfig(uk.gov.gchq.gaffer.graph.GraphConfig) Sum(uk.gov.gchq.koryphe.impl.binaryoperator.Sum) Graph(uk.gov.gchq.gaffer.graph.Graph) IsTrue(uk.gov.gchq.koryphe.impl.predicate.IsTrue) RoadUseElementGenerator(uk.gov.gchq.gaffer.doc.user.generator.RoadUseElementGenerator) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) SchemaEdgeDefinition(uk.gov.gchq.gaffer.store.schema.SchemaEdgeDefinition) IsMoreThan(uk.gov.gchq.koryphe.impl.predicate.IsMoreThan)

Example 4 with IsTrue

use of uk.gov.gchq.koryphe.impl.predicate.IsTrue in project gaffer-doc by gchq.

the class IsTrueExample method isTrue.

public void isTrue() {
    // ---------------------------------------------------------
    final IsTrue function = new IsTrue();
    // ---------------------------------------------------------
    runExample(function, null, true, false, null, "true");
}
Also used : IsTrue(uk.gov.gchq.koryphe.impl.predicate.IsTrue)

Example 5 with IsTrue

use of uk.gov.gchq.koryphe.impl.predicate.IsTrue in project Gaffer by gchq.

the class JavaPredicateToParquetPredicate method getPrimitiveFilter.

public FilterPredicate getPrimitiveFilter(final Predicate filterFunction, final String selection, final String group, final SchemaUtils schemaUtils) throws SerialisationException {
    // All supported filters will be in the if else statement below
    if (filterFunction instanceof IsEqual) {
        final IsEqual isEqual = (IsEqual) filterFunction;
        final Object[] parquetObjects = schemaUtils.getConverter(group).gafferObjectToParquetObjects(selection, isEqual.getControlValue());
        return getIsEqualFilter(selection, parquetObjects, group, schemaUtils);
    } else if (filterFunction instanceof IsLessThan) {
        final IsLessThan isLessThan = (IsLessThan) filterFunction;
        final Object[] parquetObjects = schemaUtils.getConverter(group).gafferObjectToParquetObjects(selection, isLessThan.getControlValue());
        if (isLessThan.getOrEqualTo()) {
            return getIsLessThanOrEqualToFilter(selection, parquetObjects, group, schemaUtils);
        }
        return getIsLessThanFilter(selection, parquetObjects, group, schemaUtils);
    } else if (filterFunction instanceof IsMoreThan) {
        final IsMoreThan isMoreThan = (IsMoreThan) filterFunction;
        final Object[] parquetObjects = schemaUtils.getConverter(group).gafferObjectToParquetObjects(selection, isMoreThan.getControlValue());
        if (isMoreThan.getOrEqualTo()) {
            return getIsMoreThanOrEqualToFilter(selection, parquetObjects, group, schemaUtils);
        }
        return getIsMoreThanFilter(selection, parquetObjects, group, schemaUtils);
    } else if (filterFunction instanceof IsTrue) {
        return eq(booleanColumn(selection), true);
    } else if (filterFunction instanceof IsFalse) {
        return eq(booleanColumn(selection), false);
    } else {
        fullyApplied = false;
        LOGGER.warn(filterFunction.getClass().getCanonicalName() + " is not a natively supported filter by the Parquet store, therefore execution will take longer to perform this filter.");
        return null;
    }
}
Also used : IsLessThan(uk.gov.gchq.koryphe.impl.predicate.IsLessThan) IsFalse(uk.gov.gchq.koryphe.impl.predicate.IsFalse) IsTrue(uk.gov.gchq.koryphe.impl.predicate.IsTrue) IsMoreThan(uk.gov.gchq.koryphe.impl.predicate.IsMoreThan) IsEqual(uk.gov.gchq.koryphe.impl.predicate.IsEqual)

Aggregations

IsTrue (uk.gov.gchq.koryphe.impl.predicate.IsTrue)7 Schema (uk.gov.gchq.gaffer.store.schema.Schema)5 TypeDefinition (uk.gov.gchq.gaffer.store.schema.TypeDefinition)5 Graph (uk.gov.gchq.gaffer.graph.Graph)4 AccumuloProperties (uk.gov.gchq.gaffer.accumulostore.AccumuloProperties)3 Element (uk.gov.gchq.gaffer.data.element.Element)3 SchemaEdgeDefinition (uk.gov.gchq.gaffer.store.schema.SchemaEdgeDefinition)3 View (uk.gov.gchq.gaffer.data.elementdefinition.view.View)2 GraphConfig (uk.gov.gchq.gaffer.graph.GraphConfig)2 OperationChain (uk.gov.gchq.gaffer.operation.OperationChain)2 ExportToOtherGraph (uk.gov.gchq.gaffer.operation.export.graph.ExportToOtherGraph)2 AddElements (uk.gov.gchq.gaffer.operation.impl.add.AddElements)2 GetAllElements (uk.gov.gchq.gaffer.operation.impl.get.GetAllElements)2 FileGraphLibrary (uk.gov.gchq.gaffer.store.library.FileGraphLibrary)2 GraphLibrary (uk.gov.gchq.gaffer.store.library.GraphLibrary)2 IsMoreThan (uk.gov.gchq.koryphe.impl.predicate.IsMoreThan)2 ArrayList (java.util.ArrayList)1 RoadUseElementGenerator (uk.gov.gchq.gaffer.doc.user.generator.RoadUseElementGenerator)1 FederatedStore (uk.gov.gchq.gaffer.federatedstore.FederatedStore)1 PredefinedFederatedStore (uk.gov.gchq.gaffer.federatedstore.PredefinedFederatedStore)1