Search in sources :

Example 41 with User

use of uk.gov.gchq.gaffer.user.User in project Gaffer by gchq.

the class ArrayListStoreTest method shouldAddAndGetRelatedEntities.

@Test
public void shouldAddAndGetRelatedEntities() throws OperationException {
    final Graph graph = createGraph();
    addElementsToGraph(graph);
    //set up the operation to fetch the entities
    final OperationChain<CloseableIterable<SimpleEntityDataObject>> opChain = new OperationChain.Builder().first(new GetEntities.Builder<>().addSeed(new EdgeSeed(2, 1, false)).view(new View.Builder().entity(TestGroups.ENTITY, new ViewElementDefinition.Builder().preAggregationFilter(new ElementFilter.Builder().select(TestPropertyNames.INT).execute(new IsLessThan(2)).build()).build()).build()).build()).then(new GenerateObjects.Builder<Entity, SimpleEntityDataObject>().generator(new SimpleEntityGenerator()).build()).build();
    //now do the hop
    final CloseableIterable<SimpleEntityDataObject> results = graph.execute(opChain, new User());
    //check the results by converting our edges back into SimpleDataObjects
    if (!results.iterator().hasNext()) {
        fail("No results returned");
    } else {
        for (final SimpleEntityDataObject obj : results) {
            LOGGER.info(obj.toString());
        }
        final List<SimpleEntityDataObject> resultList = Lists.newArrayList(results);
        int index = 0;
        SimpleEntityDataObject obj = resultList.get(index++);
        assertEquals(1, obj.getId());
        assertEquals(1, obj.getVisibility());
        assertEquals("Red", obj.getProperties());
        obj = resultList.get(index);
        assertEquals(2, obj.getId());
        assertEquals(1, obj.getVisibility());
        assertEquals("Orange", obj.getProperties());
    }
    results.close();
}
Also used : SimpleEntityGenerator(uk.gov.gchq.gaffer.arrayliststore.data.generator.SimpleEntityGenerator) User(uk.gov.gchq.gaffer.user.User) CloseableIterable(uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable) ViewElementDefinition(uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) Graph(uk.gov.gchq.gaffer.graph.Graph) IsLessThan(uk.gov.gchq.gaffer.function.filter.IsLessThan) OperationChain(uk.gov.gchq.gaffer.operation.OperationChain) EdgeSeed(uk.gov.gchq.gaffer.operation.data.EdgeSeed) SimpleEntityDataObject(uk.gov.gchq.gaffer.arrayliststore.data.SimpleEntityDataObject) Test(org.junit.Test)

Example 42 with User

use of uk.gov.gchq.gaffer.user.User in project Gaffer by gchq.

the class AccumuloSingleIDRetrieverTest method testEntitySeedQueryEdgesAndEntities.

private void testEntitySeedQueryEdgesAndEntities(final AccumuloStore store) throws AccumuloException, StoreException {
    setupGraph(store, numEntries);
    final User user = new User();
    // Create set to query for
    final Set<ElementSeed> ids = new HashSet<>();
    for (int i = 0; i < numEntries; i++) {
        ids.add(new EntitySeed("" + i));
    }
    final View view = new View.Builder().edge(TestGroups.EDGE).entity(TestGroups.ENTITY).build();
    final GetElements<ElementSeed, ?> operation = new GetElements<>(view, ids);
    operation.setIncludeEntities(true);
    operation.setIncludeEdges(IncludeEdgeType.ALL);
    try {
        final AccumuloSingleIDRetriever retriever = new AccumuloSingleIDRetriever(store, operation, new User());
        assertEquals(numEntries * 3, Iterables.size(retriever));
    } catch (IteratorSettingException e) {
        fail("Unable to construct SingleID Retriever");
    }
//Should find both i-B and i-C edges and entities i
}
Also used : User(uk.gov.gchq.gaffer.user.User) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) ElementSeed(uk.gov.gchq.gaffer.operation.data.ElementSeed) GetElements(uk.gov.gchq.gaffer.operation.impl.get.GetElements) IteratorSettingException(uk.gov.gchq.gaffer.accumulostore.key.exception.IteratorSettingException) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) HashSet(java.util.HashSet)

Example 43 with User

use of uk.gov.gchq.gaffer.user.User in project Gaffer by gchq.

the class AccumuloSingleIDRetrieverTest method testEntitySeedQueryOutgoingEdgesOnly.

private void testEntitySeedQueryOutgoingEdgesOnly(final AccumuloStore store) throws AccumuloException, StoreException {
    setupGraph(store, numEntries);
    final User user = new User();
    // Create set to query for
    Set<ElementSeed> ids = new HashSet<>();
    for (int i = 0; i < numEntries; i++) {
        ids.add(new EntitySeed("" + i));
    }
    final View view = new View.Builder().edge(TestGroups.EDGE).entity(TestGroups.ENTITY).build();
    AccumuloSingleIDRetriever retriever = null;
    GetElements<ElementSeed, ?> operation = new GetElements<>(view, ids);
    operation.setIncludeEntities(false);
    operation.setIncludeIncomingOutGoing(IncludeIncomingOutgoingType.OUTGOING);
    try {
        retriever = new AccumuloSingleIDRetriever(store, operation, user);
    } catch (IteratorSettingException e) {
        e.printStackTrace();
    }
    int count = 0;
    for (final Element element : retriever) {
        count++;
        assertEquals(TestGroups.EDGE, element.getGroup());
    }
    //Should find both i-B and i-C edges.
    assertEquals(numEntries * 2, count);
}
Also used : User(uk.gov.gchq.gaffer.user.User) Element(uk.gov.gchq.gaffer.data.element.Element) GetElements(uk.gov.gchq.gaffer.operation.impl.get.GetElements) IteratorSettingException(uk.gov.gchq.gaffer.accumulostore.key.exception.IteratorSettingException) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) ElementSeed(uk.gov.gchq.gaffer.operation.data.ElementSeed) HashSet(java.util.HashSet)

Example 44 with User

use of uk.gov.gchq.gaffer.user.User in project Gaffer by gchq.

the class GetElementsinRangesHandlerTest method shouldHaveNoIncomingEdges.

private void shouldHaveNoIncomingEdges(final AccumuloStore store) throws OperationException {
    // Create set to query for
    final Set<Pair<ElementSeed>> simpleEntityRanges = new HashSet<>();
    final User user = new User();
    //get Everything between 0 and 1 (Note we are using strings and string serialisers, with this ordering 0999 is before 1)
    simpleEntityRanges.add(new Pair<ElementSeed>(new EntitySeed("0"), new EntitySeed("1")));
    final View view = new View.Builder(defaultView).entity(TestGroups.ENTITY, new ViewElementDefinition.Builder().groupBy().build()).edge(TestGroups.EDGE, new ViewElementDefinition.Builder().groupBy().build()).build();
    final GetElementsInRanges<Pair<ElementSeed>, Element> operation = new GetElementsInRanges<>(view, simpleEntityRanges);
    //All Edges stored should be outgoing from our provided seeds.
    operation.setIncludeIncomingOutGoing(IncludeIncomingOutgoingType.INCOMING);
    final GetElementsInRangesHandler handler = new GetElementsInRangesHandler();
    final CloseableIterable<Element> elements = handler.doOperation(operation, user, store);
    final int count = Iterables.size(elements);
    //There should be no incoming edges to the provided range
    assertEquals(0, count);
    elements.close();
}
Also used : User(uk.gov.gchq.gaffer.user.User) GetElementsInRanges(uk.gov.gchq.gaffer.accumulostore.operation.impl.GetElementsInRanges) Element(uk.gov.gchq.gaffer.data.element.Element) ViewElementDefinition(uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) ElementSeed(uk.gov.gchq.gaffer.operation.data.ElementSeed) HashSet(java.util.HashSet) Pair(uk.gov.gchq.gaffer.accumulostore.utils.Pair)

Example 45 with User

use of uk.gov.gchq.gaffer.user.User in project gaffer-doc by gchq.

the class FederatedStoreWalkThrough method run.

public CloseableIterable<? extends Element> run() throws Exception {
    final Schema schema = new Schema.Builder().json(StreamUtil.openStreams(getClass(), schemaPath)).build();
    final HashMapGraphLibrary library = new HashMapGraphLibrary();
    library.addProperties("mapStore", getMapStoreProperties());
    library.addProperties("accumuloStore", getAccumuloStoreProperties());
    library.addSchema("roadTraffic", schema);
    // [creating a federatedstore] create a store that federates to a MapStore and AccumuloStore
    // ---------------------------------------------------------
    final Graph federatedGraph = new Graph.Builder().config(new GraphConfig.Builder().graphId(getClass().getSimpleName()).library(library).build()).storeProperties(getFederatedStoreProperties()).build();
    // ---------------------------------------------------------
    // [federatedstore properties]
    // ---------------------------------------------------------
    final FederatedStoreProperties exampleFederatedProperty = new FederatedStoreProperties();
    exampleFederatedProperty.setCacheProperties(HashMapCacheService.class.getName());
    // ---------------------------------------------------------
    printJson("FED_PROP", getFederatedStoreProperties());
    final User user = new User("user01");
    final AddGraph addMapGraph = new AddGraph.Builder().graphId("mapGraph").schema(new Schema.Builder().json(StreamUtil.openStream(getClass(), "RoadAndRoadUseWithTimesAndCardinalitiesForFederatedStore/schema/entities.json")).json(StreamUtil.openStream(getClass(), "RoadAndRoadUseWithTimesAndCardinalitiesForFederatedStore/schema/types.json")).build()).storeProperties(getMapStoreProperties()).isPublic(true).build();
    federatedGraph.execute(addMapGraph, user);
    final AddGraph addAccumuloGraph = new AddGraph.Builder().graphId("accumuloGraph").schema(new Schema.Builder().json(StreamUtil.openStream(getClass(), "RoadAndRoadUseWithTimesAndCardinalitiesForFederatedStore/schema/edges.json")).json(StreamUtil.openStream(getClass(), "RoadAndRoadUseWithTimesAndCardinalitiesForFederatedStore/schema/types.json")).build()).storeProperties(getAccumuloStoreProperties()).isPublic(true).build();
    federatedGraph.execute(addAccumuloGraph, user);
    // [add another graph] add a graph to the federated store.
    // ---------------------------------------------------------
    AddGraph addAnotherGraph = new AddGraph.Builder().graphId("AnotherGraph").schema(schema).storeProperties(getMapStoreProperties()).build();
    federatedGraph.execute(addAnotherGraph, user);
    // ---------------------------------------------------------
    improveReadabilityOfJson(addAnotherGraph);
    addAccumuloGraph.setGraphAuths(null);
    printJson("ADD_GRAPH", addAnotherGraph);
    // [remove graph] remove a graph from the federated store.
    // ---------------------------------------------------------
    RemoveGraph removeGraph = new RemoveGraph.Builder().graphId("AnotherGraph").build();
    federatedGraph.execute(removeGraph, user);
    // ---------------------------------------------------------
    improveReadabilityOfJson(removeGraph);
    printJson("REMOVE_GRAPH", removeGraph);
    // [get all graph ids] Get a list of all the graphId within the FederatedStore.
    // ---------------------------------------------------------
    final GetAllGraphIds getAllGraphIDs = new GetAllGraphIds();
    Iterable<? extends String> graphIds = federatedGraph.execute(getAllGraphIDs, user);
    // ---------------------------------------------------------
    improveReadabilityOfJson(getAllGraphIDs);
    printJson("GET_ALL_GRAPH_IDS", getAllGraphIDs);
    print("GRAPH_IDS", graphIds.toString());
    // [add elements] Create a data generator and add the edges to the federated graphs using an operation chain consisting of:
    // generateElements - generating edges from the data (note these are directed edges)
    // addElements - add the edges to the graph
    // ---------------------------------------------------------
    final OperationChain<Void> addOpChain = new OperationChain.Builder().first(new GenerateElements.Builder<String>().generator(new RoadAndRoadUseWithTimesAndCardinalitiesElementGenerator()).input(IOUtils.readLines(StreamUtil.openStream(getClass(), "RoadAndRoadUseWithTimesAndCardinalities/data.txt"))).build()).then(new AddElements()).build();
    federatedGraph.execute(addOpChain, user);
    // ---------------------------------------------------------
    printJsonAndPython("ADD_ELEMENTS", addOpChain);
    // [get elements]
    // ---------------------------------------------------------
    final OperationChain<CloseableIterable<? extends Element>> getOpChain = new OperationChain.Builder().first(new GetElements.Builder().input(new EntitySeed("10")).build()).build();
    CloseableIterable<? extends Element> elements = federatedGraph.execute(getOpChain, user);
    for (final Element element : elements) {
        print("ELEMENTS", element.toString());
    }
    printJsonAndPython("GET_ELEMENTS", getOpChain);
    // [get elements from accumulo graph]
    // ---------------------------------------------------------
    final OperationChain<CloseableIterable<? extends Element>> getOpChainOnAccumuloGraph = new OperationChain.Builder().first(new GetElements.Builder().input(new EntitySeed("10")).option(FederatedStoreConstants.KEY_OPERATION_OPTIONS_GRAPH_IDS, "accumuloGraph").build()).build();
    CloseableIterable<? extends Element> elementsFromAccumuloGraph = federatedGraph.execute(getOpChainOnAccumuloGraph, user);
    for (final Element element : elementsFromAccumuloGraph) {
        print("ELEMENTS_FROM_ACCUMULO_GRAPH", element.toString());
    }
    // [select graphs for operations]
    // ---------------------------------------------------------
    GetAllElements selectGraphsForOperations = new Builder().option(FederatedStoreConstants.KEY_OPERATION_OPTIONS_GRAPH_IDS, "graphId1, graphId2").build();
    // ---------------------------------------------------------
    printJsonAndPython("SELECT_GRAPHS_FOR_OPERATIONS", selectGraphsForOperations);
    // [do not skip failed execution]
    // ---------------------------------------------------------
    GetAllElements doNotSkipFailedExecution = new Builder().option(FederatedStoreConstants.KEY_SKIP_FAILED_FEDERATED_STORE_EXECUTE, "false").build();
    // ---------------------------------------------------------
    printJsonAndPython("DO_NOT_SKIP_FAILED_EXECUTION", doNotSkipFailedExecution);
    // [add public graph] add a graph to the federated store.
    // ---------------------------------------------------------
    AddGraph publicGraph = new AddGraph.Builder().graphId("publicGraph").parentSchemaIds(Lists.newArrayList("roadTraffic")).parentPropertiesId("mapStore").isPublic(// <-- public access
    true).graphAuths(// <-- used but irrelevant as graph has public access
    "Auth1").build();
    federatedGraph.execute(addAnotherGraph, user);
    // ---------------------------------------------------------
    improveReadabilityOfJson(publicGraph);
    printJson("ADD_PUBLIC_GRAPH", publicGraph);
    // [add private graph] add a graph to the federated store.
    // ---------------------------------------------------------
    AddGraph privateGraph = new AddGraph.Builder().graphId("privateGraph").parentSchemaIds(Lists.newArrayList("roadTraffic")).parentPropertiesId("mapStore").build();
    federatedGraph.execute(addAnotherGraph, user);
    // ---------------------------------------------------------
    improveReadabilityOfJson(privateGraph);
    printJson("ADD_PRIVATE_GRAPH", privateGraph);
    // [add secure graph] add a graph to the federated store.
    // ---------------------------------------------------------
    AddGraph addSecureGraph = new AddGraph.Builder().graphId("SecureGraph").parentSchemaIds(Lists.newArrayList("roadTraffic")).parentPropertiesId("mapStore").graphAuths("Auth1", "Auth2", "Auth3").build();
    federatedGraph.execute(addSecureGraph, user);
    // ---------------------------------------------------------
    improveReadabilityOfJson(addSecureGraph);
    printJson("ADD_SECURE_GRAPH", addSecureGraph);
    // [disallow public access]
    // ---------------------------------------------------------
    FederatedStoreProperties disallowPublicProperties = new FederatedStoreProperties();
    disallowPublicProperties.setGraphsCanHavePublicAccess(false);
    // ---------------------------------------------------------
    printJson("DISALLOW_PUBLIC_ACCESS", disallowPublicProperties.getProperties());
    // [limit custom properties]
    // ---------------------------------------------------------
    FederatedStoreProperties limitCustomProperties = new FederatedStoreProperties();
    limitCustomProperties.setCustomPropertyAuths("Auth1, Auth2, Auth3");
    // ---------------------------------------------------------
    printJson("LIMIT_CUSTOM_PROPERTIES", limitCustomProperties.getProperties());
    return elements;
}
Also used : AddElements(uk.gov.gchq.gaffer.operation.impl.add.AddElements) User(uk.gov.gchq.gaffer.user.User) FederatedStoreProperties(uk.gov.gchq.gaffer.federatedstore.FederatedStoreProperties) Schema(uk.gov.gchq.gaffer.store.schema.Schema) Builder(uk.gov.gchq.gaffer.operation.impl.get.GetAllElements.Builder) Element(uk.gov.gchq.gaffer.data.element.Element) GetElements(uk.gov.gchq.gaffer.operation.impl.get.GetElements) HashMapCacheService(uk.gov.gchq.gaffer.cache.impl.HashMapCacheService) GetAllGraphIds(uk.gov.gchq.gaffer.federatedstore.operation.GetAllGraphIds) AddGraph(uk.gov.gchq.gaffer.federatedstore.operation.AddGraph) RoadAndRoadUseWithTimesAndCardinalitiesElementGenerator(uk.gov.gchq.gaffer.doc.user.generator.RoadAndRoadUseWithTimesAndCardinalitiesElementGenerator) GetAllElements(uk.gov.gchq.gaffer.operation.impl.get.GetAllElements) RemoveGraph(uk.gov.gchq.gaffer.federatedstore.operation.RemoveGraph) HashMapGraphLibrary(uk.gov.gchq.gaffer.store.library.HashMapGraphLibrary) CloseableIterable(uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable) Graph(uk.gov.gchq.gaffer.graph.Graph) RemoveGraph(uk.gov.gchq.gaffer.federatedstore.operation.RemoveGraph) AddGraph(uk.gov.gchq.gaffer.federatedstore.operation.AddGraph) OperationChain(uk.gov.gchq.gaffer.operation.OperationChain) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed)

Aggregations

User (uk.gov.gchq.gaffer.user.User)378 Test (org.junit.jupiter.api.Test)188 Graph (uk.gov.gchq.gaffer.graph.Graph)155 Element (uk.gov.gchq.gaffer.data.element.Element)143 AddElements (uk.gov.gchq.gaffer.operation.impl.add.AddElements)128 GetElements (uk.gov.gchq.gaffer.operation.impl.get.GetElements)110 HashSet (java.util.HashSet)109 View (uk.gov.gchq.gaffer.data.elementdefinition.view.View)104 OperationChain (uk.gov.gchq.gaffer.operation.OperationChain)103 EntitySeed (uk.gov.gchq.gaffer.operation.data.EntitySeed)98 Edge (uk.gov.gchq.gaffer.data.element.Edge)85 Context (uk.gov.gchq.gaffer.store.Context)85 Entity (uk.gov.gchq.gaffer.data.element.Entity)77 Test (org.junit.Test)61 ArrayList (java.util.ArrayList)57 OperationException (uk.gov.gchq.gaffer.operation.OperationException)52 GetAllElements (uk.gov.gchq.gaffer.operation.impl.get.GetAllElements)49 CloseableIterable (uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable)48 ViewElementDefinition (uk.gov.gchq.gaffer.data.elementdefinition.view.ViewElementDefinition)45 ElementFilter (uk.gov.gchq.gaffer.data.element.function.ElementFilter)43