Examples with CloseableIterable uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable used on opensource projects

Example 6 with CloseableIterable

use of uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable in project Gaffer by gchq.

the class JcsJobTrackerIT method shouldAddJobIdToJobTrackerWhenExecuteJob.

@Test
public void shouldAddJobIdToJobTrackerWhenExecuteJob() throws OperationException, IOException, InterruptedException {
    // Given
    final OperationChain<CloseableIterable<Edge>> opChain = new OperationChain<>(new GetAllEdges());
    final User user = new User("user01");
    // When
    JobDetail jobDetails = graph.executeJob(opChain, user);
    final String jobId = jobDetails.getJobId();
    final JobDetail expectedJobDetail = new JobDetail(jobId, user.getUserId(), opChain, JobStatus.FINISHED, null);
    expectedJobDetail.setStartTime(jobDetails.getStartTime());
    int count = 0;
    while (JobStatus.RUNNING == jobDetails.getStatus() && ++count < 20) {
        Thread.sleep(100);
        jobDetails = graph.execute(new GetJobDetails.Builder().jobId(jobId).build(), user);
    }
    expectedJobDetail.setEndTime(jobDetails.getEndTime());
    // Then
    assertEquals(expectedJobDetail, jobDetails);
}

Example 7 with CloseableIterable

use of uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable in project Gaffer by gchq.

the class LoadAndQuery6 method run.

public CloseableIterable<String> run() throws OperationException {
    // [user] Create a user
    // ---------------------------------------------------------
    final User user = new User("user01");
    // ---------------------------------------------------------
    // [graph] create a graph using our schema and store properties
    // ---------------------------------------------------------
    final Graph graph = new Graph.Builder().addSchemas(getSchemas()).storeProperties(getStoreProperties()).build();
    // ---------------------------------------------------------
    // [add] Create a data generator and add the edges to the graph using an operation chain consisting of:
    // generateElements - generating edges from the data (note these are directed edges)
    // addElements - add the edges to the graph
    // ---------------------------------------------------------
    final DataGenerator6 dataGenerator = new DataGenerator6();
    final List<String> data = DataUtils.loadData(getData());
    final OperationChain addOpChain = new OperationChain.Builder().first(new GenerateElements.Builder<String>().generator(dataGenerator).objects(data).build()).then(new AddElements()).build();
    graph.execute(addOpChain, user);
    // ---------------------------------------------------------
    // [get] Create and execute an operation chain consisting of 3 operations:
    //GetAdjacentEntitySeeds - starting at vertex 1 get all adjacent vertices (vertices at other end of outbound edges)
    //GetRelatedEdges - get outbound edges
    //GenerateObjects - convert the edges back into comma separated strings
    // ---------------------------------------------------------
    final OperationChain<CloseableIterable<String>> opChain = new OperationChain.Builder().first(new GetAdjacentEntitySeeds.Builder().addSeed(new EntitySeed("1")).inOutType(IncludeIncomingOutgoingType.OUTGOING).build()).then(new GetEdges.Builder<EntitySeed>().inOutType(IncludeIncomingOutgoingType.OUTGOING).build()).then(new GenerateObjects.Builder<Edge, String>().generator(dataGenerator).build()).build();
    final CloseableIterable<String> results = graph.execute(opChain, user);
    // ---------------------------------------------------------
    log("\nFiltered edges converted back into comma separated strings. The counts have been aggregated\n");
    for (final String result : results) {
        log("RESULT", result);
    }
    return results;
}

Example 8 with CloseableIterable

use of uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable in project Gaffer by gchq.

the class ExportIT method shouldExportResultsInSet.

@Test
public void shouldExportResultsInSet() throws OperationException, IOException {
    // Given
    final OperationChain<CloseableIterable<?>> exportOpChain = new Builder().first(new GetEdges.Builder<>().addSeed(new EntitySeed(SOURCE_DIR_0)).build()).then(new ExportToSet()).then(new GenerateObjects.Builder<Edge, EntitySeed>().generator(new EntitySeedExtractor()).build()).then(new GetEdges<>()).then(new ExportToSet()).then(new GetSetExport()).build();
    // When
    final CloseableIterable<?> export = graph.execute(exportOpChain, getUser());
    // Then
    assertEquals(2, Lists.newArrayList(export).size());
}

Example 9 with CloseableIterable

use of uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable in project Gaffer by gchq.

the class GeneratorsIT method shouldConvertToDomainObjects.

@Test
public void shouldConvertToDomainObjects() throws OperationException, UnsupportedEncodingException {
    // Given
    final OperationChain<CloseableIterable<DomainObject>> opChain = new OperationChain.Builder().first(new GetElements.Builder<>().addSeed(new EntitySeed(SOURCE_1)).build()).then(new GenerateObjects.Builder<Element, DomainObject>().generator(new BasicGenerator()).outputType(new TypeReference<CloseableIterable<DomainObject>>() {
    }).build()).build();
    // When
    final List<DomainObject> results = Lists.newArrayList(graph.execute(opChain, getUser()));
    final EntityDomainObject entityDomainObject = new EntityDomainObject(SOURCE_1, "3", null);
    final EdgeDomainObject edgeDomainObject = new EdgeDomainObject(SOURCE_1, DEST_1, false, 1, 1L);
    // Then
    assertNotNull(results);
    assertEquals(2, results.size());
    assertThat(results, IsCollectionContaining.hasItems(entityDomainObject, edgeDomainObject));
}

Example 10 with CloseableIterable

use of uk.gov.gchq.gaffer.commonutil.iterable.CloseableIterable in project Gaffer by gchq.

the class LoadAndQuery6Test method shouldReturnExpectedStringsViaJson.

@Test
public void shouldReturnExpectedStringsViaJson() throws OperationException, SerialisationException {
    // Given
    final User user01 = new User("user01");
    final JSONSerialiser serialiser = new JSONSerialiser();
    final OperationChain<?> addOpChain = serialiser.deserialise(StreamUtil.openStream(LoadAndQuery.class, RESOURCE_EXAMPLE_PREFIX + "json/load.json"), OperationChain.class);
    final OperationChain<CloseableIterable<String>> queryOpChain = serialiser.deserialise(StreamUtil.openStream(LoadAndQuery.class, RESOURCE_EXAMPLE_PREFIX + "json/query.json"), OperationChain.class);
    // Setup graph
    final Graph graph = new Graph.Builder().storeProperties(StreamUtil.openStream(LoadAndQuery.class, RESOURCE_PREFIX + "mockaccumulostore.properties")).addSchemas(StreamUtil.openStreams(LoadAndQuery.class, RESOURCE_EXAMPLE_PREFIX + "schema")).build();
    // When
    // Execute the add operation chain on the graph
    graph.execute(addOpChain, user01);
    // Execute the query operation on the graph.
    final CloseableIterable<String> results = graph.execute(queryOpChain, user01);
    // Then
    verifyResults(results);
}