Search in sources :

Example 6 with GetJavaRDDOfAllElements

use of uk.gov.gchq.gaffer.spark.operation.javardd.GetJavaRDDOfAllElements in project Gaffer by gchq.

the class GetJavaRDDOfAllElementsExample method getJavaRddOfAllElementsReturningEdgesOnly.

public void getJavaRddOfAllElementsReturningEdgesOnly(final JavaSparkContext sc, final Graph graph) throws OperationException {
    ROOT_LOGGER.setLevel(Level.INFO);
    log("#### get Java RDD of elements returning edges only\n");
    printGraph();
    ROOT_LOGGER.setLevel(Level.OFF);
    final GetJavaRDDOfAllElements operation = new GetJavaRDDOfAllElements.Builder().includeEntities(false).javaSparkContext(sc).build();
    final JavaRDD<Element> rdd = graph.execute(operation, new User("user01"));
    final List<Element> elements = rdd.collect();
    ROOT_LOGGER.setLevel(Level.INFO);
    printJava("GetJavaRDDOfAllElements<ElementSeed> operation = new GetJavaRDDOfAllElements.Builder<>()\n" + "                .includeEntities(false)\n" + "                .javaSparkContext(sc)\n" + "                .build();\n" + "JavaRDD<Element> rdd = graph.execute(operation, new User(\"user01\"));\n" + "List<Element> elements = rdd.collect();");
    log("The results are:");
    log("```");
    for (final Element e : elements) {
        log(e.toString());
    }
    log("```");
    ROOT_LOGGER.setLevel(Level.OFF);
}
Also used : User(uk.gov.gchq.gaffer.user.User) Element(uk.gov.gchq.gaffer.data.element.Element) GetJavaRDDOfAllElements(uk.gov.gchq.gaffer.spark.operation.javardd.GetJavaRDDOfAllElements)

Aggregations

Element (uk.gov.gchq.gaffer.data.element.Element)6 GetJavaRDDOfAllElements (uk.gov.gchq.gaffer.spark.operation.javardd.GetJavaRDDOfAllElements)6 User (uk.gov.gchq.gaffer.user.User)6 DataOutputStream (java.io.DataOutputStream)4 ArrayList (java.util.ArrayList)4 HashSet (java.util.HashSet)4 ByteArrayOutputStream (org.apache.commons.io.output.ByteArrayOutputStream)4 Configuration (org.apache.hadoop.conf.Configuration)4 SparkConf (org.apache.spark.SparkConf)4 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)4 Test (org.junit.Test)4 Edge (uk.gov.gchq.gaffer.data.element.Edge)4 Entity (uk.gov.gchq.gaffer.data.element.Entity)4 Graph (uk.gov.gchq.gaffer.graph.Graph)4 File (java.io.File)2 AddElements (uk.gov.gchq.gaffer.operation.impl.add.AddElements)2 Key (org.apache.accumulo.core.data.Key)1 Value (org.apache.accumulo.core.data.Value)1 ByteEntityAccumuloElementConverter (uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter)1 ImportJavaRDDOfElements (uk.gov.gchq.gaffer.spark.operation.javardd.ImportJavaRDDOfElements)1