Search in sources :

Example 1 with RyaTypeWritable

use of org.apache.rya.accumulo.mr.RyaTypeWritable in project incubator-rya by apache.

the class GraphXGraphGenerator method createGraph.

public Graph<RyaTypeWritable, RyaTypeWritable> createGraph(SparkContext sc, Configuration conf) throws IOException, AccumuloSecurityException {
    StorageLevel storageLvl1 = StorageLevel.MEMORY_ONLY();
    StorageLevel storageLvl2 = StorageLevel.MEMORY_ONLY();
    ClassTag<RyaTypeWritable> RTWTag = ClassTag$.MODULE$.apply(RyaTypeWritable.class);
    RyaTypeWritable rtw = null;
    RDD<Tuple2<Object, RyaTypeWritable>> vertexRDD = getVertexRDD(sc, conf);
    RDD<Tuple2<Object, Edge>> edgeRDD = getEdgeRDD(sc, conf);
    JavaRDD<Tuple2<Object, Edge>> jrddTuple = edgeRDD.toJavaRDD();
    JavaRDD<Edge<RyaTypeWritable>> jrdd = jrddTuple.map(tuple -> tuple._2);
    RDD<Edge<RyaTypeWritable>> goodERDD = JavaRDD.toRDD(jrdd);
    return Graph.apply(vertexRDD, goodERDD, rtw, storageLvl1, storageLvl2, RTWTag, RTWTag);
}
Also used : Tuple2(scala.Tuple2) RyaTypeWritable(org.apache.rya.accumulo.mr.RyaTypeWritable) Edge(org.apache.spark.graphx.Edge) StorageLevel(org.apache.spark.storage.StorageLevel)

Example 2 with RyaTypeWritable

use of org.apache.rya.accumulo.mr.RyaTypeWritable in project incubator-rya by apache.

the class RyaVertexReader method getCurrentVertex.

@Override
public Vertex<Text, RyaTypeWritable, RyaStatementWritable> getCurrentVertex() throws IOException, InterruptedException {
    RyaStatementWritable currentStatement = reader.getCurrentValue();
    RyaStatement ryaStatement = currentStatement.getRyaStatement();
    RyaTypeWritable vertexWritable = new RyaTypeWritable();
    vertexWritable.setRyaType(ryaStatement.getSubject());
    Text vertexId = new Text(ryaStatement.getSubject().getData());
    Vertex<Text, RyaTypeWritable, RyaStatementWritable> vertex = classesConfiguration.createVertex();
    Edge<Text, RyaStatementWritable> edge = EdgeFactory.create(new Text(ryaStatement.toString()), currentStatement);
    List<Edge<Text, RyaStatementWritable>> edges = new ArrayList<Edge<Text, RyaStatementWritable>>();
    edges.add(edge);
    vertex.initialize(vertexId, vertexWritable, edges);
    return vertex;
}
Also used : RyaStatementWritable(org.apache.rya.accumulo.mr.RyaStatementWritable) ArrayList(java.util.ArrayList) RyaStatement(org.apache.rya.api.domain.RyaStatement) Text(org.apache.hadoop.io.Text) RyaTypeWritable(org.apache.rya.accumulo.mr.RyaTypeWritable) Edge(org.apache.giraph.edge.Edge)

Aggregations

RyaTypeWritable (org.apache.rya.accumulo.mr.RyaTypeWritable)2 ArrayList (java.util.ArrayList)1 Edge (org.apache.giraph.edge.Edge)1 Text (org.apache.hadoop.io.Text)1 RyaStatementWritable (org.apache.rya.accumulo.mr.RyaStatementWritable)1 RyaStatement (org.apache.rya.api.domain.RyaStatement)1 Edge (org.apache.spark.graphx.Edge)1 StorageLevel (org.apache.spark.storage.StorageLevel)1 Tuple2 (scala.Tuple2)1