Search in sources :

Example 11 with EqualTo

use of org.apache.spark.sql.sources.EqualTo in project Gaffer by gchq.

the class FilterToOperationConverterTest method testSpecifyDestination.

@Test
public void testSpecifyDestination() throws OperationException {
    final Schema schema = getSchema();
    final SQLContext sqlContext = getSqlContext("testSpecifyDestination");
    final Filter[] filters = new Filter[1];
    filters[0] = new EqualTo(SchemaToStructTypeConverter.DST_COL_NAME, "0");
    final FiltersToOperationConverter converter = new FiltersToOperationConverter(sqlContext, getViewFromSchema(schema), schema, filters);
    final AbstractGetRDD<?> operation = converter.getOperation();
    assertTrue(operation instanceof GetRDDOfElements);
    assertEquals(0, operation.getView().getEntityGroups().size());
    assertEquals(EDGE_GROUPS, operation.getView().getEdgeGroups());
    final Set<EntitySeed> seeds = new HashSet<>();
    for (final Object seed : ((GetRDDOfElements) operation).getSeeds()) {
        seeds.add((EntitySeed) seed);
    }
    assertEquals(Collections.singleton(new EntitySeed("0")), seeds);
    sqlContext.sparkContext().stop();
}
Also used : Filter(org.apache.spark.sql.sources.Filter) Schema(uk.gov.gchq.gaffer.store.schema.Schema) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) SQLContext(org.apache.spark.sql.SQLContext) EqualTo(org.apache.spark.sql.sources.EqualTo) GetRDDOfElements(uk.gov.gchq.gaffer.spark.operation.scalardd.GetRDDOfElements) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 12 with EqualTo

use of org.apache.spark.sql.sources.EqualTo in project Gaffer by gchq.

the class FilterToOperationConverterTest method testTwoGroups.

@Test
public void testTwoGroups() throws OperationException {
    final Schema schema = getSchema();
    final SQLContext sqlContext = getSqlContext("testTwoGroups");
    final Filter[] filters = new Filter[1];
    final Filter left = new EqualTo(SchemaToStructTypeConverter.GROUP, ENTITY_GROUP);
    final Filter right = new EqualTo(SchemaToStructTypeConverter.GROUP, EDGE_GROUP2);
    filters[0] = new Or(left, right);
    final FiltersToOperationConverter converter = new FiltersToOperationConverter(sqlContext, getViewFromSchema(schema), schema, filters);
    final AbstractGetRDD<?> operation = converter.getOperation();
    assertTrue(operation instanceof GetRDDOfAllElements);
    assertEquals(Collections.singleton(ENTITY_GROUP), operation.getView().getEntityGroups());
    assertEquals(Collections.singleton(EDGE_GROUP2), operation.getView().getEdgeGroups());
    sqlContext.sparkContext().stop();
}
Also used : Or(org.apache.spark.sql.sources.Or) Filter(org.apache.spark.sql.sources.Filter) GetRDDOfAllElements(uk.gov.gchq.gaffer.spark.operation.scalardd.GetRDDOfAllElements) Schema(uk.gov.gchq.gaffer.store.schema.Schema) SQLContext(org.apache.spark.sql.SQLContext) EqualTo(org.apache.spark.sql.sources.EqualTo) Test(org.junit.Test)

Aggregations

EqualTo (org.apache.spark.sql.sources.EqualTo)12 Filter (org.apache.spark.sql.sources.Filter)10 SQLContext (org.apache.spark.sql.SQLContext)9 Test (org.junit.Test)9 Schema (uk.gov.gchq.gaffer.store.schema.Schema)9 HashSet (java.util.HashSet)7 EntitySeed (uk.gov.gchq.gaffer.operation.data.EntitySeed)6 GetRDDOfElements (uk.gov.gchq.gaffer.spark.operation.scalardd.GetRDDOfElements)6 ArrayList (java.util.ArrayList)3 GreaterThan (org.apache.spark.sql.sources.GreaterThan)3 LessThan (org.apache.spark.sql.sources.LessThan)3 View (uk.gov.gchq.gaffer.data.elementdefinition.view.View)3 ConsumerFunctionContext (uk.gov.gchq.gaffer.function.context.ConsumerFunctionContext)3 IsLessThan (uk.gov.gchq.gaffer.function.filter.IsLessThan)3 IsMoreThan (uk.gov.gchq.gaffer.function.filter.IsMoreThan)3 GetRDDOfAllElements (uk.gov.gchq.gaffer.spark.operation.scalardd.GetRDDOfAllElements)3 Set (java.util.Set)2 In (org.apache.spark.sql.sources.In)2 Or (org.apache.spark.sql.sources.Or)2 FilterFunction (uk.gov.gchq.gaffer.function.FilterFunction)2