Search in sources :

Example 21 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class ElementValidatorTest method shouldReturnFalseWhenSchemaValidateWithInvalidElement.

@Test
public void shouldReturnFalseWhenSchemaValidateWithInvalidElement() {
    // Given
    final Schema schema = mock(Schema.class);
    final String group = TestGroups.EDGE;
    final Element elm = mock(Element.class);
    final SchemaElementDefinition elementDef = mock(SchemaElementDefinition.class);
    final ElementFilter filter = mock(ElementFilter.class);
    final boolean includeIsA = true;
    final ElementValidator validator = new ElementValidator(schema, includeIsA);
    given(elm.getGroup()).willReturn(group);
    given(schema.getElement(group)).willReturn(elementDef);
    given(elementDef.getValidator(includeIsA)).willReturn(filter);
    given(filter.filter(elm)).willReturn(false);
    // When
    final boolean isValid = validator.validate(elm);
    // Then
    assertFalse(isValid);
}
Also used : Schema(uk.gov.gchq.gaffer.store.schema.Schema) Element(uk.gov.gchq.gaffer.data.element.Element) ElementFilter(uk.gov.gchq.gaffer.data.element.function.ElementFilter) SchemaElementDefinition(uk.gov.gchq.gaffer.store.schema.SchemaElementDefinition) Test(org.junit.Test)

Example 22 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class ElementValidatorTest method shouldReturnTrueWhenSchemaValidateWithValidElement.

@Test
public void shouldReturnTrueWhenSchemaValidateWithValidElement() {
    // Given
    final Schema schema = mock(Schema.class);
    final String group = TestGroups.EDGE;
    final Element elm = mock(Element.class);
    final SchemaElementDefinition elementDef = mock(SchemaElementDefinition.class);
    final ElementFilter filter = mock(ElementFilter.class);
    final boolean includeIsA = true;
    final ElementValidator validator = new ElementValidator(schema, includeIsA);
    given(elm.getGroup()).willReturn(group);
    given(schema.getElement(group)).willReturn(elementDef);
    given(elementDef.getValidator(includeIsA)).willReturn(filter);
    given(filter.filter(elm)).willReturn(true);
    // When
    final boolean isValid = validator.validate(elm);
    // Then
    assertTrue(isValid);
}
Also used : Schema(uk.gov.gchq.gaffer.store.schema.Schema) Element(uk.gov.gchq.gaffer.data.element.Element) ElementFilter(uk.gov.gchq.gaffer.data.element.function.ElementFilter) SchemaElementDefinition(uk.gov.gchq.gaffer.store.schema.SchemaElementDefinition) Test(org.junit.Test)

Example 23 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class AccumuloStoreRelationTest method testBuildScanSpecifyColumnsFullView.

@Test
public void testBuildScanSpecifyColumnsFullView() throws OperationException, StoreException {
    final Schema schema = getSchema();
    final View view = getViewFromSchema(schema);
    final String[] requiredColumns = new String[] { "property1" };
    testBuildScanSpecifyColumnsWithView("testBuildScanSpecifyColumnsFullView", view, requiredColumns, e -> true);
}
Also used : Schema(uk.gov.gchq.gaffer.store.schema.Schema) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) Test(org.junit.Test)

Example 24 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class AccumuloStoreRelationTest method testBuildScanWithView.

private void testBuildScanWithView(final String name, final View view, final Predicate<Element> returnElement) throws OperationException, StoreException {
    // Given
    final SQLContext sqlContext = getSqlContext(name);
    final Schema schema = getSchema();
    final AccumuloProperties properties = AccumuloProperties.loadStoreProperties(AccumuloStoreRelationTest.class.getResourceAsStream("/store.properties"));
    final SingleUseMockAccumuloStore store = new SingleUseMockAccumuloStore();
    store.initialise(schema, properties);
    addElements(store);
    // When
    final AccumuloStoreRelation relation = new AccumuloStoreRelation(sqlContext, Collections.emptyList(), view, store, new User());
    final RDD<Row> rdd = relation.buildScan();
    final Row[] returnedElements = (Row[]) rdd.collect();
    // Then
    //  - Actual results are:
    final Set<Row> results = new HashSet<>();
    for (int i = 0; i < returnedElements.length; i++) {
        results.add(returnedElements[i]);
    }
    //  - Expected results are:
    final SchemaToStructTypeConverter schemaConverter = new SchemaToStructTypeConverter(schema, view, new ArrayList<>());
    final ConvertElementToRow elementConverter = new ConvertElementToRow(schemaConverter.getUsedProperties(), schemaConverter.getPropertyNeedsConversion(), schemaConverter.getConverterByProperty());
    final Set<Row> expectedRows = new HashSet<>();
    StreamSupport.stream(getElements().spliterator(), false).filter(returnElement).map(elementConverter::apply).forEach(expectedRows::add);
    assertEquals(expectedRows, results);
    sqlContext.sparkContext().stop();
}
Also used : SingleUseMockAccumuloStore(uk.gov.gchq.gaffer.accumulostore.SingleUseMockAccumuloStore) User(uk.gov.gchq.gaffer.user.User) AccumuloProperties(uk.gov.gchq.gaffer.accumulostore.AccumuloProperties) Schema(uk.gov.gchq.gaffer.store.schema.Schema) ConvertElementToRow(uk.gov.gchq.gaffer.spark.operation.dataframe.ConvertElementToRow) Row(org.apache.spark.sql.Row) SchemaToStructTypeConverter(uk.gov.gchq.gaffer.spark.operation.dataframe.converter.schema.SchemaToStructTypeConverter) SQLContext(org.apache.spark.sql.SQLContext) HashSet(java.util.HashSet) LinkedHashSet(java.util.LinkedHashSet) ConvertElementToRow(uk.gov.gchq.gaffer.spark.operation.dataframe.ConvertElementToRow)

Example 25 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class AccumuloStoreRelationTest method testBuildScanSpecifyColumnsAndFiltersFullView.

@Test
public void testBuildScanSpecifyColumnsAndFiltersFullView() throws OperationException, StoreException {
    final Schema schema = getSchema();
    final View view = getViewFromSchema(schema);
    final String[] requiredColumns = new String[1];
    requiredColumns[0] = "property1";
    final Filter[] filters = new Filter[1];
    filters[0] = new GreaterThan("property1", 4);
    final Predicate<Element> returnElement = (Element element) -> ((Integer) element.getProperty("property1")) > 4;
    testBuildScanSpecifyColumnsAndFiltersWithView("testBuildScanSpecifyColumnsAndFiltersFullView", view, requiredColumns, filters, returnElement);
}
Also used : Filter(org.apache.spark.sql.sources.Filter) GreaterThan(org.apache.spark.sql.sources.GreaterThan) Schema(uk.gov.gchq.gaffer.store.schema.Schema) Element(uk.gov.gchq.gaffer.data.element.Element) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) Test(org.junit.Test)

Aggregations

Schema (uk.gov.gchq.gaffer.store.schema.Schema)86 Test (org.junit.Test)63 SQLContext (org.apache.spark.sql.SQLContext)14 AccumuloProperties (uk.gov.gchq.gaffer.accumulostore.AccumuloProperties)13 View (uk.gov.gchq.gaffer.data.elementdefinition.view.View)13 User (uk.gov.gchq.gaffer.user.User)13 HashSet (java.util.HashSet)12 Filter (org.apache.spark.sql.sources.Filter)12 EqualTo (org.apache.spark.sql.sources.EqualTo)9 MockAccumuloStore (uk.gov.gchq.gaffer.accumulostore.MockAccumuloStore)9 SingleUseMockAccumuloStore (uk.gov.gchq.gaffer.accumulostore.SingleUseMockAccumuloStore)9 Element (uk.gov.gchq.gaffer.data.element.Element)9 Store (uk.gov.gchq.gaffer.store.Store)9 Before (org.junit.Before)8 SchemaException (uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException)8 Graph (uk.gov.gchq.gaffer.graph.Graph)8 OperationChain (uk.gov.gchq.gaffer.operation.OperationChain)7 SchemaEdgeDefinition (uk.gov.gchq.gaffer.store.schema.SchemaEdgeDefinition)7 InvocationTargetException (java.lang.reflect.InvocationTargetException)6 UnsupportedEncodingException (java.io.UnsupportedEncodingException)5