Search in sources :

Example 31 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class StoreTest method shouldGetJobTracker.

@Test
public void shouldGetJobTracker() throws OperationException, ExecutionException, InterruptedException, StoreException {
    // Given
    final StoreProperties properties = mock(StoreProperties.class);
    given(properties.getJobTrackerClass()).willReturn("jobTrackerClass");
    final Store store = new StoreImpl();
    final Schema schema = new Schema();
    store.initialise(schema, properties);
    // When
    final JobTracker resultJobTracker = store.getJobTracker();
    // Then
    assertSame(jobTracker, resultJobTracker);
}
Also used : Schema(uk.gov.gchq.gaffer.store.schema.Schema) JobTracker(uk.gov.gchq.gaffer.jobtracker.JobTracker) Test(org.junit.Test)

Example 32 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class StoreTest method shouldReturnTrueWhenOperationSupported.

@Test
public void shouldReturnTrueWhenOperationSupported() throws Exception {
    // Given
    final Schema schema = createSchemaMock();
    final StoreProperties properties = mock(StoreProperties.class);
    final Validatable<Integer> validatable = mock(Validatable.class);
    final Map<String, String> options = mock(HashMap.class);
    final StoreImpl store = new StoreImpl();
    final int expectedNumberOfOperations = 15;
    given(validatable.isValidate()).willReturn(true);
    given(validatable.getOptions()).willReturn(options);
    given(validatableHandler.doOperation(validatable, context, store)).willReturn(expectedNumberOfOperations);
    store.initialise(schema, properties);
    // WHen
    final Set<Class<? extends Operation>> supportedOperations = store.getSupportedOperations();
    for (final Class<? extends Operation> operationClass : supportedOperations) {
        final boolean isOperationClassSupported = store.isSupported(operationClass);
        // Then
        assertTrue(isOperationClassSupported);
    }
}
Also used : Schema(uk.gov.gchq.gaffer.store.schema.Schema) GetOperation(uk.gov.gchq.gaffer.operation.GetOperation) Operation(uk.gov.gchq.gaffer.operation.Operation) Test(org.junit.Test)

Example 33 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class AccumuloStoreRelationTest method testBuildScanSpecifyColumnsFullView.

@Test
public void testBuildScanSpecifyColumnsFullView() throws OperationException, StoreException {
    final Schema schema = getSchema();
    final View view = getViewFromSchema(schema);
    final String[] requiredColumns = new String[] { "property1" };
    testBuildScanSpecifyColumnsWithView("testBuildScanSpecifyColumnsFullView", view, requiredColumns, e -> true);
}
Also used : Schema(uk.gov.gchq.gaffer.store.schema.Schema) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) Test(org.junit.Test)

Example 34 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class AccumuloStoreRelationTest method testBuildScanWithView.

private void testBuildScanWithView(final String name, final View view, final Predicate<Element> returnElement) throws OperationException, StoreException {
    // Given
    final SQLContext sqlContext = getSqlContext(name);
    final Schema schema = getSchema();
    final AccumuloProperties properties = AccumuloProperties.loadStoreProperties(AccumuloStoreRelationTest.class.getResourceAsStream("/store.properties"));
    final SingleUseMockAccumuloStore store = new SingleUseMockAccumuloStore();
    store.initialise(schema, properties);
    addElements(store);
    // When
    final AccumuloStoreRelation relation = new AccumuloStoreRelation(sqlContext, Collections.emptyList(), view, store, new User());
    final RDD<Row> rdd = relation.buildScan();
    final Row[] returnedElements = (Row[]) rdd.collect();
    // Then
    //  - Actual results are:
    final Set<Row> results = new HashSet<>();
    for (int i = 0; i < returnedElements.length; i++) {
        results.add(returnedElements[i]);
    }
    //  - Expected results are:
    final SchemaToStructTypeConverter schemaConverter = new SchemaToStructTypeConverter(schema, view, new ArrayList<>());
    final ConvertElementToRow elementConverter = new ConvertElementToRow(schemaConverter.getUsedProperties(), schemaConverter.getPropertyNeedsConversion(), schemaConverter.getConverterByProperty());
    final Set<Row> expectedRows = new HashSet<>();
    StreamSupport.stream(getElements().spliterator(), false).filter(returnElement).map(elementConverter::apply).forEach(expectedRows::add);
    assertEquals(expectedRows, results);
    sqlContext.sparkContext().stop();
}
Also used : SingleUseMockAccumuloStore(uk.gov.gchq.gaffer.accumulostore.SingleUseMockAccumuloStore) User(uk.gov.gchq.gaffer.user.User) AccumuloProperties(uk.gov.gchq.gaffer.accumulostore.AccumuloProperties) Schema(uk.gov.gchq.gaffer.store.schema.Schema) ConvertElementToRow(uk.gov.gchq.gaffer.spark.operation.dataframe.ConvertElementToRow) Row(org.apache.spark.sql.Row) SchemaToStructTypeConverter(uk.gov.gchq.gaffer.spark.operation.dataframe.converter.schema.SchemaToStructTypeConverter) SQLContext(org.apache.spark.sql.SQLContext) HashSet(java.util.HashSet) LinkedHashSet(java.util.LinkedHashSet) ConvertElementToRow(uk.gov.gchq.gaffer.spark.operation.dataframe.ConvertElementToRow)

Example 35 with Schema

use of uk.gov.gchq.gaffer.store.schema.Schema in project Gaffer by gchq.

the class AccumuloStoreRelationTest method testBuildScanSpecifyColumnsAndFiltersFullView.

@Test
public void testBuildScanSpecifyColumnsAndFiltersFullView() throws OperationException, StoreException {
    final Schema schema = getSchema();
    final View view = getViewFromSchema(schema);
    final String[] requiredColumns = new String[1];
    requiredColumns[0] = "property1";
    final Filter[] filters = new Filter[1];
    filters[0] = new GreaterThan("property1", 4);
    final Predicate<Element> returnElement = (Element element) -> ((Integer) element.getProperty("property1")) > 4;
    testBuildScanSpecifyColumnsAndFiltersWithView("testBuildScanSpecifyColumnsAndFiltersFullView", view, requiredColumns, filters, returnElement);
}
Also used : Filter(org.apache.spark.sql.sources.Filter) GreaterThan(org.apache.spark.sql.sources.GreaterThan) Schema(uk.gov.gchq.gaffer.store.schema.Schema) Element(uk.gov.gchq.gaffer.data.element.Element) View(uk.gov.gchq.gaffer.data.elementdefinition.view.View) Test(org.junit.Test)

Aggregations

Schema (uk.gov.gchq.gaffer.store.schema.Schema)86 Test (org.junit.Test)63 SQLContext (org.apache.spark.sql.SQLContext)14 AccumuloProperties (uk.gov.gchq.gaffer.accumulostore.AccumuloProperties)13 View (uk.gov.gchq.gaffer.data.elementdefinition.view.View)13 User (uk.gov.gchq.gaffer.user.User)13 HashSet (java.util.HashSet)12 Filter (org.apache.spark.sql.sources.Filter)12 EqualTo (org.apache.spark.sql.sources.EqualTo)9 MockAccumuloStore (uk.gov.gchq.gaffer.accumulostore.MockAccumuloStore)9 SingleUseMockAccumuloStore (uk.gov.gchq.gaffer.accumulostore.SingleUseMockAccumuloStore)9 Element (uk.gov.gchq.gaffer.data.element.Element)9 Store (uk.gov.gchq.gaffer.store.Store)9 Before (org.junit.Before)8 SchemaException (uk.gov.gchq.gaffer.data.elementdefinition.exception.SchemaException)8 Graph (uk.gov.gchq.gaffer.graph.Graph)8 OperationChain (uk.gov.gchq.gaffer.operation.OperationChain)7 SchemaEdgeDefinition (uk.gov.gchq.gaffer.store.schema.SchemaEdgeDefinition)7 InvocationTargetException (java.lang.reflect.InvocationTargetException)6 UnsupportedEncodingException (java.io.UnsupportedEncodingException)5