Search in sources :

Example 56 with FreqMap

use of uk.gov.gchq.gaffer.types.FreqMap in project Gaffer by gchq.

the class AbstractAccumuloElementConverterTest method shouldDeserialiseEdgeId.

@Test
public void shouldDeserialiseEdgeId() {
    // Given
    final EdgeId expectedElementId = new EdgeSeed("source1", "dest1", true);
    final Edge edge = new Edge.Builder().source("source1").dest("dest1").directed(true).group(TestGroups.ENTITY).property(TestPropertyNames.PROP_1, new FreqMap()).property(TestPropertyNames.PROP_2, new FreqMap()).build();
    final Key key = converter.getKeysFromEdge(edge).getFirst();
    // When
    final ElementId elementId = converter.getElementId(key, false);
    // Then
    assertEquals(expectedElementId, elementId);
}
Also used : FreqMap(uk.gov.gchq.gaffer.types.FreqMap) EdgeId(uk.gov.gchq.gaffer.data.element.id.EdgeId) EdgeSeed(uk.gov.gchq.gaffer.operation.data.EdgeSeed) Edge(uk.gov.gchq.gaffer.data.element.Edge) Key(org.apache.accumulo.core.data.Key) ElementId(uk.gov.gchq.gaffer.data.element.id.ElementId) Test(org.junit.jupiter.api.Test)

Example 57 with FreqMap

use of uk.gov.gchq.gaffer.types.FreqMap in project Gaffer by gchq.

the class AbstractAccumuloElementConverterTest method shouldSerialiseAndDeserialisePropertiesWhenAllAreEmpty.

@Test
public void shouldSerialiseAndDeserialisePropertiesWhenAllAreEmpty() {
    // Given
    final Schema schema = new Schema.Builder().entity(TestGroups.ENTITY, new SchemaEntityDefinition.Builder().vertex("string").property(TestPropertyNames.PROP_1, "map").property(TestPropertyNames.PROP_2, "map").build()).type("string", String.class).type("map", new TypeDefinition.Builder().clazz(FreqMap.class).aggregateFunction(new FreqMapAggregator()).serialiser(new FreqMapSerialiser()).build()).build();
    converter = createConverter(schema);
    final Entity entity = new Entity.Builder().vertex("vertex1").property(TestPropertyNames.PROP_1, new FreqMap()).property(TestPropertyNames.PROP_2, new FreqMap()).build();
    // When 1
    final Value value = converter.getValueFromProperties(TestGroups.ENTITY, entity.getProperties());
    // Then 1
    assertTrue(value.getSize() > 0);
    // When 2
    final Properties properties = converter.getPropertiesFromValue(TestGroups.ENTITY, value);
    // Then 2
    assertEquals(entity.getProperties(), properties);
}
Also used : Entity(uk.gov.gchq.gaffer.data.element.Entity) FreqMap(uk.gov.gchq.gaffer.types.FreqMap) Schema(uk.gov.gchq.gaffer.store.schema.Schema) FreqMapAggregator(uk.gov.gchq.gaffer.types.function.FreqMapAggregator) Value(org.apache.accumulo.core.data.Value) Properties(uk.gov.gchq.gaffer.data.element.Properties) FreqMapSerialiser(uk.gov.gchq.gaffer.serialisation.FreqMapSerialiser) Test(org.junit.jupiter.api.Test)

Example 58 with FreqMap

use of uk.gov.gchq.gaffer.types.FreqMap in project Gaffer by gchq.

the class AbstractAccumuloElementConverterTest method shouldDeserialiseEntityId.

@Test
public void shouldDeserialiseEntityId() {
    // Given
    final EntityId expectedElementId = new EntitySeed("vertex1");
    final Entity entity = new Entity.Builder().vertex("vertex1").group(TestGroups.ENTITY).property(TestPropertyNames.PROP_1, new FreqMap()).property(TestPropertyNames.PROP_2, new FreqMap()).build();
    final Key key = converter.getKeyFromEntity(entity);
    // When
    final ElementId elementId = converter.getElementId(key, false);
    // Then
    assertEquals(expectedElementId, elementId);
}
Also used : EntityId(uk.gov.gchq.gaffer.data.element.id.EntityId) Entity(uk.gov.gchq.gaffer.data.element.Entity) FreqMap(uk.gov.gchq.gaffer.types.FreqMap) EntitySeed(uk.gov.gchq.gaffer.operation.data.EntitySeed) Key(org.apache.accumulo.core.data.Key) ElementId(uk.gov.gchq.gaffer.data.element.id.ElementId) Test(org.junit.jupiter.api.Test)

Example 59 with FreqMap

use of uk.gov.gchq.gaffer.types.FreqMap in project Gaffer by gchq.

the class FullSchemaLoader method createEdges.

@Override
public Map<EdgeId, Edge> createEdges() {
    final Map<EdgeId, Edge> edges = new HashMap<>();
    for (int i = 0; i <= 10; i++) {
        for (int j = 0; j < VERTEX_PREFIXES.length; j++) {
            final FreqMap freqMap = new FreqMap();
            freqMap.upsert("key");
            final Edge edge = new Edge.Builder().group(TestGroups.EDGE).source(VERTEX_PREFIXES[0] + i).dest(VERTEX_PREFIXES[j] + i).directed(false).property(TestPropertyNames.COUNT, 1L).property(TestPropertyNames.PROP_1, 1).property(TestPropertyNames.PROP_2, 1L).property(TestPropertyNames.PROP_3, "1").property(TestPropertyNames.PROP_4, freqMap).property(TestPropertyNames.PROP_5, "property").property(TestPropertyNames.DATE, Date.from(Instant.ofEpochMilli(1))).property(TestPropertyNames.TIMESTAMP, 1L).property(TestPropertyNames.VISIBILITY, "public").build();
            addToMap(edge, edges);
            final FreqMap freqMap2 = new FreqMap();
            freqMap2.upsert("key");
            final Edge edgeDir = new Edge.Builder().group(TestGroups.EDGE).source(VERTEX_PREFIXES[0] + i).dest(VERTEX_PREFIXES[j] + i).directed(true).property(TestPropertyNames.COUNT, 1L).property(TestPropertyNames.PROP_1, 1).property(TestPropertyNames.PROP_2, 1L).property(TestPropertyNames.PROP_3, "1").property(TestPropertyNames.PROP_4, freqMap2).property(TestPropertyNames.PROP_5, "property").property(TestPropertyNames.DATE, Date.from(Instant.ofEpochMilli(1))).property(TestPropertyNames.TIMESTAMP, 1L).property(TestPropertyNames.VISIBILITY, "private").build();
            addToMap(edgeDir, edges);
        }
        final FreqMap freqMap3 = new FreqMap();
        freqMap3.upsert("key");
        final Edge edge = new Edge.Builder().group(TestGroups.EDGE).source(SOURCE + i).dest(DEST + i).directed(false).property(TestPropertyNames.COUNT, 1L).property(TestPropertyNames.PROP_1, 1).property(TestPropertyNames.PROP_2, 1L).property(TestPropertyNames.PROP_3, "1").property(TestPropertyNames.PROP_4, freqMap3).property(TestPropertyNames.PROP_5, "property").property(TestPropertyNames.DATE, Date.from(Instant.ofEpochMilli(1))).property(TestPropertyNames.TIMESTAMP, 1L).property(TestPropertyNames.VISIBILITY, "public").build();
        addToMap(edge, edges);
        final FreqMap freqMap4 = new FreqMap();
        freqMap4.upsert("key");
        final Edge edgeDir = new Edge.Builder().group(TestGroups.EDGE).source(SOURCE_DIR + i).dest(DEST_DIR + i).directed(true).property(TestPropertyNames.COUNT, 1L).property(TestPropertyNames.PROP_1, 1).property(TestPropertyNames.PROP_2, 1L).property(TestPropertyNames.PROP_3, "1").property(TestPropertyNames.PROP_4, freqMap4).property(TestPropertyNames.PROP_5, "property").property(TestPropertyNames.DATE, Date.from(Instant.ofEpochMilli(1))).property(TestPropertyNames.TIMESTAMP, 1L).property(TestPropertyNames.VISIBILITY, "private").build();
        addToMap(edgeDir, edges);
    }
    return edges;
}
Also used : FreqMap(uk.gov.gchq.gaffer.types.FreqMap) HashMap(java.util.HashMap) EdgeId(uk.gov.gchq.gaffer.data.element.id.EdgeId) Edge(uk.gov.gchq.gaffer.data.element.Edge)

Example 60 with FreqMap

use of uk.gov.gchq.gaffer.types.FreqMap in project Gaffer by gchq.

the class ElementSerialisationTest method shouldSerialiseAndDeserialisePropertiesWhenAllAreEmpty.

@Test
public void shouldSerialiseAndDeserialisePropertiesWhenAllAreEmpty() throws Exception {
    // Given
    final Schema schema = new Schema.Builder().entity(TestGroups.ENTITY, new SchemaEntityDefinition.Builder().vertex("string").property(HBasePropertyNames.PROP_1, "map").property(HBasePropertyNames.PROP_2, "map").build()).type("string", String.class).type("map", new TypeDefinition.Builder().clazz(FreqMap.class).aggregateFunction(new FreqMapAggregator()).serialiser(new FreqMapSerialiser()).build()).build();
    serialisation = new ElementSerialisation(schema);
    final Entity entity = new Entity.Builder().vertex("vertex1").property(HBasePropertyNames.PROP_1, new FreqMap()).property(HBasePropertyNames.PROP_2, new FreqMap()).build();
    // When 1
    final byte[] value = serialisation.getValue(TestGroups.ENTITY, entity.getProperties());
    // Then 1
    assertArrayEquals(new byte[] { ByteArrayEscapeUtils.DELIMITER, ByteArrayEscapeUtils.DELIMITER }, value);
    // When 2
    final Properties properties = serialisation.getPropertiesFromValue(TestGroups.ENTITY, value);
    // Then 2
    assertEquals(entity.getProperties(), properties);
}
Also used : Entity(uk.gov.gchq.gaffer.data.element.Entity) FreqMap(uk.gov.gchq.gaffer.types.FreqMap) Schema(uk.gov.gchq.gaffer.store.schema.Schema) FreqMapAggregator(uk.gov.gchq.gaffer.types.function.FreqMapAggregator) Properties(uk.gov.gchq.gaffer.data.element.Properties) FreqMapSerialiser(uk.gov.gchq.gaffer.serialisation.FreqMapSerialiser) Test(org.junit.jupiter.api.Test)

Aggregations

FreqMap (uk.gov.gchq.gaffer.types.FreqMap)62 Test (org.junit.jupiter.api.Test)29 ArrayList (java.util.ArrayList)9 Edge (uk.gov.gchq.gaffer.data.element.Edge)9 Entity (uk.gov.gchq.gaffer.data.element.Entity)9 Date (java.util.Date)6 Schema (uk.gov.gchq.gaffer.store.schema.Schema)6 FunctionTest (uk.gov.gchq.koryphe.function.FunctionTest)6 Configuration (org.apache.hadoop.conf.Configuration)5 FileSystem (org.apache.hadoop.fs.FileSystem)5 Path (org.apache.hadoop.fs.Path)5 Row (org.apache.spark.sql.Row)5 SparkSession (org.apache.spark.sql.SparkSession)5 WrappedArray (scala.collection.mutable.WrappedArray)5 Element (uk.gov.gchq.gaffer.data.element.Element)5 HashMap (java.util.HashMap)4 Key (org.apache.accumulo.core.data.Key)4 Test (org.junit.Test)4 EdgeId (uk.gov.gchq.gaffer.data.element.id.EdgeId)4 ElementId (uk.gov.gchq.gaffer.data.element.id.ElementId)4