Search in sources :

Example 6 with ClassicAccumuloElementConverter

use of uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicAccumuloElementConverter in project Gaffer by gchq.

the class GetRDDOfAllElementsHandlerIT method writeFile.

private void writeFile(final KeyPackage keyPackage, final Schema schema, final String file) throws IllegalArgumentException, IOException {
    final Configuration conf = new Configuration();
    final CachableBlockFile.Writer blockFileWriter = new CachableBlockFile.Writer(FileSystem.get(conf), new Path(file), Compression.COMPRESSION_NONE, null, conf, AccumuloConfiguration.getDefaultConfiguration());
    final AccumuloElementConverter converter;
    switch(keyPackage) {
        case BYTE_ENTITY:
            converter = new ByteEntityAccumuloElementConverter(schema);
            break;
        case CLASSIC:
            converter = new ClassicAccumuloElementConverter(schema);
            break;
        default:
            throw new IllegalArgumentException("Unknown keypackage");
    }
    final Entity entity = (Entity) getElementsForIngestAggregationChecking().get(0);
    final Key key = converter.getKeyFromEntity((Entity) getElementsForIngestAggregationChecking().get(0));
    final Value value = converter.getValueFromProperties(entity.getGroup(), entity.getProperties());
    final RFile.Writer writer = new RFile.Writer(blockFileWriter, 1000);
    writer.startDefaultLocalityGroup();
    writer.append(key, value);
    writer.close();
}
Also used : Path(org.apache.hadoop.fs.Path) Entity(uk.gov.gchq.gaffer.data.element.Entity) Configuration(org.apache.hadoop.conf.Configuration) AccumuloConfiguration(org.apache.accumulo.core.conf.AccumuloConfiguration) RFile(org.apache.accumulo.core.file.rfile.RFile) ClassicAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicAccumuloElementConverter) ByteEntityAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter) AccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.AccumuloElementConverter) ClassicAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicAccumuloElementConverter) Value(org.apache.accumulo.core.data.Value) CachableBlockFile(org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile) ByteEntityAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter) Key(org.apache.accumulo.core.data.Key)

Example 7 with ClassicAccumuloElementConverter

use of uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicAccumuloElementConverter in project Gaffer by gchq.

the class RowIdAggregatorTest method setup.

@BeforeAll
public static void setup(@TempDir Path tempDir) {
    gaffer1ElementConverter = new ClassicAccumuloElementConverter(SCHEMA);
    byteEntityElementConverter = new ByteEntityAccumuloElementConverter(SCHEMA);
}
Also used : ClassicAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicAccumuloElementConverter) ByteEntityAccumuloElementConverter(uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter) BeforeAll(org.junit.jupiter.api.BeforeAll)

Aggregations

ClassicAccumuloElementConverter (uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicAccumuloElementConverter)7 ByteEntityAccumuloElementConverter (uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityAccumuloElementConverter)6 JavaSerialiser (uk.gov.gchq.gaffer.serialisation.implementation.JavaSerialiser)3 Schema (uk.gov.gchq.gaffer.store.schema.Schema)3 SchemaEdgeDefinition (uk.gov.gchq.gaffer.store.schema.SchemaEdgeDefinition)3 BeforeAll (org.junit.jupiter.api.BeforeAll)2 BeforeEach (org.junit.jupiter.api.BeforeEach)2 ByteEntityRangeFactory (uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityRangeFactory)2 ClassicRangeFactory (uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicRangeFactory)2 AccumuloConfiguration (org.apache.accumulo.core.conf.AccumuloConfiguration)1 Key (org.apache.accumulo.core.data.Key)1 Value (org.apache.accumulo.core.data.Value)1 CachableBlockFile (org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile)1 RFile (org.apache.accumulo.core.file.rfile.RFile)1 Configuration (org.apache.hadoop.conf.Configuration)1 Path (org.apache.hadoop.fs.Path)1 Before (org.junit.Before)1 BeforeClass (org.junit.BeforeClass)1 SingleUseMockAccumuloStore (uk.gov.gchq.gaffer.accumulostore.SingleUseMockAccumuloStore)1 AccumuloElementConverter (uk.gov.gchq.gaffer.accumulostore.key.AccumuloElementConverter)1