Search in sources :

Example 21 with StoreException

use of uk.gov.gchq.gaffer.store.StoreException in project Gaffer by gchq.

the class ProxyStore method fetchOperations.

@SuppressFBWarnings(value = "SIC_INNER_SHOULD_BE_STATIC_ANON")
protected Set<Class<? extends Operation>> fetchOperations() {
    try {
        URL url = getProperties().getGafferUrl("graph/operations");
        final ResponseDeserialiser<Set<Class<? extends Operation>>> responseDeserialiser = getOperationsResponseDeserialiser();
        return Collections.unmodifiableSet(doGet(url, responseDeserialiser, null));
    } catch (final StoreException e) {
        throw new GafferRuntimeException("Failed to fetch operations from remote store.", e);
    }
}
Also used : Set(java.util.Set) HashSet(java.util.HashSet) URL(java.net.URL) StoreException(uk.gov.gchq.gaffer.store.StoreException) GafferRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferRuntimeException) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings)

Example 22 with StoreException

use of uk.gov.gchq.gaffer.store.StoreException in project Gaffer by gchq.

the class ProxyStore method handleResponse.

protected <O> O handleResponse(final Response response, final ResponseDeserialiser<O> responseDeserialiser) throws StoreException {
    final String outputJson = response.hasEntity() ? response.readEntity(String.class) : null;
    if (Family.SUCCESSFUL != response.getStatusInfo().getFamily()) {
        final Error error;
        try {
            error = JSONSerialiser.deserialise(StringUtil.toBytes(outputJson), Error.class);
        } catch (final Exception e) {
            LOGGER.warn("Gaffer bad status {}. Detail: {}", response.getStatus(), outputJson);
            throw new StoreException("Delegate Gaffer store returned status: " + response.getStatus() + ". Response content was: " + outputJson);
        }
        throw new GafferWrappedErrorRuntimeException(error);
    }
    O output = null;
    if (null != outputJson) {
        try {
            output = responseDeserialiser.deserialise(outputJson);
        } catch (final SerialisationException e) {
            throw new StoreException(e.getMessage(), e);
        }
    }
    return output;
}
Also used : SerialisationException(uk.gov.gchq.gaffer.exception.SerialisationException) Error(uk.gov.gchq.gaffer.core.exception.Error) GafferWrappedErrorRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferWrappedErrorRuntimeException) GafferWrappedErrorRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferWrappedErrorRuntimeException) SerialisationException(uk.gov.gchq.gaffer.exception.SerialisationException) OperationException(uk.gov.gchq.gaffer.operation.OperationException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) StoreException(uk.gov.gchq.gaffer.store.StoreException) GafferRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferRuntimeException) StoreException(uk.gov.gchq.gaffer.store.StoreException) OperationChainDAO(uk.gov.gchq.gaffer.operation.OperationChainDAO)

Example 23 with StoreException

use of uk.gov.gchq.gaffer.store.StoreException in project Gaffer by gchq.

the class ProxyStore method doGet.

protected <O> O doGet(final URL url, final ResponseDeserialiser<O> responseDeserialiser, final Context context) throws StoreException {
    final Invocation.Builder request = createRequest(null, url, context);
    final Response response;
    try {
        response = request.get();
    } catch (final Exception e) {
        throw new StoreException("Request failed to execute via url " + url.toExternalForm(), e);
    }
    return handleResponse(response, responseDeserialiser);
}
Also used : Response(javax.ws.rs.core.Response) Invocation(javax.ws.rs.client.Invocation) GafferWrappedErrorRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferWrappedErrorRuntimeException) SerialisationException(uk.gov.gchq.gaffer.exception.SerialisationException) OperationException(uk.gov.gchq.gaffer.operation.OperationException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) StoreException(uk.gov.gchq.gaffer.store.StoreException) GafferRuntimeException(uk.gov.gchq.gaffer.core.exception.GafferRuntimeException) StoreException(uk.gov.gchq.gaffer.store.StoreException)

Example 24 with StoreException

use of uk.gov.gchq.gaffer.store.StoreException in project Gaffer by gchq.

the class GetRDDOfAllElementsHandlerIT method getGraphForDirectRDDForIngestAggregation.

private Graph getGraphForDirectRDDForIngestAggregation(final KeyPackage keyPackage, final String tableName) throws InterruptedException, AccumuloException, AccumuloSecurityException, IOException, OperationException, StoreException, TableNotFoundException {
    Schema schema = getSchemaForIngestAggregationChecking();
    final Graph graph = _getGraphForDirectRDD(keyPackage, tableName, schema, null);
    AccumuloStore accumuloStore = new AccumuloStore();
    accumuloStore.initialise(tableName, schema, PROPERTIES_B);
    // using the RFileReaderRDD
    for (int i = 0; i < 2; i++) {
        final String dir = tempFolder.getAbsolutePath() + File.separator + "files";
        File filesDir = new File(dir);
        final String file = dir + File.separator + "file" + i + ".rf";
        final String failure = tempFolder.getAbsolutePath() + File.separator + "failures";
        File failuresDir = new File(failure);
        try {
            filesDir.mkdir();
            failuresDir.mkdir();
        } catch (Exception e) {
            LOGGER.error("Failed to create directory: " + e.getMessage());
        }
        writeFile(keyPackage, graph.getSchema(), file);
        accumuloStore.getConnection().tableOperations().importDirectory(tableName, dir, failure, false);
    }
    return graph;
}
Also used : Graph(uk.gov.gchq.gaffer.graph.Graph) Schema(uk.gov.gchq.gaffer.store.schema.Schema) AccumuloStore(uk.gov.gchq.gaffer.accumulostore.AccumuloStore) CachableBlockFile(org.apache.accumulo.core.file.blockfile.impl.CachableBlockFile) File(java.io.File) RFile(org.apache.accumulo.core.file.rfile.RFile) TableNotFoundException(org.apache.accumulo.core.client.TableNotFoundException) OperationException(uk.gov.gchq.gaffer.operation.OperationException) StoreException(uk.gov.gchq.gaffer.store.StoreException) AccumuloSecurityException(org.apache.accumulo.core.client.AccumuloSecurityException) IOException(java.io.IOException) AccumuloException(org.apache.accumulo.core.client.AccumuloException)

Example 25 with StoreException

use of uk.gov.gchq.gaffer.store.StoreException in project Gaffer by gchq.

the class InputFormatTest method shouldReturnCorrectDataToMapReduceJob.

private void shouldReturnCorrectDataToMapReduceJob(final Schema schema, final KeyPackage kp, final List<Element> data, final GraphFilters graphFilters, final User user, final String instanceName, final Set<String> expectedResults, final java.nio.file.Path tempDir) throws Exception {
    AccumuloProperties properties = PROPERTIES.clone();
    SingleUseMiniAccumuloStore store = new SingleUseMiniAccumuloStore();
    String graphId = null;
    switch(kp) {
        case BYTE_ENTITY_KEY_PACKAGE:
            properties.setKeyPackageClass(ByteEntityKeyPackage.class.getName());
            graphId = "byteEntityGraph";
            break;
        case CLASSIC_KEY_PACKAGE:
            graphId = "gaffer1Graph";
            properties.setKeyPackageClass(ClassicKeyPackage.class.getName());
    }
    try {
        store.initialise(graphId, schema, properties);
    } catch (final StoreException e) {
        fail("StoreException thrown: " + e);
    }
    setupGraph(store, data);
    // Set up local conf
    final JobConf conf = new JobConf();
    conf.set("fs.default.name", "file:///");
    conf.set("mapred.job.tracker", "local");
    final FileSystem fs = FileSystem.getLocal(conf);
    // Update configuration with instance, table name, etc.
    store.updateConfiguration(conf, graphFilters, user);
    // Run Driver
    final File outputFolder = Files.createDirectories(tempDir).toFile();
    FileUtils.deleteDirectory(outputFolder);
    final Driver driver = new Driver(outputFolder.getAbsolutePath());
    driver.setConf(conf);
    driver.run(new String[] {});
    // Read results and check correct
    final SequenceFile.Reader reader = new SequenceFile.Reader(fs, new Path(outputFolder + "/part-m-00000"), conf);
    final Text text = new Text();
    final Set<String> results = new HashSet<>();
    while (reader.next(text)) {
        results.add(text.toString());
    }
    reader.close();
    assertEquals(expectedResults, results);
    FileUtils.deleteDirectory(outputFolder);
}
Also used : Path(org.apache.hadoop.fs.Path) ClassicKeyPackage(uk.gov.gchq.gaffer.accumulostore.key.core.impl.classic.ClassicKeyPackage) AccumuloProperties(uk.gov.gchq.gaffer.accumulostore.AccumuloProperties) Text(org.apache.hadoop.io.Text) SingleUseMiniAccumuloStore(uk.gov.gchq.gaffer.accumulostore.SingleUseMiniAccumuloStore) ByteEntityKeyPackage(uk.gov.gchq.gaffer.accumulostore.key.core.impl.byteEntity.ByteEntityKeyPackage) StoreException(uk.gov.gchq.gaffer.store.StoreException) SequenceFile(org.apache.hadoop.io.SequenceFile) FileSystem(org.apache.hadoop.fs.FileSystem) JobConf(org.apache.hadoop.mapred.JobConf) SequenceFile(org.apache.hadoop.io.SequenceFile) File(java.io.File) HashSet(java.util.HashSet)

Aggregations

StoreException (uk.gov.gchq.gaffer.store.StoreException)70 OperationException (uk.gov.gchq.gaffer.operation.OperationException)26 IOException (java.io.IOException)21 Path (org.apache.hadoop.fs.Path)11 Schema (uk.gov.gchq.gaffer.store.schema.Schema)11 HashSet (java.util.HashSet)10 AccumuloSecurityException (org.apache.accumulo.core.client.AccumuloSecurityException)10 Element (uk.gov.gchq.gaffer.data.element.Element)10 UnsupportedEncodingException (java.io.UnsupportedEncodingException)9 TableNotFoundException (org.apache.accumulo.core.client.TableNotFoundException)9 IteratorSettingException (uk.gov.gchq.gaffer.accumulostore.key.exception.IteratorSettingException)9 SerialisationException (uk.gov.gchq.gaffer.exception.SerialisationException)9 ArrayList (java.util.ArrayList)8 AccumuloException (org.apache.accumulo.core.client.AccumuloException)8 Configuration (org.apache.hadoop.conf.Configuration)8 Test (org.junit.jupiter.api.Test)8 User (uk.gov.gchq.gaffer.user.User)8 Set (java.util.Set)6 IteratorSetting (org.apache.accumulo.core.client.IteratorSetting)6 FileSystem (org.apache.hadoop.fs.FileSystem)6