Search in sources :

Example 16 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class BerkeleyJEKeyValueStore method get.

@Override
public StaticBuffer get(StaticBuffer key, StoreTransaction txh) throws BackendException {
    Transaction tx = getTransaction(txh);
    try {
        DatabaseEntry databaseKey = key.as(ENTRY_FACTORY);
        DatabaseEntry data = new DatabaseEntry();
        log.trace("db={}, op=get, tx={}", name, txh);
        OperationStatus status = db.get(tx, databaseKey, data, getLockMode(txh));
        if (status == OperationStatus.SUCCESS) {
            return getBuffer(data);
        } else {
            return null;
        }
    } catch (DatabaseException e) {
        throw new PermanentBackendException(e);
    }
}
Also used : StoreTransaction(org.janusgraph.diskstorage.keycolumnvalue.StoreTransaction) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException)

Example 17 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class BerkeleyJETx method commit.

@Override
public synchronized void commit() throws BackendException {
    super.commit();
    if (tx == null)
        return;
    if (log.isTraceEnabled())
        log.trace("{} committed", this.toString(), new TransactionClose(this.toString()));
    try {
        closeOpenIterators();
        tx.commit();
        tx = null;
    } catch (DatabaseException e) {
        throw new PermanentBackendException(e);
    }
}
Also used : PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) DatabaseException(com.sleepycat.je.DatabaseException)

Example 18 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class AstyanaxKeyColumnValueStore method getKeys.

@Override
public KeyIterator getKeys(KeyRangeQuery query, StoreTransaction txh) throws BackendException {
    // this query could only be done when byte-ordering partitioner is used
    // because Cassandra operates on tokens internally which means that even contiguous
    // range of keys (e.g. time slice) with random partitioner could produce disjoint set of tokens
    // returning ambiguous results to the user.
    Partitioner partitioner = storeManager.getPartitioner();
    if (partitioner != Partitioner.BYTEORDER)
        throw new PermanentBackendException("getKeys(KeyRangeQuery could only be used with byte-ordering partitioner.");
    ByteBuffer start = query.getKeyStart().asByteBuffer(), end = query.getKeyEnd().asByteBuffer();
    RowSliceQuery rowSlice = keyspace.prepareQuery(columnFamily).setConsistencyLevel(getTx(txh).getReadConsistencyLevel().getAstyanax()).withRetryPolicy(retryPolicy.duplicate()).getKeyRange(start, end, null, null, Integer.MAX_VALUE);
    // Astyanax is bad at builder pattern :(
    rowSlice.withColumnRange(query.getSliceStart().asByteBuffer(), query.getSliceEnd().asByteBuffer(), false, query.getLimit());
    // Omit final the query's key end from the result, if present in result
    final Rows<ByteBuffer, ByteBuffer> r;
    try {
        r = ((OperationResult<Rows<ByteBuffer, ByteBuffer>>) rowSlice.execute()).getResult();
    } catch (ConnectionException e) {
        throw new TemporaryBackendException(e);
    }
    Iterator<Row<ByteBuffer, ByteBuffer>> i = Iterators.filter(r.iterator(), new KeySkipPredicate(query.getKeyEnd().asByteBuffer()));
    return new RowIterator(i, query);
}
Also used : TemporaryBackendException(org.janusgraph.diskstorage.TemporaryBackendException) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) RowSliceQuery(com.netflix.astyanax.query.RowSliceQuery) Row(com.netflix.astyanax.model.Row) ByteBuffer(java.nio.ByteBuffer) Partitioner(org.janusgraph.diskstorage.cassandra.AbstractCassandraStoreManager.Partitioner) ConnectionException(com.netflix.astyanax.connectionpool.exceptions.ConnectionException) Rows(com.netflix.astyanax.model.Rows)

Example 19 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class AstyanaxStoreManager method getCompressionOptions.

@Override
public Map<String, String> getCompressionOptions(String cf) throws BackendException {
    try {
        Keyspace k = keyspaceContext.getClient();
        KeyspaceDefinition keyspaceDefinition = k.describeKeyspace();
        if (null == keyspaceDefinition) {
            throw new PermanentBackendException("Keyspace " + k.getKeyspaceName() + " is undefined");
        }
        ColumnFamilyDefinition columnFamilyDefinition = keyspaceDefinition.getColumnFamily(cf);
        if (null == columnFamilyDefinition) {
            throw new PermanentBackendException("Column family " + cf + " is undefined");
        }
        return columnFamilyDefinition.getCompressionOptions();
    } catch (ConnectionException e) {
        throw new PermanentBackendException(e);
    }
}
Also used : KeyspaceDefinition(com.netflix.astyanax.ddl.KeyspaceDefinition) ColumnFamilyDefinition(com.netflix.astyanax.ddl.ColumnFamilyDefinition) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) Keyspace(com.netflix.astyanax.Keyspace) ConnectionException(com.netflix.astyanax.connectionpool.exceptions.ConnectionException)

Example 20 with PermanentBackendException

use of org.janusgraph.diskstorage.PermanentBackendException in project janusgraph by JanusGraph.

the class CQLStoreManager method initializeCluster.

Cluster initializeCluster() throws PermanentBackendException {
    final Configuration configuration = getStorageConfig();
    final List<InetSocketAddress> contactPoints;
    try {
        contactPoints = Array.of(this.hostnames).map(hostName -> hostName.split(":")).map(array -> Tuple.of(array[0], array.length == 2 ? Integer.parseInt(array[1]) : this.port)).map(tuple -> new InetSocketAddress(tuple._1, tuple._2)).toJavaList();
    } catch (SecurityException | ArrayIndexOutOfBoundsException | NumberFormatException e) {
        throw new PermanentBackendException("Error initialising cluster contact points", e);
    }
    final Builder builder = Cluster.builder().addContactPointsWithPorts(contactPoints).withClusterName(configuration.get(CLUSTER_NAME));
    if (configuration.get(PROTOCOL_VERSION) != 0) {
        builder.withProtocolVersion(ProtocolVersion.fromInt(configuration.get(PROTOCOL_VERSION)));
    }
    if (configuration.has(AUTH_USERNAME) && configuration.has(AUTH_PASSWORD)) {
        builder.withCredentials(configuration.get(AUTH_USERNAME), configuration.get(AUTH_PASSWORD));
    }
    if (configuration.has(LOCAL_DATACENTER)) {
        builder.withLoadBalancingPolicy(new TokenAwarePolicy(DCAwareRoundRobinPolicy.builder().withLocalDc(configuration.get(LOCAL_DATACENTER)).build()));
    }
    if (configuration.get(SSL_ENABLED)) {
        try {
            final TrustManager[] trustManagers;
            try (final FileInputStream keyStoreStream = new FileInputStream(configuration.get(SSL_TRUSTSTORE_LOCATION))) {
                final KeyStore keystore = KeyStore.getInstance("jks");
                keystore.load(keyStoreStream, configuration.get(SSL_TRUSTSTORE_PASSWORD).toCharArray());
                final TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
                trustManagerFactory.init(keystore);
                trustManagers = trustManagerFactory.getTrustManagers();
            }
            final SSLContext sslContext = SSLContext.getInstance("TLS");
            sslContext.init(null, trustManagers, null);
            final JdkSSLOptions sslOptions = JdkSSLOptions.builder().withSSLContext(sslContext).build();
            builder.withSSL(sslOptions);
        } catch (NoSuchAlgorithmException | CertificateException | IOException | KeyStoreException | KeyManagementException e) {
            throw new PermanentBackendException("Error initialising SSL connection properties", e);
        }
    }
    // Build the PoolingOptions based on the configurations
    PoolingOptions poolingOptions = new PoolingOptions();
    poolingOptions.setMaxRequestsPerConnection(HostDistance.LOCAL, configuration.get(LOCAL_MAX_REQUESTS_PER_CONNECTION)).setMaxRequestsPerConnection(HostDistance.REMOTE, configuration.get(REMOTE_MAX_REQUESTS_PER_CONNECTION));
    poolingOptions.setConnectionsPerHost(HostDistance.LOCAL, configuration.get(LOCAL_CORE_CONNECTIONS_PER_HOST), configuration.get(LOCAL_MAX_CONNECTIONS_PER_HOST)).setConnectionsPerHost(HostDistance.REMOTE, configuration.get(REMOTE_CORE_CONNECTIONS_PER_HOST), configuration.get(REMOTE_MAX_CONNECTIONS_PER_HOST));
    return builder.withPoolingOptions(poolingOptions).build();
}
Also used : Match(io.vavr.API.Match) PoolingOptions(com.datastax.driver.core.PoolingOptions) GRAPH_NAME(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.GRAPH_NAME) SSLContext(javax.net.ssl.SSLContext) REMOTE_MAX_REQUESTS_PER_CONNECTION(org.janusgraph.diskstorage.cql.CQLConfigOptions.REMOTE_MAX_REQUESTS_PER_CONNECTION) TrustManager(javax.net.ssl.TrustManager) KeyStoreException(java.security.KeyStoreException) REMOTE_CORE_CONNECTIONS_PER_HOST(org.janusgraph.diskstorage.cql.CQLConfigOptions.REMOTE_CORE_CONNECTIONS_PER_HOST) StandardStoreFeatures(org.janusgraph.diskstorage.keycolumnvalue.StandardStoreFeatures) REMOTE_MAX_CONNECTIONS_PER_HOST(org.janusgraph.diskstorage.cql.CQLConfigOptions.REMOTE_MAX_CONNECTIONS_PER_HOST) Option(io.vavr.control.Option) Map(java.util.Map) Session(com.datastax.driver.core.Session) TokenAwarePolicy(com.datastax.driver.core.policies.TokenAwarePolicy) StaticBuffer(org.janusgraph.diskstorage.StaticBuffer) SSL_TRUSTSTORE_LOCATION(org.janusgraph.diskstorage.cql.CQLConfigOptions.SSL_TRUSTSTORE_LOCATION) Iterator(io.vavr.collection.Iterator) SchemaBuilder.dropKeyspace(com.datastax.driver.core.schemabuilder.SchemaBuilder.dropKeyspace) BatchStatement(com.datastax.driver.core.BatchStatement) LOCAL_MAX_CONNECTIONS_PER_HOST(org.janusgraph.diskstorage.cql.CQLConfigOptions.LOCAL_MAX_CONNECTIONS_PER_HOST) KEYSPACE(org.janusgraph.diskstorage.cql.CQLConfigOptions.KEYSPACE) KCVMutation(org.janusgraph.diskstorage.keycolumnvalue.KCVMutation) NetworkUtil(org.janusgraph.util.system.NetworkUtil) Tuple(io.vavr.Tuple) TrustManagerFactory(javax.net.ssl.TrustManagerFactory) READ_CONSISTENCY(org.janusgraph.diskstorage.cql.CQLConfigOptions.READ_CONSISTENCY) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Array(io.vavr.collection.Array) Case(io.vavr.API.Case) KeyStore(java.security.KeyStore) KeyManagementException(java.security.KeyManagementException) InetSocketAddress(java.net.InetSocketAddress) LinkedBlockingQueue(java.util.concurrent.LinkedBlockingQueue) StoreFeatures(org.janusgraph.diskstorage.keycolumnvalue.StoreFeatures) GraphDatabaseConfiguration.buildGraphConfiguration(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.buildGraphConfiguration) REPLICATION_FACTOR(org.janusgraph.diskstorage.cql.CQLConfigOptions.REPLICATION_FACTOR) KeyColumnValueStore(org.janusgraph.diskstorage.keycolumnvalue.KeyColumnValueStore) Builder(com.datastax.driver.core.Cluster.Builder) ProtocolVersion(com.datastax.driver.core.ProtocolVersion) List(java.util.List) METRICS_SYSTEM_PREFIX_DEFAULT(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.METRICS_SYSTEM_PREFIX_DEFAULT) DROP_ON_CLEAR(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.DROP_ON_CLEAR) Cluster(com.datastax.driver.core.Cluster) ATOMIC_BATCH_MUTATE(org.janusgraph.diskstorage.cql.CQLConfigOptions.ATOMIC_BATCH_MUTATE) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) HostDistance(com.datastax.driver.core.HostDistance) SSL_TRUSTSTORE_PASSWORD(org.janusgraph.diskstorage.cql.CQLConfigOptions.SSL_TRUSTSTORE_PASSWORD) AUTH_PASSWORD(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.AUTH_PASSWORD) Statement(com.datastax.driver.core.Statement) EXCEPTION_MAPPER(org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.EXCEPTION_MAPPER) KeyRange(org.janusgraph.diskstorage.keycolumnvalue.KeyRange) ThreadFactoryBuilder(com.google.common.util.concurrent.ThreadFactoryBuilder) CLUSTER_NAME(org.janusgraph.diskstorage.cql.CQLConfigOptions.CLUSTER_NAME) DistributedStoreManager(org.janusgraph.diskstorage.common.DistributedStoreManager) ThreadPoolExecutor(java.util.concurrent.ThreadPoolExecutor) BATCH_STATEMENT_SIZE(org.janusgraph.diskstorage.cql.CQLConfigOptions.BATCH_STATEMENT_SIZE) METRICS_PREFIX(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.METRICS_PREFIX) LOCAL_CORE_CONNECTIONS_PER_HOST(org.janusgraph.diskstorage.cql.CQLConfigOptions.LOCAL_CORE_CONNECTIONS_PER_HOST) REPLICATION_OPTIONS(org.janusgraph.diskstorage.cql.CQLConfigOptions.REPLICATION_OPTIONS) ONLY_USE_LOCAL_CONSISTENCY_FOR_SYSTEM_OPERATIONS(org.janusgraph.diskstorage.cql.CQLConfigOptions.ONLY_USE_LOCAL_CONSISTENCY_FOR_SYSTEM_OPERATIONS) ResultSet(com.datastax.driver.core.ResultSet) CQLTransaction.getTransaction(org.janusgraph.diskstorage.cql.CQLTransaction.getTransaction) Type(com.datastax.driver.core.BatchStatement.Type) WRITE_CONSISTENCY(org.janusgraph.diskstorage.cql.CQLConfigOptions.WRITE_CONSISTENCY) Future(io.vavr.concurrent.Future) SchemaBuilder.createKeyspace(com.datastax.driver.core.schemabuilder.SchemaBuilder.createKeyspace) AUTH_USERNAME(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.AUTH_USERNAME) Container(org.janusgraph.diskstorage.StoreMetaData.Container) LOCAL_MAX_REQUESTS_PER_CONNECTION(org.janusgraph.diskstorage.cql.CQLConfigOptions.LOCAL_MAX_REQUESTS_PER_CONNECTION) KeyColumnValueStoreManager(org.janusgraph.diskstorage.keycolumnvalue.KeyColumnValueStoreManager) StoreTransaction(org.janusgraph.diskstorage.keycolumnvalue.StoreTransaction) ExecutorService(java.util.concurrent.ExecutorService) API.$(io.vavr.API.$) SSL_ENABLED(org.janusgraph.diskstorage.cql.CQLConfigOptions.SSL_ENABLED) BackendException(org.janusgraph.diskstorage.BackendException) Configuration(org.janusgraph.diskstorage.configuration.Configuration) HashMap(io.vavr.collection.HashMap) LOCAL_DATACENTER(org.janusgraph.diskstorage.cql.CQLConfigOptions.LOCAL_DATACENTER) IOException(java.io.IOException) FileInputStream(java.io.FileInputStream) CertificateException(java.security.cert.CertificateException) REPLICATION_STRATEGY(org.janusgraph.diskstorage.cql.CQLConfigOptions.REPLICATION_STRATEGY) PROTOCOL_VERSION(org.janusgraph.diskstorage.cql.CQLConfigOptions.PROTOCOL_VERSION) TimeUnit(java.util.concurrent.TimeUnit) JdkSSLOptions(com.datastax.driver.core.JdkSSLOptions) KeyspaceMetadata(com.datastax.driver.core.KeyspaceMetadata) BaseTransactionConfig(org.janusgraph.diskstorage.BaseTransactionConfig) QueryBuilder.truncate(com.datastax.driver.core.querybuilder.QueryBuilder.truncate) Seq(io.vavr.collection.Seq) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) DCAwareRoundRobinPolicy(com.datastax.driver.core.policies.DCAwareRoundRobinPolicy) GraphDatabaseConfiguration.buildGraphConfiguration(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.buildGraphConfiguration) Configuration(org.janusgraph.diskstorage.configuration.Configuration) InetSocketAddress(java.net.InetSocketAddress) Builder(com.datastax.driver.core.Cluster.Builder) ThreadFactoryBuilder(com.google.common.util.concurrent.ThreadFactoryBuilder) CertificateException(java.security.cert.CertificateException) NoSuchAlgorithmException(java.security.NoSuchAlgorithmException) KeyManagementException(java.security.KeyManagementException) JdkSSLOptions(com.datastax.driver.core.JdkSSLOptions) PoolingOptions(com.datastax.driver.core.PoolingOptions) TokenAwarePolicy(com.datastax.driver.core.policies.TokenAwarePolicy) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) SSLContext(javax.net.ssl.SSLContext) IOException(java.io.IOException) KeyStoreException(java.security.KeyStoreException) KeyStore(java.security.KeyStore) FileInputStream(java.io.FileInputStream) TrustManager(javax.net.ssl.TrustManager) TrustManagerFactory(javax.net.ssl.TrustManagerFactory)

Aggregations

PermanentBackendException (org.janusgraph.diskstorage.PermanentBackendException)29 IOException (java.io.IOException)12 BackendException (org.janusgraph.diskstorage.BackendException)9 TemporaryBackendException (org.janusgraph.diskstorage.TemporaryBackendException)9 UncheckedIOException (java.io.UncheckedIOException)8 ConnectionException (com.netflix.astyanax.connectionpool.exceptions.ConnectionException)7 Map (java.util.Map)5 StoreTransaction (org.janusgraph.diskstorage.keycolumnvalue.StoreTransaction)5 SolrServerException (org.apache.solr.client.solrj.SolrServerException)4 KeeperException (org.apache.zookeeper.KeeperException)4 Rows (com.netflix.astyanax.model.Rows)3 CloudSolrClient (org.apache.solr.client.solrj.impl.CloudSolrClient)3 BiMap (com.google.common.collect.BiMap)2 Keyspace (com.netflix.astyanax.Keyspace)2 ColumnFamilyDefinition (com.netflix.astyanax.ddl.ColumnFamilyDefinition)2 KeyspaceDefinition (com.netflix.astyanax.ddl.KeyspaceDefinition)2 RowSliceQuery (com.netflix.astyanax.query.RowSliceQuery)2 ByteBuffer (java.nio.ByteBuffer)2 Duration (java.time.Duration)2 ConfigurationException (org.apache.cassandra.exceptions.ConfigurationException)2