Search in sources :

Example 11 with JanusGraphException

use of org.janusgraph.core.JanusGraphException in project janusgraph by JanusGraph.

the class IndexUpdateJob method workerIterationStart.

public void workerIterationStart(JanusGraph graph, Configuration config, ScanMetrics metrics) {
    this.graph = (StandardJanusGraph) graph;
    Preconditions.checkArgument(config.has(GraphDatabaseConfiguration.JOB_START_TIME), "Invalid configuration for this job. Start time is required.");
    this.jobStartTime = Instant.ofEpochMilli(config.get(GraphDatabaseConfiguration.JOB_START_TIME));
    if (indexName == null) {
        Preconditions.checkArgument(config.has(INDEX_NAME), "Need to configure the name of the index to be repaired");
        indexName = config.get(INDEX_NAME);
        indexRelationTypeName = config.get(INDEX_RELATION_TYPE);
        log.info("Read index information: name={} type={}", indexName, indexRelationTypeName);
    }
    try {
        this.managementSystem = (ManagementSystem) graph.openManagement();
        if (isGlobalGraphIndex()) {
            index = managementSystem.getGraphIndex(indexName);
        } else {
            indexRelationType = managementSystem.getRelationType(indexRelationTypeName);
            Preconditions.checkArgument(indexRelationType != null, "Could not find relation type: %s", indexRelationTypeName);
            index = managementSystem.getRelationIndex(indexRelationType, indexName);
        }
        Preconditions.checkArgument(index != null, "Could not find index: %s [%s]", indexName, indexRelationTypeName);
        log.info("Found index {}", indexName);
        validateIndexStatus();
        StandardTransactionBuilder txb = this.graph.buildTransaction();
        txb.commitTime(jobStartTime);
        writeTx = (StandardJanusGraphTx) txb.start();
    } catch (final Exception e) {
        if (null != managementSystem && managementSystem.isOpen())
            managementSystem.rollback();
        if (writeTx != null && writeTx.isOpen())
            writeTx.rollback();
        metrics.incrementCustom(FAILED_TX);
        throw new JanusGraphException(e.getMessage(), e);
    }
}
Also used : JanusGraphException(org.janusgraph.core.JanusGraphException) JanusGraphException(org.janusgraph.core.JanusGraphException) StandardTransactionBuilder(org.janusgraph.graphdb.transaction.StandardTransactionBuilder)

Example 12 with JanusGraphException

use of org.janusgraph.core.JanusGraphException in project janusgraph by JanusGraph.

the class KCVSLog method add.

/**
 * Adds the given message (content) to the timeslice for the partition identified by the provided partitionId.
 * If a persistor is specified, this persistor is used to add the message otherwise the internal delivery systems are used.
 *
 * @param content
 * @param partitionId
 * @param persistor
 * @return
 */
private Future<Message> add(StaticBuffer content, int partitionId, ExternalPersistor persistor) {
    ResourceUnavailableException.verifyOpen(isOpen, "Log", name);
    Preconditions.checkArgument(content != null && content.length() > 0, "Content is empty");
    Preconditions.checkArgument(partitionId >= 0 && partitionId < (1 << manager.partitionBitWidth), "Invalid partition id: %s", partitionId);
    final Instant timestamp = times.getTime();
    KCVSMessage msg = new KCVSMessage(content, timestamp, manager.senderId);
    FutureMessage futureMessage = new FutureMessage(msg);
    StaticBuffer key = getLogKey(partitionId, (int) (numBucketCounter.incrementAndGet() % numBuckets), getTimeSlice(timestamp));
    MessageEnvelope envelope = new MessageEnvelope(futureMessage, key, writeMessage(msg));
    if (persistor != null) {
        try {
            persistor.add(envelope.key, envelope.entry);
            envelope.message.delivered();
        } catch (JanusGraphException e) {
            envelope.message.failed(e);
            throw e;
        }
    } else if (outgoingMsg == null) {
        sendMessages(ImmutableList.of(envelope));
    } else {
        try {
            // Produces back pressure when full
            outgoingMsg.put(envelope);
            log.debug("Enqueued {} for partition {}", envelope, partitionId);
        } catch (InterruptedException e) {
            throw new JanusGraphException("Got interrupted waiting to send message", e);
        }
    }
    return futureMessage;
}
Also used : Instant(java.time.Instant) JanusGraphException(org.janusgraph.core.JanusGraphException) FutureMessage(org.janusgraph.diskstorage.log.util.FutureMessage)

Example 13 with JanusGraphException

use of org.janusgraph.core.JanusGraphException in project janusgraph by JanusGraph.

the class StandardIDPool method close.

@Override
public synchronized void close() {
    closed = true;
    try {
        waitForIDBlockGetter();
    } catch (InterruptedException e) {
        throw new JanusGraphException("Interrupted while waiting for id renewer thread to finish", e);
    }
    for (Future<?> closeBlocker : closeBlockers) {
        try {
            closeBlocker.get();
        } catch (InterruptedException e) {
            throw new JanusGraphException("Interrupted while waiting for runaway ID renewer task " + closeBlocker, e);
        } catch (ExecutionException e) {
            log.debug("Runaway ID renewer task completed with exception", e);
        }
    }
    exec.shutdownNow();
}
Also used : JanusGraphException(org.janusgraph.core.JanusGraphException) ExecutionException(java.util.concurrent.ExecutionException)

Example 14 with JanusGraphException

use of org.janusgraph.core.JanusGraphException in project janusgraph by JanusGraph.

the class StandardIDPool method nextID.

@Override
public synchronized long nextID() {
    assert currentIndex <= currentBlock.numIds();
    if (currentIndex == currentBlock.numIds()) {
        try {
            nextBlock();
        } catch (InterruptedException e) {
            throw new JanusGraphException("Could not renew id block due to interruption", e);
        }
    }
    if (currentIndex == renewBlockIndex) {
        startIDBlockGetter();
    }
    long returnId = currentBlock.getId(currentIndex);
    currentIndex++;
    if (returnId >= idUpperBound)
        throw new IDPoolExhaustedException("Reached id upper bound of " + idUpperBound);
    log.trace("partition({})-namespace({}) Returned id: {}", partition, idNamespace, returnId);
    return returnId;
}
Also used : JanusGraphException(org.janusgraph.core.JanusGraphException)

Example 15 with JanusGraphException

use of org.janusgraph.core.JanusGraphException in project janusgraph by JanusGraph.

the class Backend method initialize.

/**
 * Initializes this backend with the given configuration. Must be called before this Backend can be used
 *
 * @param config
 */
public void initialize(Configuration config) {
    try {
        // EdgeStore & VertexIndexStore
        KeyColumnValueStore idStore = storeManager.openDatabase(config.get(IDS_STORE_NAME));
        idAuthority = null;
        if (storeFeatures.isKeyConsistent()) {
            idAuthority = new ConsistentKeyIDAuthority(idStore, storeManager, config);
        } else {
            throw new IllegalStateException("Store needs to support consistent key or transactional operations for ID manager to guarantee proper id allocations");
        }
        KeyColumnValueStore edgeStoreRaw = storeManagerLocking.openDatabase(EDGESTORE_NAME);
        KeyColumnValueStore indexStoreRaw = storeManagerLocking.openDatabase(INDEXSTORE_NAME);
        // Configure caches
        if (cacheEnabled) {
            long expirationTime = configuration.get(DB_CACHE_TIME);
            Preconditions.checkArgument(expirationTime >= 0, "Invalid cache expiration time: %s", expirationTime);
            if (expirationTime == 0)
                expirationTime = ETERNAL_CACHE_EXPIRATION;
            long cacheSizeBytes;
            double cacheSize = configuration.get(DB_CACHE_SIZE);
            Preconditions.checkArgument(cacheSize > 0.0, "Invalid cache size specified: %s", cacheSize);
            if (cacheSize < 1.0) {
                // Its a percentage
                Runtime runtime = Runtime.getRuntime();
                cacheSizeBytes = (long) ((runtime.maxMemory() - (runtime.totalMemory() - runtime.freeMemory())) * cacheSize);
            } else {
                Preconditions.checkArgument(cacheSize > 1000, "Cache size is too small: %s", cacheSize);
                cacheSizeBytes = (long) cacheSize;
            }
            log.info("Configuring total store cache size: {}", cacheSizeBytes);
            long cleanWaitTime = configuration.get(DB_CACHE_CLEAN_WAIT);
            Preconditions.checkArgument(EDGESTORE_CACHE_PERCENT + INDEXSTORE_CACHE_PERCENT == 1.0, "Cache percentages don't add up!");
            long edgeStoreCacheSize = Math.round(cacheSizeBytes * EDGESTORE_CACHE_PERCENT);
            long indexStoreCacheSize = Math.round(cacheSizeBytes * INDEXSTORE_CACHE_PERCENT);
            edgeStore = new ExpirationKCVSCache(edgeStoreRaw, getMetricsCacheName(EDGESTORE_NAME), expirationTime, cleanWaitTime, edgeStoreCacheSize);
            indexStore = new ExpirationKCVSCache(indexStoreRaw, getMetricsCacheName(INDEXSTORE_NAME), expirationTime, cleanWaitTime, indexStoreCacheSize);
        } else {
            edgeStore = new NoKCVSCache(edgeStoreRaw);
            indexStore = new NoKCVSCache(indexStoreRaw);
        }
        // Just open them so that they are cached
        txLogManager.openLog(SYSTEM_TX_LOG_NAME);
        managementLogManager.openLog(SYSTEM_MGMT_LOG_NAME);
        txLogStore = new NoKCVSCache(storeManager.openDatabase(SYSTEM_TX_LOG_NAME));
        // Open global configuration
        KeyColumnValueStore systemConfigStore = storeManagerLocking.openDatabase(SYSTEM_PROPERTIES_STORE_NAME);
        systemConfig = getGlobalConfiguration(new BackendOperation.TransactionalProvider() {

            @Override
            public StoreTransaction openTx() throws BackendException {
                return storeManagerLocking.beginTransaction(StandardBaseTransactionConfig.of(configuration.get(TIMESTAMP_PROVIDER), storeFeatures.getKeyConsistentTxConfig()));
            }

            @Override
            public void close() throws BackendException {
            // Do nothing, storeManager is closed explicitly by Backend
            }
        }, systemConfigStore, configuration);
        userConfig = getConfiguration(new BackendOperation.TransactionalProvider() {

            @Override
            public StoreTransaction openTx() throws BackendException {
                return storeManagerLocking.beginTransaction(StandardBaseTransactionConfig.of(configuration.get(TIMESTAMP_PROVIDER)));
            }

            @Override
            public void close() throws BackendException {
            // Do nothing, storeManager is closed explicitly by Backend
            }
        }, systemConfigStore, USER_CONFIGURATION_IDENTIFIER, configuration);
    } catch (BackendException e) {
        throw new JanusGraphException("Could not initialize backend", e);
    }
}
Also used : ConsistentKeyIDAuthority(org.janusgraph.diskstorage.idmanagement.ConsistentKeyIDAuthority) NoKCVSCache(org.janusgraph.diskstorage.keycolumnvalue.cache.NoKCVSCache) ExpirationKCVSCache(org.janusgraph.diskstorage.keycolumnvalue.cache.ExpirationKCVSCache) JanusGraphException(org.janusgraph.core.JanusGraphException)

Aggregations

JanusGraphException (org.janusgraph.core.JanusGraphException)19 BackendException (org.janusgraph.diskstorage.BackendException)4 PropertyKey (org.janusgraph.core.PropertyKey)3 Instant (java.time.Instant)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)2 Vertex (org.apache.tinkerpop.gremlin.structure.Vertex)2 JanusGraphIndex (org.janusgraph.core.schema.JanusGraphIndex)2 JanusGraphManagement (org.janusgraph.core.schema.JanusGraphManagement)2 RelationTypeIndex (org.janusgraph.core.schema.RelationTypeIndex)2 StandardScanner (org.janusgraph.diskstorage.keycolumnvalue.scan.StandardScanner)2 StandardJanusGraph (org.janusgraph.graphdb.database.StandardJanusGraph)2 Function (com.google.common.base.Function)1 Preconditions (com.google.common.base.Preconditions)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 Duration (java.time.Duration)1 java.util (java.util)1 ArrayList (java.util.ArrayList)1