Search in sources :

Example 11 with LoadOptions

use of com.baidu.hugegraph.loader.executor.LoadOptions in project incubator-hugegraph-toolchain by apache.

the class Printer method printRealtimeProgress.

public static void printRealtimeProgress(LoadContext context) {
    LoadOptions options = context.options();
    if (!options.printProgress) {
        return;
    }
    System.out.printf(">> HugeGraphLoader worked in %s%n", options.workModeString());
    if (options.incrementalMode) {
        LoadProgress progress = context.oldProgress();
        System.out.printf("vertices/edges loaded last time: %s/%s%n", progress.vertexLoaded(), progress.edgeLoaded());
    }
    System.out.print("vertices/edges loaded this time : ");
}
Also used : LoadOptions(com.baidu.hugegraph.loader.executor.LoadOptions) LoadProgress(com.baidu.hugegraph.loader.progress.LoadProgress)

Example 12 with LoadOptions

use of com.baidu.hugegraph.loader.executor.LoadOptions in project incubator-hugegraph-toolchain by apache.

the class Printer method printFinalProgress.

public static void printFinalProgress(LoadContext context) {
    LoadOptions options = context.options();
    if (!options.printProgress) {
        return;
    }
    LoadSummary summary = context.summary();
    long vertexLoaded = summary.vertexLoaded();
    long edgeLoaded = summary.edgeLoaded();
    System.out.println(vertexLoaded + SLASH + edgeLoaded);
}
Also used : LoadOptions(com.baidu.hugegraph.loader.executor.LoadOptions) LoadSummary(com.baidu.hugegraph.loader.metrics.LoadSummary)

Example 13 with LoadOptions

use of com.baidu.hugegraph.loader.executor.LoadOptions in project incubator-hugegraph-toolchain by apache.

the class BatchInsertTask method waitThenRetry.

private int waitThenRetry(int retryCount, RuntimeException e) {
    LoadOptions options = this.options();
    if (options.retryTimes <= 0) {
        return retryCount;
    }
    if (++retryCount > options.retryTimes) {
        LOG.error("Batch insert has been retried more than {} times", options.retryTimes);
        throw e;
    }
    long interval = (1L << retryCount) * options.retryInterval;
    LOG.debug("Batch insert will sleep {} seconds then do the {}th retry", interval, retryCount);
    try {
        Thread.sleep(interval * 1000L);
    } catch (InterruptedException ignored) {
    // That's fine, just continue.
    }
    return retryCount;
}
Also used : LoadOptions(com.baidu.hugegraph.loader.executor.LoadOptions)

Aggregations

LoadOptions (com.baidu.hugegraph.loader.executor.LoadOptions)13 HugeClient (com.baidu.hugegraph.driver.HugeClient)4 IOException (java.io.IOException)3 LoadException (com.baidu.hugegraph.loader.exception.LoadException)2 LoadSummary (com.baidu.hugegraph.loader.metrics.LoadSummary)2 Vertex (com.baidu.hugegraph.structure.graph.Vertex)2 File (java.io.File)2 Test (org.junit.Test)2 SchemaManager (com.baidu.hugegraph.driver.SchemaManager)1 LoadParameter (com.baidu.hugegraph.entity.load.LoadParameter)1 LoadTask (com.baidu.hugegraph.entity.load.LoadTask)1 ExternalException (com.baidu.hugegraph.exception.ExternalException)1 InternalException (com.baidu.hugegraph.exception.InternalException)1 GroovyExecutor (com.baidu.hugegraph.loader.executor.GroovyExecutor)1 LoadMapping (com.baidu.hugegraph.loader.mapping.LoadMapping)1 LoadProgress (com.baidu.hugegraph.loader.progress.LoadProgress)1