Search in sources :

Example 6 with LoadException

use of com.baidu.hugegraph.loader.exception.LoadException in project incubator-hugegraph-toolchain by apache.

the class MappingUtil method write.

public static void write(LoadMapping mapping, String path) {
    File file = FileUtils.getFile(path);
    String json = JsonUtil.toJson(mapping);
    try {
        FileUtils.write(file, json, Constants.CHARSET);
    } catch (IOException e) {
        throw new LoadException("Failed to write mapping %s to file '%s'", e, mapping, file);
    }
}
Also used : IOException(java.io.IOException) File(java.io.File) LoadException(com.baidu.hugegraph.loader.exception.LoadException)

Example 7 with LoadException

use of com.baidu.hugegraph.loader.exception.LoadException in project incubator-hugegraph-toolchain by apache.

the class GraphStructV1 method of.

public static GraphStructV1 of(LoadContext context) {
    LoadOptions options = context.options();
    File file = FileUtils.getFile(options.file);
    try {
        String json = FileUtils.readFileToString(file, Constants.CHARSET);
        GraphStructV1 struct = JsonUtil.fromJson(json, GraphStructV1.class);
        struct.check();
        return struct;
    } catch (IOException | IllegalArgumentException e) {
        throw new LoadException("Failed to parse graph mapping description file '%s'", e, options.file);
    }
}
Also used : LoadOptions(com.baidu.hugegraph.loader.executor.LoadOptions) IOException(java.io.IOException) File(java.io.File) LoadException(com.baidu.hugegraph.loader.exception.LoadException)

Example 8 with LoadException

use of com.baidu.hugegraph.loader.exception.LoadException in project incubator-hugegraph-toolchain by apache.

the class FileLineFetcher method createCompressReader.

private static Reader createCompressReader(InputStream stream, FileSource source) throws Exception {
    Compression compression = source.compression();
    String charset = source.charset();
    switch(compression) {
        case NONE:
            return new InputStreamReader(stream, charset);
        case SNAPPY_RAW:
            Configuration config = new Configuration();
            CompressionCodec codec = ReflectionUtils.newInstance(SnappyCodec.class, config);
            CompressionInputStream sis = codec.createInputStream(stream, codec.createDecompressor());
            return new InputStreamReader(sis, charset);
        case GZIP:
        case BZ2:
        case XZ:
        case LZMA:
        case SNAPPY_FRAMED:
        case Z:
        case DEFLATE:
        case LZ4_BLOCK:
        case LZ4_FRAMED:
            CompressorStreamFactory factory = new CompressorStreamFactory();
            CompressorInputStream cis = factory.createCompressorInputStream(compression.string(), stream);
            return new InputStreamReader(cis, charset);
        default:
            throw new LoadException("Unsupported compression format '%s'", compression);
    }
}
Also used : Compression(com.baidu.hugegraph.loader.source.file.Compression) InputStreamReader(java.io.InputStreamReader) Configuration(org.apache.hadoop.conf.Configuration) CompressionInputStream(org.apache.hadoop.io.compress.CompressionInputStream) CompressorStreamFactory(org.apache.commons.compress.compressors.CompressorStreamFactory) CompressionCodec(org.apache.hadoop.io.compress.CompressionCodec) CompressorInputStream(org.apache.commons.compress.compressors.CompressorInputStream) LoadException(com.baidu.hugegraph.loader.exception.LoadException)

Example 9 with LoadException

use of com.baidu.hugegraph.loader.exception.LoadException in project incubator-hugegraph-toolchain by apache.

the class FileLineFetcher method openReader.

@Override
public void openReader(Readable readable) {
    InputStream stream = null;
    try {
        stream = readable.open();
        this.reader = createBufferedReader(stream, this.source());
    } catch (IOException e) {
        throw new LoadException("Failed to open stream for '%s'", e, readable);
    } catch (Exception e) {
        if (stream != null) {
            try {
                stream.close();
            } catch (IOException ignored) {
                LOG.warn("Failed to close stream of '{}'", readable);
            }
        }
        throw new LoadException("Failed to create reader for '%s'", e, readable);
    }
    // Mark as fresh
    this.resetStatus();
}
Also used : CompressorInputStream(org.apache.commons.compress.compressors.CompressorInputStream) CompressionInputStream(org.apache.hadoop.io.compress.CompressionInputStream) InputStream(java.io.InputStream) IOException(java.io.IOException) LoadException(com.baidu.hugegraph.loader.exception.LoadException) IOException(java.io.IOException) LoadException(com.baidu.hugegraph.loader.exception.LoadException)

Example 10 with LoadException

use of com.baidu.hugegraph.loader.exception.LoadException in project incubator-hugegraph-toolchain by apache.

the class JDBCUtil method escapePostgresql.

public static String escapePostgresql(String value) {
    StringBuilder builder = new StringBuilder(8 + value.length());
    builder.append('\'');
    try {
        Utils.escapeLiteral(builder, value, false);
    } catch (SQLException e) {
        throw new LoadException("Failed to escape '%s'", e, value);
    }
    builder.append('\'');
    return builder.toString();
}
Also used : SQLException(java.sql.SQLException) LoadException(com.baidu.hugegraph.loader.exception.LoadException)

Aggregations

LoadException (com.baidu.hugegraph.loader.exception.LoadException)32 IOException (java.io.IOException)18 File (java.io.File)10 SQLException (java.sql.SQLException)4 ArrayList (java.util.ArrayList)4 FileFilter (com.baidu.hugegraph.loader.source.file.FileFilter)3 Path (org.apache.hadoop.fs.Path)3 ServerException (com.baidu.hugegraph.exception.ServerException)2 InitException (com.baidu.hugegraph.loader.exception.InitException)2 LoadOptions (com.baidu.hugegraph.loader.executor.LoadOptions)2 LoadSummary (com.baidu.hugegraph.loader.metrics.LoadSummary)2 Readable (com.baidu.hugegraph.loader.reader.Readable)2 InputStream (java.io.InputStream)2 InputStreamReader (java.io.InputStreamReader)2 CompressorInputStream (org.apache.commons.compress.compressors.CompressorInputStream)2 CompressionInputStream (org.apache.hadoop.io.compress.CompressionInputStream)2 HugeClient (com.baidu.hugegraph.driver.HugeClient)1 HugeClientBuilder (com.baidu.hugegraph.driver.HugeClientBuilder)1 GroovyExecutor (com.baidu.hugegraph.loader.executor.GroovyExecutor)1 InputStruct (com.baidu.hugegraph.loader.mapping.InputStruct)1