Search in sources :

Example 11 with Args

use of org.neo4j.helpers.Args in project graphdb by neo4j-attic.

the class BackupTool method main.

public static void main(String[] args) {
    Args arguments = new Args(args);
    checkArguments(arguments);
    boolean full = arguments.has(FULL);
    String from = arguments.get(FROM, null);
    String to = arguments.get(TO, null);
    URI backupURI = null;
    try {
        backupURI = new URI(from);
    } catch (URISyntaxException e) {
        System.out.println("Please properly specify a location to backup as a valid URI in the form <scheme>://<host>[:port], where scheme is the target database's running mode, eg ha");
        exitAbnormally();
    }
    String module = backupURI.getScheme();
    /*
         * So, the scheme is considered to be the module name and an attempt at
         * loading the service is made.
         */
    BackupExtensionService service = null;
    if (module != null && !DEFAULT_SCHEME.equals(module)) {
        try {
            service = Service.load(BackupExtensionService.class, module);
        } catch (NoSuchElementException e) {
            System.out.println(String.format("%s was specified as a backup module but it was not found. Please make sure that the implementing service is on the classpath.", module));
            exitAbnormally();
        }
    }
    if (service != null) {
        // If in here, it means a module was loaded. Use it and substitute the
        // passed URI
        backupURI = service.resolve(backupURI);
    }
    doBackup(full, backupURI, to);
}
Also used : Args(org.neo4j.helpers.Args) URISyntaxException(java.net.URISyntaxException) URI(java.net.URI) NoSuchElementException(java.util.NoSuchElementException)

Example 12 with Args

use of org.neo4j.helpers.Args in project neo4j by neo4j.

the class ReplayRaftLog method main.

public static void main(String[] args) throws IOException {
    Args arg = Args.parse(args);
    String from = arg.get("from");
    System.out.println("From is " + from);
    String to = arg.get("to");
    System.out.println("to is " + to);
    File logDirectory = new File(from);
    System.out.println("logDirectory = " + logDirectory);
    Config config = Config.embeddedDefaults(stringMap());
    try (DefaultFileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction()) {
        LogProvider logProvider = getInstance();
        CoreLogPruningStrategy pruningStrategy = new CoreLogPruningStrategyFactory(config.get(raft_log_pruning_strategy), logProvider).newInstance();
        SegmentedRaftLog log = new SegmentedRaftLog(fileSystem, logDirectory, config.get(raft_log_rotation_size), new CoreReplicatedContentMarshal(), logProvider, config.get(raft_log_reader_pool_size), Clocks.systemClock(), new OnDemandJobScheduler(), pruningStrategy);
        // Not really, but we need to have a way to pass in the commit index
        long totalCommittedEntries = log.appendIndex();
        for (int i = 0; i <= totalCommittedEntries; i++) {
            ReplicatedContent content = readLogEntry(log, i).content();
            if (content instanceof ReplicatedTransaction) {
                ReplicatedTransaction tx = (ReplicatedTransaction) content;
                ReplicatedTransactionFactory.extractTransactionRepresentation(tx, new byte[0]).accept(element -> {
                    System.out.println(element);
                    return false;
                });
            }
        }
    }
}
Also used : Args(org.neo4j.helpers.Args) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) SegmentedRaftLog(org.neo4j.causalclustering.core.consensus.log.segmented.SegmentedRaftLog) ReplicatedTransaction(org.neo4j.causalclustering.core.state.machines.tx.ReplicatedTransaction) Config(org.neo4j.kernel.configuration.Config) CoreLogPruningStrategyFactory(org.neo4j.causalclustering.core.consensus.log.segmented.CoreLogPruningStrategyFactory) CoreLogPruningStrategy(org.neo4j.causalclustering.core.consensus.log.segmented.CoreLogPruningStrategy) OnDemandJobScheduler(org.neo4j.test.OnDemandJobScheduler) LogProvider(org.neo4j.logging.LogProvider) CoreReplicatedContentMarshal(org.neo4j.causalclustering.messaging.CoreReplicatedContentMarshal) ReplicatedContent(org.neo4j.causalclustering.core.replication.ReplicatedContent) File(java.io.File)

Example 13 with Args

use of org.neo4j.helpers.Args in project neo4j by neo4j.

the class LegacyDatabaseImpl method main.

// This has to be adapted to the way HA GDB is started in the specific old version it's used for.
public static void main(String[] args) throws Exception {
    Args arguments = Args.parse(args);
    File storeDir = new File(arguments.orphans().get(0));
    GraphDatabaseAPI db = (GraphDatabaseAPI) new TestHighlyAvailableGraphDatabaseFactory().newEmbeddedDatabaseBuilder(storeDir).setConfig(arguments.asMap()).newGraphDatabase();
    LegacyDatabaseImpl legacyDb = new LegacyDatabaseImpl(storeDir, db);
    rmiLocation(parseInt(arguments.orphans().get(1))).bind(legacyDb);
}
Also used : Args(org.neo4j.helpers.Args) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) TestHighlyAvailableGraphDatabaseFactory(org.neo4j.graphdb.factory.TestHighlyAvailableGraphDatabaseFactory) File(java.io.File)

Example 14 with Args

use of org.neo4j.helpers.Args in project neo4j by neo4j.

the class ImportTool method main.

/**
     * Runs the import tool given the supplied arguments.
     *
     * @param incomingArguments arguments for specifying input and configuration for the import.
     * @param defaultSettingsSuitableForTests default configuration geared towards unit/integration
     * test environments, for example lower default buffer sizes.
     */
public static void main(String[] incomingArguments, boolean defaultSettingsSuitableForTests) throws IOException {
    System.err.println("WARNING: neo4j-import is deprecated and support for it will be removed in a future\n" + "version of Neo4j; please use neo4j-admin import instead.\n");
    PrintStream out = System.out;
    PrintStream err = System.err;
    Args args = Args.parse(incomingArguments);
    if (ArrayUtil.isEmpty(incomingArguments) || asksForUsage(args)) {
        printUsage(out);
        return;
    }
    File storeDir;
    Collection<Option<File[]>> nodesFiles, relationshipsFiles;
    boolean enableStacktrace;
    Number processors = null;
    Input input = null;
    int badTolerance;
    Charset inputEncoding;
    boolean skipBadRelationships, skipDuplicateNodes, ignoreExtraColumns;
    Config dbConfig;
    OutputStream badOutput = null;
    IdType idType = null;
    int pageSize = UNSPECIFIED;
    Collector badCollector;
    org.neo4j.unsafe.impl.batchimport.Configuration configuration = null;
    File logsDir;
    File badFile;
    boolean success = false;
    try (FileSystemAbstraction fs = new DefaultFileSystemAbstraction()) {
        storeDir = args.interpretOption(Options.STORE_DIR.key(), Converters.<File>mandatory(), Converters.toFile(), Validators.DIRECTORY_IS_WRITABLE, Validators.CONTAINS_NO_EXISTING_DATABASE);
        Config config = Config.defaults();
        config.augment(stringMap(GraphDatabaseSettings.neo4j_home.name(), storeDir.getAbsolutePath()));
        logsDir = config.get(GraphDatabaseSettings.logs_directory);
        fs.mkdirs(logsDir);
        badFile = new File(storeDir, BAD_FILE_NAME);
        badOutput = new BufferedOutputStream(fs.openAsOutputStream(badFile, false));
        nodesFiles = extractInputFiles(args, Options.NODE_DATA.key(), err);
        relationshipsFiles = extractInputFiles(args, Options.RELATIONSHIP_DATA.key(), err);
        validateInputFiles(nodesFiles, relationshipsFiles);
        enableStacktrace = args.getBoolean(Options.STACKTRACE.key(), Boolean.FALSE, Boolean.TRUE);
        processors = args.getNumber(Options.PROCESSORS.key(), null);
        idType = args.interpretOption(Options.ID_TYPE.key(), withDefault((IdType) Options.ID_TYPE.defaultValue()), TO_ID_TYPE);
        badTolerance = parseNumberOrUnlimited(args, Options.BAD_TOLERANCE);
        inputEncoding = Charset.forName(args.get(Options.INPUT_ENCODING.key(), defaultCharset().name()));
        skipBadRelationships = args.getBoolean(Options.SKIP_BAD_RELATIONSHIPS.key(), (Boolean) Options.SKIP_BAD_RELATIONSHIPS.defaultValue(), true);
        skipDuplicateNodes = args.getBoolean(Options.SKIP_DUPLICATE_NODES.key(), (Boolean) Options.SKIP_DUPLICATE_NODES.defaultValue(), true);
        ignoreExtraColumns = args.getBoolean(Options.IGNORE_EXTRA_COLUMNS.key(), (Boolean) Options.IGNORE_EXTRA_COLUMNS.defaultValue(), true);
        badCollector = badCollector(badOutput, badTolerance, collect(skipBadRelationships, skipDuplicateNodes, ignoreExtraColumns));
        dbConfig = loadDbConfig(args.interpretOption(Options.DATABASE_CONFIG.key(), Converters.<File>optional(), Converters.toFile(), Validators.REGEX_FILE_EXISTS));
        configuration = importConfiguration(processors, defaultSettingsSuitableForTests, dbConfig, pageSize);
        input = new CsvInput(nodeData(inputEncoding, nodesFiles), defaultFormatNodeFileHeader(), relationshipData(inputEncoding, relationshipsFiles), defaultFormatRelationshipFileHeader(), idType, csvConfiguration(args, defaultSettingsSuitableForTests), badCollector, configuration.maxNumberOfProcessors());
        doImport(out, err, storeDir, logsDir, badFile, fs, nodesFiles, relationshipsFiles, enableStacktrace, input, dbConfig, badOutput, configuration);
        success = true;
    } catch (IllegalArgumentException e) {
        throw andPrintError("Input error", e, false, err);
    } catch (IOException e) {
        throw andPrintError("File error", e, false, err);
    } finally {
        if (!success && badOutput != null) {
            badOutput.close();
        }
    }
}
Also used : DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Config(org.neo4j.kernel.configuration.Config) BufferedOutputStream(java.io.BufferedOutputStream) OutputStream(java.io.OutputStream) CsvInput(org.neo4j.unsafe.impl.batchimport.input.csv.CsvInput) Input(org.neo4j.unsafe.impl.batchimport.input.Input) BadCollector(org.neo4j.unsafe.impl.batchimport.input.BadCollector) Collector(org.neo4j.unsafe.impl.batchimport.input.Collector) Collectors.badCollector(org.neo4j.unsafe.impl.batchimport.input.Collectors.badCollector) BufferedOutputStream(java.io.BufferedOutputStream) PrintStream(java.io.PrintStream) Args(org.neo4j.helpers.Args) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) Charset.defaultCharset(java.nio.charset.Charset.defaultCharset) Charset(java.nio.charset.Charset) CsvInput(org.neo4j.unsafe.impl.batchimport.input.csv.CsvInput) IOException(java.io.IOException) IdType(org.neo4j.unsafe.impl.batchimport.input.csv.IdType) Option(org.neo4j.helpers.Args.Option) StoreFile(org.neo4j.kernel.impl.storemigration.StoreFile) File(java.io.File)

Example 15 with Args

use of org.neo4j.helpers.Args in project neo4j by neo4j.

the class QuickImport method main.

public static void main(String[] arguments) throws IOException {
    Args args = Args.parse(arguments);
    long nodeCount = parseLongWithUnit(args.get("nodes", null));
    long relationshipCount = parseLongWithUnit(args.get("relationships", null));
    int labelCount = args.getNumber("labels", 4).intValue();
    int relationshipTypeCount = args.getNumber("relationship-types", 4).intValue();
    File dir = new File(args.get(ImportTool.Options.STORE_DIR.key()));
    long randomSeed = args.getNumber("random-seed", currentTimeMillis()).longValue();
    Configuration config = COMMAS;
    Extractors extractors = new Extractors(config.arrayDelimiter());
    IdType idType = IdType.valueOf(args.get("id-type", IdType.ACTUAL.name()));
    Header nodeHeader = parseNodeHeader(args, idType, extractors);
    Header relationshipHeader = parseRelationshipHeader(args, idType, extractors);
    FormattedLogProvider sysoutLogProvider = FormattedLogProvider.toOutputStream(System.out);
    org.neo4j.unsafe.impl.batchimport.Configuration importConfig = new Default() {

        @Override
        public int maxNumberOfProcessors() {
            return args.getNumber(ImportTool.Options.PROCESSORS.key(), super.maxNumberOfProcessors()).intValue();
        }

        @Override
        public int denseNodeThreshold() {
            return args.getNumber(dense_node_threshold.name(), super.denseNodeThreshold()).intValue();
        }
    };
    SimpleDataGenerator generator = new SimpleDataGenerator(nodeHeader, relationshipHeader, randomSeed, nodeCount, labelCount, relationshipTypeCount, idType);
    Input input = new DataGeneratorInput(nodeCount, relationshipCount, generator.nodes(), generator.relationships(), idType, silentBadCollector(0));
    try (FileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction()) {
        BatchImporter consumer;
        if (args.getBoolean("to-csv")) {
            consumer = new CsvOutput(dir, nodeHeader, relationshipHeader, config);
        } else {
            consumer = new ParallelBatchImporter(dir, fileSystem, importConfig, new SimpleLogService(sysoutLogProvider, sysoutLogProvider), defaultVisible(), Config.defaults());
        }
        consumer.doImport(input);
    }
}
Also used : Args(org.neo4j.helpers.Args) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) Configuration(org.neo4j.unsafe.impl.batchimport.input.csv.Configuration) SimpleLogService(org.neo4j.kernel.impl.logging.SimpleLogService) Default(org.neo4j.unsafe.impl.batchimport.Configuration.Default) IdType(org.neo4j.unsafe.impl.batchimport.input.csv.IdType) FormattedLogProvider(org.neo4j.logging.FormattedLogProvider) ParallelBatchImporter(org.neo4j.unsafe.impl.batchimport.ParallelBatchImporter) Extractors(org.neo4j.csv.reader.Extractors) Input(org.neo4j.unsafe.impl.batchimport.input.Input) Header(org.neo4j.unsafe.impl.batchimport.input.csv.Header) DataGeneratorInput.bareboneNodeHeader(org.neo4j.tooling.DataGeneratorInput.bareboneNodeHeader) DataFactories.defaultFormatRelationshipFileHeader(org.neo4j.unsafe.impl.batchimport.input.csv.DataFactories.defaultFormatRelationshipFileHeader) DataFactories.defaultFormatNodeFileHeader(org.neo4j.unsafe.impl.batchimport.input.csv.DataFactories.defaultFormatNodeFileHeader) DataGeneratorInput.bareboneRelationshipHeader(org.neo4j.tooling.DataGeneratorInput.bareboneRelationshipHeader) ParallelBatchImporter(org.neo4j.unsafe.impl.batchimport.ParallelBatchImporter) BatchImporter(org.neo4j.unsafe.impl.batchimport.BatchImporter) File(java.io.File)

Aggregations

Args (org.neo4j.helpers.Args)16 File (java.io.File)12 DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)7 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)6 PrintStream (java.io.PrintStream)3 Config (org.neo4j.kernel.configuration.Config)3 FormattedLogProvider (org.neo4j.logging.FormattedLogProvider)3 IOException (java.io.IOException)2 GraphDatabaseAPI (org.neo4j.kernel.internal.GraphDatabaseAPI)2 LogProvider (org.neo4j.logging.LogProvider)2 Input (org.neo4j.unsafe.impl.batchimport.input.Input)2 IdType (org.neo4j.unsafe.impl.batchimport.input.csv.IdType)2 BufferedOutputStream (java.io.BufferedOutputStream)1 InputStream (java.io.InputStream)1 OutputStream (java.io.OutputStream)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1 Charset (java.nio.charset.Charset)1 Charset.defaultCharset (java.nio.charset.Charset.defaultCharset)1 RemoteException (java.rmi.RemoteException)1