Search in sources :

Example 1 with ExecutionContext

use of org.neo4j.cli.ExecutionContext in project neo4j by neo4j.

the class ImportCommandTest method shouldKeepSpecifiedNeo4jHomeWhenNoAdditionalConfigIsPresent.

@Test
void shouldKeepSpecifiedNeo4jHomeWhenNoAdditionalConfigIsPresent() {
    // given
    final var homeDir = testDir.directory("other", "place");
    final var ctx = new ExecutionContext(homeDir, testDir.directory("conf"), System.out, System.err, testDir.getFileSystem());
    final var command = new ImportCommand(ctx);
    final var foo = testDir.createFile("foo.csv");
    CommandLine.populateCommand(command, "--nodes=" + foo.toAbsolutePath());
    // when
    Config resultingConfig = command.loadNeo4jConfig();
    // then
    assertEquals(homeDir, resultingConfig.get(GraphDatabaseSettings.neo4j_home));
}
Also used : ExecutionContext(org.neo4j.cli.ExecutionContext) Config(org.neo4j.configuration.Config) Test(org.junit.jupiter.api.Test)

Example 2 with ExecutionContext

use of org.neo4j.cli.ExecutionContext in project neo4j by neo4j.

the class ImportCommandTest method shouldKeepSpecifiedNeo4jHomeWhenAdditionalConfigIsPresent.

@Test
void shouldKeepSpecifiedNeo4jHomeWhenAdditionalConfigIsPresent() {
    // given
    final var homeDir = testDir.directory("other", "place");
    final var additionalConfigFile = testDir.createFile("empty.conf");
    final var ctx = new ExecutionContext(homeDir, testDir.directory("conf"), System.out, System.err, testDir.getFileSystem());
    final var command = new ImportCommand(ctx);
    final var foo = testDir.createFile("foo.csv");
    CommandLine.populateCommand(command, "--additional-config", additionalConfigFile.toAbsolutePath().toString(), "--nodes=" + foo.toAbsolutePath().toString());
    // when
    Config resultingConfig = command.loadNeo4jConfig();
    // then
    assertEquals(homeDir, resultingConfig.get(GraphDatabaseSettings.neo4j_home));
}
Also used : ExecutionContext(org.neo4j.cli.ExecutionContext) Config(org.neo4j.configuration.Config) Test(org.junit.jupiter.api.Test)

Example 3 with ExecutionContext

use of org.neo4j.cli.ExecutionContext in project neo4j by neo4j.

the class ImportCommandTest method printUsageHelp.

@Test
void printUsageHelp() {
    final var baos = new ByteArrayOutputStream();
    final var command = new ImportCommand(new ExecutionContext(Path.of("."), Path.of(".")));
    try (var out = new PrintStream(baos)) {
        CommandLine.usage(command, new PrintStream(out), CommandLine.Help.Ansi.OFF);
    }
    assertEquals("USAGE" + lineSeparator() + "" + lineSeparator() + "import [--expand-commands] [--verbose] [--cache-on-heap[=<true/false>]] [--force" + lineSeparator() + "       [=<true/false>]] [--high-io[=<true/false>]] [--ignore-empty-strings" + lineSeparator() + "       [=<true/false>]] [--ignore-extra-columns[=<true/false>]]" + lineSeparator() + "       [--legacy-style-quoting[=<true/false>]] [--multiline-fields" + lineSeparator() + "       [=<true/false>]] [--normalize-types[=<true/false>]]" + lineSeparator() + "       [--skip-bad-entries-logging[=<true/false>]] [--skip-bad-relationships" + lineSeparator() + "       [=<true/false>]] [--skip-duplicate-nodes[=<true/false>]] [--trim-strings" + lineSeparator() + "       [=<true/false>]] [--additional-config=<path>] [--array-delimiter=<char>]" + lineSeparator() + "       [--bad-tolerance=<num>] [--database=<database>] [--delimiter=<char>]" + lineSeparator() + "       [--id-type=<STRING|INTEGER|ACTUAL>] [--input-encoding=<character-set>]" + lineSeparator() + "       [--max-memory=<size>] [--processors=<num>] [--quote=<char>]" + lineSeparator() + "       [--read-buffer-size=<size>] [--report-file=<path>] --nodes=[<label>[:" + lineSeparator() + "       <label>]...=]<files>... [--nodes=[<label>[:<label>]...=]<files>...]..." + lineSeparator() + "       [--relationships=[<type>=]<files>...]..." + lineSeparator() + "" + lineSeparator() + "DESCRIPTION" + lineSeparator() + "" + lineSeparator() + "Import a collection of CSV files." + lineSeparator() + "" + lineSeparator() + "OPTIONS" + lineSeparator() + "" + lineSeparator() + "      --verbose              Enable verbose output." + lineSeparator() + "      --expand-commands      Allow command expansion in config value evaluation." + lineSeparator() + "      --database=<database>  Name of the database to import." + lineSeparator() + "                               If the database used to import into doesn't" + lineSeparator() + "                               exist prior to importing," + lineSeparator() + "                               then it must be created subsequently using" + lineSeparator() + "                               CREATE DATABASE." + lineSeparator() + "                               Default: neo4j" + lineSeparator() + "      --additional-config=<path>" + lineSeparator() + "                             Configuration file to supply additional" + lineSeparator() + "                               configuration in." + lineSeparator() + "      --report-file=<path>   File in which to store the report of the" + lineSeparator() + "                               csv-import." + lineSeparator() + "                               Default: import.report" + lineSeparator() + "      --force[=<true/false>] Force will delete any existing database files" + lineSeparator() + "                               prior to the import." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --id-type=<STRING|INTEGER|ACTUAL>" + lineSeparator() + "                             Each node must provide a unique id. This is used" + lineSeparator() + "                               to find the correct nodes when creating" + lineSeparator() + "                               relationships. Possible values are:" + lineSeparator() + "                               STRING: arbitrary strings for identifying nodes," + lineSeparator() + "                               INTEGER: arbitrary integer values for" + lineSeparator() + "                               identifying nodes," + lineSeparator() + "                               ACTUAL: (advanced) actual node ids." + lineSeparator() + "                             For more information on id handling, please see" + lineSeparator() + "                               the Neo4j Manual: https://neo4j." + lineSeparator() + "                               com/docs/operations-manual/current/tools/import/" + lineSeparator() + "                               Default: STRING" + lineSeparator() + "      --input-encoding=<character-set>" + lineSeparator() + "                             Character set that input data is encoded in." + lineSeparator() + "                               Default: UTF-8" + lineSeparator() + "      --ignore-extra-columns[=<true/false>]" + lineSeparator() + "                             If un-specified columns should be ignored during" + lineSeparator() + "                               the import." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --multiline-fields[=<true/false>]" + lineSeparator() + "                             Whether or not fields from input source can span" + lineSeparator() + "                               multiple lines, i.e. contain newline characters." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --ignore-empty-strings[=<true/false>]" + lineSeparator() + "                             Whether or not empty string fields, i.e. \"\" from" + lineSeparator() + "                               input source are ignored, i.e. treated as null." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --trim-strings[=<true/false>]" + lineSeparator() + "                             Whether or not strings should be trimmed for" + lineSeparator() + "                               whitespaces." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --legacy-style-quoting[=<true/false>]" + lineSeparator() + "                             Whether or not backslash-escaped quote e.g. \\\" is" + lineSeparator() + "                               interpreted as inner quote." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --delimiter=<char>     Delimiter character between values in CSV data." + lineSeparator() + "                               Also accepts 'TAB' and e.g. 'U+20AC' for" + lineSeparator() + "                               specifying character using unicode." + lineSeparator() + "                               Default: ," + lineSeparator() + "      --array-delimiter=<char>" + lineSeparator() + "                             Delimiter character between array elements within" + lineSeparator() + "                               a value in CSV data. Also accepts 'TAB' and e.g." + lineSeparator() + "                               'U+20AC' for specifying character using unicode." + lineSeparator() + "                               Default: ;" + lineSeparator() + "      --quote=<char>         Character to treat as quotation character for" + lineSeparator() + "                               values in CSV data. Quotes can be escaped as per" + lineSeparator() + "                               RFC 4180 by doubling them, for example \"\" would" + lineSeparator() + "                               be interpreted as a literal \". You cannot escape" + lineSeparator() + "                               using \\." + lineSeparator() + "                               Default: \"" + lineSeparator() + "      --read-buffer-size=<size>" + lineSeparator() + "                             Size of each buffer for reading input data. The" + lineSeparator() + "                               size has to at least be large enough to hold the" + lineSeparator() + "                               biggest single value in the input data. The" + lineSeparator() + "                               value can be a plain number or a byte units" + lineSeparator() + "                               string, e.g. 128k, 1m." + lineSeparator() + "                               Default: 4194304" + lineSeparator() + "      --max-memory=<size>    Maximum memory that neo4j-admin can use for" + lineSeparator() + "                               various data structures and caching to improve" + lineSeparator() + "                               performance. Values can be plain numbers, like" + lineSeparator() + "                               10000000 or e.g. 20G for 20 gigabyte, or even e." + lineSeparator() + "                               g. 70%." + lineSeparator() + "                               Default: 90%" + lineSeparator() + "      --high-io[=<true/false>]" + lineSeparator() + "                             Ignore environment-based heuristics, and assume" + lineSeparator() + "                               that the target storage subsystem can support" + lineSeparator() + "                               parallel IO with high throughput." + lineSeparator() + "                               Default: null" + lineSeparator() + "      --cache-on-heap[=<true/false>]" + lineSeparator() + "                             (advanced) Whether or not to allow allocating" + lineSeparator() + "                               memory for the cache on heap. If 'false' then" + lineSeparator() + "                               caches will still be allocated off-heap, but the" + lineSeparator() + "                               additional free memory inside the JVM will not" + lineSeparator() + "                               be allocated for the caches. Use this option to" + lineSeparator() + "                               be able to have better control over the heap" + lineSeparator() + "                               memory." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --processors=<num>     (advanced) Max number of processors used by the" + lineSeparator() + "                               importer. Defaults to the number of available" + lineSeparator() + "                               processors reported by the JVM. There is a" + lineSeparator() + "                               certain amount of minimum threads needed so for" + lineSeparator() + "                               that reason there is no lower bound for this" + lineSeparator() + "                               value. For optimal performance this value" + lineSeparator() + "                               shouldn't be greater than the number of" + lineSeparator() + "                               available processors." + lineSeparator() + "                               Default: " + Runtime.getRuntime().availableProcessors() + lineSeparator() + "      --bad-tolerance=<num>  Number of bad entries before the import is" + lineSeparator() + "                               considered failed. This tolerance threshold is" + lineSeparator() + "                               about relationships referring to missing nodes." + lineSeparator() + "                               Format errors in input data are still treated as" + lineSeparator() + "                               errors" + lineSeparator() + "                               Default: 1000" + lineSeparator() + "      --skip-bad-entries-logging[=<true/false>]" + lineSeparator() + "                             Whether or not to skip logging bad entries" + lineSeparator() + "                               detected during import." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --skip-bad-relationships[=<true/false>]" + lineSeparator() + "                             Whether or not to skip importing relationships" + lineSeparator() + "                               that refers to missing node ids, i.e. either" + lineSeparator() + "                               start or end node id/group referring to node" + lineSeparator() + "                               that wasn't specified by the node input data." + lineSeparator() + "                               Skipped relationships will be logged, containing" + lineSeparator() + "                               at most number of entities specified by" + lineSeparator() + "                               bad-tolerance, unless otherwise specified by" + lineSeparator() + "                               skip-bad-entries-logging option." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --skip-duplicate-nodes[=<true/false>]" + lineSeparator() + "                             Whether or not to skip importing nodes that have" + lineSeparator() + "                               the same id/group. In the event of multiple" + lineSeparator() + "                               nodes within the same group having the same id," + lineSeparator() + "                               the first encountered will be imported whereas" + lineSeparator() + "                               consecutive such nodes will be skipped. Skipped" + lineSeparator() + "                               nodes will be logged, containing at most number" + lineSeparator() + "                               of entities specified by bad-tolerance, unless" + lineSeparator() + "                               otherwise specified by skip-bad-entries-logging" + lineSeparator() + "                               option." + lineSeparator() + "                               Default: false" + lineSeparator() + "      --normalize-types[=<true/false>]" + lineSeparator() + "                             Whether or not to normalize property types to" + lineSeparator() + "                               Cypher types, e.g. 'int' becomes 'long' and" + lineSeparator() + "                               'float' becomes 'double'" + lineSeparator() + "                               Default: true" + lineSeparator() + "      --nodes=[<label>[:<label>]...=]<files>..." + lineSeparator() + "                             Node CSV header and data. Multiple files will be" + lineSeparator() + "                               logically seen as one big file from the" + lineSeparator() + "                               perspective of the importer. The first line must" + lineSeparator() + "                               contain the header. Multiple data sources like" + lineSeparator() + "                               these can be specified in one import, where each" + lineSeparator() + "                               data source has its own header." + lineSeparator() + "      --relationships=[<type>=]<files>..." + lineSeparator() + "                             Relationship CSV header and data. Multiple files" + lineSeparator() + "                               will be logically seen as one big file from the" + lineSeparator() + "                               perspective of the importer. The first line must" + lineSeparator() + "                               contain the header. Multiple data sources like" + lineSeparator() + "                               these can be specified in one import, where each" + lineSeparator() + "                               data source has its own header.", baos.toString().trim());
}
Also used : PrintStream(java.io.PrintStream) ExecutionContext(org.neo4j.cli.ExecutionContext) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Test(org.junit.jupiter.api.Test)

Example 4 with ExecutionContext

use of org.neo4j.cli.ExecutionContext in project neo4j by neo4j.

the class AdminCommandsIT method setup.

@BeforeEach
void setup() throws Exception {
    out = mock(PrintStream.class);
    err = mock(PrintStream.class);
    confDir = testDirectory.directory("test.conf");
    home = testDirectory.homePath("home");
    context = new ExecutionContext(home, confDir, out, err, testDirectory.getFileSystem());
    Path configFile = confDir.resolve("neo4j.conf");
    Files.createFile(configFile, PosixFilePermissions.asFileAttribute(Set.of(OWNER_READ, OWNER_WRITE)));
    GraphDatabaseSettings.strict_config_validation.name();
    Files.write(configFile, (BootloaderSettings.initial_heap_size.name() + "=$(expr 500)").getBytes());
}
Also used : Path(java.nio.file.Path) PrintStream(java.io.PrintStream) ExecutionContext(org.neo4j.cli.ExecutionContext) BeforeEach(org.junit.jupiter.api.BeforeEach)

Example 5 with ExecutionContext

use of org.neo4j.cli.ExecutionContext in project neo4j by neo4j.

the class CheckConsistencyCommandIT method printUsageHelp.

@Test
void printUsageHelp() {
    final var baos = new ByteArrayOutputStream();
    final var command = new CheckConsistencyCommand(new ExecutionContext(Path.of("."), Path.of(".")));
    try (var out = new PrintStream(baos)) {
        CommandLine.usage(command, new PrintStream(out), CommandLine.Help.Ansi.OFF);
    }
    assertThat(baos.toString().trim()).isEqualTo(String.format("Check the consistency of a database.%n" + "%n" + "USAGE%n" + "%n" + "check-consistency [--expand-commands] [--verbose] [--additional-config=<path>]%n" + "                  [--check-graph=<true/false>]%n" + "                  [--check-index-structure=<true/false>]%n" + "                  [--check-indexes=<true/false>]%n" + "                  [--check-label-scan-store=<true/false>]%n" + "                  [--check-property-owners=<true/false>]%n" + "                  [--check-relationship-type-scan-store=<true/false>]%n" + "                  [--report-dir=<path>] (--database=<database> |%n" + "                  --backup=<path>)%n" + "%n" + "DESCRIPTION%n" + "%n" + "This command allows for checking the consistency of a database or a backup%n" + "thereof. It cannot be used with a database which is currently in use.%n" + "%n" + "All checks except 'check-graph' can be quite expensive so it may be useful to%n" + "turn them off for very large databases. Increasing the heap size can also be a%n" + "good idea. See 'neo4j-admin help' for details.%n" + "%n" + "OPTIONS%n" + "%n" + "      --verbose             Enable verbose output.%n" + "      --expand-commands     Allow command expansion in config value evaluation.%n" + "      --database=<database> Name of the database to check.%n" + "      --backup=<path>       Path to backup to check consistency of. Cannot be%n" + "                              used together with --database.%n" + "      --additional-config=<path>%n" + "                            Configuration file to supply additional%n" + "                              configuration in.%n" + "      --report-dir=<path>   Directory where consistency report will be written.%n" + "                              Default: .%n" + "      --check-graph=<true/false>%n" + "                            Perform consistency checks between nodes,%n" + "                              relationships, properties, types and tokens.%n" + "                              Default: true%n" + "      --check-indexes=<true/false>%n" + "                            Perform consistency checks on indexes.%n" + "                              Default: true%n" + "      --check-index-structure=<true/false>%n" + "                            Perform structure checks on indexes.%n" + "                              Default: true%n" + "      --check-label-scan-store=<true/false>%n" + "                            Perform consistency checks on the label scan store.%n" + "                              This option is deprecated and its value will be%n" + "                              ignored. Checking of label scan store/lookup%n" + "                              index on labels is controlled by --check-graph.%n" + "      --check-relationship-type-scan-store=<true/false>%n" + "                            Perform consistency checks on the relationship type%n" + "                              scan store. This option is deprecated and its%n" + "                              value will be ignored. Checking of relationship%n" + "                              type scan store/lookup index on relationship%n" + "                              types is controlled by --check-graph.%n" + "      --check-property-owners=<true/false>%n" + "                            Perform additional consistency checks on property%n" + "                              ownership. This check is very expensive in time%n" + "                              and memory. This option is deprecated and its%n" + "                              value will be ignored."));
}
Also used : PrintStream(java.io.PrintStream) ExecutionContext(org.neo4j.cli.ExecutionContext) ByteArrayOutputStream(java.io.ByteArrayOutputStream) CheckConsistencyCommand(org.neo4j.consistency.CheckConsistencyCommand) Test(org.junit.jupiter.api.Test)

Aggregations

ExecutionContext (org.neo4j.cli.ExecutionContext)38 Test (org.junit.jupiter.api.Test)30 PrintStream (java.io.PrintStream)21 CheckConsistencyCommand (org.neo4j.consistency.CheckConsistencyCommand)13 Path (java.nio.file.Path)12 ConsistencyCheckService (org.neo4j.consistency.ConsistencyCheckService)12 ByteArrayOutputStream (java.io.ByteArrayOutputStream)10 ConsistencyFlags (org.neo4j.consistency.checking.full.ConsistencyFlags)10 Config (org.neo4j.configuration.Config)8 DatabaseLayout (org.neo4j.io.layout.DatabaseLayout)8 DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)6 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)6 ProgressMonitorFactory (org.neo4j.internal.helpers.progress.ProgressMonitorFactory)5 LogProvider (org.neo4j.logging.LogProvider)5 BeforeEach (org.junit.jupiter.api.BeforeEach)4 CommandFailedException (org.neo4j.cli.CommandFailedException)4 MemoryRecommendationsCommand.bytesToString (org.neo4j.commandline.dbms.MemoryRecommendationsCommand.bytesToString)4 FileSystem (java.nio.file.FileSystem)2 WireMockServer (com.github.tomakehurst.wiremock.WireMockServer)1 WireMock.matchingJsonPath (com.github.tomakehurst.wiremock.client.WireMock.matchingJsonPath)1