Search in sources :

Example 6 with InputException

use of org.neo4j.unsafe.impl.batchimport.input.InputException in project neo4j by neo4j.

the class InputEntityDeserializer method deserializeNextFromSource.

private boolean deserializeNextFromSource() throws IOException {
    Header.Entry[] entries = header.entries();
    if (entries.length == 0) {
        return false;
    }
    int fieldIndex = 0;
    try {
        for (; fieldIndex < entries.length; fieldIndex++) {
            // Seek the next value
            if (!data.seek(mark, delimiter)) {
                if (fieldIndex > 0) {
                    throw new UnexpectedEndOfInputException("Near " + mark);
                }
                // We're just at the end
                return false;
            }
            // Extract it, type according to our header
            Header.Entry entry = entries[fieldIndex];
            if (entry.type() != Type.IGNORE) {
                Object value = data.tryExtract(mark, entry.extractor()) ? entry.extractor().value() : null;
                deserialization.handle(entry, value);
            }
            if (mark.isEndOfLine()) {
                // We're at the end of the line, break and return an entity with what we have.
                break;
            }
        }
        return true;
    } catch (final RuntimeException e) {
        String stringValue = null;
        try {
            Extractors extractors = new Extractors('?');
            if (data.tryExtract(mark, extractors.string())) {
                stringValue = extractors.string().value();
            }
        } catch (Exception e1) {
        // OK
        }
        String message = format("ERROR in input" + "%n  data source: %s" + "%n  in field: %s" + "%n  for header: %s" + "%n  raw field value: %s" + "%n  original error: %s", data, entries[fieldIndex] + ":" + (fieldIndex + 1), header, stringValue != null ? stringValue : "??", e.getMessage());
        if (e instanceof InputException) {
            throw Exceptions.withMessage(e, message);
        }
        throw new InputException(message, e);
    }
}
Also used : Extractors(org.neo4j.csv.reader.Extractors) UnexpectedEndOfInputException(org.neo4j.unsafe.impl.batchimport.input.UnexpectedEndOfInputException) UnexpectedEndOfInputException(org.neo4j.unsafe.impl.batchimport.input.UnexpectedEndOfInputException) InputException(org.neo4j.unsafe.impl.batchimport.input.InputException) UnexpectedEndOfInputException(org.neo4j.unsafe.impl.batchimport.input.UnexpectedEndOfInputException) InputException(org.neo4j.unsafe.impl.batchimport.input.InputException) IOException(java.io.IOException)

Example 7 with InputException

use of org.neo4j.unsafe.impl.batchimport.input.InputException in project neo4j by neo4j.

the class CsvInputTest method shouldIncludeDataSourceInformationOnBadFieldValueOrLine.

@Test
public void shouldIncludeDataSourceInformationOnBadFieldValueOrLine() throws Exception {
    // GIVEN
    Iterable<DataFactory<InputNode>> data = DataFactories.nodeData(CsvInputTest.<InputNode>data(":ID,name,other:int\n" + "1,Mattias,10\n" + "2,Johan,abc\n" + "3,Emil,12"));
    Input input = new CsvInput(data, DataFactories.defaultFormatNodeFileHeader(), null, null, IdType.INTEGER, config(COMMAS), silentBadCollector(0), getRuntime().availableProcessors());
    // WHEN
    try (InputIterator<InputNode> nodes = input.nodes().iterator()) {
        try {
            assertNode(nodes.next(), 1L, new Object[] { "name", "Mattias", "other", 10 }, labels());
            nodes.next();
            fail("Should have failed");
        } catch (InputException e) {
            // THEN
            assertThat(e.getMessage(), containsString("other"));
            assertThat(e.getMessage(), containsString("abc"));
        }
    }
}
Also used : InputNode(org.neo4j.unsafe.impl.batchimport.input.InputNode) Input(org.neo4j.unsafe.impl.batchimport.input.Input) InputException(org.neo4j.unsafe.impl.batchimport.input.InputException) Test(org.junit.Test)

Example 8 with InputException

use of org.neo4j.unsafe.impl.batchimport.input.InputException in project neo4j by neo4j.

the class CsvInputTest method shouldFailOnRelationshipWithMissingEndIdField.

@Test
public void shouldFailOnRelationshipWithMissingEndIdField() throws Exception {
    // GIVEN
    Iterable<DataFactory<InputRelationship>> data = relationshipData(CsvInputTest.<InputRelationship>data(":START_ID,:END_ID,:TYPE\n" + "1,,"));
    Input input = new CsvInput(null, null, data, defaultFormatRelationshipFileHeader(), IdType.INTEGER, config(COMMAS), silentBadCollector(0), getRuntime().availableProcessors());
    // WHEN
    try (InputIterator<InputRelationship> relationships = input.relationships().iterator()) {
        relationships.next();
        fail("Should have failed");
    } catch (InputException e) {
        // THEN good
        assertThat(e.getMessage(), containsString(Type.END_ID.name()));
    }
}
Also used : Input(org.neo4j.unsafe.impl.batchimport.input.Input) InputException(org.neo4j.unsafe.impl.batchimport.input.InputException) InputRelationship(org.neo4j.unsafe.impl.batchimport.input.InputRelationship) Test(org.junit.Test)

Example 9 with InputException

use of org.neo4j.unsafe.impl.batchimport.input.InputException in project neo4j by neo4j.

the class DataFactoriesTest method shouldFailOnUnexpectedRelationshipHeaderType.

@Test
public void shouldFailOnUnexpectedRelationshipHeaderType() throws Exception {
    // GIVEN
    CharSeeker seeker = seeker(":LABEL,:START_ID,:END_ID,:TYPE");
    IdType idType = IdType.ACTUAL;
    // WHEN
    try {
        Header header = DataFactories.defaultFormatRelationshipFileHeader().create(seeker, COMMAS, idType);
        fail("Should have failed");
    } catch (InputException e) {
        // THEN
        assertThat(e.getMessage(), containsString("LABEL"));
    }
}
Also used : CharSeeker(org.neo4j.csv.reader.CharSeeker) DataFactories.defaultFormatNodeFileHeader(org.neo4j.unsafe.impl.batchimport.input.csv.DataFactories.defaultFormatNodeFileHeader) InputException(org.neo4j.unsafe.impl.batchimport.input.InputException) Test(org.junit.Test)

Example 10 with InputException

use of org.neo4j.unsafe.impl.batchimport.input.InputException in project neo4j by neo4j.

the class CsvInputTest method shouldFailOnRelationshipWithMissingStartIdField.

@Test
public void shouldFailOnRelationshipWithMissingStartIdField() throws Exception {
    // GIVEN
    Iterable<DataFactory<InputRelationship>> data = relationshipData(CsvInputTest.<InputRelationship>data(":START_ID,:END_ID,:TYPE\n" + ",1,"));
    Input input = new CsvInput(null, null, data, defaultFormatRelationshipFileHeader(), IdType.INTEGER, config(COMMAS), silentBadCollector(0), getRuntime().availableProcessors());
    // WHEN
    try (InputIterator<InputRelationship> relationships = input.relationships().iterator()) {
        relationships.next();
        fail("Should have failed");
    } catch (InputException e) {
        // THEN good
        assertThat(e.getMessage(), containsString(Type.START_ID.name()));
    }
}
Also used : Input(org.neo4j.unsafe.impl.batchimport.input.Input) InputException(org.neo4j.unsafe.impl.batchimport.input.InputException) InputRelationship(org.neo4j.unsafe.impl.batchimport.input.InputRelationship) Test(org.junit.Test)

Aggregations

InputException (org.neo4j.unsafe.impl.batchimport.input.InputException)13 Test (org.junit.Test)9 IOException (java.io.IOException)4 Input (org.neo4j.unsafe.impl.batchimport.input.Input)4 UnexpectedEndOfInputException (org.neo4j.unsafe.impl.batchimport.input.UnexpectedEndOfInputException)3 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)2 CharSeeker (org.neo4j.csv.reader.CharSeeker)2 InputRelationship (org.neo4j.unsafe.impl.batchimport.input.InputRelationship)2 Configuration (org.neo4j.unsafe.impl.batchimport.input.csv.Configuration)2 DataFactories.defaultFormatNodeFileHeader (org.neo4j.unsafe.impl.batchimport.input.csv.DataFactories.defaultFormatNodeFileHeader)2 File (java.io.File)1 DataAfterQuoteException (org.neo4j.csv.reader.DataAfterQuoteException)1 Extractors (org.neo4j.csv.reader.Extractors)1 BadCollector (org.neo4j.unsafe.impl.batchimport.input.BadCollector)1 InputNode (org.neo4j.unsafe.impl.batchimport.input.InputNode)1 CsvInput (org.neo4j.unsafe.impl.batchimport.input.csv.CsvInput)1 DataFactory (org.neo4j.unsafe.impl.batchimport.input.csv.DataFactory)1