Examples with JsonTreeReader org.apache.nifi.json.JsonTreeReader used on opensource projects

Search in sources :

Example 6 with JsonTreeReader

use of org.apache.nifi.json.JsonTreeReader in project nifi by apache.

the class TestUpdateRecord method testUpdateSimpleArray.

@Test
public void testUpdateSimpleArray() throws InitializationException, IOException {
    final JsonTreeReader jsonReader = new JsonTreeReader();
    runner.addControllerService("reader", jsonReader);
    runner.setValidateExpressionUsage(false);
    final String inputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/multi-arrays.avsc")));
    final String outputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/multi-arrays.avsc")));
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_TEXT, inputSchemaText);
    runner.enableControllerService(jsonReader);
    final JsonRecordSetWriter jsonWriter = new JsonRecordSetWriter();
    runner.addControllerService("writer", jsonWriter);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_TEXT, outputSchemaText);
    runner.setProperty(jsonWriter, "Pretty Print JSON", "true");
    runner.setProperty(jsonWriter, "Schema Write Strategy", "full-schema-attribute");
    runner.setProperty(UpdateRecord.REPLACEMENT_VALUE_STRATEGY, UpdateRecord.LITERAL_VALUES);
    runner.enableControllerService(jsonWriter);
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json"));
    runner.setProperty("/numbers[*]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    String expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json")));
    expectedOutput = expectedOutput.replaceFirst("1, null, 4", "8, 8, 8");
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
    runner.removeProperty("/numbers[*]");
    runner.clearTransferState();
    runner.enqueue("{\"numbers\":null}");
    runner.setProperty("/numbers[*]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    String content = new String(runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).toByteArray());
    assertTrue(content.contains("\"numbers\" : null"));
    runner.removeProperty("/numbers[*]");
    runner.clearTransferState();
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json"));
    runner.setProperty("/numbers[1]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json")));
    expectedOutput = expectedOutput.replaceFirst("1, null, 4", "1, 8, 4");
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
    runner.removeProperty("/numbers[1]");
    runner.clearTransferState();
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json"));
    runner.setProperty("/numbers[0..1]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json")));
    expectedOutput = expectedOutput.replaceFirst("1, null, 4", "8, 8, 4");
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
    runner.removeProperty("/numbers[0..1]");
    runner.clearTransferState();
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json"));
    runner.setProperty("/numbers[0,2]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json")));
    expectedOutput = expectedOutput.replaceFirst("1, null, 4", "8, null, 8");
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
    runner.removeProperty("/numbers[0,2]");
    runner.clearTransferState();
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json"));
    runner.setProperty("/numbers[0,1..2]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json")));
    expectedOutput = expectedOutput.replaceFirst("1, null, 4", "8, 8, 8");
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
    runner.removeProperty("/numbers[0,1..2]");
    runner.clearTransferState();
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json"));
    runner.setProperty("/numbers[0..-1][. = 4]", "8");
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/input/multi-arrays.json")));
    expectedOutput = expectedOutput.replaceFirst("1, null, 4", "1, null, 8");
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
    runner.removeProperty("/numbers[0..-1][. = 4]");
}
Also used : JsonTreeReader(org.apache.nifi.json.JsonTreeReader) JsonRecordSetWriter(org.apache.nifi.json.JsonRecordSetWriter) Test(org.junit.Test)

Example 7 with JsonTreeReader

use of org.apache.nifi.json.JsonTreeReader in project nifi by apache.

the class TestUpdateRecord method testChangingSchema.

@Test
public void testChangingSchema() throws InitializationException, IOException {
    final JsonTreeReader jsonReader = new JsonTreeReader();
    runner.addControllerService("reader", jsonReader);
    final String inputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-name-record.avsc")));
    final String outputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-name-string.avsc")));
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_TEXT, inputSchemaText);
    runner.enableControllerService(jsonReader);
    final JsonRecordSetWriter jsonWriter = new JsonRecordSetWriter();
    runner.addControllerService("writer", jsonWriter);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_TEXT, outputSchemaText);
    runner.setProperty(jsonWriter, "Pretty Print JSON", "true");
    runner.setProperty(jsonWriter, "Schema Write Strategy", "full-schema-attribute");
    runner.enableControllerService(jsonWriter);
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/person.json"));
    runner.setProperty("/name", "/name/first");
    runner.setProperty(UpdateRecord.REPLACEMENT_VALUE_STRATEGY, UpdateRecord.RECORD_PATH_VALUES);
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    final String expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/output/person-with-firstname.json")));
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
}
Also used : JsonTreeReader(org.apache.nifi.json.JsonTreeReader) JsonRecordSetWriter(org.apache.nifi.json.JsonRecordSetWriter) Test(org.junit.Test)

Example 8 with JsonTreeReader

use of org.apache.nifi.json.JsonTreeReader in project nifi by apache.

the class TestUpdateRecord method testUpdateInArray.

@Test
public void testUpdateInArray() throws InitializationException, IOException {
    final JsonTreeReader jsonReader = new JsonTreeReader();
    runner.addControllerService("reader", jsonReader);
    runner.setValidateExpressionUsage(false);
    final String inputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-address.avsc")));
    final String outputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-address.avsc")));
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_TEXT, inputSchemaText);
    runner.enableControllerService(jsonReader);
    final JsonRecordSetWriter jsonWriter = new JsonRecordSetWriter();
    runner.addControllerService("writer", jsonWriter);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_TEXT, outputSchemaText);
    runner.setProperty(jsonWriter, "Pretty Print JSON", "true");
    runner.setProperty(jsonWriter, "Schema Write Strategy", "full-schema-attribute");
    runner.enableControllerService(jsonWriter);
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/person-address.json"));
    runner.setProperty("/address[*]/city", "newCity");
    runner.setProperty(UpdateRecord.REPLACEMENT_VALUE_STRATEGY, UpdateRecord.LITERAL_VALUES);
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    final String expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/output/person-with-new-city.json")));
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
}
Also used : JsonTreeReader(org.apache.nifi.json.JsonTreeReader) JsonRecordSetWriter(org.apache.nifi.json.JsonRecordSetWriter) Test(org.junit.Test)

Example 9 with JsonTreeReader

use of org.apache.nifi.json.JsonTreeReader in project nifi by apache.

the class TestUpdateRecord method testSetRootPathAbsoluteWithMultipleValues.

@Test
public void testSetRootPathAbsoluteWithMultipleValues() throws InitializationException, IOException {
    final JsonTreeReader jsonReader = new JsonTreeReader();
    runner.addControllerService("reader", jsonReader);
    final String inputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-name-record.avsc")));
    final String outputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/name-fields-only.avsc")));
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_TEXT, inputSchemaText);
    runner.enableControllerService(jsonReader);
    final JsonRecordSetWriter jsonWriter = new JsonRecordSetWriter();
    runner.addControllerService("writer", jsonWriter);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_TEXT, outputSchemaText);
    runner.setProperty(jsonWriter, "Pretty Print JSON", "true");
    runner.setProperty(jsonWriter, "Schema Write Strategy", "full-schema-attribute");
    runner.enableControllerService(jsonWriter);
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/person.json"));
    runner.setProperty("/", "/name/*");
    runner.setProperty(UpdateRecord.REPLACEMENT_VALUE_STRATEGY, UpdateRecord.RECORD_PATH_VALUES);
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    final String expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/output/name-fields-only.json")));
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
}
Also used : JsonTreeReader(org.apache.nifi.json.JsonTreeReader) JsonRecordSetWriter(org.apache.nifi.json.JsonRecordSetWriter) Test(org.junit.Test)

Example 10 with JsonTreeReader

use of org.apache.nifi.json.JsonTreeReader in project nifi by apache.

the class TestUpdateRecord method testUpdateInNullArray.

@Test
public void testUpdateInNullArray() throws InitializationException, IOException {
    final JsonTreeReader jsonReader = new JsonTreeReader();
    runner.addControllerService("reader", jsonReader);
    runner.setValidateExpressionUsage(false);
    final String inputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-address.avsc")));
    final String outputSchemaText = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/schema/person-with-address.avsc")));
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonReader, SchemaAccessUtils.SCHEMA_TEXT, inputSchemaText);
    runner.enableControllerService(jsonReader);
    final JsonRecordSetWriter jsonWriter = new JsonRecordSetWriter();
    runner.addControllerService("writer", jsonWriter);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_ACCESS_STRATEGY, SchemaAccessUtils.SCHEMA_TEXT_PROPERTY);
    runner.setProperty(jsonWriter, SchemaAccessUtils.SCHEMA_TEXT, outputSchemaText);
    runner.setProperty(jsonWriter, "Pretty Print JSON", "true");
    runner.setProperty(jsonWriter, "Schema Write Strategy", "full-schema-attribute");
    runner.enableControllerService(jsonWriter);
    runner.enqueue(Paths.get("src/test/resources/TestUpdateRecord/input/person-with-null-array.json"));
    runner.setProperty("/address[*]/city", "newCity");
    runner.setProperty(UpdateRecord.REPLACEMENT_VALUE_STRATEGY, UpdateRecord.LITERAL_VALUES);
    runner.run();
    runner.assertAllFlowFilesTransferred(UpdateRecord.REL_SUCCESS, 1);
    final String expectedOutput = new String(Files.readAllBytes(Paths.get("src/test/resources/TestUpdateRecord/output/person-with-null-array.json")));
    runner.getFlowFilesForRelationship(UpdateRecord.REL_SUCCESS).get(0).assertContentEquals(expectedOutput);
}
Also used : JsonTreeReader(org.apache.nifi.json.JsonTreeReader) JsonRecordSetWriter(org.apache.nifi.json.JsonRecordSetWriter) Test(org.junit.Test)

Aggregations

JsonTreeReader (org.apache.nifi.json.JsonTreeReader)14 JsonRecordSetWriter (org.apache.nifi.json.JsonRecordSetWriter)12 Test (org.junit.Test)12 RecordReaderFactory (org.apache.nifi.serialization.RecordReaderFactory)2 MockRecordWriter (org.apache.nifi.serialization.record.MockRecordWriter)2 Before (org.junit.Before)2 RecordSetWriterFactory (org.apache.nifi.serialization.RecordSetWriterFactory)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial