Search in sources :

Example 66 with JsonReader

use of javax.json.JsonReader in project nifi by apache.

the class TestSiteToSiteBulletinReportingTask method testSerializedForm.

@Test
public void testSerializedForm() throws IOException, InitializationException {
    // creating the list of bulletins
    final List<Bulletin> bulletins = new ArrayList<Bulletin>();
    bulletins.add(BulletinFactory.createBulletin("group-id", "group-name", "source-id", "source-name", "category", "severity", "message"));
    // mock the access to the list of bulletins
    final ReportingContext context = Mockito.mock(ReportingContext.class);
    final BulletinRepository repository = Mockito.mock(BulletinRepository.class);
    Mockito.when(context.getBulletinRepository()).thenReturn(repository);
    Mockito.when(repository.findBulletins(Mockito.any(BulletinQuery.class))).thenReturn(bulletins);
    // creating reporting task
    final MockSiteToSiteBulletinReportingTask task = new MockSiteToSiteBulletinReportingTask();
    Mockito.when(context.getStateManager()).thenReturn(new MockStateManager(task));
    // settings properties and mocking access to properties
    final Map<PropertyDescriptor, String> properties = new HashMap<>();
    for (final PropertyDescriptor descriptor : task.getSupportedPropertyDescriptors()) {
        properties.put(descriptor, descriptor.getDefaultValue());
    }
    properties.put(SiteToSiteBulletinReportingTask.BATCH_SIZE, "1000");
    properties.put(SiteToSiteBulletinReportingTask.PLATFORM, "nifi");
    Mockito.doAnswer(new Answer<PropertyValue>() {

        @Override
        public PropertyValue answer(final InvocationOnMock invocation) throws Throwable {
            final PropertyDescriptor descriptor = invocation.getArgumentAt(0, PropertyDescriptor.class);
            return new MockPropertyValue(properties.get(descriptor));
        }
    }).when(context).getProperty(Mockito.any(PropertyDescriptor.class));
    // setup the mock initialization context
    final ComponentLog logger = Mockito.mock(ComponentLog.class);
    final ReportingInitializationContext initContext = Mockito.mock(ReportingInitializationContext.class);
    Mockito.when(initContext.getIdentifier()).thenReturn(UUID.randomUUID().toString());
    Mockito.when(initContext.getLogger()).thenReturn(logger);
    task.initialize(initContext);
    task.onTrigger(context);
    // test checking
    assertEquals(1, task.dataSent.size());
    final String msg = new String(task.dataSent.get(0), StandardCharsets.UTF_8);
    JsonReader jsonReader = Json.createReader(new ByteArrayInputStream(msg.getBytes()));
    JsonObject bulletinJson = jsonReader.readArray().getJsonObject(0);
    assertEquals("message", bulletinJson.getString("bulletinMessage"));
    assertEquals("group-name", bulletinJson.getString("bulletinGroupName"));
}
Also used : PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) MockPropertyValue(org.apache.nifi.util.MockPropertyValue) PropertyValue(org.apache.nifi.components.PropertyValue) StandardPropertyValue(org.apache.nifi.attribute.expression.language.StandardPropertyValue) JsonObject(javax.json.JsonObject) MockPropertyValue(org.apache.nifi.util.MockPropertyValue) ComponentLog(org.apache.nifi.logging.ComponentLog) MockStateManager(org.apache.nifi.state.MockStateManager) ByteArrayInputStream(java.io.ByteArrayInputStream) InvocationOnMock(org.mockito.invocation.InvocationOnMock) JsonReader(javax.json.JsonReader) Test(org.junit.Test)

Example 67 with JsonReader

use of javax.json.JsonReader in project nifi by apache.

the class TestSiteToSiteStatusReportingTask method testComponentNameFilter_nested.

@Test
public void testComponentNameFilter_nested() throws IOException, InitializationException {
    final ProcessGroupStatus pgStatus = generateProcessGroupStatus("root", "Awesome", 2, 0);
    final Map<PropertyDescriptor, String> properties = new HashMap<>();
    properties.put(SiteToSiteStatusReportingTask.BATCH_SIZE, "4");
    properties.put(SiteToSiteStatusReportingTask.COMPONENT_NAME_FILTER_REGEX, "Awesome.*processor.*");
    properties.put(SiteToSiteStatusReportingTask.COMPONENT_TYPE_FILTER_REGEX, ".*");
    MockSiteToSiteStatusReportingTask task = initTask(properties, pgStatus);
    task.onTrigger(context);
    // 3 + (3 * 3) + (3 * 3 * 3) = 39, or 10 batches of 4
    assertEquals(10, task.dataSent.size());
    final String msg = new String(task.dataSent.get(0), StandardCharsets.UTF_8);
    JsonReader jsonReader = Json.createReader(new ByteArrayInputStream(msg.getBytes()));
    JsonString componentId = jsonReader.readArray().getJsonObject(0).getJsonString("componentId");
    assertEquals("root.1.1.processor.1", componentId.getString());
}
Also used : ProcessGroupStatus(org.apache.nifi.controller.status.ProcessGroupStatus) RemoteProcessGroupStatus(org.apache.nifi.controller.status.RemoteProcessGroupStatus) PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) HashMap(java.util.HashMap) ByteArrayInputStream(java.io.ByteArrayInputStream) JsonReader(javax.json.JsonReader) JsonString(javax.json.JsonString) JsonString(javax.json.JsonString) Test(org.junit.Test)

Example 68 with JsonReader

use of javax.json.JsonReader in project nifi by apache.

the class TestSiteToSiteStatusReportingTask method testComponentTypeFilter.

@Test
public void testComponentTypeFilter() throws IOException, InitializationException {
    final ProcessGroupStatus pgStatus = generateProcessGroupStatus("root", "Awesome", 1, 0);
    final Map<PropertyDescriptor, String> properties = new HashMap<>();
    properties.put(SiteToSiteStatusReportingTask.BATCH_SIZE, "4");
    properties.put(SiteToSiteStatusReportingTask.COMPONENT_NAME_FILTER_REGEX, "Awesome.*");
    properties.put(SiteToSiteStatusReportingTask.COMPONENT_TYPE_FILTER_REGEX, "(ProcessGroup|RootProcessGroup)");
    MockSiteToSiteStatusReportingTask task = initTask(properties, pgStatus);
    task.onTrigger(context);
    // Only root pg and 3 child pgs
    assertEquals(1, task.dataSent.size());
    final String msg = new String(task.dataSent.get(0), StandardCharsets.UTF_8);
    JsonReader jsonReader = Json.createReader(new ByteArrayInputStream(msg.getBytes()));
    JsonString componentId = jsonReader.readArray().getJsonObject(0).getJsonString("componentId");
    assertEquals(pgStatus.getId(), componentId.getString());
}
Also used : ProcessGroupStatus(org.apache.nifi.controller.status.ProcessGroupStatus) RemoteProcessGroupStatus(org.apache.nifi.controller.status.RemoteProcessGroupStatus) PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) HashMap(java.util.HashMap) ByteArrayInputStream(java.io.ByteArrayInputStream) JsonReader(javax.json.JsonReader) JsonString(javax.json.JsonString) JsonString(javax.json.JsonString) Test(org.junit.Test)

Example 69 with JsonReader

use of javax.json.JsonReader in project nifi by apache.

the class TestSiteToSiteStatusReportingTask method testComponentNameFilter.

@Test
public void testComponentNameFilter() throws IOException, InitializationException {
    final ProcessGroupStatus pgStatus = generateProcessGroupStatus("root", "Awesome", 1, 0);
    final Map<PropertyDescriptor, String> properties = new HashMap<>();
    properties.put(SiteToSiteStatusReportingTask.BATCH_SIZE, "4");
    properties.put(SiteToSiteStatusReportingTask.COMPONENT_NAME_FILTER_REGEX, "Awesome.*processor.*");
    properties.put(SiteToSiteStatusReportingTask.COMPONENT_TYPE_FILTER_REGEX, ".*");
    MockSiteToSiteStatusReportingTask task = initTask(properties, pgStatus);
    task.onTrigger(context);
    // 3 processors for each of 4 groups
    assertEquals(3, task.dataSent.size());
    final String msg = new String(task.dataSent.get(0), StandardCharsets.UTF_8);
    JsonReader jsonReader = Json.createReader(new ByteArrayInputStream(msg.getBytes()));
    JsonString componentId = jsonReader.readArray().getJsonObject(0).getJsonString("componentId");
    assertEquals("root.1.processor.1", componentId.getString());
}
Also used : ProcessGroupStatus(org.apache.nifi.controller.status.ProcessGroupStatus) RemoteProcessGroupStatus(org.apache.nifi.controller.status.RemoteProcessGroupStatus) PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) HashMap(java.util.HashMap) ByteArrayInputStream(java.io.ByteArrayInputStream) JsonReader(javax.json.JsonReader) JsonString(javax.json.JsonString) JsonString(javax.json.JsonString) Test(org.junit.Test)

Example 70 with JsonReader

use of javax.json.JsonReader in project nifi by apache.

the class PutSlack method onTrigger.

@Override
public void onTrigger(final ProcessContext context, final ProcessSession session) {
    FlowFile flowFile = session.get();
    if (flowFile == null) {
        return;
    }
    JsonObjectBuilder builder = Json.createObjectBuilder();
    String text = context.getProperty(WEBHOOK_TEXT).evaluateAttributeExpressions(flowFile).getValue();
    if (text != null && !text.isEmpty()) {
        builder.add("text", text);
    } else {
        // Slack requires the 'text' attribute
        getLogger().error("FlowFile should have non-empty " + WEBHOOK_TEXT.getName());
        flowFile = session.penalize(flowFile);
        session.transfer(flowFile, REL_FAILURE);
        return;
    }
    String channel = context.getProperty(CHANNEL).evaluateAttributeExpressions(flowFile).getValue();
    if (channel != null && !channel.isEmpty()) {
        String error = validateChannel(channel);
        if (error == null) {
            builder.add("channel", channel);
        } else {
            getLogger().error("Invalid channel '{}': {}", new Object[] { channel, error });
            flowFile = session.penalize(flowFile);
            session.transfer(flowFile, REL_FAILURE);
            return;
        }
    }
    String username = context.getProperty(USERNAME).evaluateAttributeExpressions(flowFile).getValue();
    if (username != null && !username.isEmpty()) {
        builder.add("username", username);
    }
    String iconUrl = context.getProperty(ICON_URL).evaluateAttributeExpressions(flowFile).getValue();
    if (iconUrl != null && !iconUrl.isEmpty()) {
        builder.add("icon_url", iconUrl);
    }
    String iconEmoji = context.getProperty(ICON_EMOJI).evaluateAttributeExpressions(flowFile).getValue();
    if (iconEmoji != null && !iconEmoji.isEmpty()) {
        builder.add("icon_emoji", iconEmoji);
    }
    try {
        // Get Attachments Array
        if (!attachments.isEmpty()) {
            JsonArrayBuilder jsonArrayBuiler = Json.createArrayBuilder();
            for (PropertyDescriptor attachment : attachments) {
                String s = context.getProperty(attachment).evaluateAttributeExpressions(flowFile).getValue();
                JsonReader reader = Json.createReader(new StringReader(s));
                JsonObject attachmentJson = reader.readObject();
                jsonArrayBuiler.add(attachmentJson);
            }
            builder.add("attachments", jsonArrayBuiler);
        }
        JsonObject jsonObject = builder.build();
        StringWriter stringWriter = new StringWriter();
        JsonWriter jsonWriter = Json.createWriter(stringWriter);
        jsonWriter.writeObject(jsonObject);
        jsonWriter.close();
        URL url = new URL(context.getProperty(WEBHOOK_URL).getValue());
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.setRequestMethod("POST");
        conn.setDoOutput(true);
        DataOutputStream outputStream = new DataOutputStream(conn.getOutputStream());
        String payload = "payload=" + URLEncoder.encode(stringWriter.getBuffer().toString(), "UTF-8");
        outputStream.writeBytes(payload);
        outputStream.close();
        int responseCode = conn.getResponseCode();
        if (responseCode >= 200 && responseCode < 300) {
            getLogger().info("Successfully posted message to Slack");
            session.transfer(flowFile, REL_SUCCESS);
            session.getProvenanceReporter().send(flowFile, context.getProperty(WEBHOOK_URL).getValue());
        } else {
            getLogger().error("Failed to post message to Slack with response code {}", new Object[] { responseCode });
            flowFile = session.penalize(flowFile);
            session.transfer(flowFile, REL_FAILURE);
            context.yield();
        }
    } catch (JsonParsingException e) {
        getLogger().error("Failed to parse JSON", e);
        flowFile = session.penalize(flowFile);
        session.transfer(flowFile, REL_FAILURE);
    } catch (IOException e) {
        getLogger().error("Failed to open connection", e);
        flowFile = session.penalize(flowFile);
        session.transfer(flowFile, REL_FAILURE);
        context.yield();
    }
}
Also used : FlowFile(org.apache.nifi.flowfile.FlowFile) PropertyDescriptor(org.apache.nifi.components.PropertyDescriptor) DataOutputStream(java.io.DataOutputStream) JsonObject(javax.json.JsonObject) IOException(java.io.IOException) JsonWriter(javax.json.JsonWriter) URL(java.net.URL) HttpURLConnection(java.net.HttpURLConnection) StringWriter(java.io.StringWriter) StringReader(java.io.StringReader) JsonReader(javax.json.JsonReader) JsonArrayBuilder(javax.json.JsonArrayBuilder) JsonObjectBuilder(javax.json.JsonObjectBuilder) JsonParsingException(javax.json.stream.JsonParsingException)

Aggregations

JsonReader (javax.json.JsonReader)130 JsonObject (javax.json.JsonObject)110 StringReader (java.io.StringReader)78 Test (org.junit.Test)47 JsonArray (javax.json.JsonArray)44 JsonString (javax.json.JsonString)42 HashMap (java.util.HashMap)21 IOException (java.io.IOException)17 ArrayList (java.util.ArrayList)13 File (java.io.File)10 LinkedHashMap (java.util.LinkedHashMap)10 JsonParser (edu.harvard.iq.dataverse.util.json.JsonParser)9 DatasetVersion (edu.harvard.iq.dataverse.DatasetVersion)8 ByteArrayInputStream (java.io.ByteArrayInputStream)8 PropertyDescriptor (org.apache.nifi.components.PropertyDescriptor)8 InputStream (java.io.InputStream)7 Gson (com.google.gson.Gson)6 AsyncCompletionHandler (com.ning.http.client.AsyncCompletionHandler)6 Response (com.ning.http.client.Response)6 JsonParseException (edu.harvard.iq.dataverse.util.json.JsonParseException)5