Search in sources :

Example 11 with Request

use of io.openk9.entity.manager.model.payload.Request in project openk9 by smclab.

the class BasePluginDriver method invokeDataParser.

@Override
public Mono<Void> invokeDataParser(Datasource datasource, Date fromDate, Date toDate) {
    String jsonConfig = datasource.getJsonConfig();
    JsonNode jsonNode = getJsonFactory().fromJsonToJsonNode(jsonConfig);
    ObjectNode objectNode = jsonNode.toObjectNode();
    ObjectNode requestJson = getJsonFactory().createObjectNode();
    objectNode.stream().filter(e -> _containsKey(e.getKey())).forEach(e -> requestJson.set(e.getKey(), e.getValue()));
    requestJson.put("timestamp", fromDate.getTime());
    requestJson.put("datasourceId", datasource.getDatasourceId());
    Map<String, Object> headers = headersObject();
    Publisher<byte[]> request = getHttpClient().request(method(), path(), requestJson.toString(), headers);
    return Mono.from(request).then();
}
Also used : Arrays(java.util.Arrays) Datasource(io.openk9.model.Datasource) Date(java.util.Date) JsonNode(io.openk9.json.api.JsonNode) Publisher(org.reactivestreams.Publisher) Map(java.util.Map) ObjectNode(io.openk9.json.api.ObjectNode) Mono(reactor.core.publisher.Mono) HttpClient(io.openk9.http.client.HttpClient) Collectors(java.util.stream.Collectors) JsonFactory(io.openk9.json.api.JsonFactory) ObjectNode(io.openk9.json.api.ObjectNode) JsonNode(io.openk9.json.api.JsonNode)

Example 12 with Request

use of io.openk9.entity.manager.model.payload.Request in project openk9 by smclab.

the class BaseNerEnrichProcessor method process.

@Override
public Mono<ObjectNode> process(ObjectNode objectNode, DatasourceContext context, EnrichItem enrichItem, PluginDriverDTO pluginDriverName) {
    return Mono.defer(() -> {
        JsonNode datasourceConfiguration = _jsonFactory.fromJsonToJsonNode(enrichItem.getJsonConfig());
        if (!datasourceConfiguration.isObject()) {
            return Mono.error(new RuntimeException("jsonConfig must be an instance of ObjectNode " + datasourceConfiguration.toString()));
        }
        ObjectNode request = prepareRequestRawContent(objectNode, datasourceConfiguration.toObjectNode(), context, pluginDriverName);
        return Mono.from(_httpClient.request(getMethod(), getPath(), request.toString(), getHeaders())).retry(5).map(_jsonFactory::fromJsonToJsonNode).map(JsonNode::toObjectNode).map(jsonNodes -> {
            jsonNodes.put(Constants.TENANT_ID, context.getTenant().getTenantId());
            jsonNodes.put(Constants.DATASOURCE_ID, context.getDatasource().getDatasourceId());
            jsonNodes.put(Constants.CONTENT_ID, objectNode.get(Constants.CONTENT_ID));
            jsonNodes.put(Constants.RAW_CONTENT, objectNode.get(Constants.RAW_CONTENT));
            jsonNodes.put(Constants.INGESTION_ID, objectNode.get(Constants.INGESTION_ID));
            return jsonNodes;
        }).flatMap(this::_getEntityOrCreate).map(entities -> objectNode.set(entitiesField(), entities));
    });
}
Also used : ObjectNode(io.openk9.json.api.ObjectNode) JsonNode(io.openk9.json.api.JsonNode)

Aggregations

JsonNode (io.openk9.json.api.JsonNode)6 ObjectNode (io.openk9.json.api.ObjectNode)6 Tenant (io.openk9.api.aggregator.model.Tenant)3 ArrayNode (io.openk9.json.api.ArrayNode)3 JsonFactory (io.openk9.json.api.JsonFactory)3 HttpException (io.vertx.ext.web.handler.HttpException)3 Collectors (java.util.stream.Collectors)3 PermitAll (javax.annotation.security.PermitAll)3 POST (javax.ws.rs.POST)3 Path (javax.ws.rs.Path)3 Mono (reactor.core.publisher.Mono)3 Payload (io.openk9.entity.manager.dto.Payload)2 ArrayList (java.util.ArrayList)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Activate (org.osgi.service.component.annotations.Activate)2 Component (org.osgi.service.component.annotations.Component)2 Reference (org.osgi.service.component.annotations.Reference)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2