Search in sources :

Example 1 with MappingRuleCacheKey

use of org.folio.services.entity.MappingRuleCacheKey in project mod-source-record-manager by folio-org.

the class MappingRuleCache method loadMappingRules.

private CompletableFuture<Optional<JsonObject>> loadMappingRules(MappingRuleCacheKey key, Executor executor, MappingRuleDao mappingRuleDao) {
    CompletableFuture<Optional<JsonObject>> future = new CompletableFuture<>();
    executor.execute(() -> mappingRuleDao.get(key.getRecordType(), key.getTenantId()).map(optional -> optional.isPresent() ? optional : Optional.of(new JsonObject())).onComplete(ar -> {
        if (ar.failed()) {
            LOGGER.error("Failed to load mapping rules for tenant '{}' from data base", key.getTenantId(), ar.cause());
            future.completeExceptionally(ar.cause());
            return;
        }
        future.complete(ar.result());
    }));
    return future;
}
Also used : Caffeine(com.github.benmanes.caffeine.cache.Caffeine) Executor(java.util.concurrent.Executor) Promise(io.vertx.core.Promise) Vertx(io.vertx.core.Vertx) Autowired(org.springframework.beans.factory.annotation.Autowired) CompletableFuture(java.util.concurrent.CompletableFuture) MappingRuleDao(org.folio.dao.MappingRuleDao) Future(io.vertx.core.Future) AsyncLoadingCache(com.github.benmanes.caffeine.cache.AsyncLoadingCache) Component(org.springframework.stereotype.Component) Logger(org.apache.logging.log4j.Logger) Optional(java.util.Optional) MappingRuleCacheKey(org.folio.services.entity.MappingRuleCacheKey) JsonObject(io.vertx.core.json.JsonObject) LogManager(org.apache.logging.log4j.LogManager) CompletableFuture(java.util.concurrent.CompletableFuture) Optional(java.util.Optional) JsonObject(io.vertx.core.json.JsonObject)

Example 2 with MappingRuleCacheKey

use of org.folio.services.entity.MappingRuleCacheKey in project mod-source-record-manager by folio-org.

the class MappingRuleServiceImpl method updateRules.

private Future<JsonObject> updateRules(String rules, Record.RecordType recordType, String tenantId) {
    Promise<JsonObject> promise = Promise.promise();
    if (isValidJson(rules)) {
        MappingRuleCacheKey cacheKey = new MappingRuleCacheKey(tenantId, recordType);
        mappingRuleDao.update(new JsonObject(rules), recordType, tenantId).onSuccess(updatedRules -> mappingRuleCache.put(cacheKey, updatedRules)).onComplete(promise);
    } else {
        String errorMessage = "Can not update rules in non-JSON format";
        LOGGER.error(errorMessage);
        promise.fail(new BadRequestException(errorMessage));
    }
    return promise.future();
}
Also used : Resources(com.google.common.io.Resources) URL(java.net.URL) Record(org.folio.Record) Promise(io.vertx.core.Promise) Autowired(org.springframework.beans.factory.annotation.Autowired) IOException(java.io.IOException) InternalServerErrorException(javax.ws.rs.InternalServerErrorException) MappingRuleDao(org.folio.dao.MappingRuleDao) Function(java.util.function.Function) Future(io.vertx.core.Future) StandardCharsets(java.nio.charset.StandardCharsets) Logger(org.apache.logging.log4j.Logger) Charset(java.nio.charset.Charset) Service(org.springframework.stereotype.Service) Optional(java.util.Optional) MappingRuleCacheKey(org.folio.services.entity.MappingRuleCacheKey) BadRequestException(javax.ws.rs.BadRequestException) JsonObject(io.vertx.core.json.JsonObject) LogManager(org.apache.logging.log4j.LogManager) MappingRuleCacheKey(org.folio.services.entity.MappingRuleCacheKey) JsonObject(io.vertx.core.json.JsonObject) BadRequestException(javax.ws.rs.BadRequestException)

Example 3 with MappingRuleCacheKey

use of org.folio.services.entity.MappingRuleCacheKey in project mod-source-record-manager by folio-org.

the class StoredRecordChunksKafkaHandlerTest method shouldReturnFailedFutureWhenMappingRuleCacheReturnFailed.

@Test
public void shouldReturnFailedFutureWhenMappingRuleCacheReturnFailed() throws IOException {
    Record record = Json.decodeValue(TestUtil.readFileFromPath(EDIFACT_RECORD_PATH), Record.class);
    RecordsBatchResponse savedRecordsBatch = new RecordsBatchResponse().withRecords(List.of(record)).withTotalRecords(1);
    Event event = new Event().withId(UUID.randomUUID().toString()).withEventPayload(Json.encode(savedRecordsBatch));
    when(kafkaRecord.value()).thenReturn(Json.encode(event));
    when(kafkaRecord.headers()).thenReturn(List.of(KafkaHeader.header(OKAPI_HEADER_TENANT, TENANT_ID)));
    when(eventProcessedService.collectData(STORED_RECORD_CHUNKS_KAFKA_HANDLER_UUID, event.getId(), TENANT_ID)).thenReturn(Future.succeededFuture());
    when(mappingRuleCache.get(new MappingRuleCacheKey(TENANT_ID, EntityType.EDIFACT))).thenReturn(Future.failedFuture(new Exception()));
    // when
    Future<String> future = storedRecordChunksKafkaHandler.handle(kafkaRecord);
    // then
    assertTrue(future.failed());
}
Also used : MappingRuleCacheKey(org.folio.services.entity.MappingRuleCacheKey) RecordsBatchResponse(org.folio.rest.jaxrs.model.RecordsBatchResponse) Event(org.folio.rest.jaxrs.model.Event) JournalRecord(org.folio.rest.jaxrs.model.JournalRecord) Record(org.folio.rest.jaxrs.model.Record) KafkaConsumerRecord(io.vertx.kafka.client.consumer.KafkaConsumerRecord) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) DuplicateEventException(org.folio.kafka.exception.DuplicateEventException) IOException(java.io.IOException) Test(org.junit.Test)

Example 4 with MappingRuleCacheKey

use of org.folio.services.entity.MappingRuleCacheKey in project mod-source-record-manager by folio-org.

the class MarcBibErrorPayloadBuilderTest method shouldBuildPayload.

@Test
public void shouldBuildPayload(TestContext context) throws IOException {
    Async async = context.async();
    Record record = getRecordFromFile();
    when(mappingRuleCache.get(new MappingRuleCacheKey(TENANT_ID, record.getRecordType()))).thenReturn(Future.succeededFuture(Optional.of(new JsonObject(TestUtil.readFileFromPath(MAPPING_RULES_PATH)))));
    Future<DataImportEventPayload> payloadFuture = payloadBuilder.buildEventPayload(new RecordTooLargeException(LARGE_PAYLOAD_ERROR_MESSAGE), getOkapiParams(), JOB_EXECUTION_ID, record);
    payloadFuture.onComplete(ar -> {
        DataImportEventPayload result = ar.result();
        assertEquals(DI_ERROR.value(), result.getEventType());
        assertTrue(result.getContext().containsKey(ERROR_KEY));
        Record resRecordWithTitle = getRecordFromContext(result);
        assertNotNull(resRecordWithTitle.getParsedRecord());
        async.complete();
    });
}
Also used : MappingRuleCacheKey(org.folio.services.entity.MappingRuleCacheKey) Async(io.vertx.ext.unit.Async) JsonObject(io.vertx.core.json.JsonObject) Record(org.folio.rest.jaxrs.model.Record) ParsedRecord(org.folio.rest.jaxrs.model.ParsedRecord) RecordTooLargeException(org.apache.kafka.common.errors.RecordTooLargeException) DataImportEventPayload(org.folio.DataImportEventPayload) Test(org.junit.Test)

Example 5 with MappingRuleCacheKey

use of org.folio.services.entity.MappingRuleCacheKey in project mod-source-record-manager by folio-org.

the class MarcBibErrorPayloadBuilderTest method shouldBuildPayloadWhenNoMappingRulesFound.

@Test
public void shouldBuildPayloadWhenNoMappingRulesFound(TestContext context) throws IOException {
    Async async = context.async();
    Record record = getRecordFromFile();
    when(mappingRuleCache.get(new MappingRuleCacheKey(TENANT_ID, record.getRecordType()))).thenReturn(Future.succeededFuture(Optional.empty()));
    Future<DataImportEventPayload> payloadFuture = payloadBuilder.buildEventPayload(new RecordTooLargeException(LARGE_PAYLOAD_ERROR_MESSAGE), getOkapiParams(), JOB_EXECUTION_ID, record);
    payloadFuture.onComplete(ar -> {
        DataImportEventPayload result = ar.result();
        assertEquals(DI_ERROR.value(), result.getEventType());
        assertTrue(result.getContext().containsKey(ERROR_KEY));
        Record resRecordWithNoTitle = getRecordFromContext(result);
        assertNull(resRecordWithNoTitle.getParsedRecord());
        async.complete();
    });
}
Also used : MappingRuleCacheKey(org.folio.services.entity.MappingRuleCacheKey) Async(io.vertx.ext.unit.Async) Record(org.folio.rest.jaxrs.model.Record) ParsedRecord(org.folio.rest.jaxrs.model.ParsedRecord) RecordTooLargeException(org.apache.kafka.common.errors.RecordTooLargeException) DataImportEventPayload(org.folio.DataImportEventPayload) Test(org.junit.Test)

Aggregations

MappingRuleCacheKey (org.folio.services.entity.MappingRuleCacheKey)11 JsonObject (io.vertx.core.json.JsonObject)6 Record (org.folio.rest.jaxrs.model.Record)6 Test (org.junit.Test)5 Future (io.vertx.core.Future)4 Promise (io.vertx.core.Promise)4 IOException (java.io.IOException)4 Async (io.vertx.ext.unit.Async)3 KafkaConsumerRecord (io.vertx.kafka.client.consumer.KafkaConsumerRecord)3 Optional (java.util.Optional)3 RecordTooLargeException (org.apache.kafka.common.errors.RecordTooLargeException)3 LogManager (org.apache.logging.log4j.LogManager)3 Logger (org.apache.logging.log4j.Logger)3 DataImportEventPayload (org.folio.DataImportEventPayload)3 OkapiConnectionParams (org.folio.dataimport.util.OkapiConnectionParams)3 Event (org.folio.rest.jaxrs.model.Event)3 JournalRecord (org.folio.rest.jaxrs.model.JournalRecord)3 RecordsBatchResponse (org.folio.rest.jaxrs.model.RecordsBatchResponse)3 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)3 Service (org.springframework.stereotype.Service)3