Search in sources :

Example 1 with DataImportEventTypes

use of org.folio.rest.jaxrs.model.DataImportEventTypes in project mod-source-record-manager by folio-org.

the class RecordProcessedEventHandlingServiceImpl method handle.

@Override
public Future<Boolean> handle(String eventContent, OkapiConnectionParams params) {
    Promise<Boolean> promise = Promise.promise();
    DataImportEventPayload dataImportEventPayload;
    try {
        dataImportEventPayload = new ObjectMapper().readValue(eventContent, DataImportEventPayload.class);
    } catch (IOException e) {
        LOGGER.error("Failed to read eventContent {}", eventContent, e);
        promise.fail(e);
        return promise.future();
    }
    String jobExecutionId = dataImportEventPayload.getJobExecutionId();
    try {
        DataImportEventTypes eventType = DataImportEventTypes.valueOf(dataImportEventPayload.getEventType());
        jobExecutionProgressService.updateJobExecutionProgress(jobExecutionId, progress -> changeProgressAccordingToEventType(progress, eventType), params.getTenantId()).compose(updatedProgress -> updateJobExecutionIfAllRecordsProcessed(jobExecutionId, updatedProgress, params)).onComplete(ar -> {
            if (ar.failed()) {
                LOGGER.error("Failed to handle {} event", eventType, ar.cause());
                updateJobStatusToError(jobExecutionId, params).onComplete(statusAr -> promise.fail(ar.cause()));
            } else {
                promise.complete(true);
            }
        });
    } catch (Exception e) {
        LOGGER.error("Failed to handle event {}", eventContent, e);
        updateJobStatusToError(jobExecutionId, params);
        promise.fail(e);
    }
    return promise.future();
}
Also used : StatusDto(org.folio.rest.jaxrs.model.StatusDto) Date(java.util.Date) Promise(io.vertx.core.Promise) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DataImportEventPayload(org.folio.DataImportEventPayload) Autowired(org.springframework.beans.factory.annotation.Autowired) CANCELLED(org.folio.rest.jaxrs.model.JobExecution.Status.CANCELLED) IOException(java.io.IOException) COMMITTED(org.folio.rest.jaxrs.model.JobExecution.Status.COMMITTED) Future(io.vertx.core.Future) OkapiConnectionParams(org.folio.dataimport.util.OkapiConnectionParams) String.format(java.lang.String.format) Logger(org.apache.logging.log4j.Logger) JobExecutionProgress(org.folio.rest.jaxrs.model.JobExecutionProgress) JournalService(org.folio.services.journal.JournalService) JobExecutionProgressService(org.folio.services.progress.JobExecutionProgressService) DataImportEventTypes(org.folio.rest.jaxrs.model.DataImportEventTypes) Service(org.springframework.stereotype.Service) JobExecution(org.folio.rest.jaxrs.model.JobExecution) Qualifier(org.springframework.beans.factory.annotation.Qualifier) LogManager(org.apache.logging.log4j.LogManager) Progress(org.folio.rest.jaxrs.model.Progress) IOException(java.io.IOException) DataImportEventTypes(org.folio.rest.jaxrs.model.DataImportEventTypes) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) IOException(java.io.IOException) DataImportEventPayload(org.folio.DataImportEventPayload)

Example 2 with DataImportEventTypes

use of org.folio.rest.jaxrs.model.DataImportEventTypes in project mod-source-record-manager by folio-org.

the class StoredRecordChunksKafkaHandler method handle.

@Override
public Future<String> handle(KafkaConsumerRecord<String, String> record) {
    List<KafkaHeader> kafkaHeaders = record.headers();
    OkapiConnectionParams okapiConnectionParams = new OkapiConnectionParams(KafkaHeaderUtils.kafkaHeadersToMap(kafkaHeaders), vertx);
    String chunkId = okapiConnectionParams.getHeaders().get("chunkId");
    String chunkNumber = okapiConnectionParams.getHeaders().get("chunkNumber");
    String jobExecutionId = okapiConnectionParams.getHeaders().get("jobExecutionId");
    Event event = Json.decodeValue(record.value(), Event.class);
    try {
        return eventProcessedService.collectData(STORED_RECORD_CHUNKS_KAFKA_HANDLER_UUID, event.getId(), okapiConnectionParams.getTenantId()).compose(res -> {
            RecordsBatchResponse recordsBatchResponse = Json.decodeValue(event.getEventPayload(), RecordsBatchResponse.class);
            List<Record> storedRecords = recordsBatchResponse.getRecords();
            // we only know record type by inspecting the records, assuming records are homogeneous type and defaulting to previous static value
            DataImportEventTypes eventType = !storedRecords.isEmpty() && RECORD_TYPE_TO_EVENT_TYPE.containsKey(storedRecords.get(0).getRecordType()) ? RECORD_TYPE_TO_EVENT_TYPE.get(storedRecords.get(0).getRecordType()) : DI_SRS_MARC_BIB_RECORD_CREATED;
            LOGGER.debug("RecordsBatchResponse has been received, starting processing chunkId: {} chunkNumber: {} jobExecutionId: {}", chunkId, chunkNumber, jobExecutionId);
            saveCreatedRecordsInfoToDataImportLog(storedRecords, okapiConnectionParams.getTenantId());
            return recordsPublishingService.sendEventsWithRecords(storedRecords, jobExecutionId, okapiConnectionParams, eventType.value()).compose(b -> {
                LOGGER.debug("RecordsBatchResponse processing has been completed chunkId: {} chunkNumber: {} jobExecutionId: {}", chunkId, chunkNumber, jobExecutionId);
                return Future.succeededFuture(chunkId);
            }, th -> {
                LOGGER.error("RecordsBatchResponse processing has failed with errors chunkId: {} chunkNumber: {} jobExecutionId: {}", chunkId, chunkNumber, jobExecutionId, th);
                return Future.failedFuture(th);
            });
        });
    } catch (Exception e) {
        LOGGER.error("Can't process kafka record: ", e);
        return Future.failedFuture(e);
    }
}
Also used : RecordsBatchResponse(org.folio.rest.jaxrs.model.RecordsBatchResponse) Event(org.folio.rest.jaxrs.model.Event) JournalRecord(org.folio.rest.jaxrs.model.JournalRecord) Record(org.folio.rest.jaxrs.model.Record) KafkaConsumerRecord(io.vertx.kafka.client.consumer.KafkaConsumerRecord) DataImportEventTypes(org.folio.rest.jaxrs.model.DataImportEventTypes) OkapiConnectionParams(org.folio.dataimport.util.OkapiConnectionParams) KafkaHeader(io.vertx.kafka.client.producer.KafkaHeader)

Aggregations

OkapiConnectionParams (org.folio.dataimport.util.OkapiConnectionParams)2 DataImportEventTypes (org.folio.rest.jaxrs.model.DataImportEventTypes)2 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 Future (io.vertx.core.Future)1 Promise (io.vertx.core.Promise)1 KafkaConsumerRecord (io.vertx.kafka.client.consumer.KafkaConsumerRecord)1 KafkaHeader (io.vertx.kafka.client.producer.KafkaHeader)1 IOException (java.io.IOException)1 String.format (java.lang.String.format)1 Date (java.util.Date)1 LogManager (org.apache.logging.log4j.LogManager)1 Logger (org.apache.logging.log4j.Logger)1 DataImportEventPayload (org.folio.DataImportEventPayload)1 Event (org.folio.rest.jaxrs.model.Event)1 JobExecution (org.folio.rest.jaxrs.model.JobExecution)1 CANCELLED (org.folio.rest.jaxrs.model.JobExecution.Status.CANCELLED)1 COMMITTED (org.folio.rest.jaxrs.model.JobExecution.Status.COMMITTED)1 JobExecutionProgress (org.folio.rest.jaxrs.model.JobExecutionProgress)1 JournalRecord (org.folio.rest.jaxrs.model.JournalRecord)1 Progress (org.folio.rest.jaxrs.model.Progress)1