Search in sources :

Example 1 with DbWriteOperationsBlockedException

use of org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException in project nakadi by zalando.

the class TimelineService method createDefaultTimeline.

public Timeline createDefaultTimeline(final String eventTypeName, final int partitionsCount, final long retentionTime) throws TopicCreationException, InconsistentStateException, RepositoryProblemException, DuplicatedTimelineException, TimelineException, DbWriteOperationsBlockedException {
    if (featureToggleService.isFeatureEnabled(FeatureToggleService.Feature.DISABLE_DB_WRITE_OPERATIONS)) {
        throw new DbWriteOperationsBlockedException("Cannot create default timeline: write operations on DB " + "are blocked by feature flag.");
    }
    final TopicRepository repository = topicRepositoryHolder.getTopicRepository(defaultStorage.getStorage());
    final String topic = repository.createTopic(partitionsCount, retentionTime);
    try {
        final Timeline timeline = Timeline.createTimeline(eventTypeName, 1, defaultStorage.getStorage(), topic, new Date());
        timeline.setSwitchedAt(new Date());
        timelineDbRepository.createTimeline(timeline);
        eventTypeCache.updated(eventTypeName);
        return timeline;
    } catch (final InconsistentStateException | RepositoryProblemException | DuplicatedTimelineException e) {
        rollbackTopic(repository, topic);
        throw e;
    } catch (final Exception e) {
        rollbackTopic(repository, topic);
        throw new TimelineException("Failed to update event type cache, while creating timeline", e);
    }
}
Also used : DuplicatedTimelineException(org.zalando.nakadi.exceptions.runtime.DuplicatedTimelineException) Timeline(org.zalando.nakadi.domain.Timeline) TopicRepository(org.zalando.nakadi.repository.TopicRepository) RepositoryProblemException(org.zalando.nakadi.exceptions.runtime.RepositoryProblemException) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException) InconsistentStateException(org.zalando.nakadi.exceptions.runtime.InconsistentStateException) Date(java.util.Date) NakadiException(org.zalando.nakadi.exceptions.NakadiException) NoSuchEventTypeException(org.zalando.nakadi.exceptions.NoSuchEventTypeException) TopicRepositoryException(org.zalando.nakadi.exceptions.runtime.TopicRepositoryException) ServiceUnavailableException(org.zalando.nakadi.exceptions.ServiceUnavailableException) ConflictException(org.zalando.nakadi.exceptions.ConflictException) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException) TimelineException(org.zalando.nakadi.exceptions.TimelineException) InvalidCursorException(org.zalando.nakadi.exceptions.InvalidCursorException) TopicDeletionException(org.zalando.nakadi.exceptions.TopicDeletionException) InconsistentStateException(org.zalando.nakadi.exceptions.runtime.InconsistentStateException) NotFoundException(org.zalando.nakadi.exceptions.NotFoundException) TopicCreationException(org.zalando.nakadi.exceptions.TopicCreationException) UnableProcessException(org.zalando.nakadi.exceptions.UnableProcessException) DuplicatedTimelineException(org.zalando.nakadi.exceptions.runtime.DuplicatedTimelineException) AccessDeniedException(org.zalando.nakadi.exceptions.runtime.AccessDeniedException) TransactionException(org.springframework.transaction.TransactionException) InternalNakadiException(org.zalando.nakadi.exceptions.InternalNakadiException) RepositoryProblemException(org.zalando.nakadi.exceptions.runtime.RepositoryProblemException) TimelineException(org.zalando.nakadi.exceptions.TimelineException) DuplicatedTimelineException(org.zalando.nakadi.exceptions.runtime.DuplicatedTimelineException)

Example 2 with DbWriteOperationsBlockedException

use of org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException in project nakadi by zalando.

the class StorageService method createStorage.

public Result<Void> createStorage(final JSONObject json) throws DbWriteOperationsBlockedException {
    if (featureToggleService.isFeatureEnabled(FeatureToggleService.Feature.DISABLE_DB_WRITE_OPERATIONS)) {
        throw new DbWriteOperationsBlockedException("Cannot create storage: write operations on DB " + "are blocked by feature flag.");
    }
    final String type;
    final String id;
    final JSONObject configuration;
    try {
        id = json.getString("id");
        type = json.getString("storage_type");
        switch(type) {
            case "kafka":
                configuration = json.getJSONObject("kafka_configuration");
                break;
            default:
                return Result.problem(Problem.valueOf(UNPROCESSABLE_ENTITY, "Type '" + type + "' is not a valid storage type"));
        }
    } catch (JSONException e) {
        return Result.problem(Problem.valueOf(UNPROCESSABLE_ENTITY, e.getMessage()));
    }
    final Storage storage = new Storage();
    storage.setId(id);
    storage.setType(Storage.Type.valueOf(type.toUpperCase()));
    try {
        storage.parseConfiguration(objectMapper, configuration.toString());
    } catch (final IOException e) {
        return Result.problem(Problem.valueOf(UNPROCESSABLE_ENTITY, e.getMessage()));
    }
    try {
        storageDbRepository.createStorage(storage);
    } catch (final RepositoryProblemException e) {
        LOG.error("DB error occurred when creating storage", e);
        return Result.problem(Problem.valueOf(INTERNAL_SERVER_ERROR, e.getMessage()));
    } catch (final DuplicatedStorageException e) {
        return Result.problem(Problem.valueOf(CONFLICT, e.getMessage()));
    }
    return Result.ok();
}
Also used : Storage(org.zalando.nakadi.domain.Storage) DefaultStorage(org.zalando.nakadi.domain.DefaultStorage) JSONObject(org.json.JSONObject) JSONException(org.json.JSONException) IOException(java.io.IOException) RepositoryProblemException(org.zalando.nakadi.exceptions.runtime.RepositoryProblemException) DuplicatedStorageException(org.zalando.nakadi.exceptions.runtime.DuplicatedStorageException) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException)

Example 3 with DbWriteOperationsBlockedException

use of org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException in project nakadi by zalando.

the class TimelineService method createTimeline.

public void createTimeline(final String eventTypeName, final String storageId) throws AccessDeniedException, TimelineException, TopicRepositoryException, InconsistentStateException, RepositoryProblemException, DbWriteOperationsBlockedException {
    if (featureToggleService.isFeatureEnabled(FeatureToggleService.Feature.DISABLE_DB_WRITE_OPERATIONS)) {
        throw new DbWriteOperationsBlockedException("Cannot create timeline: write operations on DB " + "are blocked by feature flag.");
    }
    try {
        final EventType eventType = eventTypeCache.getEventType(eventTypeName);
        if (!adminService.isAdmin(AuthorizationService.Operation.WRITE)) {
            final Resource resource = new EventTypeResource(eventTypeName, eventType.getAuthorization());
            throw new AccessDeniedException(AuthorizationService.Operation.ADMIN, resource);
        }
        final Storage storage = storageDbRepository.getStorage(storageId).orElseThrow(() -> new UnableProcessException("No storage with id: " + storageId));
        final Timeline activeTimeline = getActiveTimeline(eventType);
        final TopicRepository currentTopicRepo = topicRepositoryHolder.getTopicRepository(activeTimeline.getStorage());
        final TopicRepository nextTopicRepo = topicRepositoryHolder.getTopicRepository(storage);
        final List<PartitionStatistics> partitionStatistics = currentTopicRepo.loadTopicStatistics(Collections.singleton(activeTimeline));
        final String newTopic = nextTopicRepo.createTopic(partitionStatistics.size(), eventType.getOptions().getRetentionTime());
        final Timeline nextTimeline = Timeline.createTimeline(activeTimeline.getEventType(), activeTimeline.getOrder() + 1, storage, newTopic, new Date());
        switchTimelines(activeTimeline, nextTimeline);
    } catch (final TopicCreationException | ServiceUnavailableException | InternalNakadiException e) {
        throw new TimelineException("Internal service error", e);
    } catch (final NoSuchEventTypeException e) {
        throw new NotFoundException("EventType \"" + eventTypeName + "\" does not exist", e);
    }
}
Also used : AccessDeniedException(org.zalando.nakadi.exceptions.runtime.AccessDeniedException) InternalNakadiException(org.zalando.nakadi.exceptions.InternalNakadiException) EventType(org.zalando.nakadi.domain.EventType) Resource(org.zalando.nakadi.plugin.api.authz.Resource) EventTypeResource(org.zalando.nakadi.domain.EventTypeResource) EventTypeResource(org.zalando.nakadi.domain.EventTypeResource) TopicCreationException(org.zalando.nakadi.exceptions.TopicCreationException) NotFoundException(org.zalando.nakadi.exceptions.NotFoundException) ServiceUnavailableException(org.zalando.nakadi.exceptions.ServiceUnavailableException) Date(java.util.Date) TimelineException(org.zalando.nakadi.exceptions.TimelineException) DuplicatedTimelineException(org.zalando.nakadi.exceptions.runtime.DuplicatedTimelineException) Timeline(org.zalando.nakadi.domain.Timeline) DefaultStorage(org.zalando.nakadi.domain.DefaultStorage) Storage(org.zalando.nakadi.domain.Storage) PartitionStatistics(org.zalando.nakadi.domain.PartitionStatistics) TopicRepository(org.zalando.nakadi.repository.TopicRepository) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException) UnableProcessException(org.zalando.nakadi.exceptions.UnableProcessException) NoSuchEventTypeException(org.zalando.nakadi.exceptions.NoSuchEventTypeException)

Example 4 with DbWriteOperationsBlockedException

use of org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException in project nakadi by zalando.

the class EventTypeService method update.

public void update(final String eventTypeName, final EventTypeBase eventTypeBase) throws TopicConfigException, InconsistentStateException, NakadiRuntimeException, ServiceTemporarilyUnavailableException, UnableProcessException, DbWriteOperationsBlockedException {
    if (featureToggleService.isFeatureEnabled(FeatureToggleService.Feature.DISABLE_DB_WRITE_OPERATIONS)) {
        throw new DbWriteOperationsBlockedException("Cannot update event type: write operations on DB " + "are blocked by feature flag.");
    }
    Closeable updatingCloser = null;
    try {
        updatingCloser = timelineSync.workWithEventType(eventTypeName, nakadiSettings.getTimelineWaitTimeoutMs());
        final EventType original = eventTypeRepository.findByName(eventTypeName);
        authorizationValidator.authorizeEventTypeAdmin(original);
        authorizationValidator.validateAuthorization(original, eventTypeBase);
        validateName(eventTypeName, eventTypeBase);
        validateSchema(eventTypeBase);
        partitionResolver.validate(eventTypeBase);
        final EventType eventType = schemaEvolutionService.evolve(original, eventTypeBase);
        eventType.setDefaultStatistic(validateStatisticsUpdate(original.getDefaultStatistic(), eventType.getDefaultStatistic()));
        updateRetentionTime(original, eventType);
    } catch (final InterruptedException e) {
        Thread.currentThread().interrupt();
        throw new ServiceTemporarilyUnavailableException("Event type is currently in maintenance, please repeat request", e);
    } catch (final TimeoutException e) {
        LOG.error("Failed to wait for timeline switch", e);
        throw new ServiceTemporarilyUnavailableException("Event type is currently in maintenance, please repeat request", e);
    } catch (final NakadiException e) {
        LOG.error("Unable to update event type", e);
        throw new NakadiRuntimeException(e);
    } finally {
        try {
            if (updatingCloser != null) {
                updatingCloser.close();
            }
        } catch (final IOException e) {
            LOG.error("Exception occurred when releasing usage of event-type", e);
        }
    }
    nakadiKpiPublisher.publish(etLogEventType, () -> new JSONObject().put("event_type", eventTypeName).put("status", "updated").put("category", eventTypeBase.getCategory()).put("authz", identifyAuthzState(eventTypeBase)).put("compatibility_mode", eventTypeBase.getCompatibilityMode()));
}
Also used : ServiceTemporarilyUnavailableException(org.zalando.nakadi.exceptions.runtime.ServiceTemporarilyUnavailableException) NakadiRuntimeException(org.zalando.nakadi.exceptions.NakadiRuntimeException) JSONObject(org.json.JSONObject) EventType(org.zalando.nakadi.domain.EventType) Closeable(java.io.Closeable) IOException(java.io.IOException) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException) TimeoutException(java.util.concurrent.TimeoutException) NakadiException(org.zalando.nakadi.exceptions.NakadiException) InternalNakadiException(org.zalando.nakadi.exceptions.InternalNakadiException)

Example 5 with DbWriteOperationsBlockedException

use of org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException in project nakadi by zalando.

the class EventTypeService method delete.

public void delete(final String eventTypeName) throws EventTypeDeletionException, AccessDeniedException, NoEventTypeException, ConflictException, ServiceTemporarilyUnavailableException, DbWriteOperationsBlockedException {
    if (featureToggleService.isFeatureEnabled(FeatureToggleService.Feature.DISABLE_DB_WRITE_OPERATIONS)) {
        throw new DbWriteOperationsBlockedException("Cannot delete event type: write operations on DB " + "are blocked by feature flag.");
    }
    Closeable deletionCloser = null;
    final EventType eventType;
    Multimap<TopicRepository, String> topicsToDelete = null;
    try {
        deletionCloser = timelineSync.workWithEventType(eventTypeName, nakadiSettings.getTimelineWaitTimeoutMs());
        final Optional<EventType> eventTypeOpt = eventTypeRepository.findByNameO(eventTypeName);
        if (!eventTypeOpt.isPresent()) {
            throw new NoEventTypeException("EventType \"" + eventTypeName + "\" does not exist.");
        }
        eventType = eventTypeOpt.get();
        authorizationValidator.authorizeEventTypeAdmin(eventType);
        final List<Subscription> subscriptions = subscriptionRepository.listSubscriptions(ImmutableSet.of(eventTypeName), Optional.empty(), 0, 1);
        if (!subscriptions.isEmpty()) {
            throw new ConflictException("Can't remove event type " + eventTypeName + ", as it has subscriptions");
        }
        topicsToDelete = transactionTemplate.execute(action -> deleteEventType(eventTypeName));
    } catch (final InterruptedException e) {
        Thread.currentThread().interrupt();
        LOG.error("Failed to wait for timeline switch", e);
        throw new EventTypeUnavailableException("Event type " + eventTypeName + " is currently in maintenance, please repeat request");
    } catch (final TimeoutException e) {
        LOG.error("Failed to wait for timeline switch", e);
        throw new EventTypeUnavailableException("Event type " + eventTypeName + " is currently in maintenance, please repeat request");
    } catch (final NakadiException e) {
        LOG.error("Error deleting event type " + eventTypeName, e);
        throw new EventTypeDeletionException("Failed to delete event type " + eventTypeName);
    } finally {
        try {
            if (deletionCloser != null) {
                deletionCloser.close();
            }
        } catch (final IOException e) {
            LOG.error("Exception occurred when releasing usage of event-type", e);
        }
    }
    if (topicsToDelete != null) {
        for (final TopicRepository topicRepository : topicsToDelete.keySet()) {
            for (final String topic : topicsToDelete.get(topicRepository)) {
                try {
                    topicRepository.deleteTopic(topic);
                } catch (TopicDeletionException e) {
                    // If a timeline was marked as deleted, then the topic does not exist, and we should proceed.
                    LOG.info("Could not delete topic " + topic, e);
                }
            }
        }
    }
    nakadiKpiPublisher.publish(etLogEventType, () -> new JSONObject().put("event_type", eventTypeName).put("status", "deleted").put("category", eventType.getCategory()).put("authz", identifyAuthzState(eventType)).put("compatibility_mode", eventType.getCompatibilityMode()));
}
Also used : EventCategory(org.zalando.nakadi.domain.EventCategory) Date(java.util.Date) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) DuplicatedEventTypeNameException(org.zalando.nakadi.exceptions.DuplicatedEventTypeNameException) TimeoutException(java.util.concurrent.TimeoutException) Subscription(org.zalando.nakadi.domain.Subscription) NakadiException(org.zalando.nakadi.exceptions.NakadiException) NoSuchEventTypeException(org.zalando.nakadi.exceptions.NoSuchEventTypeException) SchemaEvolutionService(org.zalando.nakadi.validation.SchemaEvolutionService) JSONException(org.json.JSONException) JSONObject(org.json.JSONObject) EventTypeStatistics(org.zalando.nakadi.domain.EventTypeStatistics) ConflictException(org.zalando.nakadi.exceptions.ConflictException) TimelineService(org.zalando.nakadi.service.timeline.TimelineService) ImmutableSet(com.google.common.collect.ImmutableSet) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException) EventTypeUnavailableException(org.zalando.nakadi.exceptions.runtime.EventTypeUnavailableException) TopicRepository(org.zalando.nakadi.repository.TopicRepository) TimelineException(org.zalando.nakadi.exceptions.TimelineException) TimelineSync(org.zalando.nakadi.service.timeline.TimelineSync) Collectors(java.util.stream.Collectors) TopicDeletionException(org.zalando.nakadi.exceptions.TopicDeletionException) Objects(java.util.Objects) List(java.util.List) Timeline(org.zalando.nakadi.domain.Timeline) EventTypeBase(org.zalando.nakadi.domain.EventTypeBase) ServiceTemporarilyUnavailableException(org.zalando.nakadi.exceptions.runtime.ServiceTemporarilyUnavailableException) SubscriptionDbRepository(org.zalando.nakadi.repository.db.SubscriptionDbRepository) CompatibilityMode(org.zalando.nakadi.domain.CompatibilityMode) Optional(java.util.Optional) NoEventTypeException(org.zalando.nakadi.exceptions.runtime.NoEventTypeException) CHECK_PARTITIONS_KEYS(org.zalando.nakadi.service.FeatureToggleService.Feature.CHECK_PARTITIONS_KEYS) InvalidEventTypeException(org.zalando.nakadi.exceptions.InvalidEventTypeException) EventTypeOptions(org.zalando.nakadi.domain.EventTypeOptions) InconsistentStateException(org.zalando.nakadi.exceptions.runtime.InconsistentStateException) NakadiSettings(org.zalando.nakadi.config.NakadiSettings) NotFoundException(org.zalando.nakadi.exceptions.NotFoundException) TopicCreationException(org.zalando.nakadi.exceptions.TopicCreationException) Multimap(com.google.common.collect.Multimap) TopicConfigException(org.zalando.nakadi.exceptions.runtime.TopicConfigException) UnableProcessException(org.zalando.nakadi.exceptions.UnableProcessException) PartitionResolver(org.zalando.nakadi.partitioning.PartitionResolver) Value(org.springframework.beans.factory.annotation.Value) JsonUtils(org.zalando.nakadi.util.JsonUtils) SchemaException(org.everit.json.schema.SchemaException) Enrichment(org.zalando.nakadi.enrichment.Enrichment) NoSuchPartitionStrategyException(org.zalando.nakadi.exceptions.NoSuchPartitionStrategyException) EventTypeDeletionException(org.zalando.nakadi.exceptions.runtime.EventTypeDeletionException) SchemaLoader(org.everit.json.schema.loader.SchemaLoader) AccessDeniedException(org.zalando.nakadi.exceptions.runtime.AccessDeniedException) EventType(org.zalando.nakadi.domain.EventType) Logger(org.slf4j.Logger) PartitionsCalculator(org.zalando.nakadi.repository.kafka.PartitionsCalculator) IOException(java.io.IOException) NakadiRuntimeException(org.zalando.nakadi.exceptions.NakadiRuntimeException) Component(org.springframework.stereotype.Component) TransactionTemplate(org.springframework.transaction.support.TransactionTemplate) EventTypeRepository(org.zalando.nakadi.repository.EventTypeRepository) Closeable(java.io.Closeable) Schema(org.everit.json.schema.Schema) SchemaIncompatibility(org.zalando.nakadi.validation.SchemaIncompatibility) InternalNakadiException(org.zalando.nakadi.exceptions.InternalNakadiException) EventType(org.zalando.nakadi.domain.EventType) ConflictException(org.zalando.nakadi.exceptions.ConflictException) Closeable(java.io.Closeable) IOException(java.io.IOException) NakadiException(org.zalando.nakadi.exceptions.NakadiException) InternalNakadiException(org.zalando.nakadi.exceptions.InternalNakadiException) TopicDeletionException(org.zalando.nakadi.exceptions.TopicDeletionException) JSONObject(org.json.JSONObject) EventTypeDeletionException(org.zalando.nakadi.exceptions.runtime.EventTypeDeletionException) TopicRepository(org.zalando.nakadi.repository.TopicRepository) EventTypeUnavailableException(org.zalando.nakadi.exceptions.runtime.EventTypeUnavailableException) NoEventTypeException(org.zalando.nakadi.exceptions.runtime.NoEventTypeException) Subscription(org.zalando.nakadi.domain.Subscription) DbWriteOperationsBlockedException(org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException) TimeoutException(java.util.concurrent.TimeoutException)

Aggregations

DbWriteOperationsBlockedException (org.zalando.nakadi.exceptions.runtime.DbWriteOperationsBlockedException)7 JSONObject (org.json.JSONObject)5 InternalNakadiException (org.zalando.nakadi.exceptions.InternalNakadiException)5 NoSuchEventTypeException (org.zalando.nakadi.exceptions.NoSuchEventTypeException)4 IOException (java.io.IOException)3 Date (java.util.Date)3 EventType (org.zalando.nakadi.domain.EventType)3 Subscription (org.zalando.nakadi.domain.Subscription)3 Timeline (org.zalando.nakadi.domain.Timeline)3 NakadiException (org.zalando.nakadi.exceptions.NakadiException)3 NotFoundException (org.zalando.nakadi.exceptions.NotFoundException)3 ServiceUnavailableException (org.zalando.nakadi.exceptions.ServiceUnavailableException)3 TimelineException (org.zalando.nakadi.exceptions.TimelineException)3 TopicCreationException (org.zalando.nakadi.exceptions.TopicCreationException)3 UnableProcessException (org.zalando.nakadi.exceptions.UnableProcessException)3 AccessDeniedException (org.zalando.nakadi.exceptions.runtime.AccessDeniedException)3 TopicRepository (org.zalando.nakadi.repository.TopicRepository)3 Closeable (java.io.Closeable)2 TimeoutException (java.util.concurrent.TimeoutException)2 JSONException (org.json.JSONException)2