Search in sources :

Example 1 with VersionRange

use of org.apache.kafka.metadata.VersionRange in project kafka by apache.

the class FeatureControlManagerTest method rangeMap.

@SuppressWarnings("unchecked")
private static Map<String, VersionRange> rangeMap(Object... args) {
    Map<String, VersionRange> result = new HashMap<>();
    for (int i = 0; i < args.length; i += 3) {
        String feature = (String) args[i];
        Integer low = (Integer) args[i + 1];
        Integer high = (Integer) args[i + 2];
        result.put(feature, new VersionRange(low.shortValue(), high.shortValue()));
    }
    return result;
}
Also used : HashMap(java.util.HashMap) VersionRange(org.apache.kafka.metadata.VersionRange)

Example 2 with VersionRange

use of org.apache.kafka.metadata.VersionRange in project kafka by apache.

the class FeaturesImage method write.

public void write(Consumer<List<ApiMessageAndVersion>> out) {
    List<ApiMessageAndVersion> batch = new ArrayList<>();
    for (Entry<String, VersionRange> entry : finalizedVersions.entrySet()) {
        batch.add(new ApiMessageAndVersion(new FeatureLevelRecord().setName(entry.getKey()).setMinFeatureLevel(entry.getValue().min()).setMaxFeatureLevel(entry.getValue().max()), FEATURE_LEVEL_RECORD.highestSupportedVersion()));
    }
    out.accept(batch);
}
Also used : ApiMessageAndVersion(org.apache.kafka.server.common.ApiMessageAndVersion) ArrayList(java.util.ArrayList) VersionRange(org.apache.kafka.metadata.VersionRange) FeatureLevelRecord(org.apache.kafka.common.metadata.FeatureLevelRecord)

Example 3 with VersionRange

use of org.apache.kafka.metadata.VersionRange in project kafka by apache.

the class FeatureControlManager method updateFeature.

private ApiError updateFeature(String featureName, VersionRange newRange, boolean downgradeable, Map<Integer, Map<String, VersionRange>> brokerFeatures, List<ApiMessageAndVersion> records) {
    if (newRange.min() <= 0) {
        return new ApiError(Errors.INVALID_UPDATE_VERSION, "The lower value for the new range cannot be less than 1.");
    }
    if (newRange.max() <= 0) {
        return new ApiError(Errors.INVALID_UPDATE_VERSION, "The upper value for the new range cannot be less than 1.");
    }
    VersionRange localRange = supportedFeatures.get(featureName);
    if (localRange == null || !localRange.contains(newRange)) {
        return new ApiError(Errors.INVALID_UPDATE_VERSION, "The controller does not support the given feature range.");
    }
    for (Entry<Integer, Map<String, VersionRange>> brokerEntry : brokerFeatures.entrySet()) {
        VersionRange brokerRange = brokerEntry.getValue().get(featureName);
        if (brokerRange == null || !brokerRange.contains(newRange)) {
            return new ApiError(Errors.INVALID_UPDATE_VERSION, "Broker " + brokerEntry.getKey() + " does not support the given " + "feature range.");
        }
    }
    VersionRange currentRange = finalizedVersions.get(featureName);
    if (currentRange != null && currentRange.max() > newRange.max()) {
        if (!downgradeable) {
            return new ApiError(Errors.INVALID_UPDATE_VERSION, "Can't downgrade the maximum version of this feature without " + "setting downgradable to true.");
        }
    }
    records.add(new ApiMessageAndVersion(new FeatureLevelRecord().setName(featureName).setMinFeatureLevel(newRange.min()).setMaxFeatureLevel(newRange.max()), FEATURE_LEVEL_RECORD.highestSupportedVersion()));
    return ApiError.NONE;
}
Also used : ApiMessageAndVersion(org.apache.kafka.server.common.ApiMessageAndVersion) VersionRange(org.apache.kafka.metadata.VersionRange) ApiError(org.apache.kafka.common.requests.ApiError) HashMap(java.util.HashMap) TimelineHashMap(org.apache.kafka.timeline.TimelineHashMap) FeatureMap(org.apache.kafka.metadata.FeatureMap) TreeMap(java.util.TreeMap) Map(java.util.Map) FeatureLevelRecord(org.apache.kafka.common.metadata.FeatureLevelRecord)

Example 4 with VersionRange

use of org.apache.kafka.metadata.VersionRange in project kafka by apache.

the class ClusterControlManager method replay.

public void replay(RegisterBrokerRecord record) {
    int brokerId = record.brokerId();
    List<Endpoint> listeners = new ArrayList<>();
    for (BrokerEndpoint endpoint : record.endPoints()) {
        listeners.add(new Endpoint(endpoint.name(), SecurityProtocol.forId(endpoint.securityProtocol()), endpoint.host(), endpoint.port()));
    }
    Map<String, VersionRange> features = new HashMap<>();
    for (BrokerFeature feature : record.features()) {
        features.put(feature.name(), new VersionRange(feature.minSupportedVersion(), feature.maxSupportedVersion()));
    }
    // Update broker registrations.
    BrokerRegistration prevRegistration = brokerRegistrations.put(brokerId, new BrokerRegistration(brokerId, record.brokerEpoch(), record.incarnationId(), listeners, features, Optional.ofNullable(record.rack()), record.fenced()));
    updateMetrics(prevRegistration, brokerRegistrations.get(brokerId));
    if (prevRegistration == null) {
        log.info("Registered new broker: {}", record);
    } else if (prevRegistration.incarnationId().equals(record.incarnationId())) {
        log.info("Re-registered broker incarnation: {}", record);
    } else {
        log.info("Re-registered broker id {}: {}", brokerId, record);
    }
}
Also used : BrokerFeature(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeature) Endpoint(org.apache.kafka.common.Endpoint) BrokerEndpoint(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint) TimelineHashMap(org.apache.kafka.timeline.TimelineHashMap) HashMap(java.util.HashMap) BrokerEndpoint(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint) ArrayList(java.util.ArrayList) VersionRange(org.apache.kafka.metadata.VersionRange) BrokerRegistration(org.apache.kafka.metadata.BrokerRegistration) Endpoint(org.apache.kafka.common.Endpoint) BrokerEndpoint(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint)

Example 5 with VersionRange

use of org.apache.kafka.metadata.VersionRange in project kafka by apache.

the class ClusterControlManager method registerBroker.

/**
 * Process an incoming broker registration request.
 */
public ControllerResult<BrokerRegistrationReply> registerBroker(BrokerRegistrationRequestData request, long brokerEpoch, FeatureMapAndEpoch finalizedFeatures) {
    if (heartbeatManager == null) {
        throw new RuntimeException("ClusterControlManager is not active.");
    }
    if (!clusterId.equals(request.clusterId())) {
        throw new InconsistentClusterIdException("Expected cluster ID " + clusterId + ", but got cluster ID " + request.clusterId());
    }
    int brokerId = request.brokerId();
    BrokerRegistration existing = brokerRegistrations.get(brokerId);
    if (existing != null) {
        if (heartbeatManager.hasValidSession(brokerId)) {
            if (!existing.incarnationId().equals(request.incarnationId())) {
                throw new DuplicateBrokerRegistrationException("Another broker is " + "registered with that broker id.");
            }
        } else {
            if (!existing.incarnationId().equals(request.incarnationId())) {
                // Remove any existing session for the old broker incarnation.
                heartbeatManager.remove(brokerId);
                existing = null;
            }
        }
    }
    RegisterBrokerRecord record = new RegisterBrokerRecord().setBrokerId(brokerId).setIncarnationId(request.incarnationId()).setBrokerEpoch(brokerEpoch).setRack(request.rack());
    for (BrokerRegistrationRequestData.Listener listener : request.listeners()) {
        record.endPoints().add(new BrokerEndpoint().setHost(listener.host()).setName(listener.name()).setPort(listener.port()).setSecurityProtocol(listener.securityProtocol()));
    }
    for (BrokerRegistrationRequestData.Feature feature : request.features()) {
        Optional<VersionRange> finalized = finalizedFeatures.map().get(feature.name());
        if (finalized.isPresent()) {
            if (!finalized.get().contains(new VersionRange(feature.minSupportedVersion(), feature.maxSupportedVersion()))) {
                throw new UnsupportedVersionException("Unable to register because " + "the broker has an unsupported version of " + feature.name());
            }
        }
        record.features().add(new BrokerFeature().setName(feature.name()).setMinSupportedVersion(feature.minSupportedVersion()).setMaxSupportedVersion(feature.maxSupportedVersion()));
    }
    if (existing == null) {
        heartbeatManager.touch(brokerId, true, -1);
    } else {
        heartbeatManager.touch(brokerId, existing.fenced(), -1);
    }
    List<ApiMessageAndVersion> records = new ArrayList<>();
    records.add(new ApiMessageAndVersion(record, REGISTER_BROKER_RECORD.highestSupportedVersion()));
    return ControllerResult.atomicOf(records, new BrokerRegistrationReply(brokerEpoch));
}
Also used : BrokerFeature(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeature) BrokerRegistrationRequestData(org.apache.kafka.common.message.BrokerRegistrationRequestData) ArrayList(java.util.ArrayList) BrokerRegistrationReply(org.apache.kafka.metadata.BrokerRegistrationReply) VersionRange(org.apache.kafka.metadata.VersionRange) BrokerRegistration(org.apache.kafka.metadata.BrokerRegistration) Endpoint(org.apache.kafka.common.Endpoint) BrokerEndpoint(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint) InconsistentClusterIdException(org.apache.kafka.common.errors.InconsistentClusterIdException) DuplicateBrokerRegistrationException(org.apache.kafka.common.errors.DuplicateBrokerRegistrationException) RegisterBrokerRecord(org.apache.kafka.common.metadata.RegisterBrokerRecord) BrokerEndpoint(org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint) ApiMessageAndVersion(org.apache.kafka.server.common.ApiMessageAndVersion) UnsupportedVersionException(org.apache.kafka.common.errors.UnsupportedVersionException)

Aggregations

VersionRange (org.apache.kafka.metadata.VersionRange)5 ArrayList (java.util.ArrayList)3 HashMap (java.util.HashMap)3 ApiMessageAndVersion (org.apache.kafka.server.common.ApiMessageAndVersion)3 Endpoint (org.apache.kafka.common.Endpoint)2 FeatureLevelRecord (org.apache.kafka.common.metadata.FeatureLevelRecord)2 BrokerEndpoint (org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint)2 BrokerFeature (org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeature)2 BrokerRegistration (org.apache.kafka.metadata.BrokerRegistration)2 TimelineHashMap (org.apache.kafka.timeline.TimelineHashMap)2 Map (java.util.Map)1 TreeMap (java.util.TreeMap)1 DuplicateBrokerRegistrationException (org.apache.kafka.common.errors.DuplicateBrokerRegistrationException)1 InconsistentClusterIdException (org.apache.kafka.common.errors.InconsistentClusterIdException)1 UnsupportedVersionException (org.apache.kafka.common.errors.UnsupportedVersionException)1 BrokerRegistrationRequestData (org.apache.kafka.common.message.BrokerRegistrationRequestData)1 RegisterBrokerRecord (org.apache.kafka.common.metadata.RegisterBrokerRecord)1 ApiError (org.apache.kafka.common.requests.ApiError)1 BrokerRegistrationReply (org.apache.kafka.metadata.BrokerRegistrationReply)1 FeatureMap (org.apache.kafka.metadata.FeatureMap)1