Search in sources :

Example 1 with SubTaggedMetricContextKey

use of org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey in project incubator-gobblin by apache.

the class LimiterServerResource method get.

/**
 * Request permits from the limiter server. The returned {@link PermitAllocation} specifies the number of permits
 * that the client can use.
 */
@Override
@RestMethod.Get
public void get(ComplexResourceKey<PermitRequest, EmptyRecord> key, @CallbackParam final Callback<PermitAllocation> callback) {
    try (Closeable context = this.requestTimer == null ? NoopCloseable.INSTANCE : this.requestTimer.time()) {
        PermitRequest request = key.getKey();
        String resourceId = request.getResource();
        MetricContext resourceContext = (MetricContext) broker.getSharedResource(new MetricContextFactory(), new SubTaggedMetricContextKey(resourceId, ImmutableMap.of(RESOURCE_ID_TAG, resourceId)));
        Meter permitsRequestedMeter = resourceContext.meter(PERMITS_REQUESTED_METER_NAME);
        Meter permitsGrantedMeter = resourceContext.meter(PERMITS_GRANTED_METER_NAME);
        Timer limiterTimer = resourceContext.timer(LIMITER_TIMER_NAME);
        permitsRequestedMeter.mark(request.getPermits());
        if (this.leaderFinderOpt.isPresent() && !this.leaderFinderOpt.get().isLeader()) {
            URI leaderUri = this.leaderFinderOpt.get().getLeaderMetadata().getUri();
            RestLiServiceException exception = new RestLiServiceException(HttpStatus.S_301_MOVED_PERMANENTLY, String.format("New leader <a href=\"%s\">%s</a>", leaderUri, leaderUri));
            exception.setErrorDetails(new DataMap(ImmutableMap.of(LOCATION_301, leaderUri.toString())));
            throw exception;
        } else {
            ThrottlingPolicy policy = (ThrottlingPolicy) this.broker.getSharedResource(new ThrottlingPolicyFactory(), new SharedLimiterKey(request.getResource()));
            PermitAllocation allocation;
            try (Closeable thisContext = limiterTimer.time()) {
                allocation = policy.computePermitAllocation(request);
            }
            permitsGrantedMeter.mark(allocation.getPermits());
            callback.onSuccess(allocation);
        }
    } catch (NotConfiguredException nce) {
        throw new RestLiServiceException(HttpStatus.S_422_UNPROCESSABLE_ENTITY, "No configuration for the requested resource.");
    } catch (IOException ioe) {
        // Failed to close timer context. This should never happen
        throw new RuntimeException(ioe);
    }
}
Also used : NotConfiguredException(org.apache.gobblin.broker.iface.NotConfiguredException) Meter(com.codahale.metrics.Meter) Closeable(java.io.Closeable) NoopCloseable(org.apache.gobblin.util.NoopCloseable) IOException(java.io.IOException) URI(java.net.URI) DataMap(com.linkedin.data.DataMap) SubTaggedMetricContextKey(org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey) Timer(com.codahale.metrics.Timer) RestLiServiceException(com.linkedin.restli.server.RestLiServiceException) MetricContext(org.apache.gobblin.metrics.MetricContext) MetricContextFactory(org.apache.gobblin.metrics.broker.MetricContextFactory) SharedLimiterKey(org.apache.gobblin.util.limiter.broker.SharedLimiterKey)

Example 2 with SubTaggedMetricContextKey

use of org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey in project incubator-gobblin by apache.

the class PoliciesResource method get.

@Override
public Policy get(String resourceId) {
    try {
        ThrottlingPolicy throttlingPolicy = (ThrottlingPolicy) this.broker.getSharedResource(new ThrottlingPolicyFactory(), new SharedLimiterKey(resourceId));
        Policy restliPolicy = new Policy();
        restliPolicy.setPolicyName(throttlingPolicy.getClass().getSimpleName());
        restliPolicy.setResource(resourceId);
        restliPolicy.setParameters(new StringMap(throttlingPolicy.getParameters()));
        restliPolicy.setPolicyDetails(throttlingPolicy.getDescription());
        MetricContext resourceContext = (MetricContext) broker.getSharedResource(new MetricContextFactory(), new SubTaggedMetricContextKey(resourceId, ImmutableMap.of(RESOURCE_ID_TAG, resourceId)));
        StringMap metrics = new StringMap();
        for (Map.Entry<String, Meter> meter : resourceContext.getMeters().entrySet()) {
            metrics.put(meter.getKey(), Double.toString(meter.getValue().getOneMinuteRate()));
        }
        restliPolicy.setMetrics(metrics);
        return restliPolicy;
    } catch (NotConfiguredException nce) {
        throw new RestLiServiceException(HttpStatus.S_404_NOT_FOUND, "Policy not found for resource " + resourceId);
    }
}
Also used : NotConfiguredException(org.apache.gobblin.broker.iface.NotConfiguredException) StringMap(com.linkedin.data.template.StringMap) Meter(com.codahale.metrics.Meter) SubTaggedMetricContextKey(org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey) RestLiServiceException(com.linkedin.restli.server.RestLiServiceException) MetricContext(org.apache.gobblin.metrics.MetricContext) MetricContextFactory(org.apache.gobblin.metrics.broker.MetricContextFactory) SharedLimiterKey(org.apache.gobblin.util.limiter.broker.SharedLimiterKey) ImmutableMap(com.google.common.collect.ImmutableMap) Map(java.util.Map) StringMap(com.linkedin.data.template.StringMap)

Example 3 with SubTaggedMetricContextKey

use of org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey in project incubator-gobblin by apache.

the class RestliLimiterFactory method createResource.

@Override
public SharedResourceFactoryResponse<RestliServiceBasedLimiter> createResource(SharedResourcesBroker<S> broker, ScopedConfigView<S, SharedLimiterKey> config) throws NotConfiguredException {
    S scope = config.getScope();
    if (scope != scope.rootScope()) {
        return new ResourceCoordinate<>(this, config.getKey(), scope.rootScope());
    }
    String serviceIdentifier = config.getConfig().hasPath(SERVICE_IDENTIFIER_KEY) ? config.getConfig().getString(SERVICE_IDENTIFIER_KEY) : "UNKNOWN";
    String resourceLimited = config.getKey().getResourceLimitedPath();
    MetricContextKey metricContextKey = new SubTaggedMetricContextKey(RestliServiceBasedLimiter.class.getSimpleName() + "_" + resourceLimited, ImmutableMap.of("resourceLimited", resourceLimited));
    return new ResourceInstance<>(RestliServiceBasedLimiter.builder().resourceLimited(resourceLimited).serviceIdentifier(serviceIdentifier).metricContext(broker.getSharedResource(new MetricContextFactory<S>(), metricContextKey)).requestSender(broker.getSharedResource(new RedirectAwareRestClientRequestSender.Factory<S>(), new SharedRestClientKey(RESTLI_SERVICE_NAME))).build());
}
Also used : SubTaggedMetricContextKey(org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey) SharedRestClientKey(org.apache.gobblin.restli.SharedRestClientKey) ResourceInstance(org.apache.gobblin.broker.ResourceInstance) MetricContextFactory(org.apache.gobblin.metrics.broker.MetricContextFactory) SharedResourceFactory(org.apache.gobblin.broker.iface.SharedResourceFactory) SubTaggedMetricContextKey(org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey) MetricContextKey(org.apache.gobblin.metrics.broker.MetricContextKey) ResourceCoordinate(org.apache.gobblin.broker.ResourceCoordinate)

Aggregations

MetricContextFactory (org.apache.gobblin.metrics.broker.MetricContextFactory)3 SubTaggedMetricContextKey (org.apache.gobblin.metrics.broker.SubTaggedMetricContextKey)3 Meter (com.codahale.metrics.Meter)2 RestLiServiceException (com.linkedin.restli.server.RestLiServiceException)2 NotConfiguredException (org.apache.gobblin.broker.iface.NotConfiguredException)2 MetricContext (org.apache.gobblin.metrics.MetricContext)2 SharedLimiterKey (org.apache.gobblin.util.limiter.broker.SharedLimiterKey)2 Timer (com.codahale.metrics.Timer)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 DataMap (com.linkedin.data.DataMap)1 StringMap (com.linkedin.data.template.StringMap)1 Closeable (java.io.Closeable)1 IOException (java.io.IOException)1 URI (java.net.URI)1 Map (java.util.Map)1 ResourceCoordinate (org.apache.gobblin.broker.ResourceCoordinate)1 ResourceInstance (org.apache.gobblin.broker.ResourceInstance)1 SharedResourceFactory (org.apache.gobblin.broker.iface.SharedResourceFactory)1 MetricContextKey (org.apache.gobblin.metrics.broker.MetricContextKey)1 SharedRestClientKey (org.apache.gobblin.restli.SharedRestClientKey)1