Search in sources :

Example 86 with RestRequest

use of com.github.ambry.rest.RestRequest in project ambry by linkedin.

the class FrontendRestRequestService method handleGet.

@Override
public void handleGet(final RestRequest restRequest, final RestResponseChannel restResponseChannel) {
    ThrowingConsumer<RequestPath> routingAction = requestPath -> {
        if (requestPath.matchesOperation(Operations.GET_PEERS)) {
            getPeersHandler.handle(restRequest, restResponseChannel, (result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception));
        } else if (requestPath.matchesOperation(Operations.GET_CLUSTER_MAP_SNAPSHOT)) {
            getClusterMapSnapshotHandler.handle(restRequest, restResponseChannel, (result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception));
        } else if (requestPath.matchesOperation(Operations.GET_SIGNED_URL)) {
            getSignedUrlHandler.handle(restRequest, restResponseChannel, (result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception));
        } else if (requestPath.matchesOperation(Operations.ACCOUNTS)) {
            getAccountsHandler.handle(restRequest, restResponseChannel, (result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception));
        } else if (requestPath.matchesOperation(Operations.STATS_REPORT)) {
            getStatsReportHandler.handle(restRequest, restResponseChannel, (result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception));
        } else if (requestPath.matchesOperation(Operations.NAMED_BLOB) && NamedBlobPath.parse(requestPath, restRequest.getArgs()).getBlobName() == null) {
            listNamedBlobsHandler.handle(restRequest, restResponseChannel, ((result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception)));
        } else {
            SubResource subResource = requestPath.getSubResource();
            GetBlobOptions options = buildGetBlobOptions(restRequest.getArgs(), subResource, getGetOption(restRequest, frontendConfig.defaultRouterGetOption), restRequest, requestPath.getBlobSegmentIdx());
            GetCallback routerCallback = new GetCallback(restRequest, restResponseChannel, subResource, options);
            SecurityProcessRequestCallback securityCallback = new SecurityProcessRequestCallback(restRequest, restResponseChannel, routerCallback);
            if (subResource == SubResource.Replicas) {
                securityCallback = new SecurityProcessRequestCallback(restRequest, restResponseChannel);
            }
            RestRequestMetricsGroup metricsGroup = getMetricsGroupForGet(frontendMetrics, subResource);
            RestRequestMetrics restRequestMetrics = metricsGroup.getRestRequestMetrics(restRequest.isSslUsed(), false);
            restRequest.getMetricsTracker().injectMetrics(restRequestMetrics);
            // named blob requests have their account/container in the URI, so checks can be done prior to ID conversion.
            if (requestPath.matchesOperation(Operations.NAMED_BLOB)) {
                accountAndContainerInjector.injectAccountAndContainerForNamedBlob(restRequest, metricsGroup);
            }
            securityService.processRequest(restRequest, securityCallback);
        }
    };
    preProcessAndRouteRequest(restRequest, restResponseChannel, frontendMetrics.getPreProcessingMetrics, routingAction);
}
Also used : RequestPath(com.github.ambry.rest.RequestPath) Histogram(com.codahale.metrics.Histogram) GetOption(com.github.ambry.protocol.GetOption) FrontendConfig(com.github.ambry.config.FrontendConfig) ResponseStatus(com.github.ambry.rest.ResponseStatus) LoggerFactory(org.slf4j.LoggerFactory) ByteBufferReadableStreamChannel(com.github.ambry.commons.ByteBufferReadableStreamChannel) AccountService(com.github.ambry.account.AccountService) QuotaManager(com.github.ambry.quota.QuotaManager) ByteBuffer(java.nio.ByteBuffer) ThrowingConsumer(com.github.ambry.utils.ThrowingConsumer) RequestPath(com.github.ambry.rest.RequestPath) NamedBlobDb(com.github.ambry.named.NamedBlobDb) RestRequestService(com.github.ambry.rest.RestRequestService) RestRequestMetrics(com.github.ambry.rest.RestRequestMetrics) SystemTime(com.github.ambry.utils.SystemTime) Router(com.github.ambry.router.Router) RouterErrorCode(com.github.ambry.router.RouterErrorCode) RestResponseHandler(com.github.ambry.rest.RestResponseHandler) ReadableStreamChannel(com.github.ambry.router.ReadableStreamChannel) Logger(org.slf4j.Logger) GregorianCalendar(java.util.GregorianCalendar) RestMethod(com.github.ambry.rest.RestMethod) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) RestServiceErrorCode(com.github.ambry.rest.RestServiceErrorCode) InternalKeys(com.github.ambry.rest.RestUtils.InternalKeys) ClusterMap(com.github.ambry.clustermap.ClusterMap) Utils(com.github.ambry.utils.Utils) IOException(java.io.IOException) GetBlobOptions(com.github.ambry.router.GetBlobOptions) RouterException(com.github.ambry.router.RouterException) BlobInfo(com.github.ambry.messageformat.BlobInfo) QuotaUtils(com.github.ambry.quota.QuotaUtils) AccountStatsStore(com.github.ambry.accountstats.AccountStatsStore) RestServiceException(com.github.ambry.rest.RestServiceException) GetBlobResult(com.github.ambry.router.GetBlobResult) Callback(com.github.ambry.commons.Callback) RestUtils(com.github.ambry.rest.RestUtils) AsyncOperationTracker(com.github.ambry.utils.AsyncOperationTracker) RestRequest(com.github.ambry.rest.RestRequest) GetBlobOptionsBuilder(com.github.ambry.router.GetBlobOptionsBuilder) BlobId(com.github.ambry.commons.BlobId) RestRequestMetrics(com.github.ambry.rest.RestRequestMetrics) GetBlobOptions(com.github.ambry.router.GetBlobOptions)

Example 87 with RestRequest

use of com.github.ambry.rest.RestRequest in project ambry by linkedin.

the class FrontendRestRequestService method securityPostProcessRequestCallback.

/**
 * Build a callback to use for {@link SecurityService#postProcessRequest}. This callback forwards request to the
 * {@link Router} once ID conversion is completed. In the case of some sub-resources
 * (e.g., {@link SubResource#Replicas}), the request is completed and not forwarded to the {@link Router}.
 * @param convertedId the converted blob ID to use in router requests.
 * @param restRequest the {@link RestRequest}.
 * @param restResponseChannel the {@link RestResponseChannel}.
 * @param getCallback the {@link GetCallback} to use if this is a {@link RestMethod#GET} request, or null for other
 *                    request types.
 * @param headCallback the {@link HeadCallback} to use if this is a {@link RestMethod#HEAD} request, or null for other
 *                    request types.
 * @param deleteCallback the {@link DeleteCallback} to use if this is a {@link RestMethod#DELETE} request, or null for
 *                       other request types.
 * @return the {@link Callback} to use.
 */
private Callback<Void> securityPostProcessRequestCallback(String convertedId, RestRequest restRequest, RestResponseChannel restResponseChannel, GetCallback getCallback, HeadCallback headCallback, DeleteCallback deleteCallback) {
    Callback<ReadableStreamChannel> completionCallback = (result, exception) -> submitResponse(restRequest, restResponseChannel, result, exception);
    RestMethod restMethod = restRequest.getRestMethod();
    AsyncOperationTracker.Metrics metrics;
    switch(restMethod) {
        case GET:
            metrics = frontendMetrics.getSecurityPostProcessRequestMetrics;
            break;
        case HEAD:
            metrics = frontendMetrics.headSecurityPostProcessRequestMetrics;
            break;
        case DELETE:
            metrics = frontendMetrics.deleteSecurityPostProcessRequestMetrics;
            break;
        default:
            throw new IllegalStateException("Unrecognized RestMethod: " + restMethod);
    }
    return FrontendUtils.buildCallback(metrics, result -> {
        ReadableStreamChannel response = null;
        switch(restMethod) {
            case GET:
                SubResource subResource = getRequestPath(restRequest).getSubResource();
                // inject encryption metrics if need be
                if (BlobId.isEncrypted(convertedId)) {
                    RestRequestMetrics restRequestMetrics = getMetricsGroupForGet(frontendMetrics, subResource).getRestRequestMetrics(restRequest.isSslUsed(), true);
                    restRequest.getMetricsTracker().injectMetrics(restRequestMetrics);
                }
                if (subResource == null) {
                    getCallback.markStartTime();
                    router.getBlob(convertedId, getCallback.options, getCallback, QuotaUtils.buildQuotaChargeCallback(restRequest, quotaManager, true));
                } else {
                    switch(subResource) {
                        case BlobInfo:
                        case UserMetadata:
                        case Segment:
                            getCallback.markStartTime();
                            router.getBlob(convertedId, getCallback.options, getCallback, QuotaUtils.buildQuotaChargeCallback(restRequest, quotaManager, true));
                            break;
                        case Replicas:
                            response = getReplicasHandler.getReplicas(convertedId, restResponseChannel);
                            break;
                    }
                }
                break;
            case HEAD:
                GetOption getOption = getGetOption(restRequest, frontendConfig.defaultRouterGetOption);
                // inject encryption metrics if need be
                if (BlobId.isEncrypted(convertedId)) {
                    RestRequestMetrics requestMetrics = frontendMetrics.headBlobMetricsGroup.getRestRequestMetrics(restRequest.isSslUsed(), true);
                    restRequest.getMetricsTracker().injectMetrics(requestMetrics);
                }
                headCallback.markStartTime();
                router.getBlob(convertedId, new GetBlobOptionsBuilder().operationType(GetBlobOptions.OperationType.BlobInfo).getOption(getOption).restRequest(restRequest).build(), headCallback, QuotaUtils.buildQuotaChargeCallback(restRequest, quotaManager, false));
                break;
            case DELETE:
                deleteCallback.markStartTime();
                router.deleteBlob(convertedId, getHeader(restRequest.getArgs(), Headers.SERVICE_ID, false), deleteCallback, QuotaUtils.buildQuotaChargeCallback(restRequest, quotaManager, false));
                break;
            default:
                throw new IllegalStateException("Unrecognized RestMethod: " + restMethod);
        }
        if (response != null) {
            completionCallback.onCompletion(response, null);
        }
    }, restRequest.getUri(), logger, completionCallback);
}
Also used : Histogram(com.codahale.metrics.Histogram) GetOption(com.github.ambry.protocol.GetOption) FrontendConfig(com.github.ambry.config.FrontendConfig) ResponseStatus(com.github.ambry.rest.ResponseStatus) LoggerFactory(org.slf4j.LoggerFactory) ByteBufferReadableStreamChannel(com.github.ambry.commons.ByteBufferReadableStreamChannel) AccountService(com.github.ambry.account.AccountService) QuotaManager(com.github.ambry.quota.QuotaManager) ByteBuffer(java.nio.ByteBuffer) ThrowingConsumer(com.github.ambry.utils.ThrowingConsumer) RequestPath(com.github.ambry.rest.RequestPath) NamedBlobDb(com.github.ambry.named.NamedBlobDb) RestRequestService(com.github.ambry.rest.RestRequestService) RestRequestMetrics(com.github.ambry.rest.RestRequestMetrics) SystemTime(com.github.ambry.utils.SystemTime) Router(com.github.ambry.router.Router) RouterErrorCode(com.github.ambry.router.RouterErrorCode) RestResponseHandler(com.github.ambry.rest.RestResponseHandler) ReadableStreamChannel(com.github.ambry.router.ReadableStreamChannel) Logger(org.slf4j.Logger) GregorianCalendar(java.util.GregorianCalendar) RestMethod(com.github.ambry.rest.RestMethod) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) RestServiceErrorCode(com.github.ambry.rest.RestServiceErrorCode) InternalKeys(com.github.ambry.rest.RestUtils.InternalKeys) ClusterMap(com.github.ambry.clustermap.ClusterMap) Utils(com.github.ambry.utils.Utils) IOException(java.io.IOException) GetBlobOptions(com.github.ambry.router.GetBlobOptions) RouterException(com.github.ambry.router.RouterException) BlobInfo(com.github.ambry.messageformat.BlobInfo) QuotaUtils(com.github.ambry.quota.QuotaUtils) AccountStatsStore(com.github.ambry.accountstats.AccountStatsStore) RestServiceException(com.github.ambry.rest.RestServiceException) GetBlobResult(com.github.ambry.router.GetBlobResult) Callback(com.github.ambry.commons.Callback) RestUtils(com.github.ambry.rest.RestUtils) AsyncOperationTracker(com.github.ambry.utils.AsyncOperationTracker) RestRequest(com.github.ambry.rest.RestRequest) GetBlobOptionsBuilder(com.github.ambry.router.GetBlobOptionsBuilder) BlobId(com.github.ambry.commons.BlobId) GetBlobOptionsBuilder(com.github.ambry.router.GetBlobOptionsBuilder) ByteBufferReadableStreamChannel(com.github.ambry.commons.ByteBufferReadableStreamChannel) ReadableStreamChannel(com.github.ambry.router.ReadableStreamChannel) RestRequestMetrics(com.github.ambry.rest.RestRequestMetrics) GetOption(com.github.ambry.protocol.GetOption) AsyncOperationTracker(com.github.ambry.utils.AsyncOperationTracker) RestMethod(com.github.ambry.rest.RestMethod)

Example 88 with RestRequest

use of com.github.ambry.rest.RestRequest in project ambry by linkedin.

the class AmbryIdConverterFactoryTest method testConversionForNamedBlob.

/**
 * Tests the conversion by the {@code idConverter}.
 * @param idConverter the {@link IdConverter} instance to use.
 * @param restMethod the {@link RestMethod} of the {@link RestRequest} that will be created.
 * @param signedIdMetadata the headers of the {@link RestRequest}.
 * @param expectedOutput the expected output from the {@code idConverter}.
 * @param input the input string
 * @throws Exception
 */
private void testConversionForNamedBlob(IdConverter idConverter, RestMethod restMethod, Map<String, String> signedIdMetadata, String expectedOutput, String input) throws Exception {
    JSONObject requestData = new JSONObject();
    JSONObject headers = new JSONObject();
    String contentType = "application/octet-stream";
    headers.put(RestUtils.Headers.AMBRY_CONTENT_TYPE, contentType);
    requestData.put(MockRestRequest.REST_METHOD_KEY, restMethod.name());
    requestData.put(MockRestRequest.URI_KEY, NAMED_BLOB_PATH);
    requestData.put(MockRestRequest.HEADERS_KEY, headers);
    RestRequest restRequest = new MockRestRequest(requestData, null);
    if (signedIdMetadata != null) {
        restRequest.setArg(RestUtils.InternalKeys.SIGNED_ID_METADATA_KEY, signedIdMetadata);
    }
    BlobInfo blobInfo = null;
    if (restMethod.equals(RestMethod.PUT)) {
        restRequest.setArg(RestUtils.InternalKeys.REQUEST_PATH, RequestPath.parse(NAMED_BLOB_PATH, Collections.emptyMap(), Collections.emptyList(), "Ambry-test"));
        blobInfo = new BlobInfo(new BlobProperties(-1, "service", accountId, containerId, false), new byte[0]);
    }
    IdConversionCallback callback = new IdConversionCallback();
    assertEquals("Converted ID does not match expected (Future)", expectedOutput, idConverter.convert(restRequest, input, blobInfo, callback).get(5, TimeUnit.SECONDS));
    assertEquals("Converted ID does not match expected (Callback)", expectedOutput, callback.result);
}
Also used : MockRestRequest(com.github.ambry.rest.MockRestRequest) RestRequest(com.github.ambry.rest.RestRequest) JSONObject(org.json.JSONObject) BlobProperties(com.github.ambry.messageformat.BlobProperties) BlobInfo(com.github.ambry.messageformat.BlobInfo) MockRestRequest(com.github.ambry.rest.MockRestRequest)

Example 89 with RestRequest

use of com.github.ambry.rest.RestRequest in project ambry by linkedin.

the class AmbrySecurityServiceTest method testPostBlob.

/**
 * Tests {@link AmbrySecurityService#processResponse(RestRequest, RestResponseChannel, BlobInfo, Callback)} for
 * {@link RestMethod#POST}.
 * @throws Exception
 */
private void testPostBlob() throws Exception {
    MockRestResponseChannel restResponseChannel = new MockRestResponseChannel();
    RestRequest restRequest = createRestRequest(RestMethod.POST, "/", null);
    securityService.processResponse(restRequest, restResponseChannel, DEFAULT_INFO).get();
    Assert.assertEquals("ProcessResponse status should have been set", ResponseStatus.Created, restResponseChannel.getStatus());
    Assert.assertNotNull("Date has not been set", restResponseChannel.getHeader(RestUtils.Headers.DATE));
    Assert.assertEquals("Creation time should have been set correctly", RestUtils.toSecondsPrecisionInMs(DEFAULT_INFO.getBlobProperties().getCreationTimeInMs()), RestUtils.getTimeFromDateString(restResponseChannel.getHeader(RestUtils.Headers.CREATION_TIME)).longValue());
    Assert.assertEquals("Content-Length should have been 0", "0", restResponseChannel.getHeader(RestUtils.Headers.CONTENT_LENGTH));
}
Also used : MockRestRequest(com.github.ambry.rest.MockRestRequest) RestRequest(com.github.ambry.rest.RestRequest) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel)

Example 90 with RestRequest

use of com.github.ambry.rest.RestRequest in project ambry by linkedin.

the class AmbrySecurityServiceTest method testHeadBlob.

/**
 * Tests {@link AmbrySecurityService#processResponse(RestRequest, RestResponseChannel, BlobInfo, Callback)} for
 * {@link RestMethod#HEAD}.
 * @param blobInfo the {@link BlobInfo} of the blob for which {@link RestMethod#HEAD} is required.
 * @param range the {@link ByteRange} used for a range request, or {@code null} for non-ranged requests.
 * @throws Exception
 */
private void testHeadBlob(BlobInfo blobInfo, ByteRange range) throws Exception {
    MockRestResponseChannel restResponseChannel = new MockRestResponseChannel();
    JSONObject headers = range != null ? new JSONObject().put(RestUtils.Headers.RANGE, RestTestUtils.getRangeHeaderString(range)) : null;
    RestRequest restRequest = createRestRequest(RestMethod.HEAD, "/", headers);
    Pair<Account, Container> accountAndContainer = getAccountAndContainer(blobInfo.getBlobProperties());
    insertAccountAndContainer(restRequest, accountAndContainer.getFirst(), accountAndContainer.getSecond());
    securityService.processResponse(restRequest, restResponseChannel, blobInfo).get();
    Assert.assertEquals("ProcessResponse status should have been set", range == null ? ResponseStatus.Ok : ResponseStatus.PartialContent, restResponseChannel.getStatus());
    verifyHeadersForHead(blobInfo.getBlobProperties(), range, restResponseChannel);
    verifyAccountAndContainerHeaders(restResponseChannel, accountAndContainer.getFirst(), accountAndContainer.getSecond());
    Assert.assertEquals("LifeVersion mismatch", Short.toString(blobInfo.getLifeVersion()), restResponseChannel.getHeader(RestUtils.Headers.LIFE_VERSION));
}
Also used : Account(com.github.ambry.account.Account) Container(com.github.ambry.account.Container) MockRestRequest(com.github.ambry.rest.MockRestRequest) RestRequest(com.github.ambry.rest.RestRequest) JSONObject(org.json.JSONObject) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel)

Aggregations

RestRequest (com.github.ambry.rest.RestRequest)102 MockRestRequest (com.github.ambry.rest.MockRestRequest)82 MockRestResponseChannel (com.github.ambry.rest.MockRestResponseChannel)54 JSONObject (org.json.JSONObject)50 Test (org.junit.Test)46 RestServiceException (com.github.ambry.rest.RestServiceException)34 RestResponseChannel (com.github.ambry.rest.RestResponseChannel)26 RestMethod (com.github.ambry.rest.RestMethod)23 Account (com.github.ambry.account.Account)18 StorageStatsUtilTest (com.github.ambry.server.StorageStatsUtilTest)18 ExecutionException (java.util.concurrent.ExecutionException)18 ByteBuffer (java.nio.ByteBuffer)17 RestUtils (com.github.ambry.rest.RestUtils)16 RestUtilsTest (com.github.ambry.rest.RestUtilsTest)16 ReadableStreamChannel (com.github.ambry.router.ReadableStreamChannel)16 RestServiceErrorCode (com.github.ambry.rest.RestServiceErrorCode)15 MetricRegistry (com.codahale.metrics.MetricRegistry)14 Container (com.github.ambry.account.Container)14 RequestPath (com.github.ambry.rest.RequestPath)14 FutureResult (com.github.ambry.router.FutureResult)14