Search in sources :

Example 1 with RestResponseChannel

use of com.github.ambry.rest.RestResponseChannel in project ambry by linkedin.

the class FrontendRestRequestService method handlePut.

@Override
public void handlePut(RestRequest restRequest, RestResponseChannel restResponseChannel) {
    ThrowingConsumer<RequestPath> routingAction = requestPath -> {
        if (requestPath.matchesOperation(Operations.UPDATE_TTL)) {
            ttlUpdateHandler.handle(restRequest, restResponseChannel, (r, e) -> {
                if (e instanceof RouterException && ((RouterException) e).getErrorCode() == RouterErrorCode.BlobUpdateNotAllowed) {
                    restResponseChannel.setHeader(Headers.ALLOW, TTL_UPDATE_REJECTED_ALLOW_HEADER_VALUE);
                }
                submitResponse(restRequest, restResponseChannel, null, e);
            });
        } else if (requestPath.matchesOperation(Operations.UNDELETE) && frontendConfig.enableUndelete) {
            // If the undelete is not enabled, then treat it as unrecognized operation.
            // And always send failure reason back to client for undelete
            restRequest.setArg(SEND_FAILURE_REASON, Boolean.TRUE);
            undeleteHandler.handle(restRequest, restResponseChannel, (r, e) -> {
                submitResponse(restRequest, restResponseChannel, null, e);
            });
        } else if (requestPath.matchesOperation(Operations.NAMED_BLOB)) {
            restRequest.setArg(SEND_FAILURE_REASON, Boolean.TRUE);
            namedBlobPutHandler.handle(restRequest, restResponseChannel, (r, e) -> submitResponse(restRequest, restResponseChannel, null, e));
        } else {
            throw new RestServiceException("Unrecognized operation: " + requestPath.getOperationOrBlobId(false), RestServiceErrorCode.BadRequest);
        }
    };
    preProcessAndRouteRequest(restRequest, restResponseChannel, frontendMetrics.putPreProcessingMetrics, routingAction);
}
Also used : RequestPath(com.github.ambry.rest.RequestPath) Histogram(com.codahale.metrics.Histogram) GetOption(com.github.ambry.protocol.GetOption) FrontendConfig(com.github.ambry.config.FrontendConfig) ResponseStatus(com.github.ambry.rest.ResponseStatus) LoggerFactory(org.slf4j.LoggerFactory) ByteBufferReadableStreamChannel(com.github.ambry.commons.ByteBufferReadableStreamChannel) AccountService(com.github.ambry.account.AccountService) QuotaManager(com.github.ambry.quota.QuotaManager) ByteBuffer(java.nio.ByteBuffer) ThrowingConsumer(com.github.ambry.utils.ThrowingConsumer) RequestPath(com.github.ambry.rest.RequestPath) NamedBlobDb(com.github.ambry.named.NamedBlobDb) RestRequestService(com.github.ambry.rest.RestRequestService) RestRequestMetrics(com.github.ambry.rest.RestRequestMetrics) SystemTime(com.github.ambry.utils.SystemTime) Router(com.github.ambry.router.Router) RouterErrorCode(com.github.ambry.router.RouterErrorCode) RestResponseHandler(com.github.ambry.rest.RestResponseHandler) ReadableStreamChannel(com.github.ambry.router.ReadableStreamChannel) Logger(org.slf4j.Logger) GregorianCalendar(java.util.GregorianCalendar) RestMethod(com.github.ambry.rest.RestMethod) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) RestServiceErrorCode(com.github.ambry.rest.RestServiceErrorCode) InternalKeys(com.github.ambry.rest.RestUtils.InternalKeys) ClusterMap(com.github.ambry.clustermap.ClusterMap) Utils(com.github.ambry.utils.Utils) IOException(java.io.IOException) GetBlobOptions(com.github.ambry.router.GetBlobOptions) RouterException(com.github.ambry.router.RouterException) BlobInfo(com.github.ambry.messageformat.BlobInfo) QuotaUtils(com.github.ambry.quota.QuotaUtils) AccountStatsStore(com.github.ambry.accountstats.AccountStatsStore) RestServiceException(com.github.ambry.rest.RestServiceException) GetBlobResult(com.github.ambry.router.GetBlobResult) Callback(com.github.ambry.commons.Callback) RestUtils(com.github.ambry.rest.RestUtils) AsyncOperationTracker(com.github.ambry.utils.AsyncOperationTracker) RestRequest(com.github.ambry.rest.RestRequest) GetBlobOptionsBuilder(com.github.ambry.router.GetBlobOptionsBuilder) BlobId(com.github.ambry.commons.BlobId) RestServiceException(com.github.ambry.rest.RestServiceException) RouterException(com.github.ambry.router.RouterException)

Example 2 with RestResponseChannel

use of com.github.ambry.rest.RestResponseChannel in project ambry by linkedin.

the class PostAccountContainersHandlerTest method validRequestsTest.

/**
 * Test valid request cases.
 * @throws Exception
 */
@Test
public void validRequestsTest() throws Exception {
    String accountName = theAccount.getName();
    short accountId = theAccount.getId();
    ThrowingConsumer<Collection<Container>> testAction = inputContainers -> {
        String requestBody = new String(AccountCollectionSerde.serializeContainersInJson(inputContainers));
        RestResponseChannel restResponseChannel = new MockRestResponseChannel();
        RestRequest request = createRestRequest(requestBody, accountName, null);
        ReadableStreamChannel responseChannel = sendRequestGetResponse(request, restResponseChannel);
        assertNotNull("Date has not been set", restResponseChannel.getHeader(RestUtils.Headers.DATE));
        assertEquals("Content-length is not as expected", responseChannel.getSize(), Integer.parseInt((String) restResponseChannel.getHeader(RestUtils.Headers.CONTENT_LENGTH)));
        assertEquals("Account id in response header is not as expected", accountId, Short.parseShort((String) restResponseChannel.getHeader(RestUtils.Headers.TARGET_ACCOUNT_ID)));
        RetainingAsyncWritableChannel asyncWritableChannel = new RetainingAsyncWritableChannel((int) responseChannel.getSize());
        responseChannel.readInto(asyncWritableChannel, null).get();
        Collection<Container> outputContainers = AccountCollectionSerde.containersFromInputStreamInJson(asyncWritableChannel.consumeContentAsInputStream(), accountId);
        assertEquals("Unexpected count returned", inputContainers.size(), outputContainers.size());
        for (Container container : outputContainers) {
            assertEquals("Container in account service not as expected", container, accountService.getContainerByName(accountName, container.getName()));
        }
    };
    // add new container
    testAction.accept(Collections.singleton(accountService.getRandomContainer(accountId)));
    // add multiple containers
    List<Container> containerList = new ArrayList<>();
    for (int j = 0; j < 10; j++) {
        containerList.add(new ContainerBuilder(Container.UNKNOWN_CONTAINER_ID, "Test-" + j, Container.ContainerStatus.ACTIVE, "", accountId).build());
    }
    testAction.accept(containerList);
// TODO: update existing containers when support is added
}
Also used : MockRestRequest(com.github.ambry.rest.MockRestRequest) FrontendConfig(com.github.ambry.config.FrontendConfig) FutureResult(com.github.ambry.router.FutureResult) ContainerBuilder(com.github.ambry.account.ContainerBuilder) ByteBuffer(java.nio.ByteBuffer) ThrowingConsumer(com.github.ambry.utils.ThrowingConsumer) ArrayList(java.util.ArrayList) AccountCollectionSerde(com.github.ambry.account.AccountCollectionSerde) RequestPath(com.github.ambry.rest.RequestPath) JSONObject(org.json.JSONObject) TestUtils(com.github.ambry.utils.TestUtils) LinkedList(java.util.LinkedList) RetainingAsyncWritableChannel(com.github.ambry.commons.RetainingAsyncWritableChannel) Container(com.github.ambry.account.Container) ReadableStreamChannel(com.github.ambry.router.ReadableStreamChannel) MetricRegistry(com.codahale.metrics.MetricRegistry) Properties(java.util.Properties) RestMethod(com.github.ambry.rest.RestMethod) VerifiableProperties(com.github.ambry.config.VerifiableProperties) Collection(java.util.Collection) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) RestServiceErrorCode(com.github.ambry.rest.RestServiceErrorCode) Test(org.junit.Test) ThrowingBiConsumer(com.github.ambry.utils.ThrowingBiConsumer) StandardCharsets(java.nio.charset.StandardCharsets) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) RestServiceException(com.github.ambry.rest.RestServiceException) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) Account(com.github.ambry.account.Account) RestUtils(com.github.ambry.rest.RestUtils) Assert(org.junit.Assert) RestRequest(com.github.ambry.rest.RestRequest) Collections(java.util.Collections) InMemAccountService(com.github.ambry.account.InMemAccountService) RetainingAsyncWritableChannel(com.github.ambry.commons.RetainingAsyncWritableChannel) ArrayList(java.util.ArrayList) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) Container(com.github.ambry.account.Container) MockRestRequest(com.github.ambry.rest.MockRestRequest) RestRequest(com.github.ambry.rest.RestRequest) ContainerBuilder(com.github.ambry.account.ContainerBuilder) ReadableStreamChannel(com.github.ambry.router.ReadableStreamChannel) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) Collection(java.util.Collection) Test(org.junit.Test)

Example 3 with RestResponseChannel

use of com.github.ambry.rest.RestResponseChannel in project ambry by linkedin.

the class PostBlobHandlerTest method doChunkUploadTest.

// chunkUploadTest() helpers
/**
 * Make a post request and verify behavior related to chunk uploads (for stitched blobs).
 * @param contentLength the size of the blob to upload.
 * @param chunkUpload {@code true} to send the "x-ambry-chunk-upload" header, or {@code false} to test full uploads.
 * @param uploadSession the value for the "x-ambry-chunk-upload-session" request header, or null to not set it.
 * @param maxUploadSize the value for the "x-ambry-max-upload-size" request header, or null to not set it.
 * @param blobTtlSecs the blob TTL to use.
 * @param errorChecker if non-null, expect an exception to be thrown by the post flow and verify it using this
 *                     {@link ThrowingConsumer}.
 * @throws Exception
 */
private void doChunkUploadTest(int contentLength, boolean chunkUpload, String uploadSession, Integer maxUploadSize, long blobTtlSecs, ThrowingConsumer<ExecutionException> errorChecker) throws Exception {
    JSONObject headers = new JSONObject();
    FrontendRestRequestServiceTest.setAmbryHeadersForPut(headers, blobTtlSecs, !REF_CONTAINER.isCacheable(), SERVICE_ID, CONTENT_TYPE, OWNER_ID, REF_ACCOUNT.getName(), REF_CONTAINER.getName(), null);
    if (chunkUpload) {
        headers.put(RestUtils.Headers.CHUNK_UPLOAD, true);
    }
    if (uploadSession != null) {
        headers.put(RestUtils.Headers.SESSION, uploadSession);
    }
    if (maxUploadSize != null) {
        headers.put(RestUtils.Headers.MAX_UPLOAD_SIZE, maxUploadSize);
    }
    byte[] content = TestUtils.getRandomBytes(contentLength);
    RestRequest request = getRestRequest(headers, "/", content);
    long creationTimeMs = System.currentTimeMillis();
    time.setCurrentMilliseconds(creationTimeMs);
    RestResponseChannel restResponseChannel = new MockRestResponseChannel();
    FutureResult<Void> future = new FutureResult<>();
    idConverterFactory.lastInput = null;
    postBlobHandler.handle(request, restResponseChannel, future::done);
    if (errorChecker == null) {
        future.get(TIMEOUT_SECS, TimeUnit.SECONDS);
        assertEquals("Unexpected converted ID", CONVERTED_ID, restResponseChannel.getHeader(RestUtils.Headers.LOCATION));
        Object metadata = request.getArgs().get(RestUtils.InternalKeys.SIGNED_ID_METADATA_KEY);
        if (chunkUpload) {
            Map<String, String> expectedMetadata = new HashMap<>(3);
            expectedMetadata.put(RestUtils.Headers.BLOB_SIZE, Integer.toString(contentLength));
            expectedMetadata.put(RestUtils.Headers.SESSION, uploadSession);
            expectedMetadata.put(PostBlobHandler.EXPIRATION_TIME_MS_KEY, Long.toString(Utils.addSecondsToEpochTime(creationTimeMs, blobTtlSecs)));
            assertEquals("Unexpected signed ID metadata", expectedMetadata, metadata);
        } else {
            assertNull("Signed id metadata should not be set on non-chunk uploads", metadata);
        }
        InMemoryRouter.InMemoryBlob blob = router.getActiveBlobs().get(idConverterFactory.lastInput);
        assertEquals("Unexpected blob content stored", ByteBuffer.wrap(content), blob.getBlob());
        assertEquals("Unexpected ttl stored", blobTtlSecs, blob.getBlobProperties().getTimeToLiveInSeconds());
        // check that blob size matches the actual upload size
        assertEquals("Invalid blob size", Integer.toString(contentLength), restResponseChannel.getHeader(RestUtils.Headers.BLOB_SIZE));
    } else {
        TestUtils.assertException(ExecutionException.class, () -> future.get(TIMEOUT_SECS, TimeUnit.SECONDS), errorChecker);
    }
}
Also used : HashMap(java.util.HashMap) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) InMemoryRouter(com.github.ambry.router.InMemoryRouter) RestRequest(com.github.ambry.rest.RestRequest) JSONObject(org.json.JSONObject) FutureResult(com.github.ambry.router.FutureResult) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) JSONObject(org.json.JSONObject)

Example 4 with RestResponseChannel

use of com.github.ambry.rest.RestResponseChannel in project ambry by linkedin.

the class PostBlobHandlerTest method stitchBlobAndVerify.

/**
 * Make a stitch blob call using {@link PostBlobHandler} and verify the result of the operation.
 * @param requestBody the body of the stitch request to supply.
 * @param expectedStitchedChunks the expected chunks stitched together.
 * @param errorChecker if non-null, expect an exception to be thrown by the post flow and verify it using this
 *                     {@link ThrowingConsumer}.
 * @throws Exception
 */
private void stitchBlobAndVerify(byte[] requestBody, List<ChunkInfo> expectedStitchedChunks, ThrowingConsumer<ExecutionException> errorChecker) throws Exception {
    JSONObject headers = new JSONObject();
    FrontendRestRequestServiceTest.setAmbryHeadersForPut(headers, TestUtils.TTL_SECS, !REF_CONTAINER.isCacheable(), SERVICE_ID, CONTENT_TYPE, OWNER_ID, REF_ACCOUNT.getName(), REF_CONTAINER.getName(), null);
    RestRequest request = getRestRequest(headers, "/" + Operations.STITCH, requestBody);
    RestResponseChannel restResponseChannel = new MockRestResponseChannel();
    FutureResult<Void> future = new FutureResult<>();
    idConverterFactory.lastInput = null;
    postBlobHandler.handle(request, restResponseChannel, future::done);
    if (errorChecker == null) {
        future.get(TIMEOUT_SECS, TimeUnit.SECONDS);
        assertEquals("Unexpected converted ID", CONVERTED_ID, restResponseChannel.getHeader(RestUtils.Headers.LOCATION));
        InMemoryRouter.InMemoryBlob blob = router.getActiveBlobs().get(idConverterFactory.lastInput);
        assertEquals("List of chunks stitched does not match expected", expectedStitchedChunks, blob.getStitchedChunks());
        ByteArrayOutputStream expectedContent = new ByteArrayOutputStream();
        expectedStitchedChunks.stream().map(chunkInfo -> router.getActiveBlobs().get(chunkInfo.getBlobId()).getBlob().array()).forEach(buf -> expectedContent.write(buf, 0, buf.length));
        assertEquals("Unexpected blob content stored", ByteBuffer.wrap(expectedContent.toByteArray()), blob.getBlob());
        // check actual size of stitched blob
        assertEquals("Unexpected blob size", Long.toString(getStitchedBlobSize(expectedStitchedChunks)), restResponseChannel.getHeader(RestUtils.Headers.BLOB_SIZE));
    } else {
        TestUtils.assertException(ExecutionException.class, () -> future.get(TIMEOUT_SECS, TimeUnit.SECONDS), errorChecker);
    }
}
Also used : Arrays(java.util.Arrays) FrontendConfig(com.github.ambry.config.FrontendConfig) BlobProperties(com.github.ambry.messageformat.BlobProperties) URISyntaxException(java.net.URISyntaxException) FutureResult(com.github.ambry.router.FutureResult) ByteBufferReadableStreamChannel(com.github.ambry.commons.ByteBufferReadableStreamChannel) ContainerBuilder(com.github.ambry.account.ContainerBuilder) ByteBuffer(java.nio.ByteBuffer) ThrowingConsumer(com.github.ambry.utils.ThrowingConsumer) JSONObject(org.json.JSONObject) TestUtils(com.github.ambry.utils.TestUtils) Map(java.util.Map) ChunkInfo(com.github.ambry.router.ChunkInfo) RouterErrorCode(com.github.ambry.router.RouterErrorCode) Container(com.github.ambry.account.Container) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) RestServiceErrorCode(com.github.ambry.rest.RestServiceErrorCode) Utils(com.github.ambry.utils.Utils) UUID(java.util.UUID) AccountBuilder(com.github.ambry.account.AccountBuilder) Collectors(java.util.stream.Collectors) StandardCharsets(java.nio.charset.StandardCharsets) List(java.util.List) RestServiceException(com.github.ambry.rest.RestServiceException) Stream(java.util.stream.Stream) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) MockTime(com.github.ambry.utils.MockTime) QuotaTestUtils(com.github.ambry.quota.QuotaTestUtils) Account(com.github.ambry.account.Account) RestUtils(com.github.ambry.rest.RestUtils) UnsupportedEncodingException(java.io.UnsupportedEncodingException) InMemAccountService(com.github.ambry.account.InMemAccountService) InMemoryRouter(com.github.ambry.router.InMemoryRouter) ByteArrayOutputStream(java.io.ByteArrayOutputStream) AccountService(com.github.ambry.account.AccountService) HashMap(java.util.HashMap) QuotaManager(com.github.ambry.quota.QuotaManager) ArrayList(java.util.ArrayList) QuotaConfig(com.github.ambry.config.QuotaConfig) RequestPath(com.github.ambry.rest.RequestPath) QuotaMode(com.github.ambry.quota.QuotaMode) LinkedList(java.util.LinkedList) MetricRegistry(com.codahale.metrics.MetricRegistry) Properties(java.util.Properties) RestMethod(com.github.ambry.rest.RestMethod) VerifiableProperties(com.github.ambry.config.VerifiableProperties) ClusterMap(com.github.ambry.clustermap.ClusterMap) Test(org.junit.Test) RouterException(com.github.ambry.router.RouterException) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) AmbryQuotaManager(com.github.ambry.quota.AmbryQuotaManager) Mockito(org.mockito.Mockito) QuotaMetrics(com.github.ambry.quota.QuotaMetrics) PutBlobOptionsBuilder(com.github.ambry.router.PutBlobOptionsBuilder) SimpleQuotaRecommendationMergePolicy(com.github.ambry.quota.SimpleQuotaRecommendationMergePolicy) Assert(org.junit.Assert) RestRequest(com.github.ambry.rest.RestRequest) Collections(java.util.Collections) MockClusterMap(com.github.ambry.clustermap.MockClusterMap) InMemoryRouter(com.github.ambry.router.InMemoryRouter) RestRequest(com.github.ambry.rest.RestRequest) JSONObject(org.json.JSONObject) FutureResult(com.github.ambry.router.FutureResult) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) ByteArrayOutputStream(java.io.ByteArrayOutputStream) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel)

Example 5 with RestResponseChannel

use of com.github.ambry.rest.RestResponseChannel in project ambry by linkedin.

the class FrontendTestUrlSigningServiceFactory method doRuntimeExceptionRouterTest.

// runtimeExceptionRouterTest() helpers
/**
 * Tests reactions of various methods of {@link FrontendRestRequestService} to a {@link Router} that throws
 * {@link RuntimeException}.
 * @param restMethod used to determine the method to invoke in {@link FrontendRestRequestService}.
 * @throws Exception
 */
private void doRuntimeExceptionRouterTest(RestMethod restMethod) throws Exception {
    RestRequest restRequest = createRestRequest(restMethod, referenceBlobIdStr, null, null);
    RestResponseChannel restResponseChannel = new MockRestResponseChannel();
    try {
        switch(restMethod) {
            case GET:
            case DELETE:
            case HEAD:
                doOperation(restRequest, restResponseChannel);
                fail(restMethod + " should have detected a RestServiceException because of a bad router");
                break;
            case POST:
                JSONObject headers = new JSONObject();
                setAmbryHeadersForPut(headers, Utils.Infinite_Time, !refContainer.isCacheable(), "test-serviceID", "text/plain", "test-ownerId", refAccount.getName(), refContainer.getName(), null);
                restRequest = createRestRequest(restMethod, "/", headers, null);
                doOperation(restRequest, restResponseChannel);
                fail("POST should have detected a RestServiceException because of a bad router");
                break;
            default:
                throw new IllegalArgumentException("Unrecognized RestMethod: " + restMethod);
        }
    } catch (RuntimeException e) {
        assertEquals("Unexpected error message", InMemoryRouter.OPERATION_THROW_EARLY_RUNTIME_EXCEPTION, e.getMessage());
    }
}
Also used : MockRestRequest(com.github.ambry.rest.MockRestRequest) RestRequest(com.github.ambry.rest.RestRequest) JSONObject(org.json.JSONObject) RestResponseChannel(com.github.ambry.rest.RestResponseChannel) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel) MockRestResponseChannel(com.github.ambry.rest.MockRestResponseChannel)

Aggregations

RestResponseChannel (com.github.ambry.rest.RestResponseChannel)26 RestRequest (com.github.ambry.rest.RestRequest)23 MockRestResponseChannel (com.github.ambry.rest.MockRestResponseChannel)21 JSONObject (org.json.JSONObject)15 RestServiceException (com.github.ambry.rest.RestServiceException)14 Test (org.junit.Test)13 MockRestRequest (com.github.ambry.rest.MockRestRequest)12 RestMethod (com.github.ambry.rest.RestMethod)12 RequestPath (com.github.ambry.rest.RequestPath)11 RestServiceErrorCode (com.github.ambry.rest.RestServiceErrorCode)11 RestUtils (com.github.ambry.rest.RestUtils)11 ThrowingConsumer (com.github.ambry.utils.ThrowingConsumer)11 ExecutionException (java.util.concurrent.ExecutionException)11 FutureResult (com.github.ambry.router.FutureResult)10 ReadableStreamChannel (com.github.ambry.router.ReadableStreamChannel)10 ByteBuffer (java.nio.ByteBuffer)10 FrontendConfig (com.github.ambry.config.FrontendConfig)9 MetricRegistry (com.codahale.metrics.MetricRegistry)8 Account (com.github.ambry.account.Account)8 InMemAccountService (com.github.ambry.account.InMemAccountService)8