Search in sources :

Example 1 with EmbeddedFormat

use of io.confluent.kafkarest.entities.EmbeddedFormat in project kafka-rest by confluentinc.

the class ProduceAction method produce.

private CompletableFuture<ProduceResponse> produce(String clusterId, String topicName, ProduceRequest request, ProduceController controller) {
    try {
        produceRateLimiters.rateLimit(clusterId, request.getOriginalSize());
    } catch (RateLimitExceededException e) {
        // KREST-4356 Use our own CompletionException that will avoid the costly stack trace fill.
        throw new StacklessCompletionException(e);
    }
    Instant requestInstant = Instant.now();
    Optional<RegisteredSchema> keySchema = request.getKey().flatMap(key -> getSchema(topicName, /* isKey= */
    true, key));
    Optional<EmbeddedFormat> keyFormat = keySchema.map(schema -> Optional.of(schema.getFormat())).orElse(request.getKey().flatMap(ProduceRequestData::getFormat));
    Optional<ByteString> serializedKey = serialize(topicName, keyFormat, keySchema, request.getKey(), /* isKey= */
    true);
    Optional<RegisteredSchema> valueSchema = request.getValue().flatMap(value -> getSchema(topicName, /* isKey= */
    false, value));
    Optional<EmbeddedFormat> valueFormat = valueSchema.map(schema -> Optional.of(schema.getFormat())).orElse(request.getValue().flatMap(ProduceRequestData::getFormat));
    Optional<ByteString> serializedValue = serialize(topicName, valueFormat, valueSchema, request.getValue(), /* isKey= */
    false);
    recordRequestMetrics(request.getOriginalSize());
    CompletableFuture<ProduceResult> produceResult = controller.produce(clusterId, topicName, request.getPartitionId(), request.getHeaders().stream().collect(PRODUCE_REQUEST_HEADER_COLLECTOR), serializedKey, serializedValue, request.getTimestamp().orElse(Instant.now()));
    return produceResult.handleAsync((result, error) -> {
        if (error != null) {
            long latency = Duration.between(requestInstant, Instant.now()).toMillis();
            recordErrorMetrics(latency);
            throw new StacklessCompletionException(error);
        }
        return result;
    }, executorService).thenApplyAsync(result -> {
        ProduceResponse response = toProduceResponse(clusterId, topicName, keyFormat, keySchema, valueFormat, valueSchema, result);
        long latency = Duration.between(requestInstant, result.getCompletionTimestamp()).toMillis();
        recordResponseMetrics(latency);
        return response;
    }, executorService);
}
Also used : PathParam(javax.ws.rs.PathParam) Provider(javax.inject.Provider) Produces(javax.ws.rs.Produces) ProduceRequest(io.confluent.kafkarest.entities.v3.ProduceRequest) SerializationException(org.apache.kafka.common.errors.SerializationException) MappingIterator(com.fasterxml.jackson.databind.MappingIterator) StacklessCompletionException(io.confluent.kafkarest.exceptions.StacklessCompletionException) Path(javax.ws.rs.Path) LoggerFactory(org.slf4j.LoggerFactory) CompletableFuture(java.util.concurrent.CompletableFuture) PerformanceMetric(io.confluent.rest.annotations.PerformanceMetric) ProduceRequestHeader(io.confluent.kafkarest.entities.v3.ProduceRequest.ProduceRequestHeader) Function(java.util.function.Function) Inject(javax.inject.Inject) NullNode(com.fasterxml.jackson.databind.node.NullNode) RateLimitExceededException(io.confluent.kafkarest.ratelimit.RateLimitExceededException) MediaType(javax.ws.rs.core.MediaType) Consumes(javax.ws.rs.Consumes) ProduceRequestData(io.confluent.kafkarest.entities.v3.ProduceRequest.ProduceRequestData) Objects.requireNonNull(java.util.Objects.requireNonNull) Duration(java.time.Duration) ProduceResponseData(io.confluent.kafkarest.entities.v3.ProduceResponse.ProduceResponseData) Collector(java.util.stream.Collector) ImmutableMultimap(com.google.common.collect.ImmutableMultimap) RegisteredSchema(io.confluent.kafkarest.entities.RegisteredSchema) BadRequestException(io.confluent.kafkarest.exceptions.BadRequestException) ProducerMetricsRegistry(io.confluent.kafkarest.ProducerMetricsRegistry) ExecutorService(java.util.concurrent.ExecutorService) EmbeddedFormat(io.confluent.kafkarest.entities.EmbeddedFormat) SchemaManager(io.confluent.kafkarest.controllers.SchemaManager) POST(javax.ws.rs.POST) Logger(org.slf4j.Logger) ProduceController(io.confluent.kafkarest.controllers.ProduceController) ProduceResult(io.confluent.kafkarest.entities.ProduceResult) AsyncResponse(javax.ws.rs.container.AsyncResponse) DoNotRateLimit(io.confluent.kafkarest.ratelimit.DoNotRateLimit) ResourceName(io.confluent.kafkarest.extension.ResourceAccesslistFeature.ResourceName) Instant(java.time.Instant) Suspended(javax.ws.rs.container.Suspended) ByteString(com.google.protobuf.ByteString) ProducerMetrics(io.confluent.kafkarest.ProducerMetrics) StreamingResponseFactory(io.confluent.kafkarest.response.StreamingResponseFactory) Errors(io.confluent.kafkarest.Errors) ProduceResponse(io.confluent.kafkarest.entities.v3.ProduceResponse) Optional(java.util.Optional) RecordSerializer(io.confluent.kafkarest.controllers.RecordSerializer) ProduceResponseThreadPool(io.confluent.kafkarest.resources.v3.V3ResourcesModule.ProduceResponseThreadPool) Collections(java.util.Collections) EmbeddedFormat(io.confluent.kafkarest.entities.EmbeddedFormat) ByteString(com.google.protobuf.ByteString) ProduceResponse(io.confluent.kafkarest.entities.v3.ProduceResponse) Instant(java.time.Instant) RateLimitExceededException(io.confluent.kafkarest.ratelimit.RateLimitExceededException) RegisteredSchema(io.confluent.kafkarest.entities.RegisteredSchema) ProduceResult(io.confluent.kafkarest.entities.ProduceResult) StacklessCompletionException(io.confluent.kafkarest.exceptions.StacklessCompletionException)

Example 2 with EmbeddedFormat

use of io.confluent.kafkarest.entities.EmbeddedFormat in project kafka-rest by confluentinc.

the class SchemaManagerImplTest method errorRawSchemaNotSupportedWithSchema.

@Test
public void errorRawSchemaNotSupportedWithSchema() {
    EmbeddedFormat embeddedFormatMock = mock(EmbeddedFormat.class);
    SchemaProvider schemaProviderMock = mock(SchemaProvider.class);
    expect(embeddedFormatMock.requiresSchema()).andReturn(true);
    expect(embeddedFormatMock.getSchemaProvider()).andThrow(new UnsupportedOperationException("Reason here"));
    expect(schemaProviderMock.parseSchema(TextNode.valueOf("rawSchema").toString(), emptyList(), true)).andReturn(Optional.empty());
    replay(embeddedFormatMock, schemaProviderMock);
    BadRequestException bre = assertThrows(BadRequestException.class, () -> schemaManager.getSchema(TOPIC_NAME, /* format= */
    Optional.of(embeddedFormatMock), /* subject= */
    Optional.empty(), /* subjectNameStrategy= */
    Optional.empty(), /* schemaId= */
    Optional.empty(), /* schemaVersion= */
    Optional.empty(), /* rawSchema= */
    Optional.of(TextNode.valueOf("rawSchema").toString()), /* isKey= */
    true));
    assertEquals("Raw schema not supported with format = EasyMock for class io.confluent.kafkarest.entities.EmbeddedFormat", bre.getMessage());
    assertEquals(400, bre.getCode());
}
Also used : EmbeddedFormat(io.confluent.kafkarest.entities.EmbeddedFormat) ProtobufSchemaProvider(io.confluent.kafka.schemaregistry.protobuf.ProtobufSchemaProvider) SchemaProvider(io.confluent.kafka.schemaregistry.SchemaProvider) JsonSchemaProvider(io.confluent.kafka.schemaregistry.json.JsonSchemaProvider) AvroSchemaProvider(io.confluent.kafka.schemaregistry.avro.AvroSchemaProvider) BadRequestException(io.confluent.kafkarest.exceptions.BadRequestException) Test(org.junit.jupiter.api.Test)

Example 3 with EmbeddedFormat

use of io.confluent.kafkarest.entities.EmbeddedFormat in project kafka-rest by confluentinc.

the class SchemaManagerImplTest method errorRegisteringSchema.

@Test
public void errorRegisteringSchema() throws RestClientException, IOException {
    SchemaRegistryClient schemaRegistryClientMock = mock(SchemaRegistryClient.class);
    ParsedSchema parsedSchemaMock = mock(ParsedSchema.class);
    EmbeddedFormat embeddedFormatMock = mock(EmbeddedFormat.class);
    SchemaProvider schemaProviderMock = mock(SchemaProvider.class);
    expect(embeddedFormatMock.requiresSchema()).andReturn(true);
    expect(embeddedFormatMock.getSchemaProvider()).andReturn(schemaProviderMock);
    expect(schemaProviderMock.parseSchema(TextNode.valueOf("rawString").toString(), emptyList(), true)).andReturn(Optional.of(parsedSchemaMock));
    expect(schemaRegistryClientMock.getId("subject1", parsedSchemaMock)).andThrow(new IOException("Can't get Schema"));
    expect(schemaRegistryClientMock.register("subject1", parsedSchemaMock)).andThrow(new IOException("Can't register Schema"));
    replay(schemaRegistryClientMock, embeddedFormatMock, schemaProviderMock);
    SchemaManager mySchemaManager = new SchemaManagerImpl(schemaRegistryClientMock, new TopicNameStrategy());
    RestConstraintViolationException rcve = assertThrows(RestConstraintViolationException.class, () -> mySchemaManager.getSchema(TOPIC_NAME, /* format= */
    Optional.of(embeddedFormatMock), /* subject= */
    Optional.of("subject1"), /* subjectNameStrategy= */
    Optional.empty(), /* schemaId= */
    Optional.empty(), /* schemaVersion= */
    Optional.empty(), /* rawSchema= */
    Optional.of(TextNode.valueOf("rawString").toString()), /* isKey= */
    true));
    assertEquals("Error serializing message. Error when registering schema. format = EasyMock for class io.confluent.kafkarest.entities.EmbeddedFormat, subject = subject1, schema = null\n" + "Can't register Schema", rcve.getMessage());
    assertEquals(42207, rcve.getErrorCode());
}
Also used : EmbeddedFormat(io.confluent.kafkarest.entities.EmbeddedFormat) RestConstraintViolationException(io.confluent.rest.exceptions.RestConstraintViolationException) ProtobufSchemaProvider(io.confluent.kafka.schemaregistry.protobuf.ProtobufSchemaProvider) SchemaProvider(io.confluent.kafka.schemaregistry.SchemaProvider) JsonSchemaProvider(io.confluent.kafka.schemaregistry.json.JsonSchemaProvider) AvroSchemaProvider(io.confluent.kafka.schemaregistry.avro.AvroSchemaProvider) TopicNameStrategy(io.confluent.kafka.serializers.subject.TopicNameStrategy) ParsedSchema(io.confluent.kafka.schemaregistry.ParsedSchema) IOException(java.io.IOException) SchemaRegistryClient(io.confluent.kafka.schemaregistry.client.SchemaRegistryClient) MockSchemaRegistryClient(io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient) Test(org.junit.jupiter.api.Test)

Example 4 with EmbeddedFormat

use of io.confluent.kafkarest.entities.EmbeddedFormat in project kafka-rest by confluentinc.

the class SchemaManagerImplTest method errorRawSchemaCantParseSchema.

@Test
public void errorRawSchemaCantParseSchema() {
    SchemaRegistryClient schemaRegistryClientMock = mock(SchemaRegistryClient.class);
    EmbeddedFormat embeddedFormatMock = mock(EmbeddedFormat.class);
    SchemaProvider schemaProviderMock = mock(SchemaProvider.class);
    expect(embeddedFormatMock.requiresSchema()).andReturn(true);
    expect(embeddedFormatMock.getSchemaProvider()).andThrow(new UnsupportedOperationException("Unsupported"));
    replay(embeddedFormatMock, schemaProviderMock, schemaRegistryClientMock);
    SchemaManager mySchemaManager = new SchemaManagerImpl(schemaRegistryClientMock, new TopicNameStrategy());
    BadRequestException rcve = assertThrows(BadRequestException.class, () -> mySchemaManager.getSchema(TOPIC_NAME, /* format= */
    Optional.of(embeddedFormatMock), /* subject= */
    Optional.empty(), /* subjectNameStrategy= */
    Optional.empty(), /* schemaId= */
    Optional.empty(), /* schemaVersion= */
    Optional.empty(), /* rawSchema= */
    Optional.of(TextNode.valueOf("rawSchema").toString()), /* isKey= */
    true));
    assertEquals("Raw schema not supported with format = EasyMock for class io.confluent.kafkarest.entities.EmbeddedFormat", rcve.getMessage());
    assertEquals(400, rcve.getCode());
}
Also used : EmbeddedFormat(io.confluent.kafkarest.entities.EmbeddedFormat) ProtobufSchemaProvider(io.confluent.kafka.schemaregistry.protobuf.ProtobufSchemaProvider) SchemaProvider(io.confluent.kafka.schemaregistry.SchemaProvider) JsonSchemaProvider(io.confluent.kafka.schemaregistry.json.JsonSchemaProvider) AvroSchemaProvider(io.confluent.kafka.schemaregistry.avro.AvroSchemaProvider) TopicNameStrategy(io.confluent.kafka.serializers.subject.TopicNameStrategy) BadRequestException(io.confluent.kafkarest.exceptions.BadRequestException) SchemaRegistryClient(io.confluent.kafka.schemaregistry.client.SchemaRegistryClient) MockSchemaRegistryClient(io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient) Test(org.junit.jupiter.api.Test)

Aggregations

EmbeddedFormat (io.confluent.kafkarest.entities.EmbeddedFormat)4 SchemaProvider (io.confluent.kafka.schemaregistry.SchemaProvider)3 AvroSchemaProvider (io.confluent.kafka.schemaregistry.avro.AvroSchemaProvider)3 JsonSchemaProvider (io.confluent.kafka.schemaregistry.json.JsonSchemaProvider)3 ProtobufSchemaProvider (io.confluent.kafka.schemaregistry.protobuf.ProtobufSchemaProvider)3 BadRequestException (io.confluent.kafkarest.exceptions.BadRequestException)3 Test (org.junit.jupiter.api.Test)3 MockSchemaRegistryClient (io.confluent.kafka.schemaregistry.client.MockSchemaRegistryClient)2 SchemaRegistryClient (io.confluent.kafka.schemaregistry.client.SchemaRegistryClient)2 TopicNameStrategy (io.confluent.kafka.serializers.subject.TopicNameStrategy)2 MappingIterator (com.fasterxml.jackson.databind.MappingIterator)1 NullNode (com.fasterxml.jackson.databind.node.NullNode)1 ImmutableMultimap (com.google.common.collect.ImmutableMultimap)1 ByteString (com.google.protobuf.ByteString)1 ParsedSchema (io.confluent.kafka.schemaregistry.ParsedSchema)1 Errors (io.confluent.kafkarest.Errors)1 ProducerMetrics (io.confluent.kafkarest.ProducerMetrics)1 ProducerMetricsRegistry (io.confluent.kafkarest.ProducerMetricsRegistry)1 ProduceController (io.confluent.kafkarest.controllers.ProduceController)1 RecordSerializer (io.confluent.kafkarest.controllers.RecordSerializer)1