Examples with DivolteSchema io.divolte.server.DivolteSchema used on opensource projects

Example 1 with DivolteSchema

use of io.divolte.server.DivolteSchema in project divolte-collector by divolte.

the class SchemaRegistryTest method testConfluentSinkCorrectlyAssociatedWithSchemaId.

@Test
public void testConfluentSinkCorrectlyAssociatedWithSchemaId() {
    final ValidatedConfiguration vc = new ValidatedConfiguration(() -> ConfigFactory.parseResources("schema-registry-with-confluent.conf"));
    final SchemaRegistry registry = new SchemaRegistry(vc);
    final DivolteSchema schema = registry.getSchemaBySinkName("kafka");
    assertEquals(schema.confluentId, Optional.of(12345));
}

Example 2 with DivolteSchema

use of io.divolte.server.DivolteSchema in project divolte-collector by divolte.

the class GoogleCloudPubSubFlusher method schemaFingerprint.

private static String schemaFingerprint(final DivolteSchema schema) {
    final Schema avroSchema = schema.avroSchema;
    final byte[] fingerprint;
    // SHA-256 is on the list of mandatory JCE algorithms, so this shouldn't be an issue.
    try {
        fingerprint = SchemaNormalization.parsingFingerprint("SHA-256", avroSchema);
    } catch (final NoSuchAlgorithmException e) {
        throw new RuntimeException("Cannot calculate schema fingerprint; missing SHA-256 digest algorithm", e);
    }
    return FINGERPRINT_ENCODER.encodeToString(fingerprint);
}

Example 3 with DivolteSchema

use of io.divolte.server.DivolteSchema in project divolte-collector by divolte.

the class GoogleCloudPubSubFlusherTest method processSingleMessage.

private void processSingleMessage(final Optional<Integer> confluentId) {
    final Publisher publisher = mockPublisher.orElseThrow(IllegalStateException::new);
    // Process a single message.
    final DivolteSchema schema = new DivolteSchema(MINIMAL_SCHEMA, confluentId);
    final GoogleCloudPubSubFlusher flusher = new GoogleCloudPubSubFlusher(publisher, schema);
    if (ItemProcessor.ProcessingDirective.PAUSE == flusher.process(itemFromAvroRecordBuffer(generateMessage()))) {
        flusher.heartbeat();
    }
}

Example 4 with DivolteSchema

use of io.divolte.server.DivolteSchema in project divolte-collector by divolte.

the class GoogleCloudPubSubFlusherTest method testMessageBatchSentToPublisher.

@Test
public void testMessageBatchSentToPublisher() {
    final Publisher publisher = mockPublisher.orElseThrow(IllegalStateException::new);
    // Process a bunch of messages.
    final DivolteSchema schema = new DivolteSchema(MINIMAL_SCHEMA, Optional.empty());
    final GoogleCloudPubSubFlusher flusher = new GoogleCloudPubSubFlusher(publisher, schema);
    final Queue<Item<AvroRecordBuffer>> items = Stream.generate(this::generateMessage).limit(10).map(this::itemFromAvroRecordBuffer).collect(Collectors.toCollection(() -> new ArrayBlockingQueue<>(10)));
    flusher.process(items);
    // Check the messages were all forwarded to the publisher.
    verify(publisher, times(10)).publish(any(PubsubMessage.class));
    verifyNoMoreInteractions(publisher);
}