Search in sources :

Example 11 with ChangeEvent

use of org.wikidata.query.rdf.tool.change.events.ChangeEvent in project wikidata-query-rdf by wikimedia.

the class KafkaPollerUnitTest method deleteRevision.

@Test
public void deleteRevision() throws RetryableException {
    ConsumerRecords<String, ChangeEvent> rs = makeRecords(makeRecord(makeRCEvent(Duration.ofMillis(20), 1, 1, "Q123"), "topictest", Duration.ofMillis(20)), makeRecord(makeDeleteEvent(Duration.ofMillis(21), 1, "Q123"), "othertopic", Duration.ofMillis(21)), makeRecord(makeRCEvent(Duration.ofMillis(22), 1, 2, "Q123"), "topictest", Duration.ofMillis(22)));
    Batch batch = getBatchFromRecords(rs);
    // Delete revision should always win
    assertThat(batch.changes()).hasSize(1).anyMatch(revision(-1));
}
Also used : ChangeEvent(org.wikidata.query.rdf.tool.change.events.ChangeEvent) Batch(org.wikidata.query.rdf.tool.change.KafkaPoller.Batch) Test(org.junit.Test)

Example 12 with ChangeEvent

use of org.wikidata.query.rdf.tool.change.events.ChangeEvent in project wikidata-query-rdf by wikimedia.

the class KafkaPollerUnitTest method changesFromTopics.

@Test
public void changesFromTopics() throws RetryableException {
    ConsumerRecords<String, ChangeEvent> rs = makeRecords(makeRecord(makeRCEvent(Duration.ofMillis(20), 1, 1, "Q123"), "topictest", Duration.ofMillis(20)), makeRecord(makeRCEvent(Duration.ofMillis(21), 2, 1, "Q234"), "othertopic", Duration.ofMillis(21)), makeRecord(makeRCEvent(Duration.ofMillis(20), 3, 1, "Q567"), "topictest", Duration.ofMillis(20)));
    Batch batch = getBatchFromRecords(rs);
    assertThat(batch.changes()).hasSize(3).anyMatch(title("Q123")).anyMatch(title("Q234")).anyMatch(title("Q567"));
}
Also used : ChangeEvent(org.wikidata.query.rdf.tool.change.events.ChangeEvent) Batch(org.wikidata.query.rdf.tool.change.KafkaPoller.Batch) Test(org.junit.Test)

Aggregations

ChangeEvent (org.wikidata.query.rdf.tool.change.events.ChangeEvent)12 Test (org.junit.Test)9 Batch (org.wikidata.query.rdf.tool.change.KafkaPoller.Batch)9 MetricRegistry (com.codahale.metrics.MetricRegistry)2 HashMap (java.util.HashMap)2 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)2 TopicPartition (org.apache.kafka.common.TopicPartition)2 Context (com.codahale.metrics.Timer.Context)1 ImmutableList (com.google.common.collect.ImmutableList)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1 Instant (java.time.Instant)1 ArrayList (java.util.ArrayList)1 Collections.singletonList (java.util.Collections.singletonList)1 LinkedHashMap (java.util.LinkedHashMap)1 List (java.util.List)1 Map (java.util.Map)1 Collectors.toList (java.util.stream.Collectors.toList)1 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)1