Search in sources :

Example 6 with HookNotification

use of org.apache.atlas.model.notification.HookNotification in project atlas by apache.

the class KafkaConsumerTest method testReceive.

@Test
public void testReceive() throws Exception {
    Referenceable entity = getEntity(TRAIT_NAME);
    EntityUpdateRequest message = new EntityUpdateRequest("user1", entity);
    String json = AtlasType.toV1Json(new AtlasNotificationMessage<>(new MessageVersion("1.0.0"), message));
    TopicPartition tp = new TopicPartition("ATLAS_HOOK", 0);
    List<ConsumerRecord<String, String>> klist = Collections.singletonList(new ConsumerRecord<>("ATLAS_HOOK", 0, 0L, "mykey", json));
    Map mp = Collections.singletonMap(tp, klist);
    ConsumerRecords records = new ConsumerRecords(mp);
    when(kafkaConsumer.poll(100)).thenReturn(records);
    kafkaConsumer.assign(Collections.singletonList(tp));
    AtlasKafkaConsumer consumer = new AtlasKafkaConsumer(NotificationType.HOOK, kafkaConsumer, false, 100L);
    List<AtlasKafkaMessage<HookNotification>> messageList = consumer.receive();
    assertTrue(messageList.size() > 0);
    HookNotification consumedMessage = messageList.get(0).getMessage();
    assertMessagesEqual(message, consumedMessage, entity);
}
Also used : MessageVersion(org.apache.atlas.model.notification.MessageVersion) ConsumerRecords(org.apache.kafka.clients.consumer.ConsumerRecords) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) EntityUpdateRequest(org.apache.atlas.v1.model.notification.HookNotificationV1.EntityUpdateRequest) HookNotification(org.apache.atlas.model.notification.HookNotification) Referenceable(org.apache.atlas.v1.model.instance.Referenceable) TopicPartition(org.apache.kafka.common.TopicPartition) Map(java.util.Map) Test(org.testng.annotations.Test) EntityNotificationTest(org.apache.atlas.notification.entity.EntityNotificationTest)

Example 7 with HookNotification

use of org.apache.atlas.model.notification.HookNotification in project atlas by apache.

the class KafkaNotificationTest method testReceiveKafkaMessages.

@Test
public void testReceiveKafkaMessages() throws Exception {
    kafkaNotification.send(NotificationInterface.NotificationType.HOOK, new EntityCreateRequest("u1", new Referenceable("type")));
    kafkaNotification.send(NotificationInterface.NotificationType.HOOK, new EntityCreateRequest("u2", new Referenceable("type")));
    kafkaNotification.send(NotificationInterface.NotificationType.HOOK, new EntityCreateRequest("u3", new Referenceable("type")));
    kafkaNotification.send(NotificationInterface.NotificationType.HOOK, new EntityCreateRequest("u4", new Referenceable("type")));
    NotificationConsumer<Object> consumer = kafkaNotification.createConsumers(NotificationInterface.NotificationType.HOOK, 1).get(0);
    List<AtlasKafkaMessage<Object>> messages = null;
    // fetch starting time
    long startTime = System.currentTimeMillis();
    while ((System.currentTimeMillis() - startTime) < 10000) {
        messages = consumer.receive();
        if (messages.size() > 0) {
            break;
        }
    }
    int i = 1;
    for (AtlasKafkaMessage<Object> msg : messages) {
        HookNotification message = (HookNotification) msg.getMessage();
        assertEquals(message.getUser(), "u" + i++);
    }
    consumer.close();
}
Also used : HookNotification(org.apache.atlas.model.notification.HookNotification) Referenceable(org.apache.atlas.v1.model.instance.Referenceable) EntityCreateRequest(org.apache.atlas.v1.model.notification.HookNotificationV1.EntityCreateRequest) Test(org.testng.annotations.Test)

Example 8 with HookNotification

use of org.apache.atlas.model.notification.HookNotification in project atlas by apache.

the class HookNotificationDeserializerTest method testDeserializeSplitMessage.

@Test
public void testDeserializeSplitMessage() throws Exception {
    Referenceable entity = generateVeryLargeEntityWithTrait();
    EntityUpdateRequest message = new EntityUpdateRequest("user1", entity);
    List<String> jsonMsgList = new ArrayList<>();
    AbstractNotification.createNotificationMessages(message, jsonMsgList);
    assertTrue(jsonMsgList.size() > 1);
    HookNotification deserializedMessage = deserialize(jsonMsgList);
    assertEqualMessage(deserializedMessage, message);
}
Also used : HookNotification(org.apache.atlas.model.notification.HookNotification) Referenceable(org.apache.atlas.v1.model.instance.Referenceable) ArrayList(java.util.ArrayList) EntityUpdateRequest(org.apache.atlas.v1.model.notification.HookNotificationV1.EntityUpdateRequest) Test(org.testng.annotations.Test) EntityNotificationTest(org.apache.atlas.notification.entity.EntityNotificationTest)

Example 9 with HookNotification

use of org.apache.atlas.model.notification.HookNotification in project atlas by apache.

the class HookNotificationDeserializerTest method testDeserializeLegacyMessage.

// validate deserialization of legacy message, which doesn't use MessageVersion
@Test
public void testDeserializeLegacyMessage() throws Exception {
    Referenceable entity = generateEntityWithTrait();
    EntityUpdateRequest message = new EntityUpdateRequest("user1", entity);
    String jsonMsg = AtlasType.toV1Json(message);
    HookNotification deserializedMessage = deserialize(Collections.singletonList(jsonMsg));
    assertEqualMessage(deserializedMessage, message);
}
Also used : HookNotification(org.apache.atlas.model.notification.HookNotification) Referenceable(org.apache.atlas.v1.model.instance.Referenceable) EntityUpdateRequest(org.apache.atlas.v1.model.notification.HookNotificationV1.EntityUpdateRequest) Test(org.testng.annotations.Test) EntityNotificationTest(org.apache.atlas.notification.entity.EntityNotificationTest)

Example 10 with HookNotification

use of org.apache.atlas.model.notification.HookNotification in project atlas by apache.

the class HookNotificationDeserializerTest method testDeserializeCompressedMessage.

@Test
public void testDeserializeCompressedMessage() throws Exception {
    Referenceable entity = generateLargeEntityWithTrait();
    EntityUpdateRequest message = new EntityUpdateRequest("user1", entity);
    List<String> jsonMsgList = new ArrayList<>();
    AbstractNotification.createNotificationMessages(message, jsonMsgList);
    assertTrue(jsonMsgList.size() == 1);
    String compressedMsg = jsonMsgList.get(0);
    String uncompressedMsg = AtlasType.toV1Json(message);
    assertTrue(compressedMsg.length() < uncompressedMsg.length(), "Compressed message (" + compressedMsg.length() + ") should be shorter than uncompressed message (" + uncompressedMsg.length() + ")");
    HookNotification deserializedMessage = deserialize(jsonMsgList);
    assertEqualMessage(deserializedMessage, message);
}
Also used : HookNotification(org.apache.atlas.model.notification.HookNotification) Referenceable(org.apache.atlas.v1.model.instance.Referenceable) ArrayList(java.util.ArrayList) EntityUpdateRequest(org.apache.atlas.v1.model.notification.HookNotificationV1.EntityUpdateRequest) Test(org.testng.annotations.Test) EntityNotificationTest(org.apache.atlas.notification.entity.EntityNotificationTest)

Aggregations

HookNotification (org.apache.atlas.model.notification.HookNotification)31 Test (org.testng.annotations.Test)18 ArrayList (java.util.ArrayList)13 AtlasEntitiesWithExtInfo (org.apache.atlas.model.instance.AtlasEntity.AtlasEntitiesWithExtInfo)9 Referenceable (org.apache.atlas.v1.model.instance.Referenceable)9 AtlasObjectId (org.apache.atlas.model.instance.AtlasObjectId)8 EntityCreateRequest (org.apache.atlas.v1.model.notification.HookNotificationV1.EntityCreateRequest)8 AtlasEntity (org.apache.atlas.model.instance.AtlasEntity)6 EntityNotificationTest (org.apache.atlas.notification.entity.EntityNotificationTest)6 EntityUpdateRequest (org.apache.atlas.v1.model.notification.HookNotificationV1.EntityUpdateRequest)6 Map (java.util.Map)5 EntityCreateRequestV2 (org.apache.atlas.model.notification.HookNotification.EntityCreateRequestV2)5 NotificationException (org.apache.atlas.notification.NotificationException)5 EntityUpdateRequestV2 (org.apache.atlas.model.notification.HookNotification.EntityUpdateRequestV2)4 AtlasEntityWithExtInfo (org.apache.atlas.model.instance.AtlasEntity.AtlasEntityWithExtInfo)3 EntityDeleteRequestV2 (org.apache.atlas.model.notification.HookNotification.EntityDeleteRequestV2)3 EntityPartialUpdateRequestV2 (org.apache.atlas.model.notification.HookNotification.EntityPartialUpdateRequestV2)3 MessageVersion (org.apache.atlas.model.notification.MessageVersion)2 Table (org.apache.hadoop.hive.ql.metadata.Table)2 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)2