Search in sources :

Example 11 with BasicAckableForTesting

use of org.apache.gobblin.ack.BasicAckableForTesting in project incubator-gobblin by apache.

the class RecordEnvelopeTest method testMultipleClones.

@Test
public void testMultipleClones() {
    BasicAckableForTesting ackable = new BasicAckableForTesting();
    RecordEnvelope<CopyableRecord> record = new RecordEnvelope<>(new CopyableRecord(), new MyWatermark(110));
    record.addCallBack(ackable);
    StreamEntity.ForkCloner cloner = record.forkCloner();
    RecordEnvelope<CopyableRecord> copy1 = (RecordEnvelope<CopyableRecord>) cloner.getClone();
    RecordEnvelope<CopyableRecord> copy2 = (RecordEnvelope<CopyableRecord>) cloner.getClone();
    cloner.close();
    Assert.assertEquals(record.getRecord().id, copy1.getRecord().id);
    Assert.assertEquals(((MyWatermark) copy1.getWatermark()).getId(), 110);
    Assert.assertEquals(record.getRecord().id, copy2.getRecord().id);
    Assert.assertEquals(((MyWatermark) copy2.getWatermark()).getId(), 110);
    copy1.ack();
    Assert.assertEquals(ackable.acked, 0);
    copy2.ack();
    Assert.assertEquals(ackable.acked, 1);
}
Also used : BasicAckableForTesting(org.apache.gobblin.ack.BasicAckableForTesting) Test(org.testng.annotations.Test)

Example 12 with BasicAckableForTesting

use of org.apache.gobblin.ack.BasicAckableForTesting in project incubator-gobblin by apache.

the class RecordEnvelopeTest method testClone.

@Test
public void testClone() {
    BasicAckableForTesting ackable = new BasicAckableForTesting();
    RecordEnvelope<CopyableRecord> record = new RecordEnvelope<>(new CopyableRecord(), new MyWatermark(110));
    record.addCallBack(ackable);
    RecordEnvelope<CopyableRecord> copy = (RecordEnvelope<CopyableRecord>) record.getSingleClone();
    Assert.assertEquals(record.getRecord().id, copy.getRecord().id);
    Assert.assertEquals(((MyWatermark) copy.getWatermark()).getId(), 110);
    copy.ack();
    Assert.assertEquals(ackable.acked, 1);
    try {
        record.getSingleClone();
        Assert.fail();
    } catch (IllegalStateException ise) {
    // expected, cannot clone more than once using getSingleClone
    }
}
Also used : BasicAckableForTesting(org.apache.gobblin.ack.BasicAckableForTesting) Test(org.testng.annotations.Test)

Example 13 with BasicAckableForTesting

use of org.apache.gobblin.ack.BasicAckableForTesting in project incubator-gobblin by apache.

the class StreamEntityTest method testNack.

@Test
public void testNack() {
    MyStreamEntity streamEntity = new MyStreamEntity();
    BasicAckableForTesting ackable = new BasicAckableForTesting();
    streamEntity.addCallBack(ackable);
    streamEntity.nack(new RuntimeException());
    Assert.assertEquals(ackable.nacked, 1);
    Assert.assertTrue(ackable.throwable instanceof RuntimeException);
}
Also used : BasicAckableForTesting(org.apache.gobblin.ack.BasicAckableForTesting) Test(org.testng.annotations.Test)

Example 14 with BasicAckableForTesting

use of org.apache.gobblin.ack.BasicAckableForTesting in project incubator-gobblin by apache.

the class StreamEntityTest method testAcking.

@Test
public void testAcking() {
    MyStreamEntity streamEntity = new MyStreamEntity();
    BasicAckableForTesting ackable1 = new BasicAckableForTesting();
    streamEntity.addCallBack(ackable1);
    BasicAckableForTesting ackable2 = new BasicAckableForTesting();
    streamEntity.addCallBack(ackable2);
    streamEntity.ack();
    Assert.assertEquals(ackable1.acked, 1);
    Assert.assertEquals(ackable2.acked, 1);
}
Also used : BasicAckableForTesting(org.apache.gobblin.ack.BasicAckableForTesting) Test(org.testng.annotations.Test)

Aggregations

BasicAckableForTesting (org.apache.gobblin.ack.BasicAckableForTesting)14 Test (org.testng.annotations.Test)14 StreamEntity (org.apache.gobblin.stream.StreamEntity)5 WorkUnitState (org.apache.gobblin.configuration.WorkUnitState)4 RecordEnvelope (org.apache.gobblin.stream.RecordEnvelope)2 Properties (java.util.Properties)1 State (org.apache.gobblin.configuration.State)1 IdentityForkOperator (org.apache.gobblin.fork.IdentityForkOperator)1 TaskPublisher (org.apache.gobblin.publisher.TaskPublisher)1 RowLevelPolicyChecker (org.apache.gobblin.qualitychecker.row.RowLevelPolicyChecker)1 TaskLevelPolicyCheckResults (org.apache.gobblin.qualitychecker.task.TaskLevelPolicyCheckResults)1 TaskLevelPolicyChecker (org.apache.gobblin.qualitychecker.task.TaskLevelPolicyChecker)1 FlushControlMessage (org.apache.gobblin.stream.FlushControlMessage)1 TestPartitionAwareWriterBuilder (org.apache.gobblin.writer.test.TestPartitionAwareWriterBuilder)1 TestPartitioner (org.apache.gobblin.writer.test.TestPartitioner)1 Configuration (org.apache.hadoop.conf.Configuration)1