Examples with TestEvent org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent used on opensource projects

Search in sources :

Example 1 with TestEvent

use of org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent in project apex-malhar by apache.

the class JdbcNonTransactionalBatchOutputOperatorTest method testAtLeastOnceHalfBatch.

@Test
public void testAtLeastOnceHalfBatch() {
    JdbcOperatorTest.cleanTable();
    Random random = new Random();
    TestOutputOperator outputOperator = createOperator(ProcessingMode.AT_LEAST_ONCE);
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < HALF_BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.getStore().disconnect();
    // //
    com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap attributeMap = new com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap();
    attributeMap.put(OperatorContext.PROCESSING_MODE, ProcessingMode.AT_LEAST_ONCE);
    attributeMap.put(OperatorContext.ACTIVATION_WINDOW_ID, 0L);
    attributeMap.put(DAG.APPLICATION_ID, APP_ID);
    OperatorContext context = mockOperatorContext(OPERATOR_ID, attributeMap);
    outputOperator.setup(context);
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < HALF_BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 1, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
}
Also used : Random(java.util.Random) TestEvent(org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent) OperatorContextTestHelper.mockOperatorContext(org.apache.apex.malhar.lib.helper.OperatorContextTestHelper.mockOperatorContext) OperatorContext(com.datatorrent.api.Context.OperatorContext) Test(org.junit.Test)

Example 2 with TestEvent

use of org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent in project apex-malhar by apache.

the class JdbcNonTransactionalBatchOutputOperatorTest method testAtMostOnceHalfBatch.

@Test
public void testAtMostOnceHalfBatch() {
    JdbcOperatorTest.cleanTable();
    Random random = new Random();
    TestOutputOperator outputOperator = createOperator(ProcessingMode.AT_MOST_ONCE);
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < HALF_BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.getStore().disconnect();
    // //
    com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap attributeMap = new com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap();
    attributeMap.put(OperatorContext.PROCESSING_MODE, ProcessingMode.AT_MOST_ONCE);
    attributeMap.put(OperatorContext.ACTIVATION_WINDOW_ID, 0L);
    attributeMap.put(DAG.APPLICATION_ID, APP_ID);
    OperatorContext context = mockOperatorContext(OPERATOR_ID, attributeMap);
    outputOperator.setup(context);
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(2);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 2, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
}
Also used : Random(java.util.Random) TestEvent(org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent) OperatorContextTestHelper.mockOperatorContext(org.apache.apex.malhar.lib.helper.OperatorContextTestHelper.mockOperatorContext) OperatorContext(com.datatorrent.api.Context.OperatorContext) Test(org.junit.Test)

Example 3 with TestEvent

use of org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent in project apex-malhar by apache.

the class JdbcNonTransactionalBatchOutputOperatorTest method testAtMostOnceFullBatch.

@Test
public void testAtMostOnceFullBatch() {
    JdbcOperatorTest.cleanTable();
    Random random = new Random();
    TestOutputOperator outputOperator = createOperator(ProcessingMode.AT_MOST_ONCE);
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.getStore().disconnect();
    // //
    com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap attributeMap = new com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap();
    attributeMap.put(OperatorContext.PROCESSING_MODE, ProcessingMode.AT_MOST_ONCE);
    attributeMap.put(OperatorContext.ACTIVATION_WINDOW_ID, 0L);
    attributeMap.put(DAG.APPLICATION_ID, APP_ID);
    OperatorContext context = mockOperatorContext(OPERATOR_ID, attributeMap);
    outputOperator.setup(context);
    outputOperator.beginWindow(2);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 2, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 3 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
}
Also used : Random(java.util.Random) TestEvent(org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent) OperatorContextTestHelper.mockOperatorContext(org.apache.apex.malhar.lib.helper.OperatorContextTestHelper.mockOperatorContext) OperatorContext(com.datatorrent.api.Context.OperatorContext) Test(org.junit.Test)

Example 4 with TestEvent

use of org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent in project apex-malhar by apache.

the class JdbcNonTransactionalBatchOutputOperatorTest method testBatch.

@Test
public void testBatch() {
    JdbcOperatorTest.cleanTable();
    Random random = new Random();
    TestOutputOperator outputOperator = createOperator(ProcessingMode.AT_LEAST_ONCE);
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < HALF_BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 1, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should not be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(2);
    for (int batchCounter = 0; batchCounter < HALF_BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 2, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should not be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.teardown();
}
Also used : Random(java.util.Random) TestEvent(org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent) Test(org.junit.Test)

Example 5 with TestEvent

use of org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent in project apex-malhar by apache.

the class JdbcNonTransactionalBatchOutputOperatorTest method testAtLeastOnceFullBatch.

@Test
public void testAtLeastOnceFullBatch() {
    JdbcOperatorTest.cleanTable();
    Random random = new Random();
    TestOutputOperator outputOperator = createOperator(ProcessingMode.AT_LEAST_ONCE);
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.getStore().disconnect();
    // //
    com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap attributeMap = new com.datatorrent.api.Attribute.AttributeMap.DefaultAttributeMap();
    attributeMap.put(OperatorContext.PROCESSING_MODE, ProcessingMode.AT_LEAST_ONCE);
    attributeMap.put(OperatorContext.ACTIVATION_WINDOW_ID, 0L);
    attributeMap.put(DAG.APPLICATION_ID, APP_ID);
    OperatorContext context = mockOperatorContext(OPERATOR_ID, attributeMap);
    outputOperator.setup(context);
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(0);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 0, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 2 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
    outputOperator.beginWindow(1);
    for (int batchCounter = 0; batchCounter < BATCH_SIZE; batchCounter++) {
        outputOperator.input.put(new TestEvent(random.nextInt()));
    }
    outputOperator.endWindow();
    Assert.assertEquals("Commit window id ", 1, outputOperator.getStore().getCommittedWindowId(APP_ID, OPERATOR_ID));
    Assert.assertEquals("Batch should be written", 3 * BATCH_SIZE, outputOperator.getNumOfEventsInStore(outputOperator.getStore().connection));
}
Also used : Random(java.util.Random) TestEvent(org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent) OperatorContextTestHelper.mockOperatorContext(org.apache.apex.malhar.lib.helper.OperatorContextTestHelper.mockOperatorContext) OperatorContext(com.datatorrent.api.Context.OperatorContext) Test(org.junit.Test)

Aggregations

Random (java.util.Random)5 TestEvent (org.apache.apex.malhar.lib.db.jdbc.JdbcNonTransactionalOutputOperatorTest.TestEvent)5 Test (org.junit.Test)5 OperatorContext (com.datatorrent.api.Context.OperatorContext)4 OperatorContextTestHelper.mockOperatorContext (org.apache.apex.malhar.lib.helper.OperatorContextTestHelper.mockOperatorContext)4
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial