Search in sources :

Example 1 with PostgreSQLContainer

use of org.testcontainers.containers.PostgreSQLContainer in project hazelcast by hazelcast.

the class DebeziumCdcIntegrationTest method postgres.

@Test
public void postgres() throws Exception {
    PostgreSQLContainer<?> container = postgresContainer();
    try {
        container.start();
        // given
        List<String> expectedRecords = Arrays.asList("1001/0:SYNC:Customer {id=1001, firstName=Sally, lastName=Thomas, email=sally.thomas@acme.com}", "1002/0:SYNC:Customer {id=1002, firstName=George, lastName=Bailey, email=gbailey@foobar.com}", "1003/0:SYNC:Customer {id=1003, firstName=Edward, lastName=Walker, email=ed@walker.com}", "1004/0:SYNC:Customer {id=1004, firstName=Anne, lastName=Kretchmar, email=annek@noanswer.org}", "1004/1:UPDATE:Customer {id=1004, firstName=Anne Marie, lastName=Kretchmar, email=annek@noanswer.org}", "1005/0:INSERT:Customer {id=1005, firstName=Jason, lastName=Bourne, email=jason@bourne.org}", "1005/1:DELETE:Customer {id=1005, firstName=Jason, lastName=Bourne, email=jason@bourne.org}");
        StreamSource<ChangeRecord> source = DebeziumCdcSources.debezium("postgres", "io.debezium.connector.postgresql.PostgresConnector").setProperty("database.server.name", "dbserver1").setProperty("database.hostname", container.getContainerIpAddress()).setProperty("database.port", Integer.toString(container.getMappedPort(POSTGRESQL_PORT))).setProperty("database.user", "postgres").setProperty("database.password", "postgres").setProperty("database.dbname", "postgres").setProperty("table.whitelist", "inventory.customers").build();
        Pipeline pipeline = Pipeline.create();
        pipeline.readFrom(source).withNativeTimestamps(0).<ChangeRecord>customTransform("filter_timestamps", filterTimestampsProcessorSupplier()).groupingKey(record -> (Integer) record.key().toMap().get("id")).mapStateful(LongAccumulator::new, (accumulator, customerId, record) -> {
            long count = accumulator.get();
            accumulator.add(1);
            Operation operation = record.operation();
            RecordPart value = record.value();
            Customer customer = value.toObject(Customer.class);
            return entry(customerId + "/" + count, operation + ":" + customer);
        }).setLocalParallelism(1).writeTo(Sinks.map("results"));
        // when
        HazelcastInstance hz = createHazelcastInstances(2)[0];
        Job job = hz.getJet().newJob(pipeline);
        // then
        assertEqualsEventually(() -> hz.getMap("results").size(), 4);
        // when
        try (Connection connection = getPostgreSqlConnection(container.getJdbcUrl(), container.getUsername(), container.getPassword())) {
            connection.setSchema("inventory");
            Statement statement = connection.createStatement();
            statement.addBatch("UPDATE customers SET first_name='Anne Marie' WHERE id=1004");
            statement.addBatch("INSERT INTO customers VALUES (1005, 'Jason', 'Bourne', 'jason@bourne.org')");
            statement.addBatch("DELETE FROM customers WHERE id=1005");
            statement.executeBatch();
        }
        // then
        try {
            assertEqualsEventually(() -> mapResultsToSortedList(hz.getMap("results")), expectedRecords);
        } finally {
            job.cancel();
            assertJobStatusEventually(job, JobStatus.FAILED);
        }
    } finally {
        container.stop();
    }
}
Also used : JsonProperty(com.fasterxml.jackson.annotation.JsonProperty) Arrays(java.util.Arrays) Connection(java.sql.Connection) DockerImageName(org.testcontainers.utility.DockerImageName) RunWith(org.junit.runner.RunWith) PostgreSQLContainer(org.testcontainers.containers.PostgreSQLContainer) StreamSource(com.hazelcast.jet.pipeline.StreamSource) HazelcastSerialClassRunner(com.hazelcast.test.HazelcastSerialClassRunner) JetException(com.hazelcast.jet.JetException) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) Util.entry(com.hazelcast.jet.Util.entry) Assume(org.junit.Assume) JobStatus(com.hazelcast.jet.core.JobStatus) Nonnull(javax.annotation.Nonnull) Job(com.hazelcast.jet.Job) HazelcastInstance(com.hazelcast.core.HazelcastInstance) POSTGRESQL_PORT(org.testcontainers.containers.PostgreSQLContainer.POSTGRESQL_PORT) NightlyTest(com.hazelcast.test.annotation.NightlyTest) Pipeline(com.hazelcast.jet.pipeline.Pipeline) Sinks(com.hazelcast.jet.pipeline.Sinks) Test(org.junit.Test) Category(org.junit.experimental.categories.Category) MySQLContainer(org.testcontainers.containers.MySQLContainer) Objects(java.util.Objects) List(java.util.List) LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) Statement(java.sql.Statement) Entry(java.util.Map.Entry) MYSQL_PORT(org.testcontainers.containers.MySQLContainer.MYSQL_PORT) Statement(java.sql.Statement) Connection(java.sql.Connection) Pipeline(com.hazelcast.jet.pipeline.Pipeline) LongAccumulator(com.hazelcast.jet.accumulator.LongAccumulator) HazelcastInstance(com.hazelcast.core.HazelcastInstance) Job(com.hazelcast.jet.Job) NightlyTest(com.hazelcast.test.annotation.NightlyTest) Test(org.junit.Test)

Example 2 with PostgreSQLContainer

use of org.testcontainers.containers.PostgreSQLContainer in project spring-data-jdbc by spring-projects.

the class PostgresDataSourceConfiguration method createDataSource.

/*
	 * (non-Javadoc)
	 * @see org.springframework.data.jdbc.testing.DataSourceConfiguration#createDataSource()
	 */
@Override
protected DataSource createDataSource() {
    if (POSTGRESQL_CONTAINER == null) {
        PostgreSQLContainer<?> container = new PostgreSQLContainer<>();
        container.start();
        POSTGRESQL_CONTAINER = container;
    }
    PGSimpleDataSource dataSource = new PGSimpleDataSource();
    dataSource.setUrl(POSTGRESQL_CONTAINER.getJdbcUrl());
    dataSource.setUser(POSTGRESQL_CONTAINER.getUsername());
    dataSource.setPassword(POSTGRESQL_CONTAINER.getPassword());
    return dataSource;
}
Also used : PostgreSQLContainer(org.testcontainers.containers.PostgreSQLContainer) PGSimpleDataSource(org.postgresql.ds.PGSimpleDataSource)

Example 3 with PostgreSQLContainer

use of org.testcontainers.containers.PostgreSQLContainer in project hazelcast by hazelcast.

the class PostgresCdcNetworkIntegrationTest method initPostgres.

private PostgreSQLContainer<?> initPostgres(Network network, Integer fixedExposedPort) {
    PostgreSQLContainer<?> postgres = namedTestContainer(new PostgreSQLContainer<>(AbstractPostgresCdcIntegrationTest.DOCKER_IMAGE).withDatabaseName("postgres").withUsername("postgres").withPassword("postgres"));
    if (fixedExposedPort != null) {
        Consumer<CreateContainerCmd> cmd = e -> e.withPortBindings(new PortBinding(Ports.Binding.bindPort(fixedExposedPort), new ExposedPort(POSTGRESQL_PORT)));
        postgres = postgres.withCreateContainerCmdModifier(cmd);
    }
    if (network != null) {
        postgres = postgres.withNetwork(network);
    }
    postgres.start();
    return postgres;
}
Also used : CreateContainerCmd(com.github.dockerjava.api.command.CreateContainerCmd) Arrays(java.util.Arrays) Ports(com.github.dockerjava.api.model.Ports) Connection(java.sql.Connection) AbstractPostgresCdcIntegrationTest.getConnection(com.hazelcast.jet.cdc.postgres.AbstractPostgresCdcIntegrationTest.getConnection) RunWith(org.junit.runner.RunWith) Parameters(org.junit.runners.Parameterized.Parameters) RetryStrategies(com.hazelcast.jet.retry.RetryStrategies) PostgreSQLContainer(org.testcontainers.containers.PostgreSQLContainer) HazelcastSerialParametersRunnerFactory(com.hazelcast.test.HazelcastSerialParametersRunnerFactory) StreamSource(com.hazelcast.jet.pipeline.StreamSource) Network(org.testcontainers.containers.Network) JetException(com.hazelcast.jet.JetException) CreateContainerCmd(com.github.dockerjava.api.command.CreateContainerCmd) SQLException(java.sql.SQLException) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) Util.entry(com.hazelcast.jet.Util.entry) After(org.junit.After) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) ExposedPort(com.github.dockerjava.api.model.ExposedPort) Assume(org.junit.Assume) PortBinding(com.github.dockerjava.api.model.PortBinding) ChangeRecord(com.hazelcast.jet.cdc.ChangeRecord) AbstractCdcIntegrationTest(com.hazelcast.jet.cdc.AbstractCdcIntegrationTest) Nonnull(javax.annotation.Nonnull) Job(com.hazelcast.jet.Job) UseParametersRunnerFactory(org.junit.runners.Parameterized.UseParametersRunnerFactory) HazelcastInstance(com.hazelcast.core.HazelcastInstance) POSTGRESQL_PORT(org.testcontainers.containers.PostgreSQLContainer.POSTGRESQL_PORT) NightlyTest(com.hazelcast.test.annotation.NightlyTest) HazelcastParametrizedRunner(com.hazelcast.test.HazelcastParametrizedRunner) Pipeline(com.hazelcast.jet.pipeline.Pipeline) Parameter(org.junit.runners.Parameterized.Parameter) Collection(java.util.Collection) Sinks(com.hazelcast.jet.pipeline.Sinks) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) Category(org.junit.experimental.categories.Category) Consumer(java.util.function.Consumer) ToxiproxyContainer(org.testcontainers.containers.ToxiproxyContainer) Statement(java.sql.Statement) RUNNING(com.hazelcast.jet.core.JobStatus.RUNNING) RetryStrategy(com.hazelcast.jet.retry.RetryStrategy) SECONDS(java.util.concurrent.TimeUnit.SECONDS) Assert.assertEquals(org.junit.Assert.assertEquals) ExposedPort(com.github.dockerjava.api.model.ExposedPort) PortBinding(com.github.dockerjava.api.model.PortBinding) PostgreSQLContainer(org.testcontainers.containers.PostgreSQLContainer)

Example 4 with PostgreSQLContainer

use of org.testcontainers.containers.PostgreSQLContainer in project pinpoint by naver.

the class PostgreSQLContainerFactory method newContainer.

public static JdbcDatabaseContainer newContainer(String loggerName) {
    PostgreSQLContainer container = new PostgreSQLContainer();
    container.withInitScript("init_postgresql.sql");
    Log4jLoggerFactory log4jLoggerFactory = new Log4jLoggerFactory();
    org.slf4j.Logger logger = log4jLoggerFactory.getLogger(loggerName);
    container.withLogConsumer(new Slf4jLogConsumer(logger));
    return container;
}
Also used : Log4jLoggerFactory(org.apache.logging.slf4j.Log4jLoggerFactory) PostgreSQLContainer(org.testcontainers.containers.PostgreSQLContainer) Slf4jLogConsumer(org.testcontainers.containers.output.Slf4jLogConsumer)

Aggregations

PostgreSQLContainer (org.testcontainers.containers.PostgreSQLContainer)4 HazelcastInstance (com.hazelcast.core.HazelcastInstance)2 JetException (com.hazelcast.jet.JetException)2 Job (com.hazelcast.jet.Job)2 Util.entry (com.hazelcast.jet.Util.entry)2 Pipeline (com.hazelcast.jet.pipeline.Pipeline)2 Sinks (com.hazelcast.jet.pipeline.Sinks)2 StreamSource (com.hazelcast.jet.pipeline.StreamSource)2 NightlyTest (com.hazelcast.test.annotation.NightlyTest)2 Connection (java.sql.Connection)2 Statement (java.sql.Statement)2 Arrays (java.util.Arrays)2 Nonnull (javax.annotation.Nonnull)2 Assertions.assertThatThrownBy (org.assertj.core.api.Assertions.assertThatThrownBy)2 Assume (org.junit.Assume)2 Test (org.junit.Test)2 Category (org.junit.experimental.categories.Category)2 RunWith (org.junit.runner.RunWith)2 JsonProperty (com.fasterxml.jackson.annotation.JsonProperty)1 CreateContainerCmd (com.github.dockerjava.api.command.CreateContainerCmd)1