Search in sources :

Example 11 with Fetch

use of org.molgenis.data.Fetch in project molgenis by molgenis.

the class InMemoryRepositoryTest method streamFetch.

@Test
public void streamFetch() throws IOException {
    String idAttrName = "id";
    EntityType entityType = mock(EntityType.class);
    Attribute idAttr = when(mock(Attribute.class).getName()).thenReturn(idAttrName).getMock();
    when(entityType.getIdAttribute()).thenReturn(idAttr);
    try (InMemoryRepository inMemoryRepository = new InMemoryRepository(entityType)) {
        Object id0 = 0;
        Entity entity0 = when(mock(Entity.class).get(idAttrName)).thenReturn(id0).getMock();
        Object id1 = 1;
        Entity entity1 = when(mock(Entity.class).get(idAttrName)).thenReturn(id1).getMock();
        inMemoryRepository.add(entity0);
        inMemoryRepository.add(entity1);
        Fetch fetch = new Fetch();
        @SuppressWarnings("unchecked") Consumer<List<Entity>> consumer = mock(Consumer.class);
        inMemoryRepository.forEachBatched(fetch, consumer, 1000);
        verify(consumer).accept(Arrays.asList(entity0, entity1));
    }
}
Also used : EntityType(org.molgenis.data.meta.model.EntityType) Fetch(org.molgenis.data.Fetch) Entity(org.molgenis.data.Entity) Attribute(org.molgenis.data.meta.model.Attribute) List(java.util.List) Test(org.testng.annotations.Test)

Example 12 with Fetch

use of org.molgenis.data.Fetch in project molgenis by molgenis.

the class PostgreSqlRepository method forEachBatched.

@Override
public void forEachBatched(Fetch fetch, Consumer<List<Entity>> consumer, int batchSize) {
    final Stopwatch stopwatch = createStarted();
    final JdbcTemplate template = new JdbcTemplate(dataSource);
    template.setFetchSize(batchSize);
    final Query<Entity> query = new QueryImpl<>();
    if (fetch != null) {
        query.fetch(fetch);
    }
    final String allRowsSelect = getSqlSelect(entityType, query, emptyList(), false);
    LOG.debug("Fetching [{}] data...", getName());
    LOG.trace("SQL: {}", allRowsSelect);
    RowMapper<Entity> rowMapper = postgreSqlEntityFactory.createRowMapper(entityType, fetch);
    template.query(allRowsSelect, (ResultSetExtractor<Object>) resultSet -> processResultSet(consumer, batchSize, entityType, rowMapper, resultSet));
    LOG.debug("Streamed entire repository in batches of size {} in {}.", batchSize, stopwatch);
}
Also used : BatchingQueryResult(org.molgenis.data.support.BatchingQueryResult) PostgreSqlQueryUtils.getTableAttributes(org.molgenis.data.postgresql.PostgreSqlQueryUtils.getTableAttributes) RepositoryCapability(org.molgenis.data.RepositoryCapability) java.util(java.util) PostgreSqlQueryGenerator(org.molgenis.data.postgresql.PostgreSqlQueryGenerator) Stopwatch(com.google.common.base.Stopwatch) Operator(org.molgenis.data.QueryRule.Operator) ConstraintViolation(org.molgenis.data.validation.ConstraintViolation) LoggerFactory(org.slf4j.LoggerFactory) ONE_TO_MANY(org.molgenis.data.meta.AttributeType.ONE_TO_MANY) VALUE_TOO_LONG_MSG(org.molgenis.data.postgresql.PostgreSqlExceptionTranslator.VALUE_TOO_LONG_MSG) Fetch(org.molgenis.data.Fetch) QueryImpl(org.molgenis.data.support.QueryImpl) Attribute(org.molgenis.data.meta.model.Attribute) SQLException(java.sql.SQLException) AbstractRepository(org.molgenis.data.support.AbstractRepository) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) ResultSet(java.sql.ResultSet) Objects.requireNonNull(java.util.Objects.requireNonNull) DataSource(javax.sql.DataSource) org.springframework.jdbc.core(org.springframework.jdbc.core) com.google.common.collect(com.google.common.collect) AttributeType(org.molgenis.data.meta.AttributeType) PostgreSqlNameGenerator.getJunctionTableOrderColumnName(org.molgenis.data.postgresql.PostgreSqlNameGenerator.getJunctionTableOrderColumnName) Logger(org.slf4j.Logger) MolgenisValidationException(org.molgenis.data.validation.MolgenisValidationException) Maps.newHashMap(com.google.common.collect.Maps.newHashMap) Collections.emptyList(java.util.Collections.emptyList) PostgreSqlQueryUtils.getJunctionTableAttributes(org.molgenis.data.postgresql.PostgreSqlQueryUtils.getJunctionTableAttributes) PreparedStatement(java.sql.PreparedStatement) EntityType(org.molgenis.data.meta.model.EntityType) Collectors(java.util.stream.Collectors) PostgreSqlUtils.getPostgreSqlValue(org.molgenis.data.postgresql.PostgreSqlUtils.getPostgreSqlValue) String.format(java.lang.String.format) EntityTypeUtils.isMultipleReferenceType(org.molgenis.data.support.EntityTypeUtils.isMultipleReferenceType) UnexpectedEnumException(org.molgenis.util.UnexpectedEnumException) Consumer(java.util.function.Consumer) Stream(java.util.stream.Stream) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) Collections.unmodifiableSet(java.util.Collections.unmodifiableSet) Stopwatch.createStarted(com.google.common.base.Stopwatch.createStarted) StreamSupport.stream(java.util.stream.StreamSupport.stream) Query(org.molgenis.data.Query) Entity(org.molgenis.data.Entity) Entity(org.molgenis.data.Entity) QueryImpl(org.molgenis.data.support.QueryImpl) Stopwatch(com.google.common.base.Stopwatch)

Example 13 with Fetch

use of org.molgenis.data.Fetch in project molgenis by molgenis.

the class PostgreSqlRepository method verifyUpdate.

private void verifyUpdate(List<? extends Entity> entitiesBatch, int[] counts, Attribute idAttr) {
    int nrUpdatedEntities = Arrays.stream(counts).sum();
    if (nrUpdatedEntities < entitiesBatch.size()) {
        Set<Object> existingEntityIds = findAll(entitiesBatch.stream().map(Entity::getIdValue), new Fetch().field(idAttr.getName())).map(Entity::getIdValue).collect(toSet());
        Object nonExistingEntityId = entitiesBatch.stream().map(Entity::getIdValue).filter(entityId -> !existingEntityIds.contains(entityId)).findFirst().orElseThrow(() -> new IllegalStateException("Not all entities in batch were updated but all are present in the repository."));
        throw new MolgenisValidationException(new ConstraintViolation(format("Cannot update [%s] with id [%s] because it does not exist", entityType.getId(), nonExistingEntityId.toString())));
    }
}
Also used : Fetch(org.molgenis.data.Fetch) ConstraintViolation(org.molgenis.data.validation.ConstraintViolation) MolgenisValidationException(org.molgenis.data.validation.MolgenisValidationException)

Example 14 with Fetch

use of org.molgenis.data.Fetch in project molgenis by molgenis.

the class EntityListenerRepositoryDecoratorTest method findAllStreamFetch.

@Test
public void findAllStreamFetch() {
    Fetch fetch = new Fetch();
    Object id0 = "id0";
    Object id1 = "id1";
    Entity entity0 = Mockito.mock(Entity.class);
    Entity entity1 = Mockito.mock(Entity.class);
    Stream<Object> entityIds = Stream.of(id0, id1);
    Mockito.when(delegateRepository.findAll(entityIds, fetch)).thenReturn(Stream.of(entity0, entity1));
    Stream<Entity> expectedEntities = entityListenerRepositoryDecorator.findAll(entityIds, fetch);
    Assert.assertEquals(expectedEntities.collect(Collectors.toList()), Arrays.asList(entity0, entity1));
}
Also used : Fetch(org.molgenis.data.Fetch) Entity(org.molgenis.data.Entity) Test(org.testng.annotations.Test)

Example 15 with Fetch

use of org.molgenis.data.Fetch in project molgenis by molgenis.

the class InMemoryRepositoryTest method findOneObjectFetch.

@Test
public void findOneObjectFetch() throws IOException {
    String idAttrName = "id";
    EntityType entityType = mock(EntityType.class);
    Attribute idAttr = when(mock(Attribute.class).getName()).thenReturn(idAttrName).getMock();
    when(entityType.getIdAttribute()).thenReturn(idAttr);
    try (InMemoryRepository inMemoryRepository = new InMemoryRepository(entityType)) {
        Object id = 0;
        Entity entity = when(mock(Entity.class).get(idAttrName)).thenReturn(id).getMock();
        inMemoryRepository.add(entity);
        Fetch fetch = new Fetch();
        assertEquals(inMemoryRepository.findOneById(id, fetch), entity);
    }
}
Also used : EntityType(org.molgenis.data.meta.model.EntityType) Fetch(org.molgenis.data.Fetch) Entity(org.molgenis.data.Entity) Attribute(org.molgenis.data.meta.model.Attribute) Test(org.testng.annotations.Test)

Aggregations

Fetch (org.molgenis.data.Fetch)36 Test (org.testng.annotations.Test)28 AbstractMolgenisSpringTest (org.molgenis.data.AbstractMolgenisSpringTest)12 EntityType (org.molgenis.data.meta.model.EntityType)10 Entity (org.molgenis.data.Entity)9 Attribute (org.molgenis.data.meta.model.Attribute)7 List (java.util.List)3 Query (org.molgenis.data.Query)3 TransactionDefinition (org.springframework.transaction.TransactionDefinition)3 ConstraintViolation (org.molgenis.data.validation.ConstraintViolation)2 MolgenisValidationException (org.molgenis.data.validation.MolgenisValidationException)2 Stopwatch (com.google.common.base.Stopwatch)1 Stopwatch.createStarted (com.google.common.base.Stopwatch.createStarted)1 CacheLoader (com.google.common.cache.CacheLoader)1 com.google.common.collect (com.google.common.collect)1 Lists.newArrayList (com.google.common.collect.Lists.newArrayList)1 Maps.newHashMap (com.google.common.collect.Maps.newHashMap)1 String.format (java.lang.String.format)1 PreparedStatement (java.sql.PreparedStatement)1 ResultSet (java.sql.ResultSet)1