Search in sources :

Example 16 with TableQuery

use of org.pmiops.workbench.model.TableQuery in project workbench by all-of-us.

the class CohortMaterializationServiceTest method testMaterializeCohortPersonFieldSetPersonIdWithNumberFilter.

@Test
public void testMaterializeCohortPersonFieldSetPersonIdWithNumberFilter() {
    TableQuery tableQuery = new TableQuery();
    tableQuery.setTableName("person");
    tableQuery.setColumns(ImmutableList.of("person_id"));
    ColumnFilter filter = new ColumnFilter();
    filter.setColumnName("person_id");
    filter.setValueNumber(new BigDecimal(1L));
    tableQuery.addFiltersItem(ImmutableList.of(filter));
    FieldSet fieldSet = new FieldSet();
    fieldSet.setTableQuery(tableQuery);
    MaterializeCohortResponse response = cohortMaterializationService.materializeCohort(null, SearchRequests.allGenders(), makeRequest(fieldSet, 1000));
    assertPersonIds(response, 1L);
    assertThat(response.getNextPageToken()).isNull();
}
Also used : FieldSet(org.pmiops.workbench.model.FieldSet) TableQuery(org.pmiops.workbench.model.TableQuery) ColumnFilter(org.pmiops.workbench.model.ColumnFilter) BigDecimal(java.math.BigDecimal) MaterializeCohortResponse(org.pmiops.workbench.model.MaterializeCohortResponse) BigQueryBaseTest(org.pmiops.workbench.api.BigQueryBaseTest) Test(org.junit.Test)

Example 17 with TableQuery

use of org.pmiops.workbench.model.TableQuery in project workbench by all-of-us.

the class CohortMaterializationServiceTest method testMaterializeCohortPersonFieldSetAllColumns.

@Test
public void testMaterializeCohortPersonFieldSetAllColumns() {
    TableQuery tableQuery = new TableQuery();
    tableQuery.setTableName("person");
    FieldSet fieldSet = new FieldSet();
    fieldSet.setTableQuery(tableQuery);
    MaterializeCohortResponse response = cohortMaterializationService.materializeCohort(null, SearchRequests.males(), makeRequest(fieldSet, 1000));
    ImmutableMap<String, Object> p1Map = ImmutableMap.<String, Object>builder().put("person_id", 1L).put("gender_source_value", "1").put("race_source_value", "1").put("gender_concept_id", 8507L).put("year_of_birth", 1980L).put("month_of_birth", 8L).put("day_of_birth", 1L).put("race_concept_id", 1L).put("ethnicity_concept_id", 1L).put("location_id", 1L).put("provider_id", 1L).put("care_site_id", 1L).put("person_source_value", "psv").put("gender_source_concept_id", 1L).put("race_source_concept_id", 1L).put("ethnicity_source_value", "esv").put("ethnicity_source_concept_id", 1L).build();
    assertResults(response, p1Map);
    assertThat(response.getNextPageToken()).isNull();
}
Also used : FieldSet(org.pmiops.workbench.model.FieldSet) TableQuery(org.pmiops.workbench.model.TableQuery) MaterializeCohortResponse(org.pmiops.workbench.model.MaterializeCohortResponse) BigQueryBaseTest(org.pmiops.workbench.api.BigQueryBaseTest) Test(org.junit.Test)

Example 18 with TableQuery

use of org.pmiops.workbench.model.TableQuery in project workbench by all-of-us.

the class FieldSetQueryBuilder method extractResults.

public Map<String, Object> extractResults(TableQueryAndConfig tableQueryAndConfig, List<FieldValue> row) {
    TableQuery tableQuery = tableQueryAndConfig.getTableQuery();
    List<ColumnConfig> columnConfigs = tableQuery.getColumns().stream().map(columnName -> tableQueryAndConfig.getColumn(columnName)).collect(Collectors.toList());
    Map<String, Object> results = new HashMap<>(tableQuery.getColumns().size());
    for (int i = 0; i < columnConfigs.size(); i++) {
        FieldValue fieldValue = row.get(i);
        ColumnConfig columnConfig = columnConfigs.get(i);
        if (!fieldValue.isNull()) {
            Object value;
            switch(columnConfig.type) {
                case DATE:
                    value = fieldValue.getStringValue();
                    break;
                case FLOAT:
                    value = fieldValue.getDoubleValue();
                    break;
                case INTEGER:
                    value = fieldValue.getLongValue();
                    break;
                case STRING:
                    value = fieldValue.getStringValue();
                    break;
                case TIMESTAMP:
                    value = DATE_TIME_FORMAT.print(fieldValue.getTimestampValue() / 1000L);
                    break;
                default:
                    throw new IllegalStateException("Unrecognized column type: " + columnConfig.type);
            }
            results.put(columnConfig.name, value);
        }
    }
    return results;
}
Also used : DateTimeFormat(org.joda.time.format.DateTimeFormat) BadRequestException(org.pmiops.workbench.exceptions.BadRequestException) FieldValue(com.google.cloud.bigquery.FieldValue) ColumnType(org.pmiops.workbench.config.CdrBigQuerySchemaConfig.ColumnType) DateTimeFormatter(org.joda.time.format.DateTimeFormatter) ColumnConfig(org.pmiops.workbench.config.CdrBigQuerySchemaConfig.ColumnConfig) TableQuery(org.pmiops.workbench.model.TableQuery) Operator(org.pmiops.workbench.model.Operator) SimpleDateFormat(java.text.SimpleDateFormat) Autowired(org.springframework.beans.factory.annotation.Autowired) QueryJobConfiguration(com.google.cloud.bigquery.QueryJobConfiguration) HashMap(java.util.HashMap) Collectors(java.util.stream.Collectors) FieldSet(org.pmiops.workbench.model.FieldSet) Strings(com.google.common.base.Strings) BigDecimal(java.math.BigDecimal) List(java.util.List) Service(org.springframework.stereotype.Service) Map(java.util.Map) ColumnFilter(org.pmiops.workbench.model.ColumnFilter) ParseException(java.text.ParseException) QueryParameterValue(com.google.cloud.bigquery.QueryParameterValue) Joiner(com.google.common.base.Joiner) OperatorUtils(org.pmiops.workbench.utils.OperatorUtils) ColumnConfig(org.pmiops.workbench.config.CdrBigQuerySchemaConfig.ColumnConfig) HashMap(java.util.HashMap) TableQuery(org.pmiops.workbench.model.TableQuery) FieldValue(com.google.cloud.bigquery.FieldValue)

Example 19 with TableQuery

use of org.pmiops.workbench.model.TableQuery in project workbench by all-of-us.

the class CohortMaterializationServiceTest method testMaterializeCohortPersonFieldSetPersonIdWithNumberGreaterThanOrEqualToFilter.

@Test
public void testMaterializeCohortPersonFieldSetPersonIdWithNumberGreaterThanOrEqualToFilter() {
    TableQuery tableQuery = new TableQuery();
    tableQuery.setTableName("person");
    tableQuery.setColumns(ImmutableList.of("person_id"));
    ColumnFilter filter = new ColumnFilter();
    filter.setColumnName("person_id");
    filter.setOperator(Operator.GREATER_THAN_OR_EQUAL_TO);
    filter.setValueNumber(new BigDecimal(2L));
    tableQuery.addFiltersItem(ImmutableList.of(filter));
    FieldSet fieldSet = new FieldSet();
    fieldSet.setTableQuery(tableQuery);
    MaterializeCohortResponse response = cohortMaterializationService.materializeCohort(null, SearchRequests.allGenders(), makeRequest(fieldSet, 1000));
    assertPersonIds(response, 2L, 102246L);
    assertThat(response.getNextPageToken()).isNull();
}
Also used : FieldSet(org.pmiops.workbench.model.FieldSet) TableQuery(org.pmiops.workbench.model.TableQuery) ColumnFilter(org.pmiops.workbench.model.ColumnFilter) BigDecimal(java.math.BigDecimal) MaterializeCohortResponse(org.pmiops.workbench.model.MaterializeCohortResponse) BigQueryBaseTest(org.pmiops.workbench.api.BigQueryBaseTest) Test(org.junit.Test)

Example 20 with TableQuery

use of org.pmiops.workbench.model.TableQuery in project workbench by all-of-us.

the class CohortMaterializationServiceTest method testMaterializeCohortPersonFieldSetPersonIdWithStringFilterNullNonMatch.

@Test
public void testMaterializeCohortPersonFieldSetPersonIdWithStringFilterNullNonMatch() {
    TableQuery tableQuery = new TableQuery();
    tableQuery.setTableName("person");
    tableQuery.setColumns(ImmutableList.of("person_id"));
    ColumnFilter filter = new ColumnFilter();
    filter.setColumnName("ethnicity_source_value");
    filter.setValue("esv");
    tableQuery.addFiltersItem(ImmutableList.of(filter));
    FieldSet fieldSet = new FieldSet();
    fieldSet.setTableQuery(tableQuery);
    MaterializeCohortResponse response = cohortMaterializationService.materializeCohort(null, SearchRequests.allGenders(), makeRequest(fieldSet, 1000));
    assertPersonIds(response, 1L, 2L);
    assertThat(response.getNextPageToken()).isNull();
}
Also used : FieldSet(org.pmiops.workbench.model.FieldSet) TableQuery(org.pmiops.workbench.model.TableQuery) ColumnFilter(org.pmiops.workbench.model.ColumnFilter) MaterializeCohortResponse(org.pmiops.workbench.model.MaterializeCohortResponse) BigQueryBaseTest(org.pmiops.workbench.api.BigQueryBaseTest) Test(org.junit.Test)

Aggregations

TableQuery (org.pmiops.workbench.model.TableQuery)38 FieldSet (org.pmiops.workbench.model.FieldSet)36 Test (org.junit.Test)35 BigQueryBaseTest (org.pmiops.workbench.api.BigQueryBaseTest)35 MaterializeCohortResponse (org.pmiops.workbench.model.MaterializeCohortResponse)35 ColumnFilter (org.pmiops.workbench.model.ColumnFilter)31 BigDecimal (java.math.BigDecimal)9 BadRequestException (org.pmiops.workbench.exceptions.BadRequestException)3 QueryParameterValue (com.google.cloud.bigquery.QueryParameterValue)2 HashMap (java.util.HashMap)2 List (java.util.List)2 ColumnConfig (org.pmiops.workbench.config.CdrBigQuerySchemaConfig.ColumnConfig)2 FieldValue (com.google.cloud.bigquery.FieldValue)1 QueryJobConfiguration (com.google.cloud.bigquery.QueryJobConfiguration)1 Joiner (com.google.common.base.Joiner)1 Strings (com.google.common.base.Strings)1 ParseException (java.text.ParseException)1 SimpleDateFormat (java.text.SimpleDateFormat)1 Map (java.util.Map)1 Collectors (java.util.stream.Collectors)1