Search in sources :

Example 16 with TestingTypeManager

use of io.trino.spi.type.TestingTypeManager in project trino by trinodb.

the class TestSortedRangeSet method testJsonSerialization.

@Test
public void testJsonSerialization() throws Exception {
    TestingTypeManager typeManager = new TestingTypeManager();
    TestingBlockEncodingSerde blockEncodingSerde = new TestingBlockEncodingSerde();
    ObjectMapper mapper = new ObjectMapperProvider().get().registerModule(new SimpleModule().addDeserializer(Type.class, new TestingTypeDeserializer(typeManager)).addSerializer(Block.class, new TestingBlockJsonSerde.Serializer(blockEncodingSerde)).addDeserializer(Block.class, new TestingBlockJsonSerde.Deserializer(blockEncodingSerde)));
    SortedRangeSet set = SortedRangeSet.all(BIGINT);
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), SortedRangeSet.class));
    set = SortedRangeSet.none(DOUBLE);
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), SortedRangeSet.class));
    set = SortedRangeSet.of(VARCHAR, utf8Slice("abc"));
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), SortedRangeSet.class));
    set = SortedRangeSet.of(Range.equal(BOOLEAN, true), Range.equal(BOOLEAN, false));
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), SortedRangeSet.class));
}
Also used : Type(io.trino.spi.type.Type) DecimalType.createDecimalType(io.trino.spi.type.DecimalType.createDecimalType) VarcharType.createVarcharType(io.trino.spi.type.VarcharType.createVarcharType) TestingTypeDeserializer(io.trino.spi.type.TestingTypeDeserializer) TestingBlockEncodingSerde(io.trino.spi.block.TestingBlockEncodingSerde) TestingTypeDeserializer(io.trino.spi.type.TestingTypeDeserializer) Block(io.trino.spi.block.Block) TestingBlockJsonSerde(io.trino.spi.block.TestingBlockJsonSerde) TestingTypeManager(io.trino.spi.type.TestingTypeManager) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SimpleModule(com.fasterxml.jackson.databind.module.SimpleModule) ObjectMapperProvider(io.airlift.json.ObjectMapperProvider) Test(org.testng.annotations.Test)

Example 17 with TestingTypeManager

use of io.trino.spi.type.TestingTypeManager in project trino by trinodb.

the class TestTupleDomain method testJsonSerialization.

@Test
public void testJsonSerialization() throws Exception {
    TestingTypeManager typeManager = new TestingTypeManager();
    TestingBlockEncodingSerde blockEncodingSerde = new TestingBlockEncodingSerde();
    ObjectMapper mapper = new ObjectMapperProvider().get().registerModule(new SimpleModule().addDeserializer(ColumnHandle.class, new JsonDeserializer<>() {

        @Override
        public ColumnHandle deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException {
            return new ObjectMapperProvider().get().readValue(jsonParser, TestingColumnHandle.class);
        }
    }).addDeserializer(Type.class, new TestingTypeDeserializer(typeManager)).addSerializer(Block.class, new TestingBlockJsonSerde.Serializer(blockEncodingSerde)).addDeserializer(Block.class, new TestingBlockJsonSerde.Deserializer(blockEncodingSerde)));
    TupleDomain<ColumnHandle> tupleDomain = TupleDomain.all();
    assertEquals(tupleDomain, mapper.readValue(mapper.writeValueAsString(tupleDomain), new TypeReference<TupleDomain<ColumnHandle>>() {
    }));
    tupleDomain = TupleDomain.none();
    assertEquals(tupleDomain, mapper.readValue(mapper.writeValueAsString(tupleDomain), new TypeReference<TupleDomain<ColumnHandle>>() {
    }));
    tupleDomain = TupleDomain.fromFixedValues(ImmutableMap.of(A, NullableValue.of(BIGINT, 1L), B, NullableValue.asNull(VARCHAR)));
    assertEquals(tupleDomain, mapper.readValue(mapper.writeValueAsString(tupleDomain), new TypeReference<TupleDomain<ColumnHandle>>() {
    }));
}
Also used : TestingColumnHandle(io.trino.spi.connector.TestingColumnHandle) ColumnHandle(io.trino.spi.connector.ColumnHandle) TestingBlockEncodingSerde(io.trino.spi.block.TestingBlockEncodingSerde) IOException(java.io.IOException) ObjectMapperProvider(io.airlift.json.ObjectMapperProvider) TestingColumnHandle(io.trino.spi.connector.TestingColumnHandle) Type(io.trino.spi.type.Type) TestingTypeDeserializer(io.trino.spi.type.TestingTypeDeserializer) TestingTypeDeserializer(io.trino.spi.type.TestingTypeDeserializer) JsonDeserializer(com.fasterxml.jackson.databind.JsonDeserializer) DeserializationContext(com.fasterxml.jackson.databind.DeserializationContext) Block(io.trino.spi.block.Block) TestingBlockJsonSerde(io.trino.spi.block.TestingBlockJsonSerde) TypeReference(com.fasterxml.jackson.core.type.TypeReference) TestingTypeManager(io.trino.spi.type.TestingTypeManager) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SimpleModule(com.fasterxml.jackson.databind.module.SimpleModule) JsonParser(com.fasterxml.jackson.core.JsonParser) Test(org.testng.annotations.Test)

Example 18 with TestingTypeManager

use of io.trino.spi.type.TestingTypeManager in project trino by trinodb.

the class TestDeltaLakePageSink method createPageSink.

private static ConnectorPageSink createPageSink(Path outputPath, DeltaLakeWriterStats stats) {
    HiveTransactionHandle transaction = new HiveTransactionHandle(false);
    DeltaLakeConfig deltaLakeConfig = new DeltaLakeConfig();
    DeltaLakeOutputTableHandle tableHandle = new DeltaLakeOutputTableHandle(SCHEMA_NAME, TABLE_NAME, getColumnHandles(), outputPath.toString(), Optional.of(deltaLakeConfig.getDefaultCheckpointWritingInterval()), true);
    DeltaLakePageSinkProvider provider = new DeltaLakePageSinkProvider(new GroupByHashPageIndexerFactory(new JoinCompiler(new TypeOperators()), new BlockTypeOperators()), HDFS_ENVIRONMENT, JsonCodec.jsonCodec(DataFileInfo.class), stats, deltaLakeConfig, new TestingTypeManager(), new NodeVersion("test-version"));
    return provider.createPageSink(transaction, SESSION, tableHandle);
}
Also used : NodeVersion(io.trino.plugin.hive.NodeVersion) JoinCompiler(io.trino.sql.gen.JoinCompiler) BlockTypeOperators(io.trino.type.BlockTypeOperators) HiveTransactionHandle(io.trino.plugin.hive.HiveTransactionHandle) GroupByHashPageIndexerFactory(io.trino.operator.GroupByHashPageIndexerFactory) TestingTypeManager(io.trino.spi.type.TestingTypeManager) TypeOperators(io.trino.spi.type.TypeOperators) BlockTypeOperators(io.trino.type.BlockTypeOperators)

Example 19 with TestingTypeManager

use of io.trino.spi.type.TestingTypeManager in project trino by trinodb.

the class TestRecordingHiveMetastore method createJsonCodec.

private JsonCodec<HiveMetastoreRecording.Recording> createJsonCodec() {
    ObjectMapperProvider objectMapperProvider = new ObjectMapperProvider();
    TypeDeserializer typeDeserializer = new TypeDeserializer(new TestingTypeManager());
    objectMapperProvider.setJsonDeserializers(ImmutableMap.of(Block.class, new TestingBlockJsonSerde.Deserializer(new HiveBlockEncodingSerde()), Type.class, typeDeserializer));
    objectMapperProvider.setJsonSerializers(ImmutableMap.of(Block.class, new TestingBlockJsonSerde.Serializer(new HiveBlockEncodingSerde())));
    JsonCodec<HiveMetastoreRecording.Recording> jsonCodec = new JsonCodecFactory(objectMapperProvider).jsonCodec(HiveMetastoreRecording.Recording.class);
    return jsonCodec;
}
Also used : ColumnStatisticType(io.trino.spi.statistics.ColumnStatisticType) Type(io.trino.spi.type.Type) VarcharType.createUnboundedVarcharType(io.trino.spi.type.VarcharType.createUnboundedVarcharType) HiveType(io.trino.plugin.hive.HiveType) VarcharType.createVarcharType(io.trino.spi.type.VarcharType.createVarcharType) HiveBlockEncodingSerde(io.trino.plugin.hive.util.HiveBlockEncodingSerde) TypeDeserializer(io.trino.plugin.base.TypeDeserializer) Block(io.trino.spi.block.Block) TypeDeserializer(io.trino.plugin.base.TypeDeserializer) JsonCodecFactory(io.airlift.json.JsonCodecFactory) TestingTypeManager(io.trino.spi.type.TestingTypeManager) ObjectMapperProvider(io.airlift.json.ObjectMapperProvider)

Example 20 with TestingTypeManager

use of io.trino.spi.type.TestingTypeManager in project trino by trinodb.

the class TestAvroSchemaConverter method testEmptyFieldStrategy.

@Test
public void testEmptyFieldStrategy() {
    Schema schema = SchemaBuilder.record(RECORD_NAME).fields().name("my_int").type().intType().noDefault().name("my_record").type().record("nested_record").fields().endRecord().noDefault().name("my_array").type().array().items().type("nested_record").noDefault().name("my_map").type().map().values().type("nested_record").noDefault().endRecord();
    List<Type> typesForIgnoreStrategy = ImmutableList.<Type>builder().add(INTEGER).build();
    assertEquals(new AvroSchemaConverter(new TestingTypeManager(), IGNORE).convertAvroSchema(schema), typesForIgnoreStrategy);
    assertThatThrownBy(() -> new AvroSchemaConverter(new TestingTypeManager(), FAIL).convertAvroSchema(schema)).isInstanceOf(IllegalStateException.class).hasMessage("Struct type has no valid fields for schema: '%s'", SchemaBuilder.record("nested_record").fields().endRecord());
    List<Type> typesForAddDummyStrategy = ImmutableList.<Type>builder().add(INTEGER).add(RowType.from(ImmutableList.<RowType.Field>builder().add(new RowType.Field(Optional.of(DUMMY_FIELD_NAME), BOOLEAN)).build())).add(new ArrayType(RowType.from(ImmutableList.<RowType.Field>builder().add(new RowType.Field(Optional.of(DUMMY_FIELD_NAME), BOOLEAN)).build()))).add(createType(RowType.from(ImmutableList.<RowType.Field>builder().add(new RowType.Field(Optional.of(DUMMY_FIELD_NAME), BOOLEAN)).build()))).build();
    assertEquals(new AvroSchemaConverter(new TestingTypeManager(), ADD_DUMMY).convertAvroSchema(schema), typesForAddDummyStrategy);
}
Also used : ArrayType(io.trino.spi.type.ArrayType) Type(io.trino.spi.type.Type) RowType(io.trino.spi.type.RowType) MapType(io.trino.spi.type.MapType) ArrayType(io.trino.spi.type.ArrayType) Schema(org.apache.avro.Schema) RowType(io.trino.spi.type.RowType) TestingTypeManager(io.trino.spi.type.TestingTypeManager) Test(org.testng.annotations.Test)

Aggregations

TestingTypeManager (io.trino.spi.type.TestingTypeManager)21 Type (io.trino.spi.type.Type)14 Test (org.testng.annotations.Test)14 ObjectMapperProvider (io.airlift.json.ObjectMapperProvider)8 ArrayType (io.trino.spi.type.ArrayType)7 MapType (io.trino.spi.type.MapType)6 RowType (io.trino.spi.type.RowType)6 TestingTypeDeserializer (io.trino.spi.type.TestingTypeDeserializer)6 Schema (org.apache.avro.Schema)6 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)5 SimpleModule (com.fasterxml.jackson.databind.module.SimpleModule)5 Block (io.trino.spi.block.Block)5 CatalogName (io.trino.plugin.base.CatalogName)4 FileMetastoreTableOperationsProvider (io.trino.plugin.iceberg.catalog.file.FileMetastoreTableOperationsProvider)4 TrinoHiveCatalog (io.trino.plugin.iceberg.catalog.hms.TrinoHiveCatalog)4 TestingBlockEncodingSerde (io.trino.spi.block.TestingBlockEncodingSerde)4 TestingBlockJsonSerde (io.trino.spi.block.TestingBlockJsonSerde)4 JsonCodecFactory (io.airlift.json.JsonCodecFactory)3 HdfsConfig (io.trino.plugin.hive.HdfsConfig)3 HdfsConfiguration (io.trino.plugin.hive.HdfsConfiguration)3