Search in sources :

Example 1 with TestingTypeManager

use of com.facebook.presto.common.type.TestingTypeManager in project presto by prestodb.

the class TestRange method testJsonSerialization.

@Test
public void testJsonSerialization() throws Exception {
    TestingTypeManager typeManager = new TestingTypeManager();
    TestingBlockEncodingSerde blockEncodingSerde = new TestingBlockEncodingSerde();
    ObjectMapper mapper = new JsonObjectMapperProvider().get().registerModule(new SimpleModule().addDeserializer(Type.class, new TestingTypeDeserializer(typeManager)).addSerializer(Block.class, new TestingBlockJsonSerde.Serializer(blockEncodingSerde)).addDeserializer(Block.class, new TestingBlockJsonSerde.Deserializer(blockEncodingSerde)));
    Range range = Range.all(BIGINT);
    assertEquals(range, mapper.readValue(mapper.writeValueAsString(range), Range.class));
    range = Range.equal(DOUBLE, 0.123);
    assertEquals(range, mapper.readValue(mapper.writeValueAsString(range), Range.class));
    range = Range.greaterThan(BIGINT, 0L);
    assertEquals(range, mapper.readValue(mapper.writeValueAsString(range), Range.class));
    range = Range.greaterThanOrEqual(VARCHAR, utf8Slice("abc"));
    assertEquals(range, mapper.readValue(mapper.writeValueAsString(range), Range.class));
    range = Range.lessThan(BIGINT, Long.MAX_VALUE);
    assertEquals(range, mapper.readValue(mapper.writeValueAsString(range), Range.class));
    range = Range.lessThanOrEqual(DOUBLE, Double.MAX_VALUE);
    assertEquals(range, mapper.readValue(mapper.writeValueAsString(range), Range.class));
}
Also used : Type(com.facebook.presto.common.type.Type) TestingTypeDeserializer(com.facebook.presto.common.type.TestingTypeDeserializer) TestingBlockEncodingSerde(com.facebook.presto.common.block.TestingBlockEncodingSerde) JsonObjectMapperProvider(com.facebook.airlift.json.JsonObjectMapperProvider) TestingTypeDeserializer(com.facebook.presto.common.type.TestingTypeDeserializer) Block(com.facebook.presto.common.block.Block) TestingBlockJsonSerde(com.facebook.presto.common.block.TestingBlockJsonSerde) TestingTypeManager(com.facebook.presto.common.type.TestingTypeManager) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SimpleModule(com.fasterxml.jackson.databind.module.SimpleModule) Test(org.testng.annotations.Test)

Example 2 with TestingTypeManager

use of com.facebook.presto.common.type.TestingTypeManager in project presto by prestodb.

the class TestEquatableValueSet method testJsonSerialization.

@Test
public void testJsonSerialization() throws Exception {
    TestingTypeManager typeManager = new TestingTypeManager();
    TestingBlockEncodingSerde blockEncodingSerde = new TestingBlockEncodingSerde();
    ObjectMapper mapper = new JsonObjectMapperProvider().get().registerModule(new SimpleModule().addDeserializer(Type.class, new TestingTypeDeserializer(typeManager)).addSerializer(Block.class, new TestingBlockJsonSerde.Serializer(blockEncodingSerde)).addDeserializer(Block.class, new TestingBlockJsonSerde.Deserializer(blockEncodingSerde)));
    EquatableValueSet set = EquatableValueSet.all(TestingIdType.ID);
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), EquatableValueSet.class));
    set = EquatableValueSet.none(TestingIdType.ID);
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), EquatableValueSet.class));
    set = EquatableValueSet.of(TestingIdType.ID, 1L);
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), EquatableValueSet.class));
    set = EquatableValueSet.of(TestingIdType.ID, 1L, 2L);
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), EquatableValueSet.class));
    set = EquatableValueSet.of(TestingIdType.ID, 1L, 2L).complement();
    assertEquals(set, mapper.readValue(mapper.writeValueAsString(set), EquatableValueSet.class));
}
Also used : TestingIdType(com.facebook.presto.common.type.TestingIdType) Type(com.facebook.presto.common.type.Type) TestingTypeDeserializer(com.facebook.presto.common.type.TestingTypeDeserializer) TestingBlockEncodingSerde(com.facebook.presto.common.block.TestingBlockEncodingSerde) JsonObjectMapperProvider(com.facebook.airlift.json.JsonObjectMapperProvider) TestingTypeDeserializer(com.facebook.presto.common.type.TestingTypeDeserializer) Block(com.facebook.presto.common.block.Block) TestingBlockJsonSerde(com.facebook.presto.common.block.TestingBlockJsonSerde) TestingTypeManager(com.facebook.presto.common.type.TestingTypeManager) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SimpleModule(com.fasterxml.jackson.databind.module.SimpleModule) Test(org.testng.annotations.Test)

Example 3 with TestingTypeManager

use of com.facebook.presto.common.type.TestingTypeManager in project presto by prestodb.

the class TestMarker method testJsonSerialization.

@Test
public void testJsonSerialization() throws Exception {
    TestingTypeManager typeManager = new TestingTypeManager();
    TestingBlockEncodingSerde blockEncodingSerde = new TestingBlockEncodingSerde();
    ObjectMapper mapper = new JsonObjectMapperProvider().get().registerModule(new SimpleModule().addDeserializer(Type.class, new TestingTypeDeserializer(typeManager)).addSerializer(Block.class, new TestingBlockJsonSerde.Serializer(blockEncodingSerde)).addDeserializer(Block.class, new TestingBlockJsonSerde.Deserializer(blockEncodingSerde)));
    Marker marker = Marker.above(BIGINT, 0L);
    assertEquals(marker, mapper.readValue(mapper.writeValueAsString(marker), Marker.class));
    marker = Marker.exactly(VARCHAR, utf8Slice("abc"));
    assertEquals(marker, mapper.readValue(mapper.writeValueAsString(marker), Marker.class));
    marker = Marker.below(DOUBLE, 0.123);
    assertEquals(marker, mapper.readValue(mapper.writeValueAsString(marker), Marker.class));
    marker = Marker.exactly(BOOLEAN, true);
    assertEquals(marker, mapper.readValue(mapper.writeValueAsString(marker), Marker.class));
    marker = Marker.upperUnbounded(BIGINT);
    assertEquals(marker, mapper.readValue(mapper.writeValueAsString(marker), Marker.class));
    marker = Marker.lowerUnbounded(BIGINT);
    assertEquals(marker, mapper.readValue(mapper.writeValueAsString(marker), Marker.class));
}
Also used : Type(com.facebook.presto.common.type.Type) TestingTypeDeserializer(com.facebook.presto.common.type.TestingTypeDeserializer) TestingBlockEncodingSerde(com.facebook.presto.common.block.TestingBlockEncodingSerde) JsonObjectMapperProvider(com.facebook.airlift.json.JsonObjectMapperProvider) TestingTypeDeserializer(com.facebook.presto.common.type.TestingTypeDeserializer) Block(com.facebook.presto.common.block.Block) TestingBlockJsonSerde(com.facebook.presto.common.block.TestingBlockJsonSerde) TestingTypeManager(com.facebook.presto.common.type.TestingTypeManager) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) SimpleModule(com.fasterxml.jackson.databind.module.SimpleModule) Test(org.testng.annotations.Test)

Example 4 with TestingTypeManager

use of com.facebook.presto.common.type.TestingTypeManager in project presto by prestodb.

the class TestHiveMetadata method testColumnMetadataGetter.

@Test
public void testColumnMetadataGetter() {
    TypeManager mockTypeManager = new TestingTypeManager();
    Column column1 = new Column("c1", HIVE_INT, Optional.empty(), Optional.of("some-metadata"));
    HiveColumnHandle hiveColumnHandle1 = new HiveColumnHandle(column1.getName(), HiveType.HIVE_INT, TypeSignature.parseTypeSignature("int"), 0, HiveColumnHandle.ColumnType.REGULAR, Optional.empty(), Optional.empty());
    HiveColumnHandle hidden = new HiveColumnHandle(HiveColumnHandle.PATH_COLUMN_NAME, HiveType.HIVE_INT, TypeSignature.parseTypeSignature("int"), 0, HiveColumnHandle.ColumnType.SYNTHESIZED, Optional.empty(), Optional.empty());
    Column partitionColumn = new Column("ds", HIVE_STRING, Optional.empty(), Optional.empty());
    Table mockTable = new Table("schema", "table", "user", PrestoTableType.MANAGED_TABLE, new Storage(fromHiveStorageFormat(ORC), "location", Optional.of(new HiveBucketProperty(ImmutableList.of(column1.getName()), 100, ImmutableList.of(), HIVE_COMPATIBLE, Optional.empty())), false, ImmutableMap.of(), ImmutableMap.of()), ImmutableList.of(column1), ImmutableList.of(partitionColumn), ImmutableMap.of(), Optional.empty(), Optional.empty());
    ColumnMetadata actual = HiveMetadata.columnMetadataGetter(mockTable, mockTypeManager, new HiveColumnConverter()).apply(hiveColumnHandle1);
    ColumnMetadata expected = new ColumnMetadata("c1", IntegerType.INTEGER);
    assertEquals(actual, expected);
    actual = HiveMetadata.columnMetadataGetter(mockTable, mockTypeManager, new TestColumnConverter()).apply(hidden);
    expected = ColumnMetadata.builder().setName(HiveColumnHandle.PATH_COLUMN_NAME).setType(IntegerType.INTEGER).setHidden(true).build();
    assertEquals(actual, expected);
}
Also used : ColumnMetadata(com.facebook.presto.spi.ColumnMetadata) Table(com.facebook.presto.hive.metastore.Table) Storage(com.facebook.presto.hive.metastore.Storage) HiveMetadata.decodePreferredOrderingColumnsFromStorage(com.facebook.presto.hive.HiveMetadata.decodePreferredOrderingColumnsFromStorage) Column(com.facebook.presto.hive.metastore.Column) SortingColumn(com.facebook.presto.hive.metastore.SortingColumn) TestingTypeManager(com.facebook.presto.common.type.TestingTypeManager) TypeManager(com.facebook.presto.common.type.TypeManager) HiveColumnConverter(com.facebook.presto.hive.metastore.HiveColumnConverter) TestingTypeManager(com.facebook.presto.common.type.TestingTypeManager) Test(org.testng.annotations.Test)

Example 5 with TestingTypeManager

use of com.facebook.presto.common.type.TestingTypeManager in project presto by prestodb.

the class TestHivePageSourceProvider method testNotUseRecordReaderWithInputFormatAnnotationWithoutCustomSplit.

@Test
public void testNotUseRecordReaderWithInputFormatAnnotationWithoutCustomSplit() {
    StorageFormat storageFormat = StorageFormat.create(ParquetHiveSerDe.class.getName(), HoodieParquetInputFormat.class.getName(), "");
    Storage storage = new Storage(storageFormat, "test", Optional.empty(), true, ImmutableMap.of(), ImmutableMap.of());
    HiveRecordCursorProvider recordCursorProvider = new MockHiveRecordCursorProvider();
    HiveBatchPageSourceFactory hiveBatchPageSourceFactory = new MockHiveBatchPageSourceFactory();
    Optional<ConnectorPageSource> pageSource = HivePageSourceProvider.createHivePageSource(ImmutableSet.of(recordCursorProvider), ImmutableSet.of(hiveBatchPageSourceFactory), new Configuration(), new TestingConnectorSession(new HiveSessionProperties(new HiveClientConfig().setUseRecordPageSourceForCustomSplit(true), new OrcFileWriterConfig(), new ParquetFileWriterConfig(), new CacheConfig()).getSessionProperties()), new Path("/test/"), OptionalInt.empty(), 0, 100, 200, Instant.now().toEpochMilli(), storage, TupleDomain.none(), ImmutableList.of(), ImmutableMap.of(), ImmutableList.of(), DateTimeZone.UTC, new TestingTypeManager(), new SchemaTableName("test", "test"), ImmutableList.of(), ImmutableList.of(), ImmutableMap.of(), 0, TableToPartitionMapping.empty(), Optional.empty(), false, null, null, false, null, Optional.empty(), ImmutableMap.of());
    assertTrue(pageSource.isPresent());
    assertTrue(pageSource.get() instanceof HivePageSource);
}
Also used : Path(org.apache.hadoop.fs.Path) ParquetHiveSerDe(org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe) Configuration(org.apache.hadoop.conf.Configuration) TestingConnectorSession(com.facebook.presto.testing.TestingConnectorSession) StorageFormat(com.facebook.presto.hive.metastore.StorageFormat) HoodieParquetInputFormat(org.apache.hudi.hadoop.HoodieParquetInputFormat) ConnectorPageSource(com.facebook.presto.spi.ConnectorPageSource) SchemaTableName(com.facebook.presto.spi.SchemaTableName) Storage(com.facebook.presto.hive.metastore.Storage) CacheConfig(com.facebook.presto.cache.CacheConfig) TestingTypeManager(com.facebook.presto.common.type.TestingTypeManager) Test(org.testng.annotations.Test)

Aggregations

TestingTypeManager (com.facebook.presto.common.type.TestingTypeManager)21 Test (org.testng.annotations.Test)21 Column (com.facebook.presto.hive.metastore.Column)12 HiveMaterializedViewUtils.getMaterializedDataPredicates (com.facebook.presto.hive.HiveMaterializedViewUtils.getMaterializedDataPredicates)11 MaterializedDataPredicates (com.facebook.presto.spi.MaterializedViewStatus.MaterializedDataPredicates)11 ImmutableList (com.google.common.collect.ImmutableList)11 List (java.util.List)11 JsonObjectMapperProvider (com.facebook.airlift.json.JsonObjectMapperProvider)7 TestingTypeDeserializer (com.facebook.presto.common.type.TestingTypeDeserializer)7 Type (com.facebook.presto.common.type.Type)7 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)7 SimpleModule (com.fasterxml.jackson.databind.module.SimpleModule)7 Block (com.facebook.presto.common.block.Block)6 TestingBlockEncodingSerde (com.facebook.presto.common.block.TestingBlockEncodingSerde)6 TestingBlockJsonSerde (com.facebook.presto.common.block.TestingBlockJsonSerde)6 Storage (com.facebook.presto.hive.metastore.Storage)3 CacheConfig (com.facebook.presto.cache.CacheConfig)2 StorageFormat (com.facebook.presto.hive.metastore.StorageFormat)2 ConnectorPageSource (com.facebook.presto.spi.ConnectorPageSource)2 SchemaTableName (com.facebook.presto.spi.SchemaTableName)2