Search in sources :

Example 81 with Timestamp

use of org.apache.hadoop.hive.common.type.Timestamp in project hive by apache.

the class TestParquetTimestampsHive2Compatibility method testWriteHive2ReadHive4UsingLegacyConversion.

/**
 * Tests that timestamps written using Hive2 APIs are read correctly by Hive4 APIs when legacy conversion is on.
 */
@ParameterizedTest(name = "{0}")
@MethodSource("generateTimestamps")
void testWriteHive2ReadHive4UsingLegacyConversion(String timestampString) {
    NanoTime nt = writeHive2(timestampString);
    Timestamp ts = readHive4(nt, TimeZone.getDefault().getID(), true);
    assertEquals(timestampString, ts.toString());
}
Also used : NanoTime(org.apache.hadoop.hive.ql.io.parquet.timestamp.NanoTime) Timestamp(org.apache.hadoop.hive.common.type.Timestamp) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 82 with Timestamp

use of org.apache.hadoop.hive.common.type.Timestamp in project hive by apache.

the class TestParquetTimestampsHive2Compatibility method testWriteHive4ReadHive4UsingNewConversion.

/**
 * Tests that timestamps written using Hive4 APIs are read correctly by Hive4 APIs when legacy conversion is off.
 */
@ParameterizedTest(name = "{0}")
@MethodSource("generateTimestamps")
void testWriteHive4ReadHive4UsingNewConversion(String timestampString) {
    String zoneId = "US/Pacific";
    NanoTime nt = writeHive4(timestampString, zoneId, false);
    Timestamp ts = readHive4(nt, zoneId, false);
    assertEquals(timestampString, ts.toString());
}
Also used : NanoTime(org.apache.hadoop.hive.ql.io.parquet.timestamp.NanoTime) Timestamp(org.apache.hadoop.hive.common.type.Timestamp) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 83 with Timestamp

use of org.apache.hadoop.hive.common.type.Timestamp in project hive by apache.

the class TestETypeConverter method testGetSmallBigIntConverter.

@Test
public void testGetSmallBigIntConverter() {
    Timestamp timestamp = Timestamp.valueOf("1998-10-03 09:58:31.231");
    long msTime = timestamp.toEpochMilli();
    ByteBuffer buf = ByteBuffer.allocate(12);
    buf.order(ByteOrder.LITTLE_ENDIAN);
    buf.putLong(msTime);
    buf.flip();
    // Need TimeStamp logicalType annotation here
    PrimitiveType primitiveType = createInt64TimestampType(false, TimeUnit.MILLIS);
    Writable writable = getWritableFromBinaryConverter(createHiveTypeInfo("bigint"), primitiveType, Binary.fromByteBuffer(buf));
    // Retrieve as BigInt
    LongWritable longWritable = (LongWritable) writable;
    assertEquals(msTime, longWritable.get());
}
Also used : Writable(org.apache.hadoop.io.Writable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) LongWritable(org.apache.hadoop.io.LongWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) IntWritable(org.apache.hadoop.io.IntWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) HiveDecimalWritable(org.apache.hadoop.hive.serde2.io.HiveDecimalWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) PrimitiveType(org.apache.parquet.schema.PrimitiveType) LongWritable(org.apache.hadoop.io.LongWritable) Timestamp(org.apache.hadoop.hive.common.type.Timestamp) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Example 84 with Timestamp

use of org.apache.hadoop.hive.common.type.Timestamp in project hive by apache.

the class TestETypeConverter method testGetInt64MillisTimestampConverter.

@Test
public void testGetInt64MillisTimestampConverter() throws Exception {
    Timestamp timestamp = Timestamp.valueOf("2018-07-15 15:12:20.112");
    PrimitiveType primitiveType = createInt64TimestampType(false, TimeUnit.MILLIS);
    Writable writable = getWritableFromPrimitiveConverter(null, primitiveType, timestamp.toEpochMilli());
    TimestampWritableV2 timestampWritable = (TimestampWritableV2) writable;
    assertEquals(timestamp.toEpochMilli(), timestampWritable.getTimestamp().toEpochMilli());
}
Also used : Writable(org.apache.hadoop.io.Writable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) LongWritable(org.apache.hadoop.io.LongWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) IntWritable(org.apache.hadoop.io.IntWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) HiveDecimalWritable(org.apache.hadoop.hive.serde2.io.HiveDecimalWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) PrimitiveType(org.apache.parquet.schema.PrimitiveType) Timestamp(org.apache.hadoop.hive.common.type.Timestamp) TimestampWritableV2(org.apache.hadoop.hive.serde2.io.TimestampWritableV2) Test(org.junit.Test)

Example 85 with Timestamp

use of org.apache.hadoop.hive.common.type.Timestamp in project hive by apache.

the class TestETypeConverter method testGetInt64NanosTimestampConverter.

@Test
public void testGetInt64NanosTimestampConverter() throws Exception {
    Timestamp timestamp = Timestamp.valueOf("2018-07-15 15:12:20.11223344");
    PrimitiveType primitiveType = createInt64TimestampType(false, TimeUnit.NANOS);
    long time = timestamp.toEpochSecond() * 1000000000 + timestamp.getNanos();
    Writable writable = getWritableFromPrimitiveConverter(null, primitiveType, time);
    TimestampWritableV2 timestampWritable = (TimestampWritableV2) writable;
    assertEquals(timestamp.toEpochMilli(), timestampWritable.getTimestamp().toEpochMilli());
    assertEquals(timestamp.getNanos(), timestampWritable.getNanos());
}
Also used : Writable(org.apache.hadoop.io.Writable) DoubleWritable(org.apache.hadoop.io.DoubleWritable) LongWritable(org.apache.hadoop.io.LongWritable) BytesWritable(org.apache.hadoop.io.BytesWritable) IntWritable(org.apache.hadoop.io.IntWritable) BooleanWritable(org.apache.hadoop.io.BooleanWritable) HiveDecimalWritable(org.apache.hadoop.hive.serde2.io.HiveDecimalWritable) FloatWritable(org.apache.hadoop.io.FloatWritable) PrimitiveType(org.apache.parquet.schema.PrimitiveType) Timestamp(org.apache.hadoop.hive.common.type.Timestamp) TimestampWritableV2(org.apache.hadoop.hive.serde2.io.TimestampWritableV2) Test(org.junit.Test)

Aggregations

Timestamp (org.apache.hadoop.hive.common.type.Timestamp)116 Test (org.junit.Test)36 TimestampWritableV2 (org.apache.hadoop.hive.serde2.io.TimestampWritableV2)32 Date (org.apache.hadoop.hive.common.type.Date)27 BytesWritable (org.apache.hadoop.io.BytesWritable)25 LongWritable (org.apache.hadoop.io.LongWritable)25 HiveDecimalWritable (org.apache.hadoop.hive.serde2.io.HiveDecimalWritable)24 Text (org.apache.hadoop.io.Text)22 HiveDecimal (org.apache.hadoop.hive.common.type.HiveDecimal)21 IntWritable (org.apache.hadoop.io.IntWritable)21 HiveChar (org.apache.hadoop.hive.common.type.HiveChar)20 HiveVarchar (org.apache.hadoop.hive.common.type.HiveVarchar)20 BooleanWritable (org.apache.hadoop.io.BooleanWritable)19 FloatWritable (org.apache.hadoop.io.FloatWritable)19 PrimitiveTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo)18 ArrayList (java.util.ArrayList)17 DateWritableV2 (org.apache.hadoop.hive.serde2.io.DateWritableV2)17 HiveIntervalDayTime (org.apache.hadoop.hive.common.type.HiveIntervalDayTime)16 List (java.util.List)15 ByteWritable (org.apache.hadoop.hive.serde2.io.ByteWritable)12