Examples with TimestampWritable org.apache.hadoop.hive.serde2.io.TimestampWritable used on opensource projects

Search in sources :

Example 71 with TimestampWritable

use of org.apache.hadoop.hive.serde2.io.TimestampWritable in project hive by apache.

the class TestGenericUDFQuarter method runAndVerifyTs.

private void runAndVerifyTs(String str, Integer expResult, GenericUDF udf) throws HiveException {
    DeferredObject valueObj0 = new DeferredJavaObject(str != null ? new TimestampWritable(Timestamp.valueOf(str)) : null);
    DeferredObject[] args = { valueObj0 };
    IntWritable output = (IntWritable) udf.evaluate(args);
    if (expResult == null) {
        assertNull(output);
    } else {
        assertNotNull(output);
        assertEquals("quarter() test ", expResult.intValue(), output.get());
    }
}
Also used : DeferredJavaObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredJavaObject) DeferredObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredObject) TimestampWritable(org.apache.hadoop.hive.serde2.io.TimestampWritable) IntWritable(org.apache.hadoop.io.IntWritable)

Example 72 with TimestampWritable

use of org.apache.hadoop.hive.serde2.io.TimestampWritable in project hive by apache.

the class TestGenericUDFToUnixTimestamp method testTimestamp.

public void testTimestamp() throws HiveException {
    GenericUDFToUnixTimeStamp udf = new GenericUDFToUnixTimeStamp();
    ObjectInspector valueOI = PrimitiveObjectInspectorFactory.writableTimestampObjectInspector;
    ObjectInspector[] arguments = { valueOI };
    udf.initialize(arguments);
    Timestamp ts = Timestamp.valueOf("1970-01-01 00:00:00");
    runAndVerify(udf, new TimestampWritable(ts), new LongWritable(ts.getTime() / 1000));
    ts = Timestamp.valueOf("2001-02-03 01:02:03");
    runAndVerify(udf, new TimestampWritable(ts), new LongWritable(ts.getTime() / 1000));
    // test null values
    runAndVerify(udf, null, null);
}
Also used : ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) TimestampWritable(org.apache.hadoop.hive.serde2.io.TimestampWritable) LongWritable(org.apache.hadoop.io.LongWritable) Timestamp(java.sql.Timestamp)

Example 73 with TimestampWritable

use of org.apache.hadoop.hive.serde2.io.TimestampWritable in project hive by apache.

the class TestGenericUDFTrunc method testTimestampToDateWithYearFormat.

public void testTimestampToDateWithYearFormat() throws HiveException {
    GenericUDFTrunc udf = new GenericUDFTrunc();
    ObjectInspector valueOI0 = PrimitiveObjectInspectorFactory.writableTimestampObjectInspector;
    ObjectInspector valueOI1 = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
    ObjectInspector[] initArgs = { valueOI0, valueOI1 };
    DeferredObject valueObjFmt = new DeferredJavaObject(new Text("YYYY"));
    DeferredObject valueObj0;
    DeferredObject[] evalArgs;
    // test date string
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-01 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-14 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-31 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-02 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-28 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-03 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-28 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-29 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    // test timestamp string
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-01 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-14 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-31 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-02 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-28 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-03 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-28 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-29 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
}
Also used : ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) DeferredJavaObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredJavaObject) DeferredObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredObject) TimestampWritable(org.apache.hadoop.hive.serde2.io.TimestampWritable) Text(org.apache.hadoop.io.Text)

Example 74 with TimestampWritable

use of org.apache.hadoop.hive.serde2.io.TimestampWritable in project hive by apache.

the class TestGenericUDFTrunc method testTimestampToDateWithQuarterFormat.

public void testTimestampToDateWithQuarterFormat() throws HiveException {
    GenericUDFTrunc udf = new GenericUDFTrunc();
    ObjectInspector valueOI0 = PrimitiveObjectInspectorFactory.writableTimestampObjectInspector;
    ObjectInspector valueOI1 = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
    ObjectInspector[] initArgs = { valueOI0, valueOI1 };
    DeferredObject valueObjFmt = new DeferredJavaObject(new Text("Q"));
    DeferredObject valueObj0;
    DeferredObject[] evalArgs;
    // test date string
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-01 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-14 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-31 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-02 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-28 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-03 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-28 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-29 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-05-11 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-04-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-07-01 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-07-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-12-31 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-10-01", udf, initArgs, evalArgs);
    // test timestamp string
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-01 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-14 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-31 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-02 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-28 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-03 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-28 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-29 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-05-11 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-04-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-07-01 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-07-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-12-31 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-10-01", udf, initArgs, evalArgs);
}
Also used : ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) DeferredJavaObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredJavaObject) DeferredObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredObject) TimestampWritable(org.apache.hadoop.hive.serde2.io.TimestampWritable) Text(org.apache.hadoop.io.Text)

Example 75 with TimestampWritable

use of org.apache.hadoop.hive.serde2.io.TimestampWritable in project hive by apache.

the class TestGenericUDFTrunc method testTimestampToDateWithMonthFormat.

public void testTimestampToDateWithMonthFormat() throws HiveException {
    GenericUDFTrunc udf = new GenericUDFTrunc();
    ObjectInspector valueOI0 = PrimitiveObjectInspectorFactory.writableTimestampObjectInspector;
    ObjectInspector valueOI1 = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
    ObjectInspector[] initArgs = { valueOI0, valueOI1 };
    DeferredObject valueObjFmt = new DeferredJavaObject(new Text("MON"));
    DeferredObject valueObj0;
    DeferredObject[] evalArgs;
    // test date string
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-01 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-14 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-31 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-02 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-28 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-03 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-28 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-29 00:00:00")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-02-01", udf, initArgs, evalArgs);
    // test timestamp string
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-01 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-14 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-01-31 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-01-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-02 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2014-02-28 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2014-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-03 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-28 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-02-01", udf, initArgs, evalArgs);
    valueObj0 = new DeferredJavaObject(new TimestampWritable(Timestamp.valueOf("2016-02-29 10:30:45")));
    evalArgs = new DeferredObject[] { valueObj0, valueObjFmt };
    runAndVerify("2016-02-01", udf, initArgs, evalArgs);
}
Also used : ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) DeferredJavaObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredJavaObject) DeferredObject(org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredObject) TimestampWritable(org.apache.hadoop.hive.serde2.io.TimestampWritable) Text(org.apache.hadoop.io.Text)

Aggregations

TimestampWritable (org.apache.hadoop.hive.serde2.io.TimestampWritable)87 IntWritable (org.apache.hadoop.io.IntWritable)41 Text (org.apache.hadoop.io.Text)34 DateWritable (org.apache.hadoop.hive.serde2.io.DateWritable)33 Timestamp (java.sql.Timestamp)30 HiveDecimalWritable (org.apache.hadoop.hive.serde2.io.HiveDecimalWritable)29 LongWritable (org.apache.hadoop.io.LongWritable)28 DoubleWritable (org.apache.hadoop.hive.serde2.io.DoubleWritable)25 ShortWritable (org.apache.hadoop.hive.serde2.io.ShortWritable)25 BooleanWritable (org.apache.hadoop.io.BooleanWritable)25 FloatWritable (org.apache.hadoop.io.FloatWritable)25 ByteWritable (org.apache.hadoop.hive.serde2.io.ByteWritable)24 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)24 HiveIntervalDayTimeWritable (org.apache.hadoop.hive.serde2.io.HiveIntervalDayTimeWritable)23 BytesWritable (org.apache.hadoop.io.BytesWritable)23 DeferredJavaObject (org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredJavaObject)20 DeferredObject (org.apache.hadoop.hive.ql.udf.generic.GenericUDF.DeferredObject)20 HiveVarcharWritable (org.apache.hadoop.hive.serde2.io.HiveVarcharWritable)20 HiveIntervalYearMonthWritable (org.apache.hadoop.hive.serde2.io.HiveIntervalYearMonthWritable)19 Test (org.junit.Test)19
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial