Search in sources :

Example 6 with StringableMap

use of org.apache.hadoop.hive.common.StringableMap in project hive by apache.

the class CompactorUtil method getCompactorJobQueueName.

/**
 * Get the compactor queue name if it's defined.
 * @param conf global hive conf
 * @param ci compaction info object
 * @param table instance of table
 * @return name of the queue
 */
static String getCompactorJobQueueName(HiveConf conf, CompactionInfo ci, Table table) {
    // Get queue name from the ci. This is passed through
    // ALTER TABLE table_name COMPACT 'major' WITH OVERWRITE TBLPROPERTIES('compactor.hive.compactor.job.queue'='some_queue')
    List<Function<String, String>> propertyGetters = new ArrayList<>(2);
    if (ci.properties != null) {
        StringableMap ciProperties = new StringableMap(ci.properties);
        propertyGetters.add(ciProperties::get);
    }
    if (table.getParameters() != null) {
        propertyGetters.add(table.getParameters()::get);
    }
    for (Function<String, String> getter : propertyGetters) {
        for (String p : QUEUE_PROPERTIES) {
            String queueName = getter.apply(p);
            if (queueName != null && !queueName.isEmpty()) {
                return queueName;
            }
        }
    }
    return conf.getVar(HiveConf.ConfVars.COMPACTOR_JOB_QUEUE);
}
Also used : Function(java.util.function.Function) ArrayList(java.util.ArrayList) StringableMap(org.apache.hadoop.hive.common.StringableMap)

Aggregations

StringableMap (org.apache.hadoop.hive.common.StringableMap)6 JobConf (org.apache.hadoop.mapred.JobConf)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)1 Function (java.util.function.Function)1 ValidReaderWriteIdList (org.apache.hadoop.hive.common.ValidReaderWriteIdList)1 HiveConf (org.apache.hadoop.hive.conf.HiveConf)1 Table (org.apache.hadoop.hive.metastore.api.Table)1 CompactionInfo (org.apache.hadoop.hive.metastore.txn.CompactionInfo)1 Test (org.junit.Test)1 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)1 MethodSource (org.junit.jupiter.params.provider.MethodSource)1