Search in sources :

Example 6 with ScheduledQueryKey

use of org.apache.hadoop.hive.metastore.api.ScheduledQueryKey in project hive by apache.

the class TestMetastoreScheduledQueries method createKey.

private ScheduledQueryKey createKey(String name, String string) {
    ScheduledQueryKey ret = new ScheduledQueryKey();
    ret.setScheduleName(name);
    ret.setClusterNamespace(string);
    return ret;
}
Also used : ScheduledQueryKey(org.apache.hadoop.hive.metastore.api.ScheduledQueryKey)

Example 7 with ScheduledQueryKey

use of org.apache.hadoop.hive.metastore.api.ScheduledQueryKey in project hive by apache.

the class TestMetastoreScheduledQueries method testUpdate.

@Test
public void testUpdate() throws Exception {
    ScheduledQuery schq = createScheduledQuery(createKey("update", "ns1"));
    ScheduledQueryMaintenanceRequest r = new ScheduledQueryMaintenanceRequest();
    r.setType(ScheduledQueryMaintenanceRequestType.CREATE);
    r.setScheduledQuery(schq);
    client.scheduledQueryMaintenance(r);
    r.setType(ScheduledQueryMaintenanceRequestType.ALTER);
    ScheduledQuery schq2 = createScheduledQuery2(createKey("update", "ns1"));
    schq2.getScheduleKey().setClusterNamespace("ns1");
    r.setScheduledQuery(schq2);
    client.scheduledQueryMaintenance(r);
    ScheduledQuery schq3 = client.getScheduledQuery(new ScheduledQueryKey("update", "ns1"));
    // next execution is set by remote
    schq2.setNextExecution(schq3.getNextExecution());
    assertEquals(schq2, schq3);
}
Also used : ScheduledQueryMaintenanceRequest(org.apache.hadoop.hive.metastore.api.ScheduledQueryMaintenanceRequest) ScheduledQueryKey(org.apache.hadoop.hive.metastore.api.ScheduledQueryKey) ScheduledQuery(org.apache.hadoop.hive.metastore.api.ScheduledQuery) MetastoreUnitTest(org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest) Test(org.junit.Test)

Example 8 with ScheduledQueryKey

use of org.apache.hadoop.hive.metastore.api.ScheduledQueryKey in project hive by apache.

the class TestMetastoreScheduledQueries method testOutdatedCleanup.

@Test
public void testOutdatedCleanup() throws Exception {
    String namespace = "outdatedcleanup";
    ObjectStore objStore = new ObjectStore();
    objStore.setConf(metaStore.getConf());
    objStore.deleteScheduledExecutions(0);
    ScheduledQuery schq = createScheduledQuery(new ScheduledQueryKey("q1", namespace));
    ScheduledQueryMaintenanceRequest r = new ScheduledQueryMaintenanceRequest();
    r.setType(ScheduledQueryMaintenanceRequestType.CREATE);
    r.setScheduledQuery(schq);
    objStore.scheduledQueryMaintenance(r);
    Thread.sleep(1000);
    ScheduledQueryPollRequest request = new ScheduledQueryPollRequest(namespace);
    ScheduledQueryPollResponse pollResult = objStore.scheduledQueryPoll(request);
    // will add q1 as a query being executed
    Thread.sleep(1000);
    objStore.markScheduledExecutionsTimedOut(0);
    try (PersistenceManager pm = PersistenceManagerProvider.getPersistenceManager()) {
        MScheduledExecution execution = pm.getObjectById(MScheduledExecution.class, pollResult.getExecutionId());
        assertEquals(QueryState.TIMED_OUT, execution.getState());
    }
}
Also used : ObjectStore(org.apache.hadoop.hive.metastore.ObjectStore) ScheduledQueryPollRequest(org.apache.hadoop.hive.metastore.api.ScheduledQueryPollRequest) MScheduledExecution(org.apache.hadoop.hive.metastore.model.MScheduledExecution) PersistenceManager(javax.jdo.PersistenceManager) ScheduledQueryPollResponse(org.apache.hadoop.hive.metastore.api.ScheduledQueryPollResponse) ScheduledQueryMaintenanceRequest(org.apache.hadoop.hive.metastore.api.ScheduledQueryMaintenanceRequest) ScheduledQueryKey(org.apache.hadoop.hive.metastore.api.ScheduledQueryKey) ScheduledQuery(org.apache.hadoop.hive.metastore.api.ScheduledQuery) MetastoreUnitTest(org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest) Test(org.junit.Test)

Example 9 with ScheduledQueryKey

use of org.apache.hadoop.hive.metastore.api.ScheduledQueryKey in project hive by apache.

the class MScheduledQuery method toThrift.

public static ScheduledQuery toThrift(MScheduledQuery s) {
    ScheduledQuery ret = new ScheduledQuery();
    ret.setScheduleKey(new ScheduledQueryKey(s.scheduleName, s.clusterNamespace));
    ret.setEnabled(s.enabled);
    ret.setSchedule(s.schedule);
    ret.setUser(s.user);
    ret.setQuery(s.query);
    ret.setNextExecution(s.nextExecution);
    return ret;
}
Also used : ScheduledQueryKey(org.apache.hadoop.hive.metastore.api.ScheduledQueryKey) ScheduledQuery(org.apache.hadoop.hive.metastore.api.ScheduledQuery)

Example 10 with ScheduledQueryKey

use of org.apache.hadoop.hive.metastore.api.ScheduledQueryKey in project hive by apache.

the class TestScheduledQueryStatements method testCreateFromNonDefaultDatabase.

@Test
public void testCreateFromNonDefaultDatabase() throws ParseException, Exception {
    IDriver driver = createDriver();
    driver.run("set role admin");
    driver.run("use asd");
    driver.run("create table tt (a integer)");
    // the scheduled query may reference a table inside the current database
    driver.run("create scheduled query nonDef cron '* * * * * ? *' as select 1 from tt");
    try (CloseableObjectStore os = new CloseableObjectStore(env_setup.getTestCtx().hiveConf)) {
        Optional<MScheduledQuery> sq = os.getMScheduledQuery(new ScheduledQueryKey("nonDef", "hive"));
        assertTrue(sq.isPresent());
        assertEquals("select 1 from `asd`.`tt`", sq.get().toThrift().getQuery());
    }
}
Also used : IDriver(org.apache.hadoop.hive.ql.IDriver) ScheduledQueryKey(org.apache.hadoop.hive.metastore.api.ScheduledQueryKey) MScheduledQuery(org.apache.hadoop.hive.metastore.model.MScheduledQuery) Test(org.junit.Test)

Aggregations

ScheduledQueryKey (org.apache.hadoop.hive.metastore.api.ScheduledQueryKey)17 ScheduledQuery (org.apache.hadoop.hive.metastore.api.ScheduledQuery)11 Test (org.junit.Test)10 MetastoreUnitTest (org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest)7 ScheduledQueryMaintenanceRequest (org.apache.hadoop.hive.metastore.api.ScheduledQueryMaintenanceRequest)7 ScheduledQueryPollResponse (org.apache.hadoop.hive.metastore.api.ScheduledQueryPollResponse)5 PersistenceManager (javax.jdo.PersistenceManager)4 ScheduledQueryPollRequest (org.apache.hadoop.hive.metastore.api.ScheduledQueryPollRequest)4 MScheduledExecution (org.apache.hadoop.hive.metastore.model.MScheduledExecution)4 MScheduledQuery (org.apache.hadoop.hive.metastore.model.MScheduledQuery)4 IDriver (org.apache.hadoop.hive.ql.IDriver)4 ArrayList (java.util.ArrayList)2 ObjectStore (org.apache.hadoop.hive.metastore.ObjectStore)2 ScheduledQueryProgressInfo (org.apache.hadoop.hive.metastore.api.ScheduledQueryProgressInfo)2 CronBuilder (com.cronutils.builder.CronBuilder)1 CronDefinitionBuilder (com.cronutils.model.definition.CronDefinitionBuilder)1 List (java.util.List)1 ExecutorService (java.util.concurrent.ExecutorService)1 JDOObjectNotFoundException (javax.jdo.JDOObjectNotFoundException)1 Query (javax.jdo.Query)1