Search in sources :

Example 1 with JobTableRow

use of com.airbnb.airpal.sql.beans.JobTableRow in project airpal by airbnb.

the class JobHistoryStoreDAO method addRun.

@Override
public void addRun(Job job) {
    JobDAO jobDAO = dbi.onDemand(JobDAO.class);
    TableDAO tableDAO = dbi.onDemand(TableDAO.class);
    JobTableDAO jobTableDAO = dbi.onDemand(JobTableDAO.class);
    JobOutputDAO jobOutputDAO = dbi.onDemand(JobOutputDAO.class);
    // Create the job
    long jobId = jobDAO.createJob(job);
    // Find all presto tables already represented
    Set<TableRow> tablesInDb = Collections.emptySet();
    if (job.getTablesUsed().size() > 0) {
        tablesInDb = new HashSet<>(tableDAO.getTables(new ArrayList<>(job.getTablesUsed())));
    }
    // Figure out which tables are not represented
    Sets.SetView<Table> tablesToAdd = Sets.difference(job.getTablesUsed(), Sets.newHashSet(Iterables.transform(tablesInDb, TableRow.MAP_TO_TABLE)));
    // Add tables not already represented
    tableDAO.createTables(tablesToAdd);
    Set<TableRow> tablesWithIds = Collections.emptySet();
    if (job.getTablesUsed().size() > 0) {
        tablesWithIds = new HashSet<>(tableDAO.getTables(new ArrayList<>(job.getTablesUsed())));
    }
    List<JobTableRow> jobTableRows = new ArrayList<>(job.getTablesUsed().size());
    for (TableRow tableRow : tablesWithIds) {
        jobTableRows.add(new JobTableRow(-1, jobId, tableRow.getId()));
    }
    // Add associations between Job and Table
    jobTableDAO.createJobTables(jobTableRows);
    if (job.getOutput().getLocation() != null) {
        jobOutputDAO.createJobOutput(job.getOutput(), jobId);
    }
}
Also used : JobTableDAO(com.airbnb.airpal.sql.dao.JobTableDAO) Table(com.airbnb.airpal.presto.Table) JobDAO(com.airbnb.airpal.sql.dao.JobDAO) ArrayList(java.util.ArrayList) JobTableRow(com.airbnb.airpal.sql.beans.JobTableRow) JobOutputDAO(com.airbnb.airpal.sql.dao.JobOutputDAO) Sets(com.google.common.collect.Sets) TableRow(com.airbnb.airpal.sql.beans.TableRow) JobTableRow(com.airbnb.airpal.sql.beans.JobTableRow) TableDAO(com.airbnb.airpal.sql.dao.TableDAO) JobTableDAO(com.airbnb.airpal.sql.dao.JobTableDAO)

Aggregations

Table (com.airbnb.airpal.presto.Table)1 JobTableRow (com.airbnb.airpal.sql.beans.JobTableRow)1 TableRow (com.airbnb.airpal.sql.beans.TableRow)1 JobDAO (com.airbnb.airpal.sql.dao.JobDAO)1 JobOutputDAO (com.airbnb.airpal.sql.dao.JobOutputDAO)1 JobTableDAO (com.airbnb.airpal.sql.dao.JobTableDAO)1 TableDAO (com.airbnb.airpal.sql.dao.TableDAO)1 Sets (com.google.common.collect.Sets)1 ArrayList (java.util.ArrayList)1