Examples with HivePartitionDataset org.apache.gobblin.compliance.HivePartitionDataset used on opensource projects

Example 6 with HivePartitionDataset

use of org.apache.gobblin.compliance.HivePartitionDataset in project incubator-gobblin by apache.

the class RestorableHivePartitionDatasetFinder method findDatasets.

/**
 * Will return a Singleton list of HivePartitionDataset to be restored.
 */
public List<HivePartitionDataset> findDatasets() throws IOException {
    Preconditions.checkArgument(this.state.contains(ComplianceConfigurationKeys.RESTORE_DATASET), "Missing required property " + ComplianceConfigurationKeys.RESTORE_DATASET);
    HivePartitionDataset hivePartitionDataset = HivePartitionFinder.findDataset(this.state.getProp(ComplianceConfigurationKeys.RESTORE_DATASET), this.state);
    Preconditions.checkNotNull(hivePartitionDataset, "No dataset to restore");
    return Collections.singletonList(hivePartitionDataset);
}

Example 7 with HivePartitionDataset

use of org.apache.gobblin.compliance.HivePartitionDataset in project incubator-gobblin by apache.

the class CleanableHivePartitionDatasetFinder method findDatasets.

/**
 * Will find all datasets according to whitelist, except the backup and staging tables.
 */
public List<HivePartitionDataset> findDatasets() throws IOException {
    List<HivePartitionDataset> list = new ArrayList<>();
    for (HivePartitionDataset hivePartitionDataset : super.findDatasets()) {
        CleanableHivePartitionDataset dataset = new CleanableHivePartitionDataset(hivePartitionDataset, this.fs, this.state);
        list.add(dataset);
    }
    return list;
}