Search in sources :

Example 11 with HoodieCleanerPlan

use of org.apache.hudi.avro.model.HoodieCleanerPlan in project hudi by apache.

the class HoodieClientTestHarness method createCleanMetadata.

public HoodieInstant createCleanMetadata(String instantTime, boolean inflightOnly, boolean isEmpty) throws IOException {
    HoodieCleanerPlan cleanerPlan = new HoodieCleanerPlan(new HoodieActionInstant("", "", ""), "", new HashMap<>(), CleanPlanV2MigrationHandler.VERSION, new HashMap<>());
    if (inflightOnly) {
        HoodieTestTable.of(metaClient).addInflightClean(instantTime, cleanerPlan);
    } else {
        HoodieCleanStat cleanStats = new HoodieCleanStat(HoodieCleaningPolicy.KEEP_LATEST_FILE_VERSIONS, HoodieTestUtils.DEFAULT_PARTITION_PATHS[new Random().nextInt(HoodieTestUtils.DEFAULT_PARTITION_PATHS.length)], Collections.emptyList(), Collections.emptyList(), Collections.emptyList(), instantTime);
        HoodieCleanMetadata cleanMetadata = convertCleanMetadata(instantTime, Option.of(0L), Collections.singletonList(cleanStats));
        HoodieTestTable.of(metaClient).addClean(instantTime, cleanerPlan, cleanMetadata, isEmpty);
    }
    return new HoodieInstant(inflightOnly, "clean", instantTime);
}
Also used : HoodieInstant(org.apache.hudi.common.table.timeline.HoodieInstant) HoodieCleanStat(org.apache.hudi.common.HoodieCleanStat) Random(java.util.Random) HoodieCleanMetadata(org.apache.hudi.avro.model.HoodieCleanMetadata) HoodieActionInstant(org.apache.hudi.avro.model.HoodieActionInstant) HoodieCleanerPlan(org.apache.hudi.avro.model.HoodieCleanerPlan)

Aggregations

HoodieCleanerPlan (org.apache.hudi.avro.model.HoodieCleanerPlan)11 HoodieActionInstant (org.apache.hudi.avro.model.HoodieActionInstant)6 List (java.util.List)5 HoodieCleanMetadata (org.apache.hudi.avro.model.HoodieCleanMetadata)5 HoodieCleanStat (org.apache.hudi.common.HoodieCleanStat)5 HoodieInstant (org.apache.hudi.common.table.timeline.HoodieInstant)5 IOException (java.io.IOException)4 ArrayList (java.util.ArrayList)4 HoodieIOException (org.apache.hudi.exception.HoodieIOException)4 HashMap (java.util.HashMap)3 Map (java.util.Map)3 Collectors (java.util.stream.Collectors)3 Path (org.apache.hadoop.fs.Path)3 HoodieEngineContext (org.apache.hudi.common.engine.HoodieEngineContext)3 CleanerUtils (org.apache.hudi.common.util.CleanerUtils)3 Option (org.apache.hudi.common.util.Option)3 Pair (org.apache.hudi.common.util.collection.Pair)3 HoodieWriteConfig (org.apache.hudi.config.HoodieWriteConfig)3 LogManager (org.apache.log4j.LogManager)3 Logger (org.apache.log4j.Logger)3