Search in sources :

Example 16 with HoodieTimelineArchiver

use of org.apache.hudi.client.HoodieTimelineArchiver in project hudi by apache.

the class TestCompactionCommand method generateArchive.

private void generateArchive() throws IOException {
    // Generate archive
    HoodieWriteConfig cfg = HoodieWriteConfig.newBuilder().withPath(tablePath).withSchema(HoodieTestCommitMetadataGenerator.TRIP_EXAMPLE_SCHEMA).withParallelism(2, 2).withCompactionConfig(HoodieCompactionConfig.newBuilder().retainCommits(1).archiveCommitsWith(2, 3).build()).withFileSystemViewConfig(FileSystemViewStorageConfig.newBuilder().withRemoteServerPort(timelineServicePort).build()).forTable("test-trip-table").build();
    // archive
    HoodieTableMetaClient metaClient = HoodieTableMetaClient.reload(HoodieCLI.getTableMetaClient());
    HoodieSparkTable table = HoodieSparkTable.create(cfg, context(), metaClient);
    HoodieTimelineArchiver archiver = new HoodieTimelineArchiver(cfg, table);
    archiver.archiveIfRequired(context());
}
Also used : HoodieTableMetaClient(org.apache.hudi.common.table.HoodieTableMetaClient) HoodieTimelineArchiver(org.apache.hudi.client.HoodieTimelineArchiver) HoodieWriteConfig(org.apache.hudi.config.HoodieWriteConfig) HoodieSparkTable(org.apache.hudi.table.HoodieSparkTable)

Aggregations

HoodieTimelineArchiver (org.apache.hudi.client.HoodieTimelineArchiver)16 HoodieWriteConfig (org.apache.hudi.config.HoodieWriteConfig)14 HoodieTable (org.apache.hudi.table.HoodieTable)12 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)12 HoodieInstant (org.apache.hudi.common.table.timeline.HoodieInstant)11 HoodieSparkTable (org.apache.hudi.table.HoodieSparkTable)8 ValueSource (org.junit.jupiter.params.provider.ValueSource)8 HoodieTimeline (org.apache.hudi.common.table.timeline.HoodieTimeline)7 HashMap (java.util.HashMap)6 Map (java.util.Map)6 HoodieTableMetaClient (org.apache.hudi.common.table.HoodieTableMetaClient)6 ArrayList (java.util.ArrayList)5 List (java.util.List)5 Path (org.apache.hadoop.fs.Path)5 HoodieArchivedTimeline (org.apache.hudi.common.table.timeline.HoodieArchivedTimeline)5 Test (org.junit.jupiter.api.Test)5 IOException (java.io.IOException)4 Arrays (java.util.Arrays)4 Collections (java.util.Collections)4 Comparator (java.util.Comparator)4