Search in sources :

Example 6 with HoodieDefaultTimeline

use of org.apache.hudi.common.table.timeline.HoodieDefaultTimeline in project hudi by apache.

the class CommitsCommand method showArchivedCommits.

@CliCommand(value = "commits showarchived", help = "Show the archived commits")
public String showArchivedCommits(@CliOption(key = { "includeExtraMetadata" }, help = "Include extra metadata", unspecifiedDefaultValue = "false") final boolean includeExtraMetadata, @CliOption(key = { "createView" }, mandatory = false, help = "view name to store output table", unspecifiedDefaultValue = "") final String exportTableName, @CliOption(key = { "startTs" }, mandatory = false, help = "start time for commits, default: now - 10 days") String startTs, @CliOption(key = { "endTs" }, mandatory = false, help = "end time for commits, default: now - 1 day") String endTs, @CliOption(key = { "limit" }, mandatory = false, help = "Limit commits", unspecifiedDefaultValue = "-1") final Integer limit, @CliOption(key = { "sortBy" }, help = "Sorting Field", unspecifiedDefaultValue = "") final String sortByField, @CliOption(key = { "desc" }, help = "Ordering", unspecifiedDefaultValue = "false") final boolean descending, @CliOption(key = { "headeronly" }, help = "Print Header Only", unspecifiedDefaultValue = "false") final boolean headerOnly) throws IOException {
    if (StringUtils.isNullOrEmpty(startTs)) {
        startTs = CommitUtil.getTimeDaysAgo(10);
    }
    if (StringUtils.isNullOrEmpty(endTs)) {
        endTs = CommitUtil.getTimeDaysAgo(1);
    }
    HoodieArchivedTimeline archivedTimeline = HoodieCLI.getTableMetaClient().getArchivedTimeline();
    try {
        archivedTimeline.loadInstantDetailsInMemory(startTs, endTs);
        HoodieDefaultTimeline timelineRange = archivedTimeline.findInstantsInRange(startTs, endTs);
        if (includeExtraMetadata) {
            return printCommitsWithMetadata(timelineRange, limit, sortByField, descending, headerOnly, exportTableName);
        } else {
            return printCommits(timelineRange, limit, sortByField, descending, headerOnly, exportTableName);
        }
    } finally {
        // clear the instant details from memory after printing to reduce usage
        archivedTimeline.clearInstantDetailsFromMemory(startTs, endTs);
    }
}
Also used : HoodieDefaultTimeline(org.apache.hudi.common.table.timeline.HoodieDefaultTimeline) HoodieArchivedTimeline(org.apache.hudi.common.table.timeline.HoodieArchivedTimeline) CliCommand(org.springframework.shell.core.annotation.CliCommand)

Aggregations

HoodieDefaultTimeline (org.apache.hudi.common.table.timeline.HoodieDefaultTimeline)6 HoodieInstant (org.apache.hudi.common.table.timeline.HoodieInstant)4 HoodieTimeline (org.apache.hudi.common.table.timeline.HoodieTimeline)3 CliCommand (org.springframework.shell.core.annotation.CliCommand)3 IOException (java.io.IOException)2 ArrayList (java.util.ArrayList)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2 Function (java.util.function.Function)2 Collectors (java.util.stream.Collectors)2 Stream (java.util.stream.Stream)2 FileSystem (org.apache.hadoop.fs.FileSystem)2 Path (org.apache.hadoop.fs.Path)2 HoodieCLI (org.apache.hudi.cli.HoodieCLI)2 HoodiePrintHelper (org.apache.hudi.cli.HoodiePrintHelper)2 HoodieTableHeaderFields (org.apache.hudi.cli.HoodieTableHeaderFields)2 TableHeader (org.apache.hudi.cli.TableHeader)2 HoodieTableMetaClient (org.apache.hudi.common.table.HoodieTableMetaClient)2 HoodieTableFileSystemView (org.apache.hudi.common.table.view.HoodieTableFileSystemView)2