Examples with HoodieArchivedTimeline org.apache.hudi.common.table.timeline.HoodieArchivedTimeline used on opensource projects

Example 11 with HoodieArchivedTimeline

use of org.apache.hudi.common.table.timeline.HoodieArchivedTimeline in project hudi by apache.

the class CompactionCommand method compactionsShowArchived.

@CliCommand(value = "compactions showarchived", help = "Shows compaction details for specified time window")
public String compactionsShowArchived(@CliOption(key = { "includeExtraMetadata" }, help = "Include extra metadata", unspecifiedDefaultValue = "false") final boolean includeExtraMetadata, @CliOption(key = { "startTs" }, mandatory = false, help = "start time for compactions, default: now - 10 days") String startTs, @CliOption(key = { "endTs" }, mandatory = false, help = "end time for compactions, default: now - 1 day") String endTs, @CliOption(key = { "limit" }, help = "Limit compactions", unspecifiedDefaultValue = "-1") final Integer limit, @CliOption(key = { "sortBy" }, help = "Sorting Field", unspecifiedDefaultValue = "") final String sortByField, @CliOption(key = { "desc" }, help = "Ordering", unspecifiedDefaultValue = "false") final boolean descending, @CliOption(key = { "headeronly" }, help = "Print Header Only", unspecifiedDefaultValue = "false") final boolean headerOnly) {
    if (StringUtils.isNullOrEmpty(startTs)) {
        startTs = CommitUtil.getTimeDaysAgo(10);
    }
    if (StringUtils.isNullOrEmpty(endTs)) {
        endTs = CommitUtil.getTimeDaysAgo(1);
    }
    HoodieTableMetaClient client = checkAndGetMetaClient();
    HoodieArchivedTimeline archivedTimeline = client.getArchivedTimeline();
    archivedTimeline.loadCompactionDetailsInMemory(startTs, endTs);
    try {
        return printAllCompactions(archivedTimeline, compactionPlanReader(this::readCompactionPlanForArchivedTimeline, archivedTimeline), includeExtraMetadata, sortByField, descending, limit, headerOnly);
    } finally {
        archivedTimeline.clearInstantDetailsFromMemory(startTs, endTs);
    }
}

Example 12 with HoodieArchivedTimeline

use of org.apache.hudi.common.table.timeline.HoodieArchivedTimeline in project hudi by apache.

the class IncrementalInputSplits method getArchivedMetadata.

/**
 * Returns the archived metadata in case the reader consumes untimely or it wants
 * to read from the earliest.
 *
 * <p>Note: should improve it with metadata table when the metadata table is stable enough.
 *
 * @param metaClient     The meta client
 * @param instantRange   The instant range to filter the timeline instants
 * @param commitTimeline The commit timeline
 * @param tableName      The table name
 * @return the list of archived metadata, or empty if there is no need to read the archived timeline
 */
private List<HoodieCommitMetadata> getArchivedMetadata(HoodieTableMetaClient metaClient, InstantRange instantRange, HoodieTimeline commitTimeline, String tableName) {
    if (commitTimeline.isBeforeTimelineStarts(instantRange.getStartInstant())) {
        // read the archived metadata if the start instant is archived.
        HoodieArchivedTimeline archivedTimeline = metaClient.getArchivedTimeline(instantRange.getStartInstant());
        HoodieTimeline archivedCompleteTimeline = archivedTimeline.getCommitsTimeline().filterCompletedInstants();
        if (!archivedCompleteTimeline.empty()) {
            Stream<HoodieInstant> instantStream = archivedCompleteTimeline.getInstants();
            return maySkipCompaction(instantStream).map(instant -> WriteProfiles.getCommitMetadata(tableName, path, instant, archivedTimeline)).collect(Collectors.toList());
        }
    }
    return Collections.emptyList();
}