Examples with CommitSequence org.apache.gobblin.commit.CommitSequence used on opensource projects

Example 1 with CommitSequence

use of org.apache.gobblin.commit.CommitSequence in project incubator-gobblin by apache.

the class SafeDatasetCommit method buildAndExecuteCommitSequence.

private synchronized void buildAndExecuteCommitSequence(CommitSequence.Builder builder, JobState.DatasetState datasetState, String datasetUrn) throws IOException {
    CommitSequence commitSequence = builder.addStep(buildDatasetStateCommitStep(datasetUrn, datasetState).get()).build();
    this.jobContext.getCommitSequenceStore().get().put(commitSequence.getJobName(), datasetUrn, commitSequence);
    commitSequence.execute();
    this.jobContext.getCommitSequenceStore().get().delete(commitSequence.getJobName(), datasetUrn);
}

Example 2 with CommitSequence

use of org.apache.gobblin.commit.CommitSequence in project incubator-gobblin by apache.

the class AbstractJobLauncher method executeUnfinishedCommitSequences.

private void executeUnfinishedCommitSequences(String jobName) throws IOException {
    Preconditions.checkState(this.jobContext.getCommitSequenceStore().isPresent());
    CommitSequenceStore commitSequenceStore = this.jobContext.getCommitSequenceStore().get();
    for (String datasetUrn : commitSequenceStore.get(jobName)) {
        Optional<CommitSequence> commitSequence = commitSequenceStore.get(jobName, datasetUrn);
        if (commitSequence.isPresent()) {
            commitSequence.get().execute();
        }
        commitSequenceStore.delete(jobName, datasetUrn);
    }
}