Search in sources :

Example 1 with AddDatatype

use of com.hartwig.pipeline.metadata.AddDatatype in project pipeline5 by hartwigmedical.

the class LinxGermline method output.

@Override
public LinxGermlineOutput output(final SomaticRunMetadata metadata, final PipelineStatus jobStatus, final RuntimeBucket bucket, final ResultsDirectory resultsDirectory) {
    String disruptionsTsv = metadata.sampleName() + GERMLINE_DISRUPTION_TSV;
    String driverCatalogTsv = metadata.sampleName() + GERMLINE_DRIVER_CATALOG_TSV;
    return LinxGermlineOutput.builder().status(jobStatus).maybeLinxGermlineOutputLocations(LinxGermlineOutputLocations.builder().disruptions(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(disruptionsTsv))).driverCatalog(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(driverCatalogTsv))).outputDirectory(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(), true)).build()).addFailedLogLocations(GoogleStorageLocation.of(bucket.name(), RunLogComponent.LOG_FILE)).addReportComponents(new EntireOutputComponent(bucket, Folder.root(), NAMESPACE, resultsDirectory)).addDatatypes(new AddDatatype(DataType.LINX_GERMLINE_DISRUPTIONS, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), disruptionsTsv))).addDatatypes(new AddDatatype(DataType.LINX_GERMLINE_DRIVER_CATALOG, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), driverCatalogTsv))).build();
}
Also used : ArchivePath(com.hartwig.pipeline.metadata.ArchivePath) EntireOutputComponent(com.hartwig.pipeline.report.EntireOutputComponent) AddDatatype(com.hartwig.pipeline.metadata.AddDatatype)

Example 2 with AddDatatype

use of com.hartwig.pipeline.metadata.AddDatatype in project pipeline5 by hartwigmedical.

the class Orange method output.

@Override
public OrangeOutput output(final SomaticRunMetadata metadata, final PipelineStatus jobStatus, final RuntimeBucket bucket, final ResultsDirectory resultsDirectory) {
    final String orangePdf = metadata.tumor().sampleName() + ORANGE_OUTPUT_PDF;
    final String orangeJson = metadata.tumor().sampleName() + ORANGE_OUTPUT_JSON;
    return OrangeOutput.builder().status(jobStatus).addFailedLogLocations(GoogleStorageLocation.of(bucket.name(), RunLogComponent.LOG_FILE)).addReportComponents(new EntireOutputComponent(bucket, Folder.root(), namespace(), resultsDirectory)).addDatatypes(new AddDatatype(DataType.ORANGE_OUTPUT_JSON, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), orangeJson)), new AddDatatype(DataType.ORANGE_OUTPUT_PDF, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), orangePdf))).build();
}
Also used : ArchivePath(com.hartwig.pipeline.metadata.ArchivePath) EntireOutputComponent(com.hartwig.pipeline.report.EntireOutputComponent) AddDatatype(com.hartwig.pipeline.metadata.AddDatatype)

Example 3 with AddDatatype

use of com.hartwig.pipeline.metadata.AddDatatype in project pipeline5 by hartwigmedical.

the class StagedOutputPublisher method publish.

public void publish(final PipelineState state, final SomaticRunMetadata metadata) {
    if (state.status() != PipelineStatus.FAILED && run.isPresent()) {
        List<AddDatatype> addDatatypes = state.stageOutputs().stream().map(StageOutput::datatypes).flatMap(List::stream).collect(Collectors.toList());
        SampleSet set = setResolver.resolve(metadata.set(), useOnlyDBSets);
        Optional<String> tumorSampleName = metadata.maybeTumor().map(SingleSampleRunMetadata::sampleName);
        Optional<String> refSampleName = metadata.maybeReference().map(SingleSampleRunMetadata::sampleName);
        ImmutableAnalysis.Builder alignedReadsAnalysis = eventBuilder(Type.ALIGNMENT);
        ImmutableAnalysis.Builder somaticAnalysis = eventBuilder(Type.SOMATIC);
        ImmutableAnalysis.Builder germlineAnalysis = eventBuilder(Type.GERMLINE);
        OutputIterator.from(blob -> {
            Optional<AddDatatype> dataType = addDatatypes.stream().filter(d -> blob.getName().endsWith(d.path())).findFirst();
            Blob blobWithMd5 = sourceBucket.get(blob.getName());
            if (isSecondary(blobWithMd5)) {
                alignedReadsAnalysis.addOutput(createBlob(tumorSampleName, refSampleName, dataType, blobWithMd5));
            } else {
                if (isGermline(blobWithMd5)) {
                    germlineAnalysis.addOutput(createBlob(tumorSampleName, refSampleName, dataType, blobWithMd5));
                } else if (notSecondary(blobWithMd5)) {
                    somaticAnalysis.addOutput(createBlob(tumorSampleName, refSampleName, dataType, blobWithMd5));
                }
            }
        }, sourceBucket).iterate(metadata);
        publish(PipelineComplete.builder().pipeline(ImmutablePipeline.builder().sample(tumorSampleName.orElseGet(() -> refSampleName.orElseThrow())).bucket(sourceBucket.getName()).runId(run.get().getId()).setId(set.getId()).context(context).addAnalyses(alignedReadsAnalysis.build(), somaticAnalysis.build(), germlineAnalysis.build()).version(Versions.pipelineMajorMinorVersion()).build()).build());
    }
}
Also used : StageOutput(com.hartwig.pipeline.StageOutput) Analysis(com.hartwig.events.Analysis) ImmutableAnalysisOutputBlob(com.hartwig.events.ImmutableAnalysisOutputBlob) Arrays(java.util.Arrays) ImmutableAnalysis(com.hartwig.events.ImmutableAnalysis) MD5s(com.hartwig.pipeline.metadata.MD5s) SageConfiguration(com.hartwig.pipeline.calling.sage.SageConfiguration) Aligner(com.hartwig.pipeline.alignment.Aligner) SnpGenotype(com.hartwig.pipeline.snpgenotype.SnpGenotype) Versions(com.hartwig.pipeline.tools.Versions) OutputIterator(com.hartwig.pipeline.transfer.OutputIterator) BamMetrics(com.hartwig.pipeline.metrics.BamMetrics) Blob(com.google.cloud.storage.Blob) Type(com.hartwig.events.Analysis.Type) Publisher(com.google.cloud.pubsub.v1.Publisher) PipelineStatus(com.hartwig.pipeline.execution.PipelineStatus) Molecule(com.hartwig.events.Analysis.Molecule) Bucket(com.google.cloud.storage.Bucket) Run(com.hartwig.api.model.Run) Pipeline(com.hartwig.events.Pipeline) PipelineState(com.hartwig.pipeline.PipelineState) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) PipelineComplete(com.hartwig.events.PipelineComplete) StageOutput(com.hartwig.pipeline.StageOutput) Collectors(java.util.stream.Collectors) SampleSet(com.hartwig.api.model.SampleSet) Flagstat(com.hartwig.pipeline.flagstat.Flagstat) List(java.util.List) AddDatatype(com.hartwig.pipeline.metadata.AddDatatype) AnalysisOutputBlob(com.hartwig.events.AnalysisOutputBlob) SomaticRunMetadata(com.hartwig.pipeline.metadata.SomaticRunMetadata) CramConversion(com.hartwig.pipeline.cram.CramConversion) SingleSampleRunMetadata(com.hartwig.pipeline.metadata.SingleSampleRunMetadata) GermlineCaller(com.hartwig.pipeline.calling.germline.GermlineCaller) Optional(java.util.Optional) ImmutablePipeline(com.hartwig.events.ImmutablePipeline) Predicate.not(java.util.function.Predicate.not) NotNull(org.jetbrains.annotations.NotNull) SampleSet(com.hartwig.api.model.SampleSet) ImmutableAnalysisOutputBlob(com.hartwig.events.ImmutableAnalysisOutputBlob) Blob(com.google.cloud.storage.Blob) AnalysisOutputBlob(com.hartwig.events.AnalysisOutputBlob) Optional(java.util.Optional) SingleSampleRunMetadata(com.hartwig.pipeline.metadata.SingleSampleRunMetadata) ImmutableAnalysis(com.hartwig.events.ImmutableAnalysis) AddDatatype(com.hartwig.pipeline.metadata.AddDatatype)

Example 4 with AddDatatype

use of com.hartwig.pipeline.metadata.AddDatatype in project pipeline5 by hartwigmedical.

the class Cuppa method persistedOutput.

@Override
public CuppaOutput persistedOutput(final SomaticRunMetadata metadata) {
    final String cuppaConclusionChart = cuppaConclusionChart(metadata);
    final String conclusionTxt = cuppaConclusionTxt(metadata);
    final String cuppaChart = cupReportSummaryPng(metadata);
    final String resultsCsv = cupDataCsv(metadata);
    final String featurePlot = cuppaFeaturePlot(metadata);
    return CuppaOutput.builder().status(PipelineStatus.PERSISTED).maybeCuppaOutputLocations(CuppaOutputLocations.builder().conclusionTxt(persistedDataset.path(metadata.tumor().sampleName(), DataType.CUPPA_CONCLUSION).orElse(GoogleStorageLocation.of(metadata.bucket(), PersistedLocations.blobForSet(metadata.set(), namespace(), conclusionTxt)))).summaryChartPng(persistedDataset.path(metadata.tumor().sampleName(), DataType.CUPPA_SUMMARY_CHART).orElse(GoogleStorageLocation.of(metadata.bucket(), PersistedLocations.blobForSet(metadata.set(), namespace(), cuppaChart)))).resultCsv(persistedDataset.path(metadata.tumor().sampleName(), DataType.CUPPA_RESULTS).orElse(GoogleStorageLocation.of(metadata.bucket(), PersistedLocations.blobForSet(metadata.set(), namespace(), resultsCsv)))).featurePlot(persistedDataset.path(metadata.tumor().sampleName(), DataType.CUPPA_FEATURE_PLOT).orElse(GoogleStorageLocation.of(metadata.bucket(), PersistedLocations.blobForSet(metadata.set(), namespace(), featurePlot)))).build()).addDatatypes(new AddDatatype(DataType.CUPPA_CHART, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), cuppaConclusionChart))).addDatatypes(new AddDatatype(DataType.CUPPA_CONCLUSION, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), conclusionTxt))).addDatatypes(new AddDatatype(DataType.CUPPA_CHART, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), cuppaChart))).addDatatypes(new AddDatatype(DataType.CUPPA_RESULTS, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), resultsCsv))).addDatatypes(new AddDatatype(DataType.CUPPA_FEATURE_PLOT, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), featurePlot))).build();
}
Also used : ArchivePath(com.hartwig.pipeline.metadata.ArchivePath) AddDatatype(com.hartwig.pipeline.metadata.AddDatatype)

Example 5 with AddDatatype

use of com.hartwig.pipeline.metadata.AddDatatype in project pipeline5 by hartwigmedical.

the class Cuppa method output.

@Override
public CuppaOutput output(final SomaticRunMetadata metadata, final PipelineStatus jobStatus, final RuntimeBucket bucket, final ResultsDirectory resultsDirectory) {
    final String cuppaConclusionChart = cuppaConclusionChart(metadata);
    final String conclusionTxt = cuppaConclusionTxt(metadata);
    final String cuppaSummaryChartPng = cupReportSummaryPng(metadata);
    final String resultsCsv = cupDataCsv(metadata);
    final String featurePlot = cuppaFeaturePlot(metadata);
    return CuppaOutput.builder().status(jobStatus).maybeCuppaOutputLocations(CuppaOutputLocations.builder().conclusionTxt(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(conclusionTxt))).summaryChartPng(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(cuppaSummaryChartPng))).featurePlot(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(featurePlot))).resultCsv(GoogleStorageLocation.of(bucket.name(), resultsDirectory.path(resultsCsv))).build()).addFailedLogLocations(GoogleStorageLocation.of(bucket.name(), RunLogComponent.LOG_FILE)).addReportComponents(new EntireOutputComponent(bucket, Folder.root(), namespace(), resultsDirectory)).addDatatypes(new AddDatatype(DataType.CUPPA_CHART, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), cuppaConclusionChart))).addDatatypes(new AddDatatype(DataType.CUPPA_SUMMARY_CHART, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), cuppaSummaryChartPng))).addDatatypes(new AddDatatype(DataType.CUPPA_CONCLUSION, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), conclusionTxt))).addDatatypes(new AddDatatype(DataType.CUPPA_RESULTS, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), resultsCsv))).addDatatypes(new AddDatatype(DataType.CUPPA_FEATURE_PLOT, metadata.barcode(), new ArchivePath(Folder.root(), namespace(), featurePlot))).build();
}
Also used : ArchivePath(com.hartwig.pipeline.metadata.ArchivePath) EntireOutputComponent(com.hartwig.pipeline.report.EntireOutputComponent) AddDatatype(com.hartwig.pipeline.metadata.AddDatatype)

Aggregations

AddDatatype (com.hartwig.pipeline.metadata.AddDatatype)14 ArchivePath (com.hartwig.pipeline.metadata.ArchivePath)13 EntireOutputComponent (com.hartwig.pipeline.report.EntireOutputComponent)5 RunLogComponent (com.hartwig.pipeline.report.RunLogComponent)5 SingleFileComponent (com.hartwig.pipeline.report.SingleFileComponent)4 PipelineStatus (com.hartwig.pipeline.execution.PipelineStatus)3 Folder (com.hartwig.pipeline.report.Folder)3 StartupScriptComponent (com.hartwig.pipeline.report.StartupScriptComponent)3 List (java.util.List)3 Collectors (java.util.stream.Collectors)3 Blob (com.google.cloud.storage.Blob)2 AnalysisOutputBlob (com.hartwig.events.AnalysisOutputBlob)2 PipelineComplete (com.hartwig.events.PipelineComplete)2 ResultsDirectory (com.hartwig.pipeline.ResultsDirectory)2 Aligner (com.hartwig.pipeline.alignment.Aligner)2 BashStartupScript (com.hartwig.pipeline.execution.vm.BashStartupScript)2 VirtualMachineJobDefinition (com.hartwig.pipeline.execution.vm.VirtualMachineJobDefinition)2 SingleSampleRunMetadata (com.hartwig.pipeline.metadata.SingleSampleRunMetadata)2 SomaticRunMetadata (com.hartwig.pipeline.metadata.SomaticRunMetadata)2 ReportComponent (com.hartwig.pipeline.report.ReportComponent)2