Search in sources :

Example 1 with FeatureReader

use of htsjdk.tribble.FeatureReader in project gatk by broadinstitute.

the class GatherVcfs method getReaderFromVCFUri.

private static FeatureReader<VariantContext> getReaderFromVCFUri(final Path variantPath, final int cloudPrefetchBuffer) {
    final String variantURI = variantPath.toUri().toString();
    final Function<SeekableByteChannel, SeekableByteChannel> cloudWrapper = (cloudPrefetchBuffer > 0 ? is -> SeekableByteChannelPrefetcher.addPrefetcher(cloudPrefetchBuffer, is) : Function.identity());
    return AbstractFeatureReader.getFeatureReader(variantURI, null, new VCFCodec(), false, cloudWrapper, Function.identity());
}
Also used : CloseableIterator(htsjdk.samtools.util.CloseableIterator) CommandLineProgramProperties(org.broadinstitute.barclay.argparser.CommandLineProgramProperties) java.util(java.util) ProgressLogger(org.broadinstitute.hellbender.utils.runtime.ProgressLogger) IOUtil(htsjdk.samtools.util.IOUtil) VCFHeader(htsjdk.variant.vcf.VCFHeader) Argument(org.broadinstitute.barclay.argparser.Argument) FeatureReader(htsjdk.tribble.FeatureReader) VariantContextWriterBuilder(htsjdk.variant.variantcontext.writer.VariantContextWriterBuilder) StandardArgumentDefinitions(org.broadinstitute.hellbender.cmdline.StandardArgumentDefinitions) Function(java.util.function.Function) VariantProgramGroup(org.broadinstitute.hellbender.cmdline.programgroups.VariantProgramGroup) AbstractFeatureReader(htsjdk.tribble.AbstractFeatureReader) BlockCompressedOutputStream(htsjdk.samtools.util.BlockCompressedOutputStream) RuntimeIOException(htsjdk.samtools.util.RuntimeIOException) CollectionUtil(htsjdk.samtools.util.CollectionUtil) VCFCodec(htsjdk.variant.vcf.VCFCodec) PeekableIterator(htsjdk.samtools.util.PeekableIterator) PicardCommandLineProgram(org.broadinstitute.hellbender.cmdline.PicardCommandLineProgram) Path(java.nio.file.Path) CloserUtil(htsjdk.samtools.util.CloserUtil) BlockCompressedStreamConstants(htsjdk.samtools.util.BlockCompressedStreamConstants) IOUtils(org.broadinstitute.hellbender.utils.io.IOUtils) BlockCompressedInputStream(htsjdk.samtools.util.BlockCompressedInputStream) SAMSequenceDictionary(htsjdk.samtools.SAMSequenceDictionary) Collectors(java.util.stream.Collectors) VariantContextComparator(htsjdk.variant.variantcontext.VariantContextComparator) SeekableByteChannel(java.nio.channels.SeekableByteChannel) Logger(org.apache.logging.log4j.Logger) UserException(org.broadinstitute.hellbender.exceptions.UserException) java.io(java.io) Options(htsjdk.variant.variantcontext.writer.Options) VariantContextWriter(htsjdk.variant.variantcontext.writer.VariantContextWriter) SeekableByteChannelPrefetcher(org.broadinstitute.hellbender.utils.nio.SeekableByteChannelPrefetcher) VariantContext(htsjdk.variant.variantcontext.VariantContext) Utils(org.broadinstitute.hellbender.utils.Utils) VisibleForTesting(com.google.common.annotations.VisibleForTesting) LogManager(org.apache.logging.log4j.LogManager) SeekableByteChannel(java.nio.channels.SeekableByteChannel) VCFCodec(htsjdk.variant.vcf.VCFCodec)

Example 2 with FeatureReader

use of htsjdk.tribble.FeatureReader in project gatk by broadinstitute.

the class GenomicsDBImport method traverse.

/**
     * A complete traversal from start to finish. This method will import all samples
     * specified in the input GVCF files.
     */
@Override
public void traverse() {
    // Force the progress meter to update after every batch
    progressMeter.setRecordsBetweenTimeChecks(1L);
    final int sampleCount = sampleNameToVcfUri.size();
    final int updatedBatchSize = (batchSize == DEFAULT_ZERO_BATCH_SIZE) ? sampleCount : batchSize;
    final int totalBatchCount = (sampleCount / updatedBatchSize) + (sampleCount % updatedBatchSize == 0 ? 0 : 1);
    GenomicsDBImporter importer;
    for (int i = 0, batchCount = 1; i < sampleCount; i += updatedBatchSize, ++batchCount) {
        final Map<String, FeatureReader<VariantContext>> sampleToReaderMap = getFeatureReaders(new ArrayList<>(sampleNameToVcfUri.keySet()), sampleNameToVcfUri, updatedBatchSize, sampleCount, i);
        logger.info("Importing batch " + batchCount + " with " + sampleToReaderMap.size() + " samples");
        final long variantContextBufferSize = vcfBufferSizePerSample * sampleToReaderMap.size();
        final GenomicsDBImportConfiguration.ImportConfiguration importConfiguration = createImportConfiguration(workspace, GenomicsDBConstants.DEFAULT_ARRAY_NAME, variantContextBufferSize, segmentSize, i, (i + updatedBatchSize - 1));
        try {
            importer = new GenomicsDBImporter(sampleToReaderMap, mergedHeaderLines, intervals.get(0), importConfiguration);
        } catch (final IOException e) {
            throw new UserException("Error initializing GenomicsDBImporter in batch " + batchCount, e);
        }
        try {
            importer.importBatch();
        } catch (final IOException e) {
            throw new UserException("GenomicsDB import failed in batch " + batchCount, e);
        }
        closeReaders(sampleToReaderMap);
        progressMeter.update(intervals.get(0));
        logger.info("Done importing batch " + batchCount + "/" + totalBatchCount);
    }
}
Also used : GenomicsDBImporter(com.intel.genomicsdb.GenomicsDBImporter) GenomicsDBImportConfiguration(com.intel.genomicsdb.GenomicsDBImportConfiguration) IOException(java.io.IOException) UserException(org.broadinstitute.hellbender.exceptions.UserException) FeatureReader(htsjdk.tribble.FeatureReader) AbstractFeatureReader(htsjdk.tribble.AbstractFeatureReader)

Aggregations

AbstractFeatureReader (htsjdk.tribble.AbstractFeatureReader)2 FeatureReader (htsjdk.tribble.FeatureReader)2 UserException (org.broadinstitute.hellbender.exceptions.UserException)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 GenomicsDBImportConfiguration (com.intel.genomicsdb.GenomicsDBImportConfiguration)1 GenomicsDBImporter (com.intel.genomicsdb.GenomicsDBImporter)1 SAMSequenceDictionary (htsjdk.samtools.SAMSequenceDictionary)1 BlockCompressedInputStream (htsjdk.samtools.util.BlockCompressedInputStream)1 BlockCompressedOutputStream (htsjdk.samtools.util.BlockCompressedOutputStream)1 BlockCompressedStreamConstants (htsjdk.samtools.util.BlockCompressedStreamConstants)1 CloseableIterator (htsjdk.samtools.util.CloseableIterator)1 CloserUtil (htsjdk.samtools.util.CloserUtil)1 CollectionUtil (htsjdk.samtools.util.CollectionUtil)1 IOUtil (htsjdk.samtools.util.IOUtil)1 PeekableIterator (htsjdk.samtools.util.PeekableIterator)1 RuntimeIOException (htsjdk.samtools.util.RuntimeIOException)1 VariantContext (htsjdk.variant.variantcontext.VariantContext)1 VariantContextComparator (htsjdk.variant.variantcontext.VariantContextComparator)1 Options (htsjdk.variant.variantcontext.writer.Options)1 VariantContextWriter (htsjdk.variant.variantcontext.writer.VariantContextWriter)1