Search in sources :

Example 16 with HDF5File

use of org.broadinstitute.hdf5.HDF5File in project gatk-protected by broadinstitute.

the class HDF5PCACoveragePoNUnitTest method testTargetFactorsReading.

@Test(dependsOnMethods = "testTargetNameReading")
public void testTargetFactorsReading() throws IOException {
    final HDF5File reader = new HDF5File(TEST_PON);
    final PCACoveragePoN pon = new HDF5PCACoveragePoN(reader);
    final List<String> targets = pon.getTargetNames();
    final double[] factors = pon.getTargetFactors();
    Assert.assertEquals(factors.length, targets.size());
    final List<Double> expected = readDoubleLines(TEST_PON_TARGET_FACTORS);
    Assert.assertNotNull(factors);
    Assert.assertEquals(factors.length, expected.size());
    for (int i = 0; i < expected.size(); i++) {
        Assert.assertEquals(factors[i], expected.get(i), Math.abs(expected.get(i)) * 0.0001);
    }
    reader.close();
}
Also used : HDF5File(org.broadinstitute.hdf5.HDF5File) BaseTest(org.broadinstitute.hellbender.utils.test.BaseTest) Test(org.testng.annotations.Test) BeforeTest(org.testng.annotations.BeforeTest)

Example 17 with HDF5File

use of org.broadinstitute.hdf5.HDF5File in project gatk-protected by broadinstitute.

the class HDF5PCACoveragePoNUnitTest method testLogNormalizedMatrixReading.

@Test(dependsOnMethods = { "testTargetNameReading", "testLogNormalizedSampleNameReading" })
public void testLogNormalizedMatrixReading() throws IOException {
    final HDF5File reader = new HDF5File(TEST_PON);
    final PCACoveragePoN pon = new HDF5PCACoveragePoN(reader);
    final List<String> targets = pon.getTargetNames();
    final List<String> samples = pon.getPanelSampleNames();
    final RealMatrix actual = pon.getLogNormalizedCounts();
    Assert.assertNotNull(actual);
    Assert.assertEquals(actual.getRowDimension(), targets.size());
    Assert.assertEquals(actual.getColumnDimension(), samples.size());
    final RealMatrix expected = readDoubleMatrix(TEST_PON_LOG_NORMALS);
    MathObjectAsserts.assertRealMatrixEquals(actual, expected);
}
Also used : Array2DRowRealMatrix(org.apache.commons.math3.linear.Array2DRowRealMatrix) RealMatrix(org.apache.commons.math3.linear.RealMatrix) HDF5File(org.broadinstitute.hdf5.HDF5File) BaseTest(org.broadinstitute.hellbender.utils.test.BaseTest) Test(org.testng.annotations.Test) BeforeTest(org.testng.annotations.BeforeTest)

Example 18 with HDF5File

use of org.broadinstitute.hdf5.HDF5File in project gatk by broadinstitute.

the class NormalizeSomaticReadCountsIntegrationTest method assertBetaHatsRobustToOutliers.

/**
     * Asserts that the calculation of beta hats is not significantly affected by zero-coverage outlier counts
     * We perform this check by randomly setting some coverages to zero in copy ratio space (-infinity in log space).
     * betaHats imputes 0 in log space (1 in copy ratio space) whenever coverage is below a certain low threshold
     * and should thus be robust to this type of noise.
     */
private void assertBetaHatsRobustToOutliers(final ReadCountCollection preTangentNormalized, final File ponFile) {
    try (final HDF5File ponReader = new HDF5File(ponFile)) {
        final PCACoveragePoN pon = new HDF5PCACoveragePoN(ponReader);
        final List<String> ponTargets = pon.getPanelTargetNames();
        final RealMatrix input = reorderTargetsToPoNOrder(preTangentNormalized, ponTargets);
        // randomly set some entries to zero in copy-ratio space (-infinity in log space)
        final Random random = new Random(13);
        final double noiseProportion = 0.01;
        final RealMatrix noisyInput = input.copy();
        noisyInput.walkInOptimizedOrder(new DefaultRealMatrixChangingVisitor() {

            @Override
            public double visit(final int row, final int column, final double value) {
                return random.nextDouble() < noiseProportion ? Double.NEGATIVE_INFINITY : value;
            }
        });
        final RealMatrix betaHats = PCATangentNormalizationUtils.calculateBetaHats(pon.getReducedPanelPInverseCounts(), input, PCATangentNormalizationUtils.EPSILON);
        final RealMatrix noisyBetaHats = PCATangentNormalizationUtils.calculateBetaHats(pon.getReducedPanelPInverseCounts(), noisyInput, PCATangentNormalizationUtils.EPSILON);
        final RealMatrix difference = betaHats.subtract(noisyBetaHats);
        difference.walkInOptimizedOrder(new DefaultRealMatrixPreservingVisitor() {

            @Override
            public void visit(final int row, int column, double value) {
                Assert.assertEquals(value, 0, 0.01);
            }
        });
    }
}
Also used : HDF5PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.HDF5PCACoveragePoN) DefaultRealMatrixPreservingVisitor(org.apache.commons.math3.linear.DefaultRealMatrixPreservingVisitor) Array2DRowRealMatrix(org.apache.commons.math3.linear.Array2DRowRealMatrix) RealMatrix(org.apache.commons.math3.linear.RealMatrix) DefaultRealMatrixChangingVisitor(org.apache.commons.math3.linear.DefaultRealMatrixChangingVisitor) PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.PCACoveragePoN) HDF5PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.HDF5PCACoveragePoN) HDF5File(org.broadinstitute.hdf5.HDF5File)

Example 19 with HDF5File

use of org.broadinstitute.hdf5.HDF5File in project gatk by broadinstitute.

the class CreatePanelOfNormalsIntegrationTest method assertRamPoNDuplicate.

private void assertRamPoNDuplicate(final File outputFile) {
    try (final HDF5File hdf5FilePoN = new HDF5File(outputFile)) {
        final HDF5PCACoveragePoN filePoN = new HDF5PCACoveragePoN(hdf5FilePoN);
        assertRamPoNDuplicate(filePoN);
    }
}
Also used : HDF5PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.HDF5PCACoveragePoN) HDF5File(org.broadinstitute.hdf5.HDF5File)

Example 20 with HDF5File

use of org.broadinstitute.hdf5.HDF5File in project gatk by broadinstitute.

the class NormalizeSomaticReadCountsIntegrationTest method assertTangentNormalized.

private void assertTangentNormalized(final ReadCountCollection actualReadCounts, final ReadCountCollection preTangentNormalized, final RealMatrix betaHats, final File ponFile) {
    try (final HDF5File ponReader = new HDF5File(ponFile)) {
        final PCACoveragePoN pon = new HDF5PCACoveragePoN(ponReader);
        final RealMatrix inCounts = reorderTargetsToPoNOrder(preTangentNormalized, pon.getPanelTargetNames());
        final RealMatrix actual = reorderTargetsToPoNOrder(actualReadCounts, pon.getPanelTargetNames());
        final RealMatrix ponMat = pon.getReducedPanelCounts();
        final RealMatrix projection = ponMat.multiply(betaHats);
        final RealMatrix expected = inCounts.subtract(projection);
        Assert.assertEquals(actual.getRowDimension(), expected.getRowDimension());
        Assert.assertEquals(actual.getColumnDimension(), expected.getColumnDimension());
        for (int i = 0; i < actual.getRowDimension(); i++) {
            Assert.assertEquals(actual.getRow(i), expected.getRow(i));
        }
    }
}
Also used : HDF5PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.HDF5PCACoveragePoN) Array2DRowRealMatrix(org.apache.commons.math3.linear.Array2DRowRealMatrix) RealMatrix(org.apache.commons.math3.linear.RealMatrix) PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.PCACoveragePoN) HDF5PCACoveragePoN(org.broadinstitute.hellbender.tools.pon.coverage.pca.HDF5PCACoveragePoN) HDF5File(org.broadinstitute.hdf5.HDF5File)

Aggregations

HDF5File (org.broadinstitute.hdf5.HDF5File)82 Test (org.testng.annotations.Test)58 BaseTest (org.broadinstitute.hellbender.utils.test.BaseTest)56 File (java.io.File)32 Array2DRowRealMatrix (org.apache.commons.math3.linear.Array2DRowRealMatrix)24 RealMatrix (org.apache.commons.math3.linear.RealMatrix)24 HDF5PCACoveragePoN (org.broadinstitute.hellbender.tools.pon.coverage.pca.HDF5PCACoveragePoN)20 BeforeTest (org.testng.annotations.BeforeTest)20 PCACoveragePoN (org.broadinstitute.hellbender.tools.pon.coverage.pca.PCACoveragePoN)16 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)10 HDF5Library (org.broadinstitute.hdf5.HDF5Library)6 UserException (org.broadinstitute.hellbender.exceptions.UserException)6 ArrayList (java.util.ArrayList)4 List (java.util.List)4 OptionalInt (java.util.OptionalInt)4 StandardArgumentDefinitions (org.broadinstitute.hellbender.cmdline.StandardArgumentDefinitions)4 IOException (java.io.IOException)2 UncheckedIOException (java.io.UncheckedIOException)2 DoubleStream (java.util.stream.DoubleStream)2 IntStream (java.util.stream.IntStream)2