Examples with MergingDigestCoder org.apache.beam.sdk.extensions.sketching.TDigestQuantiles.MergingDigestCoder used on opensource projects

Example 1 with MergingDigestCoder

use of org.apache.beam.sdk.extensions.sketching.TDigestQuantiles.MergingDigestCoder in project beam by apache.

the class TDigestQuantilesTest method encodeDecodeEquals.

private <T> boolean encodeDecodeEquals(MergingDigest tDigest) throws IOException {
    MergingDigest decoded = CoderUtils.clone(new MergingDigestCoder(), tDigest);
    boolean equal = true;
    // the only way to compare the two sketches is to compare them centroid by centroid.
    // Indeed, the means are doubles but are encoded as float and cast during decoding.
    // This entails a small approximation that makes the centroids different after decoding.
    Iterator<Centroid> it1 = decoded.centroids().iterator();
    Iterator<Centroid> it2 = tDigest.centroids().iterator();
    for (int i = 0; i < decoded.centroids().size(); i++) {
        Centroid c1 = it1.next();
        Centroid c2 = it2.next();
        if ((float) c1.mean() != (float) c2.mean() || c1.count() != c2.count()) {
            equal = false;
            break;
        }
    }
    return equal;
}