Search in sources :

Example 1 with Mapping

use of com.cerner.bunsen.codes.Mapping in project bunsen by cerner.

the class BroadcastableMappings method broadcast.

/**
 * Broadcast mappings stored in the given conceptMaps instance that match the given
 * conceptMapUris.
 *
 * @param conceptMaps the {@link ConceptMaps} instance with the content to broadcast
 * @param conceptMapUriToVersion map of the concept map URIs to broadcast to their versions.
 * @return a broadcast variable containing a mappings object usable in UDFs.
 */
public static Broadcast<BroadcastableMappings> broadcast(ConceptMaps conceptMaps, Map<String, String> conceptMapUriToVersion) {
    Map<String, ConceptMap> mapsToLoad = conceptMaps.getMaps().collectAsList().stream().filter(conceptMap -> conceptMap.getVersion().equals(conceptMapUriToVersion.get(conceptMap.getUrl()))).collect(Collectors.toMap(ConceptMap::getUrl, Function.identity()));
    // Expand the concept maps to load and sort them so dependencies are before
    // their dependents in the list.
    List<String> sortedMapsToLoad = sortMapsToLoad(conceptMapUriToVersion.keySet(), mapsToLoad);
    // Since this is used to map from one system to another, we use only targets
    // that don't introduce inaccurate meanings. (For instance, we can't map
    // general condition code to a more specific type, since that is not
    // representative of the source data.)
    Dataset<Mapping> mappings = conceptMaps.getMappings(conceptMapUriToVersion).filter("equivalence in ('equivalent', 'equals', 'wider', 'subsumes')");
    // Group mappings by their concept map URI
    Map<String, List<Mapping>> groupedMappings = mappings.collectAsList().stream().collect(Collectors.groupingBy(Mapping::getConceptMapUri));
    Map<String, BroadcastableConceptMap> broadcastableMaps = new HashMap<>();
    for (String conceptMapUri : sortedMapsToLoad) {
        ConceptMap map = mapsToLoad.get(conceptMapUri);
        Set<String> children = getMapChildren(map);
        List<BroadcastableConceptMap> childMaps = children.stream().map(child -> broadcastableMaps.get(child)).collect(Collectors.toList());
        BroadcastableConceptMap broadcastableConceptMap = new BroadcastableConceptMap(conceptMapUri, groupedMappings.getOrDefault(conceptMapUri, Collections.emptyList()), childMaps);
        broadcastableMaps.put(conceptMapUri, broadcastableConceptMap);
    }
    JavaSparkContext ctx = new JavaSparkContext(conceptMaps.getMaps().sparkSession().sparkContext());
    return ctx.broadcast(new BroadcastableMappings(broadcastableMaps));
}
Also used : Broadcast(org.apache.spark.broadcast.Broadcast) Dataset(org.apache.spark.sql.Dataset) ConceptMaps(com.cerner.bunsen.codes.ConceptMaps) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) Set(java.util.Set) HashMap(java.util.HashMap) Deque(java.util.Deque) Function(java.util.function.Function) Collectors(java.util.stream.Collectors) Serializable(java.io.Serializable) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) List(java.util.List) Map(java.util.Map) ConceptMapGroupUnmappedMode(org.hl7.fhir.dstu3.model.ConceptMap.ConceptMapGroupUnmappedMode) ArrayDeque(java.util.ArrayDeque) Collections(java.util.Collections) ConceptMap(org.hl7.fhir.dstu3.model.ConceptMap) Mapping(com.cerner.bunsen.codes.Mapping) SparkSession(org.apache.spark.sql.SparkSession) HashMap(java.util.HashMap) Mapping(com.cerner.bunsen.codes.Mapping) ArrayList(java.util.ArrayList) List(java.util.List) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) ConceptMap(org.hl7.fhir.dstu3.model.ConceptMap)

Aggregations

ConceptMaps (com.cerner.bunsen.codes.ConceptMaps)1 Mapping (com.cerner.bunsen.codes.Mapping)1 Serializable (java.io.Serializable)1 ArrayDeque (java.util.ArrayDeque)1 ArrayList (java.util.ArrayList)1 Collections (java.util.Collections)1 Deque (java.util.Deque)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 List (java.util.List)1 Map (java.util.Map)1 Set (java.util.Set)1 Function (java.util.function.Function)1 Collectors (java.util.stream.Collectors)1 JavaSparkContext (org.apache.spark.api.java.JavaSparkContext)1 Broadcast (org.apache.spark.broadcast.Broadcast)1 Dataset (org.apache.spark.sql.Dataset)1 SparkSession (org.apache.spark.sql.SparkSession)1 ConceptMap (org.hl7.fhir.dstu3.model.ConceptMap)1 ConceptMapGroupUnmappedMode (org.hl7.fhir.dstu3.model.ConceptMap.ConceptMapGroupUnmappedMode)1