Search in sources :

Example 16 with ConceptMap

use of org.hl7.fhir.dstu3.model.ConceptMap in project bunsen by cerner.

the class BroadcastableMappings method broadcast.

/**
 * Broadcast mappings stored in the given conceptMaps instance that match the given
 * conceptMapUris.
 *
 * @param conceptMaps the {@link ConceptMaps} instance with the content to broadcast
 * @param conceptMapUriToVersion map of the concept map URIs to broadcast to their versions.
 * @return a broadcast variable containing a mappings object usable in UDFs.
 */
public static Broadcast<BroadcastableMappings> broadcast(ConceptMaps conceptMaps, Map<String, String> conceptMapUriToVersion) {
    Map<String, ConceptMap> mapsToLoad = conceptMaps.getMaps().collectAsList().stream().filter(conceptMap -> conceptMap.getVersion().equals(conceptMapUriToVersion.get(conceptMap.getUrl()))).collect(Collectors.toMap(ConceptMap::getUrl, Function.identity()));
    // Expand the concept maps to load and sort them so dependencies are before
    // their dependents in the list.
    List<String> sortedMapsToLoad = sortMapsToLoad(conceptMapUriToVersion.keySet(), mapsToLoad);
    // Since this is used to map from one system to another, we use only targets
    // that don't introduce inaccurate meanings. (For instance, we can't map
    // general condition code to a more specific type, since that is not
    // representative of the source data.)
    Dataset<Mapping> mappings = conceptMaps.getMappings(conceptMapUriToVersion).filter("equivalence in ('equivalent', 'equals', 'wider', 'subsumes')");
    // Group mappings by their concept map URI
    Map<String, List<Mapping>> groupedMappings = mappings.collectAsList().stream().collect(Collectors.groupingBy(Mapping::getConceptMapUri));
    Map<String, BroadcastableConceptMap> broadcastableMaps = new HashMap<>();
    for (String conceptMapUri : sortedMapsToLoad) {
        ConceptMap map = mapsToLoad.get(conceptMapUri);
        Set<String> children = getMapChildren(map);
        List<BroadcastableConceptMap> childMaps = children.stream().map(child -> broadcastableMaps.get(child)).collect(Collectors.toList());
        BroadcastableConceptMap broadcastableConceptMap = new BroadcastableConceptMap(conceptMapUri, groupedMappings.getOrDefault(conceptMapUri, Collections.emptyList()), childMaps);
        broadcastableMaps.put(conceptMapUri, broadcastableConceptMap);
    }
    JavaSparkContext ctx = new JavaSparkContext(conceptMaps.getMaps().sparkSession().sparkContext());
    return ctx.broadcast(new BroadcastableMappings(broadcastableMaps));
}
Also used : Broadcast(org.apache.spark.broadcast.Broadcast) Dataset(org.apache.spark.sql.Dataset) ConceptMaps(com.cerner.bunsen.codes.ConceptMaps) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) Set(java.util.Set) HashMap(java.util.HashMap) Deque(java.util.Deque) Function(java.util.function.Function) Collectors(java.util.stream.Collectors) Serializable(java.io.Serializable) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) List(java.util.List) Map(java.util.Map) ConceptMapGroupUnmappedMode(org.hl7.fhir.dstu3.model.ConceptMap.ConceptMapGroupUnmappedMode) ArrayDeque(java.util.ArrayDeque) Collections(java.util.Collections) ConceptMap(org.hl7.fhir.dstu3.model.ConceptMap) Mapping(com.cerner.bunsen.codes.Mapping) SparkSession(org.apache.spark.sql.SparkSession) HashMap(java.util.HashMap) Mapping(com.cerner.bunsen.codes.Mapping) ArrayList(java.util.ArrayList) List(java.util.List) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) ConceptMap(org.hl7.fhir.dstu3.model.ConceptMap)

Aggregations

ConceptMap (org.hl7.fhir.dstu3.model.ConceptMap)14 Test (org.junit.Test)8 ArrayList (java.util.ArrayList)3 ConceptMapGroupComponent (org.hl7.fhir.dstu3.model.ConceptMap.ConceptMapGroupComponent)3 UriType (org.hl7.fhir.dstu3.model.UriType)3 ArrayDeque (java.util.ArrayDeque)2 HashSet (java.util.HashSet)2 SourceElementComponent (org.hl7.fhir.dstu3.model.ConceptMap.SourceElementComponent)2 ConceptMaps (com.cerner.bunsen.codes.ConceptMaps)1 Mapping (com.cerner.bunsen.codes.Mapping)1 Serializable (java.io.Serializable)1 Timestamp (java.sql.Timestamp)1 Collections (java.util.Collections)1 Deque (java.util.Deque)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1 Set (java.util.Set)1 Function (java.util.function.Function)1 Collectors (java.util.stream.Collectors)1