Search in sources :

Example 1 with BroadcastableConceptMap

use of com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap in project bunsen by cerner.

the class BroadcastableMappingsTest method testBroadcastLatest.

@Test
public void testBroadcastLatest() {
    BroadcastableConceptMap broadcastableConceptMap = broadcastLatest.getValue().getBroadcastConceptMap("uri:test:concept:map");
    BroadcastableConceptMap.CodeValue value = broadcastableConceptMap.getTarget("uri:test:source:system", "abc").get(0);
    Assert.assertEquals("uri:test:target:system", value.getSystem());
    Assert.assertEquals("123", value.getValue());
    value = broadcastableConceptMap.getTarget("uri:test:source:system", "def").get(0);
    Assert.assertEquals("uri:test:target:system", value.getSystem());
    Assert.assertEquals("xyz", value.getValue());
}
Also used : BroadcastableConceptMap(com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap) Test(org.junit.Test)

Example 2 with BroadcastableConceptMap

use of com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap in project bunsen by cerner.

the class BroadcastableMappingsTest method testDelegateMapping.

@Test
public void testDelegateMapping() {
    BroadcastableConceptMap broadcastableConceptMap = broadcast.getValue().getBroadcastConceptMap("uri:test:concept:delegating");
    BroadcastableConceptMap.CodeValue value = broadcastableConceptMap.getTarget("uri:test:source:system", "abc").get(0);
    Assert.assertEquals("uri:test:target:system", value.getSystem());
    Assert.assertEquals("123", value.getValue());
}
Also used : BroadcastableConceptMap(com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap) Test(org.junit.Test)

Example 3 with BroadcastableConceptMap

use of com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap in project bunsen by cerner.

the class ConceptMaps method broadcast.

@Override
public Broadcast<BroadcastableMappings> broadcast(Map<String, String> conceptMapUriToVersion) {
    List<ConceptMap> mapsList = getMaps().collectAsList().stream().map(row -> (ConceptMap) conceptMapRowConverter.rowToResource(row)).collect(Collectors.toList());
    Map<String, ConceptMap> mapsToLoad = mapsList.stream().filter(conceptMap -> conceptMap.getVersion().equals(conceptMapUriToVersion.get(conceptMap.getUrl()))).collect(Collectors.toMap(ConceptMap::getUrl, java.util.function.Function.identity()));
    // Expand the concept maps to load and sort them so dependencies are before
    // their dependents in the list.
    List<String> sortedMapsToLoad = sortMapsToLoad(conceptMapUriToVersion.keySet(), mapsToLoad);
    // Since this is used to map from one system to another, we use only targets
    // that don't introduce inaccurate meanings. (For instance, we can't map
    // general condition code to a more specific type, since that is not
    // representative of the source data.)
    Dataset<Mapping> mappings = getMappings(conceptMapUriToVersion).filter("equivalence in ('equivalent', 'equals', 'wider', 'subsumes')");
    // Group mappings by their concept map URI
    Map<String, List<Mapping>> groupedMappings = mappings.collectAsList().stream().collect(Collectors.groupingBy(Mapping::getConceptMapUri));
    Map<String, BroadcastableConceptMap> broadcastableMaps = new HashMap<>();
    for (String conceptMapUri : sortedMapsToLoad) {
        ConceptMap map = mapsToLoad.get(conceptMapUri);
        Set<String> children = getMapChildren(map);
        List<BroadcastableConceptMap> childMaps = children.stream().map(child -> broadcastableMaps.get(child)).collect(Collectors.toList());
        BroadcastableConceptMap broadcastableConceptMap = new BroadcastableConceptMap(conceptMapUri, groupedMappings.getOrDefault(conceptMapUri, Collections.emptyList()), childMaps);
        broadcastableMaps.put(conceptMapUri, broadcastableConceptMap);
    }
    JavaSparkContext ctx = new JavaSparkContext(getMaps().sparkSession().sparkContext());
    return ctx.broadcast(new BroadcastableMappings(broadcastableMaps));
}
Also used : SparkRowConverter(com.cerner.bunsen.spark.SparkRowConverter) Dataset(org.apache.spark.sql.Dataset) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) ConceptMapGroupComponent(org.hl7.fhir.dstu3.model.ConceptMap.ConceptMapGroupComponent) HashMap(java.util.HashMap) Deque(java.util.Deque) TargetElementComponent(org.hl7.fhir.dstu3.model.ConceptMap.TargetElementComponent) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Mapping(com.cerner.bunsen.spark.codes.Mapping) BroadcastableConceptMap(com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap) UrlAndVersion(com.cerner.bunsen.spark.codes.UrlAndVersion) FhirVersionEnum(ca.uhn.fhir.context.FhirVersionEnum) Map(java.util.Map) org.apache.spark.sql.functions.lit(org.apache.spark.sql.functions.lit) ConceptMap(org.hl7.fhir.dstu3.model.ConceptMap) BroadcastableMappings(com.cerner.bunsen.spark.codes.broadcast.BroadcastableMappings) JavaRDD(org.apache.spark.api.java.JavaRDD) SparkSession(org.apache.spark.sql.SparkSession) Broadcast(org.apache.spark.broadcast.Broadcast) SourceElementComponent(org.hl7.fhir.dstu3.model.ConceptMap.SourceElementComponent) Iterator(java.util.Iterator) UriType(org.hl7.fhir.dstu3.model.UriType) Set(java.util.Set) Row(org.apache.spark.sql.Row) Collectors(java.util.stream.Collectors) List(java.util.List) Encoder(org.apache.spark.sql.Encoder) ConceptMapGroupUnmappedMode(org.hl7.fhir.dstu3.model.ConceptMap.ConceptMapGroupUnmappedMode) FhirContexts(com.cerner.bunsen.FhirContexts) FHIRException(org.hl7.fhir.exceptions.FHIRException) Function(org.apache.spark.api.java.function.Function) ArrayDeque(java.util.ArrayDeque) AbstractConceptMaps(com.cerner.bunsen.spark.codes.base.AbstractConceptMaps) Collections(java.util.Collections) BroadcastableConceptMap(com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap) HashMap(java.util.HashMap) Mapping(com.cerner.bunsen.spark.codes.Mapping) BroadcastableMappings(com.cerner.bunsen.spark.codes.broadcast.BroadcastableMappings) ArrayList(java.util.ArrayList) List(java.util.List) JavaSparkContext(org.apache.spark.api.java.JavaSparkContext) BroadcastableConceptMap(com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap) ConceptMap(org.hl7.fhir.dstu3.model.ConceptMap)

Example 4 with BroadcastableConceptMap

use of com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap in project bunsen by cerner.

the class BroadcastableMappingsTest method testFoundMapping.

@Test
public void testFoundMapping() {
    BroadcastableConceptMap broadcastableConceptMap = broadcast.getValue().getBroadcastConceptMap("uri:test:concept:map");
    BroadcastableConceptMap.CodeValue value = broadcastableConceptMap.getTarget("uri:test:source:system", "abc").get(0);
    Assert.assertEquals("uri:test:target:system", value.getSystem());
    Assert.assertEquals("123", value.getValue());
}
Also used : BroadcastableConceptMap(com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap) Test(org.junit.Test)

Aggregations

BroadcastableConceptMap (com.cerner.bunsen.spark.codes.broadcast.BroadcastableConceptMap)4 Test (org.junit.Test)3 FhirVersionEnum (ca.uhn.fhir.context.FhirVersionEnum)1 FhirContexts (com.cerner.bunsen.FhirContexts)1 SparkRowConverter (com.cerner.bunsen.spark.SparkRowConverter)1 Mapping (com.cerner.bunsen.spark.codes.Mapping)1 UrlAndVersion (com.cerner.bunsen.spark.codes.UrlAndVersion)1 AbstractConceptMaps (com.cerner.bunsen.spark.codes.base.AbstractConceptMaps)1 BroadcastableMappings (com.cerner.bunsen.spark.codes.broadcast.BroadcastableMappings)1 ArrayDeque (java.util.ArrayDeque)1 ArrayList (java.util.ArrayList)1 Collections (java.util.Collections)1 Deque (java.util.Deque)1 HashMap (java.util.HashMap)1 HashSet (java.util.HashSet)1 Iterator (java.util.Iterator)1 List (java.util.List)1 Map (java.util.Map)1 Set (java.util.Set)1 Collectors (java.util.stream.Collectors)1