Search in sources :

Example 1 with UncheckedJaxbException

use of gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException in project beneficiary-fhir-data by CMSgov.

the class DataSetSubsetter method downloadDataSet.

/**
 * @param options the {@link ExtractionOptions} to use
 * @param dataSetS3KeyPrefix the S3 key prefix (i.e. directory) of the data set to download
 * @param downloadDirectory the Path to the directory to download the RIF files locally to
 * @return the {@link S3RifFile}s that comprise the full 1M beneficiary dummy data set
 */
private static List<RifFile> downloadDataSet(ExtractionOptions options, String dataSetS3KeyPrefix, Path downloadDirectory) {
    AmazonS3 s3Client = S3Utilities.createS3Client(options);
    TransferManager transferManager = TransferManagerBuilder.standard().withS3Client(s3Client).build();
    String dataSetPrefix = "data-random/" + dataSetS3KeyPrefix;
    String manifestSuffix = "1_manifest.xml";
    Path manifestDownloadPath = downloadDirectory.resolve(manifestSuffix);
    if (!Files.exists(manifestDownloadPath)) {
        String manifestKey = String.format("%s/%s", dataSetPrefix, manifestSuffix);
        Download manifestDownload = transferManager.download(options.getS3BucketName(), manifestKey, manifestDownloadPath.toFile());
        try {
            manifestDownload.waitForCompletion();
        } catch (AmazonClientException | InterruptedException e) {
            throw new RuntimeException(e);
        }
    }
    LOGGER.info("Manifest downloaded.");
    DataSetManifest dummyDataSetManifest;
    try {
        JAXBContext jaxbContext = JAXBContext.newInstance(DataSetManifest.class);
        Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
        dummyDataSetManifest = (DataSetManifest) jaxbUnmarshaller.unmarshal(manifestDownloadPath.toFile());
    } catch (JAXBException e) {
        throw new UncheckedJaxbException(e);
    }
    List<RifFile> rifFiles = new ArrayList<>();
    for (DataSetManifestEntry manifestEntry : dummyDataSetManifest.getEntries()) {
        String dataSetFileKey = String.format("%s/%s", dataSetPrefix, manifestEntry.getName());
        Path dataSetFileDownloadPath = downloadDirectory.resolve(manifestEntry.getName());
        if (!Files.exists(dataSetFileDownloadPath)) {
            LOGGER.info("Downloading RIF file: '{}'...", manifestEntry.getName());
            Download dataSetFileDownload = transferManager.download(options.getS3BucketName(), dataSetFileKey, dataSetFileDownloadPath.toFile());
            try {
                dataSetFileDownload.waitForCompletion();
            } catch (AmazonClientException | InterruptedException e) {
                throw new RuntimeException(e);
            }
        }
        RifFile dataSetFile = new LocalRifFile(dataSetFileDownloadPath, manifestEntry.getType());
        rifFiles.add(dataSetFile);
    }
    transferManager.shutdownNow();
    LOGGER.info("Original RIF files ready.");
    return rifFiles;
}
Also used : Path(java.nio.file.Path) AmazonS3(com.amazonaws.services.s3.AmazonS3) TransferManager(com.amazonaws.services.s3.transfer.TransferManager) S3RifFile(gov.cms.bfd.pipeline.ccw.rif.extract.s3.S3RifFile) RifFile(gov.cms.bfd.model.rif.RifFile) LocalRifFile(gov.cms.bfd.pipeline.ccw.rif.extract.LocalRifFile) DataSetManifest(gov.cms.bfd.pipeline.ccw.rif.extract.s3.DataSetManifest) LocalRifFile(gov.cms.bfd.pipeline.ccw.rif.extract.LocalRifFile) AmazonClientException(com.amazonaws.AmazonClientException) JAXBException(javax.xml.bind.JAXBException) ArrayList(java.util.ArrayList) JAXBContext(javax.xml.bind.JAXBContext) UncheckedJaxbException(gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException) Unmarshaller(javax.xml.bind.Unmarshaller) Download(com.amazonaws.services.s3.transfer.Download) DataSetManifestEntry(gov.cms.bfd.pipeline.ccw.rif.extract.s3.DataSetManifest.DataSetManifestEntry)

Example 2 with UncheckedJaxbException

use of gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException in project beneficiary-fhir-data by CMSgov.

the class CodebookVariableReader method unmarshallCodebookXml.

/**
 * @param codebookXmlStream the {@link Codebook} XML {@link InputStream} to unmarshall
 * @return the {@link Codebook} that was unmarshalled from the specified XML
 */
private static Codebook unmarshallCodebookXml(InputStream codebookXmlStream) {
    try {
        JAXBContext jaxbContext = JAXBContext.newInstance(Codebook.class.getPackageName(), Codebook.class.getClassLoader());
        Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
        InputStreamReader codebookXmlReader = new InputStreamReader(codebookXmlStream, StandardCharsets.UTF_8.name());
        Codebook codebook = (Codebook) jaxbUnmarshaller.unmarshal(codebookXmlReader);
        return codebook;
    } catch (JAXBException e) {
        throw new UncheckedJaxbException(e);
    } catch (IllegalArgumentException e) {
        throw new IllegalArgumentException("Null input stream", e);
    } catch (UnsupportedEncodingException e) {
        throw new UncheckedIOException(e);
    }
}
Also used : Codebook(gov.cms.bfd.model.codebook.model.Codebook) SupportedCodebook(gov.cms.bfd.model.codebook.extractor.SupportedCodebook) InputStreamReader(java.io.InputStreamReader) JAXBException(javax.xml.bind.JAXBException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) JAXBContext(javax.xml.bind.JAXBContext) UncheckedIOException(java.io.UncheckedIOException) UncheckedJaxbException(gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException) Unmarshaller(javax.xml.bind.Unmarshaller)

Example 3 with UncheckedJaxbException

use of gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException in project beneficiary-fhir-data by CMSgov.

the class CodebookPdfToXmlApp method writeCodebookXmlToFile.

/**
 * @param codebook the {@link Codebook} to write out
 * @param outputFile the {@link Path} of the file to write the {@link Codebook} out as XML to
 *     (which will be overwritten if it already exists)
 */
private static void writeCodebookXmlToFile(Codebook codebook, Path outputFile) {
    try (FileWriter outputWriter = new FileWriter(outputFile.toFile())) {
        JAXBContext jaxbContext = JAXBContext.newInstance(Codebook.class);
        Marshaller jaxbMarshaller = jaxbContext.createMarshaller();
        jaxbMarshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
        jaxbMarshaller.setProperty(Marshaller.JAXB_ENCODING, StandardCharsets.UTF_8.name());
        jaxbMarshaller.marshal(codebook, outputWriter);
    } catch (JAXBException e) {
        throw new UncheckedJaxbException(e);
    } catch (IOException e) {
        throw new UncheckedIOException(e);
    }
}
Also used : Marshaller(javax.xml.bind.Marshaller) FileWriter(java.io.FileWriter) JAXBException(javax.xml.bind.JAXBException) JAXBContext(javax.xml.bind.JAXBContext) UncheckedIOException(java.io.UncheckedIOException) UncheckedJaxbException(gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException)

Aggregations

UncheckedJaxbException (gov.cms.bfd.sharedutils.exceptions.UncheckedJaxbException)3 JAXBContext (javax.xml.bind.JAXBContext)3 JAXBException (javax.xml.bind.JAXBException)3 UncheckedIOException (java.io.UncheckedIOException)2 Unmarshaller (javax.xml.bind.Unmarshaller)2 AmazonClientException (com.amazonaws.AmazonClientException)1 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 Download (com.amazonaws.services.s3.transfer.Download)1 TransferManager (com.amazonaws.services.s3.transfer.TransferManager)1 SupportedCodebook (gov.cms.bfd.model.codebook.extractor.SupportedCodebook)1 Codebook (gov.cms.bfd.model.codebook.model.Codebook)1 RifFile (gov.cms.bfd.model.rif.RifFile)1 LocalRifFile (gov.cms.bfd.pipeline.ccw.rif.extract.LocalRifFile)1 DataSetManifest (gov.cms.bfd.pipeline.ccw.rif.extract.s3.DataSetManifest)1 DataSetManifestEntry (gov.cms.bfd.pipeline.ccw.rif.extract.s3.DataSetManifest.DataSetManifestEntry)1 S3RifFile (gov.cms.bfd.pipeline.ccw.rif.extract.s3.S3RifFile)1 FileWriter (java.io.FileWriter)1 IOException (java.io.IOException)1 InputStreamReader (java.io.InputStreamReader)1 UnsupportedEncodingException (java.io.UnsupportedEncodingException)1