Search in sources :

Example 1 with Duration

use of org.eclipse.sw360.datahandler.common.Duration in project sw360portal by sw360.

the class RemoteAttachmentDownloader method retrieveRemoteAttachments.

public static int retrieveRemoteAttachments(Supplier<HttpClient> httpClient, String dbAttachments, Duration downloadTimeout) throws MalformedURLException {
    AttachmentConnector attachmentConnector = new AttachmentConnector(httpClient, dbAttachments, downloadTimeout);
    AttachmentRepository attachmentRepository = new AttachmentRepository(new DatabaseConnector(httpClient, dbAttachments));
    List<AttachmentContent> remoteAttachments = attachmentRepository.getOnlyRemoteAttachments();
    log.info(format("we have %d remote attachments to retrieve", remoteAttachments.size()));
    int count = 0;
    for (AttachmentContent attachmentContent : remoteAttachments) {
        if (!attachmentContent.isOnlyRemote()) {
            log.info(format("skipping attachment (%s), which should already be available", attachmentContent.getId()));
            continue;
        }
        String attachmentContentId = attachmentContent.getId();
        log.info(format("retrieving attachment (%s) {filename=%s}", attachmentContentId, attachmentContent.getFilename()));
        log.debug("url is " + attachmentContent.getRemoteUrl());
        InputStream content = null;
        try {
            content = attachmentConnector.unsafeGetAttachmentStream(attachmentContent);
            if (content == null) {
                log.error("null content retrieving attachment " + attachmentContentId);
                continue;
            }
            try {
                long length = length(content);
                log.info(format("retrieved attachment (%s), it was %d bytes long", attachmentContentId, length));
                count++;
            } catch (IOException e) {
                log.error("attachment was downloaded but somehow not available in database " + attachmentContentId, e);
            }
        } catch (SW360Exception e) {
            log.error("cannot retrieve attachment " + attachmentContentId, e);
        } finally {
            closeQuietly(content, log);
        }
    }
    return count;
}
Also used : DatabaseConnector(org.eclipse.sw360.datahandler.couchdb.DatabaseConnector) InputStream(java.io.InputStream) AttachmentContent(org.eclipse.sw360.datahandler.thrift.attachments.AttachmentContent) IOException(java.io.IOException) AttachmentConnector(org.eclipse.sw360.datahandler.couchdb.AttachmentConnector) SW360Exception(org.eclipse.sw360.datahandler.thrift.SW360Exception)

Example 2 with Duration

use of org.eclipse.sw360.datahandler.common.Duration in project sw360portal by sw360.

the class RemoteAttachmentDownloader method main.

public static void main(String[] args) throws MalformedURLException {
    Duration downloadTimeout = durationOf(30, TimeUnit.SECONDS);
    retrieveRemoteAttachments(DatabaseSettings.getConfiguredHttpClient(), DatabaseSettings.COUCH_DB_ATTACHMENTS, downloadTimeout);
}
Also used : Duration(org.eclipse.sw360.datahandler.common.Duration)

Aggregations

IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 Duration (org.eclipse.sw360.datahandler.common.Duration)1 AttachmentConnector (org.eclipse.sw360.datahandler.couchdb.AttachmentConnector)1 DatabaseConnector (org.eclipse.sw360.datahandler.couchdb.DatabaseConnector)1 SW360Exception (org.eclipse.sw360.datahandler.thrift.SW360Exception)1 AttachmentContent (org.eclipse.sw360.datahandler.thrift.attachments.AttachmentContent)1