Search in sources :

Example 11 with OrgDisambiguatedEntity

use of org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity in project ORCID-Source by ORCID.

the class LoadRinggoldData method processDeletedIdsLine.

private void processDeletedIdsLine(String[] line) {
    String deletedSourceId = line[0];
    String replacementSourceId = line[1];
    OrgDisambiguatedEntity deletedEntity = orgDisambiguatedDao.findBySourceIdAndSourceType(deletedSourceId, RINGGOLD_SOURCE_TYPE);
    if (deletedEntity != null) {
        LOGGER.info("Deleted ID exists in DB, id={}", deletedSourceId);
        Long deletedEntityId = deletedEntity.getId();
        OrgDisambiguatedEntity replacementEntity = orgDisambiguatedDao.findBySourceIdAndSourceType(replacementSourceId, RINGGOLD_SOURCE_TYPE);
        if (replacementEntity == null) {
            LOGGER.warn("Replacement does not exist, id={}", replacementEntity);
            numDeletionsSkipped++;
        } else {
            Long replacementEntityId = replacementEntity.getId();
            orgDisambiguatedSolrDao.remove(deletedEntityId);
            orgDisambiguatedDao.replace(deletedEntityId, replacementEntityId);
            orgDisambiguatedDao.remove(deletedEntityId);
            numDeleted++;
        }
    }
}
Also used : OrgDisambiguatedEntity(org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity)

Example 12 with OrgDisambiguatedEntity

use of org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity in project ORCID-Source by ORCID.

the class LoadRinggoldData method checkForDuplicates.

private void checkForDuplicates() {
    LOGGER.info("Checking for duplicates");
    List<OrgDisambiguatedEntity> duplicates = orgDisambiguatedDao.findDuplicates();
    for (OrgDisambiguatedEntity duplicate : duplicates) {
        LOGGER.info("Found duplicate: {}\t{}\t{}\t{}\t{}\t{}\t{}", new Object[] { duplicate.getSourceType(), duplicate.getSourceId(), duplicate.getName(), duplicate.getCity(), duplicate.getRegion(), duplicate.getCountry(), duplicate.getOrgType() });
    }
    LOGGER.info("Finished checking for duplicates");
}
Also used : OrgDisambiguatedEntity(org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity)

Example 13 with OrgDisambiguatedEntity

use of org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity in project ORCID-Source by ORCID.

the class OrgDisambiguatedManagerImpl method processOrgsForIndexing.

@Override
public synchronized void processOrgsForIndexing() {
    LOGGER.info("About to process disambiguated orgs for indexing");
    List<OrgDisambiguatedEntity> entities = null;
    do {
        entities = orgDisambiguatedDaoReadOnly.findOrgsByIndexingStatus(IndexingStatus.PENDING, 0, INDEXING_CHUNK_SIZE);
        LOGGER.info("Found chunk of {} disambiguated orgs for indexing", entities.size());
        for (OrgDisambiguatedEntity entity : entities) {
            processDisambiguatedOrgInTransaction(entity);
        }
    } while (!entities.isEmpty());
}
Also used : OrgDisambiguatedEntity(org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity)

Example 14 with OrgDisambiguatedEntity

use of org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity in project ORCID-Source by ORCID.

the class OrgManagerImpl method createUpdate.

@Override
public OrgEntity createUpdate(OrgEntity org, Long orgDisambiguatedId) {
    OrgEntity existingOrg = orgDao.findByNameCityRegionAndCountry(org.getName(), org.getCity(), org.getRegion(), org.getCountry());
    if (existingOrg != null) {
        org = existingOrg;
    }
    if (org.getOrgDisambiguated() == null) {
        OrgDisambiguatedEntity disambiguatedOrg = orgDisambiguatedDao.find(orgDisambiguatedId);
        if (disambiguatedOrg == null) {
            throw new IllegalArgumentException("No such disambiguated org with id=" + orgDisambiguatedId);
        }
        org.setOrgDisambiguated(disambiguatedOrg);
    }
    if (org.getSource() == null) {
        org.setSource(new SourceEntity(sourceManager.retrieveSourceOrcid()));
    }
    return orgDao.merge(org);
}
Also used : SourceEntity(org.orcid.persistence.jpa.entities.SourceEntity) OrgDisambiguatedEntity(org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity) OrgEntity(org.orcid.persistence.jpa.entities.OrgEntity) AmbiguousOrgEntity(org.orcid.persistence.jpa.entities.AmbiguousOrgEntity)

Example 15 with OrgDisambiguatedEntity

use of org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity in project ORCID-Source by ORCID.

the class OrgManagerImpl method writeDisambiguatedOrgs.

@Override
public void writeDisambiguatedOrgs(Writer writer) {
    @SuppressWarnings("resource") CSVWriter csvWriter = new CSVWriter(writer);
    csvWriter.writeNext(DISAMBIGUATED_ORGS_HEADER);
    int firstResult = 0;
    List<OrgDisambiguatedEntity> chunk = null;
    do {
        chunk = orgDisambiguatedDao.getChunk(firstResult, CHUNK_SIZE);
        for (OrgDisambiguatedEntity orgEntity : chunk) {
            String[] line = new String[] { String.valueOf(orgEntity.getId()), orgEntity.getSourceId(), orgEntity.getSourceType(), orgEntity.getOrgType(), orgEntity.getName(), orgEntity.getCity(), orgEntity.getRegion(), orgEntity.getCountry().value(), String.valueOf(orgEntity.getPopularity()) };
            csvWriter.writeNext(line);
        }
        firstResult += chunk.size();
    } while (!chunk.isEmpty());
}
Also used : CSVWriter(au.com.bytecode.opencsv.CSVWriter) OrgDisambiguatedEntity(org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity)

Aggregations

OrgDisambiguatedEntity (org.orcid.persistence.jpa.entities.OrgDisambiguatedEntity)15 Iso3166Country (org.orcid.jaxb.model.message.Iso3166Country)3 FuzzyDateEntity (org.orcid.persistence.jpa.entities.FuzzyDateEntity)2 CSVWriter (au.com.bytecode.opencsv.CSVWriter)1 FileInputStream (java.io.FileInputStream)1 FileNotFoundException (java.io.FileNotFoundException)1 IOException (java.io.IOException)1 Date (java.util.Date)1 DocumentBuilder (javax.xml.parsers.DocumentBuilder)1 DocumentBuilderFactory (javax.xml.parsers.DocumentBuilderFactory)1 ParserConfigurationException (javax.xml.parsers.ParserConfigurationException)1 XPathExpressionException (javax.xml.xpath.XPathExpressionException)1 FundingExternalIdentifiers (org.orcid.core.adapter.impl.jsonidentifiers.FundingExternalIdentifiers)1 AmbiguousOrgEntity (org.orcid.persistence.jpa.entities.AmbiguousOrgEntity)1 OrgEntity (org.orcid.persistence.jpa.entities.OrgEntity)1 SourceEntity (org.orcid.persistence.jpa.entities.SourceEntity)1 OrgDisambiguated (org.orcid.pojo.OrgDisambiguated)1 Document (org.w3c.dom.Document)1 NodeList (org.w3c.dom.NodeList)1 SAXException (org.xml.sax.SAXException)1