Search in sources :

Example 86 with TermedStatementEntityEdit

use of org.openrefine.wikidata.updates.TermedStatementEntityEdit in project OpenRefine by OpenRefine.

the class WikibaseAPIUpdateScheduler method schedule.

@Override
public List<TermedStatementEntityEdit> schedule(List<TermedStatementEntityEdit> updates) {
    List<TermedStatementEntityEdit> result = new ArrayList<>();
    pointerFreeUpdates = new UpdateSequence();
    pointerFullUpdates = new UpdateSequence();
    allPointers = new HashSet<>();
    for (TermedStatementEntityEdit update : updates) {
        splitUpdate(update);
    }
    // Part 1: add all the pointer free updates
    result.addAll(pointerFreeUpdates.getUpdates());
    // Part 1': add the remaining new entities that have not been touched
    Set<EntityIdValue> unseenPointers = new HashSet<>(allPointers);
    unseenPointers.removeAll(pointerFreeUpdates.getSubjects());
    result.addAll(unseenPointers.stream().map(e -> new TermedStatementEntityEditBuilder(e).build()).collect(Collectors.toList()));
    // Part 2: add all the pointer full updates
    result.addAll(pointerFullUpdates.getUpdates());
    return result;
}
Also used : TermedStatementEntityEditBuilder(org.openrefine.wikidata.updates.TermedStatementEntityEditBuilder) TermedStatementEntityEdit(org.openrefine.wikidata.updates.TermedStatementEntityEdit) ArrayList(java.util.ArrayList) EntityIdValue(org.wikidata.wdtk.datamodel.interfaces.EntityIdValue) ReconEntityIdValue(org.openrefine.wikidata.schema.entityvalues.ReconEntityIdValue) HashSet(java.util.HashSet)

Example 87 with TermedStatementEntityEdit

use of org.openrefine.wikidata.updates.TermedStatementEntityEdit in project OpenRefine by OpenRefine.

the class WikibaseAPIUpdateScheduler method splitUpdate.

/**
 * Splits an update into two parts
 *
 * @param update
 */
protected void splitUpdate(TermedStatementEntityEdit update) {
    TermedStatementEntityEditBuilder pointerFreeBuilder = new TermedStatementEntityEditBuilder(update.getEntityId()).addLabels(update.getLabels(), true).addLabels(update.getLabelsIfNew(), false).addDescriptions(update.getDescriptions(), true).addDescriptions(update.getDescriptionsIfNew(), false).addAliases(update.getAliases());
    TermedStatementEntityEditBuilder pointerFullBuilder = new TermedStatementEntityEditBuilder(update.getEntityId());
    for (StatementEdit statement : update.getStatementEdits()) {
        Set<ReconEntityIdValue> pointers = extractor.extractPointers(statement.getStatement());
        if (pointers.isEmpty()) {
            pointerFreeBuilder.addStatement(statement);
        } else {
            pointerFullBuilder.addStatement(statement);
        }
        allPointers.addAll(pointers);
    }
    if (update.isNew()) {
        // If the update is new, we might need to split it
        // in two (if it refers to any other new entity).
        TermedStatementEntityEdit pointerFree = pointerFreeBuilder.build();
        if (!pointerFree.isNull()) {
            pointerFreeUpdates.add(pointerFree);
        }
        TermedStatementEntityEdit pointerFull = pointerFullBuilder.build();
        if (!pointerFull.isEmpty()) {
            pointerFullUpdates.add(pointerFull);
        }
    } else {
        // Otherwise, we just make sure this edit is done after
        // all entity creations.
        pointerFullUpdates.add(update);
    }
}
Also used : TermedStatementEntityEditBuilder(org.openrefine.wikidata.updates.TermedStatementEntityEditBuilder) ReconEntityIdValue(org.openrefine.wikidata.schema.entityvalues.ReconEntityIdValue) TermedStatementEntityEdit(org.openrefine.wikidata.updates.TermedStatementEntityEdit) StatementEdit(org.openrefine.wikidata.updates.StatementEdit)

Example 88 with TermedStatementEntityEdit

use of org.openrefine.wikidata.updates.TermedStatementEntityEdit in project OpenRefine by OpenRefine.

the class EnglishDescriptionScrutinizerTest method testGoodDesc.

@Test
public void testGoodDesc() {
    String description = "good description";
    TermedStatementEntityEdit update = new TermedStatementEntityEditBuilder(TestingData.newIdA).addDescription(Datamodel.makeMonolingualTextValue(description, "en"), true).build();
    scrutinize(update);
    assertNoWarningRaised();
}
Also used : TermedStatementEntityEditBuilder(org.openrefine.wikidata.updates.TermedStatementEntityEditBuilder) TermedStatementEntityEdit(org.openrefine.wikidata.updates.TermedStatementEntityEdit) Test(org.testng.annotations.Test)

Example 89 with TermedStatementEntityEdit

use of org.openrefine.wikidata.updates.TermedStatementEntityEdit in project OpenRefine by OpenRefine.

the class EnglishDescriptionScrutinizerTest method testAwfulDesc.

@Test
public void testAwfulDesc() {
    String description = "An awful description.";
    TermedStatementEntityEdit update = new TermedStatementEntityEditBuilder(TestingData.newIdA).addDescription(Datamodel.makeMonolingualTextValue(description, "en"), true).addLabel(Datamodel.makeMonolingualTextValue(description, "en"), true).build();
    scrutinize(update);
    assertWarningsRaised(EnglishDescriptionScrutinizer.descEndsByPunctuationSign, EnglishDescriptionScrutinizer.descBeginWithUppercase, EnglishDescriptionScrutinizer.descBeginWithArticle);
}
Also used : TermedStatementEntityEditBuilder(org.openrefine.wikidata.updates.TermedStatementEntityEditBuilder) TermedStatementEntityEdit(org.openrefine.wikidata.updates.TermedStatementEntityEdit) Test(org.testng.annotations.Test)

Example 90 with TermedStatementEntityEdit

use of org.openrefine.wikidata.updates.TermedStatementEntityEdit in project OpenRefine by OpenRefine.

the class EnglishDescriptionScrutinizerTest method testEndWithPunctuationSign.

@Test
public void testEndWithPunctuationSign() {
    String description = "description with punctuationSign.";
    TermedStatementEntityEdit update = new TermedStatementEntityEditBuilder(TestingData.newIdA).addDescription(Datamodel.makeMonolingualTextValue(description, "en"), false).build();
    scrutinize(update);
    assertWarningsRaised(EnglishDescriptionScrutinizer.descEndsByPunctuationSign);
}
Also used : TermedStatementEntityEditBuilder(org.openrefine.wikidata.updates.TermedStatementEntityEditBuilder) TermedStatementEntityEdit(org.openrefine.wikidata.updates.TermedStatementEntityEdit) Test(org.testng.annotations.Test)

Aggregations

TermedStatementEntityEdit (org.openrefine.wikidata.updates.TermedStatementEntityEdit)109 TermedStatementEntityEditBuilder (org.openrefine.wikidata.updates.TermedStatementEntityEditBuilder)102 Test (org.testng.annotations.Test)97 ItemIdValue (org.wikidata.wdtk.datamodel.interfaces.ItemIdValue)59 Statement (org.wikidata.wdtk.datamodel.interfaces.Statement)57 ConstraintFetcher (org.openrefine.wikidata.qa.ConstraintFetcher)52 Snak (org.wikidata.wdtk.datamodel.interfaces.Snak)49 StatementImpl (org.wikidata.wdtk.datamodel.implementation.StatementImpl)44 SnakGroup (org.wikidata.wdtk.datamodel.interfaces.SnakGroup)34 ValueSnak (org.wikidata.wdtk.datamodel.interfaces.ValueSnak)18 WikidataRefineTest (org.openrefine.wikidata.testing.WikidataRefineTest)13 ArrayList (java.util.ArrayList)12 StatementEdit (org.openrefine.wikidata.updates.StatementEdit)9 EntityIdValue (org.wikidata.wdtk.datamodel.interfaces.EntityIdValue)6 ReconEntityIdValue (org.openrefine.wikidata.schema.entityvalues.ReconEntityIdValue)5 NoValueSnak (org.wikidata.wdtk.datamodel.interfaces.NoValueSnak)5 PropertyIdValue (org.wikidata.wdtk.datamodel.interfaces.PropertyIdValue)5 IOException (java.io.IOException)4 List (java.util.List)4 Collectors (java.util.stream.Collectors)4