Search in sources :

Example 1 with QuantityValue

use of org.wikidata.wdtk.datamodel.interfaces.QuantityValue in project OpenRefine by OpenRefine.

the class QuantityScrutinizer method scrutinize.

@Override
public void scrutinize(Snak snak, EntityIdValue entityId, boolean added) {
    if (!added) {
        return;
    }
    if (snak instanceof ValueSnak && ((ValueSnak) snak).getValue() instanceof QuantityValue && added) {
        PropertyIdValue pid = snak.getPropertyId();
        QuantityValue value = (QuantityValue) ((ValueSnak) snak).getValue();
        if (!_fetcher.getConstraintsByType(pid, noBoundsConstraintQid).isEmpty() && (value.getUpperBound() != null || value.getLowerBound() != null)) {
            QAWarning issue = new QAWarning(boundsDisallowedType, pid.getId(), QAWarning.Severity.IMPORTANT, 1);
            issue.setProperty("property_entity", pid);
            issue.setProperty("example_value", value.getNumericValue().toString());
            issue.setProperty("example_item_entity", entityId);
            addIssue(issue);
        }
        if (!_fetcher.getConstraintsByType(pid, integerValuedConstraintQid).isEmpty() && value.getNumericValue().scale() > 0) {
            QAWarning issue = new QAWarning(integerConstraintType, pid.getId(), QAWarning.Severity.IMPORTANT, 1);
            issue.setProperty("property_entity", pid);
            issue.setProperty("example_value", value.getNumericValue().toString());
            issue.setProperty("example_item_entity", entityId);
            addIssue(issue);
        }
        List<Statement> statementList = _fetcher.getConstraintsByType(pid, allowedUnitsConstraintQid);
        Set<ItemIdValue> allowedUnits = null;
        if (!statementList.isEmpty()) {
            AllowedUnitsConstraint allowedUnitsConstraint = new AllowedUnitsConstraint(statementList.get(0));
            allowedUnits = allowedUnitsConstraint.allowedUnits;
        }
        ItemIdValue currentUnit = null;
        if (value.getUnitItemId() != null) {
            currentUnit = value.getUnitItemId();
        }
        if (allowedUnits != null && !allowedUnits.contains(currentUnit)) {
            String issueType = currentUnit == null ? noUnitProvidedType : invalidUnitType;
            QAWarning issue = new QAWarning(issueType, pid.getId(), QAWarning.Severity.IMPORTANT, 1);
            issue.setProperty("property_entity", pid);
            issue.setProperty("example_value", value.getNumericValue().toString());
            issue.setProperty("example_item_entity", entityId);
            if (currentUnit != null) {
                issue.setProperty("unit_entity", value.getUnitItemId());
            }
            addIssue(issue);
        }
    }
}
Also used : PropertyIdValue(org.wikidata.wdtk.datamodel.interfaces.PropertyIdValue) ItemIdValue(org.wikidata.wdtk.datamodel.interfaces.ItemIdValue) QuantityValue(org.wikidata.wdtk.datamodel.interfaces.QuantityValue) Statement(org.wikidata.wdtk.datamodel.interfaces.Statement) ValueSnak(org.wikidata.wdtk.datamodel.interfaces.ValueSnak) QAWarning(org.openrefine.wikidata.qa.QAWarning)

Example 2 with QuantityValue

use of org.wikidata.wdtk.datamodel.interfaces.QuantityValue in project OpenRefine by OpenRefine.

the class LaxValueMatcherTests method testQuantity.

@Test
public void testQuantity() {
    QuantityValue value1 = Datamodel.makeQuantityValue(new BigDecimal("1.234"));
    QuantityValue value2 = Datamodel.makeQuantityValue(new BigDecimal("1.2"), new BigDecimal("1"), new BigDecimal("1.3"));
    QuantityValue value3 = Datamodel.makeQuantityValue(new BigDecimal("1.35"), new BigDecimal("1.25"), new BigDecimal("1.45"));
    QuantityValue value4 = Datamodel.makeQuantityValue(new BigDecimal("1.234"), Datamodel.makeWikidataItemIdValue("Q34"));
    QuantityValue value5 = Datamodel.makeQuantityValue(new BigDecimal("1.5"), new BigDecimal("1"), new BigDecimal("2"));
    assertTrue(SUT.match(value1, value1));
    assertTrue(SUT.match(value1, value2));
    assertTrue(SUT.match(value2, value1));
    assertFalse(SUT.match(value1, value3));
    assertFalse(SUT.match(value3, value1));
    assertFalse(SUT.match(value1, value4));
    assertFalse(SUT.match(value4, value1));
    assertTrue(SUT.match(value2, value2));
    assertTrue(SUT.match(value2, value3));
    assertTrue(SUT.match(value5, value3));
    assertTrue(SUT.match(value3, value5));
}
Also used : QuantityValue(org.wikidata.wdtk.datamodel.interfaces.QuantityValue) BigDecimal(java.math.BigDecimal) Test(org.testng.annotations.Test)

Example 3 with QuantityValue

use of org.wikidata.wdtk.datamodel.interfaces.QuantityValue in project OpenRefine by OpenRefine.

the class WbQuantityExprTest method testExponent.

// for issue #341: https://github.com/Wikidata/Wikidata-Toolkit/issues/341
@Test
public void testExponent() throws SkipSchemaExpressionException, JsonProcessingException {
    setRow("38.4E+3", recon("Q42"));
    QuantityValue val = exprWithUnit.evaluate(ctxt);
    assertEquals("38400", val.getNumericValue().toString());
    assertEquals("38350", val.getLowerBound().toString());
    assertEquals("38450", val.getUpperBound().toString());
}
Also used : QuantityValue(org.wikidata.wdtk.datamodel.interfaces.QuantityValue) Test(org.testng.annotations.Test)

Example 4 with QuantityValue

use of org.wikidata.wdtk.datamodel.interfaces.QuantityValue in project OpenRefine by OpenRefine.

the class LaxValueMatcher method match.

@Override
public boolean match(Value existing, Value added) {
    if (existing instanceof EntityIdValue && added instanceof EntityIdValue) {
        // to mix up entities from different Wikibases in the same data slot
        return ((EntityIdValue) existing).getId().equals(((EntityIdValue) added).getId());
    } else if (existing instanceof StringValue && added instanceof StringValue) {
        // disregard trailing whitespace differences
        String existingStr = ((StringValue) existing).getString().trim();
        String addedStr = ((StringValue) added).getString().trim();
        // if they look like URLs, then http(s) and trailing slashes do not matter
        try {
            URI existingUrl = extraURINormalize(new URI(existingStr).normalize());
            URI addedUrl = extraURINormalize(new URI(addedStr).normalize());
            return existingUrl.equals(addedUrl);
        } catch (URISyntaxException e) {
            // fall back on basic comparison
            ;
        }
        return existingStr.equals(addedStr);
    } else if (existing instanceof MonolingualTextValue && added instanceof MonolingualTextValue) {
        // ignore differences of trailing whitespace
        MonolingualTextValue existingMTV = (MonolingualTextValue) existing;
        MonolingualTextValue addedMTV = (MonolingualTextValue) added;
        return (existingMTV.getLanguageCode().equals(addedMTV.getLanguageCode()) && existingMTV.getText().trim().equals(addedMTV.getText().trim()));
    } else if (existing instanceof QuantityValue && added instanceof QuantityValue) {
        QuantityValue existingQuantity = (QuantityValue) existing;
        QuantityValue addedQuantity = (QuantityValue) added;
        BigDecimal existingLowerBound = existingQuantity.getLowerBound();
        BigDecimal addedLowerBound = addedQuantity.getLowerBound();
        BigDecimal existingUpperBound = existingQuantity.getUpperBound();
        BigDecimal addedUpperBound = addedQuantity.getUpperBound();
        // artificially set bounds for quantities which have neither lower nor upper bounds
        if (existingLowerBound == null && existingUpperBound == null) {
            existingLowerBound = existingQuantity.getNumericValue();
            existingUpperBound = existingQuantity.getNumericValue();
        }
        if (addedLowerBound == null && addedUpperBound == null) {
            addedLowerBound = addedQuantity.getNumericValue();
            addedUpperBound = addedQuantity.getNumericValue();
        }
        if (existingQuantity.getUnit().equals(addedQuantity.getUnit()) && (existingLowerBound != null) && (addedLowerBound != null) && (existingUpperBound != null) && (addedUpperBound != null)) {
            // Consider the two values to be equal when their confidence interval overlaps
            return ((existingLowerBound.compareTo(addedLowerBound) <= 0 && addedLowerBound.compareTo(existingUpperBound) <= 0) || (addedLowerBound.compareTo(existingLowerBound) <= 0 && existingLowerBound.compareTo(addedUpperBound) <= 0));
        }
    } else if (existing instanceof GlobeCoordinatesValue && added instanceof GlobeCoordinatesValue) {
        GlobeCoordinatesValue addedCoords = (GlobeCoordinatesValue) added;
        GlobeCoordinatesValue existingCoords = (GlobeCoordinatesValue) existing;
        if (!addedCoords.getGlobeItemId().getId().equals(existingCoords.getGlobeItemId().getId())) {
            return false;
        }
        double addedMinLon = addedCoords.getLongitude() - addedCoords.getPrecision();
        double addedMaxLon = addedCoords.getLongitude() + addedCoords.getPrecision();
        double addedMinLat = addedCoords.getLatitude() - addedCoords.getPrecision();
        double addedMaxLat = addedCoords.getLatitude() + addedCoords.getPrecision();
        double existingMinLon = existingCoords.getLongitude() - existingCoords.getPrecision();
        double existingMaxLon = existingCoords.getLongitude() + existingCoords.getPrecision();
        double existingMinLat = existingCoords.getLatitude() - existingCoords.getPrecision();
        double existingMaxLat = existingCoords.getLatitude() + existingCoords.getPrecision();
        // return true when the two "rectangles" (in coordinate space) overlap (not strictly)
        return ((addedMinLon <= existingMinLon && addedMinLat <= existingMinLat && existingMinLon <= addedMaxLon && existingMinLat <= addedMaxLat) || (existingMinLon <= addedMinLon && existingMinLat <= addedMinLat && addedMinLon <= existingMaxLon && addedMinLat <= existingMaxLat));
    } else if (existing instanceof TimeValue && added instanceof TimeValue) {
        TimeValue existingTime = (TimeValue) existing;
        TimeValue addedTime = (TimeValue) added;
        if (!existingTime.getPreferredCalendarModel().equals(addedTime.getPreferredCalendarModel())) {
            return false;
        }
        int minPrecision = Math.min(existingTime.getPrecision(), addedTime.getPrecision());
        if (minPrecision <= 9) {
            // the precision is a multiple of years
            long yearPrecision = (long) Math.pow(10, 9 - minPrecision);
            long addedValue = addedTime.getYear() / yearPrecision;
            long existingValue = existingTime.getYear() / yearPrecision;
            return addedValue == existingValue;
        } else if (minPrecision == 10) {
            // month precision
            return (addedTime.getYear() == existingTime.getYear() && addedTime.getMonth() == existingTime.getMonth());
        } else if (minPrecision == 11) {
            // day precision
            return (addedTime.getYear() == existingTime.getYear() && addedTime.getMonth() == existingTime.getMonth() && addedTime.getDay() == existingTime.getDay());
        }
    // TODO possible improvements: bounds support, timezone support
    }
    // fall back to exact comparison for other datatypes
    return existing.equals(added);
}
Also used : GlobeCoordinatesValue(org.wikidata.wdtk.datamodel.interfaces.GlobeCoordinatesValue) QuantityValue(org.wikidata.wdtk.datamodel.interfaces.QuantityValue) EntityIdValue(org.wikidata.wdtk.datamodel.interfaces.EntityIdValue) MonolingualTextValue(org.wikidata.wdtk.datamodel.interfaces.MonolingualTextValue) URISyntaxException(java.net.URISyntaxException) StringValue(org.wikidata.wdtk.datamodel.interfaces.StringValue) URI(java.net.URI) BigDecimal(java.math.BigDecimal) TimeValue(org.wikidata.wdtk.datamodel.interfaces.TimeValue)

Aggregations

QuantityValue (org.wikidata.wdtk.datamodel.interfaces.QuantityValue)4 BigDecimal (java.math.BigDecimal)2 Test (org.testng.annotations.Test)2 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1 QAWarning (org.openrefine.wikidata.qa.QAWarning)1 EntityIdValue (org.wikidata.wdtk.datamodel.interfaces.EntityIdValue)1 GlobeCoordinatesValue (org.wikidata.wdtk.datamodel.interfaces.GlobeCoordinatesValue)1 ItemIdValue (org.wikidata.wdtk.datamodel.interfaces.ItemIdValue)1 MonolingualTextValue (org.wikidata.wdtk.datamodel.interfaces.MonolingualTextValue)1 PropertyIdValue (org.wikidata.wdtk.datamodel.interfaces.PropertyIdValue)1 Statement (org.wikidata.wdtk.datamodel.interfaces.Statement)1 StringValue (org.wikidata.wdtk.datamodel.interfaces.StringValue)1 TimeValue (org.wikidata.wdtk.datamodel.interfaces.TimeValue)1 ValueSnak (org.wikidata.wdtk.datamodel.interfaces.ValueSnak)1