Search in sources :

Example 1 with AtlasEntity

use of org.openstreetmap.atlas.geography.atlas.items.AtlasEntity in project atlas-checks by osmlab.

the class IntegrityCheckSparkJob method executeChecks.

/**
 * Executes all {@link BaseCheck}s on the given {@link Atlas}. Each check runs in a separate
 * thread. The checks go over all {@link AtlasEntity}s and {@link Relation}s.
 * {@link ComplexEntity}s can be processed by using the appropriate {@link Finder} and adding
 * them to the {@link Iterable} of objects.
 *
 * @param atlas
 *            the {@link Atlas} on which the checks will be run
 * @param checksToRun
 *            the set of {@link BaseCheck}s to execute
 * @param configuration
 *            {@link MapRouletteConfiguration} to create a new {@link MapRouletteClient}s
 */
@SuppressWarnings("rawtypes")
private static void executeChecks(final String country, final Atlas atlas, final Set<BaseCheck> checksToRun, final MapRouletteConfiguration configuration) {
    final Pool checkExecutionPool = new Pool(checksToRun.size(), "Check execution pool", POOL_DURATION_BEFORE_KILL);
    checksToRun.stream().filter(check -> check.validCheckForCountry(country)).forEach(check -> checkExecutionPool.queue(new RunnableCheck(country, check, new MultiIterable<>(atlas.items(), atlas.relations(), findComplexEntities(check, atlas)), MapRouletteClient.instance(configuration))));
    checkExecutionPool.close();
}
Also used : SparkJob(org.openstreetmap.atlas.generator.tools.spark.SparkJob) Atlas(org.openstreetmap.atlas.geography.atlas.Atlas) CommonConstants(org.openstreetmap.atlas.checks.constants.CommonConstants) CheckResourceLoader(org.openstreetmap.atlas.checks.base.CheckResourceLoader) SparkFilePath(org.openstreetmap.atlas.generator.tools.spark.utilities.SparkFilePath) LoggerFactory(org.slf4j.LoggerFactory) MapRouletteConfiguration(org.openstreetmap.atlas.checks.maproulette.MapRouletteConfiguration) Relation(org.openstreetmap.atlas.geography.atlas.items.Relation) Finder(org.openstreetmap.atlas.geography.atlas.items.complex.Finder) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) BaseCheck(org.openstreetmap.atlas.checks.base.BaseCheck) MapRouletteClient(org.openstreetmap.atlas.checks.maproulette.MapRouletteClient) AtlasEntity(org.openstreetmap.atlas.geography.atlas.items.AtlasEntity) Configuration(org.openstreetmap.atlas.utilities.configuration.Configuration) ComplexEntity(org.openstreetmap.atlas.geography.atlas.items.complex.ComplexEntity) StringConverter(org.openstreetmap.atlas.utilities.conversion.StringConverter) Map(java.util.Map) ConfigurationResolver(org.openstreetmap.atlas.checks.configuration.ConfigurationResolver) CheckFlagGeoJsonProcessor(org.openstreetmap.atlas.checks.event.CheckFlagGeoJsonProcessor) Duration(org.openstreetmap.atlas.utilities.scalars.Duration) Logger(org.slf4j.Logger) TaskContext(org.apache.spark.TaskContext) CoreException(org.openstreetmap.atlas.exception.CoreException) SparkFileOutput(org.openstreetmap.atlas.generator.tools.spark.utilities.SparkFileOutput) Set(java.util.Set) CommandMap(org.openstreetmap.atlas.utilities.runtime.CommandMap) CheckFlagFileProcessor(org.openstreetmap.atlas.checks.event.CheckFlagFileProcessor) MetricFileGenerator(org.openstreetmap.atlas.checks.event.MetricFileGenerator) Tuple2(scala.Tuple2) Collectors(java.util.stream.Collectors) JavaPairRDD(org.apache.spark.api.java.JavaPairRDD) Pool(org.openstreetmap.atlas.utilities.threads.Pool) Objects(java.util.Objects) Consumer(java.util.function.Consumer) FileSuffix(org.openstreetmap.atlas.streaming.resource.FileSuffix) List(java.util.List) Stream(java.util.stream.Stream) Iterables(org.openstreetmap.atlas.utilities.collections.Iterables) EventService(org.openstreetmap.atlas.checks.event.EventService) SparkFileHelper(org.openstreetmap.atlas.generator.tools.spark.utilities.SparkFileHelper) MultiIterable(org.openstreetmap.atlas.utilities.collections.MultiIterable) Optional(java.util.Optional) Rectangle(org.openstreetmap.atlas.geography.Rectangle) StringList(org.openstreetmap.atlas.utilities.collections.StringList) Time(org.openstreetmap.atlas.utilities.time.Time) Collections(java.util.Collections) Pool(org.openstreetmap.atlas.utilities.threads.Pool)

Example 2 with AtlasEntity

use of org.openstreetmap.atlas.geography.atlas.items.AtlasEntity in project atlas-checks by osmlab.

the class AbbreviatedNameCheck method flag.

/**
 * Flags {@link AtlasObject}s that has names with abbreviations.
 */
@Override
protected Optional<CheckFlag> flag(final AtlasObject object) {
    // Mark OSM identifier as we are processing it
    this.markAsFlagged(this.getUniqueOSMIdentifier((AtlasEntity) object));
    // Fetch the name
    final Optional<String> optionalName = NameTag.getNameOf(object);
    if (!optionalName.isPresent()) {
        return Optional.empty();
    }
    // Lowercase name and parse it into tokens
    final String name = optionalName.get();
    final String lowercaseName = name.toLowerCase(this.getLocale());
    final Set<String> tokens = Sets.newHashSet(NAME_SPLITTER.split(lowercaseName));
    // Flag if it has any abbreviations
    if (tokens.stream().anyMatch(this.abbreviations::contains)) {
        final CheckFlag flag = this.createFlag(object, this.getLocalizedInstruction(0, object.getOsmIdentifier(), name));
        return Optional.of(flag);
    }
    return Optional.empty();
}
Also used : AtlasEntity(org.openstreetmap.atlas.geography.atlas.items.AtlasEntity) CheckFlag(org.openstreetmap.atlas.checks.flag.CheckFlag)

Aggregations

AtlasEntity (org.openstreetmap.atlas.geography.atlas.items.AtlasEntity)2 ArrayList (java.util.ArrayList)1 Collections (java.util.Collections)1 HashSet (java.util.HashSet)1 List (java.util.List)1 Map (java.util.Map)1 Objects (java.util.Objects)1 Optional (java.util.Optional)1 Set (java.util.Set)1 Consumer (java.util.function.Consumer)1 Collectors (java.util.stream.Collectors)1 Stream (java.util.stream.Stream)1 TaskContext (org.apache.spark.TaskContext)1 JavaPairRDD (org.apache.spark.api.java.JavaPairRDD)1 BaseCheck (org.openstreetmap.atlas.checks.base.BaseCheck)1 CheckResourceLoader (org.openstreetmap.atlas.checks.base.CheckResourceLoader)1 ConfigurationResolver (org.openstreetmap.atlas.checks.configuration.ConfigurationResolver)1 CommonConstants (org.openstreetmap.atlas.checks.constants.CommonConstants)1 CheckFlagFileProcessor (org.openstreetmap.atlas.checks.event.CheckFlagFileProcessor)1 CheckFlagGeoJsonProcessor (org.openstreetmap.atlas.checks.event.CheckFlagGeoJsonProcessor)1