Search in sources :

Example 1 with FeatureExtractor

use of org.codice.ddf.spatial.geocoding.FeatureExtractor in project ddf by codice.

the class GazetteerUpdateCommandTest method testFeatureIndexing.

@Test
public void testFeatureIndexing() throws Exception {
    String resource = "example.geojson";
    final FeatureExtractor featureExtractor = spy(new FeatureExtractor() {

        @Override
        public void pushFeaturesToExtractionCallback(String resource, ExtractionCallback extractionCallback) throws FeatureExtractionException {
        /* stub */
        }
    });
    final FeatureIndexer featureIndexer = spy(new FeatureIndexer() {

        @Override
        public void updateIndex(String resource, FeatureExtractor featureExtractor, boolean create, IndexCallback callback) throws FeatureExtractionException, FeatureIndexingException {
        /* stub */
        }
    });
    gazetteerUpdateCommand.setResource(resource);
    gazetteerUpdateCommand.setFeatureExtractor(featureExtractor);
    gazetteerUpdateCommand.setFeatureIndexer(featureIndexer);
    gazetteerUpdateCommand.executeWithSubject();
    verify(featureIndexer, times(1)).updateIndex(eq(resource), eq(featureExtractor), eq(false), any(FeatureIndexer.IndexCallback.class));
}
Also used : FeatureExtractor(org.codice.ddf.spatial.geocoding.FeatureExtractor) FeatureIndexingException(org.codice.ddf.spatial.geocoding.FeatureIndexingException) FeatureExtractionException(org.codice.ddf.spatial.geocoding.FeatureExtractionException) Matchers.containsString(org.hamcrest.Matchers.containsString) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) ExtractionCallback(org.codice.ddf.spatial.geocoding.GeoEntryExtractor.ExtractionCallback) FeatureIndexer(org.codice.ddf.spatial.geocoding.FeatureIndexer) Test(org.junit.Test)

Example 2 with FeatureExtractor

use of org.codice.ddf.spatial.geocoding.FeatureExtractor in project ddf by codice.

the class CatalogFeatureIndexerTest method setUp.

@Before
public void setUp() throws SecurityServiceException, InvocationTargetException, FeatureExtractionException, UnsupportedQueryException, SourceUnavailableException, FederationException {
    Security security = mock(Security.class);
    doAnswer(invocation -> {
        Callable callback = (Callable) invocation.getArguments()[0];
        callback.call();
        return null;
    }).when(security).runWithSubjectOrElevate(any(Callable.class));
    featureExtractor = mock(FeatureExtractor.class);
    doAnswer(invocation -> {
        FeatureExtractor.ExtractionCallback callback = (FeatureExtractor.ExtractionCallback) invocation.getArguments()[1];
        callback.extracted(getExampleFeature());
        return null;
    }).when(featureExtractor).pushFeaturesToExtractionCallback(eq(RESOURCE_PATH), any(FeatureExtractor.ExtractionCallback.class));
    catalogFramework = mock(CatalogFramework.class);
    CatalogHelper catalogHelper = new CatalogHelper(FILTER_BUILDER);
    featureIndexer = new CatalogFeatureIndexer(catalogFramework, catalogHelper, generateMetacardType(), security);
    featureIndexer.setSecurity(security);
    QueryResponse queryResponse = mock(QueryResponse.class);
    Result result = mock(Result.class);
    when(result.getMetacard()).thenReturn(getExampleMetacard());
    when(queryResponse.getResults()).thenReturn(Collections.singletonList(result));
    when(catalogFramework.query(any())).thenReturn(queryResponse);
    exampleMetacard = getExampleMetacard();
}
Also used : FeatureExtractor(org.codice.ddf.spatial.geocoding.FeatureExtractor) QueryResponse(ddf.catalog.operation.QueryResponse) CatalogFramework(ddf.catalog.CatalogFramework) Security(org.codice.ddf.security.Security) Callable(java.util.concurrent.Callable) Result(ddf.catalog.data.Result) Before(org.junit.Before)

Example 3 with FeatureExtractor

use of org.codice.ddf.spatial.geocoding.FeatureExtractor in project ddf by codice.

the class GazetteerUpdateCommand method executeWithSubject.

@Override
protected Object executeWithSubject() throws Exception {
    final PrintStream console = System.out;
    final ProgressCallback progressCallback = progress -> {
        console.printf("\r%d%%", progress);
        console.flush();
    };
    final FeatureIndexer.IndexCallback featureIndexCallback = count -> {
        console.printf("\r%d features indexed", count);
        console.flush();
    };
    console.println("Updating...");
    try {
        if (isResourceGeoJSON()) {
            featureIndexer.updateIndex(resource, featureExtractor, create, featureIndexCallback);
        } else {
            geoEntryIndexer.updateIndex(resource, geoEntryExtractor, create, progressCallback);
        }
        console.println("\nDone.");
    } catch (GeoEntryExtractionException | FeatureExtractionException e) {
        LOGGER.info("Error extracting data from resource {}", resource, e);
        console.printf("Could not extract data from resource %s.%n Message: %s%n Check the logs for more details.%n", resource, e.getMessage());
    } catch (GeoEntryIndexingException | FeatureIndexingException e) {
        LOGGER.info("Error indexing data", e);
        console.printf("Could not index the  data.%n Message: %s%n Check the logs for more details.%n", e.getMessage());
    } catch (GeoNamesRemoteDownloadException e) {
        LOGGER.info("Error downloading resource from remote source {}", resource, e);
        console.printf("Could not download the GeoNames file %s.%n  Message: %s%n Check the logs for more details.%n", resource, e.getMessage());
    }
    return null;
}
Also used : PrintStream(java.io.PrintStream) GeoEntryExtractor(org.codice.ddf.spatial.geocoding.GeoEntryExtractor) Logger(org.slf4j.Logger) GeoEntryExtractionException(org.codice.ddf.spatial.geocoding.GeoEntryExtractionException) GeoEntryIndexer(org.codice.ddf.spatial.geocoding.GeoEntryIndexer) FeatureIndexer(org.codice.ddf.spatial.geocoding.FeatureIndexer) LoggerFactory(org.slf4j.LoggerFactory) Argument(org.apache.karaf.shell.api.action.Argument) FeatureExtractor(org.codice.ddf.spatial.geocoding.FeatureExtractor) ProgressCallback(org.codice.ddf.spatial.geocoding.ProgressCallback) Command(org.apache.karaf.shell.api.action.Command) Reference(org.apache.karaf.shell.api.action.lifecycle.Reference) GeoEntryIndexingException(org.codice.ddf.spatial.geocoding.GeoEntryIndexingException) GeoNamesRemoteDownloadException(org.codice.ddf.spatial.geocoding.GeoNamesRemoteDownloadException) Locale(java.util.Locale) FeatureIndexingException(org.codice.ddf.spatial.geocoding.FeatureIndexingException) Service(org.apache.karaf.shell.api.action.lifecycle.Service) Option(org.apache.karaf.shell.api.action.Option) FeatureExtractionException(org.codice.ddf.spatial.geocoding.FeatureExtractionException) SubjectCommands(org.codice.ddf.commands.catalog.SubjectCommands) GeoNamesRemoteDownloadException(org.codice.ddf.spatial.geocoding.GeoNamesRemoteDownloadException) PrintStream(java.io.PrintStream) GeoEntryExtractionException(org.codice.ddf.spatial.geocoding.GeoEntryExtractionException) FeatureIndexingException(org.codice.ddf.spatial.geocoding.FeatureIndexingException) ProgressCallback(org.codice.ddf.spatial.geocoding.ProgressCallback) FeatureExtractionException(org.codice.ddf.spatial.geocoding.FeatureExtractionException) FeatureIndexer(org.codice.ddf.spatial.geocoding.FeatureIndexer) GeoEntryIndexingException(org.codice.ddf.spatial.geocoding.GeoEntryIndexingException)

Aggregations

FeatureExtractor (org.codice.ddf.spatial.geocoding.FeatureExtractor)3 FeatureExtractionException (org.codice.ddf.spatial.geocoding.FeatureExtractionException)2 FeatureIndexer (org.codice.ddf.spatial.geocoding.FeatureIndexer)2 FeatureIndexingException (org.codice.ddf.spatial.geocoding.FeatureIndexingException)2 CatalogFramework (ddf.catalog.CatalogFramework)1 Result (ddf.catalog.data.Result)1 QueryResponse (ddf.catalog.operation.QueryResponse)1 PrintStream (java.io.PrintStream)1 Locale (java.util.Locale)1 Callable (java.util.concurrent.Callable)1 Argument (org.apache.karaf.shell.api.action.Argument)1 Command (org.apache.karaf.shell.api.action.Command)1 Option (org.apache.karaf.shell.api.action.Option)1 Reference (org.apache.karaf.shell.api.action.lifecycle.Reference)1 Service (org.apache.karaf.shell.api.action.lifecycle.Service)1 SubjectCommands (org.codice.ddf.commands.catalog.SubjectCommands)1 Security (org.codice.ddf.security.Security)1 GeoEntryExtractionException (org.codice.ddf.spatial.geocoding.GeoEntryExtractionException)1 GeoEntryExtractor (org.codice.ddf.spatial.geocoding.GeoEntryExtractor)1 ExtractionCallback (org.codice.ddf.spatial.geocoding.GeoEntryExtractor.ExtractionCallback)1