Search in sources :

Example 6 with SourceInfoResponse

use of ddf.catalog.operation.SourceInfoResponse in project ddf by codice.

the class SourceOperations method getFanoutSourceInfo.

/**
     * Retrieves the {@link SourceDescriptor} info for all {@link FederatedSource}s in the fanout
     * configuration, but the all of the source info, e.g., content types, for all of the available
     * {@link FederatedSource}s is packed into one {@link SourceDescriptor} for the
     * fanout configuration with the fanout's site name in it. This keeps the individual
     * {@link FederatedSource}s' source info hidden from the external client.
     */
private SourceInfoResponse getFanoutSourceInfo(SourceInfoRequest sourceInfoRequest) throws SourceUnavailableException {
    SourceInfoResponse response;
    SourceDescriptorImpl sourceDescriptor;
    try {
        // request
        if (sourceInfoRequest == null) {
            throw new IllegalArgumentException("SourceInfoRequest was null");
        }
        Set<SourceDescriptor> sourceDescriptors = new LinkedHashSet<>();
        Set<String> ids = sourceInfoRequest.getSourceIds();
        // specified
        if (ids != null) {
            Optional<String> notLocal = ids.stream().filter(s -> !s.equals(getId())).findFirst();
            if (notLocal.isPresent()) {
                SourceUnavailableException sourceUnavailableException = new SourceUnavailableException("Unknown source: " + notLocal.get());
                LOGGER.debug("Throwing SourceUnavailableException for unknown source: {}", notLocal.get(), sourceUnavailableException);
                throw sourceUnavailableException;
            }
        }
        // Fanout will only add one source descriptor with all the contents
        Set<Source> availableSources = frameworkProperties.getFederatedSources().values().stream().map(source -> frameworkProperties.getSourcePoller().getCachedSource(source)).filter(source -> source != null && source.isAvailable()).collect(Collectors.toSet());
        Set<ContentType> contentTypes = availableSources.stream().map(source -> frameworkProperties.getSourcePoller().getCachedSource(source)).filter(source -> source.getContentTypes() != null).map(Source::getContentTypes).flatMap(Collection::stream).collect(Collectors.toSet());
        if (catalog != null) {
            Source localSource = frameworkProperties.getSourcePoller().getCachedSource(catalog);
            if (localSource != null && localSource.isAvailable()) {
                availableSources.add(localSource);
                contentTypes.addAll(localSource.getContentTypes());
            }
        }
        // only reveal this sourceDescriptor, not the federated sources
        sourceDescriptor = new SourceDescriptorImpl(this.getId(), contentTypes);
        if (this.getVersion() != null) {
            sourceDescriptor.setVersion(this.getVersion());
        }
        sourceDescriptor.setAvailable(availableSources.size() > 0);
        sourceDescriptors.add(sourceDescriptor);
        response = new SourceInfoResponseImpl(sourceInfoRequest, null, sourceDescriptors);
    } catch (RuntimeException re) {
        throw new SourceUnavailableException("Exception during runtime while performing getSourceInfo", re);
    }
    return response;
}
Also used : LinkedHashSet(java.util.LinkedHashSet) SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) LoggerFactory(org.slf4j.LoggerFactory) TreeSet(java.util.TreeSet) HashSet(java.util.HashSet) Source(ddf.catalog.source.Source) SourceDescriptor(ddf.catalog.source.SourceDescriptor) StorageProvider(ddf.catalog.content.StorageProvider) SourceInfoResponseImpl(ddf.catalog.operation.impl.SourceInfoResponseImpl) SourceDescriptorComparator(ddf.catalog.util.impl.SourceDescriptorComparator) LinkedHashSet(java.util.LinkedHashSet) DescribableImpl(ddf.catalog.util.impl.DescribableImpl) SourceInfoResponse(ddf.catalog.operation.SourceInfoResponse) ContentType(ddf.catalog.data.ContentType) Logger(org.slf4j.Logger) SourceDescriptorImpl(ddf.catalog.source.impl.SourceDescriptorImpl) Collection(java.util.Collection) FederatedSource(ddf.catalog.source.FederatedSource) Set(java.util.Set) Collectors(java.util.stream.Collectors) FrameworkProperties(ddf.catalog.impl.FrameworkProperties) SourceInfoRequest(ddf.catalog.operation.SourceInfoRequest) List(java.util.List) CatalogProvider(ddf.catalog.source.CatalogProvider) ServiceUnavailableException(org.osgi.service.blueprint.container.ServiceUnavailableException) Optional(java.util.Optional) SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) SourceDescriptor(ddf.catalog.source.SourceDescriptor) ContentType(ddf.catalog.data.ContentType) SourceDescriptorImpl(ddf.catalog.source.impl.SourceDescriptorImpl) Source(ddf.catalog.source.Source) FederatedSource(ddf.catalog.source.FederatedSource) SourceInfoResponseImpl(ddf.catalog.operation.impl.SourceInfoResponseImpl) SourceInfoResponse(ddf.catalog.operation.SourceInfoResponse)

Example 7 with SourceInfoResponse

use of ddf.catalog.operation.SourceInfoResponse in project ddf by codice.

the class CatalogFrameworkImplTest method testGetFederatedSourcesDuplicates.

@Test
public void testGetFederatedSourcesDuplicates() {
    List<FederatedSource> federatedSources = createDefaultFederatedSourceList(true);
    // Duplicate Site
    FederatedSource siteC2 = new MockSource("C", "Site C2", "v1.0", "DDF", null, true, new Date());
    federatedSources.add(siteC2);
    // Expected Sites
    List<FederatedSource> expectedSources = createDefaultFederatedSourceList(true);
    // Mock register the federated sources in the container
    // Mock the source poller
    SourcePoller mockPoller = mock(SourcePoller.class);
    when(mockPoller.getCachedSource(isA(Source.class))).thenReturn(null);
    FrameworkProperties frameworkProperties = new FrameworkProperties();
    frameworkProperties.setSourcePoller(mockPoller);
    Map<String, FederatedSource> sources = new HashMap<>();
    for (FederatedSource federatedSource : expectedSources) {
        sources.put(federatedSource.getId(), federatedSource);
    }
    frameworkProperties.setFederatedSources(sources);
    CatalogFrameworkImpl framework = createFramework(frameworkProperties);
    // Returned Sites
    SourceInfoRequest request = new SourceInfoRequestEnterprise(true);
    SourceInfoResponse response = null;
    try {
        response = framework.getSourceInfo(request);
    } catch (SourceUnavailableException e) {
        LOGGER.debug("SourceUnavilable", e);
        fail();
    }
    Set<SourceDescriptor> sourceDescriptors = response.getSourceInfo();
    // should contain ONLY the original federated sites and the catalog framework's
    // site info (even though it has no local catalog provider configured) - hence,
    // the "+1"
    assertEquals(expectedSources.size(), sourceDescriptors.size());
}
Also used : SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) SourceDescriptor(ddf.catalog.source.SourceDescriptor) HashMap(java.util.HashMap) Matchers.anyString(org.mockito.Matchers.anyString) SourceInfoRequestEnterprise(ddf.catalog.operation.impl.SourceInfoRequestEnterprise) Date(java.util.Date) Source(ddf.catalog.source.Source) ByteSource(com.google.common.io.ByteSource) CachedSource(ddf.catalog.util.impl.CachedSource) FederatedSource(ddf.catalog.source.FederatedSource) SourcePoller(ddf.catalog.util.impl.SourcePoller) FederatedSource(ddf.catalog.source.FederatedSource) SourceInfoRequest(ddf.catalog.operation.SourceInfoRequest) SourceInfoResponse(ddf.catalog.operation.SourceInfoResponse) Test(org.junit.Test)

Example 8 with SourceInfoResponse

use of ddf.catalog.operation.SourceInfoResponse in project ddf by codice.

the class CatalogFrameworkImplTest method testGetUnavailableFederatedSources.

@Test
public void testGetUnavailableFederatedSources() {
    List<FederatedSource> federatedSources = createDefaultFederatedSourceList(false);
    CatalogProvider catalogProvider = mock(CatalogProvider.class);
    // Mock register the federated sources in the container
    SourcePollerRunner runner = new SourcePollerRunner();
    SourcePoller poller = new SourcePoller(runner);
    for (FederatedSource source : federatedSources) {
        runner.bind(source);
    }
    runner.bind(catalogProvider);
    FrameworkProperties frameworkProperties = new FrameworkProperties();
    frameworkProperties.setSourcePoller(poller);
    Map<String, FederatedSource> sources = new HashMap<>();
    for (FederatedSource federatedSource : federatedSources) {
        sources.put(federatedSource.getId(), federatedSource);
    }
    frameworkProperties.setFederatedSources(sources);
    frameworkProperties.setCatalogProviders(Collections.singletonList(catalogProvider));
    CatalogFrameworkImpl framework = createFramework(frameworkProperties);
    SourceInfoRequest request = new SourceInfoRequestEnterprise(true);
    SourceInfoResponse response = null;
    try {
        response = framework.getSourceInfo(request);
    } catch (SourceUnavailableException e) {
        fail();
    }
    Set<SourceDescriptor> sourceDescriptors = response.getSourceInfo();
    for (SourceDescriptor descriptor : sourceDescriptors) {
        LOGGER.debug("Descriptor id: {}", descriptor.getSourceId());
        if (StringUtils.isNotBlank(descriptor.getId())) {
            assertFalse(descriptor.isAvailable());
            // No contentTypes should be listed if the source is unavailable
            assertTrue(descriptor.getContentTypes().isEmpty());
        }
    }
    // The "+1" is to account for the CatalogFramework source descriptor.
    // Even if no local catalog provider is configured, the catalog
    // framework's
    // site info is included in the SourceDescriptos list.
    assertEquals(federatedSources.size() + 1, sourceDescriptors.size());
}
Also used : SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) SourceDescriptor(ddf.catalog.source.SourceDescriptor) HashMap(java.util.HashMap) Matchers.anyString(org.mockito.Matchers.anyString) SourceInfoRequestEnterprise(ddf.catalog.operation.impl.SourceInfoRequestEnterprise) SourcePoller(ddf.catalog.util.impl.SourcePoller) FederatedSource(ddf.catalog.source.FederatedSource) CatalogProvider(ddf.catalog.source.CatalogProvider) SourceInfoRequest(ddf.catalog.operation.SourceInfoRequest) SourcePollerRunner(ddf.catalog.util.impl.SourcePollerRunner) SourceInfoResponse(ddf.catalog.operation.SourceInfoResponse) Test(org.junit.Test)

Example 9 with SourceInfoResponse

use of ddf.catalog.operation.SourceInfoResponse in project ddf by codice.

the class CatalogFrameworkImplTest method testGetAllSiteNames.

@Test
public void testGetAllSiteNames() {
    String frameworkName = "DDF";
    CatalogProvider provider = new MockMemoryProvider("Provider", "Provider", "v1.0", "DDF", new HashSet<ContentType>(), true, new Date());
    List<FederatedSource> federatedSources = createDefaultFederatedSourceList(true);
    // Expected Set of Names
    Set<String> expectedNameSet = new HashSet<String>();
    for (FederatedSource curSite : federatedSources) {
        expectedNameSet.add(curSite.getId());
    }
    // Mock register the provider in the container
    // Mock the source poller
    SourcePoller mockPoller = mock(SourcePoller.class);
    when(mockPoller.getCachedSource(isA(Source.class))).thenReturn(null);
    FrameworkProperties frameworkProperties = new FrameworkProperties();
    frameworkProperties.setSourcePoller(mockPoller);
    Map<String, FederatedSource> sources = new HashMap<>();
    for (FederatedSource federatedSource : federatedSources) {
        sources.put(federatedSource.getId(), federatedSource);
    }
    frameworkProperties.setFederatedSources(sources);
    frameworkProperties.setCatalogProviders(Collections.singletonList(provider));
    CatalogFrameworkImpl framework = createFramework(frameworkProperties);
    framework.setId(frameworkName);
    // Returned Set of Names
    // Returned Sites
    SourceInfoRequest request = new SourceInfoRequestEnterprise(true);
    SourceInfoResponse response = null;
    try {
        response = framework.getSourceInfo(request);
    } catch (SourceUnavailableException e) {
        LOGGER.debug("SourceUnavilable", e);
        fail();
    }
    assert (response != null);
    Set<SourceDescriptor> sourceDescriptors = response.getSourceInfo();
    // should contain ONLY the original federated sites
    assertEquals(expectedNameSet.size(), sourceDescriptors.size());
    Set<String> returnedSourceIds = new HashSet<String>();
    for (SourceDescriptor sd : sourceDescriptors) {
        returnedSourceIds.add(sd.getSourceId());
    }
    for (String id : returnedSourceIds) {
        LOGGER.debug("returned sourceId: {}", id);
    }
    assertTrue(expectedNameSet.equals(returnedSourceIds));
}
Also used : SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) SourceDescriptor(ddf.catalog.source.SourceDescriptor) ContentType(ddf.catalog.data.ContentType) HashMap(java.util.HashMap) Matchers.anyString(org.mockito.Matchers.anyString) SourceInfoRequestEnterprise(ddf.catalog.operation.impl.SourceInfoRequestEnterprise) Date(java.util.Date) Source(ddf.catalog.source.Source) ByteSource(com.google.common.io.ByteSource) CachedSource(ddf.catalog.util.impl.CachedSource) FederatedSource(ddf.catalog.source.FederatedSource) SourcePoller(ddf.catalog.util.impl.SourcePoller) FederatedSource(ddf.catalog.source.FederatedSource) CatalogProvider(ddf.catalog.source.CatalogProvider) SourceInfoRequest(ddf.catalog.operation.SourceInfoRequest) SourceInfoResponse(ddf.catalog.operation.SourceInfoResponse) HashSet(java.util.HashSet) Test(org.junit.Test)

Example 10 with SourceInfoResponse

use of ddf.catalog.operation.SourceInfoResponse in project ddf by codice.

the class CatalogFrameworkImplTest method testGetFederatedSources.

@Test
public void testGetFederatedSources() {
    SourceInfoRequest request = new SourceInfoRequestEnterprise(true);
    SourceInfoResponse response = null;
    try {
        response = framework.getSourceInfo(request);
    } catch (SourceUnavailableException e) {
        fail();
    }
    Set<SourceDescriptor> sourceDescriptors = response.getSourceInfo();
    for (SourceDescriptor descriptor : sourceDescriptors) {
        LOGGER.debug("Descriptor id: {}", descriptor.getSourceId());
    }
    // The "+1" is to account for the CatalogFramework source descriptor.
    // Even if no local catalog provider is configured, the catalog framework's
    // site info is included in the SourceDescriptos list.
    assertEquals(federatedSources.size() + 1, sourceDescriptors.size());
}
Also used : SourceUnavailableException(ddf.catalog.source.SourceUnavailableException) SourceDescriptor(ddf.catalog.source.SourceDescriptor) SourceInfoRequest(ddf.catalog.operation.SourceInfoRequest) SourceInfoRequestEnterprise(ddf.catalog.operation.impl.SourceInfoRequestEnterprise) SourceInfoResponse(ddf.catalog.operation.SourceInfoResponse) Test(org.junit.Test)

Aggregations

SourceInfoResponse (ddf.catalog.operation.SourceInfoResponse)13 SourceDescriptor (ddf.catalog.source.SourceDescriptor)13 SourceUnavailableException (ddf.catalog.source.SourceUnavailableException)12 SourceInfoRequestEnterprise (ddf.catalog.operation.impl.SourceInfoRequestEnterprise)9 SourceInfoRequest (ddf.catalog.operation.SourceInfoRequest)6 FederatedSource (ddf.catalog.source.FederatedSource)6 HashSet (java.util.HashSet)6 Test (org.junit.Test)6 CatalogProvider (ddf.catalog.source.CatalogProvider)5 Matchers.anyString (org.mockito.Matchers.anyString)5 ContentType (ddf.catalog.data.ContentType)4 Source (ddf.catalog.source.Source)4 CatalogFramework (ddf.catalog.CatalogFramework)3 SourceInfoResponseImpl (ddf.catalog.operation.impl.SourceInfoResponseImpl)3 SourcePoller (ddf.catalog.util.impl.SourcePoller)3 HashMap (java.util.HashMap)3 Set (java.util.Set)3 ByteSource (com.google.common.io.ByteSource)2 SourceInfoRequestLocal (ddf.catalog.operation.impl.SourceInfoRequestLocal)2 SourceDescriptorImpl (ddf.catalog.source.impl.SourceDescriptorImpl)2