Search in sources :

Example 6 with Extractor

use of org.apache.gobblin.source.extractor.Extractor in project incubator-gobblin by apache.

the class GoogleAnalyticsUnsampledExtractorTest method setup.

private GoogleAnalyticsUnsampledExtractor setup(final ReportCreationStatus status, WorkUnitState wuState, final boolean throwException) throws IOException {
    Extractor actualExtractor = mock(Extractor.class);
    gaService = mock(Analytics.class);
    Management mgmt = mock(Management.class);
    when(gaService.management()).thenReturn(mgmt);
    UnsampledReports req = mock(UnsampledReports.class);
    when(mgmt.unsampledReports()).thenReturn(req);
    getReq = mock(Get.class);
    when(req.get(anyString(), anyString(), anyString(), anyString())).thenReturn(getReq);
    int pollCount = 10;
    final MutableInt countDown = new MutableInt(pollCount);
    when(getReq.execute()).then(new Answer<UnsampledReport>() {

        @Override
        public UnsampledReport answer(InvocationOnMock invocation) throws Throwable {
            countDown.decrement();
            if (countDown.intValue() == 0) {
                UnsampledReport response = new UnsampledReport();
                DriveDownloadDetails details = new DriveDownloadDetails();
                details.setDocumentId(EXPECTED_FILE_ID);
                response.setStatus(status.name()).setDownloadType(DOWNLOAD_TYPE_GOOGLE_DRIVE).setDriveDownloadDetails(details);
                return response;
            } else if (throwException) {
                throw new RuntimeException("Dummy exception.");
            }
            return new UnsampledReport();
        }
    });
    return new GoogleAnalyticsUnsampledExtractor<>(wuState, actualExtractor, gaService);
}
Also used : UnsampledReports(com.google.api.services.analytics.Analytics.Management.UnsampledReports) Management(com.google.api.services.analytics.Analytics.Management) Analytics(com.google.api.services.analytics.Analytics) DriveDownloadDetails(com.google.api.services.analytics.model.UnsampledReport.DriveDownloadDetails) InvocationOnMock(org.mockito.invocation.InvocationOnMock) Get(com.google.api.services.analytics.Analytics.Management.UnsampledReports.Get) MutableInt(org.apache.commons.lang.mutable.MutableInt) Extractor(org.apache.gobblin.source.extractor.Extractor) UnsampledReport(com.google.api.services.analytics.model.UnsampledReport)

Aggregations

Extractor (org.apache.gobblin.source.extractor.Extractor)6 ImmutableMap (com.google.common.collect.ImmutableMap)3 Method (java.lang.reflect.Method)3 Properties (java.util.Properties)3 WorkUnit (org.apache.gobblin.source.workunit.WorkUnit)3 Limiter (org.apache.gobblin.util.limiter.Limiter)3 Test (org.testng.annotations.Test)3 WorkUnitState (org.apache.gobblin.configuration.WorkUnitState)2 Analytics (com.google.api.services.analytics.Analytics)1 Management (com.google.api.services.analytics.Analytics.Management)1 UnsampledReports (com.google.api.services.analytics.Analytics.Management.UnsampledReports)1 Get (com.google.api.services.analytics.Analytics.Management.UnsampledReports.Get)1 UnsampledReport (com.google.api.services.analytics.model.UnsampledReport)1 DriveDownloadDetails (com.google.api.services.analytics.model.UnsampledReport.DriveDownloadDetails)1 Config (com.typesafe.config.Config)1 MutableInt (org.apache.commons.lang.mutable.MutableInt)1 StreamingExtractor (org.apache.gobblin.source.extractor.StreamingExtractor)1 LongWatermark (org.apache.gobblin.source.extractor.extract.LongWatermark)1 GoogleDriveExtractor (org.apache.gobblin.source.extractor.extract.google.GoogleDriveExtractor)1 GoogleDriveFsHelper (org.apache.gobblin.source.extractor.extract.google.GoogleDriveFsHelper)1