Search in sources :

Example 16 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project nifi by apache.

the class TestGeoEnrichIP method evaluatingExpressionLanguageShouldAndFindingIpFieldWithSuccessfulLookUpShouldFlowToFoundRelationship.

@Test
public void evaluatingExpressionLanguageShouldAndFindingIpFieldWithSuccessfulLookUpShouldFlowToFoundRelationship() throws Exception {
    testRunner.setProperty(GeoEnrichIP.GEO_DATABASE_FILE, "./");
    testRunner.setProperty(GeoEnrichIP.IP_ADDRESS_ATTRIBUTE, "${ip.fields:substringBefore(',')}");
    final CityResponse cityResponse = getNullLatAndLongCityResponse();
    when(databaseReader.city(InetAddress.getByName("1.2.3.4"))).thenReturn(cityResponse);
    final Map<String, String> attributes = new HashMap<>();
    attributes.put("ip.fields", "ip0,ip1,ip2");
    attributes.put("ip0", "1.2.3.4");
    testRunner.enqueue(new byte[0], attributes);
    testRunner.run();
    List<MockFlowFile> notFound = testRunner.getFlowFilesForRelationship(GeoEnrichIP.REL_NOT_FOUND);
    List<MockFlowFile> found = testRunner.getFlowFilesForRelationship(GeoEnrichIP.REL_FOUND);
    assertEquals(0, notFound.size());
    assertEquals(1, found.size());
    FlowFile finishedFound = found.get(0);
    assertNotNull(finishedFound.getAttribute("ip0.geo.lookup.micros"));
    assertEquals("Minneapolis", finishedFound.getAttribute("ip0.geo.city"));
    assertNull(finishedFound.getAttribute("ip0.geo.latitude"));
    assertNull(finishedFound.getAttribute("ip0.geo.longitude"));
    assertEquals("Minnesota", finishedFound.getAttribute("ip0.geo.subdivision.0"));
    assertEquals("MN", finishedFound.getAttribute("ip0.geo.subdivision.isocode.0"));
    assertNull(finishedFound.getAttribute("ip0.geo.subdivision.1"));
    assertEquals("TT", finishedFound.getAttribute("ip0.geo.subdivision.isocode.1"));
    assertEquals("United States of America", finishedFound.getAttribute("ip0.geo.country"));
    assertEquals("US", finishedFound.getAttribute("ip0.geo.country.isocode"));
    assertEquals("55401", finishedFound.getAttribute("ip0.geo.postalcode"));
}
Also used : CityResponse(com.maxmind.geoip2.model.CityResponse) MockFlowFile(org.apache.nifi.util.MockFlowFile) FlowFile(org.apache.nifi.flowfile.FlowFile) MockFlowFile(org.apache.nifi.util.MockFlowFile) HashMap(java.util.HashMap) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 17 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project nifi by apache.

the class IPLookupService method createRecord.

private Record createRecord(final CityResponse city) {
    if (city == null) {
        return null;
    }
    final Map<String, Object> values = new HashMap<>();
    values.put(CitySchema.CITY.getFieldName(), city.getCity().getName());
    final Location location = city.getLocation();
    values.put(CitySchema.ACCURACY.getFieldName(), location.getAccuracyRadius());
    values.put(CitySchema.METRO_CODE.getFieldName(), location.getMetroCode());
    values.put(CitySchema.TIMEZONE.getFieldName(), location.getTimeZone());
    values.put(CitySchema.LATITUDE.getFieldName(), location.getLatitude());
    values.put(CitySchema.LONGITUDE.getFieldName(), location.getLongitude());
    values.put(CitySchema.CONTINENT.getFieldName(), city.getContinent().getName());
    values.put(CitySchema.POSTALCODE.getFieldName(), city.getPostal().getCode());
    values.put(CitySchema.COUNTRY.getFieldName(), createRecord(city.getCountry()));
    final Object[] subdivisions = new Object[city.getSubdivisions().size()];
    int i = 0;
    for (final Subdivision subdivision : city.getSubdivisions()) {
        subdivisions[i++] = createRecord(subdivision);
    }
    values.put(CitySchema.SUBDIVISIONS.getFieldName(), subdivisions);
    return new MapRecord(CitySchema.GEO_SCHEMA, values);
}
Also used : MapRecord(org.apache.nifi.serialization.record.MapRecord) HashMap(java.util.HashMap) Subdivision(com.maxmind.geoip2.record.Subdivision) Location(com.maxmind.geoip2.record.Location)

Example 18 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project divolte-collector by divolte.

the class DslRecordMapperTest method shouldMapAllGeoIpFields.

@Test
public void shouldMapAllGeoIpFields() throws IOException, InterruptedException, ClosedServiceException {
    /*
         * Have to work around not being able to create a HttpServerExchange a bit.
         * We setup a actual server just to do a request and capture the HttpServerExchange
         * instance. Then we setup a DslRecordMapper instance with a mock ip2geo lookup service,
         * we then use the previously captured exchange object against our locally created mapper
         * instance to test the ip2geo mapping (using the a mock lookup service).
         */
    setupServer("minimal-mapping.groovy");
    final EventPayload payload = request("http://www.example.com");
    final File geoMappingFile = File.createTempFile("geo-mapping", ".groovy");
    copyResourceToFile("geo-mapping.groovy", geoMappingFile);
    final ImmutableMap<String, Object> mappingConfig = ImmutableMap.of("divolte.mappings.test.mapping_script_file", geoMappingFile.getAbsolutePath(), "divolte.mappings.test.schema_file", avroFile.getAbsolutePath());
    final Config geoConfig = ConfigFactory.parseMap(mappingConfig).withFallback(ConfigFactory.parseResources("base-test-server.conf")).withFallback(ConfigFactory.parseResources("reference-test.conf"));
    final ValidatedConfiguration vc = new ValidatedConfiguration(() -> geoConfig);
    final CityResponse mockResponseWithEverything = loadFromClassPath("/city-response-with-everything.json", new TypeReference<CityResponse>() {
    });
    final Map<String, Object> expectedMapping = loadFromClassPath("/city-response-expected-mapping.json", new TypeReference<Map<String, Object>>() {
    });
    final LookupService mockLookupService = mock(LookupService.class);
    when(mockLookupService.lookup(any())).thenReturn(Optional.of(mockResponseWithEverything));
    final DslRecordMapper mapper = new DslRecordMapper(vc, geoMappingFile.getAbsolutePath(), new Schema.Parser().parse(Resources.toString(Resources.getResource("TestRecord.avsc"), StandardCharsets.UTF_8)), Optional.of(mockLookupService));
    final GenericRecord record = mapper.newRecordFromExchange(payload.event);
    // Validate the results.
    verify(mockLookupService).lookup(any());
    verifyNoMoreInteractions(mockLookupService);
    expectedMapping.forEach((k, v) -> {
        final Object recordValue = record.get(k);
        assertEquals("Property " + k + " not mapped correctly.", v, recordValue);
    });
    Files.delete(geoMappingFile.toPath());
}
Also used : DslRecordMapper(io.divolte.server.recordmapping.DslRecordMapper) Config(com.typesafe.config.Config) ValidatedConfiguration(io.divolte.server.config.ValidatedConfiguration) JsonParser(com.fasterxml.jackson.core.JsonParser) CityResponse(com.maxmind.geoip2.model.CityResponse) LookupService(io.divolte.server.ip2geo.LookupService) GenericRecord(org.apache.avro.generic.GenericRecord) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) EventPayload(io.divolte.server.ServerTestUtils.EventPayload) Test(org.junit.Test)

Example 19 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project quickutil by quickutil.

the class GeoUtil method geoIPByMMDB.

public static GeoDef geoIPByMMDB(String ip) {
    String countryCode = UNKNOWN;
    String country = UNKNOWN;
    String countryChinese = UNKNOWN;
    String stateCode = UNKNOWN;
    String state = UNKNOWN;
    String stateChinese = UNKNOWN;
    String city = UNKNOWN;
    Double latitude = 0.0;
    Double longitude = 0.0;
    try {
        InetAddress ipAddr = InetAddress.getByName(ip);
        CityResponse result;
        result = mmdbReader.city(ipAddr);
        countryCode = result.getCountry().getIsoCode();
        country = result.getCountry().getName();
        countryChinese = countryChineseByCountryCode(countryCode);
        stateCode = result.getMostSpecificSubdivision().getIsoCode();
        state = stateNameByStateCode(countryCode, stateCode);
        stateChinese = stateChineseByStateCode(countryCode, stateCode);
        stateCode = stateCodeByStateCodeNewVersion(countryCode, stateCode);
        city = result.getCity().getName();
        if (countryCode != null && countryCode.equals("CN") && city != null && result.getCity().getNames().containsKey("zh-CN")) {
            if (!result.getCity().getNames().get("zh-CN").endsWith("市")) {
                city = result.getCity().getNames().get("zh-CN") + "市";
            } else {
                city = result.getCity().getNames().get("zh-CN");
            }
        }
        latitude = result.getLocation().getLatitude();
        longitude = result.getLocation().getLongitude();
    } catch (AddressNotFoundException ae) {
    } catch (UnknownHostException ue) {
    } catch (Exception e) {
        LOGGER.error(Symbol.BLANK, e);
    }
    countryCode = (countryCode == null) ? UNKNOWN : countryCode;
    country = (country == null) ? UNKNOWN : country;
    countryChinese = (countryChinese == null) ? UNKNOWN : countryChinese;
    stateCode = (stateCode == null) ? UNKNOWN : stateCode;
    state = (state == null) ? UNKNOWN : state;
    stateChinese = (stateChinese == null) ? UNKNOWN : stateChinese;
    city = (city == null) ? UNKNOWN : city;
    latitude = (latitude == null) ? 0.0 : latitude;
    longitude = (longitude == null) ? 0.0 : longitude;
    return new GeoDef(latitude, longitude, countryCode, country, countryChinese, stateCode, state, stateChinese, city, "");
}
Also used : CityResponse(com.maxmind.geoip2.model.CityResponse) UnknownHostException(java.net.UnknownHostException) AddressNotFoundException(com.maxmind.geoip2.exception.AddressNotFoundException) InetAddress(java.net.InetAddress) UnknownHostException(java.net.UnknownHostException) AddressNotFoundException(com.maxmind.geoip2.exception.AddressNotFoundException) GeoDef(com.quickutil.platform.entity.GeoDef)

Example 20 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project cas by apereo.

the class MaxmindDatabaseGeoLocationServiceTests method verifyCity.

@Test
public void verifyCity() throws Exception {
    val cityReader = mock(DatabaseReader.class);
    val cityResponse = new CityResponse(new City(), new Continent(), new Country(), new Location(10, 100, 40D, 70D, 1, 1, "UTC"), new MaxMind(), new Postal(), new Country(), new RepresentedCountry(), new ArrayList<>(), new Traits());
    when(cityReader.city(any())).thenReturn(cityResponse);
    val service = new MaxmindDatabaseGeoLocationService(cityReader, null);
    val response = service.locate("127.0.0.1");
    assertNotNull(response);
}
Also used : lombok.val(lombok.val) CityResponse(com.maxmind.geoip2.model.CityResponse) Continent(com.maxmind.geoip2.record.Continent) MaxMind(com.maxmind.geoip2.record.MaxMind) RepresentedCountry(com.maxmind.geoip2.record.RepresentedCountry) RepresentedCountry(com.maxmind.geoip2.record.RepresentedCountry) Country(com.maxmind.geoip2.record.Country) City(com.maxmind.geoip2.record.City) Postal(com.maxmind.geoip2.record.Postal) Traits(com.maxmind.geoip2.record.Traits) Location(com.maxmind.geoip2.record.Location) Test(org.junit.jupiter.api.Test)

Aggregations

CityResponse (com.maxmind.geoip2.model.CityResponse)25 InetAddress (java.net.InetAddress)13 Location (com.maxmind.geoip2.record.Location)10 HashMap (java.util.HashMap)10 Country (com.maxmind.geoip2.record.Country)9 IOException (java.io.IOException)9 City (com.maxmind.geoip2.record.City)7 AddressNotFoundException (com.maxmind.geoip2.exception.AddressNotFoundException)6 Continent (com.maxmind.geoip2.record.Continent)6 Subdivision (com.maxmind.geoip2.record.Subdivision)6 GeoIp2Exception (com.maxmind.geoip2.exception.GeoIp2Exception)5 CountryResponse (com.maxmind.geoip2.model.CountryResponse)5 Postal (com.maxmind.geoip2.record.Postal)5 Test (org.junit.Test)5 RepresentedCountry (com.maxmind.geoip2.record.RepresentedCountry)4 UnknownHostException (java.net.UnknownHostException)4 FlowFile (org.apache.nifi.flowfile.FlowFile)4 Traits (com.maxmind.geoip2.record.Traits)3 User (com.earth2me.essentials.User)2 MaxMind (com.maxmind.geoip2.record.MaxMind)2