Search in sources :

Example 1 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project elasticsearch by elastic.

the class GeoIpProcessor method retrieveCityGeoData.

private Map<String, Object> retrieveCityGeoData(InetAddress ipAddress) {
    SpecialPermission.check();
    CityResponse response = AccessController.doPrivileged((PrivilegedAction<CityResponse>) () -> {
        try {
            return dbReader.city(ipAddress);
        } catch (AddressNotFoundException e) {
            throw new AddressNotFoundRuntimeException(e);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    });
    Country country = response.getCountry();
    City city = response.getCity();
    Location location = response.getLocation();
    Continent continent = response.getContinent();
    Subdivision subdivision = response.getMostSpecificSubdivision();
    Map<String, Object> geoData = new HashMap<>();
    for (Property property : this.properties) {
        switch(property) {
            case IP:
                geoData.put("ip", NetworkAddress.format(ipAddress));
                break;
            case COUNTRY_ISO_CODE:
                String countryIsoCode = country.getIsoCode();
                if (countryIsoCode != null) {
                    geoData.put("country_iso_code", countryIsoCode);
                }
                break;
            case COUNTRY_NAME:
                String countryName = country.getName();
                if (countryName != null) {
                    geoData.put("country_name", countryName);
                }
                break;
            case CONTINENT_NAME:
                String continentName = continent.getName();
                if (continentName != null) {
                    geoData.put("continent_name", continentName);
                }
                break;
            case REGION_NAME:
                String subdivisionName = subdivision.getName();
                if (subdivisionName != null) {
                    geoData.put("region_name", subdivisionName);
                }
                break;
            case CITY_NAME:
                String cityName = city.getName();
                if (cityName != null) {
                    geoData.put("city_name", cityName);
                }
                break;
            case TIMEZONE:
                String locationTimeZone = location.getTimeZone();
                if (locationTimeZone != null) {
                    geoData.put("timezone", locationTimeZone);
                }
                break;
            case LOCATION:
                Double latitude = location.getLatitude();
                Double longitude = location.getLongitude();
                if (latitude != null && longitude != null) {
                    Map<String, Object> locationObject = new HashMap<>();
                    locationObject.put("lat", latitude);
                    locationObject.put("lon", longitude);
                    geoData.put("location", locationObject);
                }
                break;
        }
    }
    return geoData;
}
Also used : HashMap(java.util.HashMap) City(com.maxmind.geoip2.record.City) Subdivision(com.maxmind.geoip2.record.Subdivision) ConfigurationUtils.newConfigurationException(org.elasticsearch.ingest.ConfigurationUtils.newConfigurationException) IOException(java.io.IOException) AddressNotFoundException(com.maxmind.geoip2.exception.AddressNotFoundException) ElasticsearchParseException(org.elasticsearch.ElasticsearchParseException) CityResponse(com.maxmind.geoip2.model.CityResponse) Continent(com.maxmind.geoip2.record.Continent) AddressNotFoundException(com.maxmind.geoip2.exception.AddressNotFoundException) Country(com.maxmind.geoip2.record.Country) ConfigurationUtils.readStringProperty(org.elasticsearch.ingest.ConfigurationUtils.readStringProperty) ConfigurationUtils.readBooleanProperty(org.elasticsearch.ingest.ConfigurationUtils.readBooleanProperty) Location(com.maxmind.geoip2.record.Location)

Example 2 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project GNS by MobilityFirst.

the class ActiveBlockingQuerier method getLocations.

@Override
public ScriptObjectMirror getLocations(ScriptObjectMirror ipList) throws ActiveException {
    // convert ipList to a JSONArray
    JSONArray arr = null;
    try {
        arr = new JSONArray("[" + ipList.callMember("toString") + "]");
    } catch (JSONException e) {
        e.printStackTrace();
        throw new ActiveException(e.getMessage());
    }
    // resolve ip one by one
    JSONObject obj = new JSONObject();
    for (int i = 0; i < arr.length(); i++) {
        try {
            String ip = arr.getString(i);
            CityResponse loc = GeoIPUtils.getLocation_City(ip, dbReader);
            if (loc != null) {
                JSONObject value = new JSONObject();
                value.put("latitude", loc.getLocation().getLatitude());
                value.put("longitude", loc.getLocation().getLongitude());
                // continent of the location
                value.put("continent", loc.getContinent().getCode());
                obj.put(ip, value);
            }
        } catch (JSONException e) {
            continue;
        }
    }
    return string2JS(obj.toString());
}
Also used : CityResponse(com.maxmind.geoip2.model.CityResponse) JSONObject(org.json.JSONObject) ActiveException(edu.umass.cs.gnsserver.activecode.prototype.ActiveException) JSONArray(org.json.JSONArray) JSONException(org.json.JSONException)

Example 3 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project nifi by apache.

the class GeoEnrichIP method onTrigger.

@Override
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
    FlowFile flowFile = session.get();
    if (flowFile == null) {
        return;
    }
    final DatabaseReader dbReader = databaseReaderRef.get();
    final String ipAttributeName = context.getProperty(IP_ADDRESS_ATTRIBUTE).evaluateAttributeExpressions(flowFile).getValue();
    final String ipAttributeValue = flowFile.getAttribute(ipAttributeName);
    if (StringUtils.isEmpty(ipAttributeName)) {
        session.transfer(flowFile, REL_NOT_FOUND);
        getLogger().warn("FlowFile '{}' attribute '{}' was empty. Routing to failure", new Object[] { flowFile, IP_ADDRESS_ATTRIBUTE.getDisplayName() });
        return;
    }
    InetAddress inetAddress = null;
    CityResponse response = null;
    try {
        inetAddress = InetAddress.getByName(ipAttributeValue);
    } catch (final IOException ioe) {
        session.transfer(flowFile, REL_NOT_FOUND);
        getLogger().warn("Could not resolve the IP for value '{}', contained within the attribute '{}' in " + "FlowFile '{}'. This is usually caused by issue resolving the appropriate DNS record or " + "providing the processor with an invalid IP address ", new Object[] { ipAttributeValue, IP_ADDRESS_ATTRIBUTE.getDisplayName(), flowFile }, ioe);
        return;
    }
    final StopWatch stopWatch = new StopWatch(true);
    try {
        response = dbReader.city(inetAddress);
        stopWatch.stop();
    } catch (final IOException | GeoIp2Exception ex) {
        // Note IOException is captured again as dbReader also makes InetAddress.getByName() calls.
        // Most name or IP resolutions failure should have been triggered in the try loop above but
        // environmental conditions may trigger errors during the second resolution as well.
        session.transfer(flowFile, REL_NOT_FOUND);
        getLogger().warn("Failure while trying to find enrichment data for {} due to {}", new Object[] { flowFile, ex }, ex);
        return;
    }
    if (response == null) {
        session.transfer(flowFile, REL_NOT_FOUND);
        return;
    }
    final Map<String, String> attrs = new HashMap<>();
    attrs.put(new StringBuilder(ipAttributeName).append(".geo.lookup.micros").toString(), String.valueOf(stopWatch.getDuration(TimeUnit.MICROSECONDS)));
    attrs.put(new StringBuilder(ipAttributeName).append(".geo.city").toString(), response.getCity().getName());
    final Double latitude = response.getLocation().getLatitude();
    if (latitude != null) {
        attrs.put(new StringBuilder(ipAttributeName).append(".geo.latitude").toString(), latitude.toString());
    }
    final Double longitude = response.getLocation().getLongitude();
    if (longitude != null) {
        attrs.put(new StringBuilder(ipAttributeName).append(".geo.longitude").toString(), longitude.toString());
    }
    final Integer accuracy = response.getLocation().getAccuracyRadius();
    if (accuracy != null) {
        attrs.put(new StringBuilder(ipAttributeName).append(".accuracy").toString(), String.valueOf(accuracy));
    }
    int i = 0;
    for (final Subdivision subd : response.getSubdivisions()) {
        attrs.put(new StringBuilder(ipAttributeName).append(".geo.subdivision.").append(i).toString(), subd.getName());
        attrs.put(new StringBuilder(ipAttributeName).append(".geo.subdivision.isocode.").append(i).toString(), subd.getIsoCode());
        i++;
    }
    attrs.put(new StringBuilder(ipAttributeName).append(".geo.country").toString(), response.getCountry().getName());
    attrs.put(new StringBuilder(ipAttributeName).append(".geo.country.isocode").toString(), response.getCountry().getIsoCode());
    attrs.put(new StringBuilder(ipAttributeName).append(".geo.postalcode").toString(), response.getPostal().getCode());
    flowFile = session.putAllAttributes(flowFile, attrs);
    session.transfer(flowFile, REL_FOUND);
}
Also used : FlowFile(org.apache.nifi.flowfile.FlowFile) HashMap(java.util.HashMap) IOException(java.io.IOException) Subdivision(com.maxmind.geoip2.record.Subdivision) GeoIp2Exception(com.maxmind.geoip2.exception.GeoIp2Exception) StopWatch(org.apache.nifi.util.StopWatch) CityResponse(com.maxmind.geoip2.model.CityResponse) DatabaseReader(org.apache.nifi.processors.maxmind.DatabaseReader) InetAddress(java.net.InetAddress)

Example 4 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project nifi by apache.

the class TestGeoEnrichIP method successfulMaxMindResponseShouldFlowToFoundRelationshipWhenLatAndLongAreNotSet.

@Test
public void successfulMaxMindResponseShouldFlowToFoundRelationshipWhenLatAndLongAreNotSet() throws Exception {
    testRunner.setProperty(GeoEnrichIP.GEO_DATABASE_FILE, "./");
    testRunner.setProperty(GeoEnrichIP.IP_ADDRESS_ATTRIBUTE, "ip");
    final CityResponse cityResponse = getNullLatAndLongCityResponse();
    when(databaseReader.city(InetAddress.getByName("1.2.3.4"))).thenReturn(cityResponse);
    final Map<String, String> attributes = new HashMap<>();
    attributes.put("ip", "1.2.3.4");
    testRunner.enqueue(new byte[0], attributes);
    testRunner.run();
    List<MockFlowFile> notFound = testRunner.getFlowFilesForRelationship(GeoEnrichIP.REL_NOT_FOUND);
    List<MockFlowFile> found = testRunner.getFlowFilesForRelationship(GeoEnrichIP.REL_FOUND);
    assertEquals(0, notFound.size());
    assertEquals(1, found.size());
    FlowFile finishedFound = found.get(0);
    assertNotNull(finishedFound.getAttribute("ip.geo.lookup.micros"));
    assertEquals("Minneapolis", finishedFound.getAttribute("ip.geo.city"));
    assertNull(finishedFound.getAttribute("ip.geo.latitude"));
    assertNull(finishedFound.getAttribute("ip.geo.longitude"));
    assertEquals("Minnesota", finishedFound.getAttribute("ip.geo.subdivision.0"));
    assertEquals("MN", finishedFound.getAttribute("ip.geo.subdivision.isocode.0"));
    assertNull(finishedFound.getAttribute("ip.geo.subdivision.1"));
    assertEquals("TT", finishedFound.getAttribute("ip.geo.subdivision.isocode.1"));
    assertEquals("United States of America", finishedFound.getAttribute("ip.geo.country"));
    assertEquals("US", finishedFound.getAttribute("ip.geo.country.isocode"));
    assertEquals("55401", finishedFound.getAttribute("ip.geo.postalcode"));
}
Also used : CityResponse(com.maxmind.geoip2.model.CityResponse) MockFlowFile(org.apache.nifi.util.MockFlowFile) FlowFile(org.apache.nifi.flowfile.FlowFile) MockFlowFile(org.apache.nifi.util.MockFlowFile) HashMap(java.util.HashMap) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 5 with CityResponse

use of com.maxmind.geoip2.model.CityResponse in project nifi by apache.

the class IPLookupService method doLookup.

private Optional<Record> doLookup(final DatabaseReader databaseReader, final Map<String, Object> coordinates) throws LookupFailureException, InvalidDatabaseException {
    if (coordinates.get(IP_KEY) == null) {
        return Optional.empty();
    }
    final String ipAddress = coordinates.get(IP_KEY).toString();
    final InetAddress inetAddress;
    try {
        inetAddress = InetAddress.getByName(ipAddress);
    } catch (final IOException ioe) {
        getLogger().warn("Could not resolve the IP for value '{}'. This is usually caused by issue resolving the appropriate DNS record or " + "providing the service with an invalid IP address", new Object[] { coordinates }, ioe);
        return Optional.empty();
    }
    final Record geoRecord;
    if (getProperty(LOOKUP_CITY).asBoolean()) {
        final CityResponse cityResponse;
        try {
            cityResponse = databaseReader.city(inetAddress);
        } catch (final InvalidDatabaseException idbe) {
            throw idbe;
        } catch (final Exception e) {
            throw new LookupFailureException("Failed to lookup City information for IP Address " + inetAddress, e);
        }
        geoRecord = createRecord(cityResponse);
    } else {
        geoRecord = null;
    }
    final Record ispRecord;
    if (getProperty(LOOKUP_ISP).asBoolean()) {
        final IspResponse ispResponse;
        try {
            ispResponse = databaseReader.isp(inetAddress);
        } catch (final InvalidDatabaseException idbe) {
            throw idbe;
        } catch (final Exception e) {
            throw new LookupFailureException("Failed to lookup ISP information for IP Address " + inetAddress, e);
        }
        ispRecord = createRecord(ispResponse);
    } else {
        ispRecord = null;
    }
    final String domainName;
    if (getProperty(LOOKUP_DOMAIN).asBoolean()) {
        final DomainResponse domainResponse;
        try {
            domainResponse = databaseReader.domain(inetAddress);
        } catch (final InvalidDatabaseException idbe) {
            throw idbe;
        } catch (final Exception e) {
            throw new LookupFailureException("Failed to lookup Domain information for IP Address " + inetAddress, e);
        }
        domainName = domainResponse == null ? null : domainResponse.getDomain();
    } else {
        domainName = null;
    }
    final String connectionType;
    if (getProperty(LOOKUP_CONNECTION_TYPE).asBoolean()) {
        final ConnectionTypeResponse connectionTypeResponse;
        try {
            connectionTypeResponse = databaseReader.connectionType(inetAddress);
        } catch (final InvalidDatabaseException idbe) {
            throw idbe;
        } catch (final Exception e) {
            throw new LookupFailureException("Failed to lookup Domain information for IP Address " + inetAddress, e);
        }
        if (connectionTypeResponse == null) {
            connectionType = null;
        } else {
            final ConnectionType type = connectionTypeResponse.getConnectionType();
            connectionType = type == null ? null : type.name();
        }
    } else {
        connectionType = null;
    }
    final Record anonymousIpRecord;
    if (getProperty(LOOKUP_ANONYMOUS_IP_INFO).asBoolean()) {
        final AnonymousIpResponse anonymousIpResponse;
        try {
            anonymousIpResponse = databaseReader.anonymousIp(inetAddress);
        } catch (final InvalidDatabaseException idbe) {
            throw idbe;
        } catch (final Exception e) {
            throw new LookupFailureException("Failed to lookup Anonymous IP Information for IP Address " + inetAddress, e);
        }
        anonymousIpRecord = createRecord(anonymousIpResponse);
    } else {
        anonymousIpRecord = null;
    }
    if (geoRecord == null && ispRecord == null && domainName == null && connectionType == null && anonymousIpRecord == null) {
        return Optional.empty();
    }
    return Optional.ofNullable(createContainerRecord(geoRecord, ispRecord, domainName, connectionType, anonymousIpRecord));
}
Also used : ConnectionType(com.maxmind.geoip2.model.ConnectionTypeResponse.ConnectionType) IOException(java.io.IOException) ConnectionTypeResponse(com.maxmind.geoip2.model.ConnectionTypeResponse) LookupFailureException(org.apache.nifi.lookup.LookupFailureException) InvalidDatabaseException(com.maxmind.db.InvalidDatabaseException) IOException(java.io.IOException) AnonymousIpResponse(com.maxmind.geoip2.model.AnonymousIpResponse) CityResponse(com.maxmind.geoip2.model.CityResponse) LookupFailureException(org.apache.nifi.lookup.LookupFailureException) IspResponse(com.maxmind.geoip2.model.IspResponse) InvalidDatabaseException(com.maxmind.db.InvalidDatabaseException) DomainResponse(com.maxmind.geoip2.model.DomainResponse) Record(org.apache.nifi.serialization.record.Record) MapRecord(org.apache.nifi.serialization.record.MapRecord) InetAddress(java.net.InetAddress)

Aggregations

CityResponse (com.maxmind.geoip2.model.CityResponse)25 InetAddress (java.net.InetAddress)13 Location (com.maxmind.geoip2.record.Location)10 HashMap (java.util.HashMap)10 Country (com.maxmind.geoip2.record.Country)9 IOException (java.io.IOException)9 City (com.maxmind.geoip2.record.City)7 AddressNotFoundException (com.maxmind.geoip2.exception.AddressNotFoundException)6 Continent (com.maxmind.geoip2.record.Continent)6 Subdivision (com.maxmind.geoip2.record.Subdivision)6 GeoIp2Exception (com.maxmind.geoip2.exception.GeoIp2Exception)5 CountryResponse (com.maxmind.geoip2.model.CountryResponse)5 Postal (com.maxmind.geoip2.record.Postal)5 Test (org.junit.Test)5 RepresentedCountry (com.maxmind.geoip2.record.RepresentedCountry)4 UnknownHostException (java.net.UnknownHostException)4 FlowFile (org.apache.nifi.flowfile.FlowFile)4 Traits (com.maxmind.geoip2.record.Traits)3 User (com.earth2me.essentials.User)2 MaxMind (com.maxmind.geoip2.record.MaxMind)2