Search in sources :

Example 6 with DatabaseReader

use of com.maxmind.geoip2.DatabaseReader in project divolte-collector by divolte.

the class DatabaseLookupService method loadExternalDatabase.

private static DatabaseReader loadExternalDatabase(final Path location) throws IOException {
    /*
         * We have 2 strategies here.
         * 1) If compressed, we load via a stream (which means into memory).
         * 2) If it's not compressed, we pass a File in directly which allows DatabaseReader
         *    to mmap(2) the file.
         */
    final DatabaseReader reader;
    if ("gz".equals(com.google.common.io.Files.getFileExtension(location.toString()))) {
        try (final InputStream rawStream = Files.newInputStream(location);
            final InputStream bufferedStream = new BufferedInputStream(rawStream);
            final InputStream dbStream = new GZIPInputStream(bufferedStream)) {
            logger.debug("Loading compressed GeoIP database: {}", location);
            reader = new DatabaseReader.Builder(dbStream).build();
        }
    } else {
        logger.debug("Loading GeoIP database: {}", location);
        reader = new DatabaseReader.Builder(location.toFile()).build();
    }
    logger.info("Loaded GeoIP database: {}", location);
    return reader;
}
Also used : GZIPInputStream(java.util.zip.GZIPInputStream) BufferedInputStream(java.io.BufferedInputStream) GZIPInputStream(java.util.zip.GZIPInputStream) BufferedInputStream(java.io.BufferedInputStream) InputStream(java.io.InputStream) DatabaseReader(com.maxmind.geoip2.DatabaseReader)

Example 7 with DatabaseReader

use of com.maxmind.geoip2.DatabaseReader in project bender by Nextdoor.

the class GeoIpOperationTest method setup.

public GeoIpOperation setup(List<GeoProperty> geoProperties, boolean required) throws IOException {
    InputStream is = this.getClass().getResourceAsStream("my-ip-data.mmdb");
    DatabaseReader dr = new DatabaseReader.Builder(is).build();
    return new GeoIpOperation("ip_address", "geo_ip", dr, geoProperties, required);
}
Also used : InputStream(java.io.InputStream) DatabaseReader(com.maxmind.geoip2.DatabaseReader)

Example 8 with DatabaseReader

use of com.maxmind.geoip2.DatabaseReader in project Nucleus by NucleusPowered.

the class GeoIpDatabaseHandler method onRun.

private void onRun(LoadType type) {
    try {
        isLoading = true;
        // Check in case we need it.
        downloadUpdate(type == LoadType.DOWNLOAD);
        if (databaseReader != null) {
            databaseReader.close();
        }
        InputStream inputStream = new FileInputStream(countries.toFile());
        databaseReader = new DatabaseReader.Builder(inputStream).withCache(new CHMCache()).fileMode(Reader.FileMode.MEMORY).build();
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        isLoading = false;
    }
}
Also used : GZIPInputStream(java.util.zip.GZIPInputStream) FileInputStream(java.io.FileInputStream) InputStream(java.io.InputStream) CHMCache(com.maxmind.db.CHMCache) FileInputStream(java.io.FileInputStream) TimeoutException(java.util.concurrent.TimeoutException) IOException(java.io.IOException) AddressNotFoundException(com.maxmind.geoip2.exception.AddressNotFoundException)

Example 9 with DatabaseReader

use of com.maxmind.geoip2.DatabaseReader in project nutch by apache.

the class GeoIPDocumentCreator method createDocFromDomainDb.

public static NutchDocument createDocFromDomainDb(String serverIp, NutchDocument doc, DatabaseReader reader) throws UnknownHostException, IOException, GeoIp2Exception {
    DomainResponse response = reader.domain(InetAddress.getByName(serverIp));
    addIfNotNull(doc, "ip", serverIp);
    addIfNotNull(doc, "domain", response.getDomain());
    return doc;
}
Also used : DomainResponse(com.maxmind.geoip2.model.DomainResponse)

Example 10 with DatabaseReader

use of com.maxmind.geoip2.DatabaseReader in project nutch by apache.

the class GeoIPDocumentCreator method createDocFromConnectionDb.

public static NutchDocument createDocFromConnectionDb(String serverIp, NutchDocument doc, DatabaseReader reader) throws UnknownHostException, IOException, GeoIp2Exception {
    ConnectionTypeResponse response = reader.connectionType(InetAddress.getByName(serverIp));
    addIfNotNull(doc, "ip", serverIp);
    addIfNotNull(doc, "connType", response.getConnectionType().toString());
    return doc;
}
Also used : ConnectionTypeResponse(com.maxmind.geoip2.model.ConnectionTypeResponse)

Aggregations

IOException (java.io.IOException)8 DatabaseReader (com.maxmind.geoip2.DatabaseReader)6 CityResponse (com.maxmind.geoip2.model.CityResponse)5 InetAddress (java.net.InetAddress)5 GeoIp2Exception (com.maxmind.geoip2.exception.GeoIp2Exception)4 IspResponse (com.maxmind.geoip2.model.IspResponse)3 File (java.io.File)3 InputStream (java.io.InputStream)3 HashMap (java.util.HashMap)3 CHMCache (com.maxmind.db.CHMCache)2 AddressNotFoundException (com.maxmind.geoip2.exception.AddressNotFoundException)2 ConnectionTypeResponse (com.maxmind.geoip2.model.ConnectionTypeResponse)2 DomainResponse (com.maxmind.geoip2.model.DomainResponse)2 Subdivision (com.maxmind.geoip2.record.Subdivision)2 GZIPInputStream (java.util.zip.GZIPInputStream)2 FlowFile (org.apache.nifi.flowfile.FlowFile)2 DatabaseReader (org.apache.nifi.processors.maxmind.DatabaseReader)2 StopWatch (org.apache.nifi.util.StopWatch)2 JsonObject (com.google.gson.JsonObject)1 InvalidDatabaseException (com.maxmind.db.InvalidDatabaseException)1