Search in sources :

Example 11 with LookupResult

use of org.graylog2.plugin.lookup.LookupResult in project graylog2-server by Graylog2.

the class HTTPJSONPathDataAdapter method doGet.

@Override
protected LookupResult doGet(Object key) {
    String encodedKey;
    try {
        encodedKey = URLEncoder.encode(String.valueOf(key), "UTF-8").replaceAll("\\+", "%20");
    } catch (UnsupportedEncodingException ignored) {
        // UTF-8 is always supported
        encodedKey = String.valueOf(key);
    }
    final String urlString = templateEngine.transform(config.url(), ImmutableMap.of("key", encodedKey));
    if (!urlWhitelistService.isWhitelisted(urlString)) {
        LOG.error("URL <{}> is not whitelisted. Aborting lookup request.", urlString);
        publishSystemNotificationForWhitelistFailure();
        setError(UrlNotWhitelistedException.forUrl(urlString));
        return getErrorResult();
    } else {
        // we use this kind of error reporting mechanism only for whitelist errors, so we can safely clear the
        // error here
        clearError();
    }
    final HttpUrl url = HttpUrl.parse(urlString);
    if (url == null) {
        LOG.error("Couldn't parse URL <{}> - returning empty result", urlString);
        httpURLErrors.mark();
        return getErrorResult();
    }
    final Request request = new Request.Builder().get().url(url).headers(headers).build();
    final Timer.Context time = httpRequestTimer.time();
    try (final Response response = httpClient.newCall(request).execute()) {
        if (!response.isSuccessful()) {
            LOG.warn("HTTP request for key <{}> failed: {}", key, response);
            httpRequestErrors.mark();
            return getErrorResult();
        }
        final LookupResult result = parseBody(singleJsonPath, multiJsonPath, response.body().byteStream());
        if (result == null) {
            return getErrorResult();
        }
        return result;
    } catch (IOException e) {
        LOG.error("HTTP request error for key <{}>", key, e);
        httpRequestErrors.mark();
        return getErrorResult();
    } finally {
        time.stop();
    }
}
Also used : Response(okhttp3.Response) Timer(com.codahale.metrics.Timer) Request(okhttp3.Request) LookupResult(org.graylog2.plugin.lookup.LookupResult) UnsupportedEncodingException(java.io.UnsupportedEncodingException) IOException(java.io.IOException) HttpUrl(okhttp3.HttpUrl)

Example 12 with LookupResult

use of org.graylog2.plugin.lookup.LookupResult in project graylog2-server by Graylog2.

the class HTTPJSONPathDataAdapter method parseBody.

@VisibleForTesting
static LookupResult parseBody(JsonPath singleJsonPath, @Nullable JsonPath multiJsonPath, InputStream body) {
    try {
        final DocumentContext documentContext = JsonPath.parse(body);
        LookupResult.Builder builder = LookupResult.builder().cacheTTL(Long.MAX_VALUE);
        if (multiJsonPath != null) {
            try {
                final Object multiValue = documentContext.read(multiJsonPath);
                if (multiValue instanceof Map) {
                    // noinspection unchecked
                    builder = builder.multiValue((Map<Object, Object>) multiValue);
                } else if (multiValue instanceof List) {
                    // noinspection unchecked
                    final List<String> stringList = ((List<Object>) multiValue).stream().map(Object::toString).collect(Collectors.toList());
                    builder = builder.stringListValue(stringList);
                    // for backwards compatibility
                    builder = builder.multiSingleton(multiValue);
                } else {
                    builder = builder.multiSingleton(multiValue);
                }
            } catch (PathNotFoundException e) {
                LOG.warn("Couldn't read multi JSONPath from response - skipping multi value ({})", e.getMessage());
            }
        }
        try {
            final Object singleValue = documentContext.read(singleJsonPath);
            if (singleValue instanceof CharSequence) {
                return builder.single((CharSequence) singleValue).build();
            } else if (singleValue instanceof Number) {
                return builder.single((Number) singleValue).build();
            } else if (singleValue instanceof Boolean) {
                return builder.single((Boolean) singleValue).build();
            } else {
                throw new IllegalArgumentException("Single value data type cannot be: " + singleValue.getClass().getCanonicalName());
            }
        } catch (PathNotFoundException e) {
            LOG.warn("Couldn't read single JSONPath from response - returning empty result ({})", e.getMessage());
            return null;
        }
    } catch (InvalidJsonException e) {
        LOG.error("Couldn't parse JSON response", e);
        return null;
    } catch (ClassCastException e) {
        LOG.error("Couldn't assign value type", e);
        return null;
    } catch (Exception e) {
        LOG.error("Unexpected error parsing JSON response", e);
        return null;
    }
}
Also used : InvalidJsonException(com.jayway.jsonpath.InvalidJsonException) PathNotFoundException(com.jayway.jsonpath.PathNotFoundException) UnsupportedEncodingException(java.io.UnsupportedEncodingException) UrlNotWhitelistedException(org.graylog2.system.urlwhitelist.UrlNotWhitelistedException) InvalidPathException(com.jayway.jsonpath.InvalidPathException) IOException(java.io.IOException) LookupResult(org.graylog2.plugin.lookup.LookupResult) InvalidJsonException(com.jayway.jsonpath.InvalidJsonException) List(java.util.List) PathNotFoundException(com.jayway.jsonpath.PathNotFoundException) DocumentContext(com.jayway.jsonpath.DocumentContext) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 13 with LookupResult

use of org.graylog2.plugin.lookup.LookupResult in project graylog2-server by Graylog2.

the class LookupTable method setStringList.

public LookupResult setStringList(@Nonnull Object key, @Nonnull List<String> value) {
    final LookupResult result = dataAdapter().setStringList(key, value);
    cache().purge(LookupCacheKey.create(dataAdapter(), key));
    return result;
}
Also used : LookupResult(org.graylog2.plugin.lookup.LookupResult)

Example 14 with LookupResult

use of org.graylog2.plugin.lookup.LookupResult in project graylog2-server by Graylog2.

the class LookupTable method setValue.

public LookupResult setValue(@Nonnull Object key, @Nonnull Object value) {
    final LookupResult result = dataAdapter().setValue(key, value);
    cache().purge(LookupCacheKey.create(dataAdapter(), key));
    return result;
}
Also used : LookupResult(org.graylog2.plugin.lookup.LookupResult)

Example 15 with LookupResult

use of org.graylog2.plugin.lookup.LookupResult in project graylog2-server by Graylog2.

the class LookupTable method removeStringList.

public LookupResult removeStringList(@Nonnull Object key, @Nonnull List<String> value) {
    final LookupResult result = dataAdapter().removeStringList(key, value);
    cache().purge(LookupCacheKey.create(dataAdapter(), key));
    return result;
}
Also used : LookupResult(org.graylog2.plugin.lookup.LookupResult)

Aggregations

LookupResult (org.graylog2.plugin.lookup.LookupResult)17 LookupTableService (org.graylog2.lookup.LookupTableService)5 JsonPath (com.jayway.jsonpath.JsonPath)3 Test (org.junit.Test)3 Timer (com.codahale.metrics.Timer)2 IOException (java.io.IOException)2 UnsupportedEncodingException (java.io.UnsupportedEncodingException)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 DocumentContext (com.jayway.jsonpath.DocumentContext)1 InvalidJsonException (com.jayway.jsonpath.InvalidJsonException)1 InvalidPathException (com.jayway.jsonpath.InvalidPathException)1 PathNotFoundException (com.jayway.jsonpath.PathNotFoundException)1 UnknownHostException (java.net.UnknownHostException)1 Collection (java.util.Collection)1 LinkedHashMap (java.util.LinkedHashMap)1 List (java.util.List)1 Map (java.util.Map)1 Nullable (javax.annotation.Nullable)1 HttpUrl (okhttp3.HttpUrl)1