Search in sources :

Example 36 with BufferedHttpEntity

use of org.apache.http.entity.BufferedHttpEntity in project c-geo by just-radovan.

the class cgMapImg method getDrawable.

public void getDrawable(String url, int level) {
    if (url == null || url.length() == 0) {
        return;
    }
    if (geocode == null || geocode.length() == 0) {
        return;
    }
    final String fileName = settings.getStorage() + geocode + "/map_" + level;
    HttpClient client = null;
    HttpGet getMethod = null;
    HttpResponse httpResponse = null;
    HttpEntity entity = null;
    BufferedHttpEntity bufferedEntity = null;
    boolean ok = false;
    for (int i = 0; i < 3; i++) {
        if (i > 0)
            Log.w(cgSettings.tag, "cgMapImg.getDrawable: Failed to download data, retrying. Attempt #" + (i + 1));
        try {
            client = new DefaultHttpClient();
            getMethod = new HttpGet(url);
            httpResponse = client.execute(getMethod);
            entity = httpResponse.getEntity();
            // if image is to small, don't download and save, there is no map data for this zoom level
            long contentSize = entity.getContentLength();
            if (contentSize > 0 && contentSize <= MIN_MAP_IMAGE_BYTES) {
                break;
            }
            bufferedEntity = new BufferedHttpEntity(entity);
            if (bufferedEntity != null) {
                InputStream is = (InputStream) bufferedEntity.getContent();
                FileOutputStream fos = new FileOutputStream(fileName);
                int fileSize = 0;
                try {
                    byte[] buffer = new byte[4096];
                    int bytesRead;
                    while ((bytesRead = is.read(buffer)) != -1) {
                        fos.write(buffer, 0, bytesRead);
                        fileSize += bytesRead;
                    }
                    ok = true;
                } catch (IOException e) {
                    Log.e(cgSettings.tag, "cgMapImg.getDrawable (saving to cache): " + e.toString());
                } finally {
                    is.close();
                    fos.flush();
                    fos.close();
                }
                bufferedEntity = null;
                // delete image if it has no contents
                if (ok && fileSize < MIN_MAP_IMAGE_BYTES) {
                    (new File(fileName)).delete();
                }
            }
            if (ok == true) {
                break;
            }
        } catch (Exception e) {
            Log.e(cgSettings.tag, "cgMapImg.getDrawable (downloading from web): " + e.toString());
        }
    }
}
Also used : HttpEntity(org.apache.http.HttpEntity) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) InputStream(java.io.InputStream) HttpGet(org.apache.http.client.methods.HttpGet) HttpResponse(org.apache.http.HttpResponse) IOException(java.io.IOException) DefaultHttpClient(org.apache.http.impl.client.DefaultHttpClient) IOException(java.io.IOException) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) DefaultHttpClient(org.apache.http.impl.client.DefaultHttpClient) HttpClient(org.apache.http.client.HttpClient) FileOutputStream(java.io.FileOutputStream) File(java.io.File)

Example 37 with BufferedHttpEntity

use of org.apache.http.entity.BufferedHttpEntity in project c-geo by just-radovan.

the class cgAddressImg method getDrawable.

public static BitmapDrawable getDrawable(String url) {
    Bitmap imagePre = null;
    if (url == null || url.length() == 0)
        return null;
    HttpClient client = null;
    HttpGet getMethod = null;
    HttpResponse httpResponse = null;
    HttpEntity entity = null;
    BufferedHttpEntity bufferedEntity = null;
    for (int i = 0; i < 2; i++) {
        if (i > 0)
            Log.w(cgSettings.tag, "cgAddressImg.getDrawable: Failed to download data, retrying. Attempt #" + (i + 1));
        try {
            client = new DefaultHttpClient();
            getMethod = new HttpGet(url);
            httpResponse = client.execute(getMethod);
            entity = httpResponse.getEntity();
            bufferedEntity = new BufferedHttpEntity(entity);
            Log.i(cgSettings.tag, "[" + entity.getContentLength() + "B] Downloading address map " + url);
            if (bufferedEntity != null)
                imagePre = BitmapFactory.decodeStream(bufferedEntity.getContent(), null, null);
            if (imagePre != null)
                break;
        } catch (Exception e) {
            Log.e(cgSettings.tag, "cgAddressImg.getDrawable (downloading from web): " + e.toString());
        }
    }
    if (imagePre == null) {
        Log.d(cgSettings.tag, "cgAddressImg.getDrawable: Failed to obtain image");
        return null;
    }
    final BitmapDrawable image = new BitmapDrawable(imagePre);
    image.setBounds(new Rect(0, 0, imagePre.getWidth(), imagePre.getHeight()));
    // imagePre.recycle();
    imagePre = null;
    return image;
}
Also used : Bitmap(android.graphics.Bitmap) Rect(android.graphics.Rect) HttpEntity(org.apache.http.HttpEntity) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) DefaultHttpClient(org.apache.http.impl.client.DefaultHttpClient) HttpClient(org.apache.http.client.HttpClient) HttpGet(org.apache.http.client.methods.HttpGet) HttpResponse(org.apache.http.HttpResponse) BitmapDrawable(android.graphics.drawable.BitmapDrawable) DefaultHttpClient(org.apache.http.impl.client.DefaultHttpClient)

Example 38 with BufferedHttpEntity

use of org.apache.http.entity.BufferedHttpEntity in project janusgraph by JanusGraph.

the class SolrIndex method configureSolrClientsForKerberos.

private void configureSolrClientsForKerberos() throws PermanentBackendException {
    String kerberosConfig = System.getProperty("java.security.auth.login.config");
    if (kerberosConfig == null) {
        throw new PermanentBackendException("Unable to configure kerberos for solr client. System property 'java.security.auth.login.config' is not set.");
    }
    logger.debug("Using kerberos configuration file located at '{}'.", kerberosConfig);
    try (Krb5HttpClientBuilder krbBuild = new Krb5HttpClientBuilder()) {
        SolrHttpClientBuilder kb = krbBuild.getBuilder();
        HttpClientUtil.setHttpClientBuilder(kb);
        HttpRequestInterceptor bufferedEntityInterceptor = (request, context) -> {
            if (request instanceof HttpEntityEnclosingRequest) {
                HttpEntityEnclosingRequest enclosingRequest = ((HttpEntityEnclosingRequest) request);
                HttpEntity requestEntity = enclosingRequest.getEntity();
                enclosingRequest.setEntity(new BufferedHttpEntity(requestEntity));
            }
        };
        HttpClientUtil.addRequestInterceptor(bufferedEntityInterceptor);
        HttpRequestInterceptor preemptiveAuth = new PreemptiveAuth(new KerberosScheme());
        HttpClientUtil.addRequestInterceptor(preemptiveAuth);
    }
}
Also used : PredicateCondition(org.janusgraph.graphdb.query.condition.PredicateCondition) Arrays(java.util.Arrays) DocCollection(org.apache.solr.common.cloud.DocCollection) KerberosScheme(org.apache.http.impl.auth.KerberosScheme) ClusterState(org.apache.solr.common.cloud.ClusterState) Geoshape(org.janusgraph.core.attribute.Geoshape) StringUtils(org.apache.commons.lang3.StringUtils) SolrServerException(org.apache.solr.client.solrj.SolrServerException) Cardinality(org.janusgraph.core.Cardinality) IndexEntry(org.janusgraph.diskstorage.indexing.IndexEntry) KeyInformation(org.janusgraph.diskstorage.indexing.KeyInformation) Map(java.util.Map) HttpRequestInterceptor(org.apache.http.HttpRequestInterceptor) ZkStateReader(org.apache.solr.common.cloud.ZkStateReader) And(org.janusgraph.graphdb.query.condition.And) ModifiableSolrParams(org.apache.solr.common.params.ModifiableSolrParams) Set(java.util.Set) UncheckedIOException(java.io.UncheckedIOException) IOUtils(org.apache.commons.io.IOUtils) Stream(java.util.stream.Stream) SolrQuery(org.apache.solr.client.solrj.SolrQuery) Joiner(com.google.common.base.Joiner) SolrHttpClientBuilder(org.apache.solr.client.solrj.impl.SolrHttpClientBuilder) SolrInputDocument(org.apache.solr.common.SolrInputDocument) PreInitializeConfigOptions(org.janusgraph.graphdb.configuration.PreInitializeConfigOptions) Not(org.janusgraph.graphdb.query.condition.Not) AttributeUtils(org.janusgraph.graphdb.database.serialize.AttributeUtils) SimpleDateFormat(java.text.SimpleDateFormat) Constructor(java.lang.reflect.Constructor) ArrayList(java.util.ArrayList) Cmp(org.janusgraph.core.attribute.Cmp) HttpClient(org.apache.http.client.HttpClient) CachingTokenFilter(org.apache.lucene.analysis.CachingTokenFilter) StreamSupport(java.util.stream.StreamSupport) JanusGraphElement(org.janusgraph.core.JanusGraphElement) Geo(org.janusgraph.core.attribute.Geo) JanusGraphPredicate(org.janusgraph.graphdb.query.JanusGraphPredicate) Slice(org.apache.solr.common.cloud.Slice) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) CollectionAdminResponse(org.apache.solr.client.solrj.response.CollectionAdminResponse) IOException(java.io.IOException) DefaultTransaction(org.janusgraph.diskstorage.util.DefaultTransaction) SolrClient(org.apache.solr.client.solrj.SolrClient) CommonParams(org.apache.solr.common.params.CommonParams) SolrDocument(org.apache.solr.common.SolrDocument) StringReader(java.io.StringReader) UpdateRequest(org.apache.solr.client.solrj.request.UpdateRequest) Preconditions(com.google.common.base.Preconditions) INDEX_NS(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.INDEX_NS) IndexMutation(org.janusgraph.diskstorage.indexing.IndexMutation) INDEX_MAX_RESULT_SET_SIZE(org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.INDEX_MAX_RESULT_SET_SIZE) GeoToWktConverter(org.janusgraph.diskstorage.solr.transform.GeoToWktConverter) PreemptiveAuth(org.apache.solr.client.solrj.impl.PreemptiveAuth) HttpClientUtil(org.apache.solr.client.solrj.impl.HttpClientUtil) Date(java.util.Date) Spliterators(java.util.Spliterators) LoggerFactory(org.slf4j.LoggerFactory) ConfigOption(org.janusgraph.diskstorage.configuration.ConfigOption) BaseTransaction(org.janusgraph.diskstorage.BaseTransaction) IndexProvider(org.janusgraph.diskstorage.indexing.IndexProvider) Krb5HttpClientBuilder(org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder) IndexQuery(org.janusgraph.diskstorage.indexing.IndexQuery) DateFormat(java.text.DateFormat) Mapping(org.janusgraph.core.schema.Mapping) TimeZone(java.util.TimeZone) Collection(java.util.Collection) HttpEntity(org.apache.http.HttpEntity) UUID(java.util.UUID) Instant(java.time.Instant) Collectors(java.util.stream.Collectors) Replica(org.apache.solr.common.cloud.Replica) List(java.util.List) Parameter(org.janusgraph.core.schema.Parameter) HttpSolrClient(org.apache.solr.client.solrj.impl.HttpSolrClient) Optional(java.util.Optional) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) ClientUtils(org.apache.solr.client.solrj.util.ClientUtils) LBHttpSolrClient(org.apache.solr.client.solrj.impl.LBHttpSolrClient) Spliterator(java.util.Spliterator) CollectionAdminRequest(org.apache.solr.client.solrj.request.CollectionAdminRequest) Tokenizer(org.apache.lucene.analysis.Tokenizer) ConfigNamespace(org.janusgraph.diskstorage.configuration.ConfigNamespace) Condition(org.janusgraph.graphdb.query.condition.Condition) HashMap(java.util.HashMap) Function(java.util.function.Function) TemporaryBackendException(org.janusgraph.diskstorage.TemporaryBackendException) HashSet(java.util.HashSet) IndexFeatures(org.janusgraph.diskstorage.indexing.IndexFeatures) Or(org.janusgraph.graphdb.query.condition.Or) SimpleEntry(java.util.AbstractMap.SimpleEntry) TermToBytesRefAttribute(org.apache.lucene.analysis.tokenattributes.TermToBytesRefAttribute) CloudSolrClient(org.apache.solr.client.solrj.impl.CloudSolrClient) HttpEntityEnclosingRequest(org.apache.http.HttpEntityEnclosingRequest) BackendException(org.janusgraph.diskstorage.BackendException) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) Configuration(org.janusgraph.diskstorage.configuration.Configuration) BaseTransactionConfigurable(org.janusgraph.diskstorage.BaseTransactionConfigurable) RawQuery(org.janusgraph.diskstorage.indexing.RawQuery) Order(org.janusgraph.graphdb.internal.Order) Text(org.janusgraph.core.attribute.Text) BaseTransactionConfig(org.janusgraph.diskstorage.BaseTransactionConfig) ParameterType(org.janusgraph.graphdb.types.ParameterType) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) Collections(java.util.Collections) HttpEntity(org.apache.http.HttpEntity) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) PermanentBackendException(org.janusgraph.diskstorage.PermanentBackendException) HttpRequestInterceptor(org.apache.http.HttpRequestInterceptor) HttpEntityEnclosingRequest(org.apache.http.HttpEntityEnclosingRequest) SolrHttpClientBuilder(org.apache.solr.client.solrj.impl.SolrHttpClientBuilder) KerberosScheme(org.apache.http.impl.auth.KerberosScheme) PreemptiveAuth(org.apache.solr.client.solrj.impl.PreemptiveAuth) Krb5HttpClientBuilder(org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder)

Example 39 with BufferedHttpEntity

use of org.apache.http.entity.BufferedHttpEntity in project azure-tools-for-java by Microsoft.

the class ADLSGen2FSOperation method appendData.

private Observable<Long> appendData(String filePath, File src) {
    try {
        InputStreamEntity reqEntity = new InputStreamEntity(new FileInputStream(src), -1, ContentType.APPLICATION_OCTET_STREAM);
        BufferedHttpEntity entity = new BufferedHttpEntity(reqEntity);
        long len = entity.getContentLength();
        HttpPatch req = new HttpPatch(filePath);
        http.setContentType("application/octet-stream");
        return http.executeReqAndCheckStatus(req, entity, this.appendReqParams, Collections.emptyList(), 202).map(ignore -> len);
    } catch (FileNotFoundException e) {
        throw new RuntimeException(new IllegalArgumentException("Can not find the aritifact"));
    } catch (IOException e) {
        throw new RuntimeException(new IllegalArgumentException("Can not read the aritfact"));
    }
}
Also used : BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) FileNotFoundException(java.io.FileNotFoundException) IOException(java.io.IOException) FileInputStream(java.io.FileInputStream) HttpPatch(org.apache.http.client.methods.HttpPatch) InputStreamEntity(org.apache.http.entity.InputStreamEntity)

Example 40 with BufferedHttpEntity

use of org.apache.http.entity.BufferedHttpEntity in project azure-tools-for-java by Microsoft.

the class JobUtils method deployArtifact.

public static Observable<String> deployArtifact(@NotNull SparkBatchSubmission submission, @NotNull String destinationRootPath, @NotNull String artifactPath) {
    return Observable.fromCallable(() -> {
        final File file = new File(artifactPath);
        final String webHdfsUploadPath = destinationRootPath + file.getName();
        final String redirectUri;
        List<NameValuePair> params = new WebHdfsParamsBuilder("CREATE").setOverwrite("true").build();
        URIBuilder uriBuilder = new URIBuilder(webHdfsUploadPath);
        uriBuilder.addParameters(params);
        HttpUriRequest req = RequestBuilder.put(uriBuilder.build()).build();
        final CloseableHttpClient httpclient = submission.getHttpClient();
        try (final CloseableHttpResponse response = httpclient.execute(req)) {
            // two steps to upload via webhdfs
            // 1.put request the get 307 redirect uri from response
            // 2.put redirect request with file content as setEntity
            redirectUri = response.getFirstHeader("Location").getValue();
            if (StringUtils.isBlank(redirectUri)) {
                throw new UnknownServiceException("can not get valid redirect uri using webhdfs");
            }
        } catch (final Exception ex) {
            throw new UnknownServiceException("using webhdfs encounter problem:" + ex.toString());
        }
        final InputStreamEntity reqEntity = new InputStreamEntity(new FileInputStream(file), -1, ContentType.APPLICATION_OCTET_STREAM);
        reqEntity.setChunked(true);
        final BufferedHttpEntity reqEntityBuf = new BufferedHttpEntity(reqEntity);
        // setup url with redirect url and entity ,config 100 continue to header
        req = RequestBuilder.put(redirectUri).setEntity(reqEntityBuf).setConfig(RequestConfig.custom().setExpectContinueEnabled(true).build()).build();
        // execute put request
        try (final CloseableHttpResponse putResp = httpclient.execute(req)) {
            params = new WebHdfsParamsBuilder("OPEN").build();
            uriBuilder = new URIBuilder(webHdfsUploadPath);
            uriBuilder.addParameters(params);
            // return get file uri
            return uriBuilder.build().toString();
        }
    });
}
Also used : HttpUriRequest(org.apache.http.client.methods.HttpUriRequest) CloseableHttpClient(org.apache.http.impl.client.CloseableHttpClient) UnknownServiceException(java.net.UnknownServiceException) WebHdfsParamsBuilder(com.microsoft.azure.hdinsight.sdk.storage.webhdfs.WebHdfsParamsBuilder) URISyntaxException(java.net.URISyntaxException) AuthenticationException(com.microsoft.azure.hdinsight.sdk.common.AuthenticationException) FailingHttpStatusCodeException(com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException) UnknownServiceException(java.net.UnknownServiceException) HDIException(com.microsoft.azure.hdinsight.sdk.common.HDIException) ExecutionException(java.util.concurrent.ExecutionException) AzureCmdException(com.microsoft.azuretools.azurecommons.helpers.AzureCmdException) URIBuilder(org.apache.http.client.utils.URIBuilder) InputStreamEntity(org.apache.http.entity.InputStreamEntity) BufferedHttpEntity(org.apache.http.entity.BufferedHttpEntity) CloseableHttpResponse(org.apache.http.client.methods.CloseableHttpResponse)

Aggregations

BufferedHttpEntity (org.apache.http.entity.BufferedHttpEntity)43 HttpEntity (org.apache.http.HttpEntity)31 IOException (java.io.IOException)21 HttpResponse (org.apache.http.HttpResponse)16 HttpGet (org.apache.http.client.methods.HttpGet)11 Header (org.apache.http.Header)9 InputStream (java.io.InputStream)8 HttpClient (org.apache.http.client.HttpClient)7 ByteArrayInputStream (java.io.ByteArrayInputStream)6 ByteArrayOutputStream (java.io.ByteArrayOutputStream)6 HttpException (org.apache.http.HttpException)6 DefaultHttpClient (org.apache.http.impl.client.DefaultHttpClient)6 File (java.io.File)5 FileOutputStream (java.io.FileOutputStream)5 URISyntaxException (java.net.URISyntaxException)5 HttpHost (org.apache.http.HttpHost)4 HttpRequest (org.apache.http.HttpRequest)4 StatusLine (org.apache.http.StatusLine)4 AuthenticationException (org.apache.http.auth.AuthenticationException)4 CredentialsProvider (org.apache.http.client.CredentialsProvider)4