Search in sources :

Example 6 with UnsupportedEncodingException

use of java.io.UnsupportedEncodingException in project hadoop by apache.

the class KeyFieldBasedPartitioner method getPartition.

public int getPartition(K2 key, V2 value, int numReduceTasks) {
    byte[] keyBytes;
    List<KeyDescription> allKeySpecs = keyFieldHelper.keySpecs();
    if (allKeySpecs.size() == 0) {
        return getPartition(key.toString().hashCode(), numReduceTasks);
    }
    try {
        keyBytes = key.toString().getBytes("UTF-8");
    } catch (UnsupportedEncodingException e) {
        throw new RuntimeException("The current system does not " + "support UTF-8 encoding!", e);
    }
    // return 0 if the key is empty
    if (keyBytes.length == 0) {
        return 0;
    }
    int[] lengthIndicesFirst = keyFieldHelper.getWordLengths(keyBytes, 0, keyBytes.length);
    int currentHash = 0;
    for (KeyDescription keySpec : allKeySpecs) {
        int startChar = keyFieldHelper.getStartOffset(keyBytes, 0, keyBytes.length, lengthIndicesFirst, keySpec);
        // no key found! continue
        if (startChar < 0) {
            continue;
        }
        int endChar = keyFieldHelper.getEndOffset(keyBytes, 0, keyBytes.length, lengthIndicesFirst, keySpec);
        currentHash = hashCode(keyBytes, startChar, endChar, currentHash);
    }
    return getPartition(currentHash, numReduceTasks);
}
Also used : UnsupportedEncodingException(java.io.UnsupportedEncodingException) KeyDescription(org.apache.hadoop.mapreduce.lib.partition.KeyFieldHelper.KeyDescription)

Example 7 with UnsupportedEncodingException

use of java.io.UnsupportedEncodingException in project hadoop by apache.

the class SwiftRestClient method encodeUrl.

/**
   * Encode the URL. This extends {@link URLEncoder#encode(String, String)}
   * with a replacement of + with %20.
   * @param url URL string
   * @return an encoded string
   * @throws SwiftException if the URL cannot be encoded
   */
private static String encodeUrl(String url) throws SwiftException {
    if (url.matches(".*\\s+.*")) {
        try {
            url = URLEncoder.encode(url, "UTF-8");
            url = url.replace("+", "%20");
        } catch (UnsupportedEncodingException e) {
            throw new SwiftException("failed to encode URI", e);
        }
    }
    return url;
}
Also used : UnsupportedEncodingException(java.io.UnsupportedEncodingException) SwiftException(org.apache.hadoop.fs.swift.exceptions.SwiftException)

Example 8 with UnsupportedEncodingException

use of java.io.UnsupportedEncodingException in project hbase by apache.

the class ReflectionUtils method logThreadInfo.

/**
   * Log the current thread stacks at INFO level.
   * @param log the logger that logs the stack trace
   * @param title a descriptive title for the call stacks
   * @param minInterval the minimum time from the last
   */
public static void logThreadInfo(Log log, String title, long minInterval) {
    boolean dumpStack = false;
    if (log.isInfoEnabled()) {
        synchronized (ReflectionUtils.class) {
            long now = System.currentTimeMillis();
            if (now - previousLogTime >= minInterval * 1000) {
                previousLogTime = now;
                dumpStack = true;
            }
        }
        if (dumpStack) {
            try {
                ByteArrayOutputStream buffer = new ByteArrayOutputStream();
                printThreadInfo(new PrintStream(buffer, false, "UTF-8"), title);
                log.info(buffer.toString(Charset.defaultCharset().name()));
            } catch (UnsupportedEncodingException ignored) {
                log.warn("Could not write thread info about '" + title + "' due to a string encoding issue.");
            }
        }
    }
}
Also used : PrintStream(java.io.PrintStream) UnsupportedEncodingException(java.io.UnsupportedEncodingException) ByteArrayOutputStream(java.io.ByteArrayOutputStream)

Example 9 with UnsupportedEncodingException

use of java.io.UnsupportedEncodingException in project hbase by apache.

the class Base64 method encodeBytes.

// end encodeBytes
/**
   * Encodes a byte array into Base64 notation.
   * <p>
   * Valid options:
   * <ul>
   *   <li>GZIP: gzip-compresses object before encoding it.</li>
   *   <li>DONT_BREAK_LINES: don't break lines at 76 characters. <i>Note:
   *     Technically, this makes your encoding non-compliant.</i></li>
   * </ul>
   *
   * <p>
   * Example: <code>encodeBytes( myData, Base64.GZIP )</code> or
   * <p>
   * Example:
   * <code>encodeBytes( myData, Base64.GZIP | Base64.DONT_BREAK_LINES )</code>
   *
   * @param source The data to convert
   * @param off Offset in array where conversion should begin
   * @param len Length of data to convert
   * @param options Specified options
   * @see Base64#GZIP
   * @see Base64#DONT_BREAK_LINES
   * @see Base64#URL_SAFE
   * @see Base64#ORDERED
   * @return encoded byte array
   * @since 2.0
   */
public static String encodeBytes(byte[] source, int off, int len, int options) {
    if ((options & GZIP) == GZIP) {
        // Compress?
        // GZip -> Base64 -> ByteArray
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        GZIPOutputStream gzos = null;
        try {
            gzos = new GZIPOutputStream(new Base64OutputStream(baos, ENCODE | options));
            gzos.write(source, off, len);
            gzos.close();
            gzos = null;
            return new String(baos.toByteArray(), PREFERRED_ENCODING);
        } catch (UnsupportedEncodingException uue) {
            return new String(baos.toByteArray());
        } catch (IOException e) {
            LOG.error("error encoding byte array", e);
            return null;
        } finally {
            if (gzos != null) {
                try {
                    gzos.close();
                } catch (Exception e) {
                    LOG.error("error closing GZIPOutputStream", e);
                }
            }
            try {
                baos.close();
            } catch (Exception e) {
                LOG.error("error closing ByteArrayOutputStream", e);
            }
        }
    // end finally
    }
    // end Compress
    // Don't compress. Better not to use streams at all then.
    boolean breakLines = ((options & DONT_BREAK_LINES) == 0);
    int len43 = len * 4 / 3;
    byte[] outBuff = new byte[// Main 4:3
    (len43) + // padding
    ((len % 3) > 0 ? 4 : 0) + // New lines
    (breakLines ? (len43 / MAX_LINE_LENGTH) : 0)];
    int d = 0;
    int e = 0;
    int len2 = len - 2;
    int lineLength = 0;
    for (; d < len2; d += 3, e += 4) {
        encode3to4(source, d + off, 3, outBuff, e, options);
        lineLength += 4;
        if (breakLines && lineLength == MAX_LINE_LENGTH) {
            outBuff[e + 4] = NEW_LINE;
            e++;
            lineLength = 0;
        }
    // end if: end of line
    }
    if (d < len) {
        encode3to4(source, d + off, len - d, outBuff, e, options);
        e += 4;
    }
    // Return value according to relevant encoding.
    try {
        return new String(outBuff, 0, e, PREFERRED_ENCODING);
    } catch (UnsupportedEncodingException uue) {
        return new String(outBuff, 0, e);
    }
}
Also used : GZIPOutputStream(java.util.zip.GZIPOutputStream) UnsupportedEncodingException(java.io.UnsupportedEncodingException) ByteArrayOutputStream(java.io.ByteArrayOutputStream) IOException(java.io.IOException) IOException(java.io.IOException) UnsupportedEncodingException(java.io.UnsupportedEncodingException)

Example 10 with UnsupportedEncodingException

use of java.io.UnsupportedEncodingException in project hbase by apache.

the class Base64 method decode.

/**
   * Decodes data from Base64 notation, automatically detecting gzip-compressed
   * data and decompressing it.
   *
   * @param s the string to decode
   * @param options options for decode
   * @see Base64#URL_SAFE
   * @see Base64#ORDERED
   * @return the decoded data
   * @since 1.4
   */
public static byte[] decode(String s, int options) {
    byte[] bytes;
    try {
        bytes = s.getBytes(PREFERRED_ENCODING);
    } catch (UnsupportedEncodingException uee) {
        bytes = s.getBytes();
    }
    // end catch
    // Decode
    bytes = decode(bytes, 0, bytes.length, options);
    if (bytes != null && bytes.length >= 4) {
        int head = (bytes[0] & 0xff) | ((bytes[1] << 8) & 0xff00);
        if (GZIPInputStream.GZIP_MAGIC == head) {
            GZIPInputStream gzis = null;
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            try {
                gzis = new GZIPInputStream(new ByteArrayInputStream(bytes));
                byte[] buffer = new byte[2048];
                for (int length; (length = gzis.read(buffer)) >= 0; ) {
                    baos.write(buffer, 0, length);
                }
                // end while: reading input
                // No error? Get new bytes.
                bytes = baos.toByteArray();
            } catch (IOException e) {
            // Just return originally-decoded bytes
            } finally {
                try {
                    baos.close();
                } catch (Exception e) {
                    LOG.error("error closing ByteArrayOutputStream", e);
                }
                if (gzis != null) {
                    try {
                        gzis.close();
                    } catch (Exception e) {
                        LOG.error("error closing GZIPInputStream", e);
                    }
                }
            }
        // end finally
        }
    // end if: gzipped
    }
    return bytes;
}
Also used : GZIPInputStream(java.util.zip.GZIPInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) UnsupportedEncodingException(java.io.UnsupportedEncodingException) ByteArrayOutputStream(java.io.ByteArrayOutputStream) IOException(java.io.IOException) IOException(java.io.IOException) UnsupportedEncodingException(java.io.UnsupportedEncodingException)

Aggregations

UnsupportedEncodingException (java.io.UnsupportedEncodingException)1228 IOException (java.io.IOException)338 NoSuchAlgorithmException (java.security.NoSuchAlgorithmException)137 ArrayList (java.util.ArrayList)111 File (java.io.File)109 ByteArrayOutputStream (java.io.ByteArrayOutputStream)99 MessageDigest (java.security.MessageDigest)80 ByteArrayInputStream (java.io.ByteArrayInputStream)79 InputStream (java.io.InputStream)79 InputStreamReader (java.io.InputStreamReader)73 FileNotFoundException (java.io.FileNotFoundException)71 OutputStreamWriter (java.io.OutputStreamWriter)69 BufferedReader (java.io.BufferedReader)58 URL (java.net.URL)55 FileOutputStream (java.io.FileOutputStream)52 Map (java.util.Map)52 HashMap (java.util.HashMap)48 JSONException (org.json.JSONException)44 List (java.util.List)41 PrintStream (java.io.PrintStream)39