Search in sources :

Example 1 with TapException

use of cascading.tap.TapException in project SpyGlass by ParallelAI.

the class HBaseTap method obtainToken.

private void obtainToken(JobConf conf) {
    if (User.isHBaseSecurityEnabled(conf)) {
        String user = conf.getUser();
        LOG.info("obtaining HBase token for: {}", user);
        try {
            UserGroupInformation currentUser = UserGroupInformation.getCurrentUser();
            user = currentUser.getUserName();
            Credentials credentials = conf.getCredentials();
            for (Token t : currentUser.getTokens()) {
                LOG.debug("Token {} is available", t);
                if ("HBASE_AUTH_TOKEN".equalsIgnoreCase(t.getKind().toString()))
                    credentials.addToken(t.getKind(), t);
            }
        } catch (IOException e) {
            throw new TapException("Unable to obtain HBase auth token for " + user, e);
        }
    }
}
Also used : TapException(cascading.tap.TapException) Token(org.apache.hadoop.security.token.Token) IOException(java.io.IOException) Credentials(org.apache.hadoop.security.Credentials) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation)

Example 2 with TapException

use of cascading.tap.TapException in project SpyGlass by ParallelAI.

the class HBaseTapCollector method close.

@Override
public void close() {
    try {
        LOG.info("closing tap collector for: {}", tap);
        writer.close(reporter);
    } catch (IOException exception) {
        LOG.warn("exception closing: {}", exception);
        throw new TapException("exception closing HBaseTapCollector", exception);
    } finally {
        super.close();
    }
}
Also used : TapException(cascading.tap.TapException) IOException(java.io.IOException)

Example 3 with TapException

use of cascading.tap.TapException in project SpyGlass by ParallelAI.

the class JDBCTap method createConnection.

private Connection createConnection() {
    try {
        LOG.info("creating connection: {}", connectionUrl);
        Class.forName(driverClassName);
        Connection connection = null;
        if (username == null)
            connection = DriverManager.getConnection(connectionUrl);
        else
            connection = DriverManager.getConnection(connectionUrl, username, password);
        connection.setAutoCommit(false);
        return connection;
    } catch (ClassNotFoundException exception) {
        throw new TapException("unable to load driver class: " + driverClassName, exception);
    } catch (SQLException exception) {
        throw new TapException("unable to open connection: " + connectionUrl, exception);
    }
}
Also used : TapException(cascading.tap.TapException)

Example 4 with TapException

use of cascading.tap.TapException in project parquet-mr by apache.

the class ParquetTupleScheme method readSchema.

private MessageType readSchema(FlowProcess<? extends JobConf> flowProcess, Tap tap) {
    try {
        Hfs hfs;
        if (tap instanceof CompositeTap)
            hfs = (Hfs) ((CompositeTap) tap).getChildTaps().next();
        else
            hfs = (Hfs) tap;
        List<Footer> footers = getFooters(flowProcess, hfs);
        if (footers.isEmpty()) {
            throw new TapException("Could not read Parquet metadata at " + hfs.getPath());
        } else {
            return footers.get(0).getParquetMetadata().getFileMetaData().getSchema();
        }
    } catch (IOException e) {
        throw new TapException(e);
    }
}
Also used : Hfs(cascading.tap.hadoop.Hfs) CompositeTap(cascading.tap.CompositeTap) Footer(org.apache.parquet.hadoop.Footer) TapException(cascading.tap.TapException) IOException(java.io.IOException)

Example 5 with TapException

use of cascading.tap.TapException in project parquet-mr by apache.

the class ParquetTupleScheme method readSchema.

private MessageType readSchema(FlowProcess<JobConf> flowProcess, Tap tap) {
    try {
        Hfs hfs;
        if (tap instanceof CompositeTap)
            hfs = (Hfs) ((CompositeTap) tap).getChildTaps().next();
        else
            hfs = (Hfs) tap;
        List<Footer> footers = getFooters(flowProcess, hfs);
        if (footers.isEmpty()) {
            throw new TapException("Could not read Parquet metadata at " + hfs.getPath());
        } else {
            return footers.get(0).getParquetMetadata().getFileMetaData().getSchema();
        }
    } catch (IOException e) {
        throw new TapException(e);
    }
}
Also used : Hfs(cascading.tap.hadoop.Hfs) CompositeTap(cascading.tap.CompositeTap) Footer(org.apache.parquet.hadoop.Footer) TapException(cascading.tap.TapException) IOException(java.io.IOException)

Aggregations

TapException (cascading.tap.TapException)10 IOException (java.io.IOException)5 CompositeTap (cascading.tap.CompositeTap)2 Hfs (cascading.tap.hadoop.Hfs)2 Footer (org.apache.parquet.hadoop.Footer)2 Credentials (org.apache.hadoop.security.Credentials)1 UserGroupInformation (org.apache.hadoop.security.UserGroupInformation)1 Token (org.apache.hadoop.security.token.Token)1