Search in sources :

Example 1 with UnauthorizedException

use of datawave.webservice.common.exception.UnauthorizedException in project datawave by NationalSecurityAgency.

the class CachedResultsBean method status.

/**
 * Returns status of the requested cached result
 *
 * @param queryId
 * @return List of attribute names that can be used in subsequent queries
 *
 * @return {@code datawave.webservice.result.GenericResponse<String>}
 * @RequestHeader X-ProxiedEntitiesChain use when proxying request for user by specifying a chain of DNs of the identities to proxy
 * @RequestHeader X-ProxiedIssuersChain required when using X-ProxiedEntitiesChain, specify one issuer DN per subject DN listed in X-ProxiedEntitiesChain
 * @ResponseHeader X-OperationTimeInMS time spent on the server performing the operation, does not account for network or result serialization
 *
 * @HTTP 200 success
 * @HTTP 404 not found
 * @HTTP 412 not yet loaded
 * @HTTP 500 internal server error
 */
@GET
@Produces({ "application/xml", "text/xml", "application/json", "text/yaml", "text/x-yaml", "application/x-yaml", "application/x-protobuf" })
@javax.ws.rs.Path("/{queryId}/status")
@GZIP
@Interceptors({ RequiredInterceptor.class, ResponseInterceptor.class })
@Timed(name = "dw.cachedr.status", absolute = true)
public GenericResponse<String> status(@PathParam("queryId") @Required("queryId") String queryId) {
    GenericResponse<String> response = new GenericResponse<>();
    // Find out who/what called this method
    Principal p = ctx.getCallerPrincipal();
    String owner = getOwnerFromPrincipal(p);
    CachedRunningQuery crq;
    try {
        crq = retrieve(queryId, owner);
    } catch (IOException e1) {
        PreConditionFailedQueryException e = new PreConditionFailedQueryException(DatawaveErrorCode.CACHED_RESULTS_IMPORT_ERROR, e1);
        response.addException(e);
        response.setResult("CachedResult not found");
        throw new PreConditionFailedException(e, response);
    }
    if (null == crq) {
        NotFoundQueryException e = new NotFoundQueryException(DatawaveErrorCode.CACHED_RESULT_NOT_FOUND);
        response.addException(e);
        response.setResult("CachedResult not found");
        throw new NotFoundException(e, response);
    }
    if (!crq.getUser().equals(owner)) {
        UnauthorizedQueryException e = new UnauthorizedQueryException(DatawaveErrorCode.QUERY_OWNER_MISMATCH, MessageFormat.format("{0} != {1}", crq.getUser(), owner));
        response.addException(e);
        response.setResult("Current user does not match user that defined query.");
        throw new UnauthorizedException(e, response);
    }
    CachedRunningQuery.Status status = crq.getStatus();
    if (status == null) {
        response.setResult(CachedRunningQuery.Status.NONE.toString());
    } else {
        response.setResult(status.toString());
    }
    if (crq.getStatusMessage() != null && crq.getStatusMessage().isEmpty() == false) {
        response.addMessage(crq.getStatusMessage());
    }
    return response;
}
Also used : GenericResponse(datawave.webservice.result.GenericResponse) PreConditionFailedQueryException(datawave.webservice.query.exception.PreConditionFailedQueryException) NotFoundException(datawave.webservice.common.exception.NotFoundException) IOException(java.io.IOException) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) PreConditionFailedException(datawave.webservice.common.exception.PreConditionFailedException) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) Principal(java.security.Principal) DatawavePrincipal(datawave.security.authorization.DatawavePrincipal) Interceptors(javax.interceptor.Interceptors) Produces(javax.ws.rs.Produces) Timed(com.codahale.metrics.annotation.Timed) GET(javax.ws.rs.GET) GZIP(org.jboss.resteasy.annotations.GZIP)

Example 2 with UnauthorizedException

use of datawave.webservice.common.exception.UnauthorizedException in project datawave by NationalSecurityAgency.

the class CachedResultsBean method cancelLoad.

/**
 * Cancel the load process.
 *
 * @param originalQueryId
 *
 * @return datawave.webservice.result.VoidResponse
 * @RequestHeader X-ProxiedEntitiesChain use when proxying request for user by specifying a chain of DNs of the identities to proxy
 * @RequestHeader X-ProxiedIssuersChain required when using X-ProxiedEntitiesChain, specify one issuer DN per subject DN listed in X-ProxiedEntitiesChain
 * @RequestHeader query-session-id session id value used for load balancing purposes. query-session-id can be placed in the request in a Cookie header or as
 *                a query parameter
 * @ResponseHeader X-OperationTimeInMS time spent on the server performing the operation, does not account for network or result serialization
 *
 * @HTTP 200 success
 * @HTTP 401 caller is not authorized to cancel the query
 */
@PUT
@Produces({ "application/xml", "text/xml", "application/json", "text/yaml", "text/x-yaml", "application/x-yaml", "application/x-protobuf", "application/x-protostuff" })
@javax.ws.rs.Path("/{queryId}/cancel")
@GZIP
@ClearQuerySessionId
@Interceptors({ RequiredInterceptor.class, ResponseInterceptor.class })
@Timed(name = "dw.cachedr.cancel", absolute = true)
public VoidResponse cancelLoad(@PathParam("queryId") @Required("queryId") String originalQueryId) {
    // Find out who/what called this method
    Principal p = ctx.getCallerPrincipal();
    String owner = getOwnerFromPrincipal(p);
    VoidResponse response = new VoidResponse();
    try {
        // check if query is even loading
        RunningQuery query = CachedResultsBean.loadingQueryMap.get(originalQueryId);
        if (query == null) {
            NotFoundQueryException e = new NotFoundQueryException(DatawaveErrorCode.NO_QUERY_OBJECT_MATCH);
            throw new NotFoundException(e, response);
        } else {
            if (query.getSettings().getOwner().equals(owner)) {
                accumuloConnectionRequestBean.cancelConnectionRequest(originalQueryId);
                query.cancel();
                response.addMessage("CachedResults load canceled.");
            } else {
                UnauthorizedQueryException e = new UnauthorizedQueryException(DatawaveErrorCode.QUERY_OWNER_MISMATCH, MessageFormat.format("{0} != {1}", query.getSettings().getOwner(), owner));
                throw new UnauthorizedException(e, response);
            }
        }
        return response;
    } catch (DatawaveWebApplicationException e) {
        throw e;
    } catch (Exception e) {
        QueryException qe = new QueryException(DatawaveErrorCode.CANCELLATION_ERROR, e, MessageFormat.format("query_id: {0}", originalQueryId));
        log.error(qe);
        response.addException(qe.getBottomQueryException());
        int statusCode = qe.getBottomQueryException().getStatusCode();
        throw new DatawaveWebApplicationException(qe, response, statusCode);
    }
}
Also used : RunningQuery(datawave.webservice.query.runner.RunningQuery) PreConditionFailedQueryException(datawave.webservice.query.exception.PreConditionFailedQueryException) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) NoResultsQueryException(datawave.webservice.query.exception.NoResultsQueryException) QueryCanceledQueryException(datawave.webservice.query.exception.QueryCanceledQueryException) QueryException(datawave.webservice.query.exception.QueryException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) VoidResponse(datawave.webservice.result.VoidResponse) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) NotFoundException(datawave.webservice.common.exception.NotFoundException) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) Principal(java.security.Principal) DatawavePrincipal(datawave.security.authorization.DatawavePrincipal) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) PreConditionFailedQueryException(datawave.webservice.query.exception.PreConditionFailedQueryException) EJBException(javax.ejb.EJBException) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) NoResultsQueryException(datawave.webservice.query.exception.NoResultsQueryException) BatchUpdateException(java.sql.BatchUpdateException) SQLException(java.sql.SQLException) IOException(java.io.IOException) QueryCanceledQueryException(datawave.webservice.query.exception.QueryCanceledQueryException) QueryException(datawave.webservice.query.exception.QueryException) QueryCanceledException(datawave.webservice.common.exception.QueryCanceledException) PreConditionFailedException(datawave.webservice.common.exception.PreConditionFailedException) NotFoundException(datawave.webservice.common.exception.NotFoundException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) SQLSyntaxErrorException(java.sql.SQLSyntaxErrorException) NoResultsException(datawave.webservice.common.exception.NoResultsException) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) MalformedURLException(java.net.MalformedURLException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) Interceptors(javax.interceptor.Interceptors) Produces(javax.ws.rs.Produces) Timed(com.codahale.metrics.annotation.Timed) ClearQuerySessionId(datawave.annotation.ClearQuerySessionId) GZIP(org.jboss.resteasy.annotations.GZIP) PUT(javax.ws.rs.PUT)

Example 3 with UnauthorizedException

use of datawave.webservice.common.exception.UnauthorizedException in project datawave by NationalSecurityAgency.

the class QueryLogicFactoryImpl method getQueryLogic.

@Override
public QueryLogic<?> getQueryLogic(String name, Principal principal) throws IllegalArgumentException, CloneNotSupportedException {
    QueryLogic<?> logic;
    try {
        logic = (QueryLogic<?>) applicationContext.getBean(name);
        logic.setPrincipal(principal);
    } catch (ClassCastException | NoSuchBeanDefinitionException cce) {
        throw new IllegalArgumentException("Logic name '" + name + "' does not exist in the configuration");
    }
    if (!logic.canRunQuery(principal)) {
        throw new UnauthorizedException(new IllegalAccessException("User does not have required role(s): " + logic.getRoleManager().getRequiredRoles()), new VoidResponse());
    }
    logic.setLogicName(name);
    if (logic.getMaxPageSize() == 0) {
        logic.setMaxPageSize(queryLogicFactoryConfiguration.getMaxPageSize());
    }
    if (logic.getPageByteTrigger() == 0) {
        logic.setPageByteTrigger(queryLogicFactoryConfiguration.getPageByteTrigger());
    }
    return logic;
}
Also used : VoidResponse(datawave.webservice.result.VoidResponse) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) NoSuchBeanDefinitionException(org.springframework.beans.factory.NoSuchBeanDefinitionException)

Example 4 with UnauthorizedException

use of datawave.webservice.common.exception.UnauthorizedException in project datawave by NationalSecurityAgency.

the class MapReduceBean method ooziesubmit.

/**
 * Execute a Oozie workflow with the given workFlow name and runtime parameters
 *
 * @param queryParameters
 * @return
 */
@POST
@Produces({ "application/xml", "text/xml", "application/json", "text/yaml", "text/x-yaml", "application/x-yaml", "application/x-protobuf", "application/x-protostuff" })
@javax.ws.rs.Path("/ooziesubmit")
@GZIP
public GenericResponse<String> ooziesubmit(MultivaluedMap<String, String> queryParameters) {
    GenericResponse<String> response = new GenericResponse<>();
    String workFlow = queryParameters.getFirst(OozieJobConstants.WORKFLOW_PARAM);
    if (StringUtils.isBlank(workFlow)) {
        throw new BadRequestException(new IllegalArgumentException(OozieJobConstants.WORKFLOW_PARAM + " parameter missing"), response);
    }
    String parameters = queryParameters.getFirst(OozieJobConstants.PARAMETERS);
    // Find out who/what called this method
    Principal p = ctx.getCallerPrincipal();
    String sid = null;
    String userDn = p.getName();
    DatawavePrincipal datawavePrincipal = null;
    if (p instanceof DatawavePrincipal) {
        datawavePrincipal = (DatawavePrincipal) p;
        sid = datawavePrincipal.getShortName();
    } else {
        QueryException qe = new QueryException(DatawaveErrorCode.UNEXPECTED_PRINCIPAL_ERROR, MessageFormat.format("Class: {0}", p.getClass().getName()));
        response.addException(qe);
        throw new DatawaveWebApplicationException(qe, response);
    }
    OozieJobConfiguration job;
    try {
        MapReduceJobConfiguration mrConfig = this.mapReduceConfiguration.getConfiguration(workFlow);
        if (mrConfig instanceof OozieJobConfiguration) {
            job = (OozieJobConfiguration) mrConfig;
        } else {
            throw new IllegalArgumentException(workFlow + " not an Oozie job configuration");
        }
    } catch (IllegalArgumentException e) {
        BadRequestQueryException qe = new BadRequestQueryException(DatawaveErrorCode.JOB_CONFIGURATION_ERROR, e);
        response.addException(qe);
        throw new BadRequestException(qe, response);
    }
    if (job instanceof NeedCallerDetails) {
        ((NeedCallerDetails) job).setUserSid(sid);
        ((NeedCallerDetails) job).setPrincipal(p);
    }
    // Ensure that the user has the required roles and has passed the required auths
    if (null != job.getRequiredRoles() || null != job.getRequiredAuths()) {
        try {
            canRunJob(datawavePrincipal, queryParameters, job.getRequiredRoles(), job.getRequiredAuths());
        } catch (UnauthorizedQueryException qe) {
            // user does not have all of the required roles or did not pass the required auths
            response.addException(qe);
            throw new UnauthorizedException(qe, response);
        }
    }
    String id = sid + "_" + UUID.randomUUID();
    OozieClient oozieClient = null;
    Properties oozieConf = null;
    try {
        oozieClient = new OozieClient((String) job.getJobConfigurationProperties().get(OozieJobConstants.OOZIE_CLIENT_PROP));
        oozieConf = oozieClient.createConfiguration();
        job.initializeOozieConfiguration(id, oozieConf, queryParameters);
        job.validateWorkflowParameter(oozieConf, mapReduceConfiguration);
    } catch (QueryException qe) {
        log.error(qe.getMessage(), qe);
        response.addException(qe);
        throw new DatawaveWebApplicationException(qe, response);
    } catch (Exception e) {
        log.error(e.getMessage(), e);
        response.addException(new QueryException(e.getMessage(), e));
        throw new DatawaveWebApplicationException(e, response);
    } finally {
        // audit query here
        Auditor.AuditType auditType = job.getAuditType();
        log.trace("Audit type is: " + auditType.name());
        if (!auditType.equals(Auditor.AuditType.NONE)) {
            try {
                marking.validate(queryParameters);
                PrivateAuditConstants.stripPrivateParameters(queryParameters);
                queryParameters.putSingle(PrivateAuditConstants.USER_DN, userDn);
                queryParameters.putSingle(PrivateAuditConstants.COLUMN_VISIBILITY, marking.toColumnVisibilityString());
                queryParameters.putSingle(PrivateAuditConstants.AUDIT_TYPE, auditType.name());
                List<String> selectors = job.getSelectors(queryParameters, oozieConf);
                if (selectors != null && !selectors.isEmpty()) {
                    queryParameters.put(PrivateAuditConstants.SELECTORS, selectors);
                }
                // if the user didn't set an audit id, use the query id
                if (!queryParameters.containsKey(AuditParameters.AUDIT_ID)) {
                    queryParameters.putSingle(AuditParameters.AUDIT_ID, id);
                }
                auditor.audit(queryParameters);
            } catch (IllegalArgumentException e) {
                log.error("Error validating audit parameters", e);
                BadRequestQueryException qe = new BadRequestQueryException(DatawaveErrorCode.MISSING_REQUIRED_PARAMETER, e);
                response.addException(qe);
                throw new BadRequestException(qe, response);
            } catch (Exception e) {
                log.error("Error auditing query", e);
                response.addMessage("Error auditing query - " + e.getMessage());
                throw new BadRequestException(e, response);
            }
        }
    }
    // Submit the Oozie workflow.
    try {
        String jobID = null;
        try {
            jobID = oozieClient.run(oozieConf);
        } catch (Exception e) {
            throw new QueryException(DatawaveErrorCode.OOZIE_JOB_START_ERROR, e);
        }
        try {
            String jobResultstDir = oozieConf.getProperty(OozieJobConstants.OUT_DIR_PROP) + "/" + id;
            response.setResult(id);
            Path baseDir = new Path(this.mapReduceConfiguration.getMapReduceBaseDirectory());
            // Create a directory path for this job
            Path jobDir = new Path(baseDir, id);
            mapReduceState.create(id, job.getHdfsUri(), job.getJobTracker(), jobDir.toString(), jobID, jobResultstDir, parameters, workFlow);
        } catch (Exception e) {
            QueryException qe = new QueryException(DatawaveErrorCode.MAPREDUCE_STATE_PERSISTENCE_ERROR, e);
            response.addException(qe.getBottomQueryException());
            try {
                oozieClient.kill(jobID);
                // if we successfully kill the job, throw the original exception
                throw qe;
            } catch (Exception e2) {
                // throw the exception from killing the job
                throw new QueryException(DatawaveErrorCode.MAPREDUCE_JOB_KILL_ERROR, e2);
            }
        }
    } catch (QueryException qe) {
        log.error(qe.getMessage(), qe);
        response.addException(qe);
        throw new DatawaveWebApplicationException(qe, response);
    } catch (Exception e) {
        log.error(e.getMessage(), e);
        QueryException qe = new QueryException(DatawaveErrorCode.UNKNOWN_SERVER_ERROR, e.getMessage());
        response.addException(qe);
        throw new DatawaveWebApplicationException(qe, response);
    }
    return response;
}
Also used : Path(org.apache.hadoop.fs.Path) GenericResponse(datawave.webservice.result.GenericResponse) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) NeedCallerDetails(datawave.webservice.mr.configuration.NeedCallerDetails) Properties(java.util.Properties) DatawavePrincipal(datawave.security.authorization.DatawavePrincipal) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) WebApplicationException(javax.ws.rs.WebApplicationException) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) IOException(java.io.IOException) QueryException(datawave.webservice.query.exception.QueryException) BadRequestException(datawave.webservice.common.exception.BadRequestException) NotFoundException(datawave.webservice.common.exception.NotFoundException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) Auditor(datawave.webservice.common.audit.Auditor) OozieClient(org.apache.oozie.client.OozieClient) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) QueryException(datawave.webservice.query.exception.QueryException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) MapReduceJobConfiguration(datawave.webservice.mr.configuration.MapReduceJobConfiguration) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) BadRequestException(datawave.webservice.common.exception.BadRequestException) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) OozieJobConfiguration(datawave.webservice.mr.configuration.OozieJobConfiguration) Principal(java.security.Principal) ServerPrincipal(datawave.security.system.ServerPrincipal) DatawavePrincipal(datawave.security.authorization.DatawavePrincipal) POST(javax.ws.rs.POST) Produces(javax.ws.rs.Produces) GZIP(org.jboss.resteasy.annotations.GZIP)

Example 5 with UnauthorizedException

use of datawave.webservice.common.exception.UnauthorizedException in project datawave by NationalSecurityAgency.

the class MapReduceBean method submit.

/**
 * Execute a MapReduce job with the given name and runtime parameters
 *
 * @param jobName
 *            Name of the map reduce job configuration
 * @param parameters
 *            A semi-colon separated list name:value pairs. These are the required and optional parameters listed in the MapReduceConfiguration objects
 *            returned in the call to list()
 * @return {@code datawave.webservice.result.GenericResponse<String>} job id
 * @RequestHeader X-ProxiedEntitiesChain use when proxying request for user by specifying a chain of DNs of the identities to proxy
 * @RequestHeader X-ProxiedIssuersChain required when using X-ProxiedEntitiesChain, specify one issuer DN per subject DN listed in X-ProxiedEntitiesChain
 * @ResponseHeader X-OperationTimeInMS time spent on the server performing the operation, does not account for network or result serialization
 * @HTTP 200 success
 * @HTTP 204 if no data was found
 * @HTTP 400 if jobName is invalid
 * @HTTP 401 if user does not have correct roles
 * @HTTP 500 error starting the job
 */
@POST
@Produces({ "application/xml", "text/xml", "application/json", "text/yaml", "text/x-yaml", "application/x-yaml", "application/x-protobuf", "application/x-protostuff" })
@javax.ws.rs.Path("/submit")
@GZIP
public GenericResponse<String> submit(@FormParam("jobName") String jobName, @FormParam("parameters") String parameters) {
    GenericResponse<String> response = new GenericResponse<>();
    // Find out who/what called this method
    Principal p = ctx.getCallerPrincipal();
    String sid;
    Set<Collection<String>> cbAuths = new HashSet<>();
    DatawavePrincipal datawavePrincipal = null;
    if (p instanceof DatawavePrincipal) {
        datawavePrincipal = (DatawavePrincipal) p;
        sid = datawavePrincipal.getShortName();
        cbAuths.addAll(datawavePrincipal.getAuthorizations());
    } else {
        QueryException qe = new QueryException(DatawaveErrorCode.UNEXPECTED_PRINCIPAL_ERROR, MessageFormat.format("Class: {0}", p.getClass().getName()));
        response.addException(qe);
        throw new DatawaveWebApplicationException(qe, response);
    }
    // Get the MapReduceJobConfiguration from the configuration
    MapReduceJobConfiguration job;
    try {
        job = this.mapReduceConfiguration.getConfiguration(jobName);
    } catch (IllegalArgumentException e) {
        BadRequestQueryException qe = new BadRequestQueryException(DatawaveErrorCode.JOB_CONFIGURATION_ERROR, e);
        response.addException(qe);
        throw new BadRequestException(qe, response);
    }
    // Ensure that the user has the required roles and has passed the required auths
    if (null != job.getRequiredRoles() || null != job.getRequiredAuths()) {
        try {
            canRunJob(datawavePrincipal, new MultivaluedMapImpl<>(), job.getRequiredRoles(), job.getRequiredAuths());
        } catch (UnauthorizedQueryException qe) {
            // user does not have all of the required roles or did not pass the required auths
            response.addException(qe);
            throw new UnauthorizedException(qe, response);
        }
    }
    // Parse the parameters
    Map<String, String> runtimeParameters = new HashMap<>();
    if (null != parameters) {
        String[] param = parameters.split(PARAMETER_SEPARATOR);
        for (String yyy : param) {
            String[] parts = yyy.split(PARAMETER_NAME_VALUE_SEPARATOR);
            if (parts.length == 2) {
                runtimeParameters.put(parts[0], parts[1]);
            }
        }
    }
    // Check to see if the job configuration class implements specific interfaces.
    if (job instanceof NeedCallerDetails) {
        ((NeedCallerDetails) job).setUserSid(sid);
        ((NeedCallerDetails) job).setPrincipal(p);
    }
    if (job instanceof NeedAccumuloConnectionFactory) {
        ((NeedAccumuloConnectionFactory) job).setAccumuloConnectionFactory(this.connectionFactory);
    }
    if (job instanceof NeedAccumuloDetails) {
        ((NeedAccumuloDetails) job).setUsername(this.connectionPoolsConfiguration.getPools().get(this.connectionPoolsConfiguration.getDefaultPool()).getUsername());
        ((NeedAccumuloDetails) job).setPassword(this.connectionPoolsConfiguration.getPools().get(this.connectionPoolsConfiguration.getDefaultPool()).getPassword());
        ((NeedAccumuloDetails) job).setInstanceName(this.connectionPoolsConfiguration.getPools().get(this.connectionPoolsConfiguration.getDefaultPool()).getInstance());
        ((NeedAccumuloDetails) job).setZookeepers(this.connectionPoolsConfiguration.getPools().get(this.connectionPoolsConfiguration.getDefaultPool()).getZookeepers());
    }
    if (job instanceof NeedQueryLogicFactory) {
        ((NeedQueryLogicFactory) job).setQueryLogicFactory(this.queryLogicFactory);
    }
    if (job instanceof NeedQueryPersister) {
        ((NeedQueryPersister) job).setPersister(this.queryPersister);
    }
    if (job instanceof NeedQueryCache) {
        ((NeedQueryCache) job).setQueryCache(cache);
    }
    if (job instanceof NeedSecurityDomain) {
        ((NeedSecurityDomain) job).setSecurityDomain(this.jsseSecurityDomain);
    }
    // If this job is being restarted, then the jobId will be the same. The restart method
    // puts the id into the runtime parameters
    String id = runtimeParameters.get(JOB_ID);
    if (null == id)
        id = UUID.randomUUID().toString();
    org.apache.hadoop.conf.Configuration conf = new org.apache.hadoop.conf.Configuration();
    StringBuilder name = new StringBuilder().append(jobName).append("_sid_").append(sid).append("_id_").append(id);
    Job j;
    try {
        j = createJob(conf, name);
        job.initializeConfiguration(id, j, runtimeParameters, serverPrincipal);
    } catch (WebApplicationException waEx) {
        throw waEx;
    } catch (Exception e) {
        QueryException qe = new QueryException(DatawaveErrorCode.LOGIC_CONFIGURATION_ERROR, e);
        log.error(qe.getMessage(), e);
        response.addException(qe.getBottomQueryException());
        throw new DatawaveWebApplicationException(qe, response);
    }
    // Enforce that certain InputFormat classes are being used here.
    if (this.mapReduceConfiguration.isRestrictInputFormats()) {
        // Make sure that the job input format is in the list
        Class<? extends InputFormat<?, ?>> ifClass;
        try {
            ifClass = j.getInputFormatClass();
        } catch (ClassNotFoundException e1) {
            QueryException qe = new QueryException(DatawaveErrorCode.INPUT_FORMAT_CLASS_ERROR, e1);
            log.error(qe);
            response.addException(qe);
            throw new DatawaveWebApplicationException(qe, response);
        }
        if (!this.mapReduceConfiguration.getValidInputFormats().contains(ifClass)) {
            IllegalArgumentException e = new IllegalArgumentException("Invalid input format class specified. Must use one of " + this.mapReduceConfiguration.getValidInputFormats());
            QueryException qe = new QueryException(DatawaveErrorCode.INVALID_FORMAT, e);
            log.error(qe);
            response.addException(qe.getBottomQueryException());
            throw new DatawaveWebApplicationException(qe, response);
        }
    }
    try {
        j.submit();
    } catch (Exception e) {
        QueryException qe = new QueryException(DatawaveErrorCode.MAPREDUCE_JOB_START_ERROR, e);
        log.error(qe.getMessage(), qe);
        response.addException(qe.getBottomQueryException());
        throw new DatawaveWebApplicationException(qe, response);
    }
    JobID mapReduceJobId = j.getJobID();
    log.info("JOB ID: " + mapReduceJobId);
    // Create an entry in the state table
    boolean restarted = (runtimeParameters.get(JOB_ID) != null);
    try {
        if (!restarted)
            mapReduceState.create(id, job.getHdfsUri(), job.getJobTracker(), job.getJobDir(), mapReduceJobId.toString(), job.getResultsDir(), parameters, jobName);
        else
            mapReduceState.addJob(id, mapReduceJobId.toString());
    } catch (Exception e) {
        QueryException qe = new QueryException(DatawaveErrorCode.MAPREDUCE_STATE_PERSISTENCE_ERROR, e);
        log.error(qe);
        response.addException(qe.getBottomQueryException());
        try {
            j.killJob();
        } catch (IOException ioe) {
            QueryException qe2 = new QueryException(DatawaveErrorCode.MAPREDUCE_JOB_KILL_ERROR, ioe);
            response.addException(qe2);
        }
        throw new DatawaveWebApplicationException(qe, response);
    }
    response.setResult(id);
    return response;
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) ConnectionPoolsConfiguration(datawave.webservice.common.connection.config.ConnectionPoolsConfiguration) OozieJobConfiguration(datawave.webservice.mr.configuration.OozieJobConfiguration) MapReduceJobConfiguration(datawave.webservice.mr.configuration.MapReduceJobConfiguration) MapReduceConfiguration(datawave.webservice.mr.configuration.MapReduceConfiguration) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) WebApplicationException(javax.ws.rs.WebApplicationException) HashMap(java.util.HashMap) NeedQueryPersister(datawave.webservice.mr.configuration.NeedQueryPersister) NeedCallerDetails(datawave.webservice.mr.configuration.NeedCallerDetails) NeedQueryLogicFactory(datawave.webservice.mr.configuration.NeedQueryLogicFactory) DatawavePrincipal(datawave.security.authorization.DatawavePrincipal) MapReduceJobConfiguration(datawave.webservice.mr.configuration.MapReduceJobConfiguration) Configuration(org.apache.hadoop.conf.Configuration) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) NeedQueryCache(datawave.webservice.mr.configuration.NeedQueryCache) RunningJob(org.apache.hadoop.mapred.RunningJob) Job(org.apache.hadoop.mapreduce.Job) HashSet(java.util.HashSet) GenericResponse(datawave.webservice.result.GenericResponse) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) NeedAccumuloConnectionFactory(datawave.webservice.mr.configuration.NeedAccumuloConnectionFactory) IOException(java.io.IOException) DatawaveWebApplicationException(datawave.webservice.common.exception.DatawaveWebApplicationException) WebApplicationException(javax.ws.rs.WebApplicationException) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) IOException(java.io.IOException) QueryException(datawave.webservice.query.exception.QueryException) BadRequestException(datawave.webservice.common.exception.BadRequestException) NotFoundException(datawave.webservice.common.exception.NotFoundException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) UnauthorizedException(datawave.webservice.common.exception.UnauthorizedException) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) NotFoundQueryException(datawave.webservice.query.exception.NotFoundQueryException) QueryException(datawave.webservice.query.exception.QueryException) UnauthorizedQueryException(datawave.webservice.query.exception.UnauthorizedQueryException) BadRequestQueryException(datawave.webservice.query.exception.BadRequestQueryException) Collection(java.util.Collection) BadRequestException(datawave.webservice.common.exception.BadRequestException) NeedAccumuloDetails(datawave.webservice.mr.configuration.NeedAccumuloDetails) Principal(java.security.Principal) ServerPrincipal(datawave.security.system.ServerPrincipal) DatawavePrincipal(datawave.security.authorization.DatawavePrincipal) JobID(org.apache.hadoop.mapreduce.JobID) NeedSecurityDomain(datawave.webservice.mr.configuration.NeedSecurityDomain) POST(javax.ws.rs.POST) Produces(javax.ws.rs.Produces) GZIP(org.jboss.resteasy.annotations.GZIP)

Aggregations

UnauthorizedException (datawave.webservice.common.exception.UnauthorizedException)7 DatawavePrincipal (datawave.security.authorization.DatawavePrincipal)6 UnauthorizedQueryException (datawave.webservice.query.exception.UnauthorizedQueryException)6 DatawaveWebApplicationException (datawave.webservice.common.exception.DatawaveWebApplicationException)5 BadRequestQueryException (datawave.webservice.query.exception.BadRequestQueryException)5 NotFoundQueryException (datawave.webservice.query.exception.NotFoundQueryException)5 QueryException (datawave.webservice.query.exception.QueryException)5 IOException (java.io.IOException)5 Principal (java.security.Principal)5 Produces (javax.ws.rs.Produces)5 GZIP (org.jboss.resteasy.annotations.GZIP)5 BadRequestException (datawave.webservice.common.exception.BadRequestException)4 NotFoundException (datawave.webservice.common.exception.NotFoundException)4 GenericResponse (datawave.webservice.result.GenericResponse)4 PreConditionFailedQueryException (datawave.webservice.query.exception.PreConditionFailedQueryException)3 VoidResponse (datawave.webservice.result.VoidResponse)3 Interceptors (javax.interceptor.Interceptors)3 WebApplicationException (javax.ws.rs.WebApplicationException)3 Timed (com.codahale.metrics.annotation.Timed)2 ServerPrincipal (datawave.security.system.ServerPrincipal)2