Search in sources :

Example 1 with HistoryACLPolicyException

use of org.apache.tez.common.security.HistoryACLPolicyException in project tez by apache.

the class ATSHistoryACLPolicyManager method createTimelineDomain.

private void createTimelineDomain(String domainId, Configuration tezConf, DAGAccessControls dagAccessControls) throws IOException, HistoryACLPolicyException {
    TimelineDomain timelineDomain = new TimelineDomain();
    timelineDomain.setId(domainId);
    ACLConfigurationParser parser = new ACLConfigurationParser(tezConf, false);
    timelineDomain.setReaders(getMergedViewACLs(parser, dagAccessControls));
    timelineDomain.setWriters(user);
    try {
        if (timelineClient != null) {
            timelineClient.putDomain(timelineDomain);
        }
    } catch (Exception e) {
        LOG.warn("Could not post timeline domain", e);
        throw new HistoryACLPolicyException("Fail to create ACL-related domain in Timeline", e);
    }
}
Also used : TimelineDomain(org.apache.hadoop.yarn.api.records.timeline.TimelineDomain) HistoryACLPolicyException(org.apache.tez.common.security.HistoryACLPolicyException) ACLConfigurationParser(org.apache.tez.common.security.ACLConfigurationParser) IOException(java.io.IOException) HistoryACLPolicyException(org.apache.tez.common.security.HistoryACLPolicyException) TezUncheckedException(org.apache.tez.dag.api.TezUncheckedException)

Example 2 with HistoryACLPolicyException

use of org.apache.tez.common.security.HistoryACLPolicyException in project tez by apache.

the class ATSV15HistoryACLPolicyManager method createTimelineDomain.

private void createTimelineDomain(ApplicationId applicationId, String domainId, Configuration tezConf, DAGAccessControls dagAccessControls) throws IOException, HistoryACLPolicyException {
    TimelineDomain timelineDomain = new TimelineDomain();
    timelineDomain.setId(domainId);
    ACLConfigurationParser parser = new ACLConfigurationParser(tezConf, false);
    timelineDomain.setReaders(getMergedViewACLs(parser, dagAccessControls));
    timelineDomain.setWriters(user);
    // Use dummy app attempt id
    ApplicationAttemptId appAttemptId = ApplicationAttemptId.newInstance(applicationId, 1);
    try {
        if (timelineClient != null) {
            timelineClient.putDomain(appAttemptId, timelineDomain);
        }
    } catch (Exception e) {
        LOG.warn("Could not post timeline domain", e);
        throw new HistoryACLPolicyException("Fail to create ACL-related domain in Timeline", e);
    }
}
Also used : TimelineDomain(org.apache.hadoop.yarn.api.records.timeline.TimelineDomain) ApplicationAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId) HistoryACLPolicyException(org.apache.tez.common.security.HistoryACLPolicyException) ACLConfigurationParser(org.apache.tez.common.security.ACLConfigurationParser) IOException(java.io.IOException) HistoryACLPolicyException(org.apache.tez.common.security.HistoryACLPolicyException) TezUncheckedException(org.apache.tez.dag.api.TezUncheckedException)

Example 3 with HistoryACLPolicyException

use of org.apache.tez.common.security.HistoryACLPolicyException in project tez by apache.

the class ATSV15HistoryLoggingService method serviceStart.

@Override
public void serviceStart() {
    if (!historyLoggingEnabled || timelineClient == null) {
        return;
    }
    timelineClient.start();
    // create a session domain id, if it fails then disable history logging.
    try {
        sessionDomainId = createSessionDomain();
    } catch (HistoryACLPolicyException | IOException e) {
        LOG.warn("Could not setup history acls, disabling history logging.", e);
        historyLoggingEnabled = false;
        return;
    }
    eventHandlingThread = new Thread(new Runnable() {

        @Override
        public void run() {
            boolean interrupted = false;
            TezUtilsInternal.setHadoopCallerContext(appContext.getHadoopShim(), appContext.getApplicationID());
            while (!stopped.get() && !Thread.currentThread().isInterrupted() && !interrupted) {
                // Log the size of the event-queue every so often.
                if (eventCounter != 0 && eventCounter % 1000 == 0) {
                    if (eventsProcessed != 0 && !eventQueue.isEmpty()) {
                        LOG.info("Event queue stats" + ", eventsProcessedSinceLastUpdate=" + eventsProcessed + ", eventQueueSize=" + eventQueue.size());
                    }
                    eventCounter = 0;
                    eventsProcessed = 0;
                } else {
                    ++eventCounter;
                }
                synchronized (lock) {
                    try {
                        DAGHistoryEvent event = eventQueue.poll(maxPollingTimeMillis, TimeUnit.MILLISECONDS);
                        if (event == null) {
                            continue;
                        }
                        if (!isValidEvent(event)) {
                            continue;
                        }
                        try {
                            handleEvents(event);
                            eventsProcessed += 1;
                        } catch (Exception e) {
                            LOG.warn("Error handling events", e);
                        }
                    } catch (InterruptedException e) {
                        // Finish processing events and then return
                        interrupted = true;
                    }
                }
            }
        }
    }, "HistoryEventHandlingThread");
    eventHandlingThread.start();
}
Also used : DAGHistoryEvent(org.apache.tez.dag.history.DAGHistoryEvent) IOException(java.io.IOException) HistoryACLPolicyException(org.apache.tez.common.security.HistoryACLPolicyException) HistoryACLPolicyException(org.apache.tez.common.security.HistoryACLPolicyException) TezReflectionException(org.apache.tez.dag.api.TezReflectionException) IOException(java.io.IOException)

Aggregations

IOException (java.io.IOException)3 HistoryACLPolicyException (org.apache.tez.common.security.HistoryACLPolicyException)3 TimelineDomain (org.apache.hadoop.yarn.api.records.timeline.TimelineDomain)2 ACLConfigurationParser (org.apache.tez.common.security.ACLConfigurationParser)2 TezUncheckedException (org.apache.tez.dag.api.TezUncheckedException)2 ApplicationAttemptId (org.apache.hadoop.yarn.api.records.ApplicationAttemptId)1 TezReflectionException (org.apache.tez.dag.api.TezReflectionException)1 DAGHistoryEvent (org.apache.tez.dag.history.DAGHistoryEvent)1