Search in sources :

Example 1 with PulsarServerException

use of com.yahoo.pulsar.broker.PulsarServerException in project pulsar by yahoo.

the class WebSocketService method start.

public void start() throws PulsarServerException, PulsarClientException, MalformedURLException, ServletException, DeploymentException {
    if (isNotBlank(config.getGlobalZookeeperServers())) {
        this.globalZkCache = new GlobalZooKeeperCache(getZooKeeperClientFactory(), (int) config.getZooKeeperSessionTimeoutMillis(), config.getGlobalZookeeperServers(), this.orderedExecutor, this.executor);
        try {
            this.globalZkCache.start();
        } catch (IOException e) {
            throw new PulsarServerException(e);
        }
        this.configurationCacheService = new ConfigurationCacheService(getGlobalZkCache());
        log.info("Global Zookeeper cache started");
    }
    // start authorizationManager
    if (config.isAuthorizationEnabled()) {
        if (configurationCacheService == null) {
            throw new PulsarServerException("Failed to initialize authorization manager due to empty GlobalZookeeperServers");
        }
        authorizationManager = new AuthorizationManager(this.config, configurationCacheService);
    }
    // start authentication service
    authenticationService = new AuthenticationService(this.config);
    log.info("Pulsar WebSocket Service started");
}
Also used : PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) GlobalZooKeeperCache(com.yahoo.pulsar.zookeeper.GlobalZooKeeperCache) ConfigurationCacheService(com.yahoo.pulsar.broker.cache.ConfigurationCacheService) IOException(java.io.IOException) AuthorizationManager(com.yahoo.pulsar.broker.authorization.AuthorizationManager) AuthenticationService(com.yahoo.pulsar.broker.authentication.AuthenticationService)

Example 2 with PulsarServerException

use of com.yahoo.pulsar.broker.PulsarServerException in project pulsar by yahoo.

the class ProxyServer method start.

public void start() throws PulsarServerException {
    log.info("Starting web socket proxy at port {}", conf.getWebServicePort());
    try {
        RequestLogHandler requestLogHandler = new RequestLogHandler();
        Slf4jRequestLog requestLog = new Slf4jRequestLog();
        requestLog.setExtended(true);
        requestLog.setLogTimeZone("GMT");
        requestLog.setLogLatency(true);
        requestLogHandler.setRequestLog(requestLog);
        handlers.add(0, new ContextHandlerCollection());
        handlers.add(requestLogHandler);
        ContextHandlerCollection contexts = new ContextHandlerCollection();
        contexts.setHandlers(handlers.toArray(new Handler[handlers.size()]));
        HandlerCollection handlerCollection = new HandlerCollection();
        handlerCollection.setHandlers(new Handler[] { contexts, new DefaultHandler(), requestLogHandler });
        server.setHandler(handlerCollection);
        server.start();
    } catch (Exception e) {
        throw new PulsarServerException(e);
    }
}
Also used : Slf4jRequestLog(org.eclipse.jetty.server.Slf4jRequestLog) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) RequestLogHandler(org.eclipse.jetty.server.handler.RequestLogHandler) ServletContextHandler(org.eclipse.jetty.servlet.ServletContextHandler) Handler(org.eclipse.jetty.server.Handler) DefaultHandler(org.eclipse.jetty.server.handler.DefaultHandler) RequestLogHandler(org.eclipse.jetty.server.handler.RequestLogHandler) ContextHandlerCollection(org.eclipse.jetty.server.handler.ContextHandlerCollection) ContextHandlerCollection(org.eclipse.jetty.server.handler.ContextHandlerCollection) HandlerCollection(org.eclipse.jetty.server.handler.HandlerCollection) ServletException(javax.servlet.ServletException) DeploymentException(javax.websocket.DeploymentException) GeneralSecurityException(java.security.GeneralSecurityException) MalformedURLException(java.net.MalformedURLException) PulsarClientException(com.yahoo.pulsar.client.api.PulsarClientException) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) DefaultHandler(org.eclipse.jetty.server.handler.DefaultHandler)

Example 3 with PulsarServerException

use of com.yahoo.pulsar.broker.PulsarServerException in project pulsar by yahoo.

the class NamespaceService method searchForCandidateBroker.

private void searchForCandidateBroker(NamespaceBundle bundle, CompletableFuture<LookupResult> lookupFuture, boolean authoritative) {
    String candidateBroker = null;
    try {
        // check if this is Heartbeat or SLAMonitor namespace
        candidateBroker = checkHeartbeatNamespace(bundle);
        if (candidateBroker == null) {
            String broker = getSLAMonitorBrokerName(bundle);
            // checking if the broker is up and running
            if (broker != null && isBrokerActive(broker)) {
                candidateBroker = broker;
            }
        }
        if (candidateBroker == null) {
            if (!this.loadManager.isCentralized() || pulsar.getLeaderElectionService().isLeader()) {
                candidateBroker = getLeastLoadedFromLoadManager(bundle);
            } else {
                if (authoritative) {
                    // leader broker already assigned the current broker as owner
                    candidateBroker = pulsar.getWebServiceAddress();
                } else {
                    // forward to leader broker to make assignment
                    candidateBroker = pulsar.getLeaderElectionService().getCurrentLeader().getServiceUrl();
                }
            }
        }
    } catch (Exception e) {
        LOG.warn("Error when searching for candidate broker to acquire {}: {}", bundle, e.getMessage(), e);
        lookupFuture.completeExceptionally(e);
        return;
    }
    try {
        checkNotNull(candidateBroker);
        if (pulsar.getWebServiceAddress().equals(candidateBroker)) {
            // Load manager decided that the local broker should try to become the owner
            ownershipCache.tryAcquiringOwnership(bundle).thenAccept(ownerInfo -> {
                if (ownerInfo.isDisabled()) {
                    if (LOG.isDebugEnabled()) {
                        LOG.debug("Namespace bundle {} is currently being unloaded", bundle);
                    }
                    lookupFuture.completeExceptionally(new IllegalStateException(String.format("Namespace bundle %s is currently being unloaded", bundle)));
                } else {
                    pulsar.loadNamespaceDestinations(bundle);
                    lookupFuture.complete(new LookupResult(ownerInfo));
                }
            }).exceptionally(exception -> {
                LOG.warn("Failed to acquire ownership for namespace bundle {}: ", bundle, exception.getMessage(), exception);
                lookupFuture.completeExceptionally(new PulsarServerException("Failed to acquire ownership for namespace bundle " + bundle, exception));
                return null;
            });
        } else {
            if (LOG.isDebugEnabled()) {
                LOG.debug("Redirecting to broker {} to acquire ownership of bundle {}", candidateBroker, bundle);
            }
            // Now setting the redirect url
            createLookupResult(candidateBroker).thenAccept(lookupResult -> lookupFuture.complete(lookupResult)).exceptionally(ex -> {
                lookupFuture.completeExceptionally(ex);
                return null;
            });
        }
    } catch (Exception e) {
        LOG.warn("Error in trying to acquire namespace bundle ownership for {}: {}", bundle, e.getMessage(), e);
        lookupFuture.completeExceptionally(e);
    }
}
Also used : URL(java.net.URL) LoggerFactory(org.slf4j.LoggerFactory) StringUtils(org.apache.commons.lang3.StringUtils) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Matcher(java.util.regex.Matcher) Pair(org.apache.commons.lang3.tuple.Pair) Map(java.util.Map) PulsarService(com.yahoo.pulsar.broker.PulsarService) URI(java.net.URI) LocalPolicies(com.yahoo.pulsar.common.policies.data.LocalPolicies) DestinationName(com.yahoo.pulsar.common.naming.DestinationName) NamespaceBundleFactory.getBundlesData(com.yahoo.pulsar.common.naming.NamespaceBundleFactory.getBundlesData) Set(java.util.Set) StatCallback(org.apache.zookeeper.AsyncCallback.StatCallback) Collectors(java.util.stream.Collectors) String.format(java.lang.String.format) ObjectMapperFactory(com.yahoo.pulsar.common.util.ObjectMapperFactory) NamespaceIsolationPolicies(com.yahoo.pulsar.common.policies.impl.NamespaceIsolationPolicies) List(java.util.List) NamespaceBundles(com.yahoo.pulsar.common.naming.NamespaceBundles) AdminResource(com.yahoo.pulsar.broker.admin.AdminResource) Optional(java.util.Optional) Pattern(java.util.regex.Pattern) LOCAL_POLICIES_ROOT(com.yahoo.pulsar.broker.cache.LocalZooKeeperCacheService.LOCAL_POLICIES_ROOT) PulsarWebResource.joinPath(com.yahoo.pulsar.broker.web.PulsarWebResource.joinPath) AdminResource.jsonMapper(com.yahoo.pulsar.broker.admin.AdminResource.jsonMapper) LoadManager(com.yahoo.pulsar.broker.loadbalance.LoadManager) BundlesData(com.yahoo.pulsar.common.policies.data.BundlesData) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Hashing(com.google.common.hash.Hashing) LookupData(com.yahoo.pulsar.common.lookup.data.LookupData) SimpleLoadManagerImpl(com.yahoo.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl) SafeRun.safeRun(org.apache.bookkeeper.mledger.util.SafeRun.safeRun) NamespaceIsolationPolicy(com.yahoo.pulsar.common.policies.NamespaceIsolationPolicy) Lists(com.google.common.collect.Lists) LoadReport(com.yahoo.pulsar.common.policies.data.loadbalancer.LoadReport) Deserializer(com.yahoo.pulsar.zookeeper.ZooKeeperCache.Deserializer) Codec(com.yahoo.pulsar.common.util.Codec) NamespaceBundle(com.yahoo.pulsar.common.naming.NamespaceBundle) NamespaceBundleFactory(com.yahoo.pulsar.common.naming.NamespaceBundleFactory) BrokerAssignment(com.yahoo.pulsar.common.policies.data.BrokerAssignment) ServiceUnitId(com.yahoo.pulsar.common.naming.ServiceUnitId) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) LookupResult(com.yahoo.pulsar.broker.lookup.LookupResult) PulsarAdmin(com.yahoo.pulsar.client.admin.PulsarAdmin) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) NamespaceName(com.yahoo.pulsar.common.naming.NamespaceName) ServiceUnitNotReadyException(com.yahoo.pulsar.broker.service.BrokerServiceException.ServiceUnitNotReadyException) NamespaceOwnershipStatus(com.yahoo.pulsar.common.policies.data.NamespaceOwnershipStatus) ServiceConfiguration(com.yahoo.pulsar.broker.ServiceConfiguration) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) LookupResult(com.yahoo.pulsar.broker.lookup.LookupResult) KeeperException(org.apache.zookeeper.KeeperException) ServiceUnitNotReadyException(com.yahoo.pulsar.broker.service.BrokerServiceException.ServiceUnitNotReadyException) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException)

Example 4 with PulsarServerException

use of com.yahoo.pulsar.broker.PulsarServerException in project pulsar by yahoo.

the class NamespaceService method registerNamespace.

/**
     * Tried to registers a namespace to this instance
     *
     * @param namespace
     * @param ensureOwned
     * @return
     * @throws PulsarServerException
     * @throws Exception
     */
private boolean registerNamespace(String namespace, boolean ensureOwned) throws PulsarServerException {
    String myUrl = pulsar.getBrokerServiceUrl();
    try {
        NamespaceName nsname = new NamespaceName(namespace);
        // [Bug 6504511] Enable writing with JSON format
        String otherUrl = null;
        NamespaceBundle nsFullBundle = null;
        // all pre-registered namespace is assumed to have bundles disabled
        nsFullBundle = bundleFactory.getFullBundle(nsname);
        // v2 namespace will always use full bundle object
        otherUrl = ownershipCache.tryAcquiringOwnership(nsFullBundle).get().getNativeUrl();
        if (myUrl.equals(otherUrl)) {
            if (nsFullBundle != null) {
                // preload heartbeat namespace
                pulsar.loadNamespaceDestinations(nsFullBundle);
            }
            return true;
        }
        String msg = String.format("namespace already owned by other broker : ns=%s expected=%s actual=%s", namespace, myUrl, otherUrl);
        // ignore if not be owned for now
        if (!ensureOwned) {
            LOG.info(msg);
            return false;
        }
        // should not happen
        throw new IllegalStateException(msg);
    } catch (Exception e) {
        LOG.error(e.getMessage(), e);
        throw new PulsarServerException(e);
    }
}
Also used : NamespaceBundle(com.yahoo.pulsar.common.naming.NamespaceBundle) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) NamespaceName(com.yahoo.pulsar.common.naming.NamespaceName) KeeperException(org.apache.zookeeper.KeeperException) ServiceUnitNotReadyException(com.yahoo.pulsar.broker.service.BrokerServiceException.ServiceUnitNotReadyException) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException)

Example 5 with PulsarServerException

use of com.yahoo.pulsar.broker.PulsarServerException in project pulsar by yahoo.

the class NamespaceService method findBrokerServiceUrl.

/**
     * Main internal method to lookup and setup ownership of service unit to a broker
     *
     * @param bundle
     * @param authoritative
     * @param readOnly
     * @return
     * @throws PulsarServerException
     */
private CompletableFuture<LookupResult> findBrokerServiceUrl(NamespaceBundle bundle, boolean authoritative, boolean readOnly) {
    if (LOG.isDebugEnabled()) {
        LOG.debug("findBrokerServiceUrl: {} - read-only: {}", bundle, readOnly);
    }
    CompletableFuture<LookupResult> future = new CompletableFuture<>();
    // First check if we or someone else already owns the bundle
    ownershipCache.getOwnerAsync(bundle).thenAccept(nsData -> {
        if (!nsData.isPresent()) {
            if (readOnly) {
                future.completeExceptionally(new IllegalStateException(String.format("Can't find owner of ServiceUnit: %s", bundle)));
            } else {
                pulsar.getExecutor().execute(() -> {
                    searchForCandidateBroker(bundle, future, authoritative);
                });
            }
        } else if (nsData.get().isDisabled()) {
            future.completeExceptionally(new IllegalStateException(String.format("Namespace bundle %s is being unloaded", bundle)));
        } else {
            if (LOG.isDebugEnabled()) {
                LOG.debug("Namespace bundle {} already owned by {} ", bundle, nsData);
            }
            future.complete(new LookupResult(nsData.get()));
        }
    }).exceptionally(exception -> {
        LOG.warn("Failed to check owner for bundle {}: {}", bundle, exception.getMessage(), exception);
        future.completeExceptionally(exception);
        return null;
    });
    return future;
}
Also used : URL(java.net.URL) LoggerFactory(org.slf4j.LoggerFactory) StringUtils(org.apache.commons.lang3.StringUtils) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Matcher(java.util.regex.Matcher) Pair(org.apache.commons.lang3.tuple.Pair) Map(java.util.Map) PulsarService(com.yahoo.pulsar.broker.PulsarService) URI(java.net.URI) LocalPolicies(com.yahoo.pulsar.common.policies.data.LocalPolicies) DestinationName(com.yahoo.pulsar.common.naming.DestinationName) NamespaceBundleFactory.getBundlesData(com.yahoo.pulsar.common.naming.NamespaceBundleFactory.getBundlesData) Set(java.util.Set) StatCallback(org.apache.zookeeper.AsyncCallback.StatCallback) Collectors(java.util.stream.Collectors) String.format(java.lang.String.format) ObjectMapperFactory(com.yahoo.pulsar.common.util.ObjectMapperFactory) NamespaceIsolationPolicies(com.yahoo.pulsar.common.policies.impl.NamespaceIsolationPolicies) List(java.util.List) NamespaceBundles(com.yahoo.pulsar.common.naming.NamespaceBundles) AdminResource(com.yahoo.pulsar.broker.admin.AdminResource) Optional(java.util.Optional) Pattern(java.util.regex.Pattern) LOCAL_POLICIES_ROOT(com.yahoo.pulsar.broker.cache.LocalZooKeeperCacheService.LOCAL_POLICIES_ROOT) PulsarWebResource.joinPath(com.yahoo.pulsar.broker.web.PulsarWebResource.joinPath) AdminResource.jsonMapper(com.yahoo.pulsar.broker.admin.AdminResource.jsonMapper) LoadManager(com.yahoo.pulsar.broker.loadbalance.LoadManager) BundlesData(com.yahoo.pulsar.common.policies.data.BundlesData) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Hashing(com.google.common.hash.Hashing) LookupData(com.yahoo.pulsar.common.lookup.data.LookupData) SimpleLoadManagerImpl(com.yahoo.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl) SafeRun.safeRun(org.apache.bookkeeper.mledger.util.SafeRun.safeRun) NamespaceIsolationPolicy(com.yahoo.pulsar.common.policies.NamespaceIsolationPolicy) Lists(com.google.common.collect.Lists) LoadReport(com.yahoo.pulsar.common.policies.data.loadbalancer.LoadReport) Deserializer(com.yahoo.pulsar.zookeeper.ZooKeeperCache.Deserializer) Codec(com.yahoo.pulsar.common.util.Codec) NamespaceBundle(com.yahoo.pulsar.common.naming.NamespaceBundle) NamespaceBundleFactory(com.yahoo.pulsar.common.naming.NamespaceBundleFactory) BrokerAssignment(com.yahoo.pulsar.common.policies.data.BrokerAssignment) ServiceUnitId(com.yahoo.pulsar.common.naming.ServiceUnitId) Logger(org.slf4j.Logger) KeeperException(org.apache.zookeeper.KeeperException) LookupResult(com.yahoo.pulsar.broker.lookup.LookupResult) PulsarAdmin(com.yahoo.pulsar.client.admin.PulsarAdmin) Preconditions.checkNotNull(com.google.common.base.Preconditions.checkNotNull) NamespaceName(com.yahoo.pulsar.common.naming.NamespaceName) ServiceUnitNotReadyException(com.yahoo.pulsar.broker.service.BrokerServiceException.ServiceUnitNotReadyException) NamespaceOwnershipStatus(com.yahoo.pulsar.common.policies.data.NamespaceOwnershipStatus) ServiceConfiguration(com.yahoo.pulsar.broker.ServiceConfiguration) PulsarServerException(com.yahoo.pulsar.broker.PulsarServerException) CompletableFuture(java.util.concurrent.CompletableFuture) LookupResult(com.yahoo.pulsar.broker.lookup.LookupResult)

Aggregations

PulsarServerException (com.yahoo.pulsar.broker.PulsarServerException)15 KeeperException (org.apache.zookeeper.KeeperException)6 NamespaceName (com.yahoo.pulsar.common.naming.NamespaceName)5 ServiceConfiguration (com.yahoo.pulsar.broker.ServiceConfiguration)4 NamespaceBundle (com.yahoo.pulsar.common.naming.NamespaceBundle)4 Preconditions.checkArgument (com.google.common.base.Preconditions.checkArgument)3 Preconditions.checkNotNull (com.google.common.base.Preconditions.checkNotNull)3 PulsarService (com.yahoo.pulsar.broker.PulsarService)3 ServiceUnitNotReadyException (com.yahoo.pulsar.broker.service.BrokerServiceException.ServiceUnitNotReadyException)3 PulsarWebResource.joinPath (com.yahoo.pulsar.broker.web.PulsarWebResource.joinPath)3 LocalPolicies (com.yahoo.pulsar.common.policies.data.LocalPolicies)3 LoadReport (com.yahoo.pulsar.common.policies.data.loadbalancer.LoadReport)3 ObjectMapperFactory (com.yahoo.pulsar.common.util.ObjectMapperFactory)3 Lists (com.google.common.collect.Lists)2 Hashing (com.google.common.hash.Hashing)2 AdminResource (com.yahoo.pulsar.broker.admin.AdminResource)2 AdminResource.jsonMapper (com.yahoo.pulsar.broker.admin.AdminResource.jsonMapper)2 LOCAL_POLICIES_ROOT (com.yahoo.pulsar.broker.cache.LocalZooKeeperCacheService.LOCAL_POLICIES_ROOT)2 LoadManager (com.yahoo.pulsar.broker.loadbalance.LoadManager)2 SimpleLoadManagerImpl (com.yahoo.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl)2