Search in sources :

Example 1 with ConfigurationException

use of voldemort.utils.ConfigurationException in project voldemort by voldemort.

the class VoldemortServletContextListener method contextInitialized.

public void contextInitialized(ServletContextEvent event) {
    try {
        logger.info("Creating application...");
        VoldemortServer server = new VoldemortServer(VoldemortConfig.loadFromEnvironmentVariable());
        event.getServletContext().setAttribute(SERVER_KEY, server);
        event.getServletContext().setAttribute(SERVER_CONFIG_KEY, server.getVoldemortConfig());
        event.getServletContext().setAttribute(VELOCITY_ENGINE_KEY, new VelocityEngine(VOLDEMORT_TEMPLATE_DIR));
        server.start();
        logger.info("Application created.");
    } catch (ConfigurationException e) {
        logger.info("Error loading voldemort server:", e);
        throw e;
    } catch (Exception e) {
        logger.error("Error loading voldemort server:", e);
        throw new ConfigurationException(e);
    }
}
Also used : VelocityEngine(voldemort.server.http.gui.VelocityEngine) ConfigurationException(voldemort.utils.ConfigurationException) VoldemortServer(voldemort.server.VoldemortServer) ConfigurationException(voldemort.utils.ConfigurationException)

Example 2 with ConfigurationException

use of voldemort.utils.ConfigurationException in project voldemort by voldemort.

the class StorageService method removeEngine.

/**
 * Unregister and remove the engine from the storage repository. This is
 * called during deletion of stores and if there are exceptions
 * adding/opening stores
 *
 * @param engine The actual engine to remove
 * @param isReadOnly Is this read-only?
 * @param storeType The storage type of the store
 * @param truncate Should the store be truncated?
 */
public void removeEngine(StorageEngine<ByteArray, byte[], byte[]> engine, boolean isReadOnly, String storeType, boolean truncate) {
    String storeName = engine.getName();
    Store<ByteArray, byte[], byte[]> store = storeRepository.removeLocalStore(storeName);
    boolean isSlop = storeType.compareTo("slop") == 0;
    boolean isView = storeType.compareTo(ViewStorageConfiguration.TYPE_NAME) == 0;
    boolean isMetadata = storeName.compareTo(MetadataStore.METADATA_STORE_NAME) == 0;
    if (store != null) {
        if (voldemortConfig.isJmxEnabled()) {
            MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer();
            if (!isSlop && voldemortConfig.isEnableRebalanceService() && !isReadOnly && !isMetadata && !isView) {
                ObjectName name = null;
                if (this.voldemortConfig.isEnableJmxClusterName())
                    name = JmxUtils.createObjectName(metadata.getCluster().getName() + "." + JmxUtils.getPackageName(RedirectingStore.class), store.getName());
                else
                    name = JmxUtils.createObjectName(JmxUtils.getPackageName(RedirectingStore.class), store.getName());
                synchronized (mbeanServer) {
                    if (mbeanServer.isRegistered(name))
                        JmxUtils.unregisterMbean(mbeanServer, name);
                }
            }
            if (voldemortConfig.isStatTrackingEnabled()) {
                ObjectName name = null;
                if (this.voldemortConfig.isEnableJmxClusterName())
                    name = JmxUtils.createObjectName(metadata.getCluster().getName() + "." + JmxUtils.getPackageName(store.getClass()), store.getName());
                else
                    name = JmxUtils.createObjectName(JmxUtils.getPackageName(store.getClass()), store.getName());
                synchronized (mbeanServer) {
                    if (mbeanServer.isRegistered(name))
                        JmxUtils.unregisterMbean(mbeanServer, name);
                }
            }
        }
        if (voldemortConfig.isServerRoutingEnabled() && !isSlop) {
            this.storeRepository.removeRoutedStore(storeName);
            for (Node node : metadata.getCluster().getNodes()) this.storeRepository.removeNodeStore(storeName, node.getId());
        }
    }
    storeRepository.removeStorageEngine(storeName);
    // Then truncate (if needed) and close
    if (truncate) {
        engine.truncate();
    }
    engine.close();
    // Also remove any state in the StorageConfiguration (if required)
    StorageConfiguration config = storageConfigs.get(storeType);
    if (config == null) {
        throw new ConfigurationException("Attempt to close storage engine " + engine.getName() + " but " + storeType + " storage engine has not been enabled.");
    }
    config.removeStorageEngine(engine);
}
Also used : ConfigurationException(voldemort.utils.ConfigurationException) RedirectingStore(voldemort.store.rebalancing.RedirectingStore) Node(voldemort.cluster.Node) ByteArray(voldemort.utils.ByteArray) StorageConfiguration(voldemort.store.StorageConfiguration) FileBackedCachingStorageConfiguration(voldemort.store.configuration.FileBackedCachingStorageConfiguration) InMemoryStorageConfiguration(voldemort.store.memory.InMemoryStorageConfiguration) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) ReadOnlyStorageConfiguration(voldemort.store.readonly.ReadOnlyStorageConfiguration) MBeanServer(javax.management.MBeanServer) ObjectName(javax.management.ObjectName)

Example 3 with ConfigurationException

use of voldemort.utils.ConfigurationException in project voldemort by voldemort.

the class StorageService method updateStore.

public void updateStore(StoreDefinition storeDef) {
    logger.info("Updating store '" + storeDef.getName() + "' (" + storeDef.getType() + ").");
    StorageConfiguration config = storageConfigs.get(storeDef.getType());
    if (config == null)
        throw new ConfigurationException("Attempt to open store " + storeDef.getName() + " but " + storeDef.getType() + " storage engine has not been enabled.");
    config.update(storeDef);
}
Also used : ConfigurationException(voldemort.utils.ConfigurationException) StorageConfiguration(voldemort.store.StorageConfiguration) FileBackedCachingStorageConfiguration(voldemort.store.configuration.FileBackedCachingStorageConfiguration) InMemoryStorageConfiguration(voldemort.store.memory.InMemoryStorageConfiguration) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) ReadOnlyStorageConfiguration(voldemort.store.readonly.ReadOnlyStorageConfiguration)

Example 4 with ConfigurationException

use of voldemort.utils.ConfigurationException in project voldemort by voldemort.

the class StorageService method startInner.

@Override
protected void startInner() {
    registerInternalEngine(metadata, false, "metadata");
    /* Initialize storage configurations */
    for (String configClassName : voldemortConfig.getStorageConfigurations()) initStorageConfig(configClassName);
    /* Initialize view storage configuration */
    storageConfigs.put(ViewStorageConfiguration.TYPE_NAME, new ViewStorageConfiguration(voldemortConfig, metadata.getStoreDefList(), storeRepository));
    /* Initialize system stores */
    initSystemStores();
    /* Register slop store */
    if (voldemortConfig.isSlopEnabled()) {
        logger.info("Initializing the slop store using " + voldemortConfig.getSlopStoreType());
        StorageConfiguration config = storageConfigs.get(voldemortConfig.getSlopStoreType());
        if (config == null)
            throw new ConfigurationException("Attempt to open store " + SlopStorageEngine.SLOP_STORE_NAME + " but " + voldemortConfig.getSlopStoreType() + " storage engine has not been enabled.");
        // make a dummy store definition object
        StoreDefinition slopStoreDefinition = new StoreDefinition(SlopStorageEngine.SLOP_STORE_NAME, null, null, null, null, null, null, RoutingStrategyType.CONSISTENT_STRATEGY, 0, null, 0, null, 0, null, null, null, null, null, null, null, null, null, null, null, null, 0);
        SlopStorageEngine slopEngine = new SlopStorageEngine(config.getStore(slopStoreDefinition, new RoutingStrategyFactory().updateRoutingStrategy(slopStoreDefinition, metadata.getCluster())), metadata.getCluster());
        registerInternalEngine(slopEngine, false, "slop");
        storeRepository.setSlopStore(slopEngine);
        if (voldemortConfig.isSlopPusherJobEnabled()) {
            // Now initialize the pusher job after some time
            GregorianCalendar cal = new GregorianCalendar();
            cal.add(Calendar.SECOND, (int) (voldemortConfig.getSlopFrequencyMs() / Time.MS_PER_SECOND));
            Date nextRun = cal.getTime();
            logger.info("Initializing slop pusher job type " + voldemortConfig.getPusherType() + " at " + nextRun);
            scheduler.schedule("slop", (voldemortConfig.getPusherType().compareTo(BlockingSlopPusherJob.TYPE_NAME) == 0) ? new BlockingSlopPusherJob(storeRepository, metadata, failureDetector, voldemortConfig, scanPermitWrapper) : new StreamingSlopPusherJob(storeRepository, metadata, slopStreamingFailureDetector, voldemortConfig, scanPermitWrapper), nextRun, voldemortConfig.getSlopFrequencyMs());
        }
        // Create a SlopPurgeJob object and register it
        if (voldemortConfig.isSlopPurgeJobEnabled()) {
            logger.info("Initializing Slop Purge job");
            SlopPurgeJob job = new SlopPurgeJob(storeRepository, metadata, scanPermitWrapper, voldemortConfig.getSlopPurgeJobMaxKeysScannedPerSec());
            JmxUtils.registerMbean(job, JmxUtils.createObjectName(job.getClass()));
            storeRepository.registerSlopPurgeJob(job);
        }
    }
    // Create a repair job object and register it with Store repository
    if (voldemortConfig.isRepairEnabled()) {
        logger.info("Initializing repair job.");
        RepairJob job = new RepairJob(storeRepository, metadata, scanPermitWrapper, voldemortConfig.getRepairJobMaxKeysScannedPerSec());
        JmxUtils.registerMbean(job, JmxUtils.createObjectName(job.getClass()));
        storeRepository.registerRepairJob(job);
    }
    // Create a prune job object and register it
    if (voldemortConfig.isPruneJobEnabled()) {
        logger.info("Intializing prune job");
        VersionedPutPruneJob job = new VersionedPutPruneJob(storeRepository, metadata, scanPermitWrapper, voldemortConfig.getPruneJobMaxKeysScannedPerSec());
        JmxUtils.registerMbean(job, JmxUtils.createObjectName(job.getClass()));
        storeRepository.registerPruneJob(job);
    }
    List<StoreDefinition> storeDefs = new ArrayList<StoreDefinition>(this.metadata.getStoreDefList());
    logger.info("Initializing stores:");
    logger.info("Validating schemas:");
    StoreDefinitionUtils.validateSchemasAsNeeded(storeDefs);
    // first initialize non-view stores
    for (StoreDefinition def : storeDefs) if (!def.isView())
        openStore(def);
    // those stores
    for (StoreDefinition def : storeDefs) {
        if (def.isView())
            openStore(def);
    }
    initializeMetadataVersions(storeDefs);
    // enable aggregate jmx statistics
    if (voldemortConfig.isStatTrackingEnabled())
        if (this.voldemortConfig.isEnableJmxClusterName())
            JmxUtils.registerMbean(new StoreStatsJmx(this.storeStats), JmxUtils.createObjectName(metadata.getCluster().getName() + ".voldemort.store.stats.aggregate", "aggregate-perf"));
        else
            JmxUtils.registerMbean(new StoreStatsJmx(this.storeStats), JmxUtils.createObjectName("voldemort.store.stats.aggregate", "aggregate-perf"));
    List<StorageEngine> listOfDisabledStores = Lists.newArrayList();
    for (StorageEngine storageEngine : storeRepository.getAllStorageEngines()) {
        try {
            StoreVersionManager storeVersionManager = (StoreVersionManager) storageEngine.getCapability(StoreCapabilityType.DISABLE_STORE_VERSION);
            if (storeVersionManager.hasAnyDisabledVersion()) {
                listOfDisabledStores.add(storageEngine);
                logger.warn("The following store is marked as disabled: " + storageEngine.getName());
            // Must put server in offline mode.
            }
        } catch (NoSuchCapabilityException e) {
        // Not a read-only store: no-op
        }
    }
    if (listOfDisabledStores.isEmpty()) {
        logger.info("All stores initialized.");
    } else {
        throw new DisabledStoreException("All stores initialized, but the server needs to go " + "in offline mode because some store(s) are disabled.");
    }
}
Also used : DisabledStoreException(voldemort.store.DisabledStoreException) BlockingSlopPusherJob(voldemort.server.scheduler.slop.BlockingSlopPusherJob) VersionedPutPruneJob(voldemort.server.storage.prunejob.VersionedPutPruneJob) RoutingStrategyFactory(voldemort.routing.RoutingStrategyFactory) GregorianCalendar(java.util.GregorianCalendar) ArrayList(java.util.ArrayList) StoreVersionManager(voldemort.store.readonly.StoreVersionManager) StorageConfiguration(voldemort.store.StorageConfiguration) FileBackedCachingStorageConfiguration(voldemort.store.configuration.FileBackedCachingStorageConfiguration) InMemoryStorageConfiguration(voldemort.store.memory.InMemoryStorageConfiguration) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) ReadOnlyStorageConfiguration(voldemort.store.readonly.ReadOnlyStorageConfiguration) StreamingSlopPusherJob(voldemort.server.scheduler.slop.StreamingSlopPusherJob) NoSuchCapabilityException(voldemort.store.NoSuchCapabilityException) ViewStorageEngine(voldemort.store.views.ViewStorageEngine) ReadOnlyStorageEngine(voldemort.store.readonly.ReadOnlyStorageEngine) StorageEngine(voldemort.store.StorageEngine) SlopStorageEngine(voldemort.store.slop.SlopStorageEngine) FileBackedCachingStorageEngine(voldemort.store.configuration.FileBackedCachingStorageEngine) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) Date(java.util.Date) RepairJob(voldemort.server.storage.repairjob.RepairJob) ConfigurationException(voldemort.utils.ConfigurationException) StoreDefinition(voldemort.store.StoreDefinition) SlopStorageEngine(voldemort.store.slop.SlopStorageEngine) SlopPurgeJob(voldemort.server.scheduler.slop.SlopPurgeJob) StoreStatsJmx(voldemort.store.stats.StoreStatsJmx)

Example 5 with ConfigurationException

use of voldemort.utils.ConfigurationException in project voldemort by voldemort.

the class StorageService method openStore.

public StorageEngine<ByteArray, byte[], byte[]> openStore(StoreDefinition storeDef) {
    logger.info("Opening store '" + storeDef.getName() + "' (" + storeDef.getType() + ").");
    StorageConfiguration config = storageConfigs.get(storeDef.getType());
    if (config == null)
        throw new ConfigurationException("Attempt to open store " + storeDef.getName() + " but " + storeDef.getType() + " storage engine has not been enabled.");
    boolean isReadOnly = storeDef.getType().compareTo(ReadOnlyStorageConfiguration.TYPE_NAME) == 0;
    final RoutingStrategy routingStrategy = new RoutingStrategyFactory().updateRoutingStrategy(storeDef, metadata.getCluster());
    final StorageEngine<ByteArray, byte[], byte[]> engine = config.getStore(storeDef, routingStrategy);
    // Update the routing strategy + add listener to metadata
    if (storeDef.getType().compareTo(ReadOnlyStorageConfiguration.TYPE_NAME) == 0) {
        metadata.addMetadataStoreListener(storeDef.getName(), new MetadataStoreListener() {

            public void updateRoutingStrategy(RoutingStrategy updatedRoutingStrategy) {
                ((ReadOnlyStorageEngine) engine).setRoutingStrategy(updatedRoutingStrategy);
            }

            public void updateStoreDefinition(StoreDefinition storeDef) {
                return;
            }
        });
    }
    // openStore() should have atomic semantics
    try {
        registerEngine(engine, isReadOnly, storeDef.getType(), storeDef);
        if (voldemortConfig.isServerRoutingEnabled())
            registerNodeStores(storeDef, metadata.getCluster(), voldemortConfig.getNodeId());
        if (storeDef.hasRetentionPeriod())
            scheduleCleanupJob(storeDef, engine);
    } catch (Exception e) {
        removeEngine(engine, isReadOnly, storeDef.getType(), false);
        throw new VoldemortException(e);
    }
    return engine;
}
Also used : ConfigurationException(voldemort.utils.ConfigurationException) RoutingStrategyFactory(voldemort.routing.RoutingStrategyFactory) StoreDefinition(voldemort.store.StoreDefinition) RoutingStrategy(voldemort.routing.RoutingStrategy) StorageConfiguration(voldemort.store.StorageConfiguration) FileBackedCachingStorageConfiguration(voldemort.store.configuration.FileBackedCachingStorageConfiguration) InMemoryStorageConfiguration(voldemort.store.memory.InMemoryStorageConfiguration) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) ReadOnlyStorageConfiguration(voldemort.store.readonly.ReadOnlyStorageConfiguration) ByteArray(voldemort.utils.ByteArray) MetadataStoreListener(voldemort.store.metadata.MetadataStoreListener) VoldemortException(voldemort.VoldemortException) ConfigurationException(voldemort.utils.ConfigurationException) NoSuchCapabilityException(voldemort.store.NoSuchCapabilityException) DisabledStoreException(voldemort.store.DisabledStoreException) VoldemortException(voldemort.VoldemortException)

Aggregations

ConfigurationException (voldemort.utils.ConfigurationException)9 StorageConfiguration (voldemort.store.StorageConfiguration)6 FileBackedCachingStorageConfiguration (voldemort.store.configuration.FileBackedCachingStorageConfiguration)6 InMemoryStorageConfiguration (voldemort.store.memory.InMemoryStorageConfiguration)6 ReadOnlyStorageConfiguration (voldemort.store.readonly.ReadOnlyStorageConfiguration)6 ViewStorageConfiguration (voldemort.store.views.ViewStorageConfiguration)6 DisabledStoreException (voldemort.store.DisabledStoreException)3 NoSuchCapabilityException (voldemort.store.NoSuchCapabilityException)3 ByteArray (voldemort.utils.ByteArray)3 VoldemortException (voldemort.VoldemortException)2 RoutingStrategyFactory (voldemort.routing.RoutingStrategyFactory)2 StoreDefinition (voldemort.store.StoreDefinition)2 File (java.io.File)1 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Date (java.util.Date)1 GregorianCalendar (java.util.GregorianCalendar)1 MBeanServer (javax.management.MBeanServer)1 ObjectName (javax.management.ObjectName)1 Node (voldemort.cluster.Node)1