Search in sources :

Example 1 with StoreVersionManager

use of voldemort.store.readonly.StoreVersionManager in project voldemort by voldemort.

the class AdminServiceRequestHandler method handleDisableStoreVersion.

private Message handleDisableStoreVersion(VAdminProto.DisableStoreVersionRequest disableStoreVersion) {
    String storeName = disableStoreVersion.getStoreName();
    Long version = disableStoreVersion.getPushVersion();
    Properties properties = new Properties();
    try {
        properties.load(new StringReader(disableStoreVersion.getInfo()));
    } catch (IOException e) {
        logger.error("Got IOException while trying to decipher a DisableStoreVersionRequest's info.", e);
    }
    logger.info("Received DisableStoreVersionRequest:\n" + "\tstore_name: " + storeName + "\n" + "\tpush_version: " + version + "\n" + "\tinfo: " + properties.toString());
    VAdminProto.DisableStoreVersionResponse.Builder response = VAdminProto.DisableStoreVersionResponse.newBuilder();
    response.setNodeId(server.getMetadataStore().getNodeId());
    final String logMessagePrefix = "handleDisableStoreVersion returning response: ";
    try {
        StorageEngine storeToDisable = storeRepository.getStorageEngine(storeName);
        if (storeToDisable == null) {
            response.setDisableSuccess(false).setInfo("The store '" + storeName + "' does not exist!");
        } else {
            StoreVersionManager storeVersionManager = (StoreVersionManager) storeToDisable.getCapability(StoreCapabilityType.DISABLE_STORE_VERSION);
            storeVersionManager.disableStoreVersion(version);
            response.setDisableSuccess(true).setDisablePersistenceSuccess(true).setInfo("The store '" + storeName + "' version " + version + " was successfully disabled.");
        }
        logger.info(logMessagePrefix + response.getInfo());
    } catch (PersistenceFailureException e) {
        String message = "The store '" + storeName + "' version " + version + " was disabled" + " but the change could not be persisted and will thus remain in effect only" + " until the next server restart. This is likely caused by the IO subsystem" + " becoming read-only.";
        logger.error(logMessagePrefix + message, e);
        response.setDisableSuccess(true).setDisablePersistenceSuccess(false).setInfo(message);
    } catch (NoSuchCapabilityException e) {
        String message = "The store '" + storeName + "' does not support disabling versions!";
        logger.error(logMessagePrefix + message, e);
        response.setDisableSuccess(false).setInfo(message);
    } catch (Exception e) {
        String message = "The store '" + storeName + "' version " + version + " was not disabled because of an unexpected exception.";
        logger.error(logMessagePrefix + message, e);
        response.setDisableSuccess(false).setInfo(message);
    }
    if (response.getDisableSuccess()) {
        // Then we also want to put the server in offline mode
        VAdminProto.SetOfflineStateRequest offlineStateRequest = VAdminProto.SetOfflineStateRequest.newBuilder().setOfflineMode(true).build();
        handleSetOfflineState(offlineStateRequest);
    }
    return response.build();
}
Also used : StoreVersionManager(voldemort.store.readonly.StoreVersionManager) NoSuchCapabilityException(voldemort.store.NoSuchCapabilityException) IOException(java.io.IOException) Properties(java.util.Properties) MysqlStorageEngine(voldemort.store.mysql.MysqlStorageEngine) StorageEngine(voldemort.store.StorageEngine) SlopStorageEngine(voldemort.store.slop.SlopStorageEngine) ReadOnlyStorageEngine(voldemort.store.readonly.ReadOnlyStorageEngine) PersistenceFailureException(voldemort.store.PersistenceFailureException) NoSuchCapabilityException(voldemort.store.NoSuchCapabilityException) ConfigurationException(voldemort.utils.ConfigurationException) ObsoleteVersionException(voldemort.versioning.ObsoleteVersionException) StoreOperationFailureException(voldemort.store.StoreOperationFailureException) VoldemortException(voldemort.VoldemortException) IOException(java.io.IOException) PersistenceFailureException(voldemort.store.PersistenceFailureException) StoreNotFoundException(voldemort.store.StoreNotFoundException) StringReader(java.io.StringReader) VAdminProto(voldemort.client.protocol.pb.VAdminProto)

Example 2 with StoreVersionManager

use of voldemort.store.readonly.StoreVersionManager in project voldemort by voldemort.

the class VoldemortServer method validateReadOnlyStoreStatusBeforeGoingOnline.

private ReadOnlyStoreStatusValidation validateReadOnlyStoreStatusBeforeGoingOnline() {
    List<StorageEngine<ByteArray, byte[], byte[]>> storageEngines = storageService.getStoreRepository().getStorageEnginesByClass(ReadOnlyStorageEngine.class);
    if (storageEngines.isEmpty()) {
        logger.debug("There are no Read-Only stores on this node.");
        return new ReadOnlyStoreStatusValidation(true, null);
    } else {
        List<String> storesWithDisabledVersions = Lists.newArrayList();
        for (StorageEngine storageEngine : storageEngines) {
            StoreVersionManager storeVersionManager = (StoreVersionManager) storageEngine.getCapability(StoreCapabilityType.DISABLE_STORE_VERSION);
            if (storeVersionManager.hasAnyDisabledVersion()) {
                storesWithDisabledVersions.add(storageEngine.getName());
            }
        }
        if (storesWithDisabledVersions.isEmpty()) {
            if (voldemortConfig.getHighAvailabilityStateAutoCleanUp()) {
                logger.info(VoldemortConfig.PUSH_HA_STATE_AUTO_CLEANUP + "=true, so the server will attempt to delete the HA state for this node, if any.");
                FailedFetchLock failedFetchLock = null;
                try {
                    failedFetchLock = FailedFetchLock.getLock(getVoldemortConfig(), new Props());
                    failedFetchLock.removeObsoleteStateForNode(getVoldemortConfig().getNodeId());
                    logger.info("Successfully ensured that the BnP HA shared state is cleared for this node.");
                } catch (ClassNotFoundException e) {
                    return new ReadOnlyStoreStatusValidation(true, new VoldemortException("Failed to find FailedFetchLock class!", e));
                } catch (Exception e) {
                    return new ReadOnlyStoreStatusValidation(true, new VoldemortException("Exception while trying to remove obsolete HA state!", e));
                } finally {
                    IOUtils.closeQuietly(failedFetchLock);
                }
            } else {
                logger.info(VoldemortConfig.PUSH_HA_STATE_AUTO_CLEANUP + "=false, so the server will NOT attempt to delete the HA state for this node, if any.");
            }
            logger.info("No Read-Only stores are disabled. Going online as planned.");
            return new ReadOnlyStoreStatusValidation(true, null);
        } else {
            // OMG, there are disabled stores!
            StringBuilder stringBuilder = new StringBuilder();
            stringBuilder.append("Cannot go online, because the following Read-Only stores have some disabled version(s): ");
            boolean firstItem = true;
            for (String storeName : storesWithDisabledVersions) {
                if (firstItem) {
                    firstItem = false;
                } else {
                    stringBuilder.append(", ");
                }
                stringBuilder.append(storeName);
            }
            return new ReadOnlyStoreStatusValidation(false, new VoldemortException(stringBuilder.toString()));
        }
    }
}
Also used : StoreVersionManager(voldemort.store.readonly.StoreVersionManager) StorageEngine(voldemort.store.StorageEngine) ConfigurationStorageEngine(voldemort.store.configuration.ConfigurationStorageEngine) ReadOnlyStorageEngine(voldemort.store.readonly.ReadOnlyStorageEngine) Props(voldemort.utils.Props) VoldemortException(voldemort.VoldemortException) DisabledStoreException(voldemort.store.DisabledStoreException) VoldemortException(voldemort.VoldemortException) VoldemortApplicationException(voldemort.VoldemortApplicationException) UnknownHostException(java.net.UnknownHostException) FailedFetchLock(voldemort.store.readonly.swapper.FailedFetchLock)

Example 3 with StoreVersionManager

use of voldemort.store.readonly.StoreVersionManager in project voldemort by voldemort.

the class StorageService method startInner.

@Override
protected void startInner() {
    registerInternalEngine(metadata, false, "metadata");
    /* Initialize storage configurations */
    for (String configClassName : voldemortConfig.getStorageConfigurations()) initStorageConfig(configClassName);
    /* Initialize view storage configuration */
    storageConfigs.put(ViewStorageConfiguration.TYPE_NAME, new ViewStorageConfiguration(voldemortConfig, metadata.getStoreDefList(), storeRepository));
    /* Initialize system stores */
    initSystemStores();
    /* Register slop store */
    if (voldemortConfig.isSlopEnabled()) {
        logger.info("Initializing the slop store using " + voldemortConfig.getSlopStoreType());
        StorageConfiguration config = storageConfigs.get(voldemortConfig.getSlopStoreType());
        if (config == null)
            throw new ConfigurationException("Attempt to open store " + SlopStorageEngine.SLOP_STORE_NAME + " but " + voldemortConfig.getSlopStoreType() + " storage engine has not been enabled.");
        // make a dummy store definition object
        StoreDefinition slopStoreDefinition = new StoreDefinition(SlopStorageEngine.SLOP_STORE_NAME, null, null, null, null, null, null, RoutingStrategyType.CONSISTENT_STRATEGY, 0, null, 0, null, 0, null, null, null, null, null, null, null, null, null, null, null, null, 0);
        SlopStorageEngine slopEngine = new SlopStorageEngine(config.getStore(slopStoreDefinition, new RoutingStrategyFactory().updateRoutingStrategy(slopStoreDefinition, metadata.getCluster())), metadata.getCluster());
        registerInternalEngine(slopEngine, false, "slop");
        storeRepository.setSlopStore(slopEngine);
        if (voldemortConfig.isSlopPusherJobEnabled()) {
            // Now initialize the pusher job after some time
            GregorianCalendar cal = new GregorianCalendar();
            cal.add(Calendar.SECOND, (int) (voldemortConfig.getSlopFrequencyMs() / Time.MS_PER_SECOND));
            Date nextRun = cal.getTime();
            logger.info("Initializing slop pusher job type " + voldemortConfig.getPusherType() + " at " + nextRun);
            scheduler.schedule("slop", (voldemortConfig.getPusherType().compareTo(BlockingSlopPusherJob.TYPE_NAME) == 0) ? new BlockingSlopPusherJob(storeRepository, metadata, failureDetector, voldemortConfig, scanPermitWrapper) : new StreamingSlopPusherJob(storeRepository, metadata, slopStreamingFailureDetector, voldemortConfig, scanPermitWrapper), nextRun, voldemortConfig.getSlopFrequencyMs());
        }
        // Create a SlopPurgeJob object and register it
        if (voldemortConfig.isSlopPurgeJobEnabled()) {
            logger.info("Initializing Slop Purge job");
            SlopPurgeJob job = new SlopPurgeJob(storeRepository, metadata, scanPermitWrapper, voldemortConfig.getSlopPurgeJobMaxKeysScannedPerSec());
            JmxUtils.registerMbean(job, JmxUtils.createObjectName(job.getClass()));
            storeRepository.registerSlopPurgeJob(job);
        }
    }
    // Create a repair job object and register it with Store repository
    if (voldemortConfig.isRepairEnabled()) {
        logger.info("Initializing repair job.");
        RepairJob job = new RepairJob(storeRepository, metadata, scanPermitWrapper, voldemortConfig.getRepairJobMaxKeysScannedPerSec());
        JmxUtils.registerMbean(job, JmxUtils.createObjectName(job.getClass()));
        storeRepository.registerRepairJob(job);
    }
    // Create a prune job object and register it
    if (voldemortConfig.isPruneJobEnabled()) {
        logger.info("Intializing prune job");
        VersionedPutPruneJob job = new VersionedPutPruneJob(storeRepository, metadata, scanPermitWrapper, voldemortConfig.getPruneJobMaxKeysScannedPerSec());
        JmxUtils.registerMbean(job, JmxUtils.createObjectName(job.getClass()));
        storeRepository.registerPruneJob(job);
    }
    List<StoreDefinition> storeDefs = new ArrayList<StoreDefinition>(this.metadata.getStoreDefList());
    logger.info("Initializing stores:");
    logger.info("Validating schemas:");
    StoreDefinitionUtils.validateSchemasAsNeeded(storeDefs);
    // first initialize non-view stores
    for (StoreDefinition def : storeDefs) if (!def.isView())
        openStore(def);
    // those stores
    for (StoreDefinition def : storeDefs) {
        if (def.isView())
            openStore(def);
    }
    initializeMetadataVersions(storeDefs);
    // enable aggregate jmx statistics
    if (voldemortConfig.isStatTrackingEnabled())
        if (this.voldemortConfig.isEnableJmxClusterName())
            JmxUtils.registerMbean(new StoreStatsJmx(this.storeStats), JmxUtils.createObjectName(metadata.getCluster().getName() + ".voldemort.store.stats.aggregate", "aggregate-perf"));
        else
            JmxUtils.registerMbean(new StoreStatsJmx(this.storeStats), JmxUtils.createObjectName("voldemort.store.stats.aggregate", "aggregate-perf"));
    List<StorageEngine> listOfDisabledStores = Lists.newArrayList();
    for (StorageEngine storageEngine : storeRepository.getAllStorageEngines()) {
        try {
            StoreVersionManager storeVersionManager = (StoreVersionManager) storageEngine.getCapability(StoreCapabilityType.DISABLE_STORE_VERSION);
            if (storeVersionManager.hasAnyDisabledVersion()) {
                listOfDisabledStores.add(storageEngine);
                logger.warn("The following store is marked as disabled: " + storageEngine.getName());
            // Must put server in offline mode.
            }
        } catch (NoSuchCapabilityException e) {
        // Not a read-only store: no-op
        }
    }
    if (listOfDisabledStores.isEmpty()) {
        logger.info("All stores initialized.");
    } else {
        throw new DisabledStoreException("All stores initialized, but the server needs to go " + "in offline mode because some store(s) are disabled.");
    }
}
Also used : DisabledStoreException(voldemort.store.DisabledStoreException) BlockingSlopPusherJob(voldemort.server.scheduler.slop.BlockingSlopPusherJob) VersionedPutPruneJob(voldemort.server.storage.prunejob.VersionedPutPruneJob) RoutingStrategyFactory(voldemort.routing.RoutingStrategyFactory) GregorianCalendar(java.util.GregorianCalendar) ArrayList(java.util.ArrayList) StoreVersionManager(voldemort.store.readonly.StoreVersionManager) StorageConfiguration(voldemort.store.StorageConfiguration) FileBackedCachingStorageConfiguration(voldemort.store.configuration.FileBackedCachingStorageConfiguration) InMemoryStorageConfiguration(voldemort.store.memory.InMemoryStorageConfiguration) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) ReadOnlyStorageConfiguration(voldemort.store.readonly.ReadOnlyStorageConfiguration) StreamingSlopPusherJob(voldemort.server.scheduler.slop.StreamingSlopPusherJob) NoSuchCapabilityException(voldemort.store.NoSuchCapabilityException) ViewStorageEngine(voldemort.store.views.ViewStorageEngine) ReadOnlyStorageEngine(voldemort.store.readonly.ReadOnlyStorageEngine) StorageEngine(voldemort.store.StorageEngine) SlopStorageEngine(voldemort.store.slop.SlopStorageEngine) FileBackedCachingStorageEngine(voldemort.store.configuration.FileBackedCachingStorageEngine) ViewStorageConfiguration(voldemort.store.views.ViewStorageConfiguration) Date(java.util.Date) RepairJob(voldemort.server.storage.repairjob.RepairJob) ConfigurationException(voldemort.utils.ConfigurationException) StoreDefinition(voldemort.store.StoreDefinition) SlopStorageEngine(voldemort.store.slop.SlopStorageEngine) SlopPurgeJob(voldemort.server.scheduler.slop.SlopPurgeJob) StoreStatsJmx(voldemort.store.stats.StoreStatsJmx)

Aggregations

StorageEngine (voldemort.store.StorageEngine)3 ReadOnlyStorageEngine (voldemort.store.readonly.ReadOnlyStorageEngine)3 StoreVersionManager (voldemort.store.readonly.StoreVersionManager)3 VoldemortException (voldemort.VoldemortException)2 DisabledStoreException (voldemort.store.DisabledStoreException)2 NoSuchCapabilityException (voldemort.store.NoSuchCapabilityException)2 SlopStorageEngine (voldemort.store.slop.SlopStorageEngine)2 ConfigurationException (voldemort.utils.ConfigurationException)2 IOException (java.io.IOException)1 StringReader (java.io.StringReader)1 UnknownHostException (java.net.UnknownHostException)1 ArrayList (java.util.ArrayList)1 Date (java.util.Date)1 GregorianCalendar (java.util.GregorianCalendar)1 Properties (java.util.Properties)1 VoldemortApplicationException (voldemort.VoldemortApplicationException)1 VAdminProto (voldemort.client.protocol.pb.VAdminProto)1 RoutingStrategyFactory (voldemort.routing.RoutingStrategyFactory)1 BlockingSlopPusherJob (voldemort.server.scheduler.slop.BlockingSlopPusherJob)1 SlopPurgeJob (voldemort.server.scheduler.slop.SlopPurgeJob)1