Search in sources :

Example 56 with InterProcessLock

use of org.apache.curator.framework.recipes.locks.InterProcessLock in project coprhd-controller by CoprHD.

the class SchemaUtil method checkAndInitStorageSystemTypes.

public void checkAndInitStorageSystemTypes(DbClient dbClient) {
    if (onStandby) {
        _log.info("Skip StorageSystemType CF initialization on standby site");
        return;
    }
    InterProcessLock lock = null;
    try {
        lock = _coordinator.getLock(STORAGE_SYSTEM_TYPE_INIT_LOCK);
        _log.info("StorageSystemType check - waiting for StorageSystemType CF init lock");
        lock.acquire();
        StorageSystemTypesInitUtils utils = new StorageSystemTypesInitUtils(dbClient);
        utils.initializeStorageSystemTypes();
    } catch (Exception e) {
        _log.warn("Exception happend when trying to acquire lock", e);
    } finally {
        if (lock != null) {
            try {
                lock.release();
            } catch (Exception e) {
                _log.error("Fail to release lock", e);
            }
        }
    }
}
Also used : InterProcessLock(org.apache.curator.framework.recipes.locks.InterProcessLock) DatabaseException(com.emc.storageos.db.exceptions.DatabaseException) OperationException(com.netflix.astyanax.connectionpool.exceptions.OperationException) ConnectionException(com.netflix.astyanax.connectionpool.exceptions.ConnectionException) UnknownHostException(java.net.UnknownHostException)

Example 57 with InterProcessLock

use of org.apache.curator.framework.recipes.locks.InterProcessLock in project coprhd-controller by CoprHD.

the class StorageDriverManager method getLock.

private InterProcessLock getLock(String name) throws Exception {
    InterProcessLock lock = null;
    while (true) {
        try {
            // As only active site nodes will try to acquiring this lock
            // site local lock is competent
            lock = coordinator.getCoordinatorClient().getSiteLocalLock(name);
            lock.acquire();
            // got lock
            break;
        } catch (Exception e) {
            if (coordinator.isConnected()) {
                throw e;
            }
        }
    }
    return lock;
}
Also used : InterProcessLock(org.apache.curator.framework.recipes.locks.InterProcessLock) APIException(com.emc.storageos.svcs.errorhandling.resources.APIException)

Example 58 with InterProcessLock

use of org.apache.curator.framework.recipes.locks.InterProcessLock in project coprhd-controller by CoprHD.

the class DrDbHealthMonitor method checkIncrementalSyncingSite.

private void checkIncrementalSyncingSite(Site standbySite, List<Site> sitesToDegrade) {
    String siteId = standbySite.getUuid();
    int nodeCount = standbySite.getNodeCount();
    // We must wait until all the dbsvc/geodbsvc instances are back
    // or the data sync will fail anyways
    int liveDbsvcNodeCount = drUtil.getNumberOfLiveServices(siteId, Constants.DBSVC_NAME);
    int liveGeodbsvcNodeCount = drUtil.getNumberOfLiveServices(siteId, Constants.GEODBSVC_NAME);
    if (liveDbsvcNodeCount != nodeCount || liveGeodbsvcNodeCount != nodeCount) {
        log.info("Not all the dbsvc/geodbsvc instances are back. dbsvc active nodes {}, geodbsvc live nodes {}", liveDbsvcNodeCount, liveGeodbsvcNodeCount);
        boolean quorumLost = (liveDbsvcNodeCount <= nodeCount / 2) || (liveGeodbsvcNodeCount <= nodeCount / 2);
        if (quorumLost) {
            SiteMonitorResult monitorResult = coordinatorClient.getTargetInfo(siteId, SiteMonitorResult.class);
            checkEligibleForDegrade(monitorResult, standbySite, sitesToDegrade);
        }
        return;
    }
    log.info("All the dbsvc/geodbsvc instances are back. {}. Check if we need reset STANDBY_INCR_SYNCING state", standbySite.getUuid());
    InterProcessLock lock;
    try {
        lock = drUtil.getDROperationLock();
    } catch (APIException e) {
        log.warn("There are ongoing DR operations. Try again later.");
        return;
    }
    try {
        // reload site from zookeeper
        Site site = drUtil.getSiteFromLocalVdc(standbySite.getUuid());
        if (site.getState() != SiteState.STANDBY_INCR_SYNCING) {
            log.info("Skip incremental syncing state check. Site {} current state is {}", site.getUuid(), site.getState());
            return;
        }
        String dcName = drUtil.getCassandraDcId(site);
        Collection<String> ipAddrs = drUtil.getLocalSite().getHostIPv4AddressMap().values();
        if (ipAddrs.isEmpty()) {
            ipAddrs = drUtil.getLocalSite().getHostIPv6AddressMap().values();
        }
        for (String host : ipAddrs) {
            boolean isLocaldbsvcSynced = isDataCenterSynced(host, DEFAULTPORT, dcName);
            if (!isLocaldbsvcSynced) {
                return;
            }
            boolean isGeodbsvcSynced = isDataCenterSynced(host, DEFAULTGEOPORT, dcName);
            if (!isGeodbsvcSynced) {
                return;
            }
        }
        log.info("Data synced for dbsvc/geodbsvc of standby site {}", site.getUuid());
        site.setState(SiteState.STANDBY_SYNCED);
        drUtil.getCoordinator().persistServiceConfiguration(site.toConfiguration());
        updateSiteMonitorResult(standbySite);
    } catch (Exception e) {
        log.error("Failed to initiate reset STATNDBY_INCR_SYNCING standby operation. Try again later", e);
    } finally {
        try {
            lock.release();
        } catch (Exception e) {
            log.error("Failed to release the dr operation lock", e);
        }
    }
}
Also used : APIException(com.emc.storageos.svcs.errorhandling.resources.APIException) InterProcessLock(org.apache.curator.framework.recipes.locks.InterProcessLock) IOException(java.io.IOException) APIException(com.emc.storageos.svcs.errorhandling.resources.APIException) MalformedObjectNameException(javax.management.MalformedObjectNameException)

Example 59 with InterProcessLock

use of org.apache.curator.framework.recipes.locks.InterProcessLock in project coprhd-controller by CoprHD.

the class StorageDriverServiceTest method setUp.

@Before
public void setUp() throws Exception {
    // mock dependent members of StorageDriverService instance and wire into
    // them
    dbClient = mock(DbClientImpl.class);
    doNothing().when(dbClient).createObject(any(StorageSystemType.class));
    doNothing().when(dbClient).updateObject(any(StorageSystemType.class));
    coordinator = mock(CoordinatorClientImpl.class);
    coordinatorExt = mock(CoordinatorClientExt.class);
    doReturn(coordinator).when(coordinatorExt).getCoordinatorClient();
    doNothing().when(coordinator).persistServiceConfiguration(any(Configuration.class));
    service = spy(new StorageDriverService());
    service.setDbClient(dbClient);
    service.setCoordinatorExt(coordinatorExt);
    // mock 3 storage system types in db
    type1 = new StorageSystemType();
    type1.setStorageTypeDispName("systemtype1");
    type1.setStorageTypeName("systemtype1");
    type1.setIsNative(false);
    type1.setDriverStatus(StorageSystemType.STATUS.ACTIVE.toString());
    type1.setDriverName("driver1");
    type1.setDriverFileName("driverFileName1");
    type1.setDriverVersion("1.2.3.4");
    type2 = new StorageSystemType();
    type2.setStorageTypeDispName("providertype1");
    type2.setStorageTypeName("providertype1");
    type2.setIsNative(false);
    type2.setDriverStatus(StorageSystemType.STATUS.ACTIVE.toString());
    type2.setDriverName("driver1");
    type2.setDriverFileName("driverFileName1");
    type2.setDriverVersion("1.2.3.4");
    type3 = new StorageSystemType();
    type3.setStorageTypeDispName("systemtype2");
    type3.setStorageTypeName("systemtype2");
    type3.setIsNative(false);
    type3.setDriverStatus(StorageSystemType.STATUS.ACTIVE.toString());
    type3.setDriverName("driver2");
    List<StorageSystemType> types = new ArrayList<StorageSystemType>();
    types.add(type1);
    types.add(type2);
    types.add(type3);
    doReturn(types.iterator()).when(dbClient).queryIterativeObjects(eq(StorageSystemType.class), anyCollectionOf(URI.class));
    // mock that systemtype1 is in use
    Set<String> inUseTypes = new HashSet<String>();
    inUseTypes.add("systemtype1");
    doReturn(inUseTypes).when(service).getUsedStorageProviderTypes();
    // mock that no systemtype is used
    doReturn(new HashSet<String>()).when(service).getUsedStorageSystemTypes();
    // bypass pre-check for environment
    doNothing().when(service).precheckForEnv();
    // mock lock acquire and release
    InterProcessLock lock = mock(InterProcessLock.class);
    doNothing().when(lock).release();
    doReturn(lock).when(service).getStorageDriverOperationLock();
    // mock target list of installed drivers
    StorageDriversInfo drivers = new StorageDriversInfo();
    Set<String> installedDrivers = new HashSet<String>();
    installedDrivers.add("driverFileName1");
    installedDrivers.add("driverFileName2");
    drivers.setInstalledDrivers(installedDrivers);
    doReturn(drivers).when(coordinator).getTargetInfo(anyObject());
    // mock audit operation
    doNothing().when(service).auditOperation(any(OperationTypeEnum.class), anyString(), anyString(), anyObject());
    // mock file moving from tmp dir to data dir
    doNothing().when(service).moveDriverToDataDir(any(File.class));
    // mock updating target innfo
    doNothing().when(coordinator).setTargetInfo(anyObject());
    // mock progess updating
    doNothing().when(coordinatorExt).setNodeSessionScopeInfo(anyObject());
    // mock moving driver file to tmp dir
    doReturn(null).when(service).saveToTmpDir(anyString(), anyObject());
}
Also used : Configuration(com.emc.storageos.coordinator.common.Configuration) OperationTypeEnum(com.emc.storageos.services.OperationTypeEnum) ArrayList(java.util.ArrayList) StorageSystemType(com.emc.storageos.db.client.model.StorageSystemType) StorageDriversInfo(com.emc.storageos.coordinator.client.model.StorageDriversInfo) URI(java.net.URI) CoordinatorClientImpl(com.emc.storageos.coordinator.client.service.impl.CoordinatorClientImpl) DbClientImpl(com.emc.storageos.db.client.impl.DbClientImpl) InterProcessLock(org.apache.curator.framework.recipes.locks.InterProcessLock) CoordinatorClientExt(com.emc.storageos.systemservices.impl.upgrade.CoordinatorClientExt) File(java.io.File) HashSet(java.util.HashSet) Before(org.junit.Before)

Example 60 with InterProcessLock

use of org.apache.curator.framework.recipes.locks.InterProcessLock in project coprhd-controller by CoprHD.

the class RecoveryManager method triggerNodeRecovery.

/**
 * Trigger node recovery by update recovery status to 'INIT'
 */
public void triggerNodeRecovery() {
    InterProcessLock lock = null;
    try {
        lock = getRecoveryLock();
        validateNodeRecoveryStatus();
        validateClusterState();
        RecoveryStatus status = new RecoveryStatus();
        status.setStatus(RecoveryStatus.Status.INIT);
        status.setStartTime(new Date());
        persistNodeRecoveryStatus(status);
    } finally {
        releaseLock(lock);
    }
}
Also used : InterProcessLock(org.apache.curator.framework.recipes.locks.InterProcessLock) RecoveryStatus(com.emc.vipr.model.sys.recovery.RecoveryStatus) Date(java.util.Date)

Aggregations

InterProcessLock (org.apache.curator.framework.recipes.locks.InterProcessLock)98 APIException (com.emc.storageos.svcs.errorhandling.resources.APIException)25 DatabaseException (com.emc.storageos.db.exceptions.DatabaseException)21 DeviceControllerException (com.emc.storageos.exceptions.DeviceControllerException)15 IOException (java.io.IOException)15 ControllerException (com.emc.storageos.volumecontroller.ControllerException)14 Configuration (com.emc.storageos.coordinator.common.Configuration)12 CoordinatorException (com.emc.storageos.coordinator.exceptions.CoordinatorException)12 UnknownHostException (java.net.UnknownHostException)12 Site (com.emc.storageos.coordinator.client.model.Site)11 RetryableCoordinatorException (com.emc.storageos.coordinator.exceptions.RetryableCoordinatorException)11 NetworkDeviceControllerException (com.emc.storageos.networkcontroller.exceptions.NetworkDeviceControllerException)10 CheckPermission (com.emc.storageos.security.authorization.CheckPermission)9 ServiceError (com.emc.storageos.svcs.errorhandling.model.ServiceError)9 BiosCommandResult (com.emc.storageos.volumecontroller.impl.BiosCommandResult)9 ArrayList (java.util.ArrayList)9 POST (javax.ws.rs.POST)9 NetworkSystem (com.emc.storageos.db.client.model.NetworkSystem)8 Path (javax.ws.rs.Path)8 ConfigurationImpl (com.emc.storageos.coordinator.common.impl.ConfigurationImpl)6