Search in sources :

Example 16 with LockService

use of com.palantir.lock.LockService in project atlasdb by palantir.

the class SnapshotTransactionTest method testTransactionAtomicity.

@Test
public void testTransactionAtomicity() throws Exception {
    // This test runs multiple transactions in parallel, with KeyValueService.put calls throwing
    // a RuntimeException from time to time and hanging other times. which effectively kills the
    // thread. We ensure that every transaction either adds 5 rows to the table or adds 0 rows
    // by checking at the end that the number of rows is a multiple of 5.
    final TableReference tableRef = TABLE;
    Random random = new Random(1);
    final UnstableKeyValueService unstableKvs = new UnstableKeyValueService(keyValueService, random);
    final TestTransactionManager unstableTransactionManager = new TestTransactionManagerImpl(unstableKvs, timestampService, lockClient, lockService, transactionService, conflictDetectionManager, sweepStrategyManager, sweepQueue);
    ScheduledExecutorService service = PTExecutors.newScheduledThreadPool(20);
    for (int i = 0; i < 30; i++) {
        final int threadNumber = i;
        service.schedule((Callable<Void>) () -> {
            if (threadNumber == 10) {
                unstableKvs.setRandomlyThrow(true);
            }
            if (threadNumber == 20) {
                unstableKvs.setRandomlyHang(true);
            }
            Transaction transaction = unstableTransactionManager.createNewTransaction();
            BatchingVisitable<RowResult<byte[]>> results = transaction.getRange(tableRef, RangeRequest.builder().build());
            final MutableInt nextIndex = new MutableInt(0);
            results.batchAccept(1, AbortingVisitors.batching((AbortingVisitor<RowResult<byte[]>, Exception>) row -> {
                byte[] dataBytes = row.getColumns().get(PtBytes.toBytes("data"));
                BigInteger dataValue = new BigInteger(dataBytes);
                nextIndex.setValue(Math.max(nextIndex.toInteger(), dataValue.intValue() + 1));
                return true;
            }));
            // rows to the table.
            for (int j = 0; j < 5; j++) {
                int rowNumber = nextIndex.toInteger() + j;
                Cell cell = Cell.create(PtBytes.toBytes("row" + rowNumber), PtBytes.toBytes("data"));
                transaction.put(tableRef, ImmutableMap.of(cell, BigInteger.valueOf(rowNumber).toByteArray()));
                Thread.yield();
            }
            transaction.commit();
            return null;
        }, i * 20, TimeUnit.MILLISECONDS);
    }
    service.shutdown();
    service.awaitTermination(1, TimeUnit.SECONDS);
    // Verify each table has a number of rows that's a multiple of 5
    Transaction verifyTransaction = txManager.createNewTransaction();
    BatchingVisitable<RowResult<byte[]>> results = verifyTransaction.getRange(tableRef, RangeRequest.builder().build());
    final MutableInt numRows = new MutableInt(0);
    results.batchAccept(1, AbortingVisitors.batching((AbortingVisitor<RowResult<byte[]>, Exception>) row -> {
        numRows.increment();
        return true;
    }));
    Assert.assertEquals(0, numRows.toInteger() % 5);
}
Also used : ColumnMetadataDescription(com.palantir.atlasdb.table.description.ColumnMetadataDescription) Matchers.not(org.hamcrest.Matchers.not) Future(java.util.concurrent.Future) Pair(org.apache.commons.lang3.tuple.Pair) MutableLong(org.apache.commons.lang3.mutable.MutableLong) Map(java.util.Map) TableReference(com.palantir.atlasdb.keyvalue.api.TableReference) BigInteger(java.math.BigInteger) AtlasDbConstants(com.palantir.atlasdb.AtlasDbConstants) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) Cell(com.palantir.atlasdb.keyvalue.api.Cell) Set(java.util.Set) LockMode(com.palantir.lock.LockMode) Executors(java.util.concurrent.Executors) Matchers.any(org.mockito.Matchers.any) Transaction(com.palantir.atlasdb.transaction.api.Transaction) Matchers.is(org.hamcrest.Matchers.is) TransactionLockTimeoutNonRetriableException(com.palantir.atlasdb.transaction.api.TransactionLockTimeoutNonRetriableException) Matchers.containsString(org.hamcrest.Matchers.containsString) NoOpCleaner(com.palantir.atlasdb.cleaner.NoOpCleaner) Mockito.mock(org.mockito.Mockito.mock) Joiner(com.google.common.base.Joiner) ColumnRangeSelection(com.palantir.atlasdb.keyvalue.api.ColumnRangeSelection) Iterables(com.google.common.collect.Iterables) ConflictHandler(com.palantir.atlasdb.transaction.api.ConflictHandler) Expectations(org.jmock.Expectations) CachePriority(com.palantir.atlasdb.protos.generated.TableMetadataPersistence.CachePriority) TransactionFailedRetriableException(com.palantir.atlasdb.transaction.api.TransactionFailedRetriableException) Callable(java.util.concurrent.Callable) SimpleTimeDuration(com.palantir.lock.SimpleTimeDuration) AtlasDbTestCase(com.palantir.atlasdb.AtlasDbTestCase) PtBytes(com.palantir.atlasdb.encoding.PtBytes) Mockito.verifyZeroInteractions(org.mockito.Mockito.verifyZeroInteractions) Multimaps(com.google.common.collect.Multimaps) LockAwareTransactionTask(com.palantir.atlasdb.transaction.api.LockAwareTransactionTask) Lists(com.google.common.collect.Lists) LegacyTimelockService(com.palantir.lock.impl.LegacyTimelockService) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) AbortingVisitors(com.palantir.common.base.AbortingVisitors) LockRequest(com.palantir.lock.LockRequest) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) Before(org.junit.Before) LockCollections(com.palantir.lock.LockCollections) Assert.assertTrue(org.junit.Assert.assertTrue) Throwables(com.google.common.base.Throwables) Test(org.junit.Test) AtlasRowLockDescriptor(com.palantir.lock.AtlasRowLockDescriptor) Mockery(org.jmock.Mockery) ExecutionException(java.util.concurrent.ExecutionException) RowResult(com.palantir.atlasdb.keyvalue.api.RowResult) AtomicLong(java.util.concurrent.atomic.AtomicLong) TimeDuration(com.palantir.lock.TimeDuration) Matchers.hasItem(org.hamcrest.Matchers.hasItem) LockService(com.palantir.lock.LockService) LockClient(com.palantir.lock.LockClient) Assert(org.junit.Assert) Assert.assertEquals(org.junit.Assert.assertEquals) MutableInt(org.apache.commons.lang3.mutable.MutableInt) BatchColumnRangeSelection(com.palantir.atlasdb.keyvalue.api.BatchColumnRangeSelection) Sequence(org.jmock.Sequence) Random(java.util.Random) BatchingVisitable(com.palantir.common.base.BatchingVisitable) CompletionService(java.util.concurrent.CompletionService) Assert.assertThat(org.junit.Assert.assertThat) PTExecutors(com.palantir.common.concurrent.PTExecutors) Mockito.verifyNoMoreInteractions(org.mockito.Mockito.verifyNoMoreInteractions) Matchers.eq(org.mockito.Matchers.eq) Assert.fail(org.junit.Assert.fail) TrackingKeyValueService(com.palantir.atlasdb.keyvalue.impl.TrackingKeyValueService) LockDescriptor(com.palantir.lock.LockDescriptor) TableMetadata(com.palantir.atlasdb.table.description.TableMetadata) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) TransactionReadSentinelBehavior(com.palantir.atlasdb.transaction.api.TransactionReadSentinelBehavior) Collection(java.util.Collection) SweepStrategy(com.palantir.atlasdb.protos.generated.TableMetadataPersistence.SweepStrategy) Collectors(java.util.stream.Collectors) ColumnSelection(com.palantir.atlasdb.keyvalue.api.ColumnSelection) RangeRequest(com.palantir.atlasdb.keyvalue.api.RangeRequest) TransactionConflictException(com.palantir.atlasdb.transaction.api.TransactionConflictException) List(java.util.List) MultiDelegateProxy(com.palantir.common.proxy.MultiDelegateProxy) Optional(java.util.Optional) ForwardingKeyValueService(com.palantir.atlasdb.keyvalue.impl.ForwardingKeyValueService) SortedMap(java.util.SortedMap) ExecutorCompletionService(java.util.concurrent.ExecutorCompletionService) HeldLocksToken(com.palantir.lock.HeldLocksToken) Supplier(com.google.common.base.Supplier) Multimap(com.google.common.collect.Multimap) LockRefreshToken(com.palantir.lock.LockRefreshToken) AtlasDbConstraintCheckingMode(com.palantir.atlasdb.transaction.api.AtlasDbConstraintCheckingMode) ImmutableList(com.google.common.collect.ImmutableList) Suppliers(com.google.common.base.Suppliers) WriteInfo(com.palantir.atlasdb.sweep.queue.WriteInfo) ExecutorService(java.util.concurrent.ExecutorService) Matchers.hasEntry(org.hamcrest.Matchers.hasEntry) EncodingUtils(com.palantir.atlasdb.ptobject.EncodingUtils) PreCommitCondition(com.palantir.atlasdb.transaction.api.PreCommitCondition) Matchers(org.hamcrest.Matchers) TimestampCache(com.palantir.atlasdb.cache.TimestampCache) Mockito.when(org.mockito.Mockito.when) Mockito.verify(org.mockito.Mockito.verify) TimeUnit(java.util.concurrent.TimeUnit) Ignore(org.junit.Ignore) KeyValueService(com.palantir.atlasdb.keyvalue.api.KeyValueService) TransactionFailedNonRetriableException(com.palantir.atlasdb.transaction.api.TransactionFailedNonRetriableException) NameMetadataDescription(com.palantir.atlasdb.table.description.NameMetadataDescription) Collections(java.util.Collections) AbortingVisitor(com.palantir.common.base.AbortingVisitor) ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) AbortingVisitor(com.palantir.common.base.AbortingVisitor) TransactionLockTimeoutNonRetriableException(com.palantir.atlasdb.transaction.api.TransactionLockTimeoutNonRetriableException) TransactionFailedRetriableException(com.palantir.atlasdb.transaction.api.TransactionFailedRetriableException) ExecutionException(java.util.concurrent.ExecutionException) TransactionConflictException(com.palantir.atlasdb.transaction.api.TransactionConflictException) TransactionFailedNonRetriableException(com.palantir.atlasdb.transaction.api.TransactionFailedNonRetriableException) BatchingVisitable(com.palantir.common.base.BatchingVisitable) RowResult(com.palantir.atlasdb.keyvalue.api.RowResult) TableReference(com.palantir.atlasdb.keyvalue.api.TableReference) Random(java.util.Random) Transaction(com.palantir.atlasdb.transaction.api.Transaction) MutableInt(org.apache.commons.lang3.mutable.MutableInt) BigInteger(java.math.BigInteger) Cell(com.palantir.atlasdb.keyvalue.api.Cell) Test(org.junit.Test)

Example 17 with LockService

use of com.palantir.lock.LockService in project atlasdb by palantir.

the class TransactionManagerTest method shouldNotConflictIfImmutableTimestampLockExpiresEvenIfNoWritesOnNonThoroughSweptTable.

@Test
public void shouldNotConflictIfImmutableTimestampLockExpiresEvenIfNoWritesOnNonThoroughSweptTable() {
    TimelockService timelock = mock(TimelockService.class);
    LockService mockLockService = mock(LockService.class);
    TransactionManager txnManagerWithMocks = new SerializableTransactionManager(keyValueService, timelock, mockLockService, transactionService, () -> AtlasDbConstraintCheckingMode.FULL_CONSTRAINT_CHECKING_THROWS_EXCEPTIONS, conflictDetectionManager, sweepStrategyManager, NoOpCleaner.INSTANCE, TimestampTrackerImpl.createNoOpTracker(), () -> AtlasDbConstants.DEFAULT_TIMESTAMP_CACHE_SIZE, false, () -> AtlasDbConstants.DEFAULT_TRANSACTION_LOCK_ACQUIRE_TIMEOUT_MS, AbstractTransactionTest.GET_RANGES_THREAD_POOL_SIZE, AbstractTransactionTest.DEFAULT_GET_RANGES_CONCURRENCY, MultiTableSweepQueueWriter.NO_OP);
    when(timelock.getFreshTimestamp()).thenReturn(1L);
    when(timelock.lockImmutableTimestamp(any())).thenReturn(LockImmutableTimestampResponse.of(2L, LockToken.of(UUID.randomUUID())));
    txnManagerWithMocks.runTaskThrowOnConflict(txn -> {
        get(txn, TEST_TABLE, "row1", "col1");
        return null;
    });
}
Also used : LockService(com.palantir.lock.LockService) TimelockService(com.palantir.lock.v2.TimelockService) TransactionManager(com.palantir.atlasdb.transaction.api.TransactionManager) Test(org.junit.Test)

Example 18 with LockService

use of com.palantir.lock.LockService in project atlasdb by palantir.

the class TimeLockAgent method createInvalidatingTimeLockServices.

/**
 * Creates timestamp and lock services for the given client. It is expected that for each client there should
 * only be (up to) one active timestamp service, and one active lock service at any time.
 * @param client Client namespace to create the services for
 * @return Invalidating timestamp and lock services
 */
private TimeLockServices createInvalidatingTimeLockServices(String client) {
    List<String> uris = install.cluster().clusterMembers();
    ImmutableLeaderConfig leaderConfig = ImmutableLeaderConfig.builder().addLeaders(uris.toArray(new String[uris.size()])).localServer(install.cluster().localServer()).sslConfiguration(PaxosRemotingUtils.getSslConfigurationOptional(install)).quorumSize(PaxosRemotingUtils.getQuorumSize(uris)).build();
    Supplier<ManagedTimestampService> rawTimestampServiceSupplier = timestampCreator.createTimestampService(client, leaderConfig);
    Supplier<LockService> rawLockServiceSupplier = lockCreator::createThreadPoolingLockService;
    LockLog.setSlowLockThresholdMillis(JavaSuppliers.compose(TimeLockRuntimeConfiguration::slowLockLogTriggerMillis, runtime));
    return timelockCreator.createTimeLockServices(client, rawTimestampServiceSupplier, rawLockServiceSupplier);
}
Also used : LockService(com.palantir.lock.LockService) ManagedTimestampService(com.palantir.atlasdb.timelock.paxos.ManagedTimestampService) ImmutableLeaderConfig(com.palantir.atlasdb.config.ImmutableLeaderConfig)

Example 19 with LockService

use of com.palantir.lock.LockService in project atlasdb by palantir.

the class LegacyTimeLockServicesCreator method createTimeLockServices.

@Override
public TimeLockServices createTimeLockServices(String client, Supplier<ManagedTimestampService> rawTimestampServiceSupplier, Supplier<LockService> rawLockServiceSupplier) {
    log.info("Creating legacy timelock service for client {}", client);
    ManagedTimestampService timestampService = instrumentInLeadershipProxy(ManagedTimestampService.class, rawTimestampServiceSupplier, client);
    LockService lockService = instrumentInLeadershipProxy(LockService.class, rawLockServiceSupplier, client);
    // The underlying primitives are already wrapped in a leadership proxy (and must be).
    // Wrapping this means that we will make 2 paxos checks per request, which is silly.
    TimelockService legacyTimelockService = instrument(TimelockService.class, createRawLegacyTimelockService(timestampService, lockService), client);
    return TimeLockServices.create(timestampService, lockService, AsyncOrLegacyTimelockService.createFromLegacyTimelock(legacyTimelockService), timestampService);
}
Also used : LockService(com.palantir.lock.LockService) LegacyTimelockService(com.palantir.lock.impl.LegacyTimelockService) TimelockService(com.palantir.lock.v2.TimelockService) AsyncOrLegacyTimelockService(com.palantir.atlasdb.timelock.util.AsyncOrLegacyTimelockService) ManagedTimestampService(com.palantir.atlasdb.timelock.paxos.ManagedTimestampService)

Example 20 with LockService

use of com.palantir.lock.LockService in project atlasdb by palantir.

the class PaxosTimeLockServerIntegrationTest method lockServiceShouldNotAllowUsToRefreshLocksFromDifferentNamespaces.

@Test
public void lockServiceShouldNotAllowUsToRefreshLocksFromDifferentNamespaces() throws InterruptedException {
    LockService lockService1 = getLockService(CLIENT_1);
    LockService lockService2 = getLockService(CLIENT_2);
    LockRefreshToken token = lockService1.lock(LOCK_CLIENT_NAME, com.palantir.lock.LockRequest.builder(LOCK_MAP).doNotBlock().build());
    assertThat(token).isNotNull();
    assertThat(lockService1.refreshLockRefreshTokens(ImmutableList.of(token))).isNotEmpty();
    assertThat(lockService2.refreshLockRefreshTokens(ImmutableList.of(token))).isEmpty();
    lockService1.unlock(token);
}
Also used : LockService(com.palantir.lock.LockService) LockRefreshToken(com.palantir.lock.LockRefreshToken) Test(org.junit.Test)

Aggregations

LockService (com.palantir.lock.LockService)22 Test (org.junit.Test)11 LockRefreshToken (com.palantir.lock.LockRefreshToken)7 LegacyTimelockService (com.palantir.lock.impl.LegacyTimelockService)7 TimestampService (com.palantir.timestamp.TimestampService)6 KvsBackedPersistentLockService (com.palantir.atlasdb.persistentlock.KvsBackedPersistentLockService)5 NoOpPersistentLockService (com.palantir.atlasdb.persistentlock.NoOpPersistentLockService)5 PersistentLockService (com.palantir.atlasdb.persistentlock.PersistentLockService)5 LockRefreshingLockService (com.palantir.lock.client.LockRefreshingLockService)5 TimelockService (com.palantir.lock.v2.TimelockService)5 LockClient (com.palantir.lock.LockClient)4 LockRequest (com.palantir.lock.LockRequest)4 KeyValueService (com.palantir.atlasdb.keyvalue.api.KeyValueService)3 TransactionManager (com.palantir.atlasdb.transaction.api.TransactionManager)3 Cleaner (com.palantir.atlasdb.cleaner.Cleaner)2 NoOpCleaner (com.palantir.atlasdb.cleaner.NoOpCleaner)2 Cell (com.palantir.atlasdb.keyvalue.api.Cell)2 ForwardingKeyValueService (com.palantir.atlasdb.keyvalue.impl.ForwardingKeyValueService)2 TrackingKeyValueService (com.palantir.atlasdb.keyvalue.impl.TrackingKeyValueService)2 ManagedTimestampService (com.palantir.atlasdb.timelock.paxos.ManagedTimestampService)2