Search in sources :

Example 1 with RealLiveTableActor

use of org.nustaq.reallive.impl.actors.RealLiveTableActor in project kontraktor by RuedigerMoeller.

the class EmbeddedRealLive method createTable.

/**
 * WARNING: never create more than one table using the same file. This will
 * result in corrupted data for sure. As actor refs (tables) are thread save,
 * just init a singleton containing all your tables once.
 *
 * @param desc
 * @param dataDir - if null use path from description
 * @return a thread save actor reference to a newly loaded or created table
 */
public IPromise<RealLiveTable> createTable(TableDescription desc, String dataDir) {
    RealLiveTableActor table = Actors.AsActor(RealLiveTableActor.class);
    Supplier<RecordStorage> memFactory;
    if (desc.getFilePath() == null) {
        Log.Info(this, "no file specified. all data in memory " + desc.getName());
        switch(desc.getStorageType()) {
            case CACHED:
                memFactory = () -> new CachedOffHeapStorage(new OffHeapRecordStorage(desc.getKeyLen(), desc.getSizeMB(), desc.getNumEntries()), new HeapRecordStorage());
                break;
            default:
                Log.Error(this, "unknown storage type " + desc.getStorageType() + " default to PERSIST");
            case PERSIST:
                memFactory = () -> new OffHeapRecordStorage(desc.getKeyLen(), desc.getSizeMB(), desc.getNumEntries());
                break;
            case TEMP:
                memFactory = () -> new HeapRecordStorage();
                break;
        }
    } else {
        String bp = dataDir == null ? desc.getFilePath() : dataDir;
        desc.filePath(bp);
        new File(bp).mkdirs();
        String file = bp + "/" + desc.getName() + "_" + desc.getShardNo() + ".bin";
        switch(desc.getStorageType()) {
            case CACHED:
                Log.Info(this, "memory mapping file " + file);
                memFactory = () -> new CachedOffHeapStorage(new OffHeapRecordStorage(file, desc.getKeyLen(), desc.getSizeMB(), desc.getNumEntries()), new HeapRecordStorage());
                break;
            default:
                Log.Error(this, "unknown storage type " + desc.getStorageType() + " default to PERSIST");
            case PERSIST:
                Log.Info(this, "memory mapping file " + file);
                memFactory = () -> new OffHeapRecordStorage(file, desc.getKeyLen(), desc.getSizeMB(), desc.getNumEntries());
                break;
            case TEMP:
                memFactory = () -> new HeapRecordStorage();
                break;
        }
    }
    table.init(memFactory, desc).await(30_000);
    return new Promise(table);
}
Also used : Promise(org.nustaq.kontraktor.Promise) IPromise(org.nustaq.kontraktor.IPromise) OffHeapRecordStorage(org.nustaq.reallive.impl.storage.OffHeapRecordStorage) HeapRecordStorage(org.nustaq.reallive.impl.storage.HeapRecordStorage) OffHeapRecordStorage(org.nustaq.reallive.impl.storage.OffHeapRecordStorage) RecordStorage(org.nustaq.reallive.api.RecordStorage) HeapRecordStorage(org.nustaq.reallive.impl.storage.HeapRecordStorage) File(java.io.File) CachedOffHeapStorage(org.nustaq.reallive.impl.storage.CachedOffHeapStorage) OffHeapRecordStorage(org.nustaq.reallive.impl.storage.OffHeapRecordStorage) RealLiveTableActor(org.nustaq.reallive.impl.actors.RealLiveTableActor)

Example 2 with RealLiveTableActor

use of org.nustaq.reallive.impl.actors.RealLiveTableActor in project kontraktor by RuedigerMoeller.

the class Basic method testActorOutside.

@Test
public void testActorOutside() throws InterruptedException {
    RealLiveTableActor rls = Actors.AsActor(RealLiveTableActor.class);
    rls.init(() -> new OffHeapRecordStorage(32, 500, 500_000), null);
    TA ta = new TA();
    ta.runTest(rls).await();
    ta.stop();
    rls.stop();
}
Also used : RealLiveTableActor(org.nustaq.reallive.impl.actors.RealLiveTableActor) Test(org.junit.Test)

Example 3 with RealLiveTableActor

use of org.nustaq.reallive.impl.actors.RealLiveTableActor in project kontraktor by RuedigerMoeller.

the class Basic method testActorShard.

@Test
public void testActorShard() throws InterruptedException {
    RealLiveTableActor[] rls = new RealLiveTableActor[8];
    for (int i = 0; i < rls.length; i++) {
        rls[i] = Actors.AsActor(RealLiveTableActor.class);
        rls[i].init(() -> new OffHeapRecordStorage(32, 500 / rls.length, 700_000 / rls.length), null);
    }
    ShardFunc sfunc = key -> Math.abs(key.hashCode()) % rls.length;
    TableSharding sharding = new TableSharding(sfunc, rls, null);
    TA ta = Actors.AsActor(TA.class);
    // while( System.currentTimeMillis() != 0)
    {
        ta.runTest(sharding).await(50009);
    }
    ta.stop();
    sharding.stop();
}
Also used : IntStream(java.util.stream.IntStream) org.nustaq.reallive.api(org.nustaq.reallive.api) TableSpaceActor(org.nustaq.reallive.impl.tablespace.TableSpaceActor) Iterator(java.util.Iterator) Test(org.junit.Test) Promise(org.nustaq.kontraktor.Promise) PromiseLatch(org.nustaq.kontraktor.util.PromiseLatch) TableSharding(org.nustaq.reallive.impl.actors.TableSharding) org.nustaq.reallive.impl.storage(org.nustaq.reallive.impl.storage) MapRecord(org.nustaq.reallive.records.MapRecord) IPromise(org.nustaq.kontraktor.IPromise) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) org.nustaq.reallive.impl(org.nustaq.reallive.impl) Map(java.util.Map) Actor(org.nustaq.kontraktor.Actor) Actors(org.nustaq.kontraktor.Actors) ShardFunc(org.nustaq.reallive.impl.actors.ShardFunc) RealLiveTableActor(org.nustaq.reallive.impl.actors.RealLiveTableActor) ShardFunc(org.nustaq.reallive.impl.actors.ShardFunc) TableSharding(org.nustaq.reallive.impl.actors.TableSharding) RealLiveTableActor(org.nustaq.reallive.impl.actors.RealLiveTableActor) Test(org.junit.Test)

Example 4 with RealLiveTableActor

use of org.nustaq.reallive.impl.actors.RealLiveTableActor in project kontraktor by RuedigerMoeller.

the class Basic method testActor.

@Test
public void testActor() throws InterruptedException {
    RealLiveTableActor rls = Actors.AsActor(RealLiveTableActor.class, 100_000);
    rls.init(() -> new OffHeapRecordStorage(32, 500, 500_000), null);
    TA ta = Actors.AsActor(TA.class);
    ta.runTest(rls).await(20_000);
    ta.stop();
    rls.stop();
}
Also used : RealLiveTableActor(org.nustaq.reallive.impl.actors.RealLiveTableActor) Test(org.junit.Test)

Example 5 with RealLiveTableActor

use of org.nustaq.reallive.impl.actors.RealLiveTableActor in project kontraktor by RuedigerMoeller.

the class TableSpaceActor method createOrLoadTable.

@Override
public IPromise<RealLiveTable> createOrLoadTable(TableDescription desc) {
    if (tables.containsKey(desc.getName())) {
        return resolve(tables.get(desc.getName()));
    }
    RealLiveTableActor table = createTableActor(desc);
    tables.put(desc.getName(), table);
    return resolve(table);
}
Also used : RealLiveTableActor(org.nustaq.reallive.impl.actors.RealLiveTableActor)

Aggregations

RealLiveTableActor (org.nustaq.reallive.impl.actors.RealLiveTableActor)5 Test (org.junit.Test)3 IPromise (org.nustaq.kontraktor.IPromise)2 Promise (org.nustaq.kontraktor.Promise)2 File (java.io.File)1 Iterator (java.util.Iterator)1 Map (java.util.Map)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 IntStream (java.util.stream.IntStream)1 Actor (org.nustaq.kontraktor.Actor)1 Actors (org.nustaq.kontraktor.Actors)1 PromiseLatch (org.nustaq.kontraktor.util.PromiseLatch)1 org.nustaq.reallive.api (org.nustaq.reallive.api)1 RecordStorage (org.nustaq.reallive.api.RecordStorage)1 org.nustaq.reallive.impl (org.nustaq.reallive.impl)1 ShardFunc (org.nustaq.reallive.impl.actors.ShardFunc)1 TableSharding (org.nustaq.reallive.impl.actors.TableSharding)1 org.nustaq.reallive.impl.storage (org.nustaq.reallive.impl.storage)1 CachedOffHeapStorage (org.nustaq.reallive.impl.storage.CachedOffHeapStorage)1 HeapRecordStorage (org.nustaq.reallive.impl.storage.HeapRecordStorage)1