Search in sources :

Example 1 with Factory

use of cz.o2.proxima.functional.Factory in project proxima-platform by O2-Czech-Republic.

the class WebsocketReader method asFactory.

@Override
public Factory<?> asFactory() {
    final EntityDescriptor entity = getEntityDescriptor();
    final URI uri = getUri();
    final Map<String, Object> cfg = this.cfg;
    return repo -> new WebsocketReader(entity, uri, cfg);
}
Also used : ObserverUtils.asOnNextContext(cz.o2.proxima.direct.commitlog.ObserverUtils.asOnNextContext) ServerHandshake(org.java_websocket.handshake.ServerHandshake) Iterables(com.google.common.collect.Iterables) Arrays(java.util.Arrays) WebSocketClient(org.java_websocket.client.WebSocketClient) Partition(cz.o2.proxima.storage.Partition) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) StreamElement(cz.o2.proxima.storage.StreamElement) UnaryFunction(cz.o2.proxima.functional.UnaryFunction) Map(java.util.Map) URI(java.net.URI) CommitLogReader(cz.o2.proxima.direct.commitlog.CommitLogReader) AbstractStorage(cz.o2.proxima.storage.AbstractStorage) Collection(java.util.Collection) AttributeDescriptor(cz.o2.proxima.repository.AttributeDescriptor) CommitLogObserver(cz.o2.proxima.direct.commitlog.CommitLogObserver) ObserveHandle(cz.o2.proxima.direct.commitlog.ObserveHandle) UUID(java.util.UUID) StandardCharsets(java.nio.charset.StandardCharsets) Offset(cz.o2.proxima.direct.commitlog.Offset) Objects(java.util.Objects) Consumer(java.util.function.Consumer) List(java.util.List) OnNextContext(cz.o2.proxima.direct.commitlog.CommitLogObserver.OnNextContext) Optional(java.util.Optional) Position(cz.o2.proxima.storage.commitlog.Position) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) URI(java.net.URI)

Example 2 with Factory

use of cz.o2.proxima.functional.Factory in project proxima-platform by O2-Czech-Republic.

the class OpenTsdbWriter method asFactory.

@Override
public Factory<?> asFactory() {
    final EntityDescriptor entity = getEntityDescriptor();
    final URI uri = getUri();
    final Map<String, Object> cfg = getCfg();
    return repo -> new OpenTsdbWriter(entity, uri, cfg);
}
Also used : HttpWriter(cz.o2.proxima.direct.http.HttpWriter) Map(java.util.Map) Evolving(cz.o2.proxima.annotations.Evolving) ConnFactory(cz.o2.proxima.direct.http.ConnFactory) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) URI(java.net.URI) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) URI(java.net.URI)

Example 3 with Factory

use of cz.o2.proxima.functional.Factory in project proxima-platform by O2-Czech-Republic.

the class HBaseLogReader method asFactory.

@Override
public Factory<?> asFactory() {
    final URI uri = getUri();
    final EntityDescriptor entity = this.entity;
    final cz.o2.proxima.functional.Factory<ExecutorService> executorFactory = this.executorFactory;
    final byte[] serializedConf = this.serializedConf;
    return repo -> new HBaseLogReader(uri, deserialize(serializedConf, new Configuration()), entity, executorFactory);
}
Also used : BatchLogReader(cz.o2.proxima.direct.batch.BatchLogReader) Partition(cz.o2.proxima.storage.Partition) Result(org.apache.hadoop.hbase.client.Result) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) FilterList(org.apache.hadoop.hbase.filter.FilterList) TerminationContext(cz.o2.proxima.direct.batch.TerminationContext) BatchLogObservers(cz.o2.proxima.direct.batch.BatchLogObservers) ArrayList(java.util.ArrayList) ColumnPrefixFilter(org.apache.hadoop.hbase.filter.ColumnPrefixFilter) ExceptionUtils(cz.o2.proxima.util.ExceptionUtils) StreamElement(cz.o2.proxima.storage.StreamElement) BinaryComparator(org.apache.hadoop.hbase.filter.BinaryComparator) Configuration(org.apache.hadoop.conf.Configuration) URI(java.net.URI) CompareFilter(org.apache.hadoop.hbase.filter.CompareFilter) ObserveHandle(cz.o2.proxima.direct.batch.ObserveHandle) ExecutorService(java.util.concurrent.ExecutorService) Cell(org.apache.hadoop.hbase.Cell) AttributeDescriptor(cz.o2.proxima.repository.AttributeDescriptor) BatchLogObserver(cz.o2.proxima.direct.batch.BatchLogObserver) IOException(java.io.IOException) StandardCharsets(java.nio.charset.StandardCharsets) Scan(org.apache.hadoop.hbase.client.Scan) CellScanner(org.apache.hadoop.hbase.CellScanner) List(java.util.List) Slf4j(lombok.extern.slf4j.Slf4j) ResultScanner(org.apache.hadoop.hbase.client.ResultScanner) QualifierFilter(org.apache.hadoop.hbase.filter.QualifierFilter) Filter(org.apache.hadoop.hbase.filter.Filter) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) Configuration(org.apache.hadoop.conf.Configuration) ExecutorService(java.util.concurrent.ExecutorService) URI(java.net.URI)

Example 4 with Factory

use of cz.o2.proxima.functional.Factory in project proxima-platform by O2-Czech-Republic.

the class TransformingCqlFactoryTest method setup.

@Before
public void setup() throws URISyntaxException {
    statements.clear();
    factory = new TransformingCqlFactory<String>(i -> new String(i.getValue()), Arrays.asList("a", "b"), Arrays.asList(p -> "a_" + p.getFirst() + "_" + p.getSecond(), p -> "b_" + p.getFirst() + "_" + p.getSecond())) {

        @Override
        protected String createInsertStatement(StreamElement ingest) {
            String s = super.createInsertStatement(ingest);
            statements.add(s);
            return s;
        }
    };
    factory.setup(entity, new URI("cassandra://wherever/my_table/"), StringConverter.getDefault());
}
Also used : Arrays(java.util.Arrays) Repository(cz.o2.proxima.repository.Repository) URISyntaxException(java.net.URISyntaxException) AttributeDescriptor(cz.o2.proxima.repository.AttributeDescriptor) EntityDescriptor(cz.o2.proxima.repository.EntityDescriptor) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) Mockito.verify(org.mockito.Mockito.verify) ArrayList(java.util.ArrayList) PreparedStatement(com.datastax.driver.core.PreparedStatement) AttributeDescriptorBase(cz.o2.proxima.repository.AttributeDescriptorBase) ExceptionUtils(cz.o2.proxima.util.ExceptionUtils) List(java.util.List) ConfigRepository(cz.o2.proxima.repository.ConfigRepository) BoundStatement(com.datastax.driver.core.BoundStatement) StreamElement(cz.o2.proxima.storage.StreamElement) Session(com.datastax.driver.core.Session) ConfigFactory(com.typesafe.config.ConfigFactory) Optional(java.util.Optional) URI(java.net.URI) Assert(org.junit.Assert) Mockito.any(org.mockito.Mockito.any) Mockito.mock(org.mockito.Mockito.mock) Before(org.junit.Before) StreamElement(cz.o2.proxima.storage.StreamElement) URI(java.net.URI) Before(org.junit.Before)

Example 5 with Factory

use of cz.o2.proxima.functional.Factory in project proxima-platform by O2-Czech-Republic.

the class GCloudLogReader method asFactory.

@Override
public Factory<?> asFactory() {
    final GCloudStorageAccessor accessor = (GCloudStorageAccessor) getAccessor();
    final Context context = getContext();
    return repo -> new GCloudLogReader(accessor, context);
}
Also used : Context(cz.o2.proxima.direct.core.Context) Context(cz.o2.proxima.direct.core.Context) Blob(com.google.cloud.storage.Blob) BlobLogReader(cz.o2.proxima.direct.blob.BlobLogReader) Slf4j(lombok.extern.slf4j.Slf4j) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) BatchLogReader(cz.o2.proxima.direct.batch.BatchLogReader) StorageException(com.google.cloud.storage.StorageException) GCloudBlob(cz.o2.proxima.direct.gcloud.storage.GCloudBlobPath.GCloudBlob) BlobPath(cz.o2.proxima.direct.blob.BlobPath)

Aggregations

StreamElement (cz.o2.proxima.storage.StreamElement)19 Test (org.junit.Test)19 Slf4j (lombok.extern.slf4j.Slf4j)14 EntityDescriptor (cz.o2.proxima.repository.EntityDescriptor)12 Context (cz.o2.proxima.direct.core.Context)11 AttributeDescriptor (cz.o2.proxima.repository.AttributeDescriptor)11 URI (java.net.URI)11 List (java.util.List)11 WatermarkIdlePolicyFactory (cz.o2.proxima.time.WatermarkIdlePolicyFactory)10 Map (java.util.Map)10 Optional (java.util.Optional)10 HashMap (java.util.HashMap)9 CommitLogReader (cz.o2.proxima.direct.commitlog.CommitLogReader)8 Partition (cz.o2.proxima.storage.Partition)8 Position (cz.o2.proxima.storage.commitlog.Position)8 ArrayList (java.util.ArrayList)8 Collections (java.util.Collections)8 Objects (java.util.Objects)8 AtomicReference (java.util.concurrent.atomic.AtomicReference)8 BatchLogReader (cz.o2.proxima.direct.batch.BatchLogReader)7