Search in sources :

Example 41 with MongoClient

use of com.mongodb.client.MongoClient in project logging-log4j2 by apache.

the class MongoDb4Test method test.

@Test
public void test() {
    final Logger logger = LogManager.getLogger();
    logger.info("Hello log");
    try (final MongoClient mongoClient = mongoDbTestRule.getMongoClient()) {
        final MongoDatabase database = mongoClient.getDatabase("testDb");
        Assert.assertNotNull(database);
        final MongoCollection<Document> collection = database.getCollection("testCollection");
        Assert.assertNotNull(collection);
        final Document first = collection.find().first();
        Assert.assertNotNull(first);
        Assert.assertEquals(first.toJson(), "Hello log", first.getString("message"));
        Assert.assertEquals(first.toJson(), "INFO", first.getString("level"));
    }
}
Also used : MongoClient(com.mongodb.client.MongoClient) Logger(org.apache.logging.log4j.Logger) Document(org.bson.Document) MongoDatabase(com.mongodb.client.MongoDatabase) Test(org.junit.Test)

Example 42 with MongoClient

use of com.mongodb.client.MongoClient in project gora by apache.

the class TestMongoStoreMetadataAnalyzer method setUp.

@Override
@Before
public void setUp() throws Exception {
    super.setUp();
    Properties prop = DataStoreFactory.createProps();
    Configuration conf = testDriver.getConfiguration();
    MongoStoreParameters parameters = MongoStoreParameters.load(prop, conf);
    MongoClient mongoClient = MongoStore.getClient(parameters);
    mongoDatabase = mongoClient.getDatabase(parameters.getDbname());
    storeMetadataAnalyzer = DataStoreMetadataFactory.createAnalyzer(conf);
}
Also used : MongoClient(com.mongodb.client.MongoClient) Configuration(org.apache.hadoop.conf.Configuration) Before(org.junit.Before)

Example 43 with MongoClient

use of com.mongodb.client.MongoClient in project drill by apache.

the class MongoGroupScan method getScanStats.

@Override
public ScanStats getScanStats() {
    try {
        MongoClient client = storagePlugin.getClient();
        MongoDatabase db = client.getDatabase(scanSpec.getDbName());
        MongoCollection<Document> collection = db.getCollection(scanSpec.getCollectionName());
        long recordCount = collection.estimatedDocumentCount();
        float approxDiskCost = 0;
        if (recordCount != 0) {
            // toJson should use client's codec, otherwise toJson could fail on
            // some types not known to DocumentCodec, e.g. DBRef.
            DocumentCodec codec = new DocumentCodec(db.getCodecRegistry(), new BsonTypeClassMap());
            String json = collection.find().first().toJson(codec);
            approxDiskCost = json.getBytes().length * recordCount;
        }
        return new ScanStats(GroupScanProperty.ESTIMATED_TOTAL_COST, recordCount, 1, approxDiskCost);
    } catch (Exception e) {
        throw new DrillRuntimeException(e.getMessage(), e);
    }
}
Also used : MongoClient(com.mongodb.client.MongoClient) DocumentCodec(org.bson.codecs.DocumentCodec) BsonTypeClassMap(org.bson.codecs.BsonTypeClassMap) Document(org.bson.Document) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) MongoDatabase(com.mongodb.client.MongoDatabase) ScanStats(org.apache.drill.exec.physical.base.ScanStats)

Example 44 with MongoClient

use of com.mongodb.client.MongoClient in project drill by apache.

the class MongoStoragePlugin method getClient.

public synchronized MongoClient getClient(List<ServerAddress> addresses) {
    // Take the first replica from the replicated servers
    ServerAddress serverAddress = addresses.get(0);
    MongoCredential credential = clientURI.getCredential();
    String userName = credential == null ? null : credential.getUserName();
    MongoCnxnKey key = new MongoCnxnKey(serverAddress, userName);
    try {
        return addressClientMap.get(key, () -> {
            MongoClientSettings.Builder settings;
            if (clientURI.isSrvProtocol()) {
                settings = MongoClientSettings.builder().applyConnectionString(clientURI);
                logger.info("Created srv protocol connection to {}.", key);
            } else {
                settings = MongoClientSettings.builder().applyToClusterSettings(builder -> builder.hosts(addresses));
                if (credential != null) {
                    settings.credential(credential);
                }
                logger.info("Created connection to {}.", key);
            }
            // include this created
            logger.info("Number of open connections {}.", addressClientMap.size() + 1);
            return MongoClients.create(settings.build());
        });
    } catch (ExecutionException e) {
        throw new DrillRuntimeException(e);
    }
}
Also used : Convention(org.apache.calcite.plan.Convention) MongoClient(com.mongodb.client.MongoClient) MongoCredential(com.mongodb.MongoCredential) StoragePluginRulesSupplier(org.apache.drill.exec.store.StoragePluginRulesSupplier) LoggerFactory(org.slf4j.LoggerFactory) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException) AbstractGroupScan(org.apache.drill.exec.physical.base.AbstractGroupScan) DrillbitContext(org.apache.drill.exec.server.DrillbitContext) UsernamePasswordCredentials(org.apache.drill.exec.store.security.UsernamePasswordCredentials) OptimizerRulesContext(org.apache.drill.exec.ops.OptimizerRulesContext) Cache(org.apache.drill.shaded.guava.com.google.common.cache.Cache) TypeReference(com.fasterxml.jackson.core.type.TypeReference) AbstractStoragePlugin(org.apache.drill.exec.store.AbstractStoragePlugin) ServerAddress(com.mongodb.ServerAddress) RemovalListener(org.apache.drill.shaded.guava.com.google.common.cache.RemovalListener) SchemaPlus(org.apache.calcite.schema.SchemaPlus) PlainCredentialsProvider(org.apache.drill.common.logical.security.PlainCredentialsProvider) Logger(org.slf4j.Logger) MongoClients(com.mongodb.client.MongoClients) PluginRulesProviderImpl(org.apache.drill.exec.store.PluginRulesProviderImpl) CredentialsProvider(org.apache.drill.common.logical.security.CredentialsProvider) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Set(java.util.Set) IOException(java.io.IOException) MongoPluginImplementor(org.apache.drill.exec.store.mongo.plan.MongoPluginImplementor) PluginRel(org.apache.drill.exec.store.plan.rel.PluginRel) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) RelOptRule(org.apache.calcite.plan.RelOptRule) SchemaConfig(org.apache.drill.exec.store.SchemaConfig) URLEncoder(java.net.URLEncoder) List(java.util.List) ConnectionString(com.mongodb.ConnectionString) ImmutableMap(org.apache.drill.shaded.guava.com.google.common.collect.ImmutableMap) Lists(org.apache.drill.shaded.guava.com.google.common.collect.Lists) HadoopCredentialsProvider(org.apache.drill.exec.store.security.HadoopCredentialsProvider) RemovalNotification(org.apache.drill.shaded.guava.com.google.common.cache.RemovalNotification) JSONOptions(org.apache.drill.common.JSONOptions) CacheBuilder(org.apache.drill.shaded.guava.com.google.common.cache.CacheBuilder) MongoClientSettings(com.mongodb.MongoClientSettings) PlannerPhase(org.apache.drill.exec.planner.PlannerPhase) MongoSchemaFactory(org.apache.drill.exec.store.mongo.schema.MongoSchemaFactory) Collections(java.util.Collections) MongoCredential(com.mongodb.MongoCredential) ServerAddress(com.mongodb.ServerAddress) ConnectionString(com.mongodb.ConnectionString) MongoClientSettings(com.mongodb.MongoClientSettings) ExecutionException(java.util.concurrent.ExecutionException) DrillRuntimeException(org.apache.drill.common.exceptions.DrillRuntimeException)

Example 45 with MongoClient

use of com.mongodb.client.MongoClient in project drill by apache.

the class MongoRecordReader method init.

private void init(BaseMongoSubScanSpec subScanSpec) {
    List<String> hosts = subScanSpec.getHosts();
    List<ServerAddress> addresses = Lists.newArrayList();
    for (String host : hosts) {
        addresses.add(new ServerAddress(host));
    }
    MongoClient client = plugin.getClient(addresses);
    MongoDatabase db = client.getDatabase(subScanSpec.getDbName());
    this.unionEnabled = fragmentContext.getOptions().getBoolean(ExecConstants.ENABLE_UNION_TYPE_KEY);
    collection = db.getCollection(subScanSpec.getCollectionName(), BsonDocument.class);
}
Also used : MongoClient(com.mongodb.client.MongoClient) BsonDocument(org.bson.BsonDocument) ServerAddress(com.mongodb.ServerAddress) MongoDatabase(com.mongodb.client.MongoDatabase)

Aggregations

MongoClient (com.mongodb.client.MongoClient)45 Document (org.bson.Document)22 MongoDatabase (com.mongodb.client.MongoDatabase)19 Test (org.junit.Test)10 Map (java.util.Map)9 MongoClientSettings (com.mongodb.MongoClientSettings)8 ServerAddress (com.mongodb.ServerAddress)8 HashMap (java.util.HashMap)7 ConnectionString (com.mongodb.ConnectionString)6 BsonDocument (org.bson.BsonDocument)6 BsonString (org.bson.BsonString)6 Test (org.junit.jupiter.api.Test)6 List (java.util.List)5 SecureRandom (java.security.SecureRandom)4 Set (java.util.Set)4 TimeUnit (java.util.concurrent.TimeUnit)4 BsonValue (org.bson.BsonValue)4 ClientEncryptionSettings (com.mongodb.ClientEncryptionSettings)3 ClientSessionOptions (com.mongodb.ClientSessionOptions)3 ClusterFixture.getMultiMongosConnectionString (com.mongodb.ClusterFixture.getMultiMongosConnectionString)3