Search in sources :

Example 6 with FluoConfiguration

use of org.apache.fluo.api.config.FluoConfiguration in project incubator-rya by apache.

the class FluoClientFactory method getFluoClient.

/**
 * Creates a FluoClient
 * @param appName - name of Fluo application
 * @param tableName - name of Fluo table
 * @param conf - AccumuloConfiguration (must contain Accumulo User, Accumulo Instance, Accumulo Password, and Accumulo Zookeepers)
 * @return FluoClient for connecting to Fluo
 */
public static FluoClient getFluoClient(final String appName, final Optional<String> tableName, final AccumuloRdfConfiguration conf) {
    final FluoConfiguration fluoConfig = new FluoConfiguration();
    fluoConfig.setAccumuloInstance(conf.getAccumuloInstance());
    fluoConfig.setAccumuloUser(conf.getAccumuloUser());
    fluoConfig.setAccumuloPassword(conf.getAccumuloPassword());
    fluoConfig.setInstanceZookeepers(conf.getAccumuloZookeepers() + "/fluo");
    fluoConfig.setAccumuloZookeepers(conf.getAccumuloZookeepers());
    fluoConfig.setApplicationName(appName);
    if (tableName.isPresent()) {
        fluoConfig.setAccumuloTable(tableName.get());
    } else {
        fluoConfig.setAccumuloTable(appName);
    }
    return FluoFactory.newClient(fluoConfig);
}
Also used : FluoConfiguration(org.apache.fluo.api.config.FluoConfiguration)

Example 7 with FluoConfiguration

use of org.apache.fluo.api.config.FluoConfiguration in project incubator-rya by apache.

the class FluoITBase method createFluoConfig.

private FluoConfiguration createFluoConfig() {
    // Configure how the mini fluo will run.
    final FluoConfiguration config = new FluoConfiguration();
    config.setMiniStartAccumulo(false);
    config.setAccumuloInstance(instanceName);
    config.setAccumuloUser(clusterInstance.getUsername());
    config.setAccumuloPassword(clusterInstance.getPassword());
    config.setInstanceZookeepers(zookeepers + "/fluo");
    config.setAccumuloZookeepers(zookeepers);
    config.setApplicationName(getRyaInstanceName());
    config.setAccumuloTable("fluo" + getRyaInstanceName());
    return config;
}
Also used : FluoConfiguration(org.apache.fluo.api.config.FluoConfiguration)

Example 8 with FluoConfiguration

use of org.apache.fluo.api.config.FluoConfiguration in project incubator-rya by apache.

the class RyaClientExample method makeMiniFluo.

private static MiniFluo makeMiniFluo(final String username, final String password, final String instanceName, final String zookeepers, final String fluoAppName) throws AlreadyInitializedException, TableExistsException {
    // Setup the observers that will be used by the Fluo PCJ Application.
    final List<ObserverSpecification> observers = new ArrayList<>();
    observers.add(new ObserverSpecification(TripleObserver.class.getName()));
    observers.add(new ObserverSpecification(BatchObserver.class.getName()));
    observers.add(new ObserverSpecification(StatementPatternObserver.class.getName()));
    observers.add(new ObserverSpecification(JoinObserver.class.getName()));
    observers.add(new ObserverSpecification(FilterObserver.class.getName()));
    // Provide export parameters child test classes may provide to the
    // export observer.
    final HashMap<String, String> params = new HashMap<>();
    final RyaExportParameters ryaParams = new RyaExportParameters(params);
    ryaParams.setUseRyaBindingSetExporter(true);
    ryaParams.setAccumuloInstanceName(instanceName);
    ryaParams.setZookeeperServers(zookeepers);
    ryaParams.setExporterUsername(username);
    ryaParams.setExporterPassword(password);
    ryaParams.setRyaInstanceName(fluoAppName);
    final ObserverSpecification exportObserverConfig = new ObserverSpecification(QueryResultObserver.class.getName(), params);
    observers.add(exportObserverConfig);
    // Configure how the mini fluo will run.
    final FluoConfiguration config = new FluoConfiguration();
    config.setMiniStartAccumulo(false);
    config.setAccumuloInstance(instanceName);
    config.setAccumuloUser(username);
    config.setAccumuloPassword(password);
    config.setInstanceZookeepers(zookeepers + "/fluo");
    config.setAccumuloZookeepers(zookeepers);
    config.setApplicationName(fluoAppName);
    config.setAccumuloTable("fluo" + fluoAppName);
    config.addObservers(observers);
    FluoFactory.newAdmin(config).initialize(new FluoAdmin.InitializationOptions().setClearTable(true).setClearZookeeper(true));
    return FluoFactory.newMiniFluo(config);
}
Also used : ObserverSpecification(org.apache.fluo.api.config.ObserverSpecification) FluoAdmin(org.apache.fluo.api.client.FluoAdmin) HashMap(java.util.HashMap) RyaExportParameters(org.apache.rya.indexing.pcj.fluo.app.export.rya.RyaExportParameters) FluoConfiguration(org.apache.fluo.api.config.FluoConfiguration) ArrayList(java.util.ArrayList) QueryResultObserver(org.apache.rya.indexing.pcj.fluo.app.observers.QueryResultObserver)

Example 9 with FluoConfiguration

use of org.apache.fluo.api.config.FluoConfiguration in project incubator-rya by apache.

the class RyaSubGraphExportParameters method getFluoConfiguration.

/**
 * Uses underlying parameter map to build a FluoConfiguration object
 * @return - FluoConfiguration for creating a FluoClient
 */
public FluoConfiguration getFluoConfiguration() {
    final FluoConfiguration config = new FluoConfiguration();
    config.setMiniStartAccumulo(false);
    config.setAccumuloInstance(params.get(CONF_ACCUMULO_INSTANCE_NAME));
    config.setAccumuloUser(params.get(CONF_EXPORTER_USERNAME));
    config.setAccumuloPassword(params.get(CONF_EXPORTER_PASSWORD));
    config.setInstanceZookeepers(params.get(CONF_FLUO_INSTANCE_ZOOKEEPERS));
    config.setAccumuloZookeepers(params.get(CONF_ZOOKEEPER_SERVERS));
    config.setApplicationName(params.get(CONF_FLUO_APP_NAME));
    config.setAccumuloTable(params.get(CONF_FLUO_TABLE_NAME));
    return config;
}
Also used : FluoConfiguration(org.apache.fluo.api.config.FluoConfiguration)

Example 10 with FluoConfiguration

use of org.apache.fluo.api.config.FluoConfiguration in project incubator-rya by apache.

the class FluoPcjUpdaterSupplier method get.

@Override
public FluoPcjUpdater get() {
    final Configuration config = configSupplier.get();
    checkNotNull(config, "Could not create a FluoPcjUpdater because the application's configuration has not been provided yet.");
    // Ensure the correct updater type has been set.
    final PrecomputedJoinIndexerConfig indexerConfig = new PrecomputedJoinIndexerConfig(config);
    final Optional<PrecomputedJoinUpdaterType> updaterType = indexerConfig.getPcjUpdaterType();
    checkArgument(updaterType.isPresent() && updaterType.get() == PrecomputedJoinUpdaterType.FLUO, "This supplier requires the '" + PrecomputedJoinIndexerConfig.PCJ_UPDATER_TYPE + "' value be set to '" + PrecomputedJoinUpdaterType.FLUO + "'.");
    final FluoPcjUpdaterConfig fluoUpdaterConfig = new FluoPcjUpdaterConfig(indexerConfig.getConfig());
    // Make sure the required values are present.
    checkArgument(fluoUpdaterConfig.getFluoAppName().isPresent(), "Missing configuration: " + FLUO_APP_NAME);
    checkArgument(fluoUpdaterConfig.getFluoZookeepers().isPresent(), "Missing configuration: " + ACCUMULO_ZOOKEEPERS);
    checkArgument(fluoUpdaterConfig.getAccumuloZookeepers().isPresent(), "Missing configuration: " + ACCUMULO_ZOOKEEPERS);
    checkArgument(fluoUpdaterConfig.getAccumuloInstance().isPresent(), "Missing configuration: " + ACCUMULO_INSTANCE);
    checkArgument(fluoUpdaterConfig.getAccumuloUsername().isPresent(), "Missing configuration: " + ACCUMULO_USERNAME);
    checkArgument(fluoUpdaterConfig.getAccumuloPassword().isPresent(), "Missing configuration: " + ACCUMULO_PASSWORD);
    // Fluo configuration values.
    final FluoConfiguration fluoClientConfig = new FluoConfiguration();
    fluoClientConfig.setApplicationName(fluoUpdaterConfig.getFluoAppName().get());
    fluoClientConfig.setInstanceZookeepers(fluoUpdaterConfig.getFluoZookeepers().get());
    // Accumulo Fluo Table configuration values.
    fluoClientConfig.setAccumuloZookeepers(fluoUpdaterConfig.getAccumuloZookeepers().get());
    fluoClientConfig.setAccumuloInstance(fluoUpdaterConfig.getAccumuloInstance().get());
    fluoClientConfig.setAccumuloUser(fluoUpdaterConfig.getAccumuloUsername().get());
    fluoClientConfig.setAccumuloPassword(fluoUpdaterConfig.getAccumuloPassword().get());
    final FluoClient fluoClient = FluoFactory.newClient(fluoClientConfig);
    return new FluoPcjUpdater(fluoClient);
}
Also used : FluoClient(org.apache.fluo.api.client.FluoClient) FluoConfiguration(org.apache.fluo.api.config.FluoConfiguration) Configuration(org.apache.hadoop.conf.Configuration) PrecomputedJoinUpdaterType(org.apache.rya.indexing.external.PrecomputedJoinIndexerConfig.PrecomputedJoinUpdaterType) PrecomputedJoinIndexerConfig(org.apache.rya.indexing.external.PrecomputedJoinIndexerConfig) FluoConfiguration(org.apache.fluo.api.config.FluoConfiguration)

Aggregations

FluoConfiguration (org.apache.fluo.api.config.FluoConfiguration)11 ArrayList (java.util.ArrayList)3 ObserverSpecification (org.apache.fluo.api.config.ObserverSpecification)3 QueryResultObserver (org.apache.rya.indexing.pcj.fluo.app.observers.QueryResultObserver)3 HashMap (java.util.HashMap)2 FluoAdmin (org.apache.fluo.api.client.FluoAdmin)2 FluoClient (org.apache.fluo.api.client.FluoClient)2 RyaExportParameters (org.apache.rya.indexing.pcj.fluo.app.export.rya.RyaExportParameters)2 File (java.io.File)1 FileInputStream (java.io.FileInputStream)1 InputStream (java.io.InputStream)1 Properties (java.util.Properties)1 MiniFluo (org.apache.fluo.api.mini.MiniFluo)1 Configuration (org.apache.hadoop.conf.Configuration)1 PrecomputedJoinIndexerConfig (org.apache.rya.indexing.external.PrecomputedJoinIndexerConfig)1 PrecomputedJoinUpdaterType (org.apache.rya.indexing.external.PrecomputedJoinIndexerConfig.PrecomputedJoinUpdaterType)1 ConstructQueryResultObserver (org.apache.rya.indexing.pcj.fluo.app.observers.ConstructQueryResultObserver)1