Search in sources :

Example 21 with APIQueryControlResult

use of edu.snu.mist.client.APIQueryControlResult in project mist by snuspl.

the class SessionWindow method main.

/**
 * Set the environment(Hostname and port of driver, source, and sink) and submit a query.
 * @param args command line parameters
 * @throws Exception
 */
public static void main(final String[] args) throws Exception {
    final JavaConfigurationBuilder jcb = Tang.Factory.getTang().newConfigurationBuilder();
    final CommandLine commandLine = MISTExampleUtils.getDefaultCommandLine(jcb).registerShortNameOfClass(// Additional parameter
    NettySourceAddress.class).processCommandLine(args);
    if (commandLine == null) {
        // Option '?' was entered and processCommandLine printed the help.
        return;
    }
    Thread sinkServer = new Thread(MISTExampleUtils.getSinkServer());
    sinkServer.start();
    final APIQueryControlResult result = submitQuery(jcb.build());
    System.out.println("Query submission result: " + result.getQueryId());
}
Also used : APIQueryControlResult(edu.snu.mist.client.APIQueryControlResult) CommandLine(org.apache.reef.tang.formats.CommandLine) JavaConfigurationBuilder(org.apache.reef.tang.JavaConfigurationBuilder) NettySourceAddress(edu.snu.mist.examples.parameters.NettySourceAddress)

Example 22 with APIQueryControlResult

use of edu.snu.mist.client.APIQueryControlResult in project mist by snuspl.

the class UnionMist method main.

/**
 * Set the environment(Hostname and port of driver, source, and sink) and submit a query.
 * @param args command line parameters
 * @throws Exception
 */
public static void main(final String[] args) throws Exception {
    final JavaConfigurationBuilder jcb = Tang.Factory.getTang().newConfigurationBuilder();
    final CommandLine commandLine = MISTExampleUtils.getDefaultCommandLine(jcb).registerShortNameOfClass(// Additional parameter
    UnionLeftSourceAddress.class).registerShortNameOfClass(// Additional parameter
    UnionRightSourceAddress.class).processCommandLine(args);
    if (commandLine == null) {
        // Option '?' was entered and processCommandLine printed the help.
        return;
    }
    Thread sinkServer = new Thread(MISTExampleUtils.getSinkServer());
    sinkServer.start();
    final APIQueryControlResult result = submitQuery(jcb.build());
    System.out.println("Query submission result: " + result.getQueryId());
}
Also used : APIQueryControlResult(edu.snu.mist.client.APIQueryControlResult) CommandLine(org.apache.reef.tang.formats.CommandLine) UnionRightSourceAddress(edu.snu.mist.examples.parameters.UnionRightSourceAddress) JavaConfigurationBuilder(org.apache.reef.tang.JavaConfigurationBuilder)

Example 23 with APIQueryControlResult

use of edu.snu.mist.client.APIQueryControlResult in project mist by snuspl.

the class UnionMist method submitQuery.

/**
 * Submit a query unifying two sources.
 * The query reads strings from two source servers, unifies them, and send them to a sink server.
 * @return result of the submission
 * @throws IOException
 * @throws InjectionException
 */
public static APIQueryControlResult submitQuery(final Configuration configuration) throws IOException, InjectionException, URISyntaxException {
    final Injector injector = Tang.Factory.getTang().newInjector(configuration);
    final String source1Socket = injector.getNamedInstance(UnionLeftSourceAddress.class);
    final String source2Socket = injector.getNamedInstance(UnionRightSourceAddress.class);
    final SourceConfiguration localTextSocketSource1Conf = MISTExampleUtils.getLocalTextSocketSourceConf(source1Socket);
    final SourceConfiguration localTextSocketSource2Conf = MISTExampleUtils.getLocalTextSocketSourceConf(source2Socket);
    // Simple reduce function.
    final MISTBiFunction<Integer, Integer, Integer> reduceFunction = (v1, v2) -> {
        return v1 + v2;
    };
    final MISTQueryBuilder queryBuilder = new MISTQueryBuilder();
    final ContinuousStream sourceStream1 = queryBuilder.socketTextStream(localTextSocketSource1Conf).map(s -> new Tuple2(s, 1));
    final ContinuousStream sourceStream2 = queryBuilder.socketTextStream(localTextSocketSource2Conf).map(s -> new Tuple2(s, 1));
    sourceStream1.union(sourceStream2).reduceByKey(0, String.class, reduceFunction).map(s -> s.toString()).textSocketOutput(MISTExampleUtils.SINK_HOSTNAME, MISTExampleUtils.SINK_PORT);
    return MISTExampleUtils.submit(queryBuilder, configuration);
}
Also used : Injector(org.apache.reef.tang.Injector) Tang(org.apache.reef.tang.Tang) SourceConfiguration(edu.snu.mist.client.datastreams.configurations.SourceConfiguration) Tuple2(edu.snu.mist.common.types.Tuple2) URISyntaxException(java.net.URISyntaxException) IOException(java.io.IOException) CommandLine(org.apache.reef.tang.formats.CommandLine) ContinuousStream(edu.snu.mist.client.datastreams.ContinuousStream) APIQueryControlResult(edu.snu.mist.client.APIQueryControlResult) UnionLeftSourceAddress(edu.snu.mist.examples.parameters.UnionLeftSourceAddress) MISTQueryBuilder(edu.snu.mist.client.MISTQueryBuilder) Configuration(org.apache.reef.tang.Configuration) JavaConfigurationBuilder(org.apache.reef.tang.JavaConfigurationBuilder) MISTBiFunction(edu.snu.mist.common.functions.MISTBiFunction) InjectionException(org.apache.reef.tang.exceptions.InjectionException) UnionRightSourceAddress(edu.snu.mist.examples.parameters.UnionRightSourceAddress) MISTQueryBuilder(edu.snu.mist.client.MISTQueryBuilder) ContinuousStream(edu.snu.mist.client.datastreams.ContinuousStream) Injector(org.apache.reef.tang.Injector) Tuple2(edu.snu.mist.common.types.Tuple2) SourceConfiguration(edu.snu.mist.client.datastreams.configurations.SourceConfiguration)

Example 24 with APIQueryControlResult

use of edu.snu.mist.client.APIQueryControlResult in project mist by snuspl.

the class WindowAndAggregate method submitQuery.

/**
 * Submit a windowing and aggregating query.
 * The query reads strings from a source server, puts them into window,
 * concatenates all inputs in the window into a single string using toString function, and
 * print out the start and end time of the window.
 * @return result of the submission
 * @throws IOException
 * @throws InjectionException
 */
public static APIQueryControlResult submitQuery(final Configuration configuration) throws IOException, InjectionException, URISyntaxException {
    // configurations for source and sink
    final String sourceSocket = Tang.Factory.getTang().newInjector(configuration).getNamedInstance(NettySourceAddress.class);
    final SourceConfiguration localTextSocketSourceConf = MISTExampleUtils.getLocalTextSocketSourceConf(sourceSocket);
    // configurations for windowing and aggregation
    final int windowSize = 5000;
    final int windowEmissionInterval = 2500;
    final MISTFunction<WindowData<String>, String> aggregateFunc = (windowData) -> {
        return windowData.getDataCollection().toString() + ", window is started at " + windowData.getStart() + ", window is ended at " + windowData.getEnd() + ".";
    };
    final MISTQueryBuilder queryBuilder = new MISTQueryBuilder();
    queryBuilder.socketTextStream(localTextSocketSourceConf).window(new TimeWindowInformation(windowSize, windowEmissionInterval)).aggregateWindow(aggregateFunc).textSocketOutput(MISTExampleUtils.SINK_HOSTNAME, MISTExampleUtils.SINK_PORT);
    return MISTExampleUtils.submit(queryBuilder, configuration);
}
Also used : TimeWindowInformation(edu.snu.mist.common.windows.TimeWindowInformation) Tang(org.apache.reef.tang.Tang) SourceConfiguration(edu.snu.mist.client.datastreams.configurations.SourceConfiguration) MISTFunction(edu.snu.mist.common.functions.MISTFunction) NettySourceAddress(edu.snu.mist.examples.parameters.NettySourceAddress) URISyntaxException(java.net.URISyntaxException) IOException(java.io.IOException) CommandLine(org.apache.reef.tang.formats.CommandLine) APIQueryControlResult(edu.snu.mist.client.APIQueryControlResult) MISTQueryBuilder(edu.snu.mist.client.MISTQueryBuilder) Configuration(org.apache.reef.tang.Configuration) JavaConfigurationBuilder(org.apache.reef.tang.JavaConfigurationBuilder) WindowData(edu.snu.mist.common.windows.WindowData) InjectionException(org.apache.reef.tang.exceptions.InjectionException) MISTQueryBuilder(edu.snu.mist.client.MISTQueryBuilder) WindowData(edu.snu.mist.common.windows.WindowData) SourceConfiguration(edu.snu.mist.client.datastreams.configurations.SourceConfiguration) TimeWindowInformation(edu.snu.mist.common.windows.TimeWindowInformation)

Example 25 with APIQueryControlResult

use of edu.snu.mist.client.APIQueryControlResult in project mist by snuspl.

the class WindowAndAggregate method main.

/**
 * Set the environment(Hostname and port of driver, source, and sink) and submit a query.
 * @param args command line parameters
 * @throws Exception
 */
public static void main(final String[] args) throws Exception {
    final JavaConfigurationBuilder jcb = Tang.Factory.getTang().newConfigurationBuilder();
    final CommandLine commandLine = MISTExampleUtils.getDefaultCommandLine(jcb).registerShortNameOfClass(// Additional parameter
    NettySourceAddress.class).processCommandLine(args);
    if (commandLine == null) {
        // Option '?' was entered and processCommandLine printed the help.
        return;
    }
    Thread sinkServer = new Thread(MISTExampleUtils.getSinkServer());
    sinkServer.start();
    final APIQueryControlResult result = submitQuery(jcb.build());
    System.out.println("Query submission result: " + result.getQueryId());
}
Also used : APIQueryControlResult(edu.snu.mist.client.APIQueryControlResult) CommandLine(org.apache.reef.tang.formats.CommandLine) JavaConfigurationBuilder(org.apache.reef.tang.JavaConfigurationBuilder) NettySourceAddress(edu.snu.mist.examples.parameters.NettySourceAddress)

Aggregations

APIQueryControlResult (edu.snu.mist.client.APIQueryControlResult)28 JavaConfigurationBuilder (org.apache.reef.tang.JavaConfigurationBuilder)28 CommandLine (org.apache.reef.tang.formats.CommandLine)28 NettySourceAddress (edu.snu.mist.examples.parameters.NettySourceAddress)20 MISTQueryBuilder (edu.snu.mist.client.MISTQueryBuilder)11 IOException (java.io.IOException)11 URISyntaxException (java.net.URISyntaxException)11 Configuration (org.apache.reef.tang.Configuration)11 Tang (org.apache.reef.tang.Tang)11 InjectionException (org.apache.reef.tang.exceptions.InjectionException)11 SourceConfiguration (edu.snu.mist.client.datastreams.configurations.SourceConfiguration)10 MISTFunction (edu.snu.mist.common.functions.MISTFunction)4 Tuple2 (edu.snu.mist.common.types.Tuple2)4 UnionRightSourceAddress (edu.snu.mist.examples.parameters.UnionRightSourceAddress)4 ContinuousStream (edu.snu.mist.client.datastreams.ContinuousStream)3 MISTBiFunction (edu.snu.mist.common.functions.MISTBiFunction)3 ApplyStatefulFunction (edu.snu.mist.common.functions.ApplyStatefulFunction)2 TimeWindowInformation (edu.snu.mist.common.windows.TimeWindowInformation)2 WindowData (edu.snu.mist.common.windows.WindowData)2 UnionLeftSourceAddress (edu.snu.mist.examples.parameters.UnionLeftSourceAddress)2