Search in sources :

Example 11 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project buck by facebook.

the class TestRunning method addExtraXmlInfo.

/**
   * A helper method that adds extra XML.
   *
   * This includes a test name, time (in ms), message, and stack trace, when
   * present.
   * Example:
   *
   * <pre>
   * &lt;testresult name="failed_test" time="200">
   *   &lt;message>Reason for test failure&lt;/message>
   *   &lt;stacktrace>Stacktrace here&lt;/stacktrace>
   * &lt;/testresult>
   * </pre>
   *
   * @param testCase The test case summary containing one or more tests.
   * @param testEl The XML element object for the <test> tag, in which extra
   *     information tags will be added.
   */
@VisibleForTesting
static void addExtraXmlInfo(TestCaseSummary testCase, Element testEl) {
    Document doc = testEl.getOwnerDocument();
    // Loop through the test case and extract test data.
    for (TestResultSummary testResult : testCase.getTestResults()) {
        // Extract the test name and time.
        String name = Strings.nullToEmpty(testResult.getTestName());
        String time = Long.toString(testResult.getTime());
        // Create the tag: <testresult name="..." time="...">
        Element testResultEl = doc.createElement("testresult");
        testResultEl.setAttribute("name", name);
        testResultEl.setAttribute("time", time);
        testEl.appendChild(testResultEl);
        // Create the tag: <message>(Error message here)</message>
        Element messageEl = doc.createElement("message");
        String message = Strings.nullToEmpty(testResult.getMessage());
        messageEl.appendChild(doc.createTextNode(message));
        testResultEl.appendChild(messageEl);
        // Create the tag: <stacktrace>(Stacktrace here)</stacktrace>
        Element stacktraceEl = doc.createElement("stacktrace");
        String stacktrace = Strings.nullToEmpty(testResult.getStacktrace());
        stacktraceEl.appendChild(doc.createTextNode(stacktrace));
        testResultEl.appendChild(stacktraceEl);
    }
}
Also used : Element(org.w3c.dom.Element) Document(org.w3c.dom.Document) TestResultSummary(com.facebook.buck.test.TestResultSummary) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 12 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project che by eclipse.

the class CheEnvironmentEngine method normalizeLinks.

/**
     * Replaces linked to this service's name with container name which represents the service in links section.
     * The problem is that a user writes names of other services in links section in compose file.
     * But actually links are constraints and their values should be names of containers (not services) to be linked.
     * <br/>
     * For example: serviceDB:serviceDbAlias -> container_1234:serviceDbAlias <br/>
     * If alias is omitted then service name will be used.
     *
     * @param serviceToNormalizeLinks
     *         service which links will be normalized
     * @param services
     *         all services in environment
     */
@VisibleForTesting
void normalizeLinks(CheServiceImpl serviceToNormalizeLinks, Map<String, CheServiceImpl> services) {
    serviceToNormalizeLinks.setLinks(serviceToNormalizeLinks.getLinks().stream().map(link -> {
        String[] serviceNameAndAliasToLink = link.split(":", 2);
        String serviceName = serviceNameAndAliasToLink[0];
        String serviceAlias = (serviceNameAndAliasToLink.length > 1) ? serviceNameAndAliasToLink[1] : null;
        CheServiceImpl serviceLinkTo = services.get(serviceName);
        if (serviceLinkTo != null) {
            String containerNameLinkTo = serviceLinkTo.getContainerName();
            return (serviceAlias == null) ? containerNameLinkTo : containerNameLinkTo + ':' + serviceAlias;
        } else {
            throw new IllegalArgumentException("Attempt to link non existing service " + serviceName + " to " + serviceToNormalizeLinks + " service.");
        }
    }).collect(toList()));
}
Also used : CheServiceImpl(org.eclipse.che.api.environment.server.model.CheServiceImpl) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 13 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project druid by druid-io.

the class JavaScriptAggregatorFactory method compileScript.

@VisibleForTesting
static JavaScriptAggregator.ScriptAggregator compileScript(final String aggregate, final String reset, final String combine) {
    final ContextFactory contextFactory = ContextFactory.getGlobal();
    Context context = contextFactory.enterContext();
    context.setOptimizationLevel(JavaScriptConfig.DEFAULT_OPTIMIZATION_LEVEL);
    final ScriptableObject scope = context.initStandardObjects();
    final Function fnAggregate = context.compileFunction(scope, aggregate, "aggregate", 1, null);
    final Function fnReset = context.compileFunction(scope, reset, "reset", 1, null);
    final Function fnCombine = context.compileFunction(scope, combine, "combine", 1, null);
    Context.exit();
    return new JavaScriptAggregator.ScriptAggregator() {

        @Override
        public double aggregate(final double current, final ObjectColumnSelector[] selectorList) {
            Context cx = Context.getCurrentContext();
            if (cx == null) {
                cx = contextFactory.enterContext();
                // Disable primitive wrapping- we want Java strings and primitives to behave like JS entities.
                cx.getWrapFactory().setJavaPrimitiveWrap(false);
            }
            final int size = selectorList.length;
            final Object[] args = new Object[size + 1];
            args[0] = current;
            for (int i = 0; i < size; i++) {
                final ObjectColumnSelector selector = selectorList[i];
                if (selector != null) {
                    final Object arg = selector.get();
                    if (arg != null && arg.getClass().isArray()) {
                        // Context.javaToJS on an array sort of works, although it returns false for Array.isArray(...) and
                        // may have other issues too. Let's just copy the array and wrap that.
                        final Object[] arrayAsObjectArray = new Object[Array.getLength(arg)];
                        for (int j = 0; j < Array.getLength(arg); j++) {
                            arrayAsObjectArray[j] = Array.get(arg, j);
                        }
                        args[i + 1] = cx.newArray(scope, arrayAsObjectArray);
                    } else {
                        args[i + 1] = Context.javaToJS(arg, scope);
                    }
                }
            }
            final Object res = fnAggregate.call(cx, scope, scope, args);
            return Context.toNumber(res);
        }

        @Override
        public double combine(final double a, final double b) {
            final Object res = contextFactory.call(new ContextAction() {

                @Override
                public Object run(final Context cx) {
                    return fnCombine.call(cx, scope, scope, new Object[] { a, b });
                }
            });
            return Context.toNumber(res);
        }

        @Override
        public double reset() {
            final Object res = contextFactory.call(new ContextAction() {

                @Override
                public Object run(final Context cx) {
                    return fnReset.call(cx, scope, scope, new Object[] {});
                }
            });
            return Context.toNumber(res);
        }

        @Override
        public void close() {
            if (Context.getCurrentContext() != null) {
                Context.exit();
            }
        }
    };
}
Also used : ContextFactory(org.mozilla.javascript.ContextFactory) Context(org.mozilla.javascript.Context) Function(org.mozilla.javascript.Function) ScriptableObject(org.mozilla.javascript.ScriptableObject) ContextAction(org.mozilla.javascript.ContextAction) ScriptableObject(org.mozilla.javascript.ScriptableObject) ObjectColumnSelector(io.druid.segment.ObjectColumnSelector) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 14 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project druid by druid-io.

the class QueryableIndexIndexableAdapter method getBitmapIndex.

@VisibleForTesting
IndexedInts getBitmapIndex(String dimension, String value) {
    final Column column = input.getColumn(dimension);
    if (column == null) {
        return EmptyIndexedInts.EMPTY_INDEXED_INTS;
    }
    final BitmapIndex bitmaps = column.getBitmapIndex();
    if (bitmaps == null) {
        return EmptyIndexedInts.EMPTY_INDEXED_INTS;
    }
    return new BitmapCompressedIndexedInts(bitmaps.getBitmap(bitmaps.getIndex(value)));
}
Also used : IndexedLongsGenericColumn(io.druid.segment.column.IndexedLongsGenericColumn) GenericColumn(io.druid.segment.column.GenericColumn) ComplexColumn(io.druid.segment.column.ComplexColumn) Column(io.druid.segment.column.Column) DictionaryEncodedColumn(io.druid.segment.column.DictionaryEncodedColumn) IndexedFloatsGenericColumn(io.druid.segment.column.IndexedFloatsGenericColumn) BitmapCompressedIndexedInts(io.druid.segment.data.BitmapCompressedIndexedInts) BitmapIndex(io.druid.segment.column.BitmapIndex) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 15 with VisibleForTesting

use of org.apache.beam.vendor.calcite.v1_28_0.com.google.common.annotations.VisibleForTesting in project hadoop by apache.

the class InMemorySCMStore method getInitialCachedResources.

@VisibleForTesting
Map<String, String> getInitialCachedResources(FileSystem fs, Configuration conf) throws IOException {
    // get the root directory for the shared cache
    String location = conf.get(YarnConfiguration.SHARED_CACHE_ROOT, YarnConfiguration.DEFAULT_SHARED_CACHE_ROOT);
    Path root = new Path(location);
    try {
        fs.getFileStatus(root);
    } catch (FileNotFoundException e) {
        String message = "The shared cache root directory " + location + " was not found";
        LOG.error(message);
        throw (IOException) new FileNotFoundException(message).initCause(e);
    }
    int nestedLevel = SharedCacheUtil.getCacheDepth(conf);
    // now traverse individual directories and process them
    // the directory structure is specified by the nested level parameter
    // (e.g. 9/c/d/<checksum>/file)
    String pattern = SharedCacheUtil.getCacheEntryGlobPattern(nestedLevel + 1);
    LOG.info("Querying for all individual cached resource files");
    FileStatus[] entries = fs.globStatus(new Path(root, pattern));
    int numEntries = entries == null ? 0 : entries.length;
    LOG.info("Found " + numEntries + " files: processing for one resource per " + "key");
    Map<String, String> initialCachedEntries = new HashMap<String, String>();
    if (entries != null) {
        for (FileStatus entry : entries) {
            Path file = entry.getPath();
            String fileName = file.getName();
            if (entry.isFile()) {
                // get the parent to get the checksum
                Path parent = file.getParent();
                if (parent != null) {
                    // the name of the immediate parent directory is the checksum
                    String key = parent.getName();
                    // first
                    if (initialCachedEntries.containsKey(key)) {
                        LOG.warn("Key " + key + " is already mapped to file " + initialCachedEntries.get(key) + "; file " + fileName + " will not be added");
                    } else {
                        initialCachedEntries.put(key, fileName);
                    }
                }
            }
        }
    }
    LOG.info("A total of " + initialCachedEntries.size() + " files are now mapped");
    return initialCachedEntries;
}
Also used : Path(org.apache.hadoop.fs.Path) FileStatus(org.apache.hadoop.fs.FileStatus) HashMap(java.util.HashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) FileNotFoundException(java.io.FileNotFoundException) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Aggregations

VisibleForTesting (com.google.common.annotations.VisibleForTesting)1954 IOException (java.io.IOException)284 ArrayList (java.util.ArrayList)214 Map (java.util.Map)156 HashMap (java.util.HashMap)147 List (java.util.List)113 File (java.io.File)94 ImmutableMap (com.google.common.collect.ImmutableMap)72 HashSet (java.util.HashSet)67 Path (org.apache.hadoop.fs.Path)63 ImmutableList (com.google.common.collect.ImmutableList)60 Path (java.nio.file.Path)60 Set (java.util.Set)52 Matcher (java.util.regex.Matcher)46 Collectors (java.util.stream.Collectors)46 Collection (java.util.Collection)39 Optional (java.util.Optional)38 NotNull (org.jetbrains.annotations.NotNull)37 ImmutableSet (com.google.common.collect.ImmutableSet)34 TreeMap (java.util.TreeMap)34