Search in sources :

Example 1 with QueryParams

use of org.apache.atlas.query.QueryParams in project incubator-atlas by apache.

the class GraphBackedMetadataRepositoryTest method testConcurrentCalls.

@Test
public //In some cases of parallel APIs, the edge is added, but get edge by label doesn't return the edge. ATLAS-1104
void testConcurrentCalls() throws Exception {
    final HierarchicalTypeDefinition<ClassType> refType = createClassTypeDef(randomString(), ImmutableSet.<String>of());
    HierarchicalTypeDefinition<ClassType> type = createClassTypeDef(randomString(), ImmutableSet.<String>of(), new AttributeDefinition("ref", refType.typeName, Multiplicity.OPTIONAL, true, null));
    typeSystem.defineClassType(refType);
    typeSystem.defineClassType(type);
    String refId1 = createEntity(new Referenceable(refType.typeName)).get(0);
    String refId2 = createEntity(new Referenceable(refType.typeName)).get(0);
    final Referenceable instance1 = new Referenceable(type.typeName);
    instance1.set("ref", new Referenceable(refId1, refType.typeName, null));
    final Referenceable instance2 = new Referenceable(type.typeName);
    instance2.set("ref", new Referenceable(refId2, refType.typeName, null));
    ExecutorService executor = Executors.newFixedThreadPool(3);
    List<Future<Object>> futures = new ArrayList<>();
    futures.add(executor.submit(new Callable<Object>() {

        @Override
        public Object call() throws Exception {
            return createEntity(instance1).get(0);
        }
    }));
    futures.add(executor.submit(new Callable<Object>() {

        @Override
        public Object call() throws Exception {
            return createEntity(instance2).get(0);
        }
    }));
    futures.add(executor.submit(new Callable<Object>() {

        @Override
        public Object call() throws Exception {
            return discoveryService.searchByDSL(TestUtils.TABLE_TYPE, new QueryParams(10, 0));
        }
    }));
    String id1 = (String) futures.get(0).get();
    String id2 = (String) futures.get(1).get();
    futures.get(2).get();
    executor.shutdown();
    boolean validated1 = assertEdge(id1, type.typeName);
    boolean validated2 = assertEdge(id2, type.typeName);
    assertTrue(validated1 | validated2);
}
Also used : Referenceable(org.apache.atlas.typesystem.Referenceable) ExecutorService(java.util.concurrent.ExecutorService) ArrayList(java.util.ArrayList) AttributeDefinition(org.apache.atlas.typesystem.types.AttributeDefinition) Future(java.util.concurrent.Future) QueryParams(org.apache.atlas.query.QueryParams) ClassType(org.apache.atlas.typesystem.types.ClassType) Callable(java.util.concurrent.Callable) Test(org.testng.annotations.Test)

Example 2 with QueryParams

use of org.apache.atlas.query.QueryParams in project incubator-atlas by apache.

the class GraphBackedMetadataRepositoryTest method testFullTextSearch.

/**
     * Full text search requires GraphBackedSearchIndexer, and GraphBackedSearchIndexer can't be enabled in
     * GraphBackedDiscoveryServiceTest because of its test data. So, test for full text search is in
     * GraphBackedMetadataRepositoryTest:(
     */
@Test(dependsOnMethods = "testSubmitEntity")
public void testFullTextSearch() throws Exception {
    //todo fix this
    //Weird: with lucene, the test passes without sleep
    //but with elasticsearch, doesn't work without sleep. why??
    long sleepInterval = 1000;
    TestUtils.dumpGraph(TestUtils.getGraph());
    //person in hr department whose name is john
    Thread.sleep(sleepInterval);
    String response = discoveryService.searchByFullText("john", queryParams);
    Assert.assertNotNull(response);
    JSONArray results = new JSONArray(response);
    Assert.assertEquals(results.length(), 1);
    JSONObject row = (JSONObject) results.get(0);
    Assert.assertEquals(row.get("typeName"), "Person");
    //person in hr department who lives in santa clara
    response = discoveryService.searchByFullText("Jane AND santa AND clara", queryParams);
    Assert.assertNotNull(response);
    results = new JSONArray(response);
    Assert.assertEquals(results.length(), 1);
    row = (JSONObject) results.get(0);
    Assert.assertEquals(row.get("typeName"), "Manager");
    //search for person in hr department whose name starts is john/jahn
    response = discoveryService.searchByFullText("hr AND (john OR jahn)", queryParams);
    Assert.assertNotNull(response);
    results = new JSONArray(response);
    Assert.assertEquals(results.length(), 1);
    row = (JSONObject) results.get(0);
    Assert.assertEquals(row.get("typeName"), "Person");
    //verify limit and offset
    //higher limit should return all results
    results = new JSONArray(discoveryService.searchByFullText("Department", queryParams));
    assertEquals(results.length(), 5);
    //smaller limit should return those many rows
    results = new JSONArray(discoveryService.searchByFullText("Department", new QueryParams(2, 0)));
    assertEquals(results.length(), 2);
    //offset should offset the results
    results = new JSONArray(discoveryService.searchByFullText("Department", new QueryParams(5, 2)));
    assertEquals(results.length(), 3);
    //higher offset shouldn't return any rows
    results = new JSONArray(discoveryService.searchByFullText("Department", new QueryParams(2, 6)));
    assertEquals(results.length(), 0);
}
Also used : JSONObject(org.codehaus.jettison.json.JSONObject) JSONArray(org.codehaus.jettison.json.JSONArray) QueryParams(org.apache.atlas.query.QueryParams) Test(org.testng.annotations.Test)

Example 3 with QueryParams

use of org.apache.atlas.query.QueryParams in project incubator-atlas by apache.

the class EntityDiscoveryService method toGremlinQuery.

private GremlinQuery toGremlinQuery(String query, int limit, int offset) throws AtlasBaseException {
    QueryParams params = validateSearchParams(limit, offset);
    Either<NoSuccess, Expression> either = QueryParser.apply(query, params);
    if (either.isLeft()) {
        throw new AtlasBaseException(DISCOVERY_QUERY_FAILED, query);
    }
    Expression expression = either.right().get();
    Expression validExpression = QueryProcessor.validate(expression);
    GremlinQuery gremlinQuery = new GremlinTranslator(validExpression, graphPersistenceStrategy).translate();
    if (LOG.isDebugEnabled()) {
        LOG.debug("Translated Gremlin Query: {}", gremlinQuery.queryStr());
    }
    return gremlinQuery;
}
Also used : GremlinTranslator(org.apache.atlas.query.GremlinTranslator) AtlasBaseException(org.apache.atlas.exception.AtlasBaseException) AliasExpression(org.apache.atlas.query.Expressions.AliasExpression) Expression(org.apache.atlas.query.Expressions.Expression) SelectExpression(org.apache.atlas.query.Expressions.SelectExpression) NoSuccess(scala.util.parsing.combinator.Parsers.NoSuccess) QueryParams(org.apache.atlas.query.QueryParams) GremlinQuery(org.apache.atlas.query.GremlinQuery) AtlasGremlinQuery(org.apache.atlas.util.AtlasGremlinQueryProvider.AtlasGremlinQuery)

Example 4 with QueryParams

use of org.apache.atlas.query.QueryParams in project incubator-atlas by apache.

the class DataSetLineageService method getSchemaForId.

private String getSchemaForId(String typeName, String guid) throws DiscoveryException, SchemaNotFoundException {
    String configName = DATASET_SCHEMA_QUERY_PREFIX + typeName;
    if (propertiesConf.getString(configName) != null) {
        final String schemaQuery = String.format(propertiesConf.getString(configName), guid);
        int limit = AtlasConfiguration.SEARCH_MAX_LIMIT.getInt();
        return discoveryService.searchByDSL(schemaQuery, new QueryParams(limit, 0));
    }
    throw new SchemaNotFoundException("Schema is not configured for type " + typeName + ". Configure " + configName);
}
Also used : QueryParams(org.apache.atlas.query.QueryParams) SchemaNotFoundException(org.apache.atlas.typesystem.exception.SchemaNotFoundException)

Example 5 with QueryParams

use of org.apache.atlas.query.QueryParams in project incubator-atlas by apache.

the class CompiledQueryCacheKeyTest method testOnlyQueryParamsDifferent.

@Test
public void testOnlyQueryParamsDifferent() {
    CompiledQueryCacheKey e1 = new CompiledQueryCacheKey("query 1", new QueryParams(10, 10));
    CompiledQueryCacheKey e2 = new CompiledQueryCacheKey("query 1", new QueryParams(20, 10));
    assertKeysDifferent(e1, e2);
}
Also used : QueryParams(org.apache.atlas.query.QueryParams) Test(org.testng.annotations.Test)

Aggregations

QueryParams (org.apache.atlas.query.QueryParams)15 Test (org.testng.annotations.Test)8 JSONObject (org.codehaus.jettison.json.JSONObject)6 JSONArray (org.codehaus.jettison.json.JSONArray)4 ArrayList (java.util.ArrayList)2 Consumes (javax.ws.rs.Consumes)2 GET (javax.ws.rs.GET)2 Path (javax.ws.rs.Path)2 Produces (javax.ws.rs.Produces)2 WebApplicationException (javax.ws.rs.WebApplicationException)2 DiscoveryException (org.apache.atlas.discovery.DiscoveryException)2 AtlasBaseException (org.apache.atlas.exception.AtlasBaseException)2 AtlasSearchResult (org.apache.atlas.model.discovery.AtlasSearchResult)2 Referenceable (org.apache.atlas.typesystem.Referenceable)2 AttributeDefinition (org.apache.atlas.typesystem.types.AttributeDefinition)2 ClassType (org.apache.atlas.typesystem.types.ClassType)2 AtlasPerfTracer (org.apache.atlas.utils.AtlasPerfTracer)2 HashMap (java.util.HashMap)1 List (java.util.List)1 Callable (java.util.concurrent.Callable)1