Examples with WebPage org.apache.gora.examples.generated.WebPage used on opensource projects

Example 16 with WebPage

use of org.apache.gora.examples.generated.WebPage in project gora by apache.

the class TestPersistentBase method testClear.

/**
   * First we create a new WebPage object, to which we add some
   * field values. This makes the fields dirty as we have not 
   * flushed them to the datastore. We then clear the dirty
   * fields and assert that the values DO NOT exist for the 
   * field we previously made dirty.
   * We then set new values for fields, consequently making them 
   * dirty, before testing the clearing of an entirely new object
   * has all fields as null as they should be clean.
   */
@Test
public void testClear() {
    //test clear all fields
    WebPage page = WebPage.newBuilder().build();
    page.setUrl(new Utf8("http://foo.com"));
    page.getParsedContent().add(new Utf8("foo"));
    page.getOutlinks().put(new Utf8("foo"), new Utf8("bar"));
    page.setContent(ByteBuffer.wrap("foo baz bar".getBytes(Charset.defaultCharset())));
    page.clear();
    assertNull(page.getUrl());
    assertEquals(0, page.getParsedContent().size());
    assertEquals(0, page.getOutlinks().size());
    assertNull(page.getContent());
    //set fields again
    page.setUrl(new Utf8("http://bar.com"));
    page.getParsedContent().add(new Utf8("bar"));
    page.getOutlinks().put(new Utf8("bar"), new Utf8("baz"));
    page.setContent(ByteBuffer.wrap("foo baz bar barbaz".getBytes(Charset.defaultCharset())));
    //test clear new object
    page = WebPage.newBuilder().build();
    page.clear();
}

Example 17 with WebPage

use of org.apache.gora.examples.generated.WebPage in project gora by apache.

the class TestPersistentBase method testGetFields.

/**
   * Assert that the list of fields from the WebPage Schema
   * are as we expect. This is done by creating and accessing 
   * a WebPage object, then comparing the results against 
   * static fields of the WebPage.SCHEMA$.
   */
@Test
public void testGetFields() {
    WebPage page = WebPage.newBuilder().build();
    List<Field> fields = page.getSchema().getFields();
    assertEquals(WebPage.SCHEMA$.getFields(), fields);
}

Example 18 with WebPage

use of org.apache.gora.examples.generated.WebPage in project gora by apache.

the class DataStoreTestUtil method createWebPage.

private static <K> WebPage createWebPage() {
    WebPage webpage = WebPage.newBuilder().build();
    webpage.setUrl(new Utf8("url.."));
    webpage.setContent(ByteBuffer.wrap("test content".getBytes(Charset.defaultCharset())));
    webpage.setParsedContent(new ArrayList<CharSequence>());
    Metadata metadata = Metadata.newBuilder().build();
    webpage.setMetadata(metadata);
    return webpage;
}

Example 19 with WebPage

use of org.apache.gora.examples.generated.WebPage in project gora by apache.

the class DataStoreTestUtil method testPutArray.

public static void testPutArray(DataStore<String, WebPage> store) throws Exception {
    store.createSchema();
    WebPage page = WebPage.newBuilder().build();
    String[] tokens = { "example", "content", "in", "example.com" };
    page.setParsedContent(new ArrayList<CharSequence>());
    for (String token : tokens) {
        page.getParsedContent().add(new Utf8(token));
    }
    store.put("com.example/http", page);
    store.close();
}

Example 20 with WebPage

use of org.apache.gora.examples.generated.WebPage in project gora by apache.

the class TestFilterList method testOperatorMustPassOne.

/**
   * Test method for verifying 
   * {@link org.apache.gora.filter.FilterList.Operator#MUST_PASS_ONE}
   * functionality.
   */
@Test
public void testOperatorMustPassOne() {
    filter1.setFieldName(WebPage.Field.OUTLINKS.toString());
    filter1.setMapKey(new Utf8("example"));
    filter1.setFilterOp(FilterOp.EQUALS);
    filter1.setFilterIfMissing(true);
    filter1.getOperands().add(new Utf8("http://example.org"));
    filter2.setFieldName(WebPage.Field.OUTLINKS.toString());
    filter2.setFilterOp(FilterOp.EQUALS);
    filter2.setFilterIfMissing(true);
    filter2.getOperands().add(new Utf8("http://example2.org"));
    WebPage page = WebPage.newBuilder().build();
    page.getOutlinks().put(new Utf8("example"), new Utf8("http://example.org"));
    filterList = new FilterList<>(Operator.MUST_PASS_ONE, fList);
    assertTrue(filterList.filter("irrelevant", page));
}