Search in sources :

Example 6 with Formatter

use of org.voltdb.importer.formatter.Formatter in project voltdb by VoltDB.

the class TestVoltCSVFormatter method testNowhitespace.

@Test
public void testNowhitespace() throws Exception {
    ServiceReference[] refs = m_bundle.getRegisteredServices();
    ServiceReference<AbstractFormatterFactory> reference = refs[0];
    AbstractFormatterFactory o = m_bundle.getBundleContext().getService(reference);
    Properties prop = new Properties();
    prop.setProperty("nowhitespace", "true");
    FormatterBuilder builder = new FormatterBuilder("csv", prop);
    builder.setFormatterFactory(o);
    Formatter formatter = builder.create();
    try {
        Object[] results = formatter.transform(ByteBuffer.wrap("12,10.05,  test".getBytes(StandardCharsets.UTF_8)));
        fail();
    } catch (RuntimeException e) {
    }
}
Also used : Formatter(org.voltdb.importer.formatter.Formatter) Properties(java.util.Properties) FormatterBuilder(org.voltdb.importer.formatter.FormatterBuilder) AbstractFormatterFactory(org.voltdb.importer.formatter.AbstractFormatterFactory) ServiceReference(org.osgi.framework.ServiceReference) Test(org.junit.Test)

Example 7 with Formatter

use of org.voltdb.importer.formatter.Formatter in project voltdb by VoltDB.

the class TestVoltCSVFormatter method testCustomNull.

@Test
public void testCustomNull() throws Exception {
    ServiceReference[] refs = m_bundle.getRegisteredServices();
    ServiceReference<AbstractFormatterFactory> reference = refs[0];
    AbstractFormatterFactory o = m_bundle.getBundleContext().getService(reference);
    Properties prop = new Properties();
    prop.setProperty("separator", ",");
    prop.setProperty("nullstring", "empty");
    FormatterBuilder builder = new FormatterBuilder("csv", prop);
    builder.setFormatterFactory(o);
    Formatter formatter = builder.create();
    Object[] results = formatter.transform(ByteBuffer.wrap("12,empty,test".getBytes(StandardCharsets.UTF_8)));
    assertEquals(results.length, 3);
    assertEquals(results[0], "12");
    assertEquals(results[1], null);
    assertEquals(results[2], "test");
}
Also used : Formatter(org.voltdb.importer.formatter.Formatter) Properties(java.util.Properties) FormatterBuilder(org.voltdb.importer.formatter.FormatterBuilder) AbstractFormatterFactory(org.voltdb.importer.formatter.AbstractFormatterFactory) ServiceReference(org.osgi.framework.ServiceReference) Test(org.junit.Test)

Example 8 with Formatter

use of org.voltdb.importer.formatter.Formatter in project voltdb by VoltDB.

the class TestVoltCSVFormatter method testTSVBundle.

@Test
public void testTSVBundle() throws Exception {
    ServiceReference[] refs = m_bundle.getRegisteredServices();
    ServiceReference<AbstractFormatterFactory> reference = refs[0];
    AbstractFormatterFactory o = m_bundle.getBundleContext().getService(reference);
    Properties prop = new Properties();
    FormatterBuilder builder = new FormatterBuilder("tsv", prop);
    builder.setFormatterFactory(o);
    Formatter formatter = builder.create();
    Object[] results = formatter.transform(ByteBuffer.wrap("12\t10.05\ttest".getBytes(StandardCharsets.UTF_8)));
    assertEquals(results.length, 3);
    assertEquals(results[0], "12");
    assertEquals(results[1], "10.05");
    assertEquals(results[2], "test");
}
Also used : Formatter(org.voltdb.importer.formatter.Formatter) Properties(java.util.Properties) FormatterBuilder(org.voltdb.importer.formatter.FormatterBuilder) AbstractFormatterFactory(org.voltdb.importer.formatter.AbstractFormatterFactory) ServiceReference(org.osgi.framework.ServiceReference) Test(org.junit.Test)

Example 9 with Formatter

use of org.voltdb.importer.formatter.Formatter in project voltdb by VoltDB.

the class TestVoltCSVFormatter method testEscape.

@Test
public void testEscape() throws Exception {
    ServiceReference[] refs = m_bundle.getRegisteredServices();
    ServiceReference<AbstractFormatterFactory> reference = refs[0];
    AbstractFormatterFactory o = m_bundle.getBundleContext().getService(reference);
    Properties prop = new Properties();
    prop.setProperty("escape", "|");
    FormatterBuilder builder = new FormatterBuilder("csv", prop);
    builder.setFormatterFactory(o);
    Formatter formatter = builder.create();
    Object[] results = formatter.transform(ByteBuffer.wrap("12,\"10.05,|\"test|\"\"".getBytes(StandardCharsets.UTF_8)));
    assertEquals(results.length, 2);
    assertEquals(results[0], "12");
    assertEquals(results[1], "10.05,\"test\"");
}
Also used : Formatter(org.voltdb.importer.formatter.Formatter) Properties(java.util.Properties) FormatterBuilder(org.voltdb.importer.formatter.FormatterBuilder) AbstractFormatterFactory(org.voltdb.importer.formatter.AbstractFormatterFactory) ServiceReference(org.osgi.framework.ServiceReference) Test(org.junit.Test)

Example 10 with Formatter

use of org.voltdb.importer.formatter.Formatter in project voltdb by VoltDB.

the class TestVoltCSVFormatter method testCSVBundle.

@Test
public void testCSVBundle() throws Exception {
    ServiceReference[] refs = m_bundle.getRegisteredServices();
    ServiceReference<AbstractFormatterFactory> reference = refs[0];
    AbstractFormatterFactory o = m_bundle.getBundleContext().getService(reference);
    Properties prop = new Properties();
    FormatterBuilder builder = new FormatterBuilder("csv", prop);
    builder.setFormatterFactory(o);
    Formatter formatter = builder.create();
    Object[] results = formatter.transform(ByteBuffer.wrap("12,10.05,test".getBytes(StandardCharsets.UTF_8)));
    assertEquals(results.length, 3);
    assertEquals(results[0], "12");
    assertEquals(results[1], "10.05");
    assertEquals(results[2], "test");
}
Also used : Formatter(org.voltdb.importer.formatter.Formatter) Properties(java.util.Properties) FormatterBuilder(org.voltdb.importer.formatter.FormatterBuilder) AbstractFormatterFactory(org.voltdb.importer.formatter.AbstractFormatterFactory) ServiceReference(org.osgi.framework.ServiceReference) Test(org.junit.Test)

Aggregations

Formatter (org.voltdb.importer.formatter.Formatter)15 Properties (java.util.Properties)13 Test (org.junit.Test)13 ServiceReference (org.osgi.framework.ServiceReference)13 AbstractFormatterFactory (org.voltdb.importer.formatter.AbstractFormatterFactory)13 FormatterBuilder (org.voltdb.importer.formatter.FormatterBuilder)13 IOException (java.io.IOException)2 Invocation (org.voltdb.importer.Invocation)2 FormatException (org.voltdb.importer.formatter.FormatException)2 BufferedReader (java.io.BufferedReader)1 EOFException (java.io.EOFException)1 InterruptedIOException (java.io.InterruptedIOException)1 ByteBuffer (java.nio.ByteBuffer)1 FetchRequest (kafka.api.FetchRequest)1 FetchResponse (kafka.javaapi.FetchResponse)1 OffsetFetchRequest (kafka.javaapi.OffsetFetchRequest)1 OffsetFetchResponse (kafka.javaapi.OffsetFetchResponse)1 MessageAndOffset (kafka.message.MessageAndOffset)1 BlockingChannel (kafka.network.BlockingChannel)1