Search in sources :

Example 1 with DateLocaleFactory

use of com.questdb.std.time.DateLocaleFactory in project questdb by bluestreak01.

the class JsonSchemaParserTest method setUpClass.

@BeforeClass
public static void setUpClass() {
    BootstrapEnv env = new BootstrapEnv();
    env.dateFormatFactory = new DateFormatFactory();
    env.dateLocaleFactory = new DateLocaleFactory(new TimeZoneRuleFactory());
    defaultLocaleId = env.dateLocaleFactory.getDefaultDateLocale().getId();
    jsonSchemaParser = new JsonSchemaParser(env);
}
Also used : BootstrapEnv(com.questdb.BootstrapEnv) DateFormatFactory(com.questdb.std.time.DateFormatFactory) DateLocaleFactory(com.questdb.std.time.DateLocaleFactory) TimeZoneRuleFactory(com.questdb.std.time.TimeZoneRuleFactory)

Example 2 with DateLocaleFactory

use of com.questdb.std.time.DateLocaleFactory in project questdb by bluestreak01.

the class ImportManager method analyzeAndParse.

private static void analyzeAndParse(File file, PlainTextLexer parser, MetadataAwareTextParser listener, CharSequence schema, int sampleSize, boolean forceHeader) throws IOException {
    parser.clear();
    ObjList<ImportedColumnMetadata> metadata = null;
    if (schema != null) {
        BootstrapEnv env = new BootstrapEnv();
        env.dateLocaleFactory = new DateLocaleFactory(new TimeZoneRuleFactory());
        env.dateFormatFactory = new DateFormatFactory();
        JsonSchemaParser jsonSchemaParser = new JsonSchemaParser(env);
        int len = schema.length();
        long addr = Unsafe.malloc(len);
        try {
            Chars.strcpy(schema, len, addr);
            try (JsonLexer lexer = new JsonLexer(1024, 4096)) {
                lexer.parse(addr, len, jsonSchemaParser);
                lexer.parseLast();
            }
            metadata = jsonSchemaParser.getMetadata();
        } catch (JsonException e) {
            throw new IOException(e);
        } finally {
            Unsafe.free(addr, len);
        }
    }
    try (RandomAccessFile raf = new RandomAccessFile(file, "r")) {
        try (FileChannel channel = raf.getChannel()) {
            long size = channel.size();
            long bufSize = ByteBuffers.getMaxMappedBufferSize(size);
            long p = 0;
            while (p < size) {
                MappedByteBuffer buf = channel.map(FileChannel.MapMode.READ_ONLY, p, size - p < bufSize ? size - p : bufSize);
                try {
                    if (p == 0) {
                        parser.analyseStructure(ByteBuffers.getAddress(buf), buf.remaining(), sampleSize, listener, forceHeader, metadata);
                    }
                    p += buf.remaining();
                    parser.parse(ByteBuffers.getAddress(buf), buf.remaining(), Integer.MAX_VALUE, listener);
                } finally {
                    ByteBuffers.release(buf);
                }
            }
            parser.parseLast();
        }
    }
}
Also used : JsonException(com.questdb.parser.json.JsonException) BootstrapEnv(com.questdb.BootstrapEnv) ImportedColumnMetadata(com.questdb.parser.ImportedColumnMetadata) FileChannel(java.nio.channels.FileChannel) DateFormatFactory(com.questdb.std.time.DateFormatFactory) IOException(java.io.IOException) TimeZoneRuleFactory(com.questdb.std.time.TimeZoneRuleFactory) JsonLexer(com.questdb.parser.json.JsonLexer) RandomAccessFile(java.io.RandomAccessFile) MappedByteBuffer(java.nio.MappedByteBuffer) JsonSchemaParser(com.questdb.parser.JsonSchemaParser) DateLocaleFactory(com.questdb.std.time.DateLocaleFactory)

Aggregations

BootstrapEnv (com.questdb.BootstrapEnv)2 DateFormatFactory (com.questdb.std.time.DateFormatFactory)2 DateLocaleFactory (com.questdb.std.time.DateLocaleFactory)2 TimeZoneRuleFactory (com.questdb.std.time.TimeZoneRuleFactory)2 ImportedColumnMetadata (com.questdb.parser.ImportedColumnMetadata)1 JsonSchemaParser (com.questdb.parser.JsonSchemaParser)1 JsonException (com.questdb.parser.json.JsonException)1 JsonLexer (com.questdb.parser.json.JsonLexer)1 IOException (java.io.IOException)1 RandomAccessFile (java.io.RandomAccessFile)1 MappedByteBuffer (java.nio.MappedByteBuffer)1 FileChannel (java.nio.channels.FileChannel)1