Search in sources :

Example 1 with CompressionMethod

use of org.openstreetmap.osmosis.xml.common.CompressionMethod in project voltdb by VoltDB.

the class OSMImport method run.

public void run() {
    CompressionMethod compressionMethod = CompressionMethod.None;
    Sink sink = new VoltDBOsmSink(config.server);
    FastXmlReader fxr = new FastXmlReader(input, false, compressionMethod);
    fxr.setSink(sink);
    fxr.run();
}
Also used : Sink(org.openstreetmap.osmosis.core.task.v0_6.Sink) FastXmlReader(org.openstreetmap.osmosis.xml.v0_6.FastXmlReader) CompressionMethod(org.openstreetmap.osmosis.xml.common.CompressionMethod)

Example 2 with CompressionMethod

use of org.openstreetmap.osmosis.xml.common.CompressionMethod in project GeoGig by boundlessgeo.

the class OSMImportOp method parseDataFileAndInsert.

private OSMReport parseDataFileAndInsert(@Nullable File file, final InputStream dataIn, final EntityConverter converter) {
    final boolean pbf;
    final CompressionMethod compression;
    if (file == null) {
        pbf = false;
        compression = CompressionMethod.None;
    } else {
        pbf = file.getName().endsWith(".pbf");
        compression = resolveCompressionMethod(file);
    }
    RunnableSource reader;
    if (pbf) {
        reader = new OsmosisReader(dataIn);
    } else {
        reader = new org.locationtech.geogig.osm.internal.XmlReader(dataIn, true, compression);
    }
    final WorkingTree workTree = workingTree();
    if (!add) {
        workTree.delete(OSMUtils.NODE_TYPE_NAME);
        workTree.delete(OSMUtils.WAY_TYPE_NAME);
    }
    final int queueCapacity = 100 * 1000;
    final int timeout = 1;
    final TimeUnit timeoutUnit = TimeUnit.SECONDS;
    // With this iterator and the osm parsing happening on a separate thread, we follow a
    // producer/consumer approach so that the osm parse thread produces featrures into the
    // iterator's queue, and WorkingTree.insert consumes them on this thread
    QueueIterator<Feature> iterator = new QueueIterator<Feature>(queueCapacity, timeout, timeoutUnit);
    ProgressListener progressListener = getProgressListener();
    ConvertAndImportSink sink = new ConvertAndImportSink(converter, iterator, platform(), mapping, noRaw, new SubProgressListener(progressListener, 100));
    reader.setSink(sink);
    Thread readerThread = new Thread(reader, "osm-import-reader-thread");
    readerThread.start();
    Function<Feature, String> parentTreePathResolver = new Function<Feature, String>() {

        @Override
        public String apply(Feature input) {
            if (input instanceof MappedFeature) {
                return ((MappedFeature) input).getPath();
            }
            return input.getType().getName().getLocalPart();
        }
    };
    // used to set the task status name, but report no progress so it does not interfere
    // with the progress reported by the reader thread
    SubProgressListener noPorgressReportingListener = new SubProgressListener(progressListener, 0) {

        @Override
        public void setProgress(float progress) {
        // no-op
        }
    };
    workTree.insert(parentTreePathResolver, iterator, noPorgressReportingListener, null, null);
    if (sink.getCount() == 0) {
        throw new EmptyOSMDownloadException();
    }
    OSMReport report = new OSMReport(sink.getCount(), sink.getNodeCount(), sink.getWayCount(), sink.getUnprocessedCount(), sink.getLatestChangeset(), sink.getLatestTimestamp());
    return report;
}
Also used : SubProgressListener(org.locationtech.geogig.api.SubProgressListener) LineString(com.vividsolutions.jts.geom.LineString) Feature(org.opengis.feature.Feature) RunnableSource(org.openstreetmap.osmosis.core.task.v0_6.RunnableSource) Point(com.vividsolutions.jts.geom.Point) WorkingTree(org.locationtech.geogig.repository.WorkingTree) Function(com.google.common.base.Function) ProgressListener(org.locationtech.geogig.api.ProgressListener) SubProgressListener(org.locationtech.geogig.api.SubProgressListener) CompressionMethod(org.openstreetmap.osmosis.xml.common.CompressionMethod) TimeUnit(java.util.concurrent.TimeUnit) OsmosisReader(crosby.binary.osmosis.OsmosisReader)

Aggregations

CompressionMethod (org.openstreetmap.osmosis.xml.common.CompressionMethod)2 Function (com.google.common.base.Function)1 LineString (com.vividsolutions.jts.geom.LineString)1 Point (com.vividsolutions.jts.geom.Point)1 OsmosisReader (crosby.binary.osmosis.OsmosisReader)1 TimeUnit (java.util.concurrent.TimeUnit)1 ProgressListener (org.locationtech.geogig.api.ProgressListener)1 SubProgressListener (org.locationtech.geogig.api.SubProgressListener)1 WorkingTree (org.locationtech.geogig.repository.WorkingTree)1 Feature (org.opengis.feature.Feature)1 RunnableSource (org.openstreetmap.osmosis.core.task.v0_6.RunnableSource)1 Sink (org.openstreetmap.osmosis.core.task.v0_6.Sink)1 FastXmlReader (org.openstreetmap.osmosis.xml.v0_6.FastXmlReader)1