Search in sources :

Example 1 with Change

use of com.baremaps.osm.domain.Change in project baremaps by baremaps.

the class XmlChangeSpliteratorTest method tryAdvance.

@Test
void tryAdvance() throws IOException {
    try (InputStream input = DATA_OSC_XML.openStream()) {
        Spliterator<Change> spliterator = new OsmChangeSpliterator(input);
        spliterator.forEachRemaining(fileBlock -> assertNotNull(fileBlock));
        assertFalse(spliterator.tryAdvance(new HoldingConsumer<>()));
    }
}
Also used : InputStream(java.io.InputStream) HoldingConsumer(com.baremaps.stream.HoldingConsumer) Change(com.baremaps.osm.domain.Change) OsmChangeSpliterator(com.baremaps.osm.change.OsmChangeSpliterator) Test(org.junit.jupiter.api.Test)

Example 2 with Change

use of com.baremaps.osm.domain.Change in project baremaps by baremaps.

the class XmlChangeSpliteratorTest method forEachRemaining.

@Test
void forEachRemaining() throws IOException {
    try (InputStream input = DATA_OSC_XML.openStream()) {
        Spliterator<Change> spliterator = new OsmChangeSpliterator(input);
        AccumulatingConsumer<Change> accumulator = new AccumulatingConsumer<>();
        spliterator.forEachRemaining(accumulator);
        assertEquals(accumulator.values().size(), 7);
        assertEquals(accumulator.values().stream().flatMap(change -> change.getEntities().stream()).collect(Collectors.toList()).size(), 51);
    }
}
Also used : AccumulatingConsumer(com.baremaps.stream.AccumulatingConsumer) Assertions.assertNotNull(org.junit.jupiter.api.Assertions.assertNotNull) Change(com.baremaps.osm.domain.Change) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) Test(org.junit.jupiter.api.Test) OsmChangeSpliterator(com.baremaps.osm.change.OsmChangeSpliterator) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) DATA_OSC_XML(com.baremaps.testing.TestFiles.DATA_OSC_XML) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) HoldingConsumer(com.baremaps.stream.HoldingConsumer) Spliterator(java.util.Spliterator) InputStream(java.io.InputStream) InputStream(java.io.InputStream) Change(com.baremaps.osm.domain.Change) OsmChangeSpliterator(com.baremaps.osm.change.OsmChangeSpliterator) AccumulatingConsumer(com.baremaps.stream.AccumulatingConsumer) Test(org.junit.jupiter.api.Test)

Example 3 with Change

use of com.baremaps.osm.domain.Change in project baremaps by baremaps.

the class UpdateService method call.

@Override
public Void call() throws Exception {
    Header header = headerRepository.selectLatest();
    String replicationUrl = header.getReplicationUrl();
    Long sequenceNumber = header.getReplicationSequenceNumber() + 1;
    Consumer<Entity> createGeometry = new CreateGeometryConsumer(coordinates, references);
    Consumer<Entity> reprojectGeometry = new ReprojectEntityConsumer(4326, srid);
    Consumer<Change> prepareGeometries = new ChangeEntityConsumer(createGeometry.andThen(reprojectGeometry));
    Function<Change, Change> prepareChange = consumeThenReturn(prepareGeometries);
    Consumer<Change> saveChange = new SaveChangeConsumer(nodeRepository, wayRepository, relationRepository);
    URI changeUri = resolve(replicationUrl, sequenceNumber, "osc.gz");
    Blob changeBlob = blobStore.get(changeUri);
    ProgressLogger progressLogger = new ProgressLogger(changeBlob.getContentLength(), 5000);
    try (InputStream blobInputStream = changeBlob.getInputStream();
        InputStream progressInputStream = new InputStreamProgress(blobInputStream, progressLogger);
        InputStream gzipInputStream = new GZIPInputStream(progressInputStream)) {
        new OsmChangeParser().changes(gzipInputStream).map(prepareChange).forEach(saveChange);
    }
    URI stateUri = resolve(replicationUrl, sequenceNumber, "state.txt");
    Blob stateBlob = blobStore.get(stateUri);
    try (InputStream stateInputStream = stateBlob.getInputStream()) {
        State state = new OsmStateParser().state(stateInputStream);
        headerRepository.put(new Header(state.getSequenceNumber(), state.getTimestamp(), header.getReplicationUrl(), header.getSource(), header.getWritingProgram()));
    }
    return null;
}
Also used : Entity(com.baremaps.osm.domain.Entity) CreateGeometryConsumer(com.baremaps.osm.geometry.CreateGeometryConsumer) Blob(com.baremaps.blob.Blob) GZIPInputStream(java.util.zip.GZIPInputStream) InputStream(java.io.InputStream) InputStreamProgress(com.baremaps.osm.progress.InputStreamProgress) ProgressLogger(com.baremaps.osm.progress.ProgressLogger) Change(com.baremaps.osm.domain.Change) URI(java.net.URI) GZIPInputStream(java.util.zip.GZIPInputStream) ChangeEntityConsumer(com.baremaps.osm.function.ChangeEntityConsumer) Header(com.baremaps.osm.domain.Header) State(com.baremaps.osm.domain.State) ReprojectEntityConsumer(com.baremaps.osm.geometry.ReprojectEntityConsumer) OsmChangeParser(com.baremaps.osm.change.OsmChangeParser) OsmStateParser(com.baremaps.osm.state.OsmStateParser)

Example 4 with Change

use of com.baremaps.osm.domain.Change in project baremaps by baremaps.

the class OsmChangeSpliterator method tryAdvance.

@Override
public boolean tryAdvance(Consumer<? super Change> consumer) {
    try {
        if (reader.hasNext()) {
            int event = reader.next();
            switch(event) {
                case START_ELEMENT:
                    if (ELEMENT_NAME_OSMCHANGE.equals(reader.getLocalName())) {
                        return true;
                    }
                    Change entity = readChange();
                    consumer.accept(entity);
                    return true;
                case END_DOCUMENT:
                    return false;
                default:
                    return true;
            }
        } else {
            return false;
        }
    } catch (XMLStreamException e) {
        throw new StreamException(e);
    }
}
Also used : XMLStreamException(javax.xml.stream.XMLStreamException) Change(com.baremaps.osm.domain.Change) XMLStreamException(javax.xml.stream.XMLStreamException) StreamException(com.baremaps.stream.StreamException)

Example 5 with Change

use of com.baremaps.osm.domain.Change in project baremaps by baremaps.

the class OsmChangeSpliterator method readChange.

private Change readChange() throws XMLStreamException {
    switch(reader.getLocalName()) {
        case ELEMENT_NAME_CREATE:
        case ELEMENT_NAME_DELETE:
        case ELEMENT_NAME_MODIFY:
            ChangeType type = ChangeType.valueOf(reader.getLocalName().toUpperCase());
            List<Entity> elements = new ArrayList<>();
            reader.nextTag();
            while (reader.getEventType() == XMLStreamConstants.START_ELEMENT) {
                elements.add(readElement());
                reader.nextTag();
            }
            return new Change(type, elements);
        default:
            throw new StreamException("Unexpected XML element: " + reader.getLocalName());
    }
}
Also used : Entity(com.baremaps.osm.domain.Entity) ChangeType(com.baremaps.osm.domain.Change.ChangeType) ArrayList(java.util.ArrayList) Change(com.baremaps.osm.domain.Change) XMLStreamException(javax.xml.stream.XMLStreamException) StreamException(com.baremaps.stream.StreamException)

Aggregations

Change (com.baremaps.osm.domain.Change)5 InputStream (java.io.InputStream)3 OsmChangeSpliterator (com.baremaps.osm.change.OsmChangeSpliterator)2 Entity (com.baremaps.osm.domain.Entity)2 HoldingConsumer (com.baremaps.stream.HoldingConsumer)2 StreamException (com.baremaps.stream.StreamException)2 XMLStreamException (javax.xml.stream.XMLStreamException)2 Test (org.junit.jupiter.api.Test)2 Blob (com.baremaps.blob.Blob)1 OsmChangeParser (com.baremaps.osm.change.OsmChangeParser)1 ChangeType (com.baremaps.osm.domain.Change.ChangeType)1 Header (com.baremaps.osm.domain.Header)1 State (com.baremaps.osm.domain.State)1 ChangeEntityConsumer (com.baremaps.osm.function.ChangeEntityConsumer)1 CreateGeometryConsumer (com.baremaps.osm.geometry.CreateGeometryConsumer)1 ReprojectEntityConsumer (com.baremaps.osm.geometry.ReprojectEntityConsumer)1 InputStreamProgress (com.baremaps.osm.progress.InputStreamProgress)1 ProgressLogger (com.baremaps.osm.progress.ProgressLogger)1 OsmStateParser (com.baremaps.osm.state.OsmStateParser)1 AccumulatingConsumer (com.baremaps.stream.AccumulatingConsumer)1