Search in sources :

Example 1 with PackedAtlas

use of org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas in project atlas-generator by osmlab.

the class ShardedAtlasRDDLoaderTest method addTextAtlasResource.

private static void addTextAtlasResource(final String path, final String name) {
    final TextAtlasBuilder builder = new TextAtlasBuilder();
    final PackedAtlas packedAtlas = builder.read(new InputStreamResource(() -> ShardedAtlasRDDLoaderTest.class.getResourceAsStream(name)));
    final WritableResource packedAtlasResource = new ByteArrayResource();
    packedAtlas.save(packedAtlasResource);
    ResourceFileSystem.addResource(path, packedAtlasResource);
}
Also used : TextAtlasBuilder(org.openstreetmap.atlas.geography.atlas.builder.text.TextAtlasBuilder) WritableResource(org.openstreetmap.atlas.streaming.resource.WritableResource) PackedAtlas(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas) ByteArrayResource(org.openstreetmap.atlas.streaming.resource.ByteArrayResource) InputStreamResource(org.openstreetmap.atlas.streaming.resource.InputStreamResource)

Example 2 with PackedAtlas

use of org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas in project atlas-generator by osmlab.

the class WorldAtlasGeneratorIntegrationTest method testWorldAtlasGenerator.

@Test
public void testWorldAtlasGenerator() {
    ResourceFileSystem.addResource(PBF_RESOURCE, new InputStreamResource(() -> AtlasGeneratorIntegrationTest.class.getResourceAsStream("DMA_cutout.osm.pbf")));
    final String[] args = { "-pbf=" + PBF_RESOURCE, "-atlas=" + ATLAS_RESOURCE };
    final WorldAtlasGenerator worldAtlasGenerator = new WorldAtlasGenerator();
    WorldAtlasGenerator.setHadoopFileSystemConfiguration(ResourceFileSystem.simpleconfiguration());
    ResourceFileSystem.printContents();
    worldAtlasGenerator.runWithoutQuitting(args);
    ResourceFileSystem.printContents();
    final Optional<PackedAtlas> atlasOptional = ResourceFileSystem.getAtlas(ATLAS_RESOURCE);
    Assert.assertTrue(atlasOptional.isPresent());
    // NOSONAR
    final Atlas atlas = atlasOptional.get();
    Assert.assertEquals(112, atlas.numberOfNodes());
    Assert.assertEquals(240, atlas.numberOfEdges());
    Assert.assertEquals(327, atlas.numberOfAreas());
    Assert.assertEquals(10, atlas.numberOfLines());
    Assert.assertEquals(12, atlas.numberOfPoints());
    Assert.assertEquals(2, atlas.numberOfRelations());
}
Also used : Atlas(org.openstreetmap.atlas.geography.atlas.Atlas) PackedAtlas(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas) PackedAtlas(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas) InputStreamResource(org.openstreetmap.atlas.streaming.resource.InputStreamResource) AtlasGeneratorIntegrationTest(org.openstreetmap.atlas.generator.AtlasGeneratorIntegrationTest) Test(org.junit.Test)

Example 3 with PackedAtlas

use of org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas in project atlas-generator by osmlab.

the class AtlasProtoOutputFormat method save.

@Override
protected void save(final Atlas value, final AbstractWritableResource out) {
    final PackedAtlas packedValue = (PackedAtlas) value;
    packedValue.setSaveSerializationFormat(AtlasSerializationFormat.PROTOBUF);
    packedValue.save(out);
}
Also used : PackedAtlas(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas)

Example 4 with PackedAtlas

use of org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas in project atlas-generator by osmlab.

the class HadoopAtlasFileCacheTest method testCache.

@Test
public void testCache() {
    final File parent = File.temporaryFolder();
    final File parentAtlas = new File(parent + "/atlas");
    final File parentAtlasCountry = new File(parentAtlas + "/AAA");
    final String fullParentPathURI = "file://" + parentAtlas.toString();
    final HadoopAtlasFileCache cache = new HadoopAtlasFileCache(fullParentPathURI, new SlippyTilePersistenceScheme(SlippyTilePersistenceSchemeType.ZZ_SUBFOLDER), new HashMap<>());
    parentAtlasCountry.mkdirs();
    try {
        final PackedAtlasBuilder builder1 = new PackedAtlasBuilder();
        builder1.addPoint(1L, Location.CENTER, Maps.hashMap());
        final PackedAtlas atlas1 = (PackedAtlas) builder1.get();
        final PackedAtlasBuilder builder2 = new PackedAtlasBuilder();
        builder2.addPoint(2L, Location.CENTER, Maps.hashMap());
        final PackedAtlas atlas2 = (PackedAtlas) builder2.get();
        final File atlasFile1 = parentAtlasCountry.child("1/AAA_1-1-1.atlas");
        atlas1.save(atlasFile1);
        final File atlasFile2 = parentAtlasCountry.child("2/AAA_2-2-2.atlas");
        atlas2.save(atlasFile2);
        // cache miss, this will create the cached copy
        final Resource resource1 = cache.get("AAA", new SlippyTile(1, 1, 1)).get();
        final Resource resource2 = cache.get("AAA", new SlippyTile(2, 2, 2)).get();
        Assert.assertEquals(atlas1, PackedAtlas.load(resource1));
        Assert.assertEquals(atlas2, PackedAtlas.load(resource2));
        // cache hit, using cached copy
        final Resource resource3 = cache.get("AAA", new SlippyTile(1, 1, 1)).get();
        final Resource resource4 = cache.get("AAA", new SlippyTile(2, 2, 2)).get();
        Assert.assertEquals(atlas1, PackedAtlas.load(resource3));
        Assert.assertEquals(atlas2, PackedAtlas.load(resource4));
    } finally {
        cache.invalidate();
        parent.deleteRecursively();
    }
}
Also used : PackedAtlasBuilder(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasBuilder) SlippyTile(org.openstreetmap.atlas.geography.sharding.SlippyTile) PackedAtlas(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas) Resource(org.openstreetmap.atlas.streaming.resource.Resource) SlippyTilePersistenceScheme(org.openstreetmap.atlas.generator.persistence.scheme.SlippyTilePersistenceScheme) File(org.openstreetmap.atlas.streaming.resource.File) Test(org.junit.Test)

Example 5 with PackedAtlas

use of org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas in project atlas-generator by osmlab.

the class HadoopAtlasFileCacheTest method testCacheWithFetcher.

@Test
public void testCacheWithFetcher() {
    final File parent = File.temporaryFolder();
    final File parentAtlas = new File(parent + "/atlas");
    final File parentAtlasCountry = new File(parentAtlas + "/AAA");
    final String fullParentPathURI = "file://" + parentAtlas.toString();
    final HadoopAtlasFileCache cache = new HadoopAtlasFileCache(fullParentPathURI, "namespace", new SlippyTilePersistenceScheme(SlippyTilePersistenceSchemeType.ZZ_SUBFOLDER), uri -> Optional.ofNullable(FileSystemHelper.resource(uri.toString(), new HashMap<>())));
    parentAtlasCountry.mkdirs();
    try {
        final PackedAtlasBuilder builder1 = new PackedAtlasBuilder();
        builder1.addPoint(1L, Location.CENTER, Maps.hashMap());
        final PackedAtlas atlas1 = (PackedAtlas) builder1.get();
        final PackedAtlasBuilder builder2 = new PackedAtlasBuilder();
        builder2.addPoint(2L, Location.CENTER, Maps.hashMap());
        final PackedAtlas atlas2 = (PackedAtlas) builder2.get();
        final File atlasFile1 = parentAtlasCountry.child("1/AAA_1-1-1.atlas");
        atlas1.save(atlasFile1);
        final File atlasFile2 = parentAtlasCountry.child("2/AAA_2-2-2.atlas");
        atlas2.save(atlasFile2);
        // cache miss, this will create the cached copy
        final Resource resource1 = cache.get("AAA", new SlippyTile(1, 1, 1)).get();
        final Resource resource2 = cache.get("AAA", new SlippyTile(2, 2, 2)).get();
        Assert.assertEquals(atlas1, PackedAtlas.load(resource1));
        Assert.assertEquals(atlas2, PackedAtlas.load(resource2));
        // cache hit, using cached copy
        final Resource resource3 = cache.get("AAA", new SlippyTile(1, 1, 1)).get();
        final Resource resource4 = cache.get("AAA", new SlippyTile(2, 2, 2)).get();
        Assert.assertEquals(atlas1, PackedAtlas.load(resource3));
        Assert.assertEquals(atlas2, PackedAtlas.load(resource4));
    } finally {
        cache.invalidate();
        parent.deleteRecursively();
    }
}
Also used : PackedAtlasBuilder(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasBuilder) SlippyTile(org.openstreetmap.atlas.geography.sharding.SlippyTile) PackedAtlas(org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas) Resource(org.openstreetmap.atlas.streaming.resource.Resource) SlippyTilePersistenceScheme(org.openstreetmap.atlas.generator.persistence.scheme.SlippyTilePersistenceScheme) File(org.openstreetmap.atlas.streaming.resource.File) Test(org.junit.Test)

Aggregations

PackedAtlas (org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas)36 PackedAtlasBuilder (org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasBuilder)21 Atlas (org.openstreetmap.atlas.geography.atlas.Atlas)15 Test (org.junit.Test)13 File (org.openstreetmap.atlas.streaming.resource.File)12 ByteArrayResource (org.openstreetmap.atlas.streaming.resource.ByteArrayResource)11 Time (org.openstreetmap.atlas.utilities.time.Time)10 RelationBean (org.openstreetmap.atlas.geography.atlas.builder.RelationBean)9 AtlasSize (org.openstreetmap.atlas.geography.atlas.builder.AtlasSize)7 AtlasResourceLoader (org.openstreetmap.atlas.geography.atlas.AtlasResourceLoader)6 Path (java.nio.file.Path)5 CoreException (org.openstreetmap.atlas.exception.CoreException)5 Location (org.openstreetmap.atlas.geography.Location)5 Area (org.openstreetmap.atlas.geography.atlas.items.Area)5 Edge (org.openstreetmap.atlas.geography.atlas.items.Edge)5 Line (org.openstreetmap.atlas.geography.atlas.items.Line)5 Node (org.openstreetmap.atlas.geography.atlas.items.Node)5 Point (org.openstreetmap.atlas.geography.atlas.items.Point)5 PackedAtlasCloner (org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasCloner)5 ArrayList (java.util.ArrayList)4