Thursday, May 10, 2007

Hexagonal Dataset Rundown

The Icosahedral Hexagonal grid system for the next version of EarthBrowser is nearly complete. This was no trivial task and it required several helper technologies in order to be possible. One of the primary motivations of this technology was to be able to represent the northern and southern latitudes as accurately any other area on the globe, which it does seamlessly.

Unfortunately the size of the datasets are not significantly smaller as I had hoped. The size of the BMNG (Blue Marble Next Generation) dataset in the generic Plate Carée projection is just over 6 Gigabytes uncompressed. In the Hexagonal grid format it comes in roughly the same. The full 500 meter per pixel hex grid consists of 162 tiles roughly 4096x4096 in size. Each hex tile image is a square in the Gnomonic Projection centered on the hexagon tile center. A similar resolution tile set in Plate Carée would be 128 tiles of size 5700x5700. Theoretically the dataset size should be much smaller, but I added about 5% padding on each edge, also each corner of the square image has redundant data that is represented in adjacent hexagons.

Compressed into Jpeg2000 format, I can get the whole dataset down to around 250 Megabytes without noticeable compression artifacts. I think that will be better than you can get from Plate Carée since land is over-represented and less compressible than ocean areas.

In order to make all of this work, I needed a nice little tool I was using to build my Landsat 15 meter dataset. The class is called mosaic and it will take a set of geo-referenced datasets in any projection and build a new geo-referenced image in any other projection. This has made creating the hexagonal tile dataset very simple. The mosaic class can read Jpeg2000, MrSid, GeoTIFF, ECW, jpeg, png or even raw images. It also take advantage of the ability of Jpeg2000 and MrSid formats to supply reduced resolution subsets of images in order to speed up the processing. In future versions I'll add netCDF and some other neat formats out there. It's all very fast too in optimized C++ code, much faster than gdal_warp.

The mosaic class is possible due to another class that has become the very heart of the new EarthBrowser program. The raster class is useful not only in importing and exporting data from image files, but also in the graphics "game" engine for vertex buffers. Raster can be used represent any block of data based on it's height, width, depth and storage type (8-bit, 16-bit unsigned, float, double, unsigned 64-bit, etc.). It can also have an arbitrary interleaved ordering: interleaved by pixel, by line or by plane (bip, bil and bsq). It can be geo-referenced with a supplied origin, resolution, rotation, projection and datum. It can also be subsampled with various sampling kernels like bilinear, cubic convolution, cubic spline and even nearest neighbor! I tried Lanczos but failed and gave up.

Due to the flexibility of the mosaic and raster classes, I build the Hexagonal dataset from BMNG and SRTM Plus (fused elevation and bathymetry) with the very same set of calls, with just a different source dataset name and output raster format. I decided to use bilinear on SRTM and cubic convolution on BMNG too. Of course all of this functionality will be available in EarthBrowser 3.0 from the Python console or from your own imported Python scripts.

EarthBrowser 3 is going to be a quantum leap from version 2. Now if I can just nail down the rights to use the i-cubed 15 meter Landsat dataset I won't have to waste my time on that again!


Peter said...

Two years late but I figure I would ask... Can EarthBrowser expose the hex grid used for the tiling?
I would be interested to see the "nodes" of the icosohedron.
Kind regards,

marko said...

This is unquestionably uplifting news to some person with awful credit! The choice to allow the advance is made exclusively on the estimation of the guarantee vehicle. In the event that you are sufficiently fortunate to claim a high-esteem vehicle, then you might be enjoyably amazed to find that you can get a genuinely high advance sum. chicago auto title loans online