Uber H3 API examples on Urban Analytics in the city of Toulouse (France)

#Uber-H3-API-examples-on-Urban-Analytics-in-the-city-of-Toulouse-(France)

Author: Camelia Ciolac

#Author:-Camelia-Ciolac

ciolac_c@inria-alumni.fr

"The H3 geospatial indexing system is a multi-precision hexagonal tiling of the sphere indexed with hierarchical linear indexes." (https://github.com/uber/h3/blob/master/docs/overview/mainpage.md)

We use the following methods from H3 Python API:

  • h3.edge_length
  • h3.hex_area
  • h3.geo_to_h3
  • h3.h3_to_geo_boundary
  • h3.h3_get_resolution
  • h3.k_ring_distances
  • h3.h3_distance
  • h3.polyfill
  • h3.compact
  • h3.hex_ring

For data visualization we use:

  • Folium
  • deck.gl (Uber's large-scale WebGL-powered data visualization framework)

Setup steps

#Setup-steps

Virtual environment was set up as follows:

1
2
3
4
5
6
7
virtualenv -p python3 ./projenv_demo_h3   

source projenv_demo_h3/bin/activate  

pip3 install ipython==7.2.0 jupyter==1.0.0  

jupyter notebook  

Now, we list this demo's dependencies in a file called requirements_demo.txt

Use pip to install the dependencies of packages listed in our requirements_demo.txt file and verify the installation

Data sources for the examples:

#Data-sources-for-the-examples:

Bus stops: https://data.toulouse-metropole.fr/explore/dataset/arrets-de-bus0/information/

City subzones: https://data.toulouse-metropole.fr/explore/dataset/communes/information/

Residential districts: https://data.toulouse-metropole.fr/explore/dataset/recensement-population-2015-grands-quartiers-logement/information/

Notes:

1). We analyze only bus stops data for this example, however, the city of Toulouse has also trams and metro (underground) as part of the urban public transport network.
2). The download was performed on November 30th 2018

1. Preliminaries

#1.-Preliminaries

For various resolutions, how large are the hexagons at that level of the H3 spatial index:

Loading output library...

To better make sense of resolutions, we index spatiallyy with H3 a central GPS point of the French city Toulouse:

Loading output library...

Note that the index representation in H3 uses 64 bit integers, unlike Geohash indexes that use alphanumeric strings
(refer to https://github.com/uber/h3/blob/master/docs/overview/usecases.md#comparisons)

Visualize on map:

Loading output library...

Notes:

-the color scheme of hexagons boundaries was coded with green for even resolution (0,2,4,etc) and red of odd resolution(1,3,5,etc)

-starting from resolution 5 upwards it focuses on the region of the city under analysis

I. Use H3 spatial index for bucketing bus stops locations

#I.-Use-H3-spatial-index-for-bucketing-bus-stops-locations

2. Prepare date - GeoJSON file of bus stops

#2.-Prepare-date---GeoJSON-file-of-bus-stops
Loading output library...
Loading output library...

3. Index data spatially with H3

#3.-Index-data-spatially-with-H3
Loading output library...

4. Visualization with choropleth map

#4.-Visualization-with-choropleth-map
Loading output library...

5. Aggregate at a coarser resolution

#5.-Aggregate-at-a-coarser-resolution

Note: we can control in the folium map which layers to be visible, possible multiple overlayed at once

Loading output library...
Loading output library...
Loading output library...

6. K-rings and simple local outlier detection

#6.-K-rings-and-simple-local-outlier-detection
Loading output library...

Visualize local outliers on map:

Loading output library...

7. The distance (in number of hexagons) between the hexagon that indexes the city center (from the Preliminaries section) and each "cold" local outlier identified

#7.-The-distance-(in-number-of-hexagons)-between-the-hexagon-that-indexes-the-city-center-(from-the-Preliminaries-section)-and-each-"cold"-local-outlier-identified
Loading output library...

II. Fill Polygon with hexagons from H3 index

#II.-Fill-Polygon-with-hexagons-from-H3-index

8. Prepare data - GeoJSON file containing the city's subzones

#8.-Prepare-data---GeoJSON-file-containing-the-city's-subzones
Loading output library...

There are multiple subzones that form Toulouse territory, we choose for analysis the subzone named "Toulouse" which is located at the center of the Metropolitan area.

Loading output library...
Loading output library...

9. Fill a selected city subzone with hexagons from the index at the resolution 9

#9.-Fill-a-selected-city-subzone-with-hexagons-from-the-index-at-the-resolution-9

Note: if we take one hexagon used for filling the polygon of the subzone, and print its geometry, we'll discover that the coordinated are inverted as latitude,longitude. This did not happen with the hexagons used in the previous sections.

Loading output library...

10. We can approximate the area of the subzone polygone using the number of hexagons that filled it and the area of hexagon at resolution 9 (see Preliminaries section of this notebook)

#10.-We-can-approximate-the-area-of-the-subzone-polygone-using-the-number-of-hexagons-that-filled-it-and-the-area-of-hexagon-at-resolution-9-(see-Preliminaries-section-of-this-notebook)
Loading output library...

11. Merge (by left outer join) two H3 spatially indexed datasets having the same H3 index resolution

#11.-Merge-(by-left-outer-join)-two-H3-spatially-indexed-datasets-having-the-same-H3-index-resolution
Loading output library...
Loading output library...

Visualize on map

Visualize zeros (which filled nulls) in sienna(shade of brown) color

Loading output library...

12. Compacting the hexagons that fill a polygon

#12.-Compacting-the-hexagons-that-fill-a-polygon
Loading output library...

So there are 14 hexagons at resolution 7, 45 hexagons at resolution 8 and 193 hexagons at resolution 9 to fill this ity subzone's polygon

Loading output library...

III. Use H3 spatial index for spatial search (K - Nearest Neighbors)

#III.-Use-H3-spatial-index-for-spatial-search-(K---Nearest-Neighbors)

Besides bucketing locations for aggregation purposes, the spatial index can be used to speed up spatial queries.

Loading output library...
Loading output library...

We have a MultiIndex with 15 levels

Loading output library...

13. Finding K Nearest Neighbors (KNN) of a bus stop

#13.-Finding-K-Nearest-Neighbors-(KNN)-of-a-bus-stop
Loading output library...

Visualize on map

Loading output library...
Loading output library...

Now knn with k = 50 at resolution 9

Now knn with k = 50 at resolution 8

IV. Use H3 spatial index for optimized spatial join (Point within Polygon)

#IV.-Use-H3-spatial-index-for-optimized-spatial-join-(Point-within-Polygon)

14. Prepare data - GeoJSON file of the city's districts (finer granularity than subzones)

#14.-Prepare-data---GeoJSON-file-of-the-city's-districts-(finer-granularity-than-subzones)
Loading output library...

Compute area of districts:

Loading output library...
Loading output library...

Visualize all residential districts on map

Loading output library...

15. Fill all districts polygons with H3 hexagons at resolution 10 and build the Inverted Index

#15.-Fill-all-districts-polygons-with-H3-hexagons-at-resolution-10-and-build-the-Inverted-Index

Fill all districts polygons with hexagons from H3 index at resolution 10

Loading output library...

Expand lists into columns

Loading output library...

Visualize on map

Loading output library...

Create an inverted index hexagon (of H3 at resolution 10) -> districts in which it appears

Loading output library...
Loading output library...

Sum on lines and columns in this inverted index

Loading output library...
Loading output library...

16. Spatial join index-based (Point in Polygon)

#16.-Spatial-join-index-based-(Point-in-Polygon)

Now, using the newly created inverted index and the dataframe of points indexed in H3 at resolution 10. In the implementation of spatial_join below we make use of the known fact that polygons are non-overlapping. For other cases, a two-step processing is needed (first identify candidates, then do the point-in-polygon check with the subset of candidates).

Loading output library...
Loading output library...

Insights into the bus network and districts based on the spatial join result

#Insights-into-the-bus-network-and-districts-based-on-the-spatial-join-result
  • Insights by district: how many stops on each direction do the bus lines that cross the district
Loading output library...
  • Insights by bus line: which districts are served by a given bus line
Loading output library...
Loading output library...

Visualize on map

Loading output library...
Loading output library...

V. Bonus: 3D visualization with deck.gl

#V.-Bonus:-3D-visualization-with-deck.gl

17. deck.gl Hexagon layer based on bus locations points

#17.-deck.gl-Hexagon-layer-based-on-bus-locations-points

"By default, the HexagonLayer uses d3-hexbin as hexagonAggregator" (https://github.com/uber/deck.gl/blob/master/docs/layers/hexagon-layer.md)

Loading output library...
Loading output library...
Loading output library...

18. deck.gl GeoJSON layer based on the aggregation in hexagons at H3 index resolution 7

#18.-deck.gl-GeoJSON-layer-based-on-the-aggregation-in-hexagons-at-H3-index-resolution-7
Loading output library...

19. deck.gl GeoJSON layer based on the aggregation in districts after spatial join

#19.-deck.gl-GeoJSON-layer-based-on-the-aggregation-in-districts-after-spatial-join

Remember that here we work with all bus lines passing through stops, so a bus stop is counted multiple times (equal to number of bus lines stopping at it).

Loading output library...
Loading output library...

20. deck.gl Arc layer and Scatterplot layer for the route of bus 14

#20.-deck.gl-Arc-layer-and-Scatterplot-layer-for-the-route-of-bus-14
Loading output library...
Loading output library...
Loading output library...
Loading output library...
Loading output library...

The end.

#The-end.

Appendix