mobility_pipeline package

Submodules

mobility_pipeline.check_validation module

Script that checks the validity of data files

mobility_pipeline.check_validation.validate_data_files() → bool

Check the validity of data files

Note that these checks are computationally intensive, so they probably should not be included in an automated pipeline. Rather, they are for manual use.

Data files validated:

Returns

True if all files are valid, False otherwise.

mobility_pipeline.data_interface module

Stores the constants and functions to interface with data files

This file is specific to the data files we are using and their format.

mobility_pipeline.data_interface.ADMIN_ADMIN_TEMPLATE = 'data/brazil-towers-voronoi-mobility//%s-%s-admin-to-admin.csv'

Path to admin-to-admin matrix, accepts substitutions of country_id, day_id

mobility_pipeline.data_interface.ADMIN_GEOJSON_TEMPLATE = 'data/brazil-towers-voronoi-mobility//%s-shape.json'

Path to admin GeoJSON file, accepts substitution of country_id

mobility_pipeline.data_interface.ADMIN_SHAPE_PATH = 'data/brazil-towers-voronoi-mobility/gadm36_BRA_2'

Relative to py:const:DATA_PATH, path to administrative region shape file

mobility_pipeline.data_interface.ADMIN_TOWER_TEMPLATE = 'data/brazil-towers-voronoi-mobility//%s-admin-to-tower.csv'

Template that uses country identifier to make path to admin_tower matrix

mobility_pipeline.data_interface.COUNTRY_ID = 'br'

Country identifier

mobility_pipeline.data_interface.DATA_PATH = 'data/brazil-towers-voronoi-mobility/'

Path to folder containing towers, voronoi, and mobility data

mobility_pipeline.data_interface.MOBILITY_PATH = 'data/brazil-towers-voronoi-mobility/mobility_matrix_20150201.csv'

Relative to DATA_PATH, path to mobility CSV file

mobility_pipeline.data_interface.TOWERS_PATH = 'data/brazil-towers-voronoi-mobility/towers_br.csv'

Relative to DATA_PATH, path to towers CSV file

mobility_pipeline.data_interface.TOWER_ADMIN_TEMPLATE = 'data/brazil-towers-voronoi-mobility//%s-tower-to-admin.csv'

Template that uses country identifier to make path to tower_admin matrix

mobility_pipeline.data_interface.TOWER_PREFIX = 'br'

The tower name is the tower index appended to this string

mobility_pipeline.data_interface.VORONOI_PATH = 'data/brazil-towers-voronoi-mobility/brazil-voronoi.json'

Relative to DATA_PATH, path to Voronoi JSON file

mobility_pipeline.data_interface.convert_shape_to_json(shapefile_path_prefix: str, country_id: str) → None

Converts shapefile containing administrative regions to GeoJSON format

The GeoJSON file is saved at ADMIN_GEOJSON_TEMPLATE % country_id

Parameters

  • shapefile_path_prefix – Path to the .shp or .dbf shapefile, optionally without the file extension. Both the .shp and .dbf files must be present in the same directory and with the same name (except file extension).

  • country_id – Unique identifier for the country and admin level.

Returns

None

mobility_pipeline.data_interface.deserialize_mat(mat_path: str) → numpy.ndarray

Deserialize a matrix from a file

File must have been created by serialize_mat().

Parameters

mat_path – Path of matrix file

Returns

Deserialized matrix

mobility_pipeline.data_interface.load_admin_cells(identifier: str) → List[shapely.geometry.multipolygon.MultiPolygon]

Loads the administrative region cells

Data is loaded from ADMIN_GEOJSON_TEMPLATE % identifier. This is a wrapper function for load_polygons_from_json().

Returns

A list of the administrative region cells.

mobility_pipeline.data_interface.load_admin_tower(country_id: str) → numpy.ndarray

Load admin-to-tower matrix

Data loaded from ADMIN_TOWER_TEMPLATE % country_id.

Parameters

country_id – Country identifier

Returns

The admin-to-tower matrix

mobility_pipeline.data_interface.load_mobility(mobility_path: str) → pandas.core.frame.DataFrame

Loads mobility data from the file at mobility_path.

Returns

A pandas.DataFrame with columns ORIGIN, DESTINATION, and COUNT. Columns ORIGIN and DESTINATION contain numeric portions of tower names, represented as numpy.int. These numeric portions strictly increase in ORIGIN-major order, but rows may be missing if they would have had a COUNT value of 0.

mobility_pipeline.data_interface.load_polygons_from_json(filepath) → List[shapely.geometry.multipolygon.MultiPolygon]

Loads cells from given filepath to JSON.

Returns

A list of shapely.geometry.MultiPolygon objects, each of which describes a cell. If the cell can be described as a single polygon, the returned MultiPolygon will contain only 1 polygon.

mobility_pipeline.data_interface.load_tower_admin(country_id: str) → numpy.ndarray

Load tower-to-admin matrix

Data loaded from TOWER_ADMIN_TEMPLATE % country_id.

Parameters

country_id – Country identifier

Returns

The tower-to-admin matrix

mobility_pipeline.data_interface.load_towers(towers_path: str) → numpy.ndarray

Loads the tower positions from a file

Parameters

towers_path – Path to towers file

Returns

A matrix of tower coordinates with columns [longitude, latitude] and one tower per row. Row indices match the numeric portions of tower names.

mobility_pipeline.data_interface.load_voronoi_cells(voronoi_path: str) → List[shapely.geometry.multipolygon.MultiPolygon]

Loads cells

Parameters

voronoi_path – Path to file to load cells from

Returns

See load_polygons_from_json. Each returned object represents a Voronoi cell.

mobility_pipeline.data_interface.save_admin_admin(country_id: str, day_id: str, admin_admin: numpy.ndarray) → str

Save admin-to-admin matrix

Saved to ADMIN_ADMIN_TEMPLATE % (country_id, day_id).

Parameters

  • country_id – Country identifier

  • day_id – Day identifier

  • admin_admin – Admin-to-admin matrix to save

Returns

Path at which matrix was saved

mobility_pipeline.data_interface.save_admin_tower(country_id: str, mat: numpy.ndarray) → None

Save admin-to-tower matrix

Saved to ADMIN_TOWER_TEMPLATE % country_id.

Parameters

  • country_id – Country identifier

  • mat – Matrix to save

Returns

None

mobility_pipeline.data_interface.save_tower_admin(country_id: str, mat: numpy.ndarray) → None

Save tower-to-admin matrix

Saved to TOWER_ADMIN_TEMPLATE % country_id.

Parameters

  • country_id – Country identifier

  • mat – Matrix to save

Returns

None

mobility_pipeline.data_interface.serialize_mat(mat: numpy.ndarray, mat_path: str) → None

Save a matrix to a file

Matrix is saved such that it can be recovered by deserialize_mat().

Parameters

  • mat – Matrix to save

  • mat_path – File to save matrix to

Returns

None

mobility_pipeline.gen_country_matrices module

Generate admin-to-tower and tower-to-admin matries for a country

mobility_pipeline.gen_country_matrices.main()

Main function called when script run

mobility_pipeline.gen_day_mobility module

Generate admin-to-admin mobility matrix for a single day

mobility_pipeline.gen_day_mobility.main()

Called when script run

mobility_pipeline.plot_voronoi module

Tool for plotting the Voronoi tessellation described by the provided data

Note that the seeds of the tessellation are based on the provided towers file, not computed from the cells. The tool also prints to the console the number of towers and number of cells. Towers without an associated cell are shown in green, while other towers are shown in red.

mobility_pipeline.plot_voronoi.plot_polygon(axes: matplotlib.pyplot.axes, polygon: shapely.geometry.multipolygon.MultiPolygon) → None

Add a polygon to an axes

Parameters

  • axes – The axes to add the polygon to

  • polygon – The polygon to add

Returns

None

mobility_pipeline.visualize_overlaps module

Plots one Voronoi cell on top of all the administrative regions. The admins that intersect the Voronoi cell are colored by index and have the associated values from the tower-to-admin matrix printed. This lets you check that the matrix values seem reasonable.

mobility_pipeline.visualize_overlaps.I_TOWER_TO_COLOR = 1

Index of Voronoi cell to show.

mobility_pipeline.visualize_overlaps.main()

Main function that generates the plot

mobility_pipeline.visualize_overlaps.plot_polygon(axes: matplotlib.pyplot.axes, polygon: shapely.geometry.multipolygon.MultiPolygon, color, _label='') → None

Plot a polygon (or multipolygon) with matplotlib

Parameters

  • axes – The matplotlib axes to plot on

  • polygon – The polygon to plot

  • color – Color to use for shading the polygon

  • _label – Label for the polygon that will be displayed in the legend

Returns

None

Module contents