These open-source tools leverage machine learning and crowdsourced data for urban development in Latin America and the Caribbean
“We don’t have data on that.” We hear these words repeatedly while working on urban development projects with cities and countries in Latin America and the Caribbean. This is problematic when we consider the region’s population is already 80% urban. Considering the digital revolution, Latin American and Caribbean cities should be poised to generate, analyze, and disseminate urban data. Yet, all too often, our partners do not have the tools necessary for delivering urban services to informal areas, identifying and allocating lands best suited for development while protecting at-risk areas, and working with individual families to assess and improve housing conditions, among other tasks. Much of the existing technology available to support this work is either too expensive, difficult to use, or not designed for the specific conditions of cities in a developing country.
Open-source tools for Urban Planning in Latin America and the Caribbean
To address these needs, the IDB’s Housing and Urban Development Division (HUD) has developed an Open Urban Planning Toolbox, a set of open-source tools to support each step of the urban development planning process, from early design through implementation and evaluation of projects. These tools were developed by leveraging academic and private sector expertise in the context of specific IDB-funded projects in Guyana: Adequate Housing and Urban Accessibility Program (GY-L1031) and Sustainable Housing for the Hinterland (GY-L1028). All included training and testing among both civil servants and beneficiary communities. By sharing the tools through the IDB’s Code for Development initiative, we invite our fellow planners and technologists to use and improve on the Open Urban Planning Toolbox.
We have launched the Toolbox with an initial set of four tools, primarily focused on the early-stage tasks of data-driven planning and design: collection, verification, and management of urban and territorial data. Two of these tools deploy machine learning techniques to identify existing structures and project future urban growth, which are critical to understand how cities are built and how best to prepare for climate-ready growth. We have also developed a tool to quickly generate neighborhood-level maps, using existing crowdsourced data. Finally, considering the importance of ‘ground-truthing’ or verifying digitally generated data through first-hand observation as well as understanding urban challenges at the neighborhood and household scale, we have also developed a digital field data collection system to ensure faster and spatially accurate information. The next sections describe these four tools in more detail and provide the links to their repositories:
1. Identify existing structures with the Building Detection Model
City planners often lack updated digital maps of existing buildings and structures. The Building Detection Model can automatically generate a basic map of buildings from satellite images. It uses semantic segmentation, which is the process of assigning each pixel in an image into a category; in this case, the categories are ‘building’ or ‘not building’. This allows planners to create detailed digital files of hard-to-reach and remote areas, such as the Guyana hinterland, where our partners are currently planning for housing interventions.
Beyond generating a map of existing structures, urban and housing agencies can use the model’s results to estimate population size, neighborhood density, access to resources, etc., and as a base across which to extrapolate household level information. In addition, these estimates can help with downstream tasks such as measuring and detecting unplanned/informal urban sprawl, informing planned interventions and the provision of urban ecosystem services to such areas.
The model, which uses machine learning techniques for the basic categorization, was trained on data from Paramaribo (Suriname), Georgetown (Guyana) and Belize City (Belize), all of which have similar climate/vegetation, styles of architecture and patterns of urban sprawl. For improved results, the model should be re-trained for a new city with different topography and sprawl patterns.
Inputs: Satellite images
Method: Semantic segmentation with Raster Vision
Outputs: Building footprints
2. Project growth with the Urban Growth Model
While urban growth prediction models are not new, they have historically required significant resources in time, expertise, and calculations. This model automates that process, allowing urban planners to spatially extrapolate the growth of their city under different scenarios (e.g., future population size, different time horizons, sensitivity to restricted areas, degree of randomness, etc.), based on past growth and current land use.
As inputs, the model uses monochrome images with standardized size and boundaries prepared from satellite images of an urban area. These images can be physical maps, density maps, or maps denoting legislative or social boundaries, depending on the conditions of the urban area and the modeler’s discretion. Attraction features will include, for instance, proximity to transit stations, and will be assigned a positive weight; restrictor features will include conditions such as flood-risk or conservation lands and will be assigned a negative weight.
Using these inputs, a regularized spatial logistic regression model predicts future urban growth on a pixel-by-pixel level within the determined boundaries, and outputs a binary raster file showing growth.
Inputs: Processed single-feature images
Method: Lasso-regularized spatial logistic regression
Outputs: Binary prediction file
3. Generate neighborhood-level maps with the OSM Extraction Tool
Even in relatively well-established urban areas, development can take place informally or without complete records. Luckily, OpenStreetMap (OSM) uses crowdsourcing to generate detailed data at the building level. It is not always easy to download this data, however. Previously, an API called Overpass has been used to pull this data, but it requires knowledge of its own specific query language, which can be a significant barrier to urban planners without training in programming.
Our OSM Extraction Tool eliminates the need to write any code and offers a more user-friendly and intuitive interface to extract relevant shapefiles and fill data gaps. The tool allows users to select specific areas; download detailed maps of roads, buildings, and other features in those areas; and update or add information to existing GIS data. Files downloaded from the OSM Extraction Tool can be conflated (selectively merged) with existing records using JOSM (Java OpenStreetMap) editor.
Inputs: Existing GIS data (if none is available, this tool can still work with no base data)
Outputs: Shapefiles (roads, buildings, waterways, etc.)
4. Empower digital field data collection with OpenMapKit
Paper records are still quite common for field data collection in many housing agencies, where the infrastructure required to create, implement, manage and maintain survey data often includes paywalls or high levels of technical expertise. OpenMapKit (OMK) solves one of the most immediate needs for government agencies to collect and manage data from the field, incorporating automatic georeferencing into the collection process.
The kit consists of an Android mobile device and three software components: the OMK Server (which allows users to store and manage survey data), OMK Survey App (which allows users to create and edit digital surveys), and the OpenDataKit (ODK) Collection App (the main interface for users to collect survey data in the field).
Surveys are created in the office with a drag-and-drop interface (including instructions and skip logic) and downloaded onto the enumerator’s android device. Once in the field, the survey can then use all the sensors and functionality available on the phone: text, numbers, photographs, videos, annotations, signatures, etc.; for geospatial data, the software connects directly to the phone’s GPS or allows manual placement of points and polygons directly on a map.
Inputs: Android phone, Designed survey
Outputs: Georeferenced survey results
What’s next for the Open Urban Planning Toolbox?
Data-driven urban planning also demands proper modelling and understanding of dynamic urban systems. Upcoming HUD projects will see tools for this kind of contextual analysis added to the Toolbox, such as estimation of nationwide housing quality deficit, and crowd-sourced mobility mapping using General Transit Feed Specification (GTFS), a format that enables open and collaborative use of urban transport data.
Open-source software is made stronger by the community that contributes to it. We welcome users to apply the tools in their own cities, share ideas for improvement, and help identify areas of need that could be addressed with new open-source tools.
As we continue to add to our Toolbox, we will always follow key values: a shared commitment to open data and open-source tech; improved access to cutting-edge techniques, such as artificial intelligence, through active collaboration with global partners; and a focus on the participatory and democratic use of digital technology.
We’d love to hear about successful applications of these tools in the comments below. Also, please feel free to reach out for new opportunities to collaborate at firstname.lastname@example.org.