📥 Downloading National Flood Insurance Program (NFIP) Datasets with OpenFEMA API

Author: Mark Bauer

📌 Overview

This repository demonstrates how to download and work with the National Flood Insurance Program (NFIP) datasets from OpenFEMA. The workflows use the Python programming language and are designed to be efficient, easy to adapt, and easy to understand. These examples also include brief data exploration using DuckDB.

📥 The tutorial can be found in the download-examples notebook.

Notes:

• 📦 Datasets are downloaded in Parquet format whenever possible for performance and compatibility.
• 🐤 DuckDB is used for querying and previewing the data.
• 🚫 The NFIP Community Layers datasets are large and have been excluded from this repository (tracked via .gitignore), but other datasets are included both locally and on GitHub.
• 🗽 For the NFIP Policies and Claims datasets, the current focus is on a sample of New York City.

⚠️ Disclaimer

This tutorial uses the Federal Emergency Management Agency’s OpenFEMA API, but is not endorsed by FEMA. The Federal Government or FEMA cannot vouch for the data or analyses derived from these data after the data have been retrieved from the Agency's website(s).

⚖️ Read more about OpenFEMA's Terms and Conditions.

🌊 OpenFEMA API and the NFIP Claims and Policies Datasets

The NFIP Claims and Policies datasets are among the most widely used resources provided by OpenFEMA. However, these files are often very large, and bulk downloads can require significant storage and processing time.

🎯 If you're only interested in a few counties, it's more efficient to use the OpenFEMA API to retrieve just the data you need. The snippet below shows a simplified version of a function that fetches paginated NFIP data for a given county using the OpenFEMA API.

📄 Expand to see a preview of fetch_nfip_data()

# constants
BASE_URL = "https://www.fema.gov/api/open/v2/FimaNfip{}"
PAGE_SIZE = 10_000
ALLOWED_DATASETS = {"Claims", "Policies"}

# configure logging
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

# build OpenFEMA API URL
def build_url(dataset, fips, skip):
    return (
        f"{BASE_URL.format(dataset)}"
        "?$format=parquet"
        f"&$filter=countyCode eq '{fips}'"
        f"&$top={PAGE_SIZE}"
        f"&$skip={skip}"
    )

# download and save NFIP data
def fetch_nfip_data(dataset, fips, outpath=".", sleep_secs=2.0):
    """
    Downloads FEMA NFIP data (Claims or Policies) for a given county FIPS code,
    handling pagination and writing the result as a single Parquet file.
    """
    dataset_cap = dataset.capitalize()
    if dataset_cap not in ALLOWED_DATASETS:
        raise ValueError(f"Invalid dataset '{dataset}'. Must be one of {ALLOWED_DATASETS}")

    os.makedirs(outpath, exist_ok=True)
    output_file = os.path.join(outpath, f"{dataset.lower()}-{fips}.parquet")

    with tempfile.TemporaryDirectory() as tmpdir:
        skip = 0
        total_records = 0
        logging.info(f"Starting download for FIPS {fips}, dataset '{dataset_cap}'")

        while True:
            url = build_url(dataset_cap, fips, skip)
            logging.info(f"Requesting page: skip={skip}")
            page_path = os.path.join(tmpdir, f"page_{skip}.parquet")

            try:
                response = requests.get(url, timeout=60)
                response.raise_for_status()
                with open(page_path, "wb") as f:
    
    ...   

To download data, simply call the fetch_nfip_data() function and pass in your desired county FIPS codes.

The example below downloads NFIP Claims data for the five counties that make up 🗽 New York City:

# specify Couny FIPS codes
fips_list = [
    "36005", # Bronx
    "36047", # Kings (Brooklyn)
    "36061", # New York (Manhattan)
    "36081", # Queens
    "36085" # Richmond (Staten Island)
]

# Claims or Policies
dataset = "Claims"

# specify outpath
outpath = "data/"

# loop over County FIPS codes
for fips in fips_list:
    
    # pass variables to function
    fetch_nfip_data(dataset, fips, outpath)

🔧 Refer to the full function inside the download-examples notebook. Note: We download data as Parquet files for performance. With DuckDB, these can easily be converted to CSV. Please refer to the parquet-to-csv notebook.

🗺️ If you're interested in analyzing data at the national scale, check out my project, Analyzing FEMA's National Flood Insurance Program (NFIP) Data With DuckDB. That example demonstrates how to download the full dataset as a Parquet file and efficiently query it using DuckDB.

📂 Available Datasets

• FIMA NFIP Redacted Claims - v2
• FIMA NFIP Redacted Policies - v2
• NFIP Multiple Loss Properties - v1
• NFIP Residential Penetration Rates - v1
• 2023 NFIP Reinsurance Placement Information
• NFIP Community Layer Comprehensive - v1
• NFIP Community Layer No Overlaps Split - v1
• NFIP Community Layer No Overlaps Whole - v1
• NFIP Community Status Book - v1

📚 Additional Resources

• OpenFEMA
• OpenFEMA Datasets
• OpenFEMA National Flood Insurance Program Datasets
• OpenFEMA API Documentation
• OpenFEMA Developer Resources
• OpenFEMA Guide to Working with Large Data Sets

👋 Stay in Touch

Feel free to reach out.

• LinkedIn: markebauer
• Portfolio: mebauer.github.io
• GitHub: mebauer