Skip to content

DEV - Update huggingface repo for 2025 data. #1895

@traycn

Description

@traycn

Dependency

Overview

We need to remove a hotfix that loads the 2024 data instead of 2025 data, as the application is not usable in this state.

More Info

In the beginning of 2025, we decided to apply a hotfix to the API layer of our code to load 2024 data (since the 2025 data didn't exist yet). This now has become a big problem because, not only are we waiting for all of 2024 data to load into memory(10-15s), we still don't show any data on the screen (since the datepicker component dictates which data to render).

Action Items

In scripts/updateHfDataset.py

  • in dlData(), replace the 2024 url (see R1) with the new 2025 url (see R2)
  • replace ALL references of 2024*.* with the equivalent 2025*.*
    • e.g. 2024-fixed.csv becomes 2025-fixed.csv

In scripts/updateHfDataset_FilterByBoundaries.py

  • replace ALL references of 2024*.* with the equivalent 2025*.*

In DbProvider.jsx

  • update the datasets.hfYtd2025 object entry to match R3
  • modify the registerFileURL() call for requests2025.parquet to point to your corrected datasets.hfYtd2025

Testing

  • wait for the dependent issue to be completed and merged into dev
  • pull from the dev branch to include the changes from the dependent issue
  • run scripts/updateHfDataset.py locally and confirm it populates the 2025 repo on the 311-data-dev HuggingFace account
  • run the application and confirm it pulls data from the 311-data-dev HuggingFace repo

Resources

R3: Screenshots for downloading 2025 Data Locally

  1. Navigate to MyLA311 Service Request Data 2025

  2. Click on Actions in header bar and select API menu option

Image

  1. Ensure API Endpoint is selected, select data format (e.g. JSON), click Copy to Clipboard

Image


Dev Resources

  • R1: 2024_SOCRATA_API_URL
https://data.lacity.org/api/views/b7dx-7gc3/rows.csv?accessType=DOWNLOAD
  • R2: 2025_SOCRATA_API_URL
https://data.lacity.org/resource/h73f-gn57.csv
  • R3: 2025_PARQUET_URL
https://huggingface.co/datasets/311-data/2024/resolve/main/2024.parquet

Metadata

Metadata

Assignees

No one assigned

    Labels

    Complexity: Smalltickets that are prescriptive, easily understood and executableDependencyAn issue that includes dependenciesRole: FrontendReact front end workp-feature: Mapassociated with the MAP page for public viewing/interactionsize: 1ptCan be done in 6 hours

    Type

    No type

    Projects

    Status

    Icebox (on hold)

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions