[bug] Arabic SSID Truncation in OpenWISP 2 with netjsonconfig on OpenWrt

[momothefox]

Arabic SSIDs (e.g., محمد, محمدين وحسنين) configured via OpenWISP 2 and netjsonconfig on OpenWrt are truncated or corrupted, missing characters (e.g., محمد appears as م ح م). Adding spaces or multiple apostrophes (') after the SSID partially resolves the issue, but requires an excessive number of apostrophes (e.g., 10 for محمدين وحسنين) to display correctly, indicating a bug in encoding, parsing, or rendering.

Environment:
OpenWISP 2 web interface, using netjsonconfig backend for OpenWrt.
OpenWrt device (e.g., with Qualcomm Atheros QCA9880 802.11ac and QCA9550 802.11bgn, as shown in the wireless scan screenshot).
Configuration applied successfully (status: "applied," as shown in the OpenWISP Configuration screenshot).
Configuration:
Navigate to the OpenWISP Configuration page for a device.
Set the wireless SSID in the netjsonconfig JSON or UCI format to an Arabic string, such as:
option ssid 'محمد'
option ssid 'محمدين وحسنين'
Observations:
Initially, option ssid 'محمد' results in truncation, displaying only م ح م (missing د) in the UCI configuration and wireless scan (see Hex Dump and Wireless Scan screenshots).
Adding a single space after محمد (e.g., option ssid 'محمد ') displays the full محمد in the OpenWISP interface but not consistently in the UCI or wireless output.
Adding a single apostrophe after محمد (e.g., option ssid 'محمد') does not work; it requires two apostrophes (e.g., option ssid 'محمد'') to parse correctly in the UCI and wireless scan.
For longer SSIDs like محمدين وحسنين (7 characters, 18 bytes in UTF-8), truncation worsens, requiring up to 10 apostrophes (e.g., option ssid 'محمدين وحسنين'''''''''') to display correctly.
Verification:
Check the UCI configuration file on the OpenWrt device (e.g., /etc/config/wireless) and wireless scan output (e.g., via iwinfo) to confirm truncation or corruption.
Use the OpenWISP interface to monitor the "Configuration status" (see Configuration screenshot, status: "applied").
Actual Behavior
Truncation:
The SSID محمد (UTF-8: D8 85 D8 AD D8 85 D8 AF, 10 bytes) is truncated to م ح م (UTF-8: D9 85 D8 AD D9 85, 8 bytes after the opening quote) in the UCI configuration, as shown in the Hex Dump screenshot.
Longer SSIDs like محمدين وحسنين (UTF-8: D9 85 D8 AD D9 85 D8 AF D9 8A D9 86 20 D9 88 D8 AD D8 B3 D9 86 D9 8A D9 86, 18 bytes) lose more characters, requiring multiple apostrophes to parse correctly.
Workaround:
Adding spaces or apostrophes after the Arabic SSID partially resolves the issue, but the number of apostrophes required grows with SSID length (e.g., 2 for محمد, 10 for محمدين وحسنين).
The wireless scan (see Wireless Scan screenshot) shows the SSID correctly only when the exact number of apostrophes is used, but this is impractical and error-prone.
Impact:
Arabic SSIDs are unusable or require manual, non-intuitive workarounds, affecting localization and usability on OpenWrt devices managed by OpenWISP 2.

Expected Behavior
The Arabic SSID should display correctly in both the OpenWISP interface, UCI configuration, and wireless scan without truncation or corruption.
For example:
option ssid 'محمد' should render as محمد (10 bytes) in the UCI and wireless output.
option ssid 'محمدين وحسنين' should render as محمدين وحسنين (18 bytes) without requiring additional spaces or apostrophes.

System Information:

• Linux 6.8.0-1020-azure Interfaces with no addresses #23~22.04.1-Ubuntu UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
• Python 3.10