lib: posix: path to deprecate POSIX_API

[JordanYates]

Cleanup a bunch of in-tree users of POSIX_API by explicitly enabling the POSIX features they need, instead of the relying on the options enabled by POSIX_API.

Add a dedicated symbol for injecting include/zephyr/posix into the include path, so that users can opt-in to the headers without also getting POSIX_API.

[cfriedt]

This will be good for weeding-out the last remaining users of CONFIG_POSIX_API. I'd like to deprecate that option prior to the next release.

[nashif]

it would be nice to not have any of the POSIX kconfig options show up in the .config when POSIX is not enabled, i.e. when I build hello world ( west build -p -b qemu_x86 samples/hello_world/ ) right now and with this PR, I get:

$ grep POSIX build/zephyr/.config

# POSIX API Support
# POSIX Options
# CONFIG_POSIX_API is not set
# CONFIG_POSIX_ASYNCHRONOUS_IO is not set
# CONFIG_POSIX_BARRIERS is not set
# CONFIG_POSIX_C_LANG_SUPPORT_R is not set
# CONFIG_POSIX_C_LIB_EXT is not set
# POSIX device I/O
# CONFIG_POSIX_DEVICE_IO is not set
CONFIG_POSIX_OPEN_MAX=4
# end of POSIX device I/O
# CONFIG_POSIX_FD_MGMT is not set
# CONFIG_POSIX_FILE_SYSTEM_R is not set
# CONFIG_POSIX_FILE_SYSTEM is not set
# POSIX memory
CONFIG_POSIX_PAGE_SIZE=0x1000
# CONFIG_POSIX_SHARED_MEMORY_OBJECTS is not set
# CONFIG_POSIX_MAPPED_FILES is not set
# CONFIG_POSIX_MEMORY_PROTECTION is not set
# end of POSIX memory
# CONFIG_POSIX_MESSAGE_PASSING is not set
# CONFIG_POSIX_SINGLE_PROCESS is not set
# CONFIG_POSIX_MULTI_PROCESS is not set
# CONFIG_POSIX_THREADS is not set
# CONFIG_POSIX_READER_WRITER_LOCKS is not set
# POSIX scheduler options
# CONFIG_POSIX_PRIORITY_SCHEDULING is not set
# end of POSIX scheduler options
# CONFIG_POSIX_SEMAPHORES is not set
# POSIX signals
# CONFIG_POSIX_REALTIME_SIGNALS is not set
# CONFIG_POSIX_SIGNALS is not set
# end of POSIX signals
# CONFIG_POSIX_SPIN_LOCKS is not set
# POSIX synchronized I/O
# CONFIG_POSIX_FSYNC is not set
# CONFIG_POSIX_SYNCHRONIZED_IO is not set
# end of POSIX synchronized I/O
# CONFIG_POSIX_TIMERS is not set
# Miscellaneous POSIX-related options
# end of Miscellaneous POSIX-related options
CONFIG_TC_PROVIDES_POSIX_C_LANG_SUPPORT_R=y
# end of POSIX Options
# POSIX Shell Utilities
# end of POSIX Shell Utilities
# end of POSIX API Support

Why any of this needs to ALWAYS be in the .config if POSIX is not enabled? We do not do this for Bluetooth or for Networking, why do we need to have all those options in the resulting .config if posix is not enabled?

To illustrate this, when you build west build -p -b qemu_x86 samples/bluetooth/central_hr/, you get:

$ grep CONFIG_BT  build/zephyr/.config  | wc -l
259

which is fine, because you we have enabled BT.

when you build hello world:

$ grep CONFIG_BT  build/zephyr/.config
# CONFIG_BT is not set

Thats how it should be

We should do something about this.

[JordanYates]

it would be nice to not have any of the POSIX kconfig options show up in the .config when POSIX is not enabled, i.e. when I build hello world ( west build -p -b qemu_x86 samples/hello_world/ ) right now and with this PR, I get:

I agree with this statement, but as you note this is the current state both on main and in this PR. This PR is trying to limit the number of features that are enabled for users that need some individual piece of the POSIX API, not trying to fix the issue you describe.

[JordanYates]

@cfriedt The remaining twister test that is failing is already flaky on main:
west build -b qemu_x86_64/atom tests/lib/c_lib/thrd -t run will occasionally hang forever.

[jukkar]

Why do we need to manually enable CONFIG_POSIX_NETWORKING everywhere? Couldn't we select it automatically if CONFIG_NETWORKING and CONFIG_POSIX_API are set?

[JordanYates]

Why do we need to manually enable CONFIG_POSIX_NETWORKING everywhere? Couldn't we select it automatically if CONFIG_NETWORKING and CONFIG_POSIX_API are set?

It is not a given that everyone that uses NETWORKING and POSIX_API also wants POSIX_NETWORKING. Maybe they only want POSIX_THREADS, or POSIX_COUNTERS, or w/e. There is also the consideration that @cfriedt wants to deprecate POSIX_API entirely, so there is the question of what symbol are you enabling POSIX_NETWORKING based on?

[jukkar]

There is also the consideration that @cfriedt wants to deprecate POSIX_API entirely

Yes, I have heard about this. I have not yet understood why as the POSIX_API could act as a master switch for enabling Posix support. So similar as what CONFIG_NETWORKING does for net.

It is not a given that everyone that uses NETWORKING and POSIX_API also wants POSIX_NETWORKING.

Perhaps we could do imply then and allow user to disable POSIX_NETWORKING if needed. I am just trying to brainstorm here to avoid this large number of changes in this PR.

[nashif]

it would be nice to not have any of the POSIX kconfig options show up in the .config when POSIX is not enabled, i.e. when I build hello world ( west build -p -b qemu_x86 samples/hello_world/ ) right now and with this PR, I get:

I agree with this statement, but as you note this is the current state both on main and in this PR. This PR is trying to limit the number of features that are enabled for users that need some individual piece of the POSIX API, not trying to fix the issue you describe.

right, not suggesting this should be done as part of this PR :)
There is a bug about this already: #75843

btw, there is another issue with how kconfigs of posix are implemented, for example I can now do:

west build -p -b qemu_x86 samples/hello_world/ -- -DCONFIG_POSIX_TIMERS=y and nothing prevents me from enabling POSIX_TIMERS directly, but this Kconfig has no dependencies whatsoever, which in this case leads to compile errors. We should not be able to configure such options if the underlying subsystem is not enabled.

[cfriedt]

it would be nice to not have any of the POSIX kconfig options show up in the .config when POSIX is not enabled

@nashif - I have a PR in draft this address that right now. Should be merged before 4.2.0, I think. #88547

Also, @jukkar - please see the above PR. Should offer clarity.

[cfriedt]

There is also the consideration that @cfriedt wants to deprecate POSIX_API entirely, so there is the question of what symbol are you enabling POSIX_NETWORKING based on?

In #88547, POSIX Option Groups that do not require involvement from the os (e.g. string manipulation functions) are separated from POSIX Option Groups that do require OS involvement.

This allows us to very clearly gate most POSIX Option Groups with a Kconfig, removing most of them entirely from e.g. hello world.

I chose CONFIG_POSIX_SYSTEM_INTERFACES because I didn't like the idea of reusing CONFIG_POSIX_API with it's ad-hoc defaults and the baggage that comes with that, but naming for this new option is flexible. The main idea was to deprecate CONFIG_POSIX_API gracefully according to the process. We could reuse that option if the arch wg thinks that's a good idea.

[cfriedt]

This is useful for highlighting places where CONFIG_POSIX_API should be removed, but I don't think it's the right approach.

The deprecation of CONFIG_POSIX_API has been documented and processing for some time. I don't think we should be using it going forward, since it has a conflicting definition that has been used previously.

Zephyr does use a few "standalone" POSIX API calls by default as per coding guidelines. These are mainly for string processing (strnlen, srtok_r). Some time functions are also used (clock_gettime, clock_settime, gmtime_r, gettimeofday), only one of which is included in the coding guidelines. The remaining time functions should be factored-out of the various areas and native Zephyr functions should be used instead.

A more standards-compatible approach that is congruent with what Zephyr uses should be available soon in #88547. Maybe this option could be used as a stepping stone, but I kind of fear it would add a lot of unnecessary churn for an API that should be considered frozen / deprecated.

A few PRs are waiting to be reviewed / merged before 88547 will be open for review.

Thanks for highlighting many of the places where CONFIG_POSIX_API is still used in-tree.

I don't want my NAK here to be seen as n native though by any means. Thanks @JordanYates for helping to move this forward🙏

[cfriedt]

I would say the right strategy here is to deprecate the option and remove it in-tree rather than change its behaviour.

[cfriedt]

Not the ideal direction. See #88547

[JordanYates]

Updated to no longer change the POSIX_API symbol select and imply options.
Adds a dedicated symbol for the POSIX headers so that users can move away from POSIX_API before the official deprecation.

The actual deprecation of POSIX_API (through select DEPRECATED) is not done here, since we want #88547 merged first.