Skip to content

Unify the code to recursively walk structures and make compatible with Python 3. #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

albu-diku
Copy link
Contributor

  • support tuples
  • string conversion
  • define the walking internals only once

@albu-diku albu-diku force-pushed the fix/force-recursive-py3 branch from 45d89b8 to 7f148f1 Compare July 30, 2024 07:47
@albu-diku albu-diku changed the base branch from edge to test/enforce-safeinput-tests July 30, 2024 07:47
@albu-diku albu-diku mentioned this pull request Jul 30, 2024
@albu-diku albu-diku marked this pull request as ready for review August 2, 2024 13:37
@albu-diku albu-diku force-pushed the fix/force-recursive-py3 branch from c64af9a to ca47e01 Compare August 14, 2024 11:20
@albu-diku albu-diku changed the base branch from test/enforce-safeinput-tests to edge August 14, 2024 11:21
Copy link
Contributor

@jonasbardino jonasbardino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While there's definitely some general progress and valuable unit tests here I think it needs another round of adjustments to be correct and 'complete'.
Do you feel like giving it another go or should I perhaps wrap up my comments+suggestions in another PR for you to inspect?

@albu-diku
Copy link
Contributor Author

While there's definitely some general progress and valuable unit tests here I think it needs another round of adjustments to be correct and 'complete'. Do you feel like giving it another go or should I perhaps wrap up my comments+suggestions in another PR for you to inspect?

Given it another go after discussion in-person. What we arrived at is treating the data structure walker more generically and simply passing it the function to use for evaluating whether it encountered a primitive - this actually fits much better with what it already did with _force_primitive.

Of futher note is that this PR is more like a deduplication of the walking logic at this point, mostly after noticing that it had ended up unintentionally doing two separate things - fixing tuples and refactoring the logic. The former is better done as the minimal thing required in migwsgi coverage (which is what this spun out from) and in doing so makes the two PRs independent.

@@ -488,12 +490,28 @@ def verify_local_url(configuration, req_url):
return False


def is_bytes_type(thetype):
"""Return boolean indicating if val is a unicode string. We avoid
Copy link
Contributor

@jonasbardino jonasbardino Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean "if thetype corresponds to a byte sequence / string" ... looks like unfinished copy / paste.

Comparing to bytes works but perhaps it would be slightly clearer to compare with type(b"") here for symmetry with the is_unicode_type helper. One could even refactor them into one, then, but that's another matter :-)



class MigSharedBase__force_default_fs_coding_rec(MigTestCase):
"""Unit tests of mig.shared.base force_default_fs_coding_rec()"""
Copy link
Contributor

@jonasbardino jonasbardino Aug 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version of the test suite looks a lot better in terms of correctness and coverage, as we discussed previously.
I still think we should either emphasize in the docstring or a comment that we explicitly test output contents to be byte values (b"XYZ") because we already know that the force_default_fs_coding* functions always must return bytes. Without that information one should really compare the recursive output parts against the plain force_default_fs_coding(val) in one test series and perhaps add another test series to check that the non-recursive version returns bytes to complete the chain.

Copy link
Contributor

@jonasbardino jonasbardino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only minor bits remaining now. Let me know if I should try to merge and address the rest in the process.


self.assertEqual(output, b'foobar')

def test_encode_within_a_dict(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might want to duplicate the bytes/unicode permutations from the tuple tests below for these other data types for the sake of completeness if nothing else.

@jonasbardino
Copy link
Contributor

Thanks for the latest assert changes. There are some unresolved bits above which I'm slightly uncertain about. So please let me know when you want me to review this again.

* support tuples
* string conversion
* define the walking internals only once
@albu-diku albu-diku force-pushed the fix/force-recursive-py3 branch from a34927d to a40a095 Compare September 23, 2024 13:02
@albu-diku albu-diku changed the title Make the force recursive functions work on Python 3. Unify the code to recursively walk structures and make compatible with Python 3. Oct 16, 2024
@albu-diku albu-diku marked this pull request as draft October 17, 2024 12:04
@albu-diku albu-diku closed this Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants