Designing a proxy class in CPython for a class in uPython

[kc64]

I have a class in uPython that I want to interact with in CPython. self.obj is the belay device. Rather than issuing commands like self.obj('klass.attr') to get the value of the attr, I'd rather type klass.attr in CPython and have it proxy for me. Rather than self.obj('klass.__dict__') or self.obj('klass.__class__.__dict__'), I prefer to type something like klass.get() which makes the call for me to get all the attrs of the uPython object. On CPython, my __getattr__ method looks like this.

def __getattr__(self, name):
    if name in ['__belay__', '__isabstractmethod__']:
        raise AttributeError
    return self.obj(f'{self.name}.{name}')

This works. But when I try to implement a __setattr__ method, I get the typical recursive errors issues.

Has anyone tried something like this? What method(s) did you use to successfully proxy a class in C for a uPy class?

[BrianPugh]

As an experiment, I just wrote up a ProxyObject here that seems to work. Should I add this as part of the Belay library? It has some quirks, but it satisfies your use-case.

[kc64]

Thank you for such outstanding engagement and support. It is uncommon.

This is very interesting. However, I'm going to have to study how object.__setattr__ and object.__getattribute__ work. I don't yet understand how object. works in this case.

I do have a concern which might be unwarranted. At the CPython REPL:

>>> dev.klass.foo
1
>>> dev('klass.__dict__')
{}
>>> dev('klass.__class__.__dict__')
# ...throws an exception here but gives the result:
{'some_method': <function some_method at 0x2000a760>, 'foo': 1, '__module__': '__main__', 'bar': 2, '__qualname__': 'Klass'}
>>> dev.klass.foo = 10
>>> dev('klass.__dict__')
{'foo': 10}    # sets the attr on the instance
>>> dev("klass.__class__.__dict__.get('foo')")
1   # but there is still a foo on the class

[BrianPugh]

I don't yet understand how object. works in this case.

A relatively terse explanation:

1. Basically in python, foo.bar(baz) is syntatic sugar for bar(foo, baz).
2. Even though we overwrite our class's __getattribute__ method, we can still access the "vanilla" method via the fundamental building block object class. We can apply that vanilla method to our customized class.
3. The same applies for __setattr__, this way we can avoid infinite recursion as we get/set methods.
4. This still leaves a few quirks. For example, how does __call__ not trigger our __getattribute__? Well, some methods are special.

I do have a concern which might be unwarranted. At the CPython REPL:
...
# but there is still a foo on the class

This is a sharp corner of python involving class-vs-instance attributes, as well as mutable vs immutable objects. Here's a demo/explanation:

class Foo:
    int_a = 1
    mutable = []

    def __init__(self, int_b):
        self.int_b = int_b


foo = Foo(2)

# First, lets demo `int_a`; these all point to the exact same object.
print(foo.int_a)  # 1; `int_a` doesn't exist in the `foo` **instance**, so it looks it up in the `Foo` **class**
# The following 3 prints are different ways of saying the **exact same thing**
print(Foo.int_a)  # 1
print(foo.__class__.int_a)  # 1
print(type(foo).int_a)  # 1

print()

# Now, lets set the instance `foo`'s int_a
foo.int_a = 100
print(foo.int_a)  # 100
print(foo.__class__.int_a)  # 1

# When we set `foo.int_a`, it wrote the value *just for that instance*.
# Now when we do the `foo.int_a` lookup, it's looking at it's own `int_a`, **not the classes**

print()

# But now lets see what happens when we modify a mutable object, such as a list:

foo.mutable.append("I added this string to the instance!")
print(foo.mutable)  # ['I added this string to the instance!']
print(Foo.mutable)  # ['I added this string to the instance!']
# when we did get `foo.mutable`, that attribute doesn't exist in the `foo` instance,
# but it does exist in the `Foo` **class**, so we got the list from `Foo.mutable`.
# Then, we did the `append` operation, which adds an element to that list (it's mutable).
# This means that, unless directly overwritten at the instance-level, all objects of Foo
# will share the same `mutable` list.

[BrianPugh]

So actually, slight addendum:

type(obj) does NOT trigger __getattribute__, while obj.__class__ does trigger __getattribute__, so in this very-specific and quite-small-niche-of-python, they are not identical.

[kc64]

Many thanks! I have learned a lot from you.

The ProxyObject works great. I think it would make a nice addition to the Belay library.

[BrianPugh]

sounds good! I'll try and formalize it a bit more this weekend.

[kc64]

A couple observations from my testing:

1. Executing dir(proxy_object) throws an exception.
2. Adding methods to a subclass of ProxyObject does not work.

I'll keep testing.

[BrianPugh]

if you're subclassing the ProxyObject, you might probably just be better off subclassing device and making appropriate tasks and/or properties that call device. At some point there's overlap in functionality.

[BrianPugh]

@kc64 can you try out the proxy-object branch again? I fixed the 2 issues you brought up. The subclassing has a bit of a quirk:

1. If you are setting non-proxy attributes (e.g. in your __init__), you have to use object.__setattr__, otherwise it will (understandably) set the attribute on-device.
2. Conventional getting/setting an attribute defined by (1) will just work (gets/sets it locally, not on-device).

Assuming all seems good, I'll just need to write up some docs before merging.

[kc64]

@BrianPugh, this looks really good. I've done some testing and the new feature works great.

Thank you for outstanding support and for being patient with me. I was on a beach in the Caribbean when you sent me the message. I'm settled back in and caught up enough to get back into this.

[BrianPugh]

excellent! I'll let you play around with it for a few more days; if you don't find any major issues I'll merge it in and cut a release.

[kc64]

I’m sorry, but I am out of town and away from work for the week. I will be eager to see your work when I return. One thing I want to try is something I read in my research. According to one post, if you define a new dictionary, repoint your class __dict__ to it, then it will break the __setattr__ trigger. I’ll find out.

…

On Tue, Feb 25, 2025 at 9:26 AM Brian Pugh ***@***.***> wrote: @kc64 <https://github.com/kc64> can you try out the proxy-object branch again? I fixed the 2 issues you brought up. The subclassing has a bit of a quirk: 1. If you are setting non-proxy attributes (e.g. in your __init__), you have to use object.__setattr__, otherwise it will (understandably) set the attribute on-device. 2. Conventional getting/setting an attribute defined by (1) will just work (gets/sets it locally, not on-device). Assuming all seems good, I'll just need to write up some docs before merging. — Reply to this email directly, view it on GitHub <#175 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGO2L4BYSEH2SURGY5K2OT2RR4TBAVCNFSM6AAAAABXR427BCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMZRGM4TENA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[BrianPugh]

This is now in v0.23.0! In addition to everything we previously discussed, I:

1. Added a helper method Device.proxy("some_remote_object_name_here") that creates a ProxyObject from a device.
2. Added examples/11_proxy_objects to demonstrate usage.

[kc64]

This is great! Many thanks!