Hyding Hy to win the Py

[jedesroches]

Hello !

TL;DR

Why does hy2py print code for the macros and not just the result of their execution ?

Once upon a time...

I recently discovered Hy and am having a lot of fun. I was brought here thus:

0. Have a mostly Haskell-based functional programming background
1. For $work, have to implement a lot of very similar Pydantic classes and models (which is already an improvement, since we are moving away from dictionaries + bool-based validation functions. Parse, don't validate \o/)
2. Get bored. sigh, I wish I had metaprogramming like template haskell right now
3. hmm.. are there languages that transpile to python ? => discover Hy, a lisp
4. Can I actually use this ?

And thus is my question: has it already been done, in the spirit of Rescript and JS, or of jujutsu and git, to silently write Hy while the rest of your team uses python, without them noticing "too much" ? The very, very short POC I tried to write led me to believe that was not the case, because hy2py copies over macros in the generated python code... If it were possible to write a bunch of macros / eval-when-compile functions, call the macros, run hy2py and then just hand over the generated python, it would be a massive time and effort save, all the while being more fun.

---

What I'm aiming for

class Context(RootModel[Annotated[str, StringConstraints(strict=True, pattern="^[a-z_]+$")],],
    frozen=True,
):
    """Some docstring thingy"""

What I wrote

(defmacro annot-str-model [name pattern docstr]
  (setv typ (fn [pat]
    `(get RootModel
      (get Annotated
        #(str (StringConstraints :strict True :pattern ~pat))))))
  `(defclass ~name [~(typ pattern)]
    ~docstr
  ))

(annot-str-model ContextName "^[a-z-]+$" "docstr")

What I get with hy2py

import hy

def _hy_anon_var_1(name, pattern, docstr):
    typ = lambda pat: hy.models.Expression([hy.models.Symbol('get', from_parser=True), hy.models.Symbol('RootModel', from_parser=True), hy.models.Expression([hy.models.Symbol('get', from_parser=True), hy.models.Symbol('Annotated', from_parser=True), hy.models.Tuple([hy.models.Symbol('str', from_parser=True), hy.models.Expression([hy.models.Symbol('StringConstraints', from_parser=True), hy.models.Keyword('strict', from_parser=True), hy.models.Symbol('True', from_parser=True), hy.models.Keyword('pattern', from_parser=True), pat])])])])
    return hy.models.Expression([hy.models.Symbol('defclass', from_parser=True), name, hy.models.List([typ(pattern)]), docstr])
hy.macros.macro('annot-str-model')(_hy_anon_var_1)

class ContextName(RootModel[Annotated[str, StringConstraints(strict=True, pattern='^[a-z-]+$')]]):
    """docstr"""

If macros are compiled and ran at compile time, why does the "compiled" result contain both the result of the execution of the macro (the class declaration, my goal) and a definition for the macro ? Said definition should never be run at run-time, no ? So there's no point in having its code in the final program ? What am I missing ?

[scolobb]

Hi @jedesroches ! I don't have an answer to your actual question about the macros, just the intuition that the Python code produced by hy2py contains exactly everything you wrote in Hy, including macro definitions, because everything is faithfully compiled to Python.

I have been lightly using Hy more or less on and off for about a year now, and the impression that I get is that what is coming out of hy2pymay be hard to disguise human-written Python, so I guess your team will notice quite quickly 🙂 On the other hand, maybe adding an import hy to import your modules won't bother them too much, as you can directly import Hy modules from Python provided Hy is imported as well, as you might have seen in the docs.

[jedesroches]

Hello @scolobb ! Thank you for your answer, even if it is partial ^^ The fact that more complex (or, more accurately, anything requiring mangling) code would not match human-written python is expected to me, it makes sense from a compiler point of view. Unfortunately, in this context I need the resulting modules to also be maintainable by people who are still a bit too afraid of lisp to convince to use Hy; which was why I was mostly thinking of "cheating" with macros and hy2py rather than actually writing full Hy programs.

[jedesroches]

I am still curious as to why the macros end up in the final source code though.

Conceptually, my understanding of metaprogramming in a general sense is that things happen like this:

parsing => AST with macros => macro compilation => macro execution on AST => final AST without macros => compilation => bytecode

Locally in Hy,

From what I can understand by reading the macros.py file from which the macro function comes, this -- at runtime -- adds an entry to a dictionary inside the globals of a namespace with the same name as the anonymous function generated by hy to represent the macro (since it is called with install_macro(name, fn, fn) with the name pointing to the function body. Said function returns a valid hy.model which, I imagine, could perhaps be used with a combination of hy magic and exec or eval ?

Is that understanding correct, and if yes, are there examples of how this is useful ?

[Kodiologist]

Why does hy2py print code for the macros and not just the result of their execution ?

In case you want to use macros at runtime, or in another module that requires this one. Consider a program like (defmacro m [] 1) (print (hy.eval '(m))). This can only work in Python if hy2py keeps macro definitions.

Removing macro definitions is the kind of post-hy2py surgery you can do if you want cleaner Python and you know that you don't need the definitions.

[jedesroches]

Thank you for both the what happens and why - especially the require argument makes sense. And actually, I don't need post-processing, because the language already has the answer and I hadn't thought it would work: if I eval-when-compile the defmacro... it works ! I just get an import hy which is easily removed by post-processing with e.g. ruff.