Macros should be a property of a value, not a property of a binding

[afranchuk]

I've found the behavior of macro definitions confusing (as someone who's had a good amount of experience with Janet) and very different from other languages (as a PL nerd). The problem arises from the fact that defmacro only works as intended at the top-level, because the compiler only evaluates bindings with a :macro attribute as a macro. I think that (do (defmacro foo ...) (foo ...)) should work just like it does when not in a do scope. This also impacts things like macex.

I concede that, yes, a macro is just a function, however in practice a user rarely (if ever) wants to use it as a normal function (i.e., calling it to return syntax); it should operate like a macro! There should be a separate JanetType to distinguish them.

I imagine it would be cleanest to add a keyword to fn to create a macro (e.g., (fn :macro [x] ...)). On the surface this might be a breaking change, however to retain backwards-compatibility, there could be a special case on def to still accept :macro and change a function type to be a macro type instead. Since users overwhelmingly use defmacro to create macros, in practice it might not actually be a breaking change (since defmacro could be changed accordingly), but it's best to be cautious all the same.

This would also clean up hacks like as-macro.

I would be happy to contribute a patch for this (though it might be a little delayed as my free time is limited right now).

[ianthehenry]

How would this work?

(macex '(do (defmacro foo …) (foo 123))

I can’t think of how to implement macex if values are macros, since you’d need to run the code in order to compute the expansion.

I have a hard time thinking of what it means for values to be macros, although I don’t know how other languages tackle this problem. It seems like you’d just need to do runtime macro-expansion? Which seems incompatible with Janet’s compilation scheme — you lose the ability to compile a function to bytecode ahead of time.

[afranchuk]

Hmm, I understand the issue you've brought up. I would expect that particular case to do nothing: when I mentioned macex I was thinking e.g.

(do
    (defmacro foo ...)
    (macex '(foo ...)))

But yeah, emitting bytecode for a macro defined in a scope is tricky. E.g., here's a macro in a function (at macro-expand-time) that uses a parameter of the function:

(defn my-function [x]
    (defmacro uses-x [y] (if x ~(+ ,y 1) ~(- ,y 1)))
    (defmacro emits-x [y] ~(+ x ,y))
    (uses-x 1)
    (emits-x 2))

uses-x wouldn't be possible to evaluate (without embedding an eval call and potentially allowing it to access the lexical scope). emits-x is "more" possible, in a way, but only if the compiler were changed to treat macro definitions in a special way and always try to evaluate them at compile time (which IMO isn't a terrible behavior, but it would probably be a significant change). Plus the compiler would have to differentiate those two cases (probably not hard as a side effect since it'd have to have access to compile-time-scoped values). For the former case, embedding an eval call isn't that unheard of. Ultimately it's just bothersome to my purist heart that the behavior of the top-level is such a special case (obviously; see #1613), but OTOH while that is different from some other lisps and lisp-likes, it's not different from many other languages which distinguish a top-level statement scope from expression scopes.

FWIW, in the example I provide above, Janet will just happily produce code that does something the user very likely doesn't expect (producing unevaluated forms). I see that the compiler seems to lift the defmacro definitions to the top scope, which is also potentially unexpected by users. Though it does at least allow uses-x to almost work, aside from not evaluating the result. So for that reason maybe embedding an eval call is a good idea either way.

For reference, I was originally thinking that the fundamental change would be for

janet/src/core/compile.c

Lines 724 to 726 in ddc1229

	if (!(btype == JANET_BINDING_MACRO || btype == JANET_BINDING_DYNAMIC_MACRO) ||
	!janet_checktype(macroval, JANET_FUNCTION))
	return 0;

to simply be a !janet_checktype(macroval, JANET_MACRO) condition. But I see now that there are certainly a few other things that should be handled.