Faster calling conventions, "hasty call".

[markshannon]

Cffi and argument clinic take a typed description of the arguments to a function and a low-level implementation that does no dispatch, typechecking or unboxing.
Ultimately we want to be able to call directly into those low-level functions from JIT compiled code.
With that ultimate goal in mind, we also want an intermediate calling convention suitable for calling from a specializing interpreter.
Nicknamed "hasty call" as nod to "fast call", this calling convention would leave parsing, checking and re-ordering the arguments to the caller.
The callable object would be need to provide the "hasty" function pointer and a signature to enable validation and argument reordering.

For example, consider a function with the Python signature foo(a:int, b:int, *, c:Any=None)->None
with the underlying C implementation signature void foo_impl(Py_ssize_t a, Py_ssize_t b, PyObject *c).
The "hasty call" signature for foo would be PyObject *foo_hasty(PyObject *func, PyObject *args[3])

It will be up to the caller to reorder arguments and fill in defaults, so that if we call foo(b=1, a=0)
the VM would swap a and b and push None, before calling the C function foo_hasty.

It will be up to the tooling, whether cffi, argument clinic or something else, to create foo_hasty, which in this case would look something like:

PyObject *foo_hasty(PyObject *_unused, PyObject args[3]) {
    PyObject *a_obj = args[0];
    PyObject *b_obj = args[1];
    PyObject *c = args[2];
    if (!PyLong_Check(a_obj)) {
        goto error; 
    }
    if (!PyLong_Check(b_obj)) {
        goto error; 
    }
    Py_ssize_t a, b;
    if (PyLong_ToSsize_t(a_obj, &a)) {
        goto error;
    }
    if (PyLong_ToSsize_t(b_obj, &b)) {
        goto error;
    }
    foo_impl(a, b);
    Py_RETURN_NONE;
  error:
    /* Handle error */
}

The tooling will also need to create the vectorcall form to handle argument parsing when called from
unspecialized code or from other C extensions.

[arhadthedev]

    if (!PyLong_Check(a_obj)) {
        goto error; 
    }
    if (!PyLong_Check(b_obj)) {
        goto error; 
    }

Can these checks be moved into the caller too if it knows and guarantees that a and b are integers?

[markshannon]

Not easily in the PEP 659 adaptive interpreter. For an optimized trace, probably yes.
With the tooling to generate the "hasty" call interface, we should be able to generate the typed interface fairly easily.

[kmod]

+1 on moving the type checks to the caller. I feel like if this kind of calling convention is desired, why stop at moving only some of the work to the caller? Later on, wouldn't there be the same motivation to move incrementally more of the work to the caller resulting in a "hastier" calling convention?

I'm actually in favor of also moving the unboxing work to the caller so that it essentially becomes a way of describing native calls, though I suspect that that is difficult for an interpreter much more so than moving the type checks.

[antocuni]

Ultimately we want to be able to call directly into those low-level functions from JIT compiled code.

This is also a goal of HPy, although so far we just discussed it without doing any implementation. This is probably the best summary of our thoughts so far: hpyproject/hpy#129

I think we should coordinate on this. Something to consider is how to encode the C-level signatures and how fast it is to check whether a certain callable supports a certain C-level signature. For JITs, the check can be "slow" (e.g. a strcmp()) but there are cases in which we want to be able to do a very fast check, e.g. to allow Cython to "try" to call a certain C-level signature without being able to know in advance.

I wonder if hasty-calls can be integrated in such a scheme. We could imagine to have a system in which a Python callable can support multiple signatures and calling conventions. In the examble above, the Python function foo could declare that it supports vectorcall, hasty-call and the C-level signature (void)(long, long).

In your proposal, how does a function declares that it supports hasty-calls? Do you also plan to offer a way to do hasty-calls from normal C API code, or it will be used only by JITs?

[hodgestar]

Adding self to the discussion so I can follow along.

[markshannon]

Cython to "try" to call a certain C-level signature without being able to know in advance.

I don't see how this would work, but exactly the same information is available to Cython compiled code as to the VM, so maybe they can come up with something.

In your proposal, how does a function declares that it supports hasty-calls?

With a flag. METH_HASTY I guess.

Do you also plan to offer a way to do hasty-calls from normal C API code, or it will be used only by JITs?

C call can call the the hasty call function pointer, but they caller is responsible for ensuring the arguments are in the correct order, which is tricky unless you know exactly which method you are calling in advance. In which case you might as well call the lower level C function directly.

[hodgestar]

Is the plan for a JIT to be able to avoid boxing and unboxing objects when calling Python methods backed by C implementations at some point in the future?

[markshannon]

That's up to the JIT. I expect we will want to do that eventually.

[markshannon]

I should probably elaborate on what a builtin function object supporting "hasty" call would look like in CPython.

First of all it would have set the METH_HASTY flag set in the ml_flags field, and the ml_meth field would pointer to the "hasty" C function.
If the METH_HASTY flag is set then the PyMethodDef would gain a number of extra fields.

     char **parameter_names;
     PyObject *parameter_names_object; /* Should be set to NULL, will be patched by the VM */
     uint8_t total_parameter;
     uint8_t positional_only_arguments;
     uint8_t keyword_only_arguments;
     uint8_t default_values;
     uint8_t defaults[1]; /* Index into a table of "simple" values, like `None`, `0`, etc, or `NULL`.

Default values that are not "simple", will be need to handled by setting the default value to NULL and checking in the generated hasty call.
If you want more than 256 parameters, use a different calling convention.

We could also add, in future, these fields for JITs:

     funcptr low_level_func; /* The low-level C function */
     char *low_level_desc /* String describing the low-level function's types */

[carljm]

FWIW Cinder has something like what you describe as "in the future for JITs" for builtin functions, except that the typed signature description is structured, not a string (https://github.com/facebookincubator/cinder/blob/cinder/3.8/Include/methodobject.h#L111). We use this for fast calls to builtin functions from Static Python, where we can ensure in advance that the argument types are all correct.