Skip to content

Create a job Queue for remote/parallel job execution #61

@fccoelho

Description

@fccoelho

It would be very nice to create a simple decorator people could use to submit jobs, as many distributed processing libraries have: People could decorate functions they want to offload to a pipelinin queue for distributed processing. I have something similar in my Liveplots package: https://github.com/fccoelho/liveplots/blob/master/src/liveplots/xmlrpcserver.py

def enqueue(f):
    """Decorator that places the call on a queue"""
    def queued(self,*args,**kw):
        Q.put((f,(self,)+args))
    queued.__doc__ = f.__doc__
    queued.__name__ = f.__name__
    return queued

where Q is a Queue object.

Also a customized map/reduce (this could be lifted from other python map/reduce libraries) framework would see great adoption.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions