Turning a function pointer into a callable

function pointer | Python Methods and Functions

The code below shows how to get the raw low-level address of a C function and how to turn it back into a callable.

Code # 1:

import ctypes

lib = ctypes.cdll.LoadLibrary ( None )

 
# Get the sin () address from the C math library

addr = ctypes.cast (lib.sin, ctypes.c_void_p) .value

print ( "addr:" , addr)

Exit:

 addr: 140735505915760 

Code # 2: Turn address into a callable function

functype = ctypes.CFUNCTYPE (ctypes. c_double, ctypes.c_double)

func = functype (addr)

print ( "Function:" , func)

Output:

 Function: & lt; CFunctionType object at 0x1006816d0 & gt; 

Code # 3: Calling the resulting function

Output:

 func (2): 0.9092974268256817 func (0): 0.0 

An instance of CFUNCTYPE must first be created for the call. The first argument is CFUNCTYPE() — this is the return type. The following arguments — these are the types of the arguments. Once the type of the function is defined, it wraps around an integer memory address to create a callable object. The resulting object is used like any normal function accessed through ctypes
It is becoming more common in programs and libraries to use advanced code generation techniques such as just-in-time compilation, which is found in libraries such as LLVM (LLVM itself is not an abbreviation; this is the full name project).

The code below uses the llvmpy extension to create a build function, get a pointer to it, and turn it into a callable Python.

Code # 4:

print ( " func (2): " , func ( 2 ))

 

print ( " func (0): " , func ( 0 ))

from llvm.core import Module, Function, Type , Builder

 

mod = Module.new ( 'example' )

f = Function.new (mod, Type . function (

Type . double (), [ Type . Double (), Type . Double ()] , False ), 'foo' )

 

block = f.append_basic_block ( 'entry' )

builder = Builder. new (block)

  

x2 = builder.fmul (f.args [ 0 ], f.args [ 0 ])

y2 = builder.fmul (f.args [ 1 ], f.args [ 1 ])

 

r = builder.fadd (x2, y2)

builder.ret (r)

Output:

 & lt; llvm.core.Instruction object at 0x10078e990 & gt; 

Code # 5:

from llvm .ee import ExecutionEngine

 

engine = ExecutionEngine.new (mod)

ptr = engine.get_pointer_to_function (f)

ptr

Output:

 4325863440 

Code # 6: Calling the resulting function

foo = ctypes.CFUNCTYPE (ctyp es.c_double, 

ctypes.c_double, 

ctypes.c_double) (ptr)

 

print (foo ( 2 , 3 ))

 

print ( " " , foo ( 4 , 5 ))

 

print ( "" , foo ( 1 , 2 ))

Output:

 13.0 41.0 5.0 




Turning a function pointer into a callable: StackOverflow Questions

Answer #1

I was asked the same question recently, and came up with several answers. I hope it"s OK to revive this thread, as I wanted to elaborate on a few of the use cases mentioned, and add a few new ones.

Most metaclasses I"ve seen do one of two things:

  1. Registration (adding a class to a data structure):

    models = {}
    
    class ModelMetaclass(type):
        def __new__(meta, name, bases, attrs):
            models[name] = cls = type.__new__(meta, name, bases, attrs)
            return cls
    
    class Model(object):
        __metaclass__ = ModelMetaclass
    

    Whenever you subclass Model, your class is registered in the models dictionary:

    >>> class A(Model):
    ...     pass
    ...
    >>> class B(A):
    ...     pass
    ...
    >>> models
    {"A": <__main__.A class at 0x...>,
     "B": <__main__.B class at 0x...>}
    

    This can also be done with class decorators:

    models = {}
    
    def model(cls):
        models[cls.__name__] = cls
        return cls
    
    @model
    class A(object):
        pass
    

    Or with an explicit registration function:

    models = {}
    
    def register_model(cls):
        models[cls.__name__] = cls
    
    class A(object):
        pass
    
    register_model(A)
    

    Actually, this is pretty much the same: you mention class decorators unfavorably, but it"s really nothing more than syntactic sugar for a function invocation on a class, so there"s no magic about it.

    Anyway, the advantage of metaclasses in this case is inheritance, as they work for any subclasses, whereas the other solutions only work for subclasses explicitly decorated or registered.

    >>> class B(A):
    ...     pass
    ...
    >>> models
    {"A": <__main__.A class at 0x...> # No B :(
    
  2. Refactoring (modifying class attributes or adding new ones):

    class ModelMetaclass(type):
        def __new__(meta, name, bases, attrs):
            fields = {}
            for key, value in attrs.items():
                if isinstance(value, Field):
                    value.name = "%s.%s" % (name, key)
                    fields[key] = value
            for base in bases:
                if hasattr(base, "_fields"):
                    fields.update(base._fields)
            attrs["_fields"] = fields
            return type.__new__(meta, name, bases, attrs)
    
    class Model(object):
        __metaclass__ = ModelMetaclass
    

    Whenever you subclass Model and define some Field attributes, they are injected with their names (for more informative error messages, for example), and grouped into a _fields dictionary (for easy iteration, without having to look through all the class attributes and all its base classes" attributes every time):

    >>> class A(Model):
    ...     foo = Integer()
    ...
    >>> class B(A):
    ...     bar = String()
    ...
    >>> B._fields
    {"foo": Integer("A.foo"), "bar": String("B.bar")}
    

    Again, this can be done (without inheritance) with a class decorator:

    def model(cls):
        fields = {}
        for key, value in vars(cls).items():
            if isinstance(value, Field):
                value.name = "%s.%s" % (cls.__name__, key)
                fields[key] = value
        for base in cls.__bases__:
            if hasattr(base, "_fields"):
                fields.update(base._fields)
        cls._fields = fields
        return cls
    
    @model
    class A(object):
        foo = Integer()
    
    class B(A):
        bar = String()
    
    # B.bar has no name :(
    # B._fields is {"foo": Integer("A.foo")} :(
    

    Or explicitly:

    class A(object):
        foo = Integer("A.foo")
        _fields = {"foo": foo} # Don"t forget all the base classes" fields, too!
    

    Although, on the contrary to your advocacy for readable and maintainable non-meta programming, this is much more cumbersome, redundant and error prone:

    class B(A):
        bar = String()
    
    # vs.
    
    class B(A):
        bar = String("bar")
        _fields = {"B.bar": bar, "A.foo": A.foo}
    

Having considered the most common and concrete use cases, the only cases where you absolutely HAVE to use metaclasses are when you want to modify the class name or list of base classes, because once defined, these parameters are baked into the class, and no decorator or function can unbake them.

class Metaclass(type):
    def __new__(meta, name, bases, attrs):
        return type.__new__(meta, "foo", (int,), attrs)

class Baseclass(object):
    __metaclass__ = Metaclass

class A(Baseclass):
    pass

class B(A):
    pass

print A.__name__ # foo
print B.__name__ # foo
print issubclass(B, A)   # False
print issubclass(B, int) # True

This may be useful in frameworks for issuing warnings whenever classes with similar names or incomplete inheritance trees are defined, but I can"t think of a reason beside trolling to actually change these values. Maybe David Beazley can.

Anyway, in Python 3, metaclasses also have the __prepare__ method, which lets you evaluate the class body into a mapping other than a dict, thus supporting ordered attributes, overloaded attributes, and other wicked cool stuff:

import collections

class Metaclass(type):

    @classmethod
    def __prepare__(meta, name, bases, **kwds):
        return collections.OrderedDict()

    def __new__(meta, name, bases, attrs, **kwds):
        print(list(attrs))
        # Do more stuff...

class A(metaclass=Metaclass):
    x = 1
    y = 2

# prints ["x", "y"] rather than ["y", "x"]

 

class ListDict(dict):
    def __setitem__(self, key, value):
        self.setdefault(key, []).append(value)

class Metaclass(type):

    @classmethod
    def __prepare__(meta, name, bases, **kwds):
        return ListDict()

    def __new__(meta, name, bases, attrs, **kwds):
        print(attrs["foo"])
        # Do more stuff...

class A(metaclass=Metaclass):

    def foo(self):
        pass

    def foo(self, x):
        pass

# prints [<function foo at 0x...>, <function foo at 0x...>] rather than <function foo at 0x...>

You might argue ordered attributes can be achieved with creation counters, and overloading can be simulated with default arguments:

import itertools

class Attribute(object):
    _counter = itertools.count()
    def __init__(self):
        self._count = Attribute._counter.next()

class A(object):
    x = Attribute()
    y = Attribute()

A._order = sorted([(k, v) for k, v in vars(A).items() if isinstance(v, Attribute)],
                  key = lambda (k, v): v._count)

 

class A(object):

    def _foo0(self):
        pass

    def _foo1(self, x):
        pass

    def foo(self, x=None):
        if x is None:
            return self._foo0()
        else:
            return self._foo1(x)

Besides being much more ugly, it"s also less flexible: what if you want ordered literal attributes, like integers and strings? What if None is a valid value for x?

Here"s a creative way to solve the first problem:

import sys

class Builder(object):
    def __call__(self, cls):
        cls._order = self.frame.f_code.co_names
        return cls

def ordered():
    builder = Builder()
    def trace(frame, event, arg):
        builder.frame = frame
        sys.settrace(None)
    sys.settrace(trace)
    return builder

@ordered()
class A(object):
    x = 1
    y = "foo"

print A._order # ["x", "y"]

And here"s a creative way to solve the second one:

_undefined = object()

class A(object):

    def _foo0(self):
        pass

    def _foo1(self, x):
        pass

    def foo(self, x=_undefined):
        if x is _undefined:
            return self._foo0()
        else:
            return self._foo1(x)

But this is much, MUCH voodoo-er than a simple metaclass (especially the first one, which really melts your brain). My point is, you look at metaclasses as unfamiliar and counter-intuitive, but you can also look at them as the next step of evolution in programming languages: you just have to adjust your mindset. After all, you could probably do everything in C, including defining a struct with function pointers and passing it as the first argument to its functions. A person seeing C++ for the first time might say, "what is this magic? Why is the compiler implicitly passing this to methods, but not to regular and static functions? It"s better to be explicit and verbose about your arguments". But then, object-oriented programming is much more powerful once you get it; and so is this, uh... quasi-aspect-oriented programming, I guess. And once you understand metaclasses, they"re actually very simple, so why not use them when convenient?

And finally, metaclasses are rad, and programming should be fun. Using standard programming constructs and design patterns all the time is boring and uninspiring, and hinders your imagination. Live a little! Here"s a metametaclass, just for you.

class MetaMetaclass(type):
    def __new__(meta, name, bases, attrs):
        def __new__(meta, name, bases, attrs):
            cls = type.__new__(meta, name, bases, attrs)
            cls._label = "Made in %s" % meta.__name__
            return cls 
        attrs["__new__"] = __new__
        return type.__new__(meta, name, bases, attrs)

class China(type):
    __metaclass__ = MetaMetaclass

class Taiwan(type):
    __metaclass__ = MetaMetaclass

class A(object):
    __metaclass__ = China

class B(object):
    __metaclass__ = Taiwan

print A._label # Made in China
print B._label # Made in Taiwan

Edit

This is a pretty old question, but it"s still getting upvotes, so I thought I"d add a link to a more comprehensive answer. If you"d like to read more about metaclasses and their uses, I"ve just published an article about it here.

Answer #2

In short, it means there are no restrictions on the object"s use. It"s the same as any other object.

A first class object is an entity that can be dynamically created, destroyed, passed to a function, returned as a value, and have all the rights as other variables in the programming language have.

Depending on the language, this can imply:

  • being expressible as an anonymous literal value
  • being storable in variables
  • being storable in data structures
  • having an intrinsic identity (independent of any given name)
  • being comparable for equality with other entities
  • being passable as a parameter to a procedure/function
  • being returnable as the result of a procedure/function
  • being constructible at runtime
  • being printable
  • being readable
  • being transmissible among distributed processes
  • being storable outside running processes

Source.

In C++ functions themselves are not first class objects, however:

  • You can override the "()" operator making it possible to have an object function, which is first class.
  • Function pointers are first class.
  • boost bind, lambda and function do offer first class functions

In C++, classes are not first class objects but instances of those classes are. In Python both the classes and the objects are first class objects. (See this answer for more details about classes as objects).

Here is an example of Javascript first class functions:

// f: function that takes a number and returns a number
// deltaX: small positive number
// returns a function that is an approximate derivative of f
function makeDerivative( f, deltaX )
{
    var deriv = function(x)
    { 
       return ( f(x + deltaX) - f(x) )/ deltaX;
    }
    return deriv;
}
var cos = makeDerivative( Math.sin, 0.000001);
// cos(0)     ~> 1
// cos(pi/2)  ~> 0

Source.

Entities that are not first class objects are referred to as second-class objects. Functions in C++ are second class because they can"t be dynamically created.

Regarding the edit:

EDIT. When one says "everything is an object" (like in Python), does he indeed mean that "everything is first-class"?

The term object can be used loosely and doesn"t imply being first class. And it would probably make more sense to call the whole concept "first class entities". But in Python they do aim to make everything first class. I believe the intent of the person who made your statement meant first class.

Tutorials