[Cython] bug: constructor declarations not working

[Marcus Brinkmann]

Hi,

I want to declare and define C++ classes from Cython. Sure, I could
write them in native C++, but it's just small glue code, and I prefer to
let Cython handle the GIL and all the other good stuff.

First, for reference, the following works (c.f.
tests/run/cpp_classes_def.pyx):

== cython --cplus example1.pyx ============
cdef cppclass Foo "Foo":
int _foo
__init__(int foo) nogil:
this._foo = foo
__dealloc__() nogil:
pass
===========================================

This will emit a declaration for Foo and the implementations of the
constructor and destructor (output is truncated to show relevant parts
only):

struct Foo {
int _foo;
Foo(int);
virtual ~Foo(void);
};
Foo::Foo(int __pyx_v_foo) {
this->_foo = __pyx_v_foo;
}
Foo::~Foo(void) {
}

Now, I want to export the class to other cython files. The following
does not work, and I think it's a bug:

== example2.pxd ===========================
cdef extern cppclass Foo:
int _foo
__init__(int foo)
__dealloc__()
===========================================

== cython --cplus example2.pyx ============
cdef cppclass Foo:
__init__(int foo) nogil:
this._foo = foo
__dealloc__() nogil:
pass
===========================================

== cython --cplus example3.pyx ============
from example2 cimport Foo

cdef Foo* foo_p = new Foo(2)
===========================================

This fails for example2 with an obscure error message:

$ cython --cplus example2.pyx
example2.pyx:3:8: undeclared name not builtin: this
...more spurious errors...

For example3, Cython runs through but trying to compile shows the actual
problem (which you can also see in example2 if you shift things around a
bit):

$ g++ -fPIC -shared -o example3.so -I /usr/include/python2.7 example3.cpp
example3.cpp: In function ‘void initexample3()’:
example3.cpp:775:38: error: no matching function for call to ‘Foo::Foo(int)’
__pyx_v_8example3_foo_p = new Foo(2);

This is in the generated code:

/*--- Type declarations ---*/
struct Foo;
struct Foo {
int _foo;
virtual ~Foo(void);
};

It's missing the constructor declaration!

I traced this a bit down the Compiler/Symtab.py and found this part in
CppClassScope::declare_var:

4e07fc52 (Robert Bradshaw 2012-08-21 00:46:00 -0700 2112)
if name != "this" and (defining or name != "<init>"):
4e07fc52 (Robert Bradshaw 2012-08-21 00:46:00 -0700 2113)
self.var_entries.append(entry)

Here defining is 0, so the declaration is skipped. I don't know Cython
internals, so I am hoping someone who does can fix this easily given the
above information or point me in the right direction.

Thanks a lot for Cython, it's very useful!

Marcus Brinkmann

[Marcus Brinkmann]

Hi,

ok, this is not as easy as I originally thought. I fixed a couple of
things that prevented the suggested syntax from working, but the linking
stage has additional challenges, as there is currently no support for
cimporting a native cppclass from one module to another (akin to the
"api" feature or the "public" symbol import function feature). It's
also not sure how this should work exactly, so I am giving up.

I settled on a work around using "cdef public" C functions and making
use of cimport and the automatic symbol import feature. This does not
give access to the full type, but it's sufficient for my use case.

The following two changes were somewhat useful in my explorations:

1. Fix the "this" pointer misreference. This is an actual bug, so I
made a pull request:

https://github.com/cython/cython/pull/316

2. The missing constructor declaration generated when cimporting the pxd
file is easy to fix by not testing for "defining or name != "<init>") in
CppClassScope. However, this breaks a test case, because
NewExprNode.inferType adds the implicit default constructor under the
name "<init>" without a proper cname. So I made a larger change that
fixes this, too:

https://github.com/lambdafu/cython/tree/lambdafu/cpp_declare_constructor

I didn't make a pull request, because currently cimport'ing an external
cppclass declaration is not useful without some way to add the runtime
dependency as well...

Thanks and sorry for the noise. It wasn't entirely clear to me what the
state of cppclass support in Cython is until I read the source code.

Marcus

Post by Marcus Brinkmann
Hi,
I want to declare and define C++ classes from Cython. Sure, I could
write them in native C++, but it's just small glue code, and I prefer to
let Cython handle the GIL and all the other good stuff.
First, for reference, the following works (c.f.
== cython --cplus example1.pyx ============
int _foo
this._foo = foo
pass
===========================================
This will emit a declaration for Foo and the implementations of the
constructor and destructor (output is truncated to show relevant parts
struct Foo {
int _foo;
Foo(int);
virtual ~Foo(void);
};
Foo::Foo(int __pyx_v_foo) {
this->_foo = __pyx_v_foo;
}
Foo::~Foo(void) {
}
Now, I want to export the class to other cython files. The following
== example2.pxd ===========================
int _foo
__init__(int foo)
__dealloc__()
===========================================
== cython --cplus example2.pyx ============
this._foo = foo
pass
===========================================
== cython --cplus example3.pyx ============
from example2 cimport Foo
cdef Foo* foo_p = new Foo(2)
===========================================
$ cython --cplus example2.pyx
example2.pyx:3:8: undeclared name not builtin: this
...more spurious errors...
For example3, Cython runs through but trying to compile shows the actual
problem (which you can also see in example2 if you shift things around a
$ g++ -fPIC -shared -o example3.so -I /usr/include/python2.7 example3.cpp
example3.cpp:775:38: error: no matching function for call to
‘Foo::Foo(int)’
__pyx_v_8example3_foo_p = new Foo(2);
/*--- Type declarations ---*/
struct Foo;
struct Foo {
int _foo;
virtual ~Foo(void);
};
It's missing the constructor declaration!
I traced this a bit down the Compiler/Symtab.py and found this part in
4e07fc52 (Robert Bradshaw 2012-08-21 00:46:00 -0700 2112) if name
4e07fc52 (Robert Bradshaw 2012-08-21 00:46:00 -0700 2113)
self.var_entries.append(entry)
Here defining is 0, so the declaration is skipped. I don't know Cython
internals, so I am hoping someone who does can fix this easily given the
above information or point me in the right direction.
Thanks a lot for Cython, it's very useful!
Marcus Brinkmann
_______________________________________________
cython-devel mailing list
https://mail.python.org/mailman/listinfo/cython-devel

[Robert Bradshaw]

On Sat, Aug 16, 2014 at 9:07 PM, Marcus Brinkmann

Hi,
ok, this is not as easy as I originally thought. I fixed a couple of things
that prevented the suggested syntax from working, but the linking stage has
additional challenges, as there is currently no support for cimporting a
native cppclass from one module to another (akin to the "api" feature or the
"public" symbol import function feature). It's also not sure how this
should work exactly, so I am giving up.
I settled on a work around using "cdef public" C functions and making use of
cimport and the automatic symbol import feature. This does not give access
to the full type, but it's sufficient for my use case.
1. Fix the "this" pointer misreference. This is an actual bug, so I made a
https://github.com/cython/cython/pull/316
2. The missing constructor declaration generated when cimporting the pxd
file is easy to fix by not testing for "defining or name != "<init>") in
CppClassScope. However, this breaks a test case, because
NewExprNode.inferType adds the implicit default constructor under the name
"<init>" without a proper cname. So I made a larger change that fixes this,
https://github.com/lambdafu/cython/tree/lambdafu/cpp_declare_constructor
I didn't make a pull request, because currently cimport'ing an external
cppclass declaration is not useful without some way to add the runtime
dependency as well...
Thanks and sorry for the noise. It wasn't entirely clear to me what the
state of cppclass support in Cython is until I read the source code.

Thanks.

As you observed, defining C++ classes in Cython is still somewhat of a
beta feature.

Post by Marcus Brinkmann
Hi,
I want to declare and define C++ classes from Cython. Sure, I could
write them in native C++, but it's just small glue code, and I prefer to
let Cython handle the GIL and all the other good stuff.
First, for reference, the following works (c.f.
== cython --cplus example1.pyx ============
int _foo
this._foo = foo
pass
===========================================
This will emit a declaration for Foo and the implementations of the
constructor and destructor (output is truncated to show relevant parts
struct Foo {
int _foo;
Foo(int);
virtual ~Foo(void);
};
Foo::Foo(int __pyx_v_foo) {
this->_foo = __pyx_v_foo;
}
Foo::~Foo(void) {
}
Now, I want to export the class to other cython files. The following
== example2.pxd ===========================
int _foo
__init__(int foo)
__dealloc__()
===========================================
== cython --cplus example2.pyx ============
this._foo = foo
pass
===========================================
== cython --cplus example3.pyx ============
from example2 cimport Foo
cdef Foo* foo_p = new Foo(2)
===========================================
$ cython --cplus example2.pyx
example2.pyx:3:8: undeclared name not builtin: this
...more spurious errors...
For example3, Cython runs through but trying to compile shows the actual
problem (which you can also see in example2 if you shift things around a
$ g++ -fPIC -shared -o example3.so -I /usr/include/python2.7 example3.cpp
example3.cpp:775:38: error: no matching function for call to
‘Foo::Foo(int)’
__pyx_v_8example3_foo_p = new Foo(2);
/*--- Type declarations ---*/
struct Foo;
struct Foo {
int _foo;
virtual ~Foo(void);
};
It's missing the constructor declaration!
I traced this a bit down the Compiler/Symtab.py and found this part in
4e07fc52 (Robert Bradshaw 2012-08-21 00:46:00 -0700 2112) if name
4e07fc52 (Robert Bradshaw 2012-08-21 00:46:00 -0700 2113)
self.var_entries.append(entry)
Here defining is 0, so the declaration is skipped. I don't know Cython
internals, so I am hoping someone who does can fix this easily given the
above information or point me in the right direction.
Thanks a lot for Cython, it's very useful!
Marcus Brinkmann
_______________________________________________
cython-devel mailing list
https://mail.python.org/mailman/listinfo/cython-devel

_______________________________________________
cython-devel mailing list
https://mail.python.org/mailman/listinfo/cython-devel