Cython 0.21 alpha 1 released

Hi all,

given the overwhelming lack of negative feedback on the latest alpha
release, and the recent changes that made it into the master branch, I
think it's time for a first beta release.

http://www.cython.org/release/Cython-0.21b1.tar.gz

http://www.cython.org/release/Cython-0.21b1.tar.gz.asc

This release features faster calls to C implemented Python functions and
methods in simple (non-kwargs) cases, so you might see a net win if your
code calls a lot into previously unoptimised builtins or other native
extension modules (including Cython modules). There's also a bit of a
speedup for Python method calls in general, in addition to the long list of
improvements that were already in the last alpha.

Complete changelog:

https://github.com/cython/cython/blob/ebafd54916e1fe976a282e77b4d59577c793bfca/CHANGES.rst

Please give it a try with your code. Any regressions that you find now have
a chance of being fixed before the final release. Positive success stories
are also appreciated.

Have fun,

Stefan

Daπid

2014-08-12 19:44:31 UTC

Post by Stefan Behnel
Please give it a try with your code. Any regressions that you find now have
a chance of being fixed before the final release. Positive success stories
are also appreciated.

Hi,

I have tested it against numpy and scipy master on Fedora 20 and only got
one error in Scipy, I was getting it before too:

======================================================================
ERROR: test_fitpack.TestSplder.test_kink
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File
"/usr/lib64/python2.7/site-packages/scipy/interpolate/tests/test_fitpack.py",
line 329, in test_kink
splder(spl2, 2) # Should work
File "/usr/lib64/python2.7/site-packages/scipy/interpolate/fitpack.py",
line 1198, in splder
"and is not differentiable %d times") % n)
ValueError: The spline has internal repeated knots and is not
differentiable 2 times

Arfrever Frehtes Taifersar Arahesis

2014-08-14 08:18:17 UTC

Results of test suite of Cython 0.21 beta 1 with different versions of Python:
Python 2.6: 1 error, 2 failures
Python 2.7: 1 error, 0 failures
Python 3.2: 1 error, 0 failures
Python 3.3: 1 error, 0 failures
Python 3.4: 1 error, 0 failures

The 1 error is the same with all versions of Python.
Output with Python 2.6:

======================================================================
ERROR: compiling (c) and running bufaccess
----------------------------------------------------------------------
Traceback (most recent call last):
File "runtests.py", line 1022, in run
check_thread_termination()
File "runtests.py", line 1633, in check_thread_termination
raise PendingThreadsError("left-over threads found after running test")
PendingThreadsError: left-over threads found after running test

======================================================================
FAIL: Doctest: double_dealloc_T796
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib64/python2.6/doctest.py", line 2163, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for double_dealloc_T796
File "/tmp/Cython-0.21b1/tests-2.6/run/c/double_dealloc_T796/double_dealloc_T796.so", line unknown line number, in double_dealloc_T796

----------------------------------------------------------------------
File "/tmp/Cython-0.21b1/tests-2.6/run/c/double_dealloc_T796/double_dealloc_T796.so", line ?, in double_dealloc_T796
Failed example:
del x
Expected:
SimpleGarbage(1) __dealloc__
Collector.__dealloc__
collect 0
Got:
SimpleGarbage(1) __dealloc__
Collector.__dealloc__
collect 128

======================================================================
FAIL: Doctest: double_dealloc_T796
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib64/python2.6/doctest.py", line 2163, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for double_dealloc_T796
File "/tmp/Cython-0.21b1/tests-2.6/run/cpp/double_dealloc_T796/double_dealloc_T796.so", line unknown line number, in double_dealloc_T796

----------------------------------------------------------------------
File "/tmp/Cython-0.21b1/tests-2.6/run/cpp/double_dealloc_T796/double_dealloc_T796.so", line ?, in double_dealloc_T796
Failed example:
del x
Expected:
SimpleGarbage(1) __dealloc__
Collector.__dealloc__
collect 0
Got:
SimpleGarbage(1) __dealloc__
Collector.__dealloc__
collect 128

----------------------------------------------------------------------
Ran 8832 tests in 6785.081s

FAILED (failures=2, errors=1)
ALL DONE

--
Arfrever Frehtes Taifersar Arahesis

Stefan Behnel

2014-08-28 18:39:54 UTC

Hi,

thanks for testing.

Post by Arfrever Frehtes Taifersar Arahesis
Python 2.6: 1 error, 2 failures
Python 2.7: 1 error, 0 failures
Python 3.2: 1 error, 0 failures
Python 3.3: 1 error, 0 failures
Python 3.4: 1 error, 0 failures
The 1 error is the same with all versions of Python.
======================================================================
ERROR: compiling (c) and running bufaccess
----------------------------------------------------------------------
File "runtests.py", line 1022, in run
check_thread_termination()
File "runtests.py", line 1633, in check_thread_termination
raise PendingThreadsError("left-over threads found after running test")
PendingThreadsError: left-over threads found after running test
======================================================================

There should be some output in the log where the test is run that shows
what threads are still running here. Could you look that up?

Post by Arfrever Frehtes Taifersar Arahesis
======================================================================
FAIL: Doctest: double_dealloc_T796
----------------------------------------------------------------------
File "/usr/lib64/python2.6/doctest.py", line 2163, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for double_dealloc_T796
File "/tmp/Cython-0.21b1/tests-2.6/run/cpp/double_dealloc_T796/double_dealloc_T796.so", line unknown line number, in double_dealloc_T796
----------------------------------------------------------------------
File "/tmp/Cython-0.21b1/tests-2.6/run/cpp/double_dealloc_T796/double_dealloc_T796.so", line ?, in double_dealloc_T796
del x
SimpleGarbage(1) __dealloc__
Collector.__dealloc__
collect 0
SimpleGarbage(1) __dealloc__
Collector.__dealloc__
collect 128

This might just be a problem with the test. Does this patch help?

=========================================
diff -r f7b5d79a04e6 -r e2f2cf8efcc7 tests/run/double_dealloc_T796.pyx
--- a/tests/run/double_dealloc_T796.pyx Thu Aug 28 20:21:14 2014 +0200
+++ b/tests/run/double_dealloc_T796.pyx Thu Aug 28 20:31:06 2014 +0200
@@ -1,4 +1,5 @@
"""
+>>> gc.collect()

x = SimpleGarbage()

SimpleGarbage(1) __cinit__

del x

=========================================

Stefan

Arfrever Frehtes Taifersar Arahesis

2014-08-31 08:29:05 UTC

There should be some output in the log where the test is run that shows
what threads are still running here. Could you look that up?

Output for Cython 0.21 beta 2:

compiling (c) and running bufaccess ... Doctest: bufaccess.__test__.acquire_failure1 ... ok
Doctest: bufaccess.__test__.acquire_failure2 ... ok
Doctest: bufaccess.__test__.acquire_failure3 ... ok
Doctest: bufaccess.__test__.acquire_failure4 ... ok
Doctest: bufaccess.__test__.acquire_failure5 ... ok
Doctest: bufaccess.__test__.acquire_nonbuffer1 ... ok
Doctest: bufaccess.__test__.acquire_nonbuffer2 ... ok
Doctest: bufaccess.__test__.acquire_raise ... ok
Doctest: bufaccess.__test__.acquire_release ... ok
Doctest: bufaccess.__test__.as_argument ... ok
Doctest: bufaccess.__test__.as_argument_defval ... ok
Doctest: bufaccess.__test__.as_argument_not_none ... ok
Doctest: bufaccess.__test__.assign_temporary_to_object ... ok
Doctest: bufaccess.__test__.assign_to_object ... ok
Doctest: bufaccess.__test__.basic_struct ... ok
Doctest: bufaccess.__test__.bufdefaults1 ... ok
Doctest: bufaccess.__test__.buffer_cast ... ok
Doctest: bufaccess.__test__.buffer_cast_fails ... ok
Doctest: bufaccess.__test__.buffer_nogil ... ok
Doctest: bufaccess.__test__.buffer_nogil_oob ... ok
Doctest: bufaccess.__test__.c_contig ... ok
Doctest: bufaccess.__test__.c_contig_2d ... ok
Doctest: bufaccess.__test__.cascaded_buffer_assignment ... ok
Doctest: bufaccess.__test__.cdef_assignment ... ok
Doctest: bufaccess.__test__.complex_dtype ... ok
Doctest: bufaccess.__test__.complex_inplace ... ok
Doctest: bufaccess.__test__.complex_struct_dtype ... ok
Doctest: bufaccess.__test__.complex_struct_inplace ... ok
Doctest: bufaccess.__test__.explicitly_release_buffer ... ok
Doctest: bufaccess.__test__.f_contig ... ok
Doctest: bufaccess.__test__.f_contig_2d ... ok
Doctest: bufaccess.__test__.forin_assignment ... ok
Doctest: bufaccess.__test__.get_int_2d ... ok
Doctest: bufaccess.__test__.get_int_2d_uintindex ... ok
Doctest: bufaccess.__test__.inplace_operators ... ok
Doctest: bufaccess.__test__.list_comprehension ... ok
Doctest: bufaccess.__test__.mixed_get ... ok
Doctest: bufaccess.__test__.nested_packed_struct ... ok
Doctest: bufaccess.__test__.nested_struct ... ok
Doctest: bufaccess.__test__.no_negative_indices ... ok
Doctest: bufaccess.__test__.packed_struct ... ok
Doctest: bufaccess.__test__.printbuf_float ... ok
Doctest: bufaccess.__test__.printbuf_int_2d ... ok
Doctest: bufaccess.__test__.printbuf_object ... ok
Doctest: bufaccess.__test__.printbuf_td_cy_int ... ok
Doctest: bufaccess.__test__.printbuf_td_h_cy_short ... ok
Doctest: bufaccess.__test__.printbuf_td_h_double ... ok
Doctest: bufaccess.__test__.printbuf_td_h_short ... ok
Doctest: bufaccess.__test__.printbuf_td_h_ushort ... ok
Doctest: bufaccess.__test__.readonly ... ok
Doctest: bufaccess.__test__.safe_get ... ok
Doctest: bufaccess.__test__.set_int_2d ... ok
Doctest: bufaccess.__test__.strided ... ok
Doctest: bufaccess.__test__.test_inplace_assignment ... ok
Doctest: bufaccess.__test__.tuple_buffer_assignment1 ... ok
Doctest: bufaccess.__test__.tuple_buffer_assignment2 ... ok
Doctest: bufaccess.__test__.typedbuffer1 ... ok
Doctest: bufaccess.__test__.typedbuffer2 ... ok
Doctest: bufaccess.__test__.unsafe_get ... ok
Doctest: bufaccess.__test__.unsafe_get_nonegative ... ok
Doctest: bufaccess.__test__.wraparound_directive ... ok
Doctest: bufaccess.__test__.writable ... ok
warning: left-over threads found after running test:
...<HistorySavingThread(Thread-1, started 140020768708352)>
ERROR

This might just be a problem with the test. Does this patch help?
=========================================
diff -r f7b5d79a04e6 -r e2f2cf8efcc7 tests/run/double_dealloc_T796.pyx
--- a/tests/run/double_dealloc_T796.pyx Thu Aug 28 20:21:14 2014 +0200
+++ b/tests/run/double_dealloc_T796.pyx Thu Aug 28 20:31:06 2014 +0200
@@ -1,4 +1,5 @@
"""
+>>> gc.collect()

x = SimpleGarbage()

SimpleGarbage(1) __cinit__

del x

=========================================

This patch does not help (for Cython 0.21 beta 2).

--
Arfrever Frehtes Taifersar Arahesis

Stefan Behnel

2014-09-08 17:51:51 UTC

Hi,

thanks again for testing.

There should be some output in the log where the test is run that shows
what threads are still running here. Could you look that up?

compiling (c) and running bufaccess ... Doctest: bufaccess.__test__.acquire_failure1 ... ok
[...]
...<HistorySavingThread(Thread-1, started 140020768708352)>
ERROR

That's from the IPython tests. It seems to start that thread automatically,
don't know if we can do anything about it. In any case, it's not harmful
and the rest of the test has passed.

This might just be a problem with the test. Does this patch help?
=========================================
diff -r f7b5d79a04e6 -r e2f2cf8efcc7 tests/run/double_dealloc_T796.pyx
--- a/tests/run/double_dealloc_T796.pyx Thu Aug 28 20:21:14 2014 +0200
+++ b/tests/run/double_dealloc_T796.pyx Thu Aug 28 20:31:06 2014 +0200
@@ -1,4 +1,5 @@
"""
+>>> gc.collect()

x = SimpleGarbage()

SimpleGarbage(1) __cinit__

del x

=========================================

This patch does not help (for Cython 0.21 beta 2).

Then I hope that this fixes it:

https://github.com/cython/cython/commit/268810da652f702415a18c846b98ee376df8d011

Stefan

Stefan Behnel

2014-09-08 18:03:54 UTC

There should be some output in the log where the test is run that shows
what threads are still running here. Could you look that up?

compiling (c) and running bufaccess ... Doctest: bufaccess.__test__.acquire_failure1 ... ok
[...]
...<HistorySavingThread(Thread-1, started 140020768708352)>
ERROR

That's from the IPython tests. It seems to start that thread automatically,
don't know if we can do anything about it.

Seems like we can:

https://github.com/cython/cython/commit/33cabb92abe015a3c2ccda136df4fc9ea681d406

Stefan

Peter Schay

2014-08-18 22:46:20 UTC

Hi,
The 0.21b1 release has been working great for my project (7500 lines of pyx
and 7500 py) and the only detectable difference was that it caught a bug
with "a = b or c".
Performance is same for some of my benchmarks and perhaps a few % faster
for others.
Now I am looking forward to trying the new enums which hopefully will let
me eliminate some duplicated constants.
Finally, since I work at a big company and they are going to want to
inspect my code at some point, the new language regarding generated code is
much appreciated and very timely.
Cython is fantastic; thank you and everyone who has been working on it.

One question: Is there any expected time-frame for freelists to support
cdef subclasses? Soon? Never? Somewhere in between?

Thanks,
Pete

Stefan Behnel

2014-08-28 18:11:53 UTC

Post by Peter Schay
One question: Is there any expected time-frame for freelists to support
cdef subclasses? Soon? Never? Somewhere in between?

Hard to say. This isn't easy as supporting subtypes means that we need full
control over what the parent types are doing in their
allocation/deallocation chain, and that may not be entirely trivial to
determine. The current call chain isn't made for that.

In any case, no-one's currently working on this AFAIK.

Stefan

Martin Bammer

2014-08-13 08:34:09 UTC

Hi,

following error occurs with Cython 0.21b1 on Windows:

Error compiling Cython file:
------------------------------------------------------------
...
pickle.dump(self.pErrors, F, -1)

@cython.ccall
@cython.locals(bParseP = cython.bint)
@cython.returns(cython.bint)
def LoadCache(self, bParseP):
^
------------------------------------------------------------

PClib.pyx:1054:4: Compiler crash in AnalyseDeclarationsTransform

File 'ModuleNode.py', line 103, in analyse_declarations:
ModuleNode(PClib.pyx:1:0,
full_module_name = 'PClib')
File 'Nodes.py', line 387, in analyse_declarations:
StatListNode(PClib.pyx:6:0)
File 'Nodes.py', line 387, in analyse_declarations:
StatListNode(PClib.pyx:714:0)
File 'Nodes.py', line 4245, in analyse_declarations:
PyClassDefNode(PClib.pyx:714:0,
name = u'Parser')
File 'Nodes.py', line 387, in analyse_declarations:
StatListNode(PClib.pyx:715:4)
File 'Nodes.py', line 331, in analyse_declarations:
CompilerDirectivesNode(PClib.pyx:1054:4)
File 'Nodes.py', line 387, in analyse_declarations:
StatListNode(PClib.pyx:1054:4)
File 'Nodes.py', line 2268, in analyse_declarations:
CFuncDefNode(PClib.pyx:1054:4,
args = [...]/2,
modifiers = [...]/0,
overridable = True,
visibility = 'private')
File 'Nodes.py', line 2277, in declare_cpdef_wrapper:
CFuncDefNode(PClib.pyx:1054:4,
args = [...]/2,
modifiers = [...]/0,
overridable = True,
visibility = 'private')
File 'Nodes.py', line 2338, in call_self_node:
CFuncDefNode(PClib.pyx:1054:4,
args = [...]/2,
modifiers = [...]/0,
overridable = True,
visibility = 'private')

Compiler crash traceback from this point on:
File "c:\python27\lib\site-packages\Cython\Compiler\Nodes.py", line 2338,
in call_self_node
type_entry = self.type.args[0].type.entry
AttributeError: 'PyObjectType' object has no attribute 'entry'

Stefan Behnel

2014-08-28 18:34:58 UTC

Hi,

here is a second beta:

http://www.cython.org/release/Cython-0.21b2.tar.gz

http://www.cython.org/release/Cython-0.21b2.tar.gz.asc

Since the last beta, there were a couple of improvements for the new method
call optimisation, as well as some fixes (usually noted in the relevant
mailing list threads already).

We might have forgotten to mention previously that inner functions are
supported inside of cdef functions in 0.21, a feature that I consider worth
noting (and testing :).

Complete changelog:

https://github.com/cython/cython/blob/d4b87c6d818406275905c87cf89479bebff53c6f/CHANGES.rst

This would be a good time to bump any pending bug reports that you think
should be considered for this release.

Stefan

Stefan Behnel

2014-08-29 17:38:51 UTC

[ forgot to change the subject to attract attention :) ]

Post by Stefan Behnel
Hi,
http://www.cython.org/release/Cython-0.21b2.tar.gz
http://www.cython.org/release/Cython-0.21b2.tar.gz.asc
Since the last beta, there were a couple of improvements for the new method
call optimisation, as well as some fixes (usually noted in the relevant
mailing list threads already).
We might have forgotten to mention previously that inner functions are
supported inside of cdef functions in 0.21, a feature that I consider worth
noting (and testing :).
https://github.com/cython/cython/blob/d4b87c6d818406275905c87cf89479bebff53c6f/CHANGES.rst
This would be a good time to bump any pending bug reports that you think
should be considered for this release.
Stefan

Martin Bammer

2014-08-30 15:53:45 UTC

Hi,

I've tested now with the lastet cython beta. Result is: same speed, but
bigger pyd-files.
Example:
with 20.2: 357888 bytes
with 21b2: 387072 bytes

Martin

Ivan Smirnov

2014-09-08 08:55:05 UTC