On Wed, Nov 4, 2009 at 9:04 PM, Anne Archibald <peridot.faceted@[...].com> wrote:

>  2009/11/4 Charles R Harris <charlesr.harris@[...].com>:
>  >
>  >
>  > On Wed, Nov 4, 2009 at 8:39 PM, Anne Archibald <
>  peridot.faceted@[...].com>
>  > wrote:
>  >>
>  >> 2009/11/4 Charles R Harris <charlesr.harris@[...].com>:
>  >> > I see the following test errors on ubuntu_x86_64 9.10
>  >> >
>  >> > ERROR: Check linalg works with non-aligned memory
>  >> > ----------------------------------------------------------------------
>  >> > Traceback (most recent call last):
>  >> >   File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in
>  runTest
>  >> >     self.test(*self.arg)
>  >> >   File
>  >> >
>  >> >
>  "/usr/local/lib/python2.6/dist-packages/scipy/linalg/tests/test_decomp.py",
>  >> > line 1067, in test_aligned_mem
>  >> >     eig(z.T, overwrite_a=True)
>  >> >   File
>  "/usr/local/lib/python2.6/dist-packages/scipy/linalg/decomp.py",
>  >> > line
>  >> > 158, in eig
>  >> >     geev, = get_lapack_funcs(('geev',),(a1,))
>  >> >   File
>  "/usr/local/lib/python2.6/dist-packages/scipy/linalg/lapack.py",
>  >> > line
>  >> > 82, in get_lapack_funcs
>  >> >     raise ValueError("Non-aligned array cannot be passed to LAPACK
>  >> > without
>  >> > copying")
>  >> > ValueError: Non-aligned array cannot be passed to LAPACK without
>  copying
>  >> [...]
>  >> > I suspect the last failure is an mpmath version problem, and maybe the
>  >> > second also. Is anyone else seeing the other error?
>  >>
>  >> This error is related to ticket 794:
>  >> http://projects.scipy.org/scipy/ticket/794
>  >> Under certain poorly-defined circumstances, passing non-aligned
>  >> matrices to ATLAS can cause a segfault. I've posted a support request
>  >> on the ATLAS tracker, hoping to find out whether it's supposed to be
>  >> able to deal with non-aligned matrices, and in the meantime I have
>  >> code to raise exceptions when non-aligned matrices are fed to LAPACK.
>  >>
>  >> I don't know whether the bug occurs at all on 64-bit systems, and I
>  >> don't know whether it's the result of us abusing ATLAS or a bug in
>  >> ATLAS. If it's a bug in ATLAS, I don't know for sure what triggers it
>  >> - in particular, whether alignment actually solves the problem on all
>  >> platforms.
>  >>
>  >>
>  >> One solution would be to annotate all relevant f2py LAPACK wrappers
>  >> with the new align8 keyword and hope that cures the problem (by
>  >> forcing a copy of all not-sufficiently-aligned arrays on all LAPACK
>  >> installations).
>  >>
>  >> Another "solution" would be to simply remove the failing test and tell
>  >> people not to feed non-aligned matrices full of outrageous values into
>  >> ATLAS on pain of segfaults. (The bug was filed by someone who'd had a
>  >> segfault while working with an unpickled array, so it does bite some
>  >> people.)
>  >>
>  >> A third option would be to raise ValueErrors any time non-aligned
>  >> matrices are supposed to be passed to LAPACK. (This is what happens
>  >> now, but the tests fail when this happens; if this were the permanent
>  >> decision I would of course fix the tests to expect exceptions.)
>  >>
>  >> It might be possible, though I'm not sure how and it would be
>  >> complicated, to trigger all this working-around only when using ATLAS.
>  >>
>  >>
>  >> I'm not sure how best to deal with this. What do you think?
>  >>
>  >
>  > I haven't a clue as to what the best thing to do is. Using aligned
>  matrices
>  > might be the best workaround. Were the alignment problems on SPARC? I
>  recall
>  > a thread on the topic but don't remember what all was said.
> 
>  I don't know about SPARC, but I was able to trigger them on my
>  Pentium-M machine using the SSE2 version of ATLAS. (Which means, among
>  other things, that this problem is going to affect a reasonable number
>  of users.)
> 
>  Part of the problem with using aligned matrices is that I'm not
>  confident I know which routines need aligned matrices: I found
>  segfaults using eig and svd, but many others were fine - on my
>  machine. svd was a particular problem because as I understand the
>  f2py, the matrix is already always copied on input (and adding the
>  aligned flag didn't help, though aligning the array before I called
>  svd made the problem go away).
> 
> 
ATLAS is a moving target and who knows what will cause trouble in the
future. On top of that, it is trying to get the best performance and that is
likely to mean code somewhere that likes aligned data. I've now read the
ticket and I think you are right to just use the new alignment option added
to f2py. I don't think you should wait too long on the ATLAS folks.

Chuck