cPickle bug in AS Python [was Re: Shelve bug in ActiveState Python? [or Python as Houdini]]
by Bob Kline other posts by this author
May 13 2008 3:03PM messages near this date
view in the new Beta List Site
Re: Shelve bug in ActiveState Python? [or Python as Houdini]
|
Re: Shelve bug in ActiveState Python? [or Python as Houdini]
Trent Mick wrote:
> Bob Kline wrote:
>
> > I'm looking at behavior in ActiveState's Python which looks suspiciously
> > like a bug. I've got a number of steps before I'm ready to file a real
> > bug report (I'm a minor release behind (2.5.1.1), for one thing, so I'll
> > need to test with the latest release, and creating a repro case for this
> > one won't be trivial), but I'd like to do a little reality check here on
> > the listserv before I plow ahead.
> >
> > Boiling the code down to its essentials, and with a few debugging output
> > statements thrown in, what we have looks something like this:
> >
> >
>
> Does seem supicious. I don't have much experience with shelve, but it
> should just crash. Do you have another platform that you can try to
> repro on?
Tried it on Linux (various machines, OS versions, Python versions) and I
get a RuntimeError exception thrown ("maximum recursion depth
exceeded"). I've been able to eliminate the classes I was trying to
feed to shelve, and in fact, eliminated the shelve module itself from
the repro case. I can replicate the behavior by parsing an XML document
and asking cPickle to serialize the DOM object. So I'd say we have two
issues, only one of which is specific to the ActiveState distribution.
The other question, which I'll turn over to the core Python team, is why
the pickling implementation is blowing a recursion limit of 1000 when
the maximum level of nesting in the document I'm working with is nine
(ten if you count the unnamed head of the tree), given that the pickle
module's documentation claims to handle recursive references correctly
(in contrast to the marshal module).
The original problem is the one specific to Python on Windows: why is
Python crashing instead of throwing the exception (whose documented
purpose is to prevent the crash in the first place)?
As I say, this is happening on every version of ActiveState Python I've
tried it on (up through 2.5.2.2) on various flavors of Windows (XP
Professional/Intel 3.20 GHz/1GB RAM, as well as Windows Server 2003
R2/Intel Xeon 3.00 GHz x 4/8 GB RAM).
> A repro code snippet would, of course, help a lot.
>
Attached. I replaced the text values in the document with random
garbage of the same length in order to avoid the possibility of
divulging sensitive data, but the document's still well-formed, and the
behavior's identical.
> Does the crash "go away" if ClinicalTrial is made simpler. How about if
> ClinicalTrial is made a new-style class? (I'm flailing here. :)
>
See notes above.
Let me know if you'd prefer that I file this bug directly myself. We
have a workaround (avoiding pickling altogether by changing the
processing logic), if you're uncertain about where to set the priority
(though any problem which crashes the Python processor can't be
something you'd want hanging around for long).
--
Bob Kline
http://www.rksystems.com
mailto:bkline@rksystems.com
Thread:
Bob Kline
Trent Mick
Bob Kline
Trent Mick
|