Re: [Numpy-discussion] Histograms of extremely large data sets
by Eric Jones other posts by this author
Dec 13 2006 8:35PM messages near this date
Re: [Numpy-discussion] Histograms of extremely large data sets
|
[Numpy-discussion] a.T
Glad to here it worked for you.
see ya,
eric
Cameron Walsh wrote:
> Thanks very much, Eric. That line fixed it for me, although I'm still
> not sure why it broke with the last line.
>
> Your weave_histogram works a charm and is around 16 times faster than
> any of the other options I've tried. On my laptop it took 30 seconds
> to generate a histogram from 500 million numbers, which is fine.
>
> Thanks and best regards,
>
> Cameron.
>
>
> On 14/12/06, eric jones <eric@[...].com> wrote:
>
> > Hmmm.
> >
> > ? Not sure. ?
> >
> > Change that line to this instead which should work as well.
> >
> > code = array_converter.declaration_code(self, templatize, inline)
> >
> > Both work for me.
> >
> > eric
> >
> > Cameron Walsh wrote:
> >
> >> On 13/12/06, Cameron Walsh <cameron.walsh@[...].com> wrote:
> >>
> >>
> >>> On 13/12/06, eric jones <eric@[...].com> wrote 290 lines of
> >>> awesome code and a fantastic explanation:
> >>>
> >>>
> >>>
> >>>> Hey Cameron,
> >>>>
> >>>> I wrote a simple weave based histogram function that should work for
> >>>> your problem. It should work for any array input data type. The needed
> >>>> files (and a few tests and examples) are attached.
> >>>>
> >>>>
> >> [...]
> >>
> >> Hi Eric,
> >>
> >> I've ran test_weave_histogram.py and histogram_speed.py, but each one
> >> seems to fail on calling typed_array_converter.declaration_code() as
> >> follows:
> >>
> >> Traceback (most recent call last):
> >> File "histogram_speed.py", line 26, in <module>
> >> res2 = histogram(data, bins)
> >> File "/home/cameron/repos/wavesmaker/trunk/code/gui/process_modules/eric_histo/weave_h
istogram.py",
> >> line 67, in histogram
> >> compiler='gcc')
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/inline_tools.py",
> >> line 339, in inline
> >> **kw)
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/inline_tools.py",
> >> line 447, in compile_function
> >> verbose=verbose, **kw)
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/ext_tools.py",
> >> line 353, in compile
> >> kw,file = self.build_kw_and_file(location,kw)
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/ext_tools.py",
> >> line 334, in build_kw_and_file
> >> file = self.generate_file(location=location)
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/ext_tools.py",
> >> line 295, in generate_file
> >> code = self.module_code()
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/ext_tools.py",
> >> line 203, in module_code
> >> self.function_code(),
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/ext_tools.py",
> >> line 269, in function_code
> >> all_function_code += func.function_code()
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/inline_tools.py",
> >> line 83, in function_code
> >> decl_code = indent(self.arg_declaration_code(),4)
> >> File "/usr/local/lib/python2.5/site-packages/scipy/weave/inline_tools.py",
> >> line 62, in arg_declaration_code
> >> arg_strings.append(arg.declaration_code(inline=1))
> >> File "/home/cameron/repos/wavesmaker/trunk/code/gui/process_modules/eric_histo/typed_a
rray_converter.py",
> >> line 18, in declaration_code
> >> code = super(typed_array_converter, self).declaration_code(templatize,
> >> TypeError: super() argument 1 must be type, not classobj
> >>
> >> I've tried this with Python2.4 and Python2.5 with the same results.
> >>
> >> What do I need to change, since it seems to have worked for you but not for me.
> >>
> >> Thanks and best regards,
> >>
> >> Cameron.
> >> _______________________________________________
> >> Numpy-discussion mailing list
> >> Numpy-discussion@[...].org
> >> http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >>
> >>
> >>
> > _______________________________________________
> > Numpy-discussion mailing list
> > Numpy-discussion@[...].org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@[...].org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@[...].org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Thread:
Cameron Walsh
Rick White
Brian Granger
Eric Jones
Cameron Walsh
Rick White
Eric Jones
Cameron Walsh
Cameron Walsh
Eric Jones
Giorgio Luciano
Sven Schreiber
Christopher Barker
Cameron Walsh
Eric Jones
|