IO Time with NeuroTools

[pierre]

If anybody has a great idea to speed up the loading time of the NeuroTools object, I think we should discussed it and find a solution to speed up the whole stuff. Otherwise it will be a bottleneck and nobody will use it. Now, when you have file stored with cpickle, loading/saving time are good. The problem is for the text file, and unfortunately, this is the standard output of pyNN. So, either we should add an option allowing pyNN to write directly, at the end of the simulation, a pickled NeuroTools object (that can be build while the file is post processed by pyNN), either we should find others solutions. Suggestions are welcomed :-)

[emuller]

hdf5

[pierre]

Yes, hdf5 will be a solution, but not handle yet in NeuroTools... Nevertheless, I found a nice implementation with the bisect package of python that considerably reduces the loading times of rather big files, so now this is even very fast to load file and build SpikeTrains? object from text file

[pierre]

Thanks to the bisect package, the loading time even for a text file are now user friendly. Maybe it can still be improved, but at least this is really efficient for 0.1