How to increase sorting speed and reading of *.mcd files?

Dear MCS or any other users with advice,

We would like any advice to speed up our sorting process of spikes recorded from mouse retinas. We use Offline Sorter v3 from Plexon to sort MCD data files. The process of opening channels is quite slow because the data files are relatively large (multi-GB).

With a SATA2 hard drive, we reached a top speed of 40 MB/s. We installed a solid state drive, Intel X25-M G2, but this only increased that rate to 60 MB/s. We were hoping to get closer to the >200 MB/s theoretical speed.

The software reads each channel from disk using the MC_Stream.dll. This process seems to happen in 1KB blocks, but we're not sure.

Is there a way to increase the block size or some other way to increase the speed that Offline Sorter can get the data into memory? I’m asking here because this seems to be a limitation due to MC_Stream.dll and not due to Offline Sorter. If that is incorrect, please let me know. The drives are formatted under WinXP with a 4KB allocation block size.

(Alignment of waveforms is another bottleneck, and we're increasing CPU power for that.)

Thanks!
Rene’ Renteria
University of Texas Health Science Center at San Antonio

rrenteri wrote:
Is there a way to increase the block size or some other way to increase the speed that Offline Sorter can get the data into memory? I’m asking here because this seems to be a limitation due to MC_Stream.dll and not due to Offline Sorter. If that is incorrect, please let me know. The drives are formatted under WinXP with a 4KB allocation block size.
Thanks!

There is no internal block size to adjust in MC_Stream.dll. All reading is done with the standard library functions.
I have loaded a 1 GB file with spikes into offline sorter in just less than 50 s. I can't comment on offline sorter, but I suppose that the data are in memory after loading, so that opening a channel should not be a problem of disk speed anymore

regards
Hans

Thank you. We're still trying to find the bottleneck. It seems that the sorter reads the file from disk but only to get information on number of waveforms and channels. Then it reads from disk again when asked to open an individual channel. It seems that processing happens after each block of disk reading as the sorter uses MC_Stream.dll to decode the mcd file. Or something like that.

Our goal is to speed up all the aspects of sorting. For example, aligning waveforms is a major bottleneck, and that is mostly CPU-limited from what we can tell. So we're moving to an overclocked i7-920 PC.

Best,
Rene'