Speed tricks for analyzing Large Data Files

Some flow researchers with large event count files, ask us for tips to speed up the FlowJo analysis. Here are a couple quickies.

Preprocessing

If the analysis workflow allows it, one can do some pre-processing. In FlowJo, gate on Cells of interest (eliminating debris and doublets), then use the Export feature to create smaller fcs files.

Ram Disk

One can benefit from setting up a RAM disk, especially if the fcs files are  greater than 10MB. (If you need more RAM for your Mac, check out www.crucial.com). On MacOSX, one can mount a 2Gig RAM Disk easily by pasting the text below into Terminal:

diskutil erasevolume HFS+ "ramdisk" `hdiutil attach -nomount ram://4000000`

Or one can use a program like Tmpdisk to create a RAM Disk.

Make sure you create a folder on the RAM disk for your flowjo workspace file and fcs files (loading the fcs files into flowjo from the RAM disk), and copy the RAM Disk folder to your hard drive when done with the analysis.

Comments

[Keith]

My 3 tips for accelerating data analysis:

1. Upgrade your hard drive to an SSD

2. Max out your RAM

3. Work with local copies of your fcs files.

I recommend maxing out the RAM in your computer AND upgrading to an SSD. Once you have an SSD you won't need to use a RAM disk, but the extra RAM is always a good idea.

If your files are located on a network drive, you should copy them to your local drive before you begin analysis. Accessing big files over a slow network connection greatly impedes things, especially when you are trying to refresh multiple large files.

Time is money - you shouldn't be afraid to invest a few $$ to speed up data analysis. It pays off in the end!