The Daily Dongle

Transformation Differences

One of the most frequently questions I'm asked is, "Why does the data look different in FlowJo as compared to DIVA?"

It is a result of the biexponential transformation.

DIVA calculates the "best" transform and typically uses a width basis around -100 by default.  Therefore, the first two decades are compressed into the linear space around 0. 

Since the fcs3.0 standard does not include a keyword for exporting the transform used, it is up to the analysis program to apply the transform to the data.

FlowJo's default is set at -10 by default.  Hence, only the first decade is compressed into the linear space.  This leads to a difference in the visualization of data between DIVA and FlowJo.

We encourage users to start with a width basis of -10 and increase it if necessary.  Too much initial compression can lead to the accidental loss of dim events because they are compressed into the negative population.  While a width basis of -100 often makes the data appear 'cleaner', we recommend users start with a minor transform and scale it up as necessary.  If you prefer to start with a default width basis of -100 in FlowJo, you can set it in the preferences.

For Mac: FlowJo>preferences>workspace tab>digital data

For PC: Edit>preferences>fcs3.0 scaling preferences

Other aquistion programs use different default transforms or no transformation.  You can also turn the transform off in FlowJo by default if you prefer.

- Question or comments can be sent to nick@treestar.com

 

 

comp-matrix confetti? badges to the rescue

You might have noticed that starting with version 7.6 there are new ornaments in the workspace window, to the left of the sample names. We call them "badges" :

 

each of the three has a different context, for now, I'd like to focus on the right-most - which represents compensation. The color of the compensation badge, if grey, means the sample lacks compensation, and if a color is assigned, then the badge color matches the compensation matrix color (as seen in the comp editor list):

 

The data I'm showing you here all belongs to the same experiment, the same run, all tubes are in sequence. So.. why are there so many "Acquisition-defined" matrices? Badges certainly can help spot this problem, but as for the explanation:

It appears that the software which creates some of the files uses a slight variation from one file to the next in some nth decimal place of the spillover values. FlowJo simply detects these as different and assigns each unique set of values to a separate matrix:

 

comparing the matrix above to the one below, you can notice in the PE-Cy7 row that there are minute variations in the spills:

This "granularity" gets FLowJo into trouble with users switching over from their other software - users report that some of the macros for transformation, for instance, will not work.

The workaround is two fold:

1. you could drag a compensation matrix from the comp-editor, to the workspace window, and drop it on the name of the group of your experiment. This applies the matrix to all samples in that group allowing the comp-macros in the Graph Window's T button to work correctly.

2. you could import your single-stain controls and derive a comp-matrix in FlowJo, then use that instead of the acquisition matrix.

DiVa data vs. FlowJo, 2005 vs. 2007

One of our astute users (just kidding, all our users are astute!) sent in a question regarding display changes of DiVa data from an older 2005 version of FlowJo, vs. how we plot DiVa data today. The difference is most striking in the lowest decade where the newer versions of FlowJo (Left) show picket fencing.

 

Engineering had a go at this one, here's the transcript:

Back in 2005, DiVa could only export integer data (18 bit).  This allowed FlowJo to randomize within the range of one channel, because it knew exactly what the resolution of the incoming data was (i.e., it knew the distance between one channel and the next to be 1.0 units).  With the advent of floating point data, FlowJo can no longer make such assumptions.  (Well, it could, but that might be wrong in some cases, so it doesn't).  Therefore, it no longer can randomize enough to spread the data at the very low values across to hide the picket fencing.  Of course, this becomes invisible when Transformation is on, which is what the default in FlowJo is, and is why we haven't noticed this previously.

So, if you want to look at the data without transformation (ugh), and want to completely remove the picket fencing, then export the data as integer data [edit: we're not sure how to do it, so ask BD].  Best is to leave transformation on.

 

DiVa training materials

Mr Blais forwarded me a link to some BD DIVA training materials:

http://bdbiosciences.com/immunocytometry_systems/support/training/online/

fixed support for large DiVa files in FJ for windows

FlowJo for Windows 7.1.3 and older would not be able to handle certain large diva files, creating corrupted displays and/or stat problems such as this one:

this has now been fixed in 7.2 beta. here's an example 11 color, 2.5M event file. I've tested 4 of these on a machine with 1GB RAM and it seems to behave!

How to completely disable biexponential scaling in FlowJo

Many FlowJo users want to disable bi-exponential transforms. This post will show you how, for Version 7 (Windows) and Version 8 (Macintosh).

Also, before you give up on Biexponential transforms, please be sure to understand what you are giving up. There is a great paper by the Herzenberg lab showing the need for this visualization technique. Another one by Parks/Moore/Roederer. There is also a tutorial on how to set this up properly in Windows version (mac tutorial coming soon)

FlowJo V7.1.3: this can be done effectively thru DiVa preferences or the Graph Window.

If you change it in your preferences you will not see transforms, ever, for any NEW workspaces.

However, FlowJo embeds biexponential settings into saved workspaces. So if you bring a workspace from another computer, and if that workspace has biexponential scaling enabled, this will over-ride the local preferences set.

In that case you'll have to change this in Transforms tab.

DiVa preferences:
1. Open FlowJo
2. Click Windows menu, choose Compensation Editor.
3. Click Transformations tab, and Set DiVa preferences button
4. un-check the "auto-transform" button, click OK and Apply

Graph Window:
1. Open FlowJo
2. Open a Graph Window of a DiVa file (FCS3, compensated)
3. format one of the axes to show a "comp: FLx" parameter (any compensated fluorescent parameter)
4. notice the T button next to that axis. Click on it until you notice that the 0 value moves to the corner.
5. right-click the T button and choose "Apply to All Compensated Parameters"

FlowJo V8

FlowJo V8 also has a preference to set BiExponential transforms.

Please note: if you open a saved workspace where biexponential transforms have been enabled, you may not disable them. Once these are on, they stay on - you would have to change the prefs and re-analyze the data. This tutorial applies only to NEW analysis.

1. Open FlowJo
2. Click FlowJo/Preferences. In the Workspace tab, disable custom visualization
3. click "Define.." button (this brings up more DiVa preferences)
4. un-click "enable transformation"
5. click OK, Save. Restart FlowJo.

BD FACSDiva sofware 5.0.1 on Macintosh via parallels

Is it true? Is it necessary? is it pretty? well.. the answer is "yes" to at least one of those questions: It's true. You can run DiVa on a mac. First, we need to get windows XP on a mac. I use parallels with the following settings:

Of course I don't have a cytometer connected to the iMac, I had to connect the USB dongle that BD requires. This was possible once I enabled the USB autoconnect.

And here it is, the unphotoshopped proof:

the samples I imported were some 200MB files. during the screenshot, I was modifying the quad gates. You can see the CPU utilization is pretty high... but doing the same in diva 4 on my native windows workstation acts about the same. No big surprize there. It's not slow, it's careful (TM).

DiVa scatter parameter scaling in FlowJo

FlowJo has a set of preferences for dealing with DiVa parameter scaling, digital transformation and number of decades. Most frequently asked question is,

how do I set the scale of all my diva side scatter parameters to Logarithmic?

To master the scaling of your parameters, launch preferences and edit your DiVa prefs until your graphs look how you want.

If you use the Macintosh version, you should follow the instructions on this page. Please note, setting these preferences requires a restart of the program:
http://www.flowjo.com/v8/html/divadata.html

if you use the Windows version, messing with preferences causes an instant update of all graphs.

• in your Workspace window, click Windows menu, compensation editor.
• In the window that comes up, open transformation tab. On the bottom, click Set Diva Prefs.
• in the dialog that comes up, make your choices for scaling, then click OK.

more on data visualization

it appears the paper I suggested here is pretty popular with our users. Here is an example email I got (digested, rephrased) which should clarify a lot of points about data visualization in FlowJo.

I have already collected numerous data sets with compensation (DiVa based BD cytometer) and am now wondering if there is any way to recover my data and if what i've done is Ok and usable.

DiVa software outputs your cytometers data in either FCS2 or FCS3 format. FCS3 files contain the original 14 bit ("16,384 discrete levels") dynamic range in 18 bits (262144). Not sure why. Anyway, in addition to containing the raw, linear 14bit values, they also contain a matrix of compensation spillovers which is applied on-the-fly as you view the data to produce the pretty compensated plots.

when DiVa data is loaded into FlowJo you can view the original, uncompensated parameters:

• mac: the compensated parameters have the <brackets> around them, and uncompensated ones do not.  You can view the uncompensated parameters by option-clicking the graph window axes.
• PC: the compensated parameters have the Comp: prefix, whereas the uncompensated ones do not.

when i import my LSR II compensated data into flowjo it DOESNT appear to be compensated;
there is no purple line next to the samples

I bet you're saving FCS2.0 files. DiVa has the option to export FCS files as FCS 2 or 3.

If I compensate DiVa data in FlowJo, am I compensating my data twice??

no, when you re-compensate a software compensated sample, the effect of compensation is not cumulative. The new compensation spillovers replace the old ones.

How do I switch from the normal Log scaling to the bi-exponential view (or logicale view)?

In the mac version, click Platforms, compensate.., define bi-exp transform. This is only possible once the purple bar is next to your sample in the workspace window.

In the PC version, you also need the purlple bar, if there is, and you open a graph window, you will find a little T button in the graph window which toggles the transform on/off interactively per-parameter. Right click on the T button to explore more options.

DiVa log-shmog...

I get a lot of emails like

I acquired my data in log, but your software is showing it in linear...

This is usually wrt DiVa files.

DiVa files are in FCS3.0 format (see this and that )and all parameters are linear. FlowJo has preferences which will try to scale the data in log if the parameter is fluorescent. Scatter parameters can be either, so we just leave it to however the user sets it in the preferences.

To set these prefs in Mac, open preferences, click Define... and mess with the checkboxes on the DiVa data settings.

To set these prefs in PC, click Edit menu, 32bit data prefs.