The Daily Dongle

FlowJo V10 build for linux

Hello Readers, 

we're still not running in linux. We are stuck setting up stepping thru C++ code in linux (debugging.) if you or someone you know is a C++ developer comfortable with Linux, we'd like to see a resume and discuss a contract. I believe currently our search is limited to Oregon, but we will consider other candidates based on their resume.

last updated by Maciej Dec 28th 2011 

 

known issues: 

- 64 bit linux / 64 bit Java VM required. 
- we run into random but frequent engine crashes when loading data. We're not sure if this is because we're developing this on VMware or if becasue there is a linux-specific engine bug. Your feedback on this behavior would help us figure this one out. 
- dragging files from file system to FJ window won't work yet. Do this via the "add samples" button. 
- this build follows the regular V10 beta licensing and release schedule (expires weekly on Wednesdays, requires beta site license. )

contact simm@treestar.com with any questions about FJ linux, otherwise download, try it, and let me know if you crash :)

-ms  

 

 

 

Clone Gates

Please note: this functionality requires FlowJo for Mac 9.3b5 or later.

Introduction

The use case for this feature came from our clinically inclined users who wanted one version of a gate propagated to different points in a hierarchy. This used to be done by one of two ways, neither of which is optimal in terms of workflow:

FJ10 in linux 64 bit = 30 million events / second

since our support to the linux savvy fj users has not been top-notch, I installed ubuntu 10.10 64 bit and will be spending some time in that testing & fixing FJ as needed. It's now my job to make FJ10 and linux bff's ;D

So, today i got my IDE running sun's java, put my honkin files on a ramdisk and got to testing:

above, i got nine 10.5M event files repainting the contents of the gate in the graph window. From moving the gate to update in the layout, 3s elapse for each gate movement. I have the "synch" attribute enabled so each gate movement in one sample is instantly propagated to the others & all 9 files are recalced. since this test involves manipulation of 9 x 10.5 M events, im claiming roughly 30M events /second performance on this 16 core 2.2GHz machine.

FJ on linux is not without problems, but at least i'll be working on resolving those. Performance wise this matches or exceeds win7 b64 bit, pending some more detailed testing.

beta testing license

Howdy all. 

Sorry to all the beta testers who got kicked off the license. Testing is closed. There was a bit of turnover in beta testing management, so things did not play out smoothly. We had to purge this license because we noticed a growth of signups of users who were not using the license for testing beta versions. 

Until we revamp the system to block license abuse the signup page for the beta license has also been closed. 

To make this up to our genuine beta testers,  we're discussing ways to reward beta testers in future beta campaigns. Details will follow as we get closer to FlowJo V10 beta testing

pre-emptive ib4 "when is FJ10 beta out" : I don't know, but when I do, you will too :) 

FlowJo Java, threaded

Got to spend some quality time with my testing 16 core today: 

FJ hitting all 16 cores, memory utilization ~8GB = instant gate recalcs :)

 

while at it, we made some performance improvements: 

This is the first time I'm showing a timing from FJ v10 (Java.) This version is currently in Alpha but we hope to have public beta soonish (Spring?)

Please note that this drastic change in performance in 7.6.4 and V10 will only be observed on:

- computers with more than 8 CPU's. I saw no change with 2/4 CPU models.  
- .. "distributed engines" setting in performance preferences turned ON 
- .. storage device fast enough to saturate all your CPU's with data 

the main reason for this improvement was the architecture of FJ's distributed engines - they ran in series, now - parallel. Also, the improved timings show FJ as running in 64 bit mode. I am not yet sure if 7.6.4 will ship in 64 bit compatible mode, but 10 definitely will.  64bitness is NOT required for this improvement to work. 

FJML keyword examples

I've compiled this as a primer to FJML keywords for those who prefer to learn by example. To use these examples, first open your Mac FlowJo to a new workspace with some data, then click Workspace menu, Add Keyword. Once you have created this new keyword, you can double-click its cell in the workspace window, just like in Excel, and enter formulas. Here are some to try (using LMD data made by CXP software.) 

This syntax..	..renders as:	notes
FJML=<>	<>	anything to the right of equals that is not valid FJML is rendered as text.
FJML=<key $SYS>	CXP	assuming $SYS = CXP
FJML=File Made By: <key $SYS>	File Made By: CXP	assuming $SYS = CXP
FJML=<stat /gate1:Freq. of parent>	80.1	assuming there is a gate called “gate1” with a stat 80.1 % :)
FJML=<key $P2B> * 2	16 * 2	assuming Parameter2 exists and its B=16. Notice the failed attempt to invoke a math formula gets parsed as a text string.
FJML={{<key $P2B> * 2}}	32	the presence of {{ }} brackets around the expression forces FJ to evaluate the contents as a formula.
FJML={{sub(<key $CYT>, 1, 3)}}	Interpreter Error.	assumes $CYT = Cytomics FC 500. Fails to parse as shown, see below for corrected example.
FJML={{sub(“<key $CYT>”, 1, 3)}}	Cyt	notice the quotes around the keyword!

finally here's a harder one: convert BTIM/ETIM keywords' values (HH:MM:SS) into number of seconds the acquisition took. Since this expression is longer, I'll do it outside the table, split into the components of extracting the hours, days and seconds separately. This code does not take into account the clock resetting/starting over every 24 hrs ;) 

to calculate hours in a day and convert to seconds:

3600* abs(num(sub("<key $BTIM>", 1, 2)) - num(sub("<key $eTIM>", 1, 2)))

The "abs" stands for absolute value, and negates any negative sign. 

"num" converts a string to a number. This is in case we get something like "06" for HH. 

"sub" stands for substring, with arguments of starting/stopping character index. 

Next, minutes -> seconds: 

60* abs(num(sub("<key $BTIM>", 4, 5)) - num(sub("<key $eTIM>", 4, 5)))

and finally, seconds (no conversion :) ): 

abs(num(sub("<key $BTIM>", 7, 8)) - num(sub("<key $eTIM>", 7, 8)))

So the combined expression to paste into the keyword, is:

FJML={{  

 3600* abs(num(sub("<key $BTIM>", 1, 2)) - num(sub("<key $eTIM>", 1, 2))) +

60* abs(num(sub("<key $BTIM>", 4, 5)) - num(sub("<key $eTIM>", 4, 5))) +

abs(num(sub("<key $BTIM>", 7, 8)) - num(sub("<key $eTIM>", 7, 8)))   

 }} 

FlowJo and Lab Archives

you might have noticed that starting with Friday's release of 7.6.3 there is a new field in Preferences' License tab: 

Should you click it and enter your Lab Archives credentials & restart FJ...

..you would add the option "Add Remote Samples" to your workspace menu. If you click on that menu item,  you will get a window to browse your Lab Archives root: 

anyone with Lab Archives accounts - please let us know how you find this new feature. Anyone without a Lab Archives acct, you can contact them about it. 

maciej simm

 

FJRealData

In addition to fjTextData, Mac version of FlowJo has another port for opening text files, called fjRealData. the primary difference between the two is header construction syntax and the fact that "real" format uses doubles, while 'text' uses integers. Data formatted in this way can be read directly into FlowJo V9 for analysis and/or exported as FCS files for future uses

please note - FJ PC users should instead convert their CSV files into FCS files using text-to-FCS utility provided on this blog a few posts back. 

 

 

Format of this data file:

Line1 = "FJREALDATA"           //(allows floating point valus)
M <tab> N                              
Key1 <tab> value1
...
KeyM <tab> valueM
P1Value1 <tab> P2Value1 <tab> ... PjValue1 <tab> Parm#Key1Name
P1Value2 <tab> P2Value2 <tab> ... PjValue2 <tab> Parm#Key2Name
...
P1ValueN <tab> P2ValueN <tab> ... PjValueN <tab> Parm#KeyNName
{values...}

M: Number of keyword name / value pairs specified one at a time

N: Number of parameter keywords that are specified in a matrix format

Key1, Value1:  The name and value for the first of M keywords specified one at a time (Use this for things like $FIL or other attributes you might want to display in the workspace, like custom keyword values, e.g., treatment, time, etc.)

PjValuek:  The value of parameter keyword number "k" for parameter number "j"

Parm#KeykName: The name of parameter keyword number "k", using # to specify where the number is to be inserted

possible values for P#DISPLAY:

LINEAR  ... linear axis
MANUAL = (x, y)   .... log axis

x = number of decades
y = offset of lowest displayed value on log scale

Look at the file for an example.  Download 4Channel

 

9.1 is expiring

It's true. FlowJo 9.1 is expiring. 

9.2 is here: http://www.flowjo.com/home/macintosh.html (with expiration removed.) 

This was done to encourage users to keep their version up to date (easier on tech support) but apparently it rather angered some of you. Our Apologies, won't happen again :) 

Quad Gates Using FJML

This is an interesting application of FJLM/Vertex gating in FlowJo V9.2 - driving the center position of a quad gate based on another gate's position or summary stat. 

Let's say you like using quad gates - and you want to tie the center position to more less where the negative cells end on x/y axes. Here's how I assembled that.