The Daily Dongle

FlowJo V10 build for linux

Hello Readers, 

we're still not running in linux. We are stuck setting up stepping thru C++ code in linux (debugging.) if you or someone you know is a C++ developer comfortable with Linux, we'd like to see a resume and discuss a contract. I believe currently our search is limited to Oregon, but we will consider other candidates based on their resume.

last updated by Maciej Dec 28th 2011 

 

known issues: 

- 64 bit linux / 64 bit Java VM required. 
- we run into random but frequent engine crashes when loading data. We're not sure if this is because we're developing this on VMware or if becasue there is a linux-specific engine bug. Your feedback on this behavior would help us figure this one out. 
- dragging files from file system to FJ window won't work yet. Do this via the "add samples" button. 
- this build follows the regular V10 beta licensing and release schedule (expires weekly on Wednesdays, requires beta site license. )

contact simm@treestar.com with any questions about FJ linux, otherwise download, try it, and let me know if you crash :)

-ms  

 

 

 

FJ10 in linux 64 bit = 30 million events / second

since our support to the linux savvy fj users has not been top-notch, I installed ubuntu 10.10 64 bit and will be spending some time in that testing & fixing FJ as needed. It's now my job to make FJ10 and linux bff's ;D

So, today i got my IDE running sun's java, put my honkin files on a ramdisk and got to testing:

above, i got nine 10.5M event files repainting the contents of the gate in the graph window. From moving the gate to update in the layout, 3s elapse for each gate movement. I have the "synch" attribute enabled so each gate movement in one sample is instantly propagated to the others & all 9 files are recalced. since this test involves manipulation of 9 x 10.5 M events, im claiming roughly 30M events /second performance on this 16 core 2.2GHz machine.

FJ on linux is not without problems, but at least i'll be working on resolving those. Performance wise this matches or exceeds win7 b64 bit, pending some more detailed testing.

minor changes to Table Editor in next 7.x beta

I'm working on some added functionality to CSV saving/batching from the table editor.  

 

the panel above is found in Edit -> Preferences -> Tables. 

The first one is self explanatory.. if unchecked, you won't get the "mean/average" rows added to your csv output. 

The second one - normally if you batch your table output to "c:\table.csv" and the file already exists, flowjo will create an output file called "table-1.csv" in that same path. 

If the "append" preference is checked, FlowJo will instead append the current table to the existing file's contents (adds after current content, ie at the bottom of the existing file. )  

Anything else here while I'm at it? 

Autobatching & Command Line tutorial

Download autobatch tutorial [MOV, 18.2MB]

Summary: 

Auto-Batching is a feature which can be utilized when running flowjo in Command-Line Mode (CLI). CLI was developed to give the set of features necessitated by large volumes of data set that may need automated pre-processing prior to 'attended' analysis - pre-gating, compensation calculations,  exporting gates to smaller FCS files, are some examples of tasks that fall into that category. 

AutoBatch is best initiated from an existing template which has things like: 

- dynamic live groups with group-owned-gates
- tables defined in the Table Editor
- layouts defined in the Layout Editor

AutoBatch in Table Editor: 

- open an existing Table in your Table Editor. Click Edit menu >> Include in Batches. 
- define an output for your table that gets written to disk. Click Batch button, then set the option for "File", pick a flavor & output destination, and hit "Save" and "Apply" buttons. This commits the autobatch settings to this particular table.
- notice the tables which were autobatch enabled, are listed in BOLD typeface.

AutoBatch in Layout Editor: 

- open an existing layout, click Layout menu >> Include in Batches. 
- Click Batch button and define some sort of File output (PDF, PPT etc.) and destination. Remember to hit "Save" and "Apply". 
- notice autobatched layouts are listed in BOLD typeface. 

AutoBatching file export/concatenation:

- this is initiated slightly differently from the above two, since there is no "batch" button. 
- to autobatch the export or concatenation of UNGATED FCS files to csv/fcs data on disk, right-click the group definition at the top level, click Export/Concat. In the Export/Concat dialog select your destination (currently ignored, defaults to template location) and file flavor, as well as whether or not to concatenate the data. Lastly, check the "save in template" checkbox - this commits your actions to the template's memory. 
- to autobatch the export or concatenation of GATED data (ex. "merge all my T cell gates into one CSV/FCS file"), right-click the group definition at that gate's level, then click Export/Concat. After this point the directions match the "ungated" export/concat. 

Executing "AutoBatch" templates: 

this requires either access to the command prompt or ability to generate and execute .bat (batch) files. 
the command syntax follows: 

c:\path\flowjo.exe c:\path\template.wspt c:\path\experiment1 -save -quit

where the flowjo exectuable (path + exe name)  is called with four arguments: the input template (path + document name), the input data (path), and two optional arguments to save the executed work into a workspace file. The "-quit" argument is optional but recommended to free up any runaway RAM consumed by each batch :) 

working on tab delimited text files -> flowjo

One of the very first posts on the DD, and one of the most frequent - and oldest- of feature requests, is to read in non-flow files into FlowJo. Currently the mac version sort of does this, but you have to hand-write the text header. 

I think we can improve that by just supporting any (for starters, tab delim'd data) 2D array text file and figure out the ranges on the fly. And even make an FCS file out of it. I hope to have a demoable prototype before CDW. For now I'm limiting my scope to 10bit FCS2 format (bigger/smaller data will be normalized to "fit" in ten bits).

BD's multicolor flow utils

http://www.bdbiosciences.com/research/multicolor/tools/index.jsp

neat!

FlowJo translation project

We're starting to translate FlowJo into other languages. Please leave a comment on this post if your language should be added to the list. Currently we are making progress in Chinese and Polish (we just happen to have prolific translators fluent in these, as well as German and French, which will likely be next.)

 Anything else?  

We might be interested in translators for languages other than the four mentioned - unicode experience is a must. Let us know if you'd be into it, and maybe we can discuss how to make it worth your while (probably limited to feel-good-points and T shirts.) FlowJo pratar inte Svenska!

7.5 beta - 3D overlays, with transparency

quicktime tutorial, ~1.5MB: click on the graphic below to download the movie explaining how it works.

Text version: 

- edit prefs file, include "offset3D="1" somewhere in the <prefs... > element. 

- restart FJ

- open a graph window, format Y axis as histogram

- drag that node to the layout

- drag other gates on top of the first plot

- right-click, offset histograms

- use the handle on the lower-right edge of the plot to adjust perspective. 

- use the legend (right-click) to set coloring, line weight, style etc.. as usual. 

visualizing apache server stats in FlowJo

Using the old FJTEXTDATA trick, I loaded some HTTP stats into FlowJo and viewed it as listmode data. First, I chose to correlate Days of Week and Months of Year vs. Hits. I extracted Days of Week by doing a number series (1-7 repeated ). I did the same trick for months assigning them values of 1-12. FInally each event was tagged with that day's hits. So one event on this scale is one day, and the values of that event are {hits, day of week(1-7) and month of year (1-12). Enough words. Pictures: 

Sundays got labeled as 8 for some reason, otherwise looks as predicted. 

Gated for Saturday and Sunday events only, here's a kinetics platform view (each timeslice is ~1 year) 

vs weekday stats..

and even in 3D: 

hacking polyvariate platform for precise controls

If you've ever used the polyvariate platform, you might have noticed the only way to control it is with your mouse, which is imprecise.

if you would like full hands-on control of the PV platform, save your FJ workspace as XML (file/save as xml) and look for these snippets:

<VectorTransformation VectorPivot="0.25" XMin="0" XMax ="1" YMin="-1" YMax="0">
<VectorDefinition Parameter="FSC-A" Pivot="0" XFactor="1" YFactor="0" />
<VectorDefinition Parameter="SSC-A" Pivot="1" XFactor="0" YFactor="1" />
<VectorDefinition Parameter="&lt;FITC-A&gt;" Pivot="0.25" XFactor="0" YFactor="-1" />
<VectorDefinition Parameter="APC-W" Pivot="0.25" XFactor="-1" YFactor="0" />
</VectorTransformation>

the top level element variables are the radius of the circle (out of 1 max), and positions of the X and Y axis sliders.

all of the subsequent elements show three variables: the pivot point, and the magnitude of the radial vector expressed in X and Y components: (1,0) is 3 o'clock, (0,0) is the origin, (-1, -1) is a median between 6 and 9 o'clock

You can change the values for each attribute, save your .xml, rename it to .xml.jo and re-open in FJ.