BEE: Brain Extraction Evaluation Web-service
INC is pleased to announce a Brain Extraction Evaluation Web Service (BEE). The goal of this service is to provide interested users with (i) a quantitative evaluation of the performance of their "in-house" brain extraction algorithms on a downloadable test dataset and (ii) a performance comparison against McStrip, BSE, and BET. Each algorithm can be evaluated against two sets of expertly hand-stripped masks for 15 test volumes* with respect to percent correct boundary captured, average distance between brain-mask and target boundaries, and misclassified tissue volume. The location of the results will be e-mailed to you as soon as they become available.
*Acquired on a Siemens 1.5T scanner using 3D FLASH: TR 35 ms, TE 6 ms, FA 45 degrees, 1 NEX, FOV 165 x 220 mm, matrix 192 x 256, voxel dimensions 0.86 x 0.86 x 1.0mm. These subject volumes have undergone "facial modification" to ensure anonymity. The BET, BSE and McStrip algorithms tended to perform slightly worse with the anonymized volumes than with the originals; the differences were on the order of two percent on boundary and similarity metrics.
How to Test Your Algorithm
1. Go to the NEUROVIA anonymous FTP site (ftp://neurovia.umn.edu/pub/BEE) and move to the input directory. Download the 15 test T1-weighted MRI volumes. They can be downloaded as individual files named subject00.img...subject14.img and subject00.hdr...subject14.hdr or as one file in a gzip-ed tar ball. There are a total of 30 files: 15 image files and 15 header files in Analyze format.
2. Run your brain extraction algorithm (BEA) on one or more of the 15 image volumes.
3. Make sure that your files containing the extracted brains (or brain masks) are in Analyze format and that they have the same orientation as the original volumes, i.e., left-right, anterior-to-posterior, and inferior-to-superior.
4. Name your files using the following convention: subjectNUMBER_NAME.img and subjectNUMBER_NAME.hdr, where "NUMBER" is the subject number (from 00 to 14), and "NAME" is a shorthand for your BEA, e.g., subject03_YOURS.img.
5. If you wish to upload compressed files, we support .gz and .Z file extensions. Remember to follow the above naming convention before you compress your files.
6. Return to our anonymous FTP site (ftp://neurovia.umn.edu/pub/BEE). Create your own directory and upload your stripped-brain or binary-strip-mask volumes (one per file) to that directory. Be sure to include header (.hdr) as well as image (.img) files.
7. Go to BEE Submission (link below)
8. Wait for an e-mail reply containing the location of postscript files with your results. The minimum processing time is 4 hours, and each postscript file (one per brain volume) will be approximately 900K. Click here for an interpretation of the results contained in the postscript files.
9. If you do not receive a reply within two working days, please contact us via e-mail at .