Objective: The applicability of existing ROC analyses [1] to the evaluation of fMRI processing pipelines for real datasets is unknown due to their reliance on simulation. Since the NPAIRS approach works on real data [2, 3], we used its prediction and reproducibility performance metrics in order to evaluate the impact of preprocessing steps and statistical models in single-subject analysis.

Methods: A 16-subject, block-design, static-force motor-activation fMRI dataset was used.

(1). Evaluation of the Impact of Preprocessing Steps
The impact of the preprocessing steps was investigated with NPAIRS Canonical Variates Analysis (CVA). For each pipeline tested, the Euclidean distance of each prediction and reproducibility pair from (1, 1) was calculated and compared for pipelines with and without a particular preprocessing step, and a Wilcoxin matched-pair rank-sum test was performed on these distances for the comparison, across the 16 subjects.

(2). Evaluation of Analytic Models
In this study, FSL.FEAT, SPM2 and NPAIRS.GLM were chosen as the univariate GLM (General Linear Model)-based models, and NPAIRS.CVA was selected as the multivariate analytic model to evaluate. In total, nine heterogenerous processing pipelines with these baseline-activation models were built with identical preprocessing parameter settings (e.g. 2-pixel spatial smoothing) except for the following:
NPAIRS.CVA0--2 cosine-basis detrending, 5 Principal Components (#PCs);
NPAIRS.CVA1--optimized #PCs;
NPAIRS.CVA2--optimized detrending and #PCs;
NPAIRS.CVA3--optimized smoothing, detrending, #PCs;
FSL(3.2).FEAT--128 sec. high-pass filtering cutoff;
NPAIRS.GLM1--2 cosine basis detrending;
NPAIRS.GLM2--optimized detrending options generated from CVA2;
NPAIRS.GLM3--optimized smoothing and detrending options from CVA3;
SPM2--high-pass filtering 128 sec. cutoff.
The variance of the SPMs (Statistical Parametric Maps) generated by the nine models was analyzed through NPAIRS.CVA with between-subject variance removal.

Results & Discussion: (1). Table 1 demonstrates that for block-designs, slice timing correction and global intensity normalization have little impact on the fMRI processing pipeline, but, in order of importance, spatial smoothing, low-pass filtering, temporal detrending, motion correction and high-pass filtering significantly improve pipeline performance. (2). Figure 1 illustrates that the differences between GLM and CVA SPMs accounts for the largest variance (CV1) followed by individually-optimized smoothing (CV2).

Conclusions: These NPAIRS evaluation results demonstrate that the most important pipeline choices include univariate-or-multivariate data-analysis approaches followed by spatial smoothing optimization.

References & Acknowledgements:
[1] Skudlarski P., et al. (1999). NeuroImage 9(3): 311-329.
[2] Strother S. C., et al. (2004). NeuroImage 23(S): 196-207.
[3] LaConte S., et al. (2003). NeuroImage 18(1): 10-27.

This work is supported in part by NIH grant EB002013.