SPNT and Bad Science

For those of you who have yet to hear about the Smooth Pursuit Neck Torsion test, it's had an impeccable research pedigree. It's one of the few tests that has great separation of the data between normal controls and injured patients, meaning that injured patients produce a value on the test that isn't found in normal subjects. The test is also objective, very difficult to fake, and part of an ENG battery that's been performed since the 1950's. That's why we were surprised to come across the paper by Kongsted which seemed to show that the test result produced by SPNT (Gain), was not statistically different in a whiplash group and a normal control group. As usual, once you peer "under the hood" it becomes pretty clear why they got this result. For more on those details, see below. However, this brings up a much bigger issue about research and academics. Clearly, the Kongsted group is not made up of clinicians, but of academic researchers. Why is this clear? For clinicians performing research, the easiest subjects to recruit are patients (we have lots of those) and the hardest are "normal controls" (we don't see those). As discussed below, this group had no problem finding almost twice as many normal controls as patient subjects.

The bigger issue is why are academics (who don't normally see these patients) getting involved in research about a test that has only clinical value? The motive becomes clear when one investigates that the source of funding for the study was in fact that auto insurance industry in Denmark.

Here's the editorial just submitted by Micheal Freeman any myself...

Editorial Submission on Kongsted, A., et al., Are smooth pursuit eye movements altered in chronic whiplash-associated disorders? A cross-sectional study. Clin Rehabil, 2007. 21(11): p. 1038-49
Christopher J. Centeno, M.D., Michael Freeman, Ph.D., M.P.H.

We read with much interest the research on the Smooth Pursuit Neck Torsion test and WAD published by Kongsted.[1] Since this study had very different conclusions from other research on this diagnostic test, we decided to investigate further.

The authors state, “The sample size needed to reach a power of 90% with a significance level of 5%... Under these conditions we required 40 cases and 60 controls.” With this statement we were surprised to find only 34 subjects. It also seemed odd to us that controls outnumbered subjects almost 2 to 1. Usually normal controls are very difficult to recruit. Table 1 shows that only 15 of their subjects had daily neck pain, 12 had daily headache, and only 4 had daily dizziness. Even if we add back the subjects without daily, but severe pain that occurred on an infrequent basis that leaves at most (15 + 11) 26 WAD subjects who had any type of symptom complaint that would likely cause them to seek specialty care. This is now significantly less than their power calculation revealed would be needed.

Yet another issue with the WAD group is the number of patients with moderate or severe daily neck pain (15/34) vs. the number of patients with moderate or severe daily dizziness (4/34). If one looks at this data from the other direction, there are 2/34 WAD subjects with mild occasional, regular, or daily neck pain, but 11/34 WAD subjects with mild occasional, regular, or daily dizziness. This difference is significant, as Treleaven found that this test to be positive in 90% of patients who reported dizziness vs. 54% of patients who did not.[2] Clearly the group most likely to have a positive test (WAD patients with dizziness) is skewed toward mild symptoms.

In order to compare this tested WAD group to the severity of prior studies, we contacted Treleaven. While her SPNT paper did not include severity data, this study was part of a more comprehensive investigation which did measure this data. Mean (SEM) NDI of their WAD non- dizzy group was 34.4(2.0) and WAD dizzy group was 46.4(2.1). If one looks at the interpretation of this data by the original authors of the neck disability index (NDI), Treleaven’s non-dizzy group would be categorized as “severely disabled” and her dizzy group would be categorized as “completely disabled”.[3] While Kongsted did not use NDI, the review of their pain frequency data above would clearly define these groups as significantly less severe than either of the Trealeven subgroups.
The authors also state, “A box plot revealed that patients with severe neck pain or headache during the previous two weeks tended to obtain lower gains than patients with mild or moderate pain…” This is not surprising, as above, we discuss that the recruited subjects had very mild symptom frequency. Here the authors confirm our suspicion that their own analysis showed that patients with more frequent symptoms tended to have more positive tests. The authors also state, “Lower gains were slightly more frequent in patients with dizziness, but the results were almost similar to those for patients without dizziness (Figure 3).” Again, this trend is not surprising given the fact that the dizziness in the subject group is heavily skewed toward mild symptoms. It should be noted that the total number of subjects with dizziness is only 27, more than 30% lower than the n required by their power calculation to reach significance with an expected gain difference of 5%.

From our review of this paper, the conclusion should read, "When we compared a statically underpowered group of mild WAD and even more mild dizzy WAD patients to a larger control group, we did not detect significant differences in gain between our small mild WAD group and the normal control group."

1. Kongsted, A., et al., Are smooth pursuit eye movements altered in chronic whiplash-associated disorders? A cross-sectional study. Clin Rehabil, 2007. 21(11): p. 1038-49.
2. Treleaven, J., G. Jull, and N. LowChoy, The relationship of cervical joint position error to balance and eye movement disturbances in persistent whiplash. Man Ther, 2006. 11(2): p. 99-106.
3. Vernon, H. and S. Mior, The Neck Disability Index: a study of reliability and validity. J Manipulative Physiol Ther, 1991. 14(7): p. 409-15.