Two recent graduates of the Teachers College Columbia University speech-language pathology program, supervised by Cate Crowley, created and presented this poster at the ASHA convention in Chicago, 2013 detailing construct validity concerns with the PLS-5. The work was initially done as part of Cate’s Assessment and Evaluation class at Teachers College.
Alexandra Gibson and Elana Winters presented this poster at ASHA’s convention in Chicago in November 2013. Alexandra and Elana created this poster based on work they did in the Assessment and Evaluation course at Teachers College Columbia University. This course is required as part of the master’s SLP program there.
In that course the students must analyze the most widely used standardized tests to identify disorder/disability for children and adolescents. They analyze the tests’ validity, reliability, and significant biases among other factors. The information for the analysis is primarily from the test itself—generally the interpretive and technical manuals and the other information provided to anyone who purchases the test. That material is analyzed in light of the standards of the field, including the work of Dollaghan, Vance and Plante, Restrepo, and going back to even McCauley and Swisher‘s work of 1984. In the class students find and incorporate into their analysis any research on current or older versions of the test. Another part of the task is to analyze the significant biases including cultural and linguistic biases and biases related to socio-economic background.
I am always surprised when SLPs, psychologists, and other evaluators don’t know the most basic information about the assessment materials they use to identify disabilities. I can’t understand how anyone can use any test to identify disability without knowing its sensitivity or specificity, how spectrum bias might affect the discriminant accuracy of the test, significant bias issues, what the reference standards are, or why Standard Error of Measurement is critical even if the test is valid, reliable, and free of significant bias. There is just too much at stake when we are evaluating a child or adolescent who is having trouble in the classroom (What’s wrong with identifying a child with a disability when he/she doesn’t have one?) and we are asked to identify whether there is a disorder/disability requiring and IEP, or “something else.”
It really isn’t that difficult to learn. In the 15 years I have been teaching Assessment and Evaluation I have never had a student who could not learn this in a semester. This is further reinforced in the clinical diagnostics course the students take generally in their last semester of the master’s SLP program. But most clinical graduate programs especially SLP, psychology, and OT seem to drop the ball in this area. Any clinical professional must be able to analyze any assessment material they use. Otherwise they will not be able to distinguish a disorder/disability from “something else.”
It is a challenge to determine whether a child or adolescent has a disability, but that is our job—according to the federal law on special education, state regulations, and our professional codes of ethics. We have a great deal of information on how to distinguish a disability from “something else.” We need advanced degrees because it is not just giving a test and getting a single standardized score—we certainly know that even if the test is valid reliable and free of significant bias. It it does require a level of competence, knowledge of the research, and skill in evidence-based practice which we owe to the child or adolescent being evaluated and their families.
Congratulations to Alexandra and Elana!
