...
But back to that 10% figure--why so much emphasis on time-consuming (thus expensive) teacher evaluation models for a small gain of a small percent? That gave rise for me to once again look to my favorite evaluation system and the evidence supporting it. I found this statement and things immediately went off the rails:
A summary:
1. The research claimed here is incestuous; it was carried out by an affiliated organization.
2. This is not disclosed properly (see image above, source is on page 3 of this document).
3. A potential lack of credible (academic) expertise.
4. No evidence it was peer reviewed.
----------------------------------------------------------------
So the first problem was trying to just find the article, which was strangely difficult--red flag #1. When you can't even find the abstract, that's odd. Finally going to Google Scholar, it turns out that the latter one has just two citations (one by a book, one by the RAND Corp.):
The reasons for this became apparent pretty quickly:
1. Dr. Lindsey Basileo works for Learning Sciences International. Their About page states, "Our Centers for research and development include Learning Sciences Marzano Center with Dr. Robert J. Marzano, to develop instructional frameworks as well as evaluation and growth systems." So this isn't independent research, and a gross lack of disclosure. I would say this is Red Flag #2, but honestly, by academic standards, it's game over at this point--credibility is out the window. Especially when they didn't make the connection between product and reviewer/researcher clear.
2. Dr. Basileo's PhD is in Criminology and Criminal Justice (along with her MS and BS). So while an expert, her expertise doesn't seem particularly relevant.
3. Michael Toth is the CEO of Learning Sciences International. Same problem as #1. Additionally, a few "suspicious" things: Googling the organization he was former president of ("National Center for the Profession of Teaching") in quotes seems to indicate said organization now exists only on his biography (Guidestar indicates it may have either merged with another organization or gone defunct). I also noticed he's referred to only as "Michael Toth"; unlike the "Dr." they put in front of Marzano and Basileo and there's no mention of his education on his LinkedIn. (Admittedly this proves nothing, but given their focus on marketing, I find it suspicious they'd leave off the alphabet soup and the credibility it lends.)
4. At this point, I think we can safely say this research supporting Marzano's system is incestuous; it came from an affiliated (parent?) organization and this wasn't properly disclosed. Further, between these obvious problems, the lack of citations, and the difficulty in even finding a copy of the research in question, I think I can safely assume it was not peer-reviewed. Which from my perspective, renders an already discredited bit of research totally worthless.
