Monday, July 17, 2017

Public Education's PR Problem (and more spurious "research")

So I finally found an article linking to that old statement I made that teachers make very little impact on test scores (~10%).  The Atlantic article itself is great and highlights the PR/perception (and lack of leadership...) issues facing American public education.  This has been long known; the late Dr. Gerald Bracey wrote about it in 2010 and touched on using the bad perception as a "scare tactic" back in 2007.


But back to that 10% figure--why so much emphasis on time-consuming (thus expensive) teacher evaluation models for a small gain of a small percent?  That gave rise for me to once again look to my favorite evaluation system and the evidence supporting it.  I found this statement and things immediately went off the rails:

A summary: 
1.  The research claimed here is incestuous; it was carried out by an affiliated organization.
2.  This is not disclosed properly (see image above, source is on page 3 of this document).
3.  A potential lack of credible (academic) expertise.
4.  No evidence it was peer reviewed.

So the first problem was trying to just find the article, which was strangely difficult--red flag #1.  When you can't even find the abstract, that's odd.  Finally going to Google Scholar, it turns out that the latter one has just two citations (one by a book, one by the RAND Corp.):

The reasons for this became apparent pretty quickly:

1.  Dr. Lindsey Basileo works for Learning Sciences International.  Their About page states, "Our Centers for research and development include Learning Sciences Marzano Center with Dr. Robert J. Marzano, to develop instructional frameworks as well as evaluation and growth systems."  So this isn't independent research, and a gross lack of disclosure.  I would say this is Red Flag #2, but honestly, by academic standards, it's game over at this point--credibility is out the window.  Especially when they didn't make the connection between product and reviewer/researcher clear.

2.  Dr. Basileo's PhD is in Criminology and Criminal Justice (along with her MS and BS).  So while an expert, her expertise doesn't seem particularly relevant.

3.  Michael Toth is the CEO of Learning Sciences International.  Same problem as #1.  Additionally, a few "suspicious" things:  Googling the organization he was former president of ("National Center for the Profession of Teaching") in quotes seems to indicate said organization now exists only on his biography (Guidestar indicates it may have either merged with another organization or gone defunct).    I also noticed he's referred to only as "Michael Toth"; unlike the "Dr." they put in front of Marzano and Basileo and there's no mention of his education on his LinkedIn.  (Admittedly this proves nothing, but given their focus on marketing, I find it suspicious they'd leave off the alphabet soup and the credibility it lends.)

4.  At this point, I think we can safely say this research supporting Marzano's system is incestuous; it came from an affiliated (parent?) organization and this wasn't properly disclosed.  Further, between these obvious problems, the lack of citations, and the difficulty in even finding a copy of the research in question, I think I can safely assume it was not peer-reviewed.  Which from my perspective, renders an already discredited bit of research totally worthless.


  1. Hi Ellis,
    I'm doing research into Marzano's strategies and their usage across subjects in a specific county in FL and came across your blog. Is there a way I could contact you outside of the blog? I'd love to ask you a few questions.


  2. thanks for your excellent analysis Ellis, we need more teachers like you doing this sort of thing. ResearchED is a grass roots organisation trying to get teachers together to present this sort of analysis. I will read with interest all your other analysis of Marzano. ReseachEd can be found here -

    Also, you maybe interested in a paper by Prof John O'Neill who describes Hattie (i think also Marzano) as 'policy entrepreneurs' -

    "public policy discourse becomes problematic when the terms used are ambiguous, unclear or vague" (p1). The "discourse seeks to portray the public sector as ‘ineffective, unresponsive, sloppy, risk-averse and innovation-resistant’ yet at the same time it promotes celebration of public sector 'heroes' of reform and new kinds of public sector 'excellence'. Relatedly, Mintrom (2000) has written persuasively in the American context, of the way in which ‘policy entrepreneurs’ position themselves politically to champion, shape and benefit from school reform discourses" (p2).

    keep up the good work!

  3. Extremely helpful post. This is my first time i visit here. I discovered such a large number of intriguing stuff in your blog particularly its exchange. Truly its extraordinary article. Keep it up. act test dates

  4. This suspicion by ages and dependably is taken care of causing peeps out supposition and spots formal education as thing which sacral. best past papers