Since this is the season of ghouls, goblins, pumpkin (spice in everything), elections (in VA), and reporting (well, not particularly seasonal at MNA since we haven’t stopped writing at all this year!), I want to share a few thoughts on reporting.
Recently, I read (rather thoroughly) a summative evaluation report completed by another evaluation firm. I have to say, I was disappointed – not because of its format, layout, dataviz, etc., – those were professionally done, but with the long list of recommendations (or areas of enhancements) they made to the client. This list, shall I say, was spooky, bordering on condescension of the client and their work in the communities they serve.
At the end of it, the report left me wondering, why would an evaluator:
1. Provide a laundry list of what the client didn’t do at all / well as proposed to the funders
2. Not provide ways and methods to remediate the issues, process(es), and mechanisms at play
3. Not mention clearly that they didn’t conduct a formative evaluation (it was an annual report only per their contract deliverable)
4. Bash their clients publicly as though it was a “gotcha” moment of triumph for them, and
5. Derive “evidence” from correlations which are certainly not causations.
When evaluators as practitioners say that we are collaborative, then why don’t we (well, some of us) walk the talk? Who benefits from the report and its findings and what are the broader impacts?
Neither party is happy. Money is spent. Seldom any gains are made.
Needless to say to most of the experienced evaluators here, but formative feedback, periodic check-ins are essential for any evaluation process – big or small. If the clients “forget” to inform us of the happenings, it’s our responsibility to ask. We are equal stakeholders when it comes to information gathering, data access, sharing, and exchange.
If the evaluation finds issues with the program -its planning or implementation, how about creating a succinct public facing document with a list of actionable and doable recommendations within the context of the program and another memo-type document with details outlining issues as observed with potential solutions, remediation methods, and ideas?
Being spooky in evaluation reporting doesn’t help anyone, folks. Just saying.
Happy Halloween!
Pic taken from the internet. Would absolutely love to credit whoever did the jack o’ lantern. This continues to be my all time favorite.
