Key Findings from Listen for Good Six-Month Survey Results
The data from the Listen for Good six-month grantee surveys have provided positive feedback and detailed areas where growth can occur.
Positive findings related to grantee experience with Listen for Good implementation:
- Grantees appreciated the Listen for Good survey template and the flexibility to administer surveys in ways most appropriate for their organization
- Grantees value the Listen for Good technical assistance and supports, especially the early support offered in the survey design phase
- Program managers are finding useful variation in the feedback data they receive from constituents
- SurveyMonkey was found to be helpful when interpreting results
- Grantees are using results for logistical and programmatic changes
- Agency leaders affirm that the Listen for Good process will yield actionable data for their organization
Challenges grantees encounter with Listen for Good implementation:
- The most common challenge in designing the survey was developing the custom questions
- Grantees are struggling with low response rates and unique constituent needs
Resulting Organizational Change:
- Grantees feel that participating in Listen for Good has better equipped them to collect constituent feedback
- Agency leaders see engaging in Listen for Good as beneficial to both their internal operations and their service to constituents
- Barriers to adopting and implementing organization-wide constituent feedback processes are capacity, resources, and low response rates
Key Findings from 2016 Grantee Interview Themes
Feedback Practice and Research Grantees
- There continues to be a positive sense of momentum. Many grantees are seeing a growing interest among a broader audience and increasing sophistication in their conversations about feedback.
- Key components that contribute to building a culture of feedback are leadership support, internal capacity around implementation, and relationships with the constituents from whom grantees are collecting feedback.
- Some grantees have seen a shift in the degree to which they are making the case for feedback. Conversations are focused more on advanced aspects of “how” rather than “why”.
- Grantees providing capacity building and support around feedback see greater demand among their audiences since last year, including more donor feedback participation, more people attending feedback-related events, and greater uptake in consulting and related services.
Foundation Openness Grantees
- Grantees were all making individual progress, with some developing digestible content around the topic of openness and clearer messages for communicating about openness over time.
- While grantees are on track and not experiencing pushback in the field around the idea of openness, they saw less uptake in their work than they had hoped.
- Grantees did not experience any level of synergy with each other in the way the feedback practice grantees did, nor was there any sense of cohesiveness of views in the sector or field around openness.
Key Findings from Media Analysis: Two-Year Follow Up Analysis From Baseline Report
- Fewer instances of relevant “chatter” overall - compared to previous year, there is slightly less content with greater alignment around beneficiary feedback loops*
- Relevant content shows up most frequently in blogs – this suggests that beneficiary feedback loops are a popular topic of “informal” discourse in the field
- Within feedback discourse, discussion around using feedback and closing the loop have decreased
- Fewer instances referred to both systematic feedback and closed loop feedback
- More instances of feedback were specifically from beneficiaries
*We attribute this decrease in “chatter” around beneficiary feedback loops to our decision to refrain from writing about and encouraging media coverage of our newly launched Listen for Good initiative as it was highly experimental and unproven during most of 2016.
Key Findings from Theory of Change Progress and Lessons
- Shared Insight’s work on feedback practice is on track to date. While it is early to see changes in the desired outcomes, there seems to be forward momentum particularly related to nonprofit practice of feedback loops and related discourse. The addition of Listen for Good helps expand the scope and scale of efforts in this area of work. Progress related to funder engagement around feedback practice may be less on track overall; although there is explicit funder engagement in Listen for Good, it remains to be seen how engaged funders will be.
- Shared Insight’s work with its feedback research grantees is also on track to date. With the addition of the Innovations for Poverty Action grant, the two areas of research in the theory of change are now being covered. Given the timeline of the projects, it will be awhile before we know what findings result from the research.
- In the area of increasing foundation openness, Shared Insight is slightly off track. While individual grantees seem to be meeting expectations, it does not appear there have been synergistic effects as hoped. Unlike other focus areas, Shared Insight has not augmented its approach since the first year grants to address early gaps or new opportunities. However, Shared Insight released a new open RFP in March 2016 to address this gap.
- Overall, Shared Insight has taken an emergent approach to its grantmaking, acknowledging from the beginning that, unlike an approach that begins with a strong set of hypotheses about change, there is much to learn about how to improve philanthropy so that foundations better listen and share to support greater impact. This work to date is laying the foundation for long-term changes that potentially will occur outside the life of Shared Insight itself and progress should be assessed with this timeframe in mind.
Key Findings from Grantee Interviews
- Grantees are making valuable connections with other Shared Insight grantees, particularly those implementing and helping others implement feedback loops
- Rising sense of momentum around the idea of feedback loops, but less so about actual implementation
- Ideas about how to change philanthropic practice around openness are varied- for some it is top-down, focusing on leadership; for others it is focusing on program staff
- One noted headwind was that many communications staff at foundations are reluctant to engage in openness conversations
- Some grantees cautioned against a tone that suggests the work around feedback loops are "new" and to avoid appearing exclusive
Some challenging aspects of implementing feedback loops for our grantees are:
- Maintaining quality implementation (including training and prioritizing among staff and volunteers)
- Closing the loop in a meaningful way
- Making sense of feedback
- Ensuring feedback is candid and varied enough to be constructive
- Getting quality information at scale
Key Findings from Media Analysis: One-Year Follow Up Analysis From Baseline Report
- More Chatter, More Alignment - Compared to previous year, there is slightly more content with greater alignment around beneficiary feedback loops
- Similar Venues, New Voices - A similar set of publications, including blogs and periodicals, contain content related to feedback while several new blogs launched that cover beneficiary feedback issues
- There was an increase in the number of authors writing about feedback loops
- More instances of media pieces advocating for feedback use compared to previous year, both in nonprofit and foundation practice
- The uptick in discourse is likely connected to the Fund for Shared Insight launch, but shows promise of taking off beyond Shared Insight’s work.
Key Findings from Feedback Loops and Openness: A Snapshot of the Field Baseline Report
- Foundations understand conceptually what beneficiary feedback loops mean, but few have strong internal practices for intentionally collecting and putting to use feedback that comes from "the people they seek to help"
- The three most common barriers to implementing feedback loops into foundation practice are organizational capacity, organizational culture and technical challenges
- Prior to the launch of Shared Insight, ORS Impact found some instances of feedback-focused content in a broad based review of sector-related blogs, reports and publications but definite room for more voices discussing this work
- The two most common barriers to foundation openness are organizational culture, including a fear of sharing failures, followed by time and resources
As a funder group, we reflected together on these findings and agreed that while the baseline report indicates that nonprofits and foundations seem to be talking about feedback loops, there isn’t a widespread understanding of how to do it well, how to integrate it into practice, and how to take action based on the feedback. We suspect there is an overstatement of the extent to which foundations actually listen to feedback and share openly – but we believe the upside is that there is good intent and a willingness to do more in these areas.
Our conclusion thus far, is that there is an opportunity for Shared Insight to develop better tools and resources/frameworks for nonprofits and to demonstrate the value of feedback loops and openness to nonprofits and even more so to foundation audiences. We believe more depth of discourse about feedback loops and openness is needed and it is possible that our future funding to increase foundation openness will be more oriented toward listening and action.