Impact
How Listen4Good Has Made a Difference
We think it’s important to look at the impact we’ve had on organizations in multiple ways. Our overall focus, though, is simple: To actively review our results and use them to improve what we do. Our own feedback loop!
of Listen4Good organizations make or have plans to make changes in response to feedback within 1 year
More than
of organizations continue to collect feedback one year after graduating from Listen4Good
of organizations in recent surveys affirm that L4G has influenced their use of feedback data, as a complement to outcome data, to inform internal decision-making
Feedback Stories
Check out these stories to get a sense of what it’s like for organizations to experience Listen4Good.
Case Studies
Explore how organizations have used Listen4Good to drive meaningful change for their communities.
Evaluations
During our first five years, Listen4Good was routinely assessed by third-party evaluation firm, ORS Impact, along with a range of other research partners. Read studies from them and other research collaborators below.
-
Feedback and Equity: Connecting the Dots
ORS Impact engaged in an open-ended inquiry to clarify the conceptual connection between feedback and equity, aiming to identify examples and patterns of feedback practices to help guide nonprofits and funders in what to look for, what to focus on, and how to create practices that support equity work. ORS interviewed multiple staff members at six nonprofits that participated in Listen4Good, exploring the extent to which feedback contributes to their understanding of the inequities their clients face and how they leverage feedback and listening practices to give clients more control over resources and decisions.
- Feedback is contributing to organizations’ equity work in two main ways: insights from feedback inform changes in programs and organizational policies that advance equity, and the act of listening itself is influencing organizational change.
- For clients, feedback is personal; their feedback includes perspectives, personal experiences, and feelings they choose to share to contribute to organizations’ understanding of their lived experience and barriers they face. While most feel heard and valued, there are still opportunities for improvement.
- Organizations are shifting from having power over clients to building power with clients, and feedback and listening contributed to those practices.
- Feedback and listening contributed in different ways to organizations’ equity work; it acted as a catalyst, mirror, or compass, depending on how feedback and listening were leveraged by each organization.
- Collecting client feedback doesn’t automatically make organizations more equitable; organizations must intentionally design processes and create a culture that infuses equity into the way they gather and use feedback to understand and address the inequities clients face.
-
Feedback Sustainability
ORS Impact surveyed nonprofits one year after the end of their participation in Listen4Good to learn if they were still implementing high-quality feedback loops and what that work looked like after they were no longer receiving support through Listen4Good. To gauge the quality of feedback, organizations were asked to what extent they continued using practices associated with the five steps identified by Listen4Good as comprising a high-quality feedback loop: design, collection, analysis, response, reporting back to survey takers.- A majority of organizations in the 2017 and 2018 cohorts continued to collect feedback after their participation in Listen4Good ended. The COVID-19 pandemic affected feedback work in different ways, but no organization stopped collecting feedback indefinitely, signaling that feedback work is sustainable, even in challenging times.
- Almost all organizations are engaging high- and medium-quality practices in steps 2 and 3 of the feedback loop, data collection and data analysis. Responding to data and closing the loop continue to be the most difficult steps for organizations.
- There are specific practices within each step of a high-quality feedback loop that can help advance equity, diversity or inclusion (such as by uncovering disparities in services or making the survey process more inclusive). Most organizations are engaging in these practices during data collection and data analysis, but half or less are engaging in these practices in other steps of the loop.
- At least half of the organizations from both cohorts reported that feedback work after Listen4Good was primarily driven by one person tasked with collecting and analyzing data and then sharing it with other staff.
-
Funder Engagement: Learning from L4G Co-funders
For this report, ORS Impact interviewed co-funders that nominated nonprofits to participate in Listen4Good, examining the extent to which the feedback program may have led to changes in the funders’ thinking, organizational culture, and practices around feedback.- More than three out of four funders overall reported making changes related to feedback.
- Returning funders, those that nominated nonprofits in more than one Listen4Good cohort, have different experiences than new funders, reporting more concrete changes in feedback practices and being more likely to say that Listen4Good contributed to changes.
- Both new and returning funders mentioned additional benefits of Listen4Good, including their organization’s increased buy-in into the power of feedback; the opportunity to participate in a new/better way of getting feedback; increased funder knowledge and understanding regarding feedback and the capacity needed to do it well: and more activity around closing the loop with clients.
- Most funders overall reported plans to continue their feedback work internally and with their grantees. Many also reported additional feedback work outside of Listen4Good.
- Most funders overall see a relationship between feedback practice and equity, diversity, and inclusion, with the largest share of those funders pointing to a link between feedback and consciously centering clients and valuing their experiences.
-
The Listen4Good Journey So Far
This report by ORS Impact, Fund for Shared Insight’s learning and evaluation partner, looks at the impact of Listen4Good on the funders and nonprofits that participated in the feedback program in its first four years.- L4G has had strong impacts across three cohorts of nonprofits on their capacity to collect feedback, gain insights, take actions, and see organizational effects by the end of the L4G grant. For example, 81 percent of participating nonprofits made at least one change in response to feedback in programming, operations, client-staff interactions, or offering new services.
- L4G has had some impacts on co-funders, despite Shared Insight’s light touch approach to engaging with them. For example, 33 percent of interviewed co-funders reported changes to their foundation practices related to feedback from grantees or communities and mentioned that L4G contributed to that change. (Since Shared Insight began increasing its engagement activities with co-funders beginning with the 2019 cohort, ORS will assess — and include in future reports — whether these increased efforts lead to better outcomes.)
- L4G has made some contributions to meaningfully connecting nonprofits and foundations, but contributions are uneven, and there is opportunity for greater impact.
- There is some progress — and more opportunity — for using feedback to advance equity and inclusion. For example, Shared Insight might further assist more organizations in segmenting data by sub-groups to identify differences in client experience, interpreting findings and developing appropriate responses to those differences.
-
Listen4Good Data Analysis
In early 2017, Shared Insight engaged Harder+Company Community Research to analyze the L4G surveys’ aggregate dataset, consisting of more than 29,000 responses; summarize the data and identify patterns and/or differences in respondent feedback; and assess the effectiveness and utility of the Net Promoter System overall and specifically in the nonprofit context.- A majority of respondents provided very positive feedback to the organizations that served them.
- Youth under age 18 and people receiving education-related services gave significantly lower ratings on survey questions than did other respondents.
- The Listen4Good survey tool is understandable to the people participating nonprofits serve.
- This analysis provides some evidence that the Net Promoter System works in the nonprofit context. It also reveals the need for further exploration.
-
Listen4Good: 2016 – 2017 Cross-Cohort Findings
Over the past three years, ORS Impact has served as learning and evaluation partner to Shared Insight and L4G. This report analyzes data collected from 104 out of the 112 nonprofit partners from the 2016 and 2017 cohorts, providing a high-level view of how L4G has worked across the two grantmaking periods, and identifying key lessons and implications for new L4G partners under the current program model, for planned changes to the program, and for evaluation efforts moving forward. L4G Impact on Nonprofits- Organizations from both cohorts report improvements in their ability to implement feedback-related tasks, but there may be a ceiling effect to this growth.
- Organizations with higher ability to implement feedback loops are more likely to see impact on program effectiveness.
- L4G provides organizational insights through both new information and data to confirm prior beliefs.
- Organizations that gain more insights are more likely to see impact on program effectiveness.
- Organizations are translating these insights into changes to better serve clients.
- L4G advances organizations’ equity, diversity, and inclusion work by providing clients a seat at the table at various steps in the process and allowing them to shape the services that benefit their communities.
- L4G improves organizations’ ability to serve clients while changing organizational culture and values around listening to the people they seek to help.
- All organizations plan to continue their feedback practice after their L4G grant.
- L4G increases leadership and staff commitment to feedback practice.
- Organizations find the L4G model effective in supporting their feedback practice.
- The amount of TA provided has remained adequate despite changes in the model, and organizations highly value the SurveyMonkey platform.
Listen4Good 201602017 Cohort Findings by ORS Impact is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
