Component 1: Community Team Structure, Organization and Systems Building

Effective collaboration and coordination are cornerstones of programs that seek to improve the health and development of children and families. This component of the evaluation will describe and assess the community coalitions within the two community initiatives (Miami-Children’s Initiative and New Town Success Zone) serving as local implementation sites for the ECCS CoIIN activities. The state-level ECCS leadership and advisory group will also be assessed in order to evaluate the broader system-level collaboration development, which is essential for supporting future efforts in other communities. This evaluation will assess each network’s implementation, processes, and network composition and structure. To describe and measure collaboration among the three networks (two place-based community teams and state-level ECCS leadership/advisory group) the PARTNER survey will be adapted to align with the aims of this initiative. PARTNER is a quantitative social network analysis and collaboration tool that measures how well each network is working in terms of: identifying partners and leveraging resources; demonstrating how the collaboratives are progressing over time; and assessing collaborative and participant outcomes. Each year, active participants from each of the three networks will complete the brief online PARTNER survey. Data will be examined using the social network analysis tool provided by PARTNER and will also be uploaded to SAS 9.4 or SPSS V.22 for more in-depth analysis (see Table 7), including generating graphic representations of the collaborative networks and network process scores (e.g. the number and quality of relationships; roles adopted by each network member; and agreement on collaborative goals and outcomes). Survey reports will be disseminated annually.

Component 2: Community Team CQI Capacity and Implementation

Qualitative methods will be used to add contextual information to data collected through the PARTNER survey, and to better understand in-depth individual stakeholder perceptions of how the ECCS strategies are being implemented. Focus groups provide rich, in-depth information on how participants perceive the community collaboration activities, and how those perceptions are discussed in social groups. This will accomplish several goals, including adding depth and context to complement the results of the quantitative survey; providing additional information about services that are being provided and received; providing information on how groups discuss their collaborations in the community with each other; and providing important feedback for each community, as well as the overall ECCS effort. Participants from the three Partner networks (two community teams, one state team) will also participate in focus group discussions held twice yearly in each community or at statewide meetings. The results of these discussions will complement the findings from the PARTNER Survey by enriching, explaining, and adding context to interpretation of results. Groups will consist of up to 20 participants (or separated into smaller groups of 8-10 participants to facilitate discussion as needed), a facilitator, and a co-facilitator. Results from the PARTNER survey may be expanded upon or clarified in the focus groups, and results from the focus groups will inform the next PARTNER survey through an iterative process. Specifically, focus groups will gather participant perspectives on the processes (successes, challenges, next steps) and outcomes of the ECCS initiatives, including: identification and use of core process and outcome indicators; successful community-based strategies; strategy development and testing (CQI process, PDSA cycles); participation in peer-to-peer mentoring; data sharing, integration strategy formulated for home visiting and early learning sectors; and sustainability. 

Component 3: Data Analysis Technical Assistance (based on data availability)

As communities and systems engage in data sharing and integration as part of the ECCS systems building and their CQI processes, the evaluation team will provide technical assistance as needed in analyzing: local team adherence to PDSA principles (use of iterative cycles; prediction-based test of change; small-scale testing; use of data over time; documentation), data collected through monthly PDSA cycles (including duration, sample size/scale, and changes tested), and outcomes. These data will be reported back to teams to assist the communities and the state-level team in tracking and reflecting upon data (e.g. using run charts) as they test strategies to improve developmental and maternal depression screening and referral rates. The evaluation team will also assist the initiative with larger data-based investigations, such as: assessing data completeness and quality; data linkage; descriptive and inferential statistics; subgroup analyses; and spatial analysis to examine spatial patterns among sub-populations in participating place-based communities. Research questions will be developed in partnership with communities, the state team, and stakeholders. Data summaries and analysis reports will be provided to local communities and state partners.