This blog is part of a series in collaboration with Brock University. Written by a student in the ‘Program Evaluation in Professional Practice’ course, this blog draws on a student’s first-hand experience conducting a program evaluation during a placement with the Bounce Back LeagueOpens in a new window. In this blog, Ashley Romano, a 4th year undergraduate student, offers practical tips to engage children in program evaluations, and make evaluation fun!
Designing activities that cater to the children’s skills and interest is a routine part of any children’s sport program. In the context of program evaluation, choosing data collection methods that work for kids is just as important.
Research has shown children are capable, accurate, and valuable communicators, yet many data collection methods come with considerable limitations related to effectively engaging and understanding children’s experiences (Driessnack & Furukawa, 2011). To overcome these limitations, creative approaches to data collection account for children’s skills and abilities while making evaluation engaging and fun.
Using an evaluation of a children’s sport program—the Bounce Back LeagueOpens in a new window—as an example, this blog explores creative approaches to data collection, including movement-based methods, and offers tips and tricks for implementing movement-based methods with children.
Limitations of traditional data collection methods
Traditional approaches to data collection, such as surveys or assessments, may not be suitable for children for several reasons. First, methods that rely on adults to observe and/or report on children’s progress or experiences (e.g., physical literacy assessments, progress reports) diminish what children have to offer. Children think differently than adults, but not less (Carter & Ford, 2013). Second, many self-report techniques, such as surveys, were developed for adults and adapted to be “child friendly” (Driessnack & Furukawa, 2011), but these adaptations do not always ensure sensitivity to the childhood experience. Often, these methods receive low response rates, have poor reliability, and are influenced by social desirability (i.e., wanting to give the “right” answer; Soland et al., 2019). In contrast, creative approaches to data collection can be developmentally tailored, provide valuable insight directly from children’s experiences, and save time by doubling as a program activity.
Creative approaches to data collection
The term “creative” fits nicely within the scope of methods for collecting data with children. Creative approaches incorporate developmentally appropriate strategies, including but not limited to, individual or group interviews, creative play, or thinking (Christian et al., 2010). Creative approaches resonate with the childhood experience and cater to children’s skills and interests (Carter & Ford, 2013). Movement-based methods are one example of a creative approach to data collection that is particularly relevant for evaluations of sport and physical programs. These methods tend to be low-cost, easy to implement, are typically well-received by children due to the active nature of involvement, and can help improve on-task behaviour and performance when engaging in these methods (Savina et al., 2016).
Movement-based methods can be understood as integrating physical activity into data collection and can take many forms, such as cooperative games or team sport in which children use the structure or rules of the game to provide feedback about a program or their experiences. For example, in Ship, Shore, AnchorOpens in a new window, children run to the wall labelled “yes/agree” or “no/disagree” when questions are called out about the program. This method is explained in more detail below.
Movement-based methods can be used to capture how and why sport fosters development through children’s perspectives. Using movement is an engaging process that helps children ‘buy in’—and it is fun!
Movement-based methods in the Bounce Back League
The Bounce Back League (BBL)Opens in a new window is a nationally-run trauma-sensitive sport programOpens in a new window for children aged eight to 12 years old. BBL uses a proactive approach to build children’s skills, including resiliency, as many who attend BBL are at risk of experiencing trauma or adversity. In the BBL, program evaluations are crucial to understand children’s program experiences, and given its sport-focused nature, movement-based methods are a natural fit. While several movement-based methods have been used to inform BBL evaluation, Ship, Shore, AnchorOpens in a new window is a notable example.
Ship, Shore, Anchor uses space and movement to elicit children’s thoughts and ideas about their program experience and can be run by program staff or an external evaluator. To prepare, staff label one wall “yes/agree” and the opposite wall “no/disagree.” Another space or object (e.g., a hula-hoop or mat) is labelled the anchor—a neutral space for children who are unsure or do not feel comfortable answering. A staff member then calls out “I-statements” that relate to children’s experiences (e.g., “I feel safe at BBL”) or outcomes (e.g., “I learned to not give up when things are difficult”). How children respond to the questions can be documented by a facilitator. At the end of the activity, staff can foster a reflective discussion in a debrief with the children that is aligned with the evaluation purpose. This discussion can include questions such as “What skills did you use today?” or “What did you learn about yourself through this activity?”. If feasible, the debrief can be audio-recorded or notes can be taken and analyzed as part of the evaluation data.
Tips and tricks for implementing movement-based methods
- Use activities that children know and love: It can be difficult to create a movement-based activity from scratch, so start with an activity that program participants already know and love. Making slight modifications for data collection will be easier knowing the activity is familiar and well-received.
- Demonstrate directions: Making modifications to well-known activities, even if minimal, can be confusing for children who are used to playing an activity a certain way. When explaining instructions, it can be beneficial for facilitators to physically or visually demonstrate changes to the structure or rules.
- Keep it simple: Children have short attention spans and benefit from clear instructions (Carter & Ford, 2013). Keeping language short and simple is important for both the instructions and evaluation questions. “ I-statementsOpens in a new window” and signs/labels are easy ways to keep evaluation questions and instructions clear and simple.
- Check in: It is important to continuously check in with children to ensure they understand what is going on, what is being asked of them, and why. A check in could be as simple as seeking a thumbs up for “I get it,” thumbs down for “I don’t get it,” or a thumbs sideways for “I kind of get it, but not completely.”
