Evaluations and Research
 
  
Evaluations are an important tool in building and maintaining organizational effectiveness. DREAM believes in using academically rigorous evaluations to determine areas of strength and challenge within the organization. We couple our own internal findings with external research to determine our overall value to children, mentors, families, and communities.
Philosophy
DREAM has a Theory of Change – a model that explains the mechanisms and results that construct the organization’s impact. Here’s the model:
The basic idea: Every experience a kid has with DREAM hits on at least one (if not all) of our direct and indirect outcomes - expanding social horizons by meeting cool young people who go to college, engaging in constructive risk taking by attending overnight winter camping at Camp DREAM, increasing world perspective by traveling with fellow DREAMers across the U.S. These experiences increase a child's expectations and aspirations for what is possible in their life - it pushes them to try new things and to DREAM big.
The experiences, the adventures in DREAM are what fuel DREAMers to challenge themselves and others. Each kid's path through DREAM is different but the key is: we're using a mix of some incredible personal and interpersonal outcomes to help kids DREAM big about what is possible for their lives.
Our obligation to provide the highest quality mentoring relationships starts with the Board of Directors and runs through the staff and every volunteer. To learn more about our Board Policies and Ends Statements please check them out here.
Why do we do evaluations? Because DREAM is a young, evolving, learning organization. We believe that regular and thorough reflection is one of the best tools for informing organizational practice. We utilize a mixture of quantitative and qualitative tools to gather data to gather a diversity of information and perspectives. We believe that evaluation is cyclical - reflection, analysis, action, reflection - that it should be ongoing and never static. Finally, DREAM is a community that is driven by its mentors, children, parents, alumni, partners, and staff members, and we use our evaluations to strengthen ownership of the organization at all levels. The reflections provided by members of our community are what drive this organization forward in DREAMing BIG.
Want to hear more about what chlidren think about constructive risk-taking at Camp DREAM? ...
DREAM Reports
DREAM embarks on a comprehensive evaluation of all children, parents, and mentors every 3-5 years. These evaluations are structured by our Theory of Change and seek to provide an overall picture of the organization’s impact – assessing areas of strength, challenge, and recent development.
In addition we have regular mini-reports on our constituents and specific programs.
Outside Research
Here are some links to other research related to DREAM's work
as well as a listing of our evaluation partners...
Mentoring Research
Camp and Adventures Evaluation
Evaluation Partners
OUR CONSTITUENTS
Want to learn more about the different constituent groups that we survey? Here’s some information about who they are and what kind of data we collect from them.
- Children : exploring the impact that DREAM has on its participants' well being.
- Mentors : understanding the impact DREAM has on its student mentors as well as exposing the influence that the mentors have on their DREAM partners.
- Parents /Families : showing the influence that DREAM has on the families and neighborhoods involved in the program, as well as the affect of DREAM on the child participants as perceived by parents or caregivers.
- Alumni : understanding any long-term influences DREAM has on its mentors.
- Staff : providing structure for staff development and program assessment.
Children
Children are the heart of our organization. DREAM takes seriously the evaluation of our impact on child and youth participants. We believe their opinions of the program are not only essential in maintaining healthy participation in DREAM, but that they will also inform the future growth of our organization – telling staff and mentors what’s most important, challenging, exciting, and, of course, fun.
In the past DREAM has attempted to track children over time by developing a “New DREAMer” background information survey for all children and youth beginning with the program. This would be administered upon entry. This survey was followed up a “Post Program” survey administered in the Spring.
More recently, we’ve adapted the content of these two surveys into a single survey to be administered on a regular schedule in the spring. This allows us to track program participants over longer periods of time, focusing on long term outcomes and looking at the “feedback loop” mechanism within our Theory of Change.
Our evaluation instruments have been developed for The DREAM Program to focus on the assessment of our Theory of Change outcomes. These assessments draw from instruments suggested by the Vermont Mentoring Partnership and have been designed with input from Cynthia Char, professional evaluator. The surveys are designed to be accessible by children of different ages, and are user friendly in format. Here are a few of our Child Surveys:
Data is gathered at the individual DREAM sites across the state. Once collected, all data is stored at the central DREAM office. Findings are reported either as a group aggregate, or with individual pseudonyms. Results are written up in a formal Impact Studies, available for download above.
Data from the 2009 children surveys have also been broken into Local Program Specific Reports. Read all about the outcomes of a program you're most interested in below.
Mentors
Often times the positive impact DREAM has on its child participants overshadows the good that the organization is doing for its mentors. As truly student run programs, there are a wealth of opportunities available to DREAM mentors to learn and hone transferable skills as well as to reap personal satisfaction from their efforts.
DREAM uses internet based surveys (hosted by SurveyMonkey.com), written mini-surveys, and focus groups to gather data from our mentors.
DREAM Mentors are responsible for governing their local programs. This means mentors have primary responsibility for the major components of their organization, including recruiting, fundraising, planning Fridays and Adventures. Mentors learn to effectively communicate with parents, community members, college administrators, and children while also gaining the experience of peer-to-peer leadership.
DREAM believes, as with our children, that mentors gain an increased sense of possibility from working with the organization. DREAM enhances the college experience and provides mentors with strong transferable leadership skills, a world-wide social network of peers, a broadened world view, and a deepened understanding of children. There is still limited research as to the long term effects of mentoring for mentors and DREAM aspires to be at the forefront of this research area.
Our surveys are designed to be personal reflection tools for individual mentors to gain perspective on their own growth through DREAM, as well as to provide the organization with critical feedback on areas of primary development amongst the mentor population and overall mentor ownership of their organization.
Below are a handful of quotes from mentors over the years articulating some of the most important things they’ve learned from DREAM:
“From DREAM I've learned how just a few hours of our time a week can mean so much to the kids living in these communities. The kids are always SO eager and excited to come to DREAM, it makes everything worth it. Also (my second #1), more personally, I've learned a greater sense of leadership by organizing events that bring our DREAM program together on a larger scale.”
“DREAM has given me confidence in the power of local, grassroots movements to provide young people with the moral guidance they so desperately need.”
“I think the main thing I've learned about are some of my own boundaries and limitations as far as working with children, and that given plenty of time and planning, those boundaries are much broader than I had imagined!”
“That leading is tough, but when it is done right you can create amazing change!”
“That adventure is one of the most powerful ways of building increased self-reliance in kids.”
“My actions mean more to other people than I ever realized.”
“That I really do wanna have kids someday.”
And some additional anecdotes shared from our 2007 Mentor Survey:
“I was just in San Juan, Puerto Rico for vacation and saw someone wearing a DREAM shirt. You can't believe how happy I was to see a fellow DREAMer! Of course we stopped and chatted for a bit and it made me realize what kind of global influence DREAM has already had and what kind of reach it can have in the future.”
“I think one of the best parts of DREAM is the community focus. I love being part of a program where you not only go to visit your mentee but all of their neighbors and friends, it creates a home away from home when I'm at school.”
“DREAM has changed Dartmouth for me and I couldn't imagine school without [my mentee]!”
“Joining DREAM was one of the pivotal moments in my college career, if not my entire life. The program has meant more to me, and created more positive change in my own life than I can even believe… It has been incredible mentoring [my mentee] and watching her grow up. I feel so lucky to know her, I am in awe of her. I love her and I love being her mentor. I really feel like she feels happy and safe in our relationship, and I am glad that she has a person to come to with any issues that she might have. Adolescents are so often overlooked when it comes to people who need mentors because they are thought of as too old, but it seems to me that it is one of the most important times to have someone to come to. We have created that safe space here in DREAM, I see it in my relationship with [my mentee], and I see it with the other teen mentors in Elm Street.”
“With my kid, the smallest things have always gone a long way towards my appreciation of the power of this program. Like the time he skipped rocks and even harmlessly flirted with a girl his age down by the creek near the Square. Or the time he spontaneously began asking me if I remembered "that time we had a lot of fun," unable to elaborate further; for a boy with dramatic across-the-board developmental delays, moments like those are incredibly moving. DREAM has been integral to my college experience. To take just one example, the young woman (now graduated) who insisted that I join when I was a freshman has gone on to be both a mentor and a crucial example in all areas of my life. The commitment that DREAM represents has helped propel me through challenging times. Most of all, I continue to believe what I told this year's potential freshman recruits: for the children, of course, but also for the mentors, DREAM is nothing less than a better way of being in the world.”
Our findings are reported either as a group aggregate, or with individual pseudonyms. Regular results are written up in a formal impact reports available above for download, exposing both DREAM’s significance in the mentors’ lives and also target areas in need of work. Overall data trends are shared with college/university partners and other research partners. Data pertinent to individual programs is forwarded to the program’s chairs.
Parents/Families
DREAM provides opportunities for parents to be involved in their children’s lives and to share ideas on how to strengthen their neighborhoods. Parents are invested members of the program, making important contributions, and appreciating the work the program does in their communities. One DREAM father spoke to the value he places on the program to a group of mentors and fellow parents:
"I don't know if it's like this for every parent here, but I have a really good relationship with my kids' mentors. I mean, if they came to my house and asked to borrow my TV and I was watching the Super Bowl, I would give them my TV."
DREAM surveys the parents of its child participants to corroborate findings from child surveys and also to determine what effects the program is having on parents themselves and their larger neighborhoods. Specifically, DREAM works to ensure that the parents have positive, healthy interactions with their neighbors, and are actively engaged in the running of DREAM.
As part of our organizational outcome evaluations DREAM administers a written parent survey to all DREAM parents every 3-5 years. These surveys cover material similar to the child surveys with additional questions regarding parent involvement and overarching neighborhood impacts that parents may be observing. In addition to this DREAM may add an optional page that provides direct feedback to the Vermont Children’s Forum, a youth advocacy organization with which we partner.
In the past, DREAM has attempted to survey a “control” group of parents from demographically similar neighborhoods by adding an additional set of questions regarding overall community health. We have yet to achieve a control group that is large enough to be statistically relevant, so we currently rely on external data regarding families living in poverty to provide comparison data.
Here are examples of the “control” and DREAM parent surveys:
Findings are reported either as a group aggregate, or with individual pseudonyms. Results are written up in formal impact studies available for download above. The results report on DREAM’s significance in parents’ lives, the worth of DREAM to their children, and target areas in need of work.
When we first began the Elm Street DREAM Program in Winooski, VT parents provided these initial comments on the program’s first year:
"The program really seems to help foster the children's' self-esteem and teaches them how to build positive peer relationships. The mentors are positive role models for all kids."
"[The kids] can't wait until Friday when they can get out and do different things and enjoy the one on one time also."
"All the children have nothing but positive things to say about the program."
"This is the best program for my kids because they’re always bored—it’s lifted their self-esteem, gave them something to look forward to. I really hope they continue this program because with a large family sometimes I don’t get the time with them and DREAM is wonderful! I really hope they continue."
Alumni
DREAM’s Alumni Organization is made up of mentors and mentees who have graduated out of their Local Programs and want to maintain a meaningful connection to the organization. Our Alumni are all over the world, working in a variety of different professions, with one thing in common – DREAM! Learn more about our Alumni Organization governance, fundraising, and events here.
We survey our Alumni in order to learn more about the long-term effects of DREAM. We believe that DREAM provides mentors and children with long lasting experiences and transferable skills that provide a foundation for professional and leadership development.
Our Alumni surveys are administered online using Survey Monkey. A sample survey can be viewed here: (link to pdf of mentor survey) Our first full report of Alumni Data will be reported in 2009! In the meantime here are some favorite memories from DREAM Alumni:
“So many memories! How about a simple one... watching my DREAM partner and other kids spread manure at the beautiful Dartmouth Organic Farm. The mentors were worried because spreading manure was all that was left to be done that season. But the kids dug right in and enjoyed every moment of it!” Erica Mintzer, Dartmouth ‘01
“Whalewatching trip-- the bus went the wrong way, we were hopelessly late, and when I finally had to announce to the whole group that we had missed the boat and weren't going to see whales, James said, "But we still get to hang out with our partners, right?" Devon Green, Dartmouth ‘02
“That wherever you meet Dartmouth students, no matter what country, state, circumstance, someone is always wearing a DREAM tee shirt!” Allegra Love, Dartmouth ‘03
“Mud sliding at one of the first UVM Chili Cook offs !” Stephanie Gergely, UVM ‘05
“The Spring Things, Chili Cookoffs, Talent Shows, and other big events were, of course, memorable... But it was the tiny, indescribable micro-moments that made what DREAM is all about feel real. ” Kristin Brisee, UVM ’06
“Driving a 10 passenger van singing cheesy songs at the top of my lungs with Josie from Saint Mikes to Franklin Square every Friday at 2:30. AND running, playing, swimming, eating, etc. in the woods with fun folks and kids in 2005 for the summer. HOLLER CAMP DREAM!” Kristin Beattie, St. Michael’s College ‘06
Staff
The DREAM Central Office Staff also undergo quarterly staff evaluation. Staff evaluation focuses on the core competencies of each position and individualized staff development, as well as assessing strengths and weaknesses of the organization. To learn more about our survey tools and staff Individualized DREAM Plans (“IDPs”) please contact Elizabeth Haag at
|
