Monitoring and evaluation are important processes to identify what works. The evidence of what works is for the tens of thousands of young people whose schools and colleges are working with HeppSY. Based on the evidence, HeppSY practitioners renew existing projects and activities and develop new ones annually to meet young people’s learning needs.
This page contains the essential facts about HeppSY evaluation.
What is monitoring? Monitoring in this project is like a school’s student attendance record system. HeppSY collects the students’ information from 47 schools and colleges in South Yorkshire and North Derbyshire.
What is evaluation? HeppSY evaluation assesses the students’ learning outcomes through robust evaluation tools. For different activities and projects, the students are asked to respond to a quiz, poll or survey, complete a creative task, write a reflection log, or answer questions in a one-to-one interview or a group discussion.
The HeppSY Programme is evaluated annually through monitoring and evaluation reports about the students’ learning outcomes. The Learner Survey is the core of the HeppSY programme evaluation. We need large numbers of students from all 47 schools and colleges working with HeppSY to respond to the survey every year so that the survey respondents’ learning outcomes can represent the learning outcomes of all the young people HeppSY intends to work with.
HeppSY endeavours to produce bespoke evaluation reports for all partner schools and colleges based on the Learner Survey. The robustness of the evaluation reports for schools and colleges largely relies on the number and make-up of the students in each school or college who completed the Learner Survey annually.
HeppSY also commissions external evaluators to evaluate the programmes’ impact.
Alongside running projects to help students improve their academic performance and make informed choices about their future, we also systematically evaluate the projects by collecting student feedback at the beginning and end of their learning journey.
Aligning the projects under three strands: Targeted Outreach, Strategic Outreach, and Attainment Raising, HeppSY practitioners aim to implement evaluations for the following projects in the academic year 2023/24:
| Strands | Projects | Student Year Groups | Evaluation Methods |
|---|---|---|---|
| Attainment raising | Grade Shifter | Year 11 | Surveys (at the beginning and the end of the project); Interviews |
| GOALS Breakfast Club | Year 7 – 9 | Surveys (at the beginning and the end of the project) | |
| The Elephant Group – Meta Method | Year 10 – 11 | Surveys (at the beginning and the end of the project); Reflection log | |
| Strategic Outreach | HE Can | Year 9 | Learner Survey at the beginning, a short survey at the end of the project; Interviews |
| SHE Can | Year 9 | Learner Survey at the beginning, a short survey at the end of the project; Interviews | |
| HEadstart | Year 12 or Level 3 Year 1 | Learner Survey at the beginning, a short survey at the end of the project; Interviews | |
| Targeted outreach | Central offer | Year 11 – 13 and Level 3 Year 1 and 2 | Learner Survey in Winter term; Reflective quiz at the end of each activity |
| Campus tour | Year 11 – 13 and Level 3 Year 1 and 2 | A survey after the campus tour | |
| Sheffield College self-led project | Level 3 Year 1 | Learner Survey at the beginning, a short survey at the end of the project; | |
| Barnsley College self-led project | Level 3 Year 1 | Learner Survey at the beginning, a short survey at the end of the project; |
Over 7,880 learners who worked with HeppSY since the academic year 2017/18 have progressed to higher education, according to the records released by the Higher Education Statistic Agency (HESA) in 2022. The statistics do not include the students who entered higher education qualification courses in further education colleges or private higher education providers.
At the start of the academic year, HeppSY collects student data for all new students in HeppSY schools and colleges. The first step is publishing a HeppSY privacy notice in each school and college. Parents and carers are notified of this by our partner schools and colleges through newsletters and have the option to opt out of data collection. After the notice period, schools and colleges share student data with us for monitoring and evaluation.
Data protection is a lawful duty that HeppSY takes very seriously.
HeppSY collects the students’ data on the grounds of public task. It means that the data collection is for the public’s interests, such as improving social equality, ensuring effective use of funding and gathering evidence of what works. If HeppSY does not collect the students’ personal data, HeppSY has no means to allocate limited resources to support the students in need. In that case, these students would lose the opportunity to narrow or close the knowledge, information and attainment gaps between them and other more fortunate students.
HeppSY always puts the security and integrity of personal data first. The drives used to store the students’ personal data are exclusive and protected with multiple security checks. The files with students’ personal data are encrypted even in the secured drives. All HeppSY staff must pass a strict Disclosure and Barring Service (DBS) check. All the Evaluation and Data team members hold enhanced DBS certificates in order to process the student data.
HeppSY also puts the students’ rights to their personal data at the heart of the data protection policy. The students participating in evaluations are well informed of the aims and processes, and their data will not be processed without their consent.
Students and parents can download our privacy notice explaining the type of data being collected and how the data will be used below:
For schools and colleges engaging in specific projects (particularly those in the Attainment Raising strand), we may ask for further data related to attainment and attendance. This is also outlined in the privacy notice.
For more information, please contact the Evaluation and Data Team at heppsydata@shu.ac.uk
A cohort of Netherwood students’ progress from 21/22 to 22/23
Netherwood Academy is a secondary school located in a rural area between Barnsley and Rotherham with 1,200 students enrolled. Over one-third of students come from low-income families, making them eligible for free school meals. Nine out of ten students live in areas with fewer than average young people progressing to HE after compulsory education.
In the academic year 2021/22, Netherwood Academy made good use of the HeppSY workshops provided for Year 10 students, with 944 times of attendance.
The Year 10 students in Netherwood Academy participated in two Learner Surveys and answered the same questions in the Winter term of academic year 21/22 and then 22/23. The comparison of their learning outcomes reveals significant progress in one academic year.
Figure 1: Improved expectation to progress to sixth form or college
From Year 10 to Year 11, 8% more students expected to progress to sixth forms, and 4% more students expected to progress to colleges after the current study.
Figure 2: Improved perceived fitness for higher education and sense of belonging
From Year 10 to Year 11, 13% more students strongly agreed “higher education is for people like me”; 22% more students strongly agreed “I would fit in well with others”; 21% more students strongly agreed “I have the academic ability to succeed”; 20% more students strongly agreed “I could cope with the level of study required”.
These positive changes mean more students are confident in their future studies and have started considering higher education. Of course, students’ progress is driven by multiple factors. HeppSY activities are part of the contributing factors.
We want to ensure that the HeppSY programme offers support to students as intended, so we need strong and thoughtful evaluation practices. Every year, we report on how our programmes and projects are working, either in general or for specific partners or groups of students.
From November 2017 to 2023, HeppSY conducted an annual Learner Survey to evaluate the students’ understanding of higher education, their expectations, and important factors related to their academic performance and choices. Annual partnership reports provide insights from the analysis of survey data. Please click the report covers below to read the reports.
As an important part of our impact evaluation, several matched cohort analyses have been submitted to the OfS evidence bank by HeppSY. The cohort analyses involve tracking groups of students longitudinally to determine associations and causal relationships between participation in outreach activity and outcomes related to HE progression, attitudes and knowledge.
The focus of Uni Connect is to provide targeted higher education (HE) outreach to young people in Years 7 to 13 living in particular geographic areas. This report brings together multiple sources of student data to examine the relationship between engagement in HeppSY activity and progression to HE.
Download
This report tests whether an association between HeppSY programme engagement, and expectations of applying to HE and HE knowledge would be observed over a longer timeframe – with responses at Wave 2 and Wave 4 of the survey serving as pre- and post-intervention measures.
Download
Building on previous cohort-level analyses which focused on students’ expectations of progressing to HE (Fletcher and Brown, 2020a), this report considered another important outcome: knowledge of higher education. Drawing on data from Wave 1 and Wave 2 of the CFE annual Learner Survey, and HeppSY and partner activity data held in the Higher Education Access Tracker (HEAT), the aim of this project was to investigate the association between the number of contact hours that students participated in between waves and their self-reported HE knowledge at Wave 2, focussing on a Year 10 (W1) and a Year 12(W1) cohort of students.
Download
This is the first matched cohort analysis report to evaluate the impact of contact hours that the students participated in HeppSY activities on the students’ expectations of applying to higher education. The analysis suggests that the more a Year 10 student interacts with the HeppSY programme, the greater the odds are that they will maintain or increase their expectations of progression to HE study at age 18 or 19.
Download
#DitchtheDoubt 2023 was developed for UCP students who face additional barriers beyond those of their peers and ran across 9 centres between February and May 2023. Bespoke programmes were offered to identified centres of need. These included SHE Can (4 centres), aimed at high achieving working class girls who need to ‘find their voice’ and increase their self efficacy; HE Can for White working-class boys (2 centres); and Amplify aimed at Black students (3 centres). This report analyses the efficacy of the programme and makes recommendations about future projects.
Download
ElevatED was developed as a shadow programme to the Linacre Institute’s ‘Reach Higher’ programme to support students from our target schools and colleges with the knowledge, guidance and confidence to aim for degree level study at Oxford, Cambridge and other Russell Group universities. This report analysed the efficacy of the programme.
Download
#DitchtheDoubt was developed for UCP students who face additional barriers beyond those of their peers and ran across 7 centres between March 2022 and June 2022. A total of 93 students participated. Without a more intensive and bespoke intervention between March and June 2022, the UCP students who participated in this project were expected not to have the confidence to overcome the barriers, and not to access and make the best use of other resources and opportunities open to them. This report analyses the efficacy of the programme and makes recommendations about future projects.
Download
This report explores how the HeppSY programme performed against its output targets contained in its logic model. Five themes were identified from the logic model: developing a collaborative approach; relationships with partners; programme implementation; evaluation and monitoring; and legacy.
Download
