Gender Balance in Computing: Making the link between informal and formal learning in computing
Our Gender Balance in Computing research programme is the largest national effort to date to explore ways to encourage more girls and young women to engage with computing. The research programme comprises a number of interventions relating to four different research areas and has been running since 2018. Today we are sharing the evaluation report from the Informal Learning strand of the programme.
There is still an imbalance in the proportion of girls who choose Computer Science at GCSE and A level. However, the proportion of girls attending non-formal computing initiatives (40%) is much higher than those that select Computer Science at GCSE according to a survey conducted in 2019 by the Raspberry Pi Foundation. This points to an opportunity to capitalise on girls’ interest in non-formal computing learning. In fact, prior work undertaken in this field informs us of a potential link between non-formal computing learning and studying computing in a more formal setting.
When female pupils participate in non-formal computing learning initiatives, they may not see how this learning relates to them achieving their long-term goals in computing. This may be the reason that girls’ higher participation in informal initiatives is not translated into them choosing to study computing in formal education (at GCSE or A level). This intervention aimed to address this key barrier by focusing on strengthening the links between non-formal computing learning and studying computing more formally within schools.
Design and implementation of non-formal learning interventions
There were two interventions in this project: secondary school pupils used an adapted Apps for Good course and primary schools ran after-school Code Clubs. In both interventions, existing non-formal learning resources were modified with the aim to help children and young people better understand how the skills they use in the non-formal setting could help them when studying computing more formally. The resource adaptations were made jointly by the Behavioural Insights Team, Apps for Good, and the Raspberry Pi Foundation, and drew on the behavioural science concepts of endowed progress and self-persuasion.
Previous research suggests that girls lack confidence in their ability to succeed in computing. However, self-persuasion activities draw on the theory that indirectly persuading yourself or others is more effective in increasing confidence than direct persuasion activities, such as another person telling you that you are good at a subject. Similarly, the endowed progress concept suggests that girls see computing more favourably when the progress they have already made towards a goal such as studying GCSE computer science is emphasised.
Secondary school intervention
In the secondary school intervention, year 8 pupils from 57 schools took part in twelve, hour-long, teacher-led sessions adapted from one of the courses offered by Apps for Good. Throughout the sessions, learners developed an app to solve a problem that was relevant to them. An initial run of this intervention was launched in December 2019; however, it was disrupted by COVID-19 in March 2020 when some course delivery had to move online. The trial was repeated between April and July 2021. Data was collected in both runs.
Figure 1 – A screenshot from the secondary school intervention showing how skills learnt in the sessions could contribute towards working in the tech industry.
All schools used the adapted materials and pupil attitudes towards computing were measured quantitatively through the collection of survey data directly before and after the intervention. Interviews with staff and pupils were also conducted to understand their experiences with the intervention and how the intervention may or may not have fulfilled the intended outcomes.
Primary school intervention
The primary school intervention was evaluated by using a randomised controlled trial. Participating schools were assigned randomly to either a control group who used the existing Code Club projects or to a treatment group who used the adapted Code Club projects, referred to as Code Club+. Both groups ran clubs for 12 weeks in after-school sessions with pupils aged 7–11 years old, and a total of 54 schools took part across the two intervention runs. The intervention was initially run between December 2019 and March 2020 and was disrupted by COVID-19. The intervention was repeated from October 2021 through to February 2022.
Quantitative data on attitudes towards computing was collected through pupil surveys. Observations of Code Club sessions, interviews with staff, and pupil discussion groups were also conducted to understand the mechanisms of change and the diversity of programme delivery.
"...I want to do this first before we go home, can we not get this done? Can I take a picture at least?" A pupil from observation (Code Club report, p. 34)
Lessons learnt from this research
The evaluation found that the secondary school intervention was not always implemented as intended across schools. Due to demands on teacher time and limitations in the timetable, teachers had to cut and condense the content. However, after this intervention, a small, positive change was observed in girls’ attitudes towards computing. The attitude change for boys was also small, but negative. Some teachers thought that girls enjoyed the teamwork aspect of the intervention and that girls’ attitudes to computing changed when they saw that computing was relevant to real-world problems and when they interacted with computing industry role models. Teachers also felt that the intervention needed to introduce more technical aspects of computing.
“Probably that it was fun to do a project in a team in computing specifically — usually I’m not good at computing but because you’re working in a group, other people could help and I felt better about it.” (Female pupil, Apps for Good report, p. 23)
For the primary school intervention, the evaluation found that the estimated effect of the Code Club+ intervention on attitudes towards computing and stated intention to select GCSE computer science was positive relative to the Code Club group, but was not statistically significant and was small in magnitude. The COVID-19 context likely posed important challenges to both interventions, but additional factors that may have limited the impact of the Code Club+ intervention may include an already high engagement with Code Club and participants' pre-existing interest in computing. Nonetheless, the Code Club resources were said to be high quality and easily accessible not only in terms of hardware needs but also for teachers with limited computing knowledge.
"That was really interesting because you got the resources, but that gave you more of the specifics about how to start off with the first session and then how to move them forward, and what my role would be facilitating and what the children would be encouraged to do." (Primary teacher, Code Club report, p. 37).
Recommendations from independent evaluators
Several recommendations were made to refine the design and delivery of the secondary school intervention. For instance, it was recommended that in future iterations, the links between informal and formal learning should be made even more explicit, not only for pupils but also for teachers. A balance of engaging activities and technical skills was recommended as this was perceived by the teachers as critical in improving girls’ confidence to pursue the subject. The evaluators suggested conducting a randomised controlled trial of a refined Apps for Good intervention to better estimate the impact of the intervention for girls and explore the possible negative effects on boys’ attitudes towards computing.
As with the secondary school intervention, the evaluators recommended modifying the Code Club+ materials to make the activities linking non-formal and formal learning even more salient. Other suggestions included introducing a teacher training session about the Code Club+ materials, introducing functionality to allow teachers to track and view their pupils’ progress, and incorporating more role models into the Code Club+ sessions.
You can find the full evaluation report about the non-formal learning intervention on the Gender Balance in Computing page. If you would like to find out more about the research programme and be notified when we publish new findings, you can sign up to our Gender Balance in Computing newsletter.