My research on improving teaching and learning strategies of software testing in computer science education.
My Orcid ID is: 0000-0002-0680-4443.
I sometimes toot about my research on my Mastodon account @niels76@mastodon.online about my research, but more often about other things that interest me, or that fill me with wonder.
Some of my projects can be found on GitHub.com/nielsdoorn. Feel free to contribute.
Follow me on Strava! I like to go for a run now and then. It helps me to think and also to clear my mind. I often get my best ideas while working out, but I also tend to forget most of them immediately.
☕ 🧘 🌳 🐱 🐔 🥞 🚲 📷
Came here for the Test Informed Learning with Examples repository? Look no further! It can be found at tile-repository.github.io/.
by Niels Doorn
This summer holiday is turning out to be a very interesting working holiday so far. Earlier I reported about the great time I had at the Prague Summer School on the future of schools, the last few days I have been doing something completely different, namely working as a Red Cross volunteer on one of the larger festivals of The Netherlands, the “Zwarte Cross” (translates to “Black Cross” where cross refers to the motorcross which is an important part of the festival). So I was working at the Black Cross for the Red Cross haha! After three intense days and versy short nights, it is now time to travel to Florence, Italy, for the Conference on Software Engineering and Education (CSEE&T), to present our accepted article called “Learning Critical Testing Literacy Through Puzzles: An Experience Report” (pre-print can be found here). Due to my other commitment, I will only be able to attend the second day of the conference. In this post I describe my experiences.
There is still a lot of debate on how AI is shaping education. The keynote talked about AI augmented engineering, human-ai collaboration, the ethics, and the impact. The role as a teacher changes because of these developments. The point of the presenter is that we should move from preparing students in using AI+ to them _leading with AI, however, a major problem to achieve this is the gap between industry and education. Most forms of IT education struggle to have a authentic environment. The presenter proposes more collaboration with industry and the use of AI as a partner during the development process. What I missed in the talk was the learning. What happens to how students learn when partnering with AI? I understand that we should not ignore the use of AI, but I like my students to have the knowledge and skills required to develop without AI, just as I like pilots knowing how to fly a plane despite using auto pilot most of the time. The presenter does agree with that, however, what does means for the CS curriculum and for learning it self, stays, in my opinion, vague. The presenter argues that AI should be included in all subjects of the curriculum, and that it should be the foundation of Software Engineering education. This way, universities can lead the way into using AI.
Four posters are presented at the conference, one of them by my colleague and co-author Bart Th. Knaack. Allthough I am slightly biased, his poster looked really good. His poster is about testing, the others addressed topics related to supporting students with code comprehension, drafting systematic literature reviews, and tutoring in programming education. I always like looking at and reading posters, creating good posters is an art in my opinion. Also, getting your poster at a conference un-wrinkled, and hanging your poster stable and straight is an underestimated skill no LLM can help you with.
One of the talks was about debugging, linting, and testing Scratch projects. Existing tools have there own problems regarding learning. The researchers developed a extension to Scratch with debugging tutorials. In the tutorial, bugs in an existing Scratch game are shown and step by step they can be fixed. Tests are run after each fix, and feedbak is given. Also, help can be requested, after which hints are given. The tutorial is very interactive and completely in style with Scratch. I like that it incorporates many educational best-practices and that it creates a useful addition to the Scratch eco-system. The main objective is to change the mental model of students to find bugs. The tool can be found here and the source code can be found on GitHub.
One of the most intrigueing things in CS education is the use of Parson’s problems. The notion that these problems have a similar effect on learning than writing code is facinating. Tiered tasks is also very interesting in this regard. At this session, a tool, CodeOwl, was presented, which can generate tiered tasks for Parsons problems. It can generate Parsons problems for three different tiers in different programming languages. It takes into account the context, problem statements and the programming concepts that should be trained based on existing code that serves as an input.
This was followed by a talk about enabling self-paced, problem based learning by an intelligent tutoring system. Benefits would be to support students with adaptive feedback on problems that have multiple valid solution paths. This is relevant for courses that use LLM’s, but reuires a combination of result based feedback strategies and code-based feedback strategies. The developed system works for MatLab exercises, and is useful in Engineering courses.
Final talk of this session was about a 2D Dungeon Crawling RPG to learn C programming. It is a visual aid to learn C using Gamification. It has a focus on teaching the C language internal behaviour. The RPG illustrates the execution process of the C language. The goal of the project is to support any C program, which seems very hard! Game mechanics include reducing hit points when a player chooses an incorrect path in the game. The game looks very in style with the old RPG’s from when I was young, so I liked that aspects. I also like the idea behind it.
First presented study was about the integration of VR in authentic computation education. There has been a lot of attention to VR in education, but adoption into the curriculum remains low. This study has a focus of the teachers point of view on the pros and cons of integrating VR. I think it would be beneficial if students are able to use VR in the solutions they develop. It can help to experience the use of an app from the perspective of the intendet audience.
Second talk was an exploratory study on pacing students who follow asynchronous online programmes. These students struggle with procrastination and binge-learning. Self-Determination Theory tells us that self-regulation needs to be supported by design in courses. The hypotheses is that design of staged deadlines, the use of badges, and other motivational support can help with that. Results show that most students meet the deadlines. Quizes to earn badges where very popular, and reflection response was high. Students indicate shifts to more systematic approaches to their work. Badges received mixed appreciation, the ability to share badges was missed by students. A very interesting study, not only for remote learning, but als for part-time students who have a higher load of self-study compared to full-time students.
The third talk was about a Capture the Flag (CTF) event to improve the knowledge on cyber-security of students and job applicants. They used ExoMy Mars Rovers, cool looking 3D printed colourful little robots, developed by ESA. 120 People took part in the qualification event, 40 participants made it to the main event. They created a miniature space station where the robots needed to drive around. The challenges used had to do with encryption and cyber-security concepts such as ransomware.
The fourth talk was about humour in software testing education. So more related to my own research, and humour is almost never bad for student engagement. They used memes, lolcommits (taking a selfie when committing code), the java-faker library (used to generate test data with appropriate cultural references such as Harry Potter instead of US presidents), and rickrolling-ci (if a build fails, you will be Rick rolled). I find this very creative. Students enjoyed this approach, and many even found it less frustrating to do software testing. It makes it also less intimidating. One student mentioned that the use of emotions in relation to code makes it more personal and less like code from a LLM. It also made students try out more edge case testing. It also helped in social interaction with other students, it gave them a sense of belonging. These are very positive results, also in regard to creativity, it changed the perception of software testing as being more creative. It also showed to have a stronger positive effect on female students. This talk sparked some discussion on the use of humour in education. I find this remarkable. Maybe a false comparison, but during talks on the use of GenAI in education nobody raises any ethical questions (anymore) but if humour is involved, some people have great ethical concerns.
Final talk was about gender representation in Software Engineering Education on YouTube. With men being overpresentated in SE, this has an implicit effect on how people perceive the field (as in terms of who belongs in the field). Taking several aspects into account (read the paper for more information), the results show that most videos are on technical and commercial topics. Very few on social domains. For every woman named, 4.6 men where named in German videos, in English videos the ratio was 1.8 men per woman. A similar ratio was observed in the agency of people named. Bias indicator show almost exclusive male-centric terminology. It shows a typical Dude culture.
The first presentation was about using a project together with the space industry as a capstone project. The results showed that the students experience was much better than from a simulated project. One of the interesting things was the way of assessing using real world quality reports.
In this session was our presentation. After some minor technical problems, the presentation went well. The people in the room where interested and asked very relevant and good questions (good preparation for my defense).
The third presentation was about human errors in software engineering. These errors are a major source of defects in software. The study shows how this can be addressed in a course using real world examples.
The fourth presentation is a combined presentation of two papers on industry involvement and a transnational computing programme. During a course a real world simulation was done.
The CSEE&T ended with a lovely banquet in a beatiful restaurant near the city center, however, CSEE&T was followed by the QRS conference the next day at the same venue. As CSEE&T participants, we were invited to join the keynote speaches of QRS on Wednesday. Bart and I went to the first one, which addressed trust issues on the hardware continuum, which is an interesting topic for many reasons, one of which is the complexity created by all the different hardware on all devices, many of which do not receive proper security updates.
After this keynote and a cup of lovely Italian coffee, we went to the train station to travel to Bologna. Buying train tickets was an very interesting experience, out of the blue, without us asking, an employee of one of the Italian railway companies helped us by changing our tickets to a different train, which was delayed, and because of this, could be taken by us. This saved us some time, but most of all, I have never experienced such initiative at railway stations. The high speed train took us to Bologna in about 30 minutes. The train is mainly inside exceptional long tunnels, so landscape wise the trip is not very exciting. In Bologna we toured the city and went into several churches and cathedrals. There are many different ones in the city center, each with their own atmosphere. I particulaire like the smaller churches.
Bologna is a lovely city with a calm vibe and beautiful buildings. We took the oppertunity to eat some pasta and we had a great time. The train trip back to Florence was equally boring in terms of outside views but went without any problems.
Despite being in Florence for a couple of days already, I had not seen much of the city yet. For thursday I booked two tours to get to known the city and it’s history a bit more. The first tour showed all the highlights, explained the history of the most important people and families, and gave some insights in Italian livestyle. It was a very nice and inspiring tour, with an excellent guide. It is amazing to hear and see the history unfold by walking through the city. The Medici family, the famous artists like Michelangelo and Leonardo da Vinci, the political system in those days, and how this combination led to such a city with all it’s splendor! The second tour started in the evening and took a different approach, it told the stories of the mysteries of Florence. There are many! Some funny, some really sad, but all of them full of unexpected twists. The guide showed us details I would otherwise have missed. The tour visited some of the same places as the first one, but told stories from a different perspective. It was a lot of fun!
Due to announced railway strikes, it looked like I needed an alternative to my adventures plan to return by train. I therefore already purchased a ticket for the Flix-bus to get me to Zurich. The bus left at 00:05 which felt like Thursday, but technically is Friday, and messed with my mind for days. It was better then I expected to be honest. I booked a panorama seat on advice of Bart, and I was very happy with that decision. The bus stopped in Bologna, and Milan. I did not sleep very well, but everything considered, not too bad. After Milan, the bus was a lot less crowded, and during the last two hours I looked out of the panorama window and enjoyed the view of the Alps. In Zurich, I could change my ticket to an earlier ICE, reducing my waiting time with four hours! I also had a nice breakfast and was able to buy a Starbucks cup of Zurich (making this summer a serious challenge to store all extra cups of Prague, Bologna, Florence and now Zurich).
tags: conference - presentation - Italy - Florence