AI support for learning to program

Learning to program with Chat Scratch

This week, continuing both the themes of educational AI and evaluation methods,  we will focus on how AI can help children to learn how to program in Scratch. We'll look at this paper: ChatScratch: An AI-Augmented System Toward Autonomous Visual Programming Learning for Children Aged 6-12. 

In doing these tasks, you will build up your skill in critically reading academic research papers, which will be useful for your honours projects.

  1. Familiarise yourself with an overview of the paper.  For example, you could:
    1. Watch the video summary or read its transcript here
    2. Read the abstract and conclusion
    3. Ask ELM or another GenAI tool to summarise the pdf of the paper for you

It’s a busy time of the semester, so you can choose which aspect of the paper you want to focus on in the most depth. (if you have time, feel free to do both aspects). You could do a technical deep dive OR a research critique. We will go over both during the class this week.

Technical Deep Dive

  1. Read the Design Goals in section 3.3 and then pause reading. Think: If you had to implement a system which met these goals, how would you do it from a technical point of view?
  2. Find out how the authors designed and implemented their system by reading section 4.
  3. Are you surprised by any of their design choices? If so, why?

  4. Can you foresee any limitations in the design and implementation?

     

    Research Critique

    1. Look at Figures 2 and 3 so you understand what ChatScratch is like
    2. Read sections 5 and 6 of the paper

    3. Complete the table (adapted from Table 2 in the paper) below to help you think about the reliability of each source of evidence.

      d. Do you agree with the authors’ claims based on the results? Why? Are there any dodgy claims which you don’t think are supported by the evidence?

       

Data

Evaluation Metrics

How informative is this kind of evidence? [Weak, Medium, High]

Note down any criticisms of this kind of data collection

Visual element count

Assets Richness

 

 

Expert ratings

Expert ratings on assets

 

 

Creativity Support Index Questionnaires

Creativity Support Index

 

 

Code Quality Rubric Scores

Dr Scratch Rubric

 

 

Code Retention and Expansion

Retention and expansion measures

 

 

Video recordings

Coding by researchers

 

 

Artifact-based interview

Semi-structured interview

 

 

 

Tasks for everyone

  1. Keep track of dodgy claims or questionable approaches which you encounter in the paper and post them to EdStem
  2. If you find confusing bits as you read, post questions in EdStem and I will cover them in the class

 

License
All rights reserved The University of Edinburgh