While the range of AI techniques and technologies researched in classrooms and discussed at conferences continues to grow, the ethical consequences are rarely fully considered – at least, while there is much ethics research for AI in general, there is very little published work considering the ethics of AIED in particular. In short, as a field (while we apply our university research regulations), we are continuing to work without any fully-developed moral groundings specific to AIED.

In fact, as the AIED community is aware, the field of AIED raises an indeterminate number of as yet unanswered ethical questions. To begin with, concerns exist about the large volumes of data collected to support AIED (such as the recording of student competencies, emotions, strategies and misconceptions). Who owns and who is able to access this data, what are the privacy concerns, and who should be considered responsible if something goes wrong?

Other major ethical concerns centre on AIED computational approaches. How should the data be analysed, interpreted and shared? How should the biases (conscious or unconscious), that might impact negatively on the civil rights of individual students, be remedied – especially given that the scale of AIED in the coming years is likely to amplify any design biases (e.g. about gender, age, race, social status, income inequality…)?

However, and this is all too often ignored, the ethics of AIED cannot be reduced to questions about data or computational approaches. AIED research also needs to account for the ethics of education (which, although the subject of decades of research, is most often overlooked). For example, AIED research needs to address the fact that many of its educational assumptions are contested by the learning sciences community.

All that said, the ethics of data, computational approaches, and education are the ‘known unknowns’. But what about the ‘unknown unknowns’, the ethical issues raised by AIED – i.e., at the intersection of data, computation and education – that have yet to be even identified?

Ethics in AIED questions include:

  • What are the criteria for ethically acceptable AIED?
  • How does the transient nature of student goals, interests and emotions impact on the ethics of AIED?
  • What are the AIED ethical obligations of private organisations (developers of AIED products) and public authorities (schools and universities involved in AIED research)?
  • How might schools, students and teachers opt out from, or challenge, how they are represented in large datasets?
  • What are the ethical implications of many ITS and other AIED approaches adopting an instructionist approach to learning?
  • What are the ethical implications of not being able to easily interrogate how AIED deep decisions (using multi-level neural networks) are made?

Strategies are also needed for risk amelioration, since AI algorithms are vulnerable to hacking and manipulation. And where AIED interventions target behavioural change (such as by ‘nudging’ individuals towards a particular course of action), the entire sequence of AIED enhanced pedagogical activity also needs to be ethically warranted. And finally, it is important to recognise another perspective on AIED ethical questions: in each instance, the ethical cost of inaction and failure to innovate must be balanced against the potential for AIED innovation to result in real benefits for learners, educators and educational institutions.