- Nigel Bosch
University of Illinois
Machine-learned models of student behaviors, emotions, and outcomes do not always generalize well to populations that differ in demographic composition from the training data. This phenomenon can result from differences in familiarity with computers, socioeconomic status, feelings of belonging, cultural constraints, and many other differences. Lack of model generalization leads to systematic biases when models are less accurate for some groups of students than others, especially for traditionally underrepresented groups (e.g., women in STEM) because data for these groups are inherently less available for model training. Our work focuses on quantifying such model biases and developing methods to reduce them. We first identify biases by measuring the accuracy of student models across groups. For example, in one study we built models that used the sentiment of student-written text as a predictor of course engagement and found that models were biased across genders. That is, models trained with only data from male students were less accurate when applied to their peers, which can be an issue in courses where male students represent the vast majority of students (and thus available training data). We are also researching methodologies to reduce such biases once they have been identified, focusing primarily on transfer learning and semi-supervised machine learning methods. In the transfer learning approach, we train models on a dataset that is external to the eventual target dataset, focusing on external datasets with high demographic heterogeneity. We then use reuse parts of the external model when training models on the target dataset, thus leveraging the diversity in the external dataset. In the semi-supervised approach, we learn features from minority student data in an unsupervised fashion (where data are more widely available) before training a supervised model. Through these methods we hope to enable adaptive educational systems that are truly open and egalitarian.
- Shayan Doroudi
Carnegie Mellon University Equity in AIED: Who Cares?
Artificial intelligence algorithms used in educational environments rarely have objectives that are concerned with whether the algorithms are equitable. As a result, although it is not immediately obvious, certain algorithms proposed for artificial intelligence in education and educational data mining can lead to widening the gap between high-performing students and low-performing students. I will briefly demonstrate this in two very different settings. First, I show that when fitting the popular Bayesian knowledge tracing (BKT) algorithm to data coming from a mix of student populations, the error in the parameters can lead to worse outcomes (more under practice) for low-performing students than for high-performing students. Second, I will show that for a recently proposed game-theoretic mechanism for incentivizing students to accurately self-assess their work, the mechanism inherently leads to artificially deflating the grades of low-performing students. The hope is to generate discussion around what kinds of checks we should make as researchers to ensure our algorithms do not act inequitably and how we can develop algorithms that explicitly try to reduce inequity rather than implicitly increasing it.
- Garron Hillaire
The Open University, UK From climate change to the world cup: ethics for artificial intelligence and emotion detection in chat messages
In this five-minute lightning presentation, the intersection of Emotional Intelligence (EI) and Artificial Intelligence (AI) will be examined to consider ethics. When AI supports humans to augment their intelligence it can be referred to as Intelligence Augmentation (IA) . The ability model of EI includes the ability to identify emotions in oneself and others . By considering how the ability model of EI can be supported through IA this workshop considers ethical questions of Emotional Intelligence Augmentation (EIA).
When computers support the identification of emotion expression in chat discussions what should such a technology do when an AI detects discrimination based on nationalism. The lightning talk will demonstrate how text detection of negative valence could potentially identify situations where social perspectives from a national majority are negative toward a national minority in two contexts:
- A) discussions about the Paris accords
- B) discussions about the FIFA world cup
By viewing the social perspective to be negative on a speaker based on their national identity the ethical frameworks of Consequentialism and Duty will be used to determine what might be ethical in these two contexts. The contexts are referenced when evaluating the presence of social injustice that merits “special” minority rights .
- Iris Howley
If an Algorithm Is Openly Accessible, and No One Can Understand It, Is It Actually Open?
As blackbox algorithms play an increasing role in classroom decision-making, calls to “open” these algorithms and explain the inputs and latent variables that determine the decision outcomes grow increasingly louder. However, even systems using open algorithms face similar concerns. Using open algorithms does not mean that stakeholders (e.g., instructors, students, parents, etc.) of the system understand the connection between features in the underlying machine learning model and the outcomes displayed to them. Some algorithms (i.e., multilevel neural networks) are too complex to easily interrogate the decision-making process, but other algorithms (e.g., Bayesian Knowledge Tracing (BKT), classification, etc.) are considerably more comprehendible, but teachers and students still do not understand them. This work asks, what are the ethical implications of providing students and teachers with algorithmic decision-making software they could interrogate, but due to lack of knowledge, cannot interrogate and how might researchers help bridge that gap?
First steps of this work focus on OARS, an assessment and learning system described in Bassen et al. (2018), that uses BKT to predict student skill mastery. BKT is built on a Hidden Markov Model where student skill mastery is output as either “mastered” or “unmastered.” Parameters in this model include probabilities related to existing mastery, slipping (i.e., forgetting), learning, and guessing (Bassen et al., 2018). In relation to deep learning approaches, BKT is rather simple to interrogate for relationships between parameters, however, just because it is possible to understand, does not mean the students and instructors who use BKT-based systems do understand it.
I performed eight semi-structured interviews with instructors discussing their use of educational technology tools in the classroom. 6 of the interviewees used OARS previously, and so could speak directly to their understanding of the underlying BKT model, and how (or why) they trust the skill predictions it produces. In general, interviewees trusted the OARS output without a firm grasp on the underlying decision-making, such as P7, a professor at a community college, who said, “I have not thought much about the specific algorithm because I tend on being more trusting in the algorithm than not.”
From initial interviews on this topic, it is apparent that the space of interrogating AIED algorithmic decisions is a multi-dimensional issue. Along the openness algorithm dimension, there are algorithms too difficult to interrogate, algorithms that are complex but explainable, and then there are blackbox commercial algorithms. Along the user motivation dimension, there are users who want to understand the algorithm and have the prior knowledge to do so, users who want to understand but lack the requisite prior knowledge, and users who are less motivated to understand algorithmic processes. User motivation also intersects with system trust, although exactly how is not yet known. Investigation into these issues is necessary to understand when and how intervention should be provided to users of algorithmically enhanced learning (AEL) environments.
As artificial intelligence in education researchers, we must ask, what are the ethical implications of deploying systems that our instructors and students, cannot understand, but could if provided with the appropriate scaffolding? If students and instructors are using our systems to change their approach to learning, should they not first understand, when possible, the outputs their AEL environments are producing? My current project begins to examine how to scaffold the learning of the algorithmic decision-making process for instructors, and how this relates to trust in the algorithmically enhanced learning environment.
5. Manon Knockaert and Jean-Marc Van Gyseghem
Université de Namur
H2020 TeSLA project: Data protection law and e-learning get well together
New technologies’ emergence enable e-learning activities. By given the possibility to grant access to teachings to a large number of people with various lifestyle, one of the main purpose of e-learning is the democratization of teaching and education. In the one hand, the increasing use of metrics and algorithms in order to reinforce the effectiveness of online education and, mainly, e-assessment. This phenomenon raises many ethical issues. The ethical reflection could find some answers into the new Regulation adopted by the European Union about the protection of personal data. Privacy law and ethics are intrinsically linked in order to ensure a social acceptability for the use of technologies. One example of this possibility is the TeSLA project.The TeSLA project provides to educational institutions, an adaptive trusted e-assessment system for assuring e-assessment processes in online and blended environments. It will support both continuous and final assessment to improve the trust level across students, teachers and institutions.
The objective of the H2020 project TeSLA is the designing and development of a set of tools for academic institutions. TeSLA is using biometrics data from students such as facial and vocal recognition in order to ensure the real identity of the student that participates to the e-assessment. One of the challenge is to put in place an ethical system that respects privacy of students without reducing trustability and correctness of the identification and authorship. Following the new requirement of the General Data Protection Regulation, TeSLA adopts a privacy by design/default approach. The analysis of this approach, combined with ethical principles, shed light on what kind and what amount of personal data can be processed (including the collect), who controls the processing of personal data, who can have access to the data and how to insure a fair balance between openness and transparency. Therefore, legal and ethical considerations are closely connected and their coexistence enriches each other.
6. Mike Sharples
The Open University
Ethics for AI and informal learning
Most learning during a person’s lifetime takes place outside classrooms, in the home, workplace, or outdoors. Since the early 2000s, research has explored the design of mobile, contextual, seamless and ubiquitous learning with technologies. These technologies can track how a person gains knowledge and seeks support across locations, times, devices, and social engagements, throughout a lifetime. We learn, in part, by creating plausible narratives for ourselves to explain our beliefs and actions. AI has the potential to intercede and change those narratives, to create different stories that bind our lives and others. This is starting to be seen in elections and in response to political events, where the type of software that generates personalised adverts is producing ubiquitous political messages and fake news across multiple media channels that create alternate narratives.
Closer to home, the FutureLearn platform was designed from an explicit pedagogy of learning through conversation, drawing on the Conversation Theory of Gordon Pask. The aim is to enable pervasive conversations amongst learners and educators linked to each piece of learning content. This has been more successful that originally conceived, with each course having conversations of up to half a million contributions, and a single learning item attracting up to 65,000 learner comments and replies. From the outset, in the 1970s, Pask conceived Conversation Theory as embracing computers as conversational partners. We now have technology through text analysis, learner modelling and story generation, for AI systems to contribute to those conversations. Such methods have already been introduced into online teaching forums, without the knowledge of students: “It was not until the end of the Spring 2016 term that we told the students that one of their teaching assistants secretly was an AI agent” (Eicher, Polepeddi & Goel, 2017).
The ethics of AIED needs to consider not only the influence of AI systems on classroom teaching and online courses, but the new ways that AI can influence informal learning by creating plausible alternative narratives and by guiding conversations.
7. Alessandra Silveira
Evaluating a personalized course recommendation system and the implications of its use in the university
As the applications of artificial intelligence in education become more prevalent and visible in research and practice, the domain in which it operates becomes one of loose constraints. At UC Berkeley, the personalized course recommendation system AskOski.berkeley.edu uses a word vector model to sequence millions of student enrollment data to learn the conceptual relationships between courses across departments and majors. The hypothesis assumes that conceptually similar courses will be mapped closely together like semantically similar words are in most natural language processing tasks. The distributed cognitive process of course selection encodes facets of information about courses that extend beyond their course descriptions. Previous work has shown that we can surface conceptually similar courses using this method. This work extends the practice to more ephemeral and socially constructed concepts of Diversity, Equity and Inclusion to evaluate the efficacy and relevance of the courses that the model surfaces. This work also investigates what a “good recommendation” looks like, as course selection in higher education lives beyond degree satisfying requirements. What is a “good recommendation” if the model recommends an algorithmically sound course using novel personalized modeling, but that is far above a students’ ability, and causes stress or negative feelings in their academic experience? This work utilizes a mixed method approach to evaluate such course recommendations, as well as investigate the underlying assumptions and implications of using such modeling without scrutable data.