Principles of Computing
Over the past few summers, I've had the opportunity to teaching introductory programming in Python to high schoolers, freshmen, and incoming MHCI students at CMU.
Access the open learning materials
Automatically evaluate the quality of educational multiple-choice questions according to a 19-critera rubric.
Try out the beta here
Students participate in the creation and evaluation of educational content, all while learning in the process.
Read about my recent work
Using LLMs such as GPT-4, I apply rubrics for evaluating the quality of educational questions.
Read about my recent work
We have crowdworkers participate in a variety of tasks such as evaluating question quality, providing the skills a question assesses, or creating a hint for a question.
Read about my recent work
My team recently won the $1M compeition to modernize, accelerate, and improve the ways in which we identify effective learning tools and processes that improve learning.
Check out the press release
Over the past few summers, I've had the opportunity to teaching introductory programming in Python to high schoolers, freshmen, and incoming MHCI students at CMU.
Access the open learning materials
I redesigned and taught a graduate-level course on designing, evaluating, and using educational technology in the classroom that leverage learning science principles.
Checkout the student feedback: 2023, 2024
I taught a gradudate-level course on introductory statistics, designed to prepare students for their future coursework and assist them in evaluating research designs.
Access the open learning materials
I updated and taught a graduate-level course on effective strategies for measuring student learning, enhancing student engagement, and assessing educational effectiveness.
I'll have student reviews soon :)
Teaching two graduate-level courses "Evidence-Based Educational Design" and "Tools for Online Learning", while also building tools and researching ways to improve the learning engineering process by harnessing (pedagogically informed) LLM-based methods.
METALS Faculty website
Provided faculty support for integrating technologylearning in their courses, enhanced CS courses at the undergraduate and graduate levels, and developed interactive tools to assist faculty in teaching across multiple domains.
Eberly Center's website
Devised heuristics to assess skill tagging production model saturation, evaluated alternate skill rescaling approaches for improved student learning modeling, and investigated the effects of post-assessment resource provision on learners.
Coursera's website
Conducted classroom observations to provide data-driven feedback to instructors, worked with clients to construct and revise statements of teaching philosophy, and lead microteaching workshops to provide instructors with teaching feedback.
Graduate Teaching Fellow's website
Created customized frontend websites to fulfill diverse client requirements, developed robust backend systems and managed extensive databases, and designed Android and iOS apps for surveys and location tracking purposes.
Westat's website| Year | Type | Publication |
|---|---|---|
| 2024 | Thesis 🤠 |
Steven Moore (2024). Toward Automation to Support Creation and Evaluation of Pedagogically Valid Multiple-Choice Question Assessments at Scale. For the degree ofDoctor of Philosophy in Human-Computer Interaction at Carnegie Mellon University |
| 2024 | Conference | Steven Moore, Norman Bier, John Stamper (2024). Assessing Educational Quality: Comparative Analysis of Crowdsourced, Expert, and AI-Driven Rubric Applications. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol. 12, forthcoming |
| 2024 | Conference | Steven Moore, Eamon Costello, Huy A Nguyen, John Stamper (2024). An Automatic Question Usability Evaluation Toolkit. In Proceedings of the 25th International Conference on Artificial Intelligence in Education, pp. 31-46 |
| 2024 | Conference 🏅 |
Steven Moore, Robin Schmucker, Tom Mitchell, John Stamper (2024). Automated Generation and Tagging of Knowledge Components from Multiple-Choice Questions. In Proceedings of the 11th ACM Conference on Learning @ Scale, pp. 122-133 |
| 2024 | Poster |
Ilya Musabirov, Mohi Reza, Steven Moore, Pan Chen, Harsh Kumar, Tong Li, Haochen Song, Jiakai Shi, Koby Choy, Thomas Price, John Stamper, Norman Bier, Nina Deliu, Sofia Villar, Anna Rafferty, Audrey Durand, Joseph Jay Williams (2024). Platform-based Adaptive Experimental Research in Education: Lessons Learned from Digital Learning Challenge. In Companion Proceedings in LAK24: 14th International Learning Analytics and Knowledge, pp. 37-40 |
| 2023 | Conference | Steven Moore, Huy A Nguyen, Tianying Chen, John Stamper (2023). Assessing the Quality of Multiple-Choice Questions Using GPT-4 and Rule-Based Methods. In Proceedings of the 18th European Conference on Technology Enhanced Learning, pp. 229-245 |
| 2023 | Conference |
Steven Moore, Ellen Fang, Huy A Nguyen, John Stamper (2023). Crowdsourcing the Evaluation of Multiple-Choice Questions Using Item-Writing Flaws and Bloom’s Taxonomy. In Proceedings of the 10th ACM Conference on Learning @ Scale, pp. 25-34 |
| 2023 | Journal |
Hassan Khosravi, Paul Denny, Steven Moore, John Stamper (2023). Learnersourcing in the Age of AI: Student, Educator and Machine Partnerships for Content Creation. In Computers and Education: Artificial Intelligence, 100151 |
| 2023 | Conference |
Steven Moore, Huy A Nguyen, Norman Bier, Tanvi Domadia, John Stamper (2023). Who Writes Tomorrow’s Learning Activities? Exploring Community College Student Participation in Learnersourcing. In Proceedings of the 17th International Conference of the Learning Sciences |
| 2023 | Practioner Report |
Norman Bier, John Stamper, Steven Moore, Darren Siegel, Ariel Anbar (2023). OLI Torus: A Next-Generation, Open Platform for Adaptive Courseware Development, Delivery, and Research. In Companion Proceedings in LAK23: 13th International Learning Analytics and Knowledge, pp. 57-60 |
| 2023 | Poster |
Steven Moore, Huy A Nguyen, John Stamper (2023). Students’ Domain Confidence and their Participation in Optional Learnersourcing Activities. In Companion Proceedings in LAK23: 13th International Learning Analytics and Knowledge, pp. 147-149 |
| 2022 | Conference |
Steven Moore, Huy A Nguyen, Norman Bier, Tanvi Domadia, John Stamper (2022). Assessing the Quality of Student-Generated Short Answer Questions Using GPT-3. In Proceedings of the 17th European Conference on Technology Enhanced Learning, pp. 243-257 |
| 2022 | Conference |
Huy A Nguyen, Shravya Bhat, Steven Moore, Norman Bier, John Stamper (2022). Towards Generalized Methods for Automatic Question Generation in Educational Domains. In Proceedings of the 17th European Conference on Technology Enhanced Learning, pp. 272-284 |
| 2022 | Poster |
Shravya Bhat, Huy A Nguyen, Steven Moore, John Stamper, Majd Sakr, Eric Nyberg (2022). Towards Automated Generation and Evaluation of Questions in Educational Domains. In Proceedings of the International Conference on Educational Data Mining, pp. 701-704 |
| 2022 | Conference |
Steven Moore, Huy A Nguyen, John Stamper (2022). Leveraging Students to Generate Skill Tags that Inform Learning Analytics. In Proceedings of the 16th International Conference of the Learning Sciences, pp. 791-798 |
| 2022 | Conference |
Steven Moore, Huy A Nguyen, John Stamper (2022). Participation and Success with Optional Self-Explanation for Students in Online Undergraduate Chemistry Courses. In Proceedings of the 16th International Conference of the Learning Sciences, pp. 1381-1384 |
| 2021 | Conference |
Steven Moore, Huy A Nguyen, John Stamper (2021). Examining the Effects of Student Participation and Performance on the Quality of Learnersourcing Multiple-Choice Questions. In Proceedings of the 8th ACM Conference on Learning @ Scale, pp. 209-220 |
| 2021 | Conference |
Huy A Nguyen, Michelle Lim, Steven Moore, Majd Sakr, Eric Nyberg, John Stamper (2021). Exploring Metrics for the Analysis of Code Submissions in an Introductory Data Science Course. In Proceedings of the 11th International Conference on Learning Analytics & Knowledge, pp. 632-638 |
| 2021 | Conference |
Steven Moore, John Stamper, Norman Bier, Mary Jean Blink (2021). A Human-Centered Approach to Data Driven Iterative Course Improvement. In Proceedings of the 17th International Conference on Remote Engineering and Virtual Instrumentation, pp. 742-761 |
| 2020 | Conference |
Steven Moore, Huy A Nguyen, John Stamper (2020). Towards Crowdsourcing the Identification of Knowledge Components. In Proceedings of the 7th ACM Conference on Learning @ Scale, pp. 245-248 |
| 2020 | Conference |
Rosalyn Abbott, Conrad Zapanta, Michael Melville, Steven Moore (2020). WIP: Incorporating Interactive Modules Related to Cell Culture and Plasmid Design into Introduction to Biomedical Engineering. In American Society of Engineering Education, pp. 234-238 |
| 2020 | Conference 🏅 |
Steven Moore, Huy A Nguyen, John Stamper (2020). Utilizing Crowdsourcing and Topic Modeling to Generate Knowledge Components for Math and Writing Problems. In Proceedings of the 28th International Conference on Computers in Education, pp. 31-40 |
| 2020 | Conference |
Steven Moore, Huy A Nguyen, John Stamper (2020). Evaluating Crowdsourcing and Topic Modeling in Generating Knowledge Components from Explanations. In Proceedings of the 21st International Conference on Artificial Intelligence in Education, pp. 398-410 |
| 2020 | Poster |
Steven Moore, Huy A Nguyen, John Stamper (2020). Crowdsourcing Explanations for Improving Assessment Content and Identifying Knowledge Components. In Proceedings of the 14th International Conference of the Learning Sciences, pp. 2637-2638 |
| 2019 | Workshop |
Norman Bier, Steven Moore, Martin Van Velsen (2019). Instrumenting Courseware and Leveraging Data with the Open Learning Initiative (OLI). In Companion Proceedings 9th International Learning Analytics & Knowledge Conference, pp. 990-1001 |
| 2019 | Workshop |
Steven Moore, John Stamper (2019). Exploring Expertise through Visualizing Agent Policies and Human Strategies in Open-Ended Games. In Proceedings of the Workshops of the 12th International Conference on Educational Data Mining, pp. 30-37 |
| 2019 | Demo |
John Stamper, Paulo Carvalho, Steven Moore, Kenneth Koedinger (2019). Tigris: An Online Workflow Tool for Sharing Educational Data and Analytic Methods. In Companion Proceedings 9th International Learning Analytics & Knowledge Conference, pp. 189 |
| 2019 | Conference |
John Stamper, Steven Moore (2019). Exploring Teachable Humans and Teachable Agents: Human Strategies Versus Agent Policies and the Basis of Expertise. In Proceedings of the 20th International Conference on Artificial Intelligence in Education, pp. 269-274 |
| 2019 | Conference 🏅 |
Steven Moore, John Stamper (2019). Decision Support for an Adversarial Game Environment Using Automatic Hint Generation. In Proceedings of the 15th International Conference on Intelligent Tutoring Systems, pp. 82-88 |
| Year | Venue | Publication |
|---|---|---|
| 2024 | L@S | Steven Moore, Anjali Singh, Xinyi Lu, Hyoungwook Jin, Hassan Khosravi, Paul Denny, Christopher Brooks, Xu Wang, Juho Kim, John Stamper (2024). Learnersourcing: Student-generated Content @ Scale. (2nd annual workshop) at Learning @ Scale 2024 https://sites.google.com/andrew.cmu.edu/learnersourcing |
| 2024 | EDM | Neil Heffernan, Rose Wang, Christopher MacLellan, Arto Hellas, Chenglu Li, Candace Walkington, Joshua Littenberg-Tobias, David Joyner, Steven Moore, Adish Singla, Zach Pardos, Maciej Pankiewicz, Juho Kim, Shashank Sonkar, Clayton Cohn, Anthony Botelho, Andrew Lan, Lan Jiang, Mingyu Feng, Tanja Kaser, Eamon Worden (2024). Leveraging Large Language Models for Next-Generation Educational Technologies. at Educatonal Data Mining 2024 https://sites.google.com/view/llmworkshopedm/home |
| 2023 | NeurIPS | Paul Denny, Sumit Gulwani, Neil Heffernan, Tanja Käser, Steven Moore, Anna Rafferty, Adish Singla (2023). Generative AI for Education (GAIED): Advances, Opportunities, and Challenges. at Neural Information Processing Systems 2023 https://gaied.org/neurips2023 |
| 2023 | AIED | Steven Moore, Richard Tong, Anjali Singh, Zitao Liu, Xiangen Hu, Yu Lu, Joleen Liang, Chen Cao, Hassan Khosravi, Paul Denny, Chris Brooks, John Stamper (2023). Empowering Education with LLMs-The Next-Gen Interface and Content Generation. at Artificial Intelligence in Education 2023 https://ai4ed.cc/workshops/aied2023 |
| 2023 | LAK | Steven Moore, Anjali Singh, Hassan Khosravi, Paul Denny, Chris Brooks, John Stamper (2023). Partnerships for Cocreating Educational Content. at Learning Analytics and Knowledge 2023 https://sites.google.com/andrew.cmu.edu/partnerships-for-cocreating-ed |
| 2022 | L@S | Steven Moore, John Stamper, Christopher Brooks, Paul Denny, Hassan Khosravi (2023). Learnersourcing: Student-generated Content @ Scale. at Learning @ Scale 2022 https://sites.google.com/andrew.cmu.edu/learnersourcing (proceedings) |