Recent & Upcoming Activities

If you are interested in a PhD in NLP, information retrieval, or data science please get in touch with Claudia Hauff (c.hauff@tudelft.nl)

  • November 2018: Claudia Hauff will be at EMNLP 2018.

  • October 2018: Felipe Moraes and Claudia Hauff will be at CIKM 2018 to present their full paper Contrasting search as a learning activity with instructor-designed learning, which was also co-authored by one of our Master students (Sindunuraga Rikarno Putra).

  • September 2018: Two full papers by Lambda-Lab members will be presented at the 13th European Conference on Technology Enhanced Learning.  The paper SRLx: A Personalized Learner Interface for MOOCs by Dan Davis, Vasileios Triglianos, Claudia Hauff and Geert-Jan Houben is a collaboration between the University of Lugano and TU Delft. The paper Can I have a Mooc2Go, please? On The Viability of Mobile vs. Stationary Learning by Yue Zhao, Tarmo Robal, Christoph Lofi and Claudia Hauff is a collaboration with Tallinn University of Technology. The latter paper received a best paper nomination!

  • August 2018: Felipe Moraes will be at DESIRES 2018 to present our collaborative search framework.

  • July 2018: The Computers & Education journal has published our review on innovations in online learning strategies. Preprint available here.

  • July 2018: Felipe Moraes and Claudia Hauff attended SIGIR 2018, where they presented demos on collaborative search [PDF] and A/B testing [PDF].

  • June 2018: Guanliang Chen presented his research on collecting a large-scale dataset that can be used for educational question generation in the 12th International AAAI Conference on Web and Social Media (ICWSM). The slides are available online.

  • June 2018: Guanliang Chen presented his research on knowledge tracing in the 11th Workshop on Innovative Use of NLP for Building Educational Applications (co-located with NAACL HLT). The poster is available online.

  • May 2018: Claudia Hauff gave an invited keynote talk at CORIA, the French chapter of Information Retrieval conference on the topic of Search As Learning. The slides are available online.

  • April 2018: Tarmo Robal's and Yue Zhao's work on IntelliEye: Enhancing MOOC Learners’ Video Watching Experience with Real-Time Attention Tracking was accepted as a full paper at ACM Hypertext 2018.

  • March 2018: Guanliang Chen's work on creating a dataset for learning question generation (aptly named LearningQ: A Large-scale Dataset for Educational Question Generation) was accepted at ICWSM 2018 as a full paper.  

Why the Name?

It is tradition in our research group to form teams around Greek letters, usually a letter that is associated with a major topic of the team. And thus we picked Lambda to highlight Learning. Sigma (which would have been another obvious choice due to Search) was already taken.

Research Lines

Lambda-Lab brings together research expertise in data science and information retrieval and applies that expertise to the domain of online learning, more specifically to Massive Open Online Learning. TU Delft is one of the largest MOOC providers in Europe and as such offers a unique research environment to investigate and explore human learning in a computational and large-scale manner. With more than 50 TU Delft MOOCs running on the edX platform and nearly 2 million registered learners (as of July 2018), we have a rare opportunity to empirically explore and evaluate learning interventions. The research we conduct can be split along four research lines, with the first two being our focus going into the next years:

  • Beyond the MOOC platform: enriching large-scale online learning through external data sources.
  • Search as learning: enriching large-scale online learning through search-based adaptations as search is an important component of the learning process.
  • Within the MOOC platform: enriching large-scale online learning through MOOC environment adaptations.
  • Data analytics in the classroom: enriching the students’ and teachers’ experience in the traditional classroom. This research line is not focused on Massive Open Online Learning, but instead the more traditional higher-education classroom.

Researchers Involved

Several researchers of the WIS group contribute to this research line:

  • Claudia Hauff is an Associate Professor at WIS and the lab leader of Lambda-Lab. She is the daily supervisor of all MSc and PhD projects within this research line. Contact her via c.hauff@tudelft.nl with any questions about Lambda-Lab.
  • Dimitrios Bountouridis is a Postdoc (financed by NWA project VWData) who works on fairness and bias in news recommender systems (and yes, there is also a connection to MOOCs in the project).
  • Guanliang Chen: the focus of his PhD research is learner analytics beyond the MOOC platform, funded by TU Delft's Extension School.
  • Dan Davis' PhD research is centered around the impact of the online learning environment on learners and learning, funded as part of CEL.
  • Yue Zhao (PhD student, financed via the CSC) employs data analytics to make sense of MOOC learners' behaviour.
  • Felipe Moraes's PhD is financed by NWO (via Claudia Hauff's TOP2 grant) and works on collaborative search questions in the setting of MOOCs.
  • Geert-Jan Houben is the head of WIS and provides expertise in user modeling and Web systems.

as well as a number of Master students:

  • Kilian Grashoff's focus is the implementation of digital workspaces for large-scale collaborative search.
  • Daan Rennings is working on axiomatic approaches for deep IR models.
  • Nirmal Roy is investigating local word embeddings for different IR tasks.
  • Maria Gatou is exploring gamification in MOOCs.
  • Tim Molenaar is researching how to improve work-wellbeing via personal digital assistants.

We also actively collaborate on projects/papers with other researchers:

  • Vassileios Triglianos and Cesare Pautasso, University of Lugano.
  • Tarmo Robal, Tallinn University of Technology.
  • René Kizilcec, Stanford University.
  • Markus Krause, Telefonica.
  • Christoph Lofi, Delft University of Technology.

Past members of Lambda-Lab include:

  • Mónica Marrero worked as research engineer (financed by Delft Data Science) who focused on building systems enabling large-scale online experiments.
  • MSc student Sindu Sindunuraga Rikarno Putra investigated single-user and collaborative search for learning, and implementing a collaborative search system in the process; he heavily contributed to a SIGIR 2018 demo paper, a DESIRES paper and a CIKM paper! Sindu defended his thesis in August 2018.
  • MSc student Ioana Jivet explored how to increase self-regulation in MOOC learners through the use of an interactive learning tracker. She defended her thesis in September 2016.  Currently, Ioana is working as a PhD student at the Welten-instituut. Her thesis work contributed to a full paper at LAK 2017.
  • MSc student Yingying Bao investigated the prevalence of cheating in MOOCs. She defended her thesis in February 2017. Her thesis work contributed to a short paper at EDM 2017.
  • MSc student Jochem de Goede designed a workbench to increase the reproducibility of MOOC experiments. He defended his thesis in August 2017.
  • MSc student Sambit Praharaj collaborated with researchers from Lugano to bring data analytics to the higher-education classroom. His thesis work (defended in August 2017) contributed to a full paper at UMAP 2017. Sambit is now a PhD student at the Welten-instituut.

Software

We are regularly producing software artifacts in our lab. Here we list the more mature open-source contributions we have made:

  • SearchX is a scalable collaborative search system, developed to research large-scale search and sensemaking experiments.
  • APONE is an academic environment for online experiments. We have used in our research and teaching.

Overview of our Work

To get a quick overview of the kind of questions we have looked at in the past year(s), take a look at the slides, they capture the essence of our approach and each of our publications in 1-2 slides (check the speaker notes as well for more insights).

Publications

+++ 2019 +++

Dimitrios Bountouridis, Jaron Harambam, Mykola Makhortykh, Monica Marrero, Nava Tintarev and Claudia Hauff:
SIREN: A Simulation Framework for Understanding the Effects of Recommender Systems in Online News Environments.
Accepted as a full paper at the 2nd ACM Conference on Fairness, Accountability, and Transparency.

+++ 2018 +++

Felipe Moraes, Sindunuraga Rikarno Putra and Claudia Hauff:
Contrasting search as a learning activity with instructor-designed learning.
Accepted as a full paper at CIKM 2018.

Sindunuraga Rikarno Putra, Kilian Grashoff, Felipe Moraes, Claudia Hauff:
On the Development of a Collaborative Search System.
Accepted as full paper at DESIRES 2018.

Yue Zhao, Tarmo Robal, Christoph Lofi, Claudia Hauff:
Can I have a Mooc2Go, please? On The Viability of Mobile vs. Stationary Learning.
Accepted as full paper at EC-TEL 2018. Best Paper Nominee.

Daniel Davis, Vasileios Triglianos, Claudia Hauff and Geert-Jan Houben:
SRLx: A Personalized Learner Interface for MOOCs.
Accepted as full paper at EC-TEL 2018.

Dan Davis, Guanliang Chen, Claudia Hauff, Geert-Jan Houben:
Activating learning at scale: A review of innovations in online learning strategies. Computers & Education 125: 327-344 (2018)

Guanliang Chen, Dan Davis, Markus Krause, Efthimia Aivaloglou, Claudia Hauff, Geert-Jan Houben:
From Learners to Earners: Enabling MOOC Learners to Apply Their Skills and Earn Money in an Online Market Place. TLT 11(2): 264-274 (2018)

Guanliang Chen, Claudia Hauff, Geert-Jan Houben:
Feature Engineering for Second Language Acquisition Modeling. BEA@NAACL-HLT 2018: 356-364

Tarmo Robal, Yue Zhao, Christoph Lofi, Claudia Hauff:
IntelliEye: Enhancing MOOC Learners' Video Watching Experience through Real-Time Attention Tracking. HT 2018: 106-114

Guanliang Chen, Jie Yang, Claudia Hauff, Geert-Jan Houben:
LearningQ: A Large-Scale Dataset for Educational Question Generation. ICWSM 2018: 481-490

Tarmo Robal, Yue Zhao, Christoph Lofi, Claudia Hauff:
Webcam-based Attention Tracking in Online Learning: A Feasibility Study. IUI 2018: 189-197

Dan Davis, René F. Kizilcec, Claudia Hauff, Geert-Jan Houben:
The half-life of MOOC knowledge: a randomized trial evaluating knowledge retention and retrieval practice in MOOCs. LAK 2018: 1-10

Dan Davis, Daniel Seaton, Claudia Hauff, Geert-Jan Houben:
Toward large-scale learning design: categorizing course designs in service of supporting learning outcomes. L@S 2018: 4:1-4:10

Sindunuraga Rikarno Putra, Felipe Moraes, Claudia Hauff:
SearchX: Empowering Collaborative Search Research. SIGIR 2018: 1265-1268

Mónica Marrero, Claudia Hauff:
A/B Testing with APONE. SIGIR 2018: 1269-1272

Yue Zhao, Tarmo Robal, Christoph Lofi, Claudia Hauff:
Stationary vs. Non-stationary Mobile Learning in MOOCs. UMAP (Adjunct Publication) 2018: 299-303

+++ 2017 +++

Yue Zhao, Christoph Lofi, Claudia Hauff:
Scalable Mind-Wandering Detection for MOOCs: A Webcam-Based Approach. EC-TEL 2017: 330-344
Best Paper Nominee.

Yingying Bao, Guanliang Chen, Claudia Hauff:
On the Prevalence of Multiple-Account Cheating in Massive Open Online Learning. EDM 2017

Dan Davis, Ioana Jivet, René F. Kizilcec, Guanliang Chen, Claudia Hauff, Geert-Jan Houben:
Follow the successful crowd: raising MOOC completion rates through social comparison at scale. LAK 2017: 454-463

Guanliang Chen, Dan Davis, Markus Krause, Claudia Hauff, Geert-Jan Houben:
Buying time: enabling learners to become earners with a real-world paid task recommender system. LAK 2017: 578-579

Yue Zhao, Dan Davis, Guanliang Chen, Christoph Lofi, Claudia Hauff, Geert-Jan Houben:
Certificate Achievement Unlocked: How Does MOOC Learners' Behaviour Change? UMAP (Adjunct Publication) 2017: 83-88

Vasileios Triglianos, Sambit Praharaj, Cesare Pautasso, Alessandro Bozzon, Claudia Hauff:
Measuring Student Behaviour Dynamics in a Large Interactive Classroom Setting. UMAP 2017: 212-220

+++ 2016 +++

Dan Davis, Guanliang Chen, Tim Van der Zee, Claudia Hauff, Geert-Jan Houben:
Retrieval Practice and Study Planning in MOOCs: Exploring Classroom-Based Self-regulated Learning Strategies at Scale. EC-TEL 2016: 57-71
Best Student Paper Award.

Vasileios Triglianos, Cesare Pautasso, Alessandro Bozzon, Claudia Hauff:
Inferring Student Attention with ASQ. EC-TEL 2016: 306-320

Dan Davis, Guanliang Chen, Claudia Hauff, Geert-Jan Houben:
Gauging MOOC Learners' Adherence to the Designed Learning Path. EDM 2016: 54-61

Guanliang Chen, Dan Davis, Claudia Hauff, Geert-Jan Houben:
Learning Transfer: Does It Take Place in MOOCs? An Investigation into the Uptake of Functional Programming in Practice. L@S 2016: 409-418
Best Paper Nominee.


Guanliang Chen, Dan Davis, Jun Lin, Claudia Hauff, Geert-Jan Houben:
Beyond the MOOC platform: gaining insights about learners from the social web. WebSci 2016: 15-24