Irene Ai Lin

A few things I've researched & built:

Confidently Wrong: Predicting LLM Miscalibration

LLM Evaluation · Python · Logistic Regression · SHAP · 2026

Can we predict when an LLM is wrong from the way it communicates certainty? We constructed a 594-response evaluation dataset across GPT-4o, Claude Haiku, and Llama 3.3 70B, engineering linguistic features around certainty, hedging, and authority to predict factual correctness. Using logistic regression and SHAP, we found that model- and domain-conditioned features improved AUC from 0.556 to 0.638, with hedge density emerging as the strongest linguistic predictor.

poster preview

Curricula for the Concerto with RoboPianist

Reinforcement Learning · PyTorch · Robotics · Music · 2026

Can curriculum learning help reinforcement learning agents master increasingly difficult piano pieces? I developed MIDI-derived difficulty metrics to construct training curricula for RoboPianist, then trained SAC policies for 2M environment steps per repertoire and evaluated policy performance using return and note-level F1. Curriculum learning improved onset F1 from 0.331 to 0.358, while overall return and note accuracy remained largely unchanged.

poster preview

Seamless

Cofounder · Stanford Lean LaunchPad · LLMs · Vector Search · 2026

We built Seamless to explore how AI could turn the relationships formed at enterprise events into something people could actually act on. In ten weeks through Stanford's Lean LaunchPad, our team built a relationship intelligence platform combining vector search with an LLM-powered assistant, interviewed 200+ potential users, and iterated with four enterprise design partners. The experience made me fall in love with building at the intersection of AI, product, and people, and I'm excited to return to Lean LaunchPad next year as a course assistant, helping the next cohort build and test ideas of their own!

Protecting Against Propaganda

LLMs · Human-AI Interaction · OpenAI API · 2023

Could an LLM help people recognize when online content is trying to persuade them? For my first research project in computer science, I worked with Stanford researchers to investigate how language models could support more critical news consumption. We built a Chrome extension that analyzes webpages in real time, using LLM prompting to identify persuasive techniques and surface contextual explanations while a user reads, and evaluated prompting strategies for encouraging reflection without simply telling users what to believe.

system flow