My research lies at the intersection of Human Computer Interaction (HCI) and Artificial Intelligence (AI),
with a strong focus on advancing user interface
understanding,
automation and
generation, and Gemini code
generation for UIs and apps. Motivated by challenges in these tasks, I work on
deep learning research that are applicable for general domains, such as
neural representations,
architectures
and learning methods.
In my spare time, I enjoy building and fixing things in the physical world
-- Check out our treehouse project.
Bio · Yang is a Senior Staff Research Scientist at Google DeepMind, and an Affiliate Associate Professor at University of Washington CSE. He is inducted by the ACM as a Distinguished Scientist for his "research contributions intersecting human-computer interaction and artificial intelligence". Yang has significantly contributed to UI understanding, automation, and generation by developing novel deep learning methods (e.g., Fourier Positional Encoding, and area attention) and impactful benchmarks (e.g., widget captioning, screen2words, seq2act and rico), helping to establish deep learning as a core approach in this area. He also pioneered on-device interactive machine learning on Android, leading to impactful product features like next app prediction and Gesture Search. Yang holds a Ph.D. in Computer Science from the Chinese Academy of Sciences, and completed postdoctoral research at UC Berkeley EECS. Yang has extensively published in top-tier venues across both the HCI and ML fields such as CHI, UIST, ICML, NeurIPS, ICLR, ACL, EMNLP, CVPR and KDD, earning multiple honors including Best Paper Awards from CHI and CVPR, and a Lasting Impact Award from UIST. A dedicated community member, he regularly serves as Area Chair/SPC for leading conferences across both fields. Yang co-edited the Springer book "AI for HCI: A Modern Approach", and organized inaugural workshops bridging the HCI and AI/ML field, including the first AI&HCI workshop at ICML.