Yue Yang (杨樾)
pronounced as yoo-eh

Moore 103 (SIG Lab), 3300 Walnut St
Philadelphia, PA 19104, USA
Email: yueyang1 [at] seas.upenn.edu

Google Scholar   /   GitHub   /   LinkedIn   /   Youtube   /   pixiv   /   CV
About me

Hi! My name is Yue Yang (杨樾). I am a second-year Ph.D. student in Computer and Information Science at the University of Pennsylvania, affiliated with Penn NLP. I am grateful to be advised by Prof. Chris Callison-Burch and Prof. Mark Yatskar.

I am interested in the intersection area of Natural Language Processing (NLP) and Computer Vision (CV), aka Multimodal. My current research focuses on two directions to marry vision and language:

  • Vision helps Language: Leverage visual knowledge to alleviate the reporting bias of language models.
  • Language helps Vision: Language as intermediate, explict representation to improve explanability on vision tasks.

Before Penn, I was an undergrad in Mechanical Engineering at the College of Energy Engineering at Zhejiang University. I worked with Prof. Yuqi Huang to study Computational Fluid Dynamics (CFD) during undergrads.

Penn initially admitted me as a master's student in Robotics in 2018. Now I switch to Artificial Intelligence and temporarily farewell to the hardware (Not wholly True, I designed a sensor pack for R2D2, which students of CIS-521 will use).

Shuyan Zhou, Harry Li Zhang, Yue Yang, Veronica Qing Lyu, Pengcheng Yin, Chris Callison-Burch, Graham Neubig
Annual Meeting of the Association for Computational Linguistics (ACL), 2022
TL;DR: Procedures are inherently hierarchical. To "host a party", one may need to "clean the house", which in turn may require "putting away the clothes". We develop a simple and efficient method that links steps (e.g. "clean the house") in an article to other articles with similar intents (e.g. "how to deep lean your house"), which proceeds recursively to form the KB.
Yue Yang, Joongwon Kim, Artemis Panagopoulou, Mark Yatskar and Chris Callison-Burch
CVPR 2022 @ ODRUM, 2022, spotlight talk
TL;DR: This work proposes a novel system that induces schemata from web videos and generalizes them to capture unseen tasks with the goal of improving video retrieval performance, and demonstrates that the schemata induced by the system are better than those generated by other models.
Yue Yang, Artemis Panagopoulou, Qing Lyu, Li Zhang, Mark Yatskar and Chris Callison-Burch
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
TL;DR: This work proposes the Visual Goal-Step Inference (VGSI) task where a model is given a textual goal and must choose a plausible step towards that goal from among four candidate images. We construct a VGSI dataset from wikiHow and show that SOTA multimodal models struggle on it.
Yue Yang, Yuqi Huang and Jisheng Zhao
Applied Thermal Engineering 169 (2020): 114932.
TL;DR: Numerically studied the dewing and defogging progress of a truck cabin; The external flow field was analyzed to get convective heat transfer coefficient; Effect of air velocity, temperature and humidity was considered and compared; Optimal control strategy of air conditioner was concluded based on these studies.
Yue Yang, Jinxing Chen, Yuqi Huang*, Jiangang Chen and Yuan Ji
13th International Conference on Heat Transfer, Fluid Mechanics and Thermodynamics
TL;DR: This paper aims to explore the optimum gap of tractor-trailers to reduce the aerodynamic drag by analyzing the mechanisms and laws of the flow which will decrease their energy consumption.
University of Pennsylvania, Philadelphia, PA, USA
  • Ph.D. in Computer and Information Science (2020 - present)
  • M.S. in Robotics (2018 - 2020)
  • Zhejiang University, Hangzhou, China
  • B.E. in Mechanical Engineering (2014 - 2018)
  • Experiences
    Tencent AI Lab, Seattle, WA, USA
    Research Scientist Intern (May. 2022 to Sept. 2022)
    University of Pennsylvania & Coursera, Philadelphia, PA, USA
    Associate Instructor (Sept. 2020 to Sept. 2022)
    University of Pennsylvania, Philadelphia, PA, USA
    Research Assistant (May. 2020 to present)

    Head Teaching Assistant, CIS-521 Artificial Intelligence, University of Pennsylvania
    Fall2019; Fall 2020; Summer 2021; Fall 2021; Spring 2022

    Teaching Assistant, CIS-530 Computational Linguistics, University of Pennsylvania
    Spring 2021
    CLUNCH, University of Pennsylvania, Philadelphia, PA, USA
    Investigate Procedural Events in a Multimodal Fashion, November 22, 2021. slides

    Website source from Jon Barron.