I'm a Ph.D student @ Yonsei MIR lab, advised by Youngjae Yu.
I do vision-to-language machine learning and knowledge understanding.
My long-term goal is understanding the operations of human understanding using computational methods.
PublicationsFusing Pre-trained Language Models with Multimodal Prompts through Reinforcement Learning (CVPR 2023)
I am one of the two lead authors of this work
What did I do?
I made the main idea work with the architecture choices and the stability rewards
Programming, including all models and experiments
Experiment design & practice
Paper writeup
Demo
ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning (ICCV 2021)
I am one of the two lead authors of this work
What did I do?
I came up with the main idea of using the set-level clustering-based mutual information as criterion
Programming, including the data curation pipeline and toy experiments
Paper writeup
Web & Demo
Transitional Adaptation of Pretrained Models for Visual Storytelling (CVPR 2021)
I am one of the two lead authors of this work
What did I do?
I designed the auxilliary loss for visual storytelling and suggested split-training
Programming, including all models and experiments
Experiment design & practice
Paper writeup
Programming Skills Machine Learning
PyTorch: For debugging & fast prototyping
Jax: For scaling up to multi-gpu or tpu environments
Gradio: For demos & data explorers
Education Bachelor’s Degree (Yonsei University, ~2018)
Double Major
Computer Science
Philosophy
Minor
English Language & Literature
Master’s Degree (Seoul National University, ~2022)
Major
Computer Engineering (Machine Learning)
Doctor’s Degree (Yonsei University)
Major
Artificial Intelligence (Multimodal Learning)
ExperienceTeaching Assistant
IoT/AI/Big-Data (IAB) (Undergraduate Course, Fall 2019)
Artificial Intelligence (AI) (Graduate Course, Spring 2021)
Computer Vision (CV) (Undergraduate Course, Fall 2021)