Jiwan Chung
Search
Duplicate
Try Notion
Jiwan Chung
I'm a Ph.D student @ SNU Vision & Learning Lab, advised by Gunhee Kim.
I do machine learning, NLP and multimodal knowledge understanding.
My long-term goal is machine epistemology, a.k.a understanding the operations of human understanding using computational methods.
✉️ Mail / 🎓 Google Scholar / 💻 Github My main point of contact is via Email!
👩🏻‍💻 Research Interest
Currently I am delving into the following topics.
Please feel free to contact me if you are interested in co-working or have any inquiry.
Mutimodality
Teaching multimodal context to pretrained language models
Constrained Decoding
Transporting constrained search from inference time to training time for better inference speed and performance
Analysis on Prompt-Based Learning
One thing is clear: prompts in pretrained models do not work as we humans intended
I am currently interested in analysing this phenomenon thoroughly and finding at least some reasons behind it
📚 Publications
Multimodal Knowledge Alignment with Reinforcement Learning (arXiv 2022)
I am one of the two lead authors of this work
What did I do?
I made the main idea work with architecture choices and stability rewards
Programming, including all models and experiments
Experiment design & practice
Paper writeup
Demo
ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning (ICCV 2021)
I am one of the two lead authors of this work
What did I do?
I came up with the main idea of using the set-level clustering-based mutual information as criterion
Programming, including the data curation pipeline and toy experiments
Paper writeup
Web & Demo
Transitional Adaptation of Pretrained Models for Visual Storytelling (CVPR 2021)
I am one of the two lead authors of this work
What did I do?
I designed the auxilliary loss for visual storytelling and suggested split-training
Programming, including all models and experiments
Experiment design & practice
Paper writeup
Character Grounding and Re-identification in Story of Videos and Text Descriptions (ECCV 2020)
I am a co-author of this work
What did I do?
Idea discussion
Experiment planning
Paper writeup assistance
⌨️ Programming Skills
Machine Learning
PyTorch: For debugging & fast prototyping
Jax: For scaling up to multi-gpu or tpu environments
Web: some experience on VueJS, SvelteJS, MongoDB, Dgraph, Azure, etc.
🐣 Education
Bachelor’s Degree (Yonsei University, ~2019)
Double Major
Computer Science
Philosophy
Minor
English Language & Literature
💬 Experience
Teaching Assistant
IoT/AI/Big-Data (IAB) (Undergraduate Course, Fall 2019)
Artificial Intelligence (AI) (Graduate Course, Spring 2021)
Computer Vision (CV) (Undergraduate Course, Fall 2021)