I'm interested in where perception meets understanding.
I'm a PhD student at Yonsei University MIR Lab, advised by Youngjae Yu. Prior to this, I received my MS in Computer Engineering from Seoul National University, advised by Gunhee Kim. During my PhD, I did internships at Naver and LG AI Research.
My research goal is understanding how knowledge emerges and develops by replicating these processes in machines. Arguably, the most fundamental source of human knowledge lies in perception, which forms the basis of how we interpret and learn from the world around us. To reflect this, my current focus is on multimodal artificial intelligence, where I explore challenges that arise from introducing multimodal inputs and outputs to large language models (LLMs).
Currently seeking research internship opportunities anytime through the end of 2025! 🙌
News
- Sep 2024 - 3 papers have been accepted to EMNLP 2024. See you at Miami!
Publications (* equal contribution)
-
Towards Visual Text Design Transfer Across Languages
Jiwan Chung*, Yejin Choi*, Sumin Shim, Giyeong Oh, Youngjae Yu
NeurIPS Datasets and Benchmarks 2024
(Paper link will be updated shortly.) -
Selective Vision is the Challenge for Visual Reasoning: A Benchmark for Visual Argument Understanding
Jiwan Chung*, Sungjae Lee*, Minseo Kim, Seungju Han, Ashkan Yousefpour, Jack Hessel, Youngjae Yu
EMNLP 2024
github | dataset -
Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!
Jiwan Chung, Seungwon Lim, Jaehyun Jeon, Seungbeen Lee, Youngjae Yu
EMNLP 2024
github | dataset -
Language models as compilers: Simulating pseudocode execution improves algorithmic reasoning in language models
Hyungjoo Chae, Yeonghyeon Kim, Seungone Kim, Kai Tzu-iunn Ong, Beong-woo Kwak, Moohyeon Kim, Seonghwan Kim, Taeyoon Kwon, Jiwan Chung, Youngjae Yu, Jinyoung Yeo
EMNLP 2024
-
CANVAS: Commonsense-Aware Navigation System for Intuitive Human-Robot Interaction
Suhwan Choi, Yongjun Cho, Minchan Kim, Jaeyoon Jung, Myunchul Joe, Yubeen Park, Minseo Kim, Sungwoong Kim, Sungjae Lee, Hwiseong Park, Jiwan Chung, Youngjae Yu
arXiv 2024
-
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics
Seungbeen Lee, Seungwon Lim, Seungju Han, Giyeong Oh, Hyungjoo Chae, Jiwan Chung, Minju Kim, Beong-woo Kwak, Yeonsoo Lee, Dongha Lee, Jinyoung Yeo, Youngjae Yu
arXiv 2024
-
HyperCLOVA X Technical Report
HyperCLOVA X Team
arXiv 2024
-
Long Story Short: a Summarize-then-Search Method for Long Video Question Answering
Jiwan Chung, Youngjae Yu
BMVC 2023
github -
VLIS: Unimodal Language Models Guide Multimodal Language Generation
Jiwan Chung, Youngjae Yu
EMNLP 2023
github -
Reading books is great, but not if you are driving! visually grounded reasoning about defeasible commonsense norms
Seungju Han, Junhyeok Kim, Jack Hessel, Liwei Jiang, Jiwan Chung, Yejin Son, Yejin Choi, Youngjae Yu
EMNLP 2023
-
Fusing pre-trained language models with multimodal prompts through reinforcement learning
Jiwan Chung*, Youngjae Yu*, Heeseung Yun, Jack Hessel, Jae Sung Park, Ximing Lu, Rowan Zellers, Prithviraj Ammanabrolu, Ronan Le Bras, Gunhee Kim, Yejin Choi
CVPR 2023
github -
ACAV100m: Automatic curation of large-scale datasets for audio-visual video representation learning
Jiwan Chung*, Sangho Lee*, Youngjae Yu, Gunhee Kim, Thomas Breuel, Gal Chechik, Yale Song
ICCV 2021
webpage | github -
Transitional adaptation of pretrained models for visual storytelling
Jiwan Chung*, Youngjae Yu*, Heeseung Yun, Jongseok Kim, Gunhee Kim
CVPR 2021
github -
Character grounding and re-identification in story of videos and text descriptions
Youngjae Yu, Jongseok Kim, Heeseung Yun, Jiwan Chung, Gunhee Kim
ECCV 2020
Education
- Ph.D. in Artificial Intelligence, 2026 (expected) , Yonsei University
- M.S. in Computer Engineering, 2023 , Seoul National University
- B.S. in Computer Science, 2019 , Yonsei University