Ziyang Chen

About

I am a fourth-year Ph.D. student in Computer Vision at the University of Michigan, advised by Andrew Owens. Before that, I was a Master's student in UMich ECE (2021) and I did my bachelor's degree at UM-SJTU Joint Institute from Shanghai Jiao Tong University (2019).
My primary research interest lies in Computer Vision and Deep Learning. I have spent wonderful years in computer vision group @ Michigan. During my Ph.D. I have been fortunate to intern at Meta Codec Avatars Lab and SODA (Sound Design AI) Group of Adobe Research. I am honored to collaborate with Prof. David Fouhey and Prof. Alex Wong.

I'm currently seeking full-time opportunities starting in Summer 2025. Please feel free to reach out if you have any openings!

[2024/11] Check out our new work MultiFoley for creative Foley sound generation!
[2024/09] Our work Images that Sound: Composing Images and Sounds on a Single Canvas is accepted at NeurIPS 2024!
[2024/05] I am co-organizing Sight and Sound workshop at CVPR 2024 this year!
[2024/05] Check out our latest work Images that Sound: a visual spectrogram art made by multimodal compositional generation!
[2024/05] I joined Adobe Research as a research scientist intern this summer, working with Prem Seetharaman and Justin Salamon!
[2024/02] Two first-author papers accepted at CVPR 2024, one with highlight. See you in Seattle!

(* indicates equal contribution)

Video-Guided Foley Sound Generation with Multimodal Controls
Ziyang Chen, Prem Seetharaman, Bryan Russell, Oriol Nieto, David Bourgin, Andrew Owens, Justin Salamon
arXiv 2024
project page · paper · bibtex

Images that Sound: Composing Images and Sounds on a Single Canvas
Ziyang Chen, Daniel Geng, Andrew Owens
NeurIPS 2024
CVPR 2024 AI Art Gallery
project page · paper · code · bibtex

Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark
Ziyang Chen, Israel D. Gebru, Christian Richardt, Anurag Kumar, William Laney, Andrew Owens, Alexander Richard
CVPR 2024 (Highlight -- 2.8% accept rate)
project page · paper · dataset · bibtex

Binding Touch to Everything: Learning Unified Multimodal Tactile Representations
Fengyu Yang*, Chao Feng*, Ziyang Chen*, ...... , Andrew Owens, Alex Wong
CVPR 2024
project page · paper · code · bibtex

Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation
Ziyang Chen, Shengyi Qian, Andrew Owens
ICCV 2023
project page · paper · code · bibtex

Conditional Generation of Audio from Video via Foley Analogies
Yuexi Du, Ziyang Chen, Justin Salamon, Bryan Russell, Andrew Owens
CVPR 2023
project page · paper · code · bibtex

Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
Chao Feng, Ziyang Chen, Andrew Owens
CVPR 2023 (Highlight -- 2.5% accept rate)
project page · paper · code · bibtex

Sound Localization by Self-Supervised Time Delay Estimation
Ziyang Chen, David F. Fouhey, Andrew Owens
ECCV 2022
project page · paper · slides · code · bibtex

Mix and Localize: Localizing Sound Sources in Mixtures
Xixi Hu*, Ziyang Chen*, Andrew Owens
CVPR 2022
project page · paper · slides · code · bibtex

Structure from Silence: Learning Scene Structure from Ambient Sound
Ziyang Chen*, Xixi Hu*, Andrew Owens
CoRL 2021 (Oral)
project page · paper · slides · code · bibtex