Ziyang Chen

About

I am a third-year Ph.D. student in Computer Vision at the University of Michigan, advised by Andrew Owens. Before that, I was a Master's student in UMich ECE (2021) and I did my bachelor's degree at UM-SJTU Joint Institute from Shanghai Jiao Tong University (2019).
My primary research interest lies in Computer Vision and Deep Learning. I have spent wonderful years in computer vision group @ Michigan. During my Ph.D. I have been fortunate to intern at Codec Avatars Lab, Meta with Israel D. Gebru and Alexander Richard. I am honored to collaborate with David Fouhey, Justin Salamon and Bryan Russell.

[2024/05] I am co-organizing Sight and Sound workshop at CVPR 2024 this year!
[2024/05] Check out our latest work Images that Sound: a visual spectrogram art made by multimodal compositional generation!
[2024/05] I joined Adobe Research as a research scientist intern this summer, working with Prem Seetharaman and Justin Salamon!
[2024/02] Two first-author papers accepted at CVPR 2024, one with highlight. See you in Seattle!
[2023/07] Our work Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation is accepted at ICCV 2023!

(* indicates equal contribution)

Images that Sound: Composing Images and Sounds on a Single Canvas
Ziyang Chen, Daniel Geng, Andrew Owens
arXiv 2024
CVPR 2024 AI Art Gallery
project page · paper · code · bibtex

Real Acoustic Fields: An Audio-Visual Room Acoustics Dataset and Benchmark
Ziyang Chen, Israel D. Gebru, Christian Richardt, Anurag Kumar, William Laney, Andrew Owens, Alexander Richard
CVPR 2024 (Highlight -- 2.8% accept rate)
project page · paper · dataset · bibtex

Binding Touch to Everything: Learning Unified Multimodal Tactile Representations
Fengyu Yang*, Chao Feng*, Ziyang Chen*, ...... , Andrew Owens, Alex Wong
CVPR 2024
project page · paper · code · bibtex

GPS-to-3D: Lifting Tourist Photos to 3D Using 2D GPS-Conditioned Diffusion
Chao Feng, Ziyang Chen, Aleksander Holynski, Alexei A. Efros, Andrew Owens
In submission

Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation
Ziyang Chen, Shengyi Qian, Andrew Owens
ICCV 2023
project page · paper · code · bibtex

Conditional Generation of Audio from Video via Foley Analogies
Yuexi Du, Ziyang Chen, Justin Salamon, Bryan Russell, Andrew Owens
CVPR 2023
project page · paper · code · bibtex

Self-Supervised Video Forensics by Audio-Visual Anomaly Detection
Chao Feng, Ziyang Chen, Andrew Owens
CVPR 2023 (Highlight -- 2.5% accept rate)
project page · paper · code · bibtex

Sound Localization by Self-Supervised Time Delay Estimation
Ziyang Chen, David F. Fouhey, Andrew Owens
ECCV 2022
project page · paper · slides · code · bibtex

Mix and Localize: Localizing Sound Sources in Mixtures
Xixi Hu*, Ziyang Chen*, Andrew Owens
CVPR 2022
project page · paper · slides · code · bibtex

Structure from Silence: Learning Scene Structure from Ambient Sound
Ziyang Chen*, Xixi Hu*, Andrew Owens
CoRL 2021 (Oral)
project page · paper · slides · code · bibtex