Ziwei Wang

Ziwei Wang is currently an assistant professor in School of Electrical and Electronic Engineering, Nanyang Technological University. Before joining NTU, he was a postdoc fellow in Robotics Institute, Carnegie Mellon University, with Prof. Changliu Liu. He received the Ph.D and the B.S degrees from the Department of Automation, Tsinghua University in 2023 and the Department of Physics, Tsinghua University in 2018 respectively. His research goal is to design foundation models (FMs) for robotics including grounding FMs to the physical scene and deploying FMs in resource-limited robots. He has published over 20 scientific papers in TPAMI, IJCV, RAL, CVPR, ICCV, ECCV, NeurIPS, IROS and ICRA. He serves as a regular reviewer member for a variety of conferences and journals.

I am looking for self-motivated PhD/Postdoc/Master/research assistant. If you are interested in joining our lab, please drop me an email at ziwei.wang@ntu.edu.sg with your CV.

Email / CV / Google Scholar / Github / Twitter / Zhihu

News

2024-09: I am invited to give a talk in Recent Advances in Video Content Understanding and Generation Tutorial at ECCV 2024.

2024-09: One paper is accepted to NeurIPS 2024.

2024-07: Two papers are accepted to ECCV 2024.

2024-06: Our Perception and embodied INtElligence (PINE) Lab is established.

2024-05: One paper is accepted to RAL.

2024-05: One paper is accepted to IJCV.

2024-02: Three papers are accepted to CVPR 2024.

2024-02: One paper is accepted to TIP.

2023-10: One paper is accepted to TPAMI.

2023-09: One paper is accepted to NeurIPS 2023.

Recent Works

EmbodiedSAM: Online Segment Any 3D Thing in Real Time
Xiuwei Xu, Huangxing Chen, Linqing Zhao, Ziwei Wang, Jie Zhou, Jiwen Lu
Arxiv, 2024
[PDF] [Website] [Code] [Video]

We propose an online Segment Anything Model for real-time 3D instance segmentation to process the streaming RGB-D videos acquired in diverse embodied tasks.

Embodied Instruction Following in Unknown Environments
Zhenyu Wu, Ziwei Wang, Xiuwei Xu, Jiwen Lu, Haibin Yan
Arxiv, 2024
[PDF] [Website] [Code and Dataset (coming soon)]

Our embodied agent efficiently explores the unknown environment to generate feasible plans with existing objects to accomplish abstract instructions. which can complete complex human instructions such as making breakfast, tidying bedrooms and cleaning bathrooms in house-level scenes.

ManiCM: Real-time 3D Diffusion Policy via Consistency Model for Robotic Manipulation
Guanxing Lu, Zifeng Gao, Tianxing Chen, Wenxun Dai, Ziwei Wang, Yansong Tang
Arxiv, 2024
[PDF] [Website] [Code]

We propose a real-time robotic manipulation model that imposes the consistency constraint to the diffusion process, so that the model can generate robot actions in only one-step inference with high success rate. We achieve 16ms average runtime for 3D diffusion policy generation.

Selected Publications

* means equal contribution
† means corresponding author

ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
Guanxing Lu, Shiyi Zhang, Ziwei Wang^†, Changliu Liu, Jiwen Lu, Yansong Tang
European Conference on Computer Vision (ECCV), 2024
[PDF] [Code] [Website]

	3D Small Object Detection with Dynamic Spatial Pruning Xiuwei Xu, Zhihao Sun, Ziwei Wang, Hongming Liu, Jie Zhou, Jiwen Lu European Conference on Computer Vision (ECCV), 2024 [PDF] [Code] [Website]
	StableLego: Stability Analysis of Block Stacking Assembly Ruixuan Liu, Kangle Deng, Ziwei Wang, Changliu Liu IEEE Robotics and Automation Letters (RAL), 2024 [PDF] [Code and Dataset]
	Learning Generalizable Mixed-Precision Quantization via Attribution Imitation Ziwei Wang, Han Xiao, Jie Zhou, Jiwen Lu International Journal of Computer Vision (IJCV), 2024. [PDF] [Code]
	Towards Accurate Data-free Quantization for Diffusion Models Changyuan Wang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, Jiwen Lu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR, Highlight), 2024. [PDF] [Supp] [Code]
	Memory-based Adapters for Online 3D Scene Perception Xiuwei Xu, Chong Xia, Ziwei Wang, Linqing Zhao, Yueqi Duan, Jie Zhou, Jiwen Lu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Code] [Website]
	MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory Yinan Liang, Ziwei Wang, Xiuwei Xu, Yansong Tang, Jie Zhou, Jiwen Lu Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023. [PDF] [Supp] [Code]
	Learning Accurate Performance Predictors for Ultrafast Automated Model Compression Ziwei Wang, Jiwen Lu, Han Xiao, Shengyu Liu, Jie Zhou International Journal of Computer Vision (IJCV), 2023. [PDF] [Code]
	Category-level Shape Estimation for Densely Cluttered Objects Zhenyu Wu, Ziwei Wang, Jiwen Lu, Haibin Yan IEEE International Conference on Robotics and Automation (ICRA), 2023. [PDF] [Code]
	Quantformer: Learning Extremely Low-precision Vision Transformers Ziwei Wang, Changyuan Wang, Xiuwei Xu, Jie Zhou, Jiwen Lu IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2022. [PDF] [Supp] [Code]
	Planning Irregular Object Packing via Hierarchical Reinforcement Learning Sichao Huang, Ziwei Wang, Jie Zhou, Jiwen Lu IEEE Robotics and Automation Letters (RAL), 2022 [PDF] [Robot Demo] [Simulation Demo] [Code]
	Smart Explorer: Recognizing Objects in Dense Clutter via Interactive Exploration Zhenyu Wu, Ziwei Wang, Zibu Wei, Yi Wei, Haibin Yan IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022. [PDF] [Demo] [Code]
	GE-Grasp: Efficeint Target Oriented Grasping in Dense Clutter Zhan Liu, Ziwei Wang, Sichao Huang, Jie Zhou, Jiwen Lu IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022. [PDF] [Demo] [Code]
	Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search Han Xiao, Ziwei Wang, Zheng Zhu, Jie Zhou, Jiwen Lu IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. [PDF] [Supplement] [Code]
	Learning Efficient Binarized Object Detectors with Information Compression Ziwei Wang, Jiwen Lu, Ziyi Wu, Jie Zhou IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2022. [PDF] [Supplement] [Code]
	Generalizable Mixed-Precision Quantization via Attribution Rank Preservation Ziwei Wang, Han Xiao, Jiwen Lu, Jie Zhou IEEE International Conference on Computer Vision (ICCV), 2021. [PDF] [Supplement] [Code]
	BiDet: An Efficient Binarized Object Detector Ziwei Wang, Ziyi Wu, Jiwen Lu, Jie Zhou IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020 [PDF] [Code]
	Learning Channel-wise Interactions for Binary Convolutional Neural Networks Ziwei Wang, Jiwen Lu, Chenxin Tao, Jie Zhou IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019 [PDF] [Code]