Fangyun Wei
I am a Principal Researcher at Microsoft Research Asia. My research spans visual generative models, multimodal learning, embodied intelligence, and data-efficient learning.
I received my Ph.D. from the School of Computer Science, Faculty of Engineering, at the University of Sydney, under the supervision of Prof. Chang Xu. I earned my master's degree in Intelligence Science and Technology from the School of EECS at Peking University and my bachelor's degree in Electronic Information Engineering from Shandong University.
I serve as an Area Chair for ICML, ICLR, NeurIPS, and AAAI.
Publication
2026
- Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative ModelsECCV
- Lens: Rethinking Training Efficiency for Foundational Text-to-Image ModelsarXiv
- InsightTok: Improving Text and Face Fidelity in Discrete Tokenization for Autoregressive Image GenerationECCVW
- MobileManiBench: Simplifying Model Verification for Mobile ManipulationECCV
- CustomX: Unified Character, Action, and Scene Customization in Video World ModelsECCV
- Spatia: Video Generation with Updatable Spatial MemoryCVPR
2025
- From Virtual Games to Real-World PlayarXiv
- Fast Autoregressive Models for Continuous Latent GenerationarXiv
- Modeling Multi-Modal Cross-Interaction for Multi-Label Few-Shot Image Classification Based on Local Feature SelectionTOMM
- Low-Shot Video Object SegmentationTPAMI
- EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time TestNeurIPS
- VideoVLA: Video Generators Can Be Generalizable Robot ManipulatorsNeurIPS
- Improved Noise Schedule for Diffusion TrainingICCV
- Minimizing Labeled, Maximizing Unlabeled: An Image-Driven Approach for Video Instance SegmentationCVPR
- UniGraspTransformer: Simplified Policy Distillation for Scalable Dexterous Robotic GraspingCVPR
- Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal ModelsCVPRW
2024
- A Large-Scale Human-Centric Benchmark for Referring Expression Comprehension in the LMM EraNeurIPS
- CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic ManipulationarXiv
- LoRASC: Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded LearningEMNLP
- Towards Online Continuous Sign Language Recognition and TranslationEMNLP
- EAGLE-2: Faster Inference of Language Models with Dynamic Draft TreesEMNLP
- Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%NeurIPS
- A Hong Kong Sign Language Corpus Collected from Sign-interpreted TV NewsLREC-COLING
- Rethinking Generative Large Language Model Evaluation for Semantic ComprehensionICML
- Beyond Text: Frozen Large Language Models in Visual Signal ComprehensionCVPR
- AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API CallsICML
- EAGLE: Speculative Sampling Requires Rethinking Feature UncertaintyICML
- A Simple Baseline for Spoken Language to Sign Language Translation with 3D AvatarsECCV
- Hybrid Proposal Refiner: Revisiting DETR Series from the Faster R-CNN PerspectiveCVPR
- RAIN: Your Language Models Can Align Themselves without FinetuningICLR
Earlier publications 2019–2023
2023
- Exploring Non-additive Randomness on ViT against Query-Based Black-Box AttacksBMVC
- AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image CollectionsSIGGRAPH Asia
- Improving Continuous Sign Language Recognition with Cross-Lingual SignsICCV
- CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive LearningCVPR
- Two-shot Video Object SegmentationCVPR
- Natural Language-Assisted Sign Language RecognitionCVPR
- DeepMIM: Deep Supervision for Masked Image ModelingWACV
- Side Adapter Network for Open-Vocabulary Semantic SegmentationCVPR
- ADPL: Adaptive Dual Path Learning for Domain Adaptation of Semantic SegmentationTPAMI
- TinyMIM: An Empirical Study of Distilling MIM Pre-trained ModelsCVPR
- Iterative Proposal Refinement for Weakly-Supervised Video GroundingCVPR
2022
- Attentive Mask CLIPICCV
- Two-Stream Network for Sign Language Recognition and TranslationNeurIPS
- A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-Language ModelECCV
- AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video AvatarsNeurIPS
- Conditional DETR V2: Efficient Detection Transformer with Box QueriesarXiv
- Boosting Zero-shot Learning via Contrastive Optimization of Attribute RepresentationsTNNLS
- Unsupervised Prompt Learning for Vision-Language ModelsarXiv
- Frame-wise Action Representations for Long Videos via Sequence Contrastive LearningCVPR
- Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language ModelCVPR
- A Simple Multi-Modality Transfer Learning Baseline for Sign Language TranslationCVPR
- Cross-Model Pseudo-Labeling for Semi-Supervised Action RecognitionCVPR
2021
- Towards Tokenized Human Dynamics RepresentationarXiv
- Bootstrap Your Object Detector via Mixed TrainingNeurIPS
- Semi-Supervised Semantic Segmentation via Adaptive Equalization LearningNeurIPS
- ADNet: Leveraging Error-Bias Towards Normal Direction in Face AlignmentICCV
- Dual Path Learning for Domain Adaptation of Semantic SegmentationICCV
- End-to-End Semi-Supervised Object Detection with Soft TeacherICCV
- Aligning Pretraining for Detection via Object-Level Contrastive LearningNeurIPS
- High-Fidelity and Arbitrary Face EditingCVPR
2020
- Global Context NetworksTPAMI
- Point-Set Anchors for Object Detection, Instance Segmentation and Pose EstimationECCV
- RelationNet++: Bridging Visual Representations for Object Detection via Transformer DecoderNeurIPS
- Restoring Negative Information in Few-Shot Object DetectionNeurIPS
- Group Sampling for Scale Invariant Face DetectionTPAMI
- Design and Interpretation of Universal Adversarial Patches in Face DetectionECCV