Lin Song

Hello! My name is Lin Song (宋林). I am currently with Joy Future Academy, JD. Previously, I was a Research Scientist at Tencent AI Lab in Shenzhen, China. In July 2022, I received my PhD from the College of Artificial Intelligence at Xi’an Jiaotong University, advised by Jian Sun and Hongbin Sun. My research focuses on multimodal foundation models, computer vision, image and video generation and editing, and machine learning.
Google Scholar/GitHub/Email

News

[08/2026] We released JoyAI-Video-Edit, a real-time, open-ended video editing model with an online demo and open-source checkpoints.
[05/2026] We released JoyAI-Image, a unified multimodal foundation model for visual understanding, generation, and editing.
[09/2025] MindOmni was accepted by NeurIPS 2025.
[06/2025] LoRA-Gen and HaploVL were accepted by ICML 2025.
[09/2024] MambaTree was accepted by NeurIPS 2024 as a Spotlight paper.
[02/2024] Three papers were accepted by CVPR 2024.
[02/2024] We proposed YOLO-World, a real-time model for detecting everything.
[10/2023] Four papers were accepted by ICCV2023, NeurIPS2023 and ICLR2024
[05/2023] We introduce and release the code of GPT4Tools and BoxSnake.
[01/2023] One paper was accepted by ICLR 2023.
[04/2022] We won the 2rd Place on LVIS Challenges (Workshop at ICCV 2021).
[09/2021] One paper was accepted by NeurIPS 2021.
[06/2021] We won the 1st Place on Streaming Perception Challenges (Workshop on Autonomous Driving at CVPR 2021).

Selected Publications

Conference

* indicates equal contribution
^ indicates corresponding author

	JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion Project Leader Yicheng Xiao, Wenxun Dai, Xinran Qin, Lin Song, Maoquan Zhang, Hang Xu, Yukang Chen, et al. arXiv preprint, 2026* [paper][code][model][demo]
	JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation Project Leader Lin Song, Wenbo Li, Guoqing Ma, Wei Tang, Bo Wang, Yuan Zhang, Yijun Yang, et al. Technical Report, 2026* [paper][code][model][demo]
	From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation Cheng Cheng, Lin Song, Di An, Yicheng Xiao, Xuchong Zhang, Hongbin Sun, Ying Shan International Conference on Learning Representations (ICLR), 2026 [paper]
	HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation Yicheng Xiao, Lin Song^*, Rui Yang, Cheng Cheng, Zunnan Xu, Zhaoyang Zhang, Yixiao Ge, Xiu Li^, Ying Shan arXiv preprint, 2025* [paper][code]
	MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO Yicheng Xiao, Lin Song^, Yukang Chen, Yingmin Luo, Yuxin Chen, Yukang Gan, Wei Huang, Xiu Li, Xiaojuan Qi, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2025 [paper][code][project]
	LoRA-Gen: Specializing Large Language Model via Online LoRA Generation Yicheng Xiao, Lin Song, Rui Yang, Cheng Cheng, Yixiao Ge, Xiu Li^, Ying Shan International Conference on Machine Learning (ICML), 2025 [paper]
	HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding Rui Yang, Lin Song^, Yicheng Xiao, Runhui Huang, Yixiao Ge, Ying Shan, Hengshuang Zhao^ International Conference on Machine Learning (ICML), 2025 [paper][code]
	MambaTree: Tree Topology is All You Need in State Space Model Yicheng Xiao, Lin Song^, Shaoli Huang, Jiangshan Wang, Siyu Song, Yixiao Ge, Xiu Li^, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2024 (Spotlight)** [paper][code]
	YOLO-World: Real-time Open-vocabulary Object Detection Tianheng Cheng, Lin Song^*, Yixiao Ge, Wenyu Liu, Xinggang Wang, Ying Shan IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024* [paper][code]
	LoRA-Sparse: Low-Rank Approximation for Sparse Large Language Models *Lin Song^*, Yukang Chen, Shuai Yang, Xiaohan Ding, Yixiao Ge, Ying-Cong Chen, Ying Shan IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024
	Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng, Lin Song, Ruoyi Xue, Hang Wang, Hongbin Sun, Yixiao Ge, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2023 [paper][code]
	GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction Rui Yang, Lin Song^*, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2023* [paper][code]
	BoxSnake: Polygonal Instance Segmentation with Box Supervision Rui Yang, Lin Song^*, Yixiao Ge, Xiu Li^ International Conference on Computer Vision (ICCV), 2023* [paper][code]
	DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detection Jinrong Yang, Lin Song, Songtao Liu, Zeming Li, Xiaoping Li, Hongbin Sun, Jian Sun, Nanning Zheng International Conference on Learning Representations (ICLR), 2023 [paper][code]
	Dynamic Grained Encoder for Vision Transformer Lin Song, Songyang Zhang, Songtao Liu, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2021 [paper] [code]
	End-to-End Object Detection with Fully Convolutional Network Jianfeng Wang, Lin Song, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 [paper] [code]
	Fine-Grained Dynamic Head for Object Detection Lin Song, Yanwei Li, Zhengkai Jiang, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2020 [paper] [code] [slides]
	Rethinking Learnable Tree Filter for Generic Feature Transform Lin Song, Yanwei Li, Zhengkai Jiang, Zeming Li, Xiangyu Zhang, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2020 [paper] [code] [slides]
	Learning Dynamic Routing for Semantic Segmentation Yanwei Li, Lin Song, Yukang Chen, Zeming Li, Xiangyu Zhang, Xingang Wang, Jian Sun IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020 (Oral) [paper] [code] [slides]
	Learnable Tree Filter for Structure-preserving Feature Transform Lin Song, Yanwei Li, Zeming Li, Gang Yu, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2019 [paper] [code]
	TACNet: Transition-aware Context Network for Spatio-temporal Action Detection Lin Song, Shiwei Zhang, Gang Yu, Hongbin Sun IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 [paper]

Journal

	GLNet: Global Local Network for Weakly Supervised Action Localization Shiwei Zhang, Lin Song, Changxin Gao, Nong Sang IEEE Transactions on Multimedia (TMM) [paper]
	NIPM-sWMF: Toward Efficient FPGA Design for High-Definition Large-Disparity Stereo Matching Xuchong Zhang, Hongbin Sun, Shiqiang Chen, Lin Song, Nanning Zheng IEEE Transactions on Circuits and Systems for Video Technology (TCSVT) [paper]

Experience

Joy Future Academy, JD
Areas: Multimodal Foundation Models, Image and Video Generation
Researcher, Present

Tencent AI Lab (Vision Computing Center)
Areas: Visual Perception and Multimodal Learning
Research Scientist, 2022.8 - 2025

Megvii (Face++)
Areas: Object Detection, Image Segmentation and Action Localization
Full Time Research Intern, 2019.5 - 2022.6
Research Intern, 2017.11 - 2019.5

School of Microelectronics, Xidian University
Areas: Integrated Circuit
Research Intern, 2017.07 - 2017.09

Activity

Reviewer: CVPR, ICCV, NeurIPS, ICLR, AAAI, TPAMI, IJCV, TMM, TCSVT
Seminar talk: "Rethinking Learnable Tree Filter for Generic Feature Transform", 2020 [slides]

Award

LVIS Challenge (Workshop at ICCV 2021), 2022, 2nd Place
Streaming Perception Challenges (Workshop on Autonomous Driving at CVPR 2021), 2021, 1st
ActivityNet-AVA, 2018, 1st
OpenImage Object Detection, 2018, GOLD (solo)
National Undergraduate Electronics Design Contest, 2015, 1st
National Undergraduate Intelligent Car Competition, 2014, 1st

	JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion Project Leader Yicheng Xiao, Wenxun Dai, Xinran Qin, Lin Song, Maoquan Zhang, Hang Xu, Yukang Chen, et al. arXiv preprint, 2026* [paper][code][model][demo]
	JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation Project Leader Lin Song, Wenbo Li, Guoqing Ma, Wei Tang, Bo Wang, Yuan Zhang, Yijun Yang, et al. Technical Report, 2026* [paper][code][model][demo]
	From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation Cheng Cheng, Lin Song, Di An, Yicheng Xiao, Xuchong Zhang, Hongbin Sun, Ying Shan International Conference on Learning Representations (ICLR), 2026 [paper]
	HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation Yicheng Xiao, Lin Song^*, Rui Yang, Cheng Cheng, Zunnan Xu, Zhaoyang Zhang, Yixiao Ge, Xiu Li^, Ying Shan arXiv preprint, 2025* [paper][code]
	MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO Yicheng Xiao, Lin Song^, Yukang Chen, Yingmin Luo, Yuxin Chen, Yukang Gan, Wei Huang, Xiu Li, Xiaojuan Qi, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2025 [paper][code][project]
	LoRA-Gen: Specializing Large Language Model via Online LoRA Generation Yicheng Xiao, Lin Song, Rui Yang, Cheng Cheng, Yixiao Ge, Xiu Li^, Ying Shan International Conference on Machine Learning (ICML), 2025 [paper]
	HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding Rui Yang, Lin Song^, Yicheng Xiao, Runhui Huang, Yixiao Ge, Ying Shan, Hengshuang Zhao^ International Conference on Machine Learning (ICML), 2025 [paper][code]
	MambaTree: Tree Topology is All You Need in State Space Model Yicheng Xiao, Lin Song^, Shaoli Huang, Jiangshan Wang, Siyu Song, Yixiao Ge, Xiu Li^, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2024 (Spotlight)** [paper][code]
	YOLO-World: Real-time Open-vocabulary Object Detection Tianheng Cheng, Lin Song^*, Yixiao Ge, Wenyu Liu, Xinggang Wang, Ying Shan IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024* [paper][code]
	LoRA-Sparse: Low-Rank Approximation for Sparse Large Language Models *Lin Song^*, Yukang Chen, Shuai Yang, Xiaohan Ding, Yixiao Ge, Ying-Cong Chen, Ying Shan IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024
	Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng, Lin Song, Ruoyi Xue, Hang Wang, Hongbin Sun, Yixiao Ge, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2023 [paper][code]
	GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction Rui Yang, Lin Song^*, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, Ying Shan Conference on Neural Information Processing Systems (NeurIPS), 2023* [paper][code]
	BoxSnake: Polygonal Instance Segmentation with Box Supervision Rui Yang, Lin Song^*, Yixiao Ge, Xiu Li^ International Conference on Computer Vision (ICCV), 2023* [paper][code]
	DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detection Jinrong Yang, Lin Song, Songtao Liu, Zeming Li, Xiaoping Li, Hongbin Sun, Jian Sun, Nanning Zheng International Conference on Learning Representations (ICLR), 2023 [paper][code]
	Dynamic Grained Encoder for Vision Transformer Lin Song, Songyang Zhang, Songtao Liu, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2021 [paper] [code]
	End-to-End Object Detection with Fully Convolutional Network Jianfeng Wang, Lin Song, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021 [paper] [code]
	Fine-Grained Dynamic Head for Object Detection Lin Song, Yanwei Li, Zhengkai Jiang, Zeming Li, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2020 [paper] [code] [slides]
	Rethinking Learnable Tree Filter for Generic Feature Transform Lin Song, Yanwei Li, Zhengkai Jiang, Zeming Li, Xiangyu Zhang, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2020 [paper] [code] [slides]
	Learning Dynamic Routing for Semantic Segmentation Yanwei Li, Lin Song, Yukang Chen, Zeming Li, Xiangyu Zhang, Xingang Wang, Jian Sun IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020 (Oral) [paper] [code] [slides]
	Learnable Tree Filter for Structure-preserving Feature Transform Lin Song, Yanwei Li, Zeming Li, Gang Yu, Hongbin Sun, Jian Sun, Nanning Zheng Conference on Neural Information Processing Systems (NeurIPS), 2019 [paper] [code]
	TACNet: Transition-aware Context Network for Spatio-temporal Action Detection Lin Song, Shiwei Zhang, Gang Yu, Hongbin Sun IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019 [paper]

News

Selected Publications

Conference

JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion

JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation

HaploOmni: Unified Single Transformer for Multimodal Video Understanding and Generation

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation

HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding

MambaTree: Tree Topology is All You Need in State Space Model

YOLO-World: Real-time Open-vocabulary Object Detection

LoRA-Sparse: Low-Rank Approximation for Sparse Large Language Models

Meta-Adapter: An Online Few-shot Learner for Vision-Language Model

GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction

BoxSnake: Polygonal Instance Segmentation with Box Supervision

DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detection

Dynamic Grained Encoder for Vision Transformer

End-to-End Object Detection with Fully Convolutional Network

Fine-Grained Dynamic Head for Object Detection

Rethinking Learnable Tree Filter for Generic Feature Transform

Learning Dynamic Routing for Semantic Segmentation

Learnable Tree Filter for Structure-preserving Feature Transform

TACNet: Transition-aware Context Network for Spatio-temporal Action Detection

Journal

GLNet: Global Local Network for Weakly Supervised Action Localization

NIPM-sWMF: Toward Efficient FPGA Design for High-Definition Large-Disparity Stereo Matching

Experience

Activity

Award