MELD: Multi-Dimensional Collaborative Deployment Mechanism of MoE-based Edge LLMs for 6G Ubiquitous Intelligence

Purpose and goal

Edge deployment of Large Language Models (LLMs) plays a vital role in ensuring low latency, reducing communication overhead, and enhancing privacy, bridging the gap unaddressed by cloud and on-device LLMs. However, edge LLMs face daunting challenges due to resource constraints and highly dynamic, heterogeneous environments. This project establishes the theoretical foundations and practical methodologies for realizing high-performance and ubiquitous edge LLMs.

Expected effects and result

This project establishes the theoretical foundations and practical methodologies for realizing high-performance and ubiquitous edge LLMs. High qualiity papers will be published and related sofaware will be released.

Planned approach and implementation

To address these challenges, advanced strategies are proposed in this project. Inter-server and intra-server collaborative deployment methods partition models based on expert activation paths and similarities, ensuring efficient distribution across edge servers and optimal expert scheduling within each server.