Next Generation Infrastructure for Foundation Models
| Reference number | |
| Coordinator | Lindholmen Science Park AB - AI Sweden |
| Funding from Vinnova | SEK 10 000 000 |
| Project duration | May 2023 - October 2025 |
| Status | Ongoing |
| Venture | Advanced digitalization - Enabling technologies |
| Call | Advanced and innovative digitalization 2023 - call one |
Purpose and goal
The project aims to develop the next generation infrastructure to train and implement foundation models such as BERT, GPT-3, Dall-E and CLIP. The planned infrastructure will be based on a decentralized learning architecture to work around many of the limitations found in the current centralized infrastructure.
Expected effects and result
The proposed infrastructure will have higher performance, increase collaboration between stakeholders, lead to a hardware-agnostic solution, have better privacy and security, as well as reduced costs and increased availability. The project will have a positive impact on the global foundation model ecosystem and has high potential for commercialization.
Planned approach and implementation
The project will create a prototype and demonstrate the next generation infrastructure for foundation models, controlled by a central server at AI Sweden. The project has two work packages where WP1 will carry out benchmarks on the Swedish GPT-SW3 model and WP2 will develop the decentralized architecture. For dissemination there is also a work package, as spreading knowledge is also an objective.