Combining AI models using depth upscaling for extra performance
A new technique in AI model development called “depth upscaling,” has been used to create the Solar 10.7 B model. This model, despite having only 11 billion parameters, outperforms models with up to 30 billion parameters, even surpassing the recent Mixtral 8X7B model. Depth upscaling involves merging multiple AI models by concatenating different layers from … Read more