the near future,NVIDIA Corporation andMetaJointly launched by the companyLlama-3.1-Minitron 4B AISmall model.This model isLlama 3On the basis of this, significant updates have been made,By combining customer data withLlama 3.1 405BAnd NVIDIANemotronModels are created for specific fields“Super model”.

Previously, MetaThe company istwo thousand and twenty-fouryearsevenmonthtwenty-fourThe day released its strongest open sourceAIModelLlama 3.1This model has three different versions（8BThe70Band405B）Among them405BVersion includesfour thousand and fiftyWith billions of parameters, it is one of the largest models in parameter scale in recent years.Llama 3.1stayLlama 3On the basis of this, significant updates have been made, mainly used to drive chatbots, and equipped with abilities such as multilingual dialogue, writing high-quality computer code, and solving complex mathematical problems.

And Llama-3.1-Minitron 4B AIThe model is a collaboration between NVIDIA CorporationMetaCompany, based onLlama-3.1The series of models are carefully crafted through cutting-edge model pruning and distillation techniques. Pruning（Pruning）andPruning distillation（Distillation）allIt is a model compression technique,This model can not only be used in application scenarios such as chatbots, but also perform various tasks such as programming, answering mathematical questions, and generating images.

Iterative model pruning and refinement program

pruneyesBy removing unimportant weights or connections from the neural network, the complexity and size of the model can be reduced while maintaining its performance as much as possible. The unimportant weights or connections are usually determined based on their contribution to the model output. distillationthen isUse a pre trained large model (teacher model) to guide the training of a small model (student model). By having the teacher model interact with the student model“teaching”It can enable student models to learn the knowledge of teacher models, thereby reducing the size and computational complexity of the models while maintaining high performance.

The researchers introduced that,althoughLlama-3.1-Minitron 4BThe parameter quantity is onlyfortyBillion, far lower than its predecessorNemotronIn the seriesone hundred and fiftyBillion ginseng quantity model, but using pruning and distillation techniquesTechnique, this model is used for multimodal learning and understanding（MMLU）The score has increased16%.In addition,Training large language models requires a significant amount of computational resources and time. Through pruning and distillation techniques,Llama-3.1-Minitron 4BThe training cost can be reduced at mostone point eightTwice. This is a huge cost savings for businesses and research institutions, enabling more institutions to afford advanced technologyAIApplication and research and development of technology.andIn machine learning, labeled data is an expensive resource. Reducing the demand for training labels means that models can be deployed faster and reliance on large amounts of labeled data can be reduced. This is an important advantage for many practical application scenarios.

NVIDIA CEOHuang RenxundialogueMeta CEOMark Zuckerberg

According to reports, both parties have already beenAIThere have been relevant collaborations on the model.NVIDIAtotwo thousand and twenty-fouryearsevenmonthtwenty-threedayLaunched“NVIDIA AI Foundry”OEM service, customers can use itMetaofLlamaseriesAIModel (such asLlama 3.1）Combining Nvidia's software, computing, and professional knowledge to customize and build for specific fields“Super model”. Nvidia has also launched“NVIDIA NIM”Inference microservices, a set of accelerated inference microservice solutions that allow enterprises to operate on various computing platforms such as cloud computing, data centers, workstations, and morePCEtc.) RunningAIModel. By using industry standardsAPIDevelopers can easily deploy and manage itAIModel, thereby improving development efficiency and model performance.

NVIDIA, as a global leaderGPUManufacturers and providers of artificial intelligence computing platforms have strong technological capabilities. andMetaThe company (formerly known asFacebook）As a giant of social media and Internet technology, there is a huge demand for artificial intelligence, especially in its chat robots and virtual reality（VR）And augmented reality（AR）In other fields. NVIDIA andMetaAs the two giants in the field of artificial intelligence, achieving through cooperationI got itTechnological complementarity and market expansion. NVIDIA, with its powerfulGPUComputational power andAITechnology stack, forMetaProvided solid hardware and software support; andMetaBy utilizing its rich data and application scenarios in social media, metaverse, and other fields, it has promotedAIThe implementation and application of technology. The cooperation between both parties not only promotes technological innovation and market competitiveness, but also jointly promotes the development and application of artificial intelligence technology.

With the continuous development of artificial intelligence technology and the expansion of application scenarios, NVIDIA andMetaThe cooperation will continue to deepen. Both parties will continue to explore new cooperation models and technological innovation points, and jointly promote the development and application of artificial intelligence technology. At the same time, both sides will actively address challenges and issues in the field of artificial intelligence, contributing to the construction of a safer, more trustworthy, and sustainable artificial intelligence ecosystem.