Google has announced a new open model called VaultGemma based on recent findings around differential privacy. The AI model has 1-billion parameters and is based on the Gemma 2 from Google’s family of small language models. 

It has been known historically that adding differential privacy to an AI model could prevent it from regurgitating output that is identical to the data it was trained on. This is done by introducing some amount of noise during training. 

However, adding noise to training data reduces the accuracy of an AI model. 

Google teamed up with Google DeepMind on a research titled, “Scaling Laws for Differentially Private Language Models,” which looked into the volume of randomised noise as compared to the size of the overall training dataset. 

The researchers conducted experiments with different model sizes and noise-batch ratios to understand how differential privacy could be scaled while maintaining good output. 

A blog posted by Google claimed that this is the largest open model on differential privacy that can be used to develop high-utility AI models.

Developers can download VaultGemma from Hugging Face and Kaggle. Google has also released the weights so users can fine-tune the AI model to build their own versions.