Hi! 👋 Welcome to The Big Y!
A couple of large-scale models have been the talk of the (AI) town for quite some time now, including GPT-3, BERT, and DALL·E. Now, a group out of Stanford has aptly named these types of models as foundational models.
Foundational models are massive models that have been trained on broad data at a very large scale and can be applied to a variety of downstream tasks. The models generally work with different modalities, but we see NLP and text-based models as the leaders at the moment.
The technology behind foundational models isn’t new, since neural networks and self-supervised learning have been around for a while, it is the scale of these models that are now making an impact.
Coming out of the Stanford Institute for Human-Centered Artificial Intelligence (HAI) is the Center for Research on Foundational Models (CRFM), which has been tasked with digging into the impacts of foundational models. Here is the 200+ page report they recently released that covers not only the technical implications from foundational models but also the possible impacts across society.
Many application developers and researchers use these foundational models in their work so any impacts that may be hidden in the foundational models can be perpetuated downstream. If there is bias, sexism, or racism, for example, baked into a foundational model like GPT-3, then a product based on it will likely also exhibit this bias.
There are great opportunities that can come with these large-scale foundational models, but since they are still in their nascent stages, we must be careful relying on their results without close scrutiny.
Disney is using advanced cameras and sensors combined with AI to create a new generation of robots. Not only will these new advanced robots interact with guests at the theme parks, but they’ll also play a role as stunt performers.
Thanks for reading! Share this with a friend if you think they'd like it too. Have a great week! 😁
🎙 The Big Y Podcast: Listen on Spotify, Apple Podcasts, Stitcher, Substack