The data platform Snowflake is hosting its annual user conference, BUILD 2024, bringing together data scientists and developers and sharing new functionality across its platform that will enable customers to get more value from their data and build AI functionality on top of it.
New updates across Snowflake platform enable greater collaboration, flexibility, and security
First, the company announced several new capabilities for its flagship platform that will enable customers to better collaborate around their data.
The new Snowflake Internal Marketplace allows users to discover data, apps, and AI services created by different teams in the company. The marketplace also facilitates sharing of fine-tuned LLMs, which improves cross-team collaboration on generative AI use cases.
Listings in the marketplace benefit from AI via Copilot for Listings, an assistant that can answer questions on structured data to help users understand if shared data is relevant to them.
Additionally, the company announced an integration of the Snowflake Native App Framework with Snowpark Container Services, enabling developers to build applications in their preferred language and distribute them across clouds and regions. Additionally, new support for Snowpark ML Modeling API provides access to familiar Python frameworks to preprocess data, do feature engineering, and train models in Snowflake.
Snowflake is also introducing a new Egress Cost Optimizer that removes fees for transferring data out of a cloud service, enabling greater flexibility for resource allocation.
For customers that have deployed Snowflake on Virtual Private Snowflake or use AWS PrivateLink, the company has a few new security-related updates. Snowflake Native App Support for Secure Deployments now allows those customers to install and use Snowflake Native Apps while reducing exposure to external threats. The new Snowflake Native App Compliance Badging will enable customers to more easily recognize apps that meet their internal compliance requirements.
“To keep up with the evolving data and AI landscape, enterprises need instant access to all of the data within their organizations, supplemented with their customers’ and partners’ data, in order to build powerful AI apps at scale for crucial decision-making,” said Prasanna Krishnan, head of collaboration and Horizon at Snowflake. “That’s where Snowflake’s industry-leading cross-cloud collaboration comes in, enabling organizations to not only access and share their most valuable data, but also apps and AI models, both internally and externally with third-parties. Snowflake’s latest innovations unlock new ways for teams to collaborate on these initiatives and bring AI and apps into production faster, while handling security and governance implications to reduce risk.”
Unistore is a single platform for transactional and analytical data
Unistore is a new platform that will simplify data architectures by enabling transactional and analytical data to exist side-by-side in a single platform. It is powered by Hybrid Tables, which is a table type that supports transactional workloads by enabling high concurrency point operations.
According to Snowflake, traditional data architectures typically require that these data types be separate, leading to data silos and governance gaps, and also making it difficult to transfer data between systems.
Hybrid Tables identify when a query is transactional or analytical, which allows Unistore to provide customers the most optimal performance for those queries.
“The general availability of Hybrid Tables are the next iteration of Snowflake’s journey, empowering enterprises to execute both transactional and analytics use cases from a single platform,” said Carl Perry, head of core services at Snowflake. “With Hybrid Tables, which power Unistore, enterprises also benefit from Snowflake’s unified security and governance across all of their data, so they can spend less time worrying about their data protections, and more time accelerating innovation with the AI Data Cloud.”
Snowflake Intelligence enables agentic AI capabilities on Snowflake
Snowflake Intelligence is an AI platform that allows users to create data agents that can analyze, summarize, and take action on data.
Coming soon to private preview, this new platform will enable organizations to get greater value from their existing data. It can integrate with third-party tools and data sources, like a database of sales transactions, documents in knowledge bases like SharePoint, and productivity tools like Slack, Salesforce, and Google Workspace.
It is built on Cortex AI, the company’s managed AI engine, and utilizes Cortex Search for running queries on unstructured data and Cortex Analyst for running queries on structured data.
Snowflake Intelligence also integrates with the Snowflake Horizon Catalog, which provides data governance and discovery. This integration makes the platform compatible with open data formats like Apache Iceberg and Apache Polaris.
AI/ML model training updates
For companies building their own AI products, Snowflake also announced a number of new updates related to training models that will enable customers to build conversational apps for structured and unstructured data, run batch LLM inference, and train custom models on GPU-powered containers.
In Cortex AI, support will soon be added for multimodal inputs, a Snowflake Connector for Microsoft SharePoint is being added, new document preprocessing capabilities are being introduced, and more.
Customers will also be able to do batch inferencing, which significantly reduces costs compared to processing individual requests. The company is additionally adding new pre-trained LLMs, embedding model sizes, and context window lengths, which will enable customers to further make decisions based on cost.
Snowflake ML is adding support for Container Runtime as well, which will allow customers to distribute training jobs across GPUs.
Finally, the company is introducing Observability for ML Model, utilizing monitoring technology from TruEra to provide visibility into models running inference on Snowflake.
“The latest innovations to Snowflake Cortex AI and Snowflake ML enable data teams and developers to accelerate the path to delivering trusted AI with their enterprise data, so they can build chatbots faster, improve the cost and performance of their AI initiatives, and accelerate ML development,” said Baris Gultekin, head of AI at Snowflake.