New Relic is hoping to provide customers with a better understanding of the health and state of their AI applications with the introduction of New Relic AI monitoring.
“New Relic AI monitoring brings the power of observability to engineers working on AI by providing the necessary insights to debug, monitor, and improve AI applications, ensuring that they operate as intended, deliver accurate results, and meet emerging standards for responsible use,” New Relic wrote in a blog post back when the platform was first introduced into early access last November.
The new solution, which is now generally available, will enable customers to quickly determine the root cause of AI applications issues, enabling more efficient troubleshooting and resolution.
It provides a holistic view of AI applications and their infrastructure, tracking metrics like number of requests, response time, and token usage. From a single view, users can see trends in AI responses, analyze sentiment, and see user feedback.
Customers can also trace the life cycle of AI responses, which is helpful for fixing performance and quality issues, such as bias, toxicity, and hallucinations.
The tool also provides a method for comparing different models in terms of performance, cost, and quality issues. “By tracking the token usage across AI models, you can identify which models are the most expensive to run. You can then choose less expensive models to optimize your AI application architecture,” New Relic wrote.
New Relic currently has over 60 integrations across the AI ecosystem, including popular LLMs, machine learning libraries, vector databases, and orchestration frameworks.