Kong recently unveiled a collection of six open-source AI plugins for its Kong Gateway 3.6, which offer integration with multiple LLMs and significantly advance AI technology accessibility for developers and platform teams.
According to Kong, the goal of this release is to enhance developer productivity by simplifying the integration process of LLMs into various products. It ensures a secure, compliant, and controllable environment for AI requests within organizations.
“Today marks a significant milestone in our journey towards democratizing AI for developers and enterprises worldwide. By open-sourcing this suite of innovative AI capabilities, including no-code AI plugins, we’re removing the barriers to AI adoption and making it possible for developers to leverage multiple LLMs effortlessly and ship AI-powered applications faster. At the same time, we’re providing governance and visibility to all the AI traffic that is being generated by an organization,“ said Marco Palladino, CTO and co-founder of Kong Inc.
Kong’s introduction of AI capabilities into its Kong Gateway via the “ai-proxy” plugin enables the incorporation of multiple LLMs, such as those offered by OpenAI, Azure AI, and others, providing a unified interface that allows developers to switch between models without altering their application code. It also simplifies the management of AI credentials by centralizing them within the Kong Gateway. This approach enhances security by abstracting credential management away from applications, enabling easier credential rotation and updates, according to Kong.
The “ai-proxy” plugin extends its functionality into the realm of analytics by enabling the collection of detailed Layer 7 AI metrics,which includes tracking request and response token counts, and usage data across different LLM providers and models. With support for integration with analytics platforms like Datadog and New Relic, alongside Kong Gateway’s native logging plugins, developers gain valuable insights into the usage patterns of their AI integrations.
Additionally, the introduction of no-code AI integration plugins, mostly the “ai-request-transformer” and “ai-response-transformer,” empower users to inject AI capabilities directly into API requests and responses, enabling functionalities like real-time API response translation for broader internationalization efforts without the need for coding.
Kong also introduced plugins that specialize in the creation, decoration, and governance of AI prompts. The “ai-prompt-template” plugin allows for the centralized management of prompt templates, enabling swift updates and compliance with approved templates without needing to modify application code. The “ai-prompt-decorator” and “ai-prompt-guard” plugins provide mechanisms for ensuring that AI prompts are both consistent with organizational policies and protected against unauthorized or sensitive content.