Llm

Over the past year I’ve been running a hands-on workshop called “New Relic AI Monitoring Mastery” at developer events. The premise is simple: attendees build a small AI chatbot from scratch using Python, Flask, and the OpenAI API, then instrument it with New Relic to see exactly what’s happening inside. This post captures the key ideas and lessons from those sessions - what metrics matter, how to get visibility quickly, and why model choice has a bigger operational impact than most people expect. ...

AI applications are complex and distributed, making effective monitoring challenging. Combining the New Relic intelligent observability platform with Pulumi’s infrastructure-as-code and secret management solutions allows for an end-to-end “observability as code” approach. This method enables teams to: Define artificial intelligence (AI) and large language model (LLM) monitoring instrumentation along with cloud resources programmatically. Securely manage API keys and cloud account credentials. Automatically deploy New Relic instrumentation alongside AI applications and infrastructure. Benefits include: ...