Analyzing BC Telemetry with AI with the “BC Telemetry Buddy”

I did a thing …

TL;DR

I created an MCP for VSCode that makes it possible for you to just “talk” to your Telemetry without touching KQL or getting to know any tools.  It’s available on the marketplace, you don’t need extra subscriptions to fancy models or tools – it just uses your chat subscription (I only tested it with Github Copilot), and that’s it.

Repo: https://github.com/waldo1001/waldo.BCTelemetryBuddy
Marketplace: https://marketplace.visualstudio.com/items?itemName=waldoBC.bc-telemetry-buddy

Simply run the Setup Wizard, and you should be good to go!

Issues?  It’s an OpenSource project, so use the issues on github.

There’s more to it, so let me tell you a story how I came about creating this, and why this has been consuming pretty much all my free time for the past 10 days (no, it’s not the development part, it’s what you can do with the tool ) 

The story

Before I start the story – let me first say a few things…

Obviously, this project has been massively inspired by the work I did the past year together with Dmitry Katson.  Honestly, there isn’t one person I learned more from about AI, than Dmitry – for me he is THE guru of anything BC AI and it was a privilege to have had the chance to work on an incredibly interesting project: the Troubleshooting Buddy.  We demonstrated that last year at Directions and half a year later at BCTechDays.  If you want to see that session:

Now, while we were incredibly proud of the work we did, at BCTechDays, it became painfully clear that quite some people found it far too complex .. some people even claimed it was “too much too soon”. 

It took me a while – but I had to respectfully disagree.  It was not “too much too soon” .. It was rather “too much too late”.  Because quite honestly – besides the fact we coded a marvel of an AI copilot .. It was deterministic and too difficult to maintain.  Or in other words .. It was not an agent.  It looked like it was one, but it wasn’t even close to an agent…

So whatever we did – it was already “too old” and “obsolete” .. .   There were already new possibilities that made this just the wrong architecture.  Do know, I didn’t mind ONE SINGLE SECOND (and there were many…) of spending the time to the troubleshooting buddy though – the things I learned (most of all thanks to Dmitry) was invaluable!

That said …

One morning (last week), I got up after a terrible night of sleep wondering why I couldn’t find a certain Telemetry-related issue.  And since I’m doing lots of AI lately – I wondered – why can’t I just ask?

You know where this is going ..

I opened my ChatGPT (yep, I have a subscription on ChatGPT, and I’m not ashamed for it ), and asked whether it would be possible to create some kind of VSCode extension that allows me to simply ask questions on BC Telemetry – and doing that by using the already working GitHub Copilot subscription in VSCode.  In fact, I still have the first prompt here.

It became a 4-hour during conversation going back and forward, all resulting in an instruction set of what I thought could be an working solution to have some kind of way to “talk about BC Telemetry” in VSCode.

Since we both (the agent and me) agreed this was possible, I decided to sacrifice the next few nights to come up with an mvp (an mvp of the mcp if you will ).  In fact, I decided it would be incredibly interesting to track all the prompts and design changes, so I created some copilot instructions to always log all my prompts and all design changes in some md files.  This resulted into a huge promptlog and a huge designwalkthrough.  No idea if this is interesting to anybody – but it sure enables me to track / document the flow of my vibe-coding experience, which was the exact intention of these files and one of the motivators for this challenge ;-).

The results after a week of “vibe coding by night”?

Well, I’ll just give you a few pointers on how I think it should be used, but it’s all just your own imagination on how you can use this tool, and even combine it with other tools (eg, I didn’t go into combining the stacktrace analysis with reading AL files or symbols .. and doing code changes from telemetry conclusions .. or anything like that …).

The setup

I would advice to simply create a new workspace: the settings are stored per workspace, so you can have a workspace per AppInsights endpoint, and store analysis reports and queries per workspace and organize your work like that.  A workspace could be a customers.. I mean .. your choice ;-).

When you installed the buddy from the marketplace, simply open the Setup Wizard:

It’s a few steps that are clearly explained, with some test-buttons and all – it should support multiple types of authentication.  For me, AzureCLI works wonderfully well, so I made that the default.

Once set up, reload your environment, and you’re good to go!  The setup should look something like:

The MCP

The MCP is bundled in the VSCode Extension, so when you do “MCP: List Server”, you’ll see it there, and you can start it there.  Always good to start it and watch the output – if it ends with “Discovered 11 tools” (in time this number could change) – that’s a very good start!

If not – tip: ask you copilot to help you set it up (refer to my github repo while doing so).

Now, the process is set up this way that the copilot is motivated to find out about:

• certain BC-related events (there is a “smart” event discovery to help the copilot focus on certain BC related eventids)
• common field names and datatypes to try to help the copilot to interpret the fields the right way
• mapping tenantid’s to customer names based on the CompanyName – to let the user focus on a customer without being bothered by the tenantid, although the copilot is motivated (is how I like to describe it ;-)) to use tenantid instead of filters on company names
• Look into online resources (you can set up online resources in the setup, like Microsoft learn, BCTech, and such)…. For example:

• Look into local resources, like KQL files (yes, just imagine you liked a certain analysis – simply ask to save the query, and it will be saved in your repo, which can be of use later on)
• …

Yeah, there are quite some BC related functions to help copilot to come up with the right KQL.  I tried to set up a userguide for further reference ;-).

The Chatmode

I also included an experimental (give me feedback) chatmode.  In a way, this is an example for you to create your own chatmodes.  If you don’t know what chatmodes are: Use chat modes in VS Code.  So if you would like to do certain types of analysis, with a limited set of tools for the agent (to let it focus on the ones it has available) – chatmodes are AWESOME!

So I included an example which you can make available like so:

Your first question

By means of test, you could ask:

You’ll see that the agent will be getting the event catalogue to find out “what are errors in Business Central”, and then perform a series of analysis.  Like .. Insane!

Example output (not even close to the level of detail you could get to)

From here, you could dive deeper into a company, certain errors, callstacks, timings, .. Whatever!

Some tips

One chat, one context

In one chat, you can simply ask about the previous output, because it keeps all context.  So if you ask about all errors, it might give you customer names.  At that point, simply ask “ok, give me a deeper analysis on customer x”, and it will do just that.

Even more, if you drill into a certain customer, you might see this tool

Which is a good sign, because it will search for the customer name’s tenantid, and then filter on tenant instead of company (which is what you want ;-))

But .. that also means if you switch context yourself (in your mine) and you don’t create a new chat, it might take your next question as a follow up of the previous ones, which might be confusing at some points..

I want graphs in a PDF

Even graphs are possible.  And yes, also PDF or PNG or .. .  All you need to do is ask.  It might give you options (which type of graphs) – I particularly like python graphs.  But mermaid or anything like that – just ask, you’ll be amazed .

A very simple example:

And as a result:

I know .. I guess if you ask vague questions, you get vague answers

Let it look into stacktraces and SQLStatements

When i’m looking into performance, I like to also look into stacktraces and SQLStatements.  Simply ask for a deep SQL analysis, or to look into stacktraces to try to find similarities (like in deadlocks analysis).

It will also discover custom events

There is a reason I didn’t want to rely on only the Microsoft provided documentation, simply because a lot of us are using custom telemetry.  That’s why the tool “get_event_catalog” is there, with a custom KQL to discover events based on telemetry.  So when I ask:

Since I obviously have “iFacto Telemetry” installed (more info: Handling Business Central Telemetry like a boss: iFacto Telemetry – Pt. 3) it answers:

Create your own chatmodes

Again – the chatmodes are powerful.  And the one I provide, just an example.  Create your own for example to create your own Performance report.  Describe what you expect from it, what it needs to contain, may be help it a bit in analysing specific points (eg, look into the mount of extensions installed, or the usages of the search on pages, or …) by providing your own KQL, or just eventids, or not even that: just a structure and points you definitely want (or don’t want) in the report.  Explain how you want the report (an md file makes sense), and may be even export it as PDF .. Even with charts and all that.

I mean .. the possibilities are endless.. .

Conclusion

Use and abuse ;-).  I’m in full prep-mode for Directions, so I won’t touch the buddy anymore until after Directions EMEA, but let’s assemble feedback in the “issues” on github in the meantime.  Tips, tricks – EVERYTHING is interesting to gather ;-).

I hope you’ll enjoy!

Original Post https://waldo.be/2025/10/22/i-did-a-thing-again-bc-telemetry-buddy/