Accurate

You don't have to choose - OpenTelemetry has an Azure Monitor exporter. I guess that's part of the advantage - YOU want to run Azure Monitor, but a different deployment of same app may want to run Prometheus/Grafana - and this gives you the option of using any of them without changing your code (except for config).

That is what is good about Azure Monitor by itself. I do not have to do anything in the code. Azure, as the hosting platform provides the telemetry and does all the plumbing. Other cloud providers have similar things as well. If you run something on-prem, I can see the need for that. There might be other scenarios. However, when cloud providers are in use, I think that features like telemetry are built in, without the need to change anything in the code.

Managed Prometheus and Grafana have been fused with Azure Monitor. AppInsights is now a component of Azure Monitor. When it comes to monitoring, you are spoiled for choice. It boils down to your preferred tools and the breadth and depth of coverage. The advantage of Azure Monitor is that you can ingest signals from your "entire infrastructure" and other tools and services. This allows you to correlate the information and carry out advanced diagnostics on the logs, metrics, and traces.

Depends on the size and circumstances of the team. For a new team getting started and bootstrapping, the initial team members will do all this ground work themselves. Sometimes, they will just be 1 to 3 people. So they need to know how to code, deploy, handle kubernetes, monitor and deal with database and security issues. Sprinkle on some PKI issues, email setup, etc, etc... Tools like this are meant for such individuals (and also more advanced teams with greater delineation of duties). It reduces the cognitive load while allowing them stay in control of everything going on. They are able to rev quickly, experiment and adapt till they find the sweet spot that works for them that will allow frequent, repeatable deployments and features are added and the project grows... My thoughts (also speaking from personal experience).

If you're a one-person shop trying to get off the ground, unless you have unlimited runway and bandwidth, this is probably more of a v2.0 / do-it-when-you're-profitable kind of thing. This stuff is cool and immensely useful for finding hot-spots, bottlenecks, and overall project health, but you will bog yourself down if you focus on these kinds of things instead of the core product. Gotta keep minimal viable product in mind. If you run out of funding the project will never see the light of day. No amount of telemetry, metrics, or logging will fix that.

Prometheus stores it. You can query prometheus directly or via a visualization tool like Grafana

@@dauchande but where does Prometheus store it? Disc? Can it be configured other storage media like Azure storage table or blob?

@@andrewiecisa2907 Prometheus itself is a time-series database, so you don't store its data in other Azure databases. There is a Managed Prometheus offering in Azure, or you can use a compatible database called Grafana Mimir that can store data in cloud blob storage such as Azure blob storage. Then use something like Grafana Agent to remote-write the prometheus data to Mimir

Worked parallel to a team of about a dozen devs on a brand new greenfield project w/ a couple dozen or so micro apps and services (the tail was wagging the dog for sure, w/ the devs calling all the shots). They were generating on the order of gigabytes of logging data every night, and they weren't even live. Our DBA was furious with them. They ran themselves right into the ground -- the project was poop-canned about 5 years into it. Millions of dollars wasted, other than padding their resumes, and the sad part was they were just fine with that and moved on to make the same mistakes in future projects w/ other organizations. Your point is well taken, esp. for the people in charge funding these development projects. Buyer beware.

Adam and Eve were first, not you!!