Monitoring and Observability

Azure Monitor is not a separate resource to deploy — every Azure resource emits into it by default. It collects two fundamentally different kinds of telemetry:

A Log Analytics workspace is the Log Analytics query engine plus the storage for every log table sent to it. Bookshelf uses one workspace, law-bookshelf, as the shared destination for `cae-bookshelf’s and `aks-bookshelf’s container logs, `afd-bookshelf’s access logs, and the Application Insights component below — consolidating them lets a single KQL query join a request across every layer it passed through.

Kusto Query Language (KQL) reads like a pipeline: a table name, then |-separated operators that filter, project, summarize or join, each acting on the output of the one before it. Every example below can be run from Logs on law-bookshelf itself, or from the Logs blade of the specific resource (which pre-scopes the query to that resource).

A diagnostic setting is the explicit link between one resource’s logs/metrics and one or more destinations (a Log Analytics workspace, an event hub, or a storage account for long-term archive). Without one, a resource’s logs are never sent anywhere durable — platform metrics keep flowing regardless, but resource-log tables like ContainerAppConsoleLogs or AzureDiagnostics stay empty. For cae-bookshelf specifically, the diagnostic setting only takes effect once the environment’s logs destination is azure-monitor (see Container Apps).

Application Insights is Azure Monitor’s application performance monitoring (APM) feature: request rates, response times, dependency calls (catalog-api calling pg-bookshelf, orders-api calling docdb-bookshelf), exceptions, and distributed traces that follow one request across catalog-api, orders-api and notifier. A workspace-based Application Insights component — the only kind the portal creates today — stores its data as extra tables inside a Log Analytics workspace (law-bookshelf), so KQL queries can join AppRequests, AppExceptions and AppDependencies with the container and platform logs above in one query.

An action group is a reusable notification target — one or more email, SMS, voice, webhook, Azure Function, Logic App or ITSM receivers, defined once and referenced by many alert rules. Bookshelf defines one, ag-bookshelf-oncall, with an email receiver for the on-call rotation:

Derived from Microsoft Learn — Azure CLI reference: az monitor action-group.

A metric alert evaluates a metric-based condition on a fixed schedule (near-real-time, typically every one to five minutes) and fires the action group when it is breached. This one watches catalog-api’s replica count on `cae-bookshelf:

Derived from Microsoft Learn — Azure CLI reference: az monitor metrics alert and Microsoft Learn — Types of Azure Monitor alerts.

A log search alert is the log-data equivalent: instead of a built-in metric condition, it runs a KQL query on a schedule and fires when the result set crosses a threshold — the natural way to alert on one of the KQL queries above (e.g. "more than 10 ERROR lines from catalog-api in the last 5 minutes"). It is created with az monitor scheduled-query create rather than metrics alert create, since it targets a Log Analytics workspace and a query rather than a metric namespace.

A workbook is an interactive report — text, KQL query results (tables, charts), and parameters the reader can change — authored once in the portal and reused across incidents, distinct from a one-off ad hoc query. Azure Monitor ships gallery templates for Container Apps, AKS and Application Insights that already chart request rate, failure rate, latency and dependency health; cloning one of those and adding a Bookshelf-specific KQL query (such as the Front Door error-count query above) is the usual starting point, rather than authoring a workbook from a blank canvas. A dashboard is the simpler, older alternative: a fixed grid of tiles (metric charts, a workbook, a static markdown tile) pinned from individual resource blades, better suited to an always-on big-screen view than to interactive investigation. Both are ARM resources under the hood and can be defined as Bicep/ARM templates for repeatable deployment rather than only clicked together in the portal, which is the practical way to keep a workbook’s definition in source control alongside the rest of Bookshelf’s infrastructure-as-code (covered on Infrastructure as Code).

Derived from Microsoft Learn — Azure Workbooks and Microsoft Learn — Create and share dashboards of Log Analytics data.

These three sit next to Azure Monitor rather than inside it — each answers a different "is something wrong" question that resource-level metrics and logs cannot:

Derived from Microsoft Learn — What is Azure Service Health?, Microsoft Learn — Azure Advisor introduction, and Microsoft Learn — What is Azure Network Watcher?.

This removes law-bookshelf, appi-bookshelf, ag-bookshelf-oncall, and every alert rule and diagnostic setting created on this page, along with the rest of Bookshelf’s resources in $RG. Application Insights and Log Analytics workspaces have no soft-delete purge step of their own to worry about (unlike Key Vault or API Management), so deleting the resource group is sufficient.