Skip to main content
Demo Project
DocsBook DemoStart Free Trial

check the patterns and investigate what needs more attention

The main items needing attention are operational rather than normal application traffic:

  • otelcol-contrib: 147 errors, the largest error contributor. Investigate collector failures first; these may affect telemetry completeness.
  • SYSTEM monitor alerts: 12 errors, all monitor.alert.triggered events reporting latency alerts. Review the associated monitors and affected endpoints to determine whether latency is sustained or threshold-related.
  • flagd EventStream errors: 13 in flagd, with one EventStream error also appearing in recommendation, payment, fraud-detection, and ad. This suggests a shared flagd/EventStream dependency issue rather than isolated service failures.
  • Node disk pressure: recurring warning reports insufficient free space on the image filesystem, with 90% of 71.6 GiB used. This is an infrastructure risk and should be addressed before workloads or telemetry collection are impacted.
  • Configuration warnings: otel-demo repeatedly reports missing data/binaryData fields in ConfigMaps. Validate whether these ConfigMaps are expected to contain configuration; otherwise suppress or correct the noisy warning.

Normal high-volume patterns are dominated by successful product-catalog reads, checkout/payment activity, and frontend traffic. Their high occurrence counts do not by themselves indicate incidents.

7
KQL
status_code == "ERROR" | summarize count() by resource.service.name, name
8 tool calls
list_log_patterns
→ [{"first_seen_at":"2026-09-24T00:16:19.457941Z","id":14090939327,"is_error":false,"last_seen_at":"2026-10-26T18:52:39.924744Z","level":null,"occurrence_count":457520,"pattern":"severity_text;badge-info⇒INFO Found <*> products from database <*>","sample_message":"severity_text;badge-info⇒INFO Found 10 products from database attributes;text-textWeak⇒{\"log\":{\"original\":\"Found 10 products from database\"},\"products\":10,\"session\":{\"id\":\"271b5a91-2eba-4a25-88a1-6d22ab011280\"}} session;right-badge-neutral⇒271b5a91-2eba-4a25-88a1-6d22ab011280","service":"product-catalog","state":"acknowledged"},{"first_seen_at":"2026-08-31T15:11:24.331052Z","id":13485440045,"is_error":false,"last_seen_at":"2026-10-26T18:52:39.924744Z","level":null,"occurrence_count":90048,"pattern":"request_type;neutral⇒incoming rpc.method;neutral⇒oteldemo.ProductCatalogService/ListProducts span_name;neutral⇒oteldemo.ProductCatalogService/ListProducts protocol;right-badge-neutral⇒rpc duration;right-badge-neutral⇒<*> ms session;right-badge-neutral⇒<*> user <*>","sample_message":"request_type;neutral⇒incoming rpc.method;neutral⇒oteldemo.ProductCatalogService/ListProducts span_name;neutral⇒oteldemo.ProductCatalogService/ListProducts attributes;text-textWeak⇒{\"demo\":{\"product\":{\"count\":10}},\"rpc\":{\"method\":\"oteldemo.ProductCatalogService/ListProducts\",\"response\":{\"status_code\":\"OK\"},\"system\":{\"name\":\"grpc\"}},\"server\":{\"address\":\"10.42.0.175\",\"port\":8080},\"session\":{\"id\":\"d7630f4f-6ef8-4098-91cb-43c9dbf3911e\"},\"user\":{\"id\":\"14f3f2de-405e-40c1-9ac9-8ec99447bfab\"}} protocol;right-badge-neutral⇒rpc duration;right-badge-neutral⇒1.6 ms session;right-badge-neutral⇒d7630f4f-6ef8-4098-91cb-43c9dbf3911e user id;right-badge-neutral⇒14f3f2de-405e-40c1-9ac9-8ec99447bfab","service":"product-catalog","state":"acknowledged"},{"first_seen_at":"2026-09-01T15:44:00.099Z","id":13603205113,"is_error":false,"last_seen_at":"2026-10-26T18:52:39.924744Z","level":null,"occurrence_
run_queryString "status_code == \"ERROR\" | summarize count() by resource.service.name"
→ Results (7 rows): recommendation | 5.0 payment | 5.0 otelcol-contrib | 147.0 fraud-detection | 5.0 flagd | 13.0 ad | 5.0 SYSTEM | 12.0
run_queryString "status_code == \"ERROR\" | summarize count() by resource.service.name, name"
→ Results (7 rows): recommendation | /flagd.evaluation.v2.Service/EventStream | 5.0 payment | grpc.flagd.evaluation.v2.Service/EventStream | 5.0 otelcol-contrib | null | 147.0 fraud-detection | flagd.evaluation.v2.Service/EventStream | 5.0 flagd | flagd.evaluation.v2.Service/EventStream | 13.0 ad | flagd.evaluation.v2.Service/EventStream | 5.0 SYSTEM | monitor.alert.triggered | 12.0
run_queryString "level in (\"WARN\", \"ERROR\", \"FATAL\") | summarize count() by resource.service.name, level"
→ Results (4 rows): null | WARN | 12.0 otelcol-contrib | ERROR | 147.0 otel-demo | WARN | 60.0 SYSTEM | ERROR | 12.0
run_queryString "status_code == \"ERROR\" | summarize count() by bin_auto(timestamp), resource.service.name"
→ Results (95 rows): 1.79304066e9 | SYSTEM | 1.0 1.79304036e9 | SYSTEM | 1.0 1.79304009e9 | flagd | 1.0 1.79304009e9 | ad | 1.0 1.79304006e9 | SYSTEM | 1.0 1.79303976e9 | SYSTEM | 1.0 1.79303973e9 | payment | 1.0 1.7930397e9 | recommendation | 1.0 1.79303964e9 | flagd | 1.0 1.79303964e9 | fraud-detection | 1.0 1.79303949e9 | flagd | 1.0 1.79303949e9 | ad | 1.0 1.7930394e9 | SYSTEM | 1.0 1.79303913e9 | payment | 1.0 1.7930391e9 | SYSTEM | 1.0 1.7930391e9 | recommendation | 1.0 1.79303904e9 | flagd | 1.0 1.79303904e9 | fraud-detection | 1.0 1.79303886e9 | flagd | 1.0 1.79303886e9 | ad | 1.0 ... +75 more
sample_logsString "status_code == \"ERROR\""
→ Sample logs: - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting f0514134-7aba-571f-a83f-12b14035b434 2026-10-26T18:51:06.162946Z null monitor.aler - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting 5612af8a-ac3a-5a2e-b6c2-d7bce3100f31 2026-10-26T18:46:05.135268Z null monitor.aler - [null] flagd.evaluation.v2.Service/EventStream (flagd): null 33890f7d-5d0b-57c2-b5a1-205bc4d9d25a 2026-10-26T18:41:53.812725Z c62eb050ae9954bce5c1591090e39d - [null] flagd.evaluation.v2.Service/EventStream (ad): null 97ec81a3-4d04-5432-a3ad-2d30983d595b 2026-10-26T18:41:53.810778Z c62eb050ae9954bce5c1591090e39d - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting 3d4445e0-d5c0-50a3-978a-a331f9904541 2026-10-26T18:41:04.338884Z null monitor.aler
sample_logsString "level == \"ERROR\""
→ Sample logs: - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting f0514134-7aba-571f-a83f-12b14035b434 2026-10-26T18:51:06.162946Z null monitor.aler - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting 5612af8a-ac3a-5a2e-b6c2-d7bce3100f31 2026-10-26T18:46:05.135268Z null monitor.aler - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting 3d4445e0-d5c0-50a3-978a-a331f9904541 2026-10-26T18:41:04.338884Z null monitor.aler - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting e247e285-cdd7-5e99-adf2-174fa74a9693 2026-10-26T18:36:04.295545Z null monitor.aler - [ERROR] monitor.alert.triggered (SYSTEM): Latency: alerting e9e2d3cc-bda8-5298-9d30-49b7dcf7ddc0 2026-10-26T18:30:15.266585Z null monitor.aler
sample_logsString "level == \"WARN\""
→ Sample logs: - [WARN] null (otel-demo): No data/binaryData field in configmap fdd5996b-6b23-5e21-a537-cc51a5645369 2026-10-26T18:52:39.76576 - [WARN] null (otel-demo): No data/binaryData field in configmap 24dcbcfd-abf7-5bd2-a93a-dcdd7b2ccb19 2026-10-26T18:51:39.67298 - [WARN] null (otel-demo): No data/binaryData field in configmap a6f3ae77-714c-529e-8d79-e2e1b04c9e0e 2026-10-26T18:50:39.56729 - [WARN] null (null): Insufficient free disk space on the node's image filesystem (90% of 71.6 GiB used). Failed to free s - [WARN] null (otel-demo): No data/binaryData field in configmap 5ee1e166-30aa-50c4-a50c-10fb8a5ad0b5 2026-10-26T18:49:39.46705

AI can use your project data and connected integrations. Review actions before relying on them.