Home

Telemetry and Data Quality: Measuring the Offer Funnel

January 22, 2023

You cannot improve the offer funnel if you cannot measure it. Telemetry should show what request came in, what was returned, how fast it happened, and whether the traveler converted. Treat the funnel like a product with instrumentation baked in.

Define the funnel stages

  • Search received: Request metadata, traveler context, correlation IDs.
  • Offers generated: Count, composition, pricing methods used.
  • Offers viewed: Which offers reached the UI or partner API response.
  • Offer selected and ordered: Conversion events, including reasons when travelers abandon.

Data quality guardrails

Telemetry is only helpful if data is trustworthy. Implement automated checks:

  1. Schema validation for every event before it leaves the service.
  2. Sampling of payloads against business rules (e.g., total price matches sum of components).
  3. Anomaly detection for sudden drops in offer counts or spikes in errors.
Golden signals to monitor
  • Latency percentiles per dependency (inventory, pricing, ancillaries).
  • Error code distribution grouped by partner or channel.
  • Offer-to-order conversion by market and fare brand.
  • Time to first meaningful interaction after new releases.

Tooling considerations

Adopt centralized logging and metrics platforms. Ensure developers can trace a single request across microservices using consistent correlation IDs. Provide dashboards for operations, but also create self-service notebooks so analysts can explore data without engineering bottlenecks.

With disciplined telemetry and data quality checks, the offer funnel becomes a feedback loop. Teams can prioritize fixes based on measurable impact rather than intuition.

2026 technical upgrade: commercial observability from Offer to Order

Updated 5 November 2026. An HTTP 200 response is not necessarily a successful retail outcome. A search response may be technically valid yet contain no usable Offers; a payment authorization may succeed while Order creation remains uncertain; and a confirmed Order may not propagate to the systems responsible for delivery. The operating question is therefore not only whether an API is healthy, but whether a customer can complete a commercially valid journey. The following is an independent practitioner framework, not a prescribed IATA telemetry standard.

Separate technical success from commercial success

Define a small, mutually exclusive set of business outcomes at each boundary. For shopping, distinguish valid offers returned, valid no-availability, validation rejected, dependency timeout, and partial response. For commitment, distinguish Order confirmed, Order rejected, and commit outcome unknown. The last category matters: an API timeout after submitting OrderCreate must not be counted as a confirmed failure or blindly retried as a fresh sale. The correct recovery action is to reconcile against an authoritative Order reference and payment state before deciding whether to retry.

Record both technical and commercial outcomes, with explicit definitions for numerator, denominator and exclusions. An Offer availability rate should not silently include malformed requests in its denominator; conversely, excluding dependency failures can conceal an outage. Publish the definition alongside each dashboard so commercial, engineering and partner teams interpret the same number.

Instrument the journey as correlated operations

Use distributed tracing for the path from shopping gateway through pricing, availability, product assembly, Offer presentation, selection, payment and Order creation. OpenTelemetry's observability primer describes traces, metrics and logs; its semantic conventions standardize common technical attributes. These are general observability references, not airline-specific event definitions. Custom commercial fields proposed here should be documented and versioned independently.

Attach a generated journey correlation key at the edge, then propagate trace context through synchronous requests and asynchronous messages. Keep a separate stable business operation identifier for retries and delayed callbacks. An Order identifier should only be attached once the authoritative system provides it. Avoid treating a client-generated request identifier as proof that an Order exists. Store trace and business identifiers in a controlled lookup table, not as uncontrolled metric labels.

Suggested event fields include event time, observation time, channel category, market grouping, operation type, technical outcome, commercial outcome, retry classification, dependency class, software release and a pseudonymous correlation reference. Keep passenger names, contact details, payment credentials, raw Offers and unrestricted Order payloads out of default logs. Where investigative access to sensitive evidence is necessary, apply role-based access, retention rules and redaction before collection.

Build a commercial-impact signal, not a fictional revenue number

Prioritize alarms using a sequence of evidence. First identify an abnormal deviation from a matched baseline: for example, the proportion of eligible shopping requests that return a usable Offer falls from 97% to 71% for a specific route group and channel. Next confirm the deviation is not caused by traffic-mix shifts, valid no-availability, planned schedule changes or data-quality gaps. Finally estimate affected opportunities using the observed eligible request count and historical conversion ranges. Report the result as an estimated exposure interval, never as proven lost revenue.

For an illustrative 10,000 eligible shopping requests, a fall from 97% to 71% represents 2,600 fewer requests with usable Offers. If the historical conversion rate for comparable eligible requests is 1–3%, that suggests approximately 26–78 potentially affected Orders before considering substitution, demand changes and recoveries. Multiplying by a headline average basket value would be an additional assumption, not a measured financial loss. A defensible incident record preserves each assumption and the data window used.

Protect against misleading aggregate dashboards

Segment carefully by channel, partner integration class, journey type, geography, request size and deployment cohort, while imposing minimum sample thresholds. An aggregate p95 latency improvement can coexist with a severe regression for a low-volume partner. Equally, one partner's traffic surge can make global Offer conversion look worse without any change in system performance. Use cohort-normalized comparisons, absolute request counts and confidence intervals when drawing conclusions from small samples.

Distinguish observed no-availability from inferred availability loss. A legitimate zero-Offer response is not automatically a defect. Use controlled synthetic journeys or independent reference checks to validate suspicious patterns; do not assume that a cache, inventory or pricing subsystem is wrong solely because one source returns a different result. Track freshness and timestamps of the reference evidence.

Recommended operational metrics and owners

  • Offer usability: eligible searches with at least one usable Offer divided by eligible searches, segmented by channel and market; owned jointly by retail product and engineering.
  • Commit uncertainty: OrderCreate attempts with unresolved outcome after a defined reconciliation window divided by all attempts; owned by Order operations.
  • Payment/Order divergence: authorizations without a matched confirmed Order beyond the agreed investigation window; owned by payment and Order reconciliation.
  • Dependency impact: request cohorts affected by pricing, availability, ancillary or identity failures, including timeouts and fallback paths; owned by the relevant platform teams.
  • Detection and recovery: time from first measurable anomaly to alert, containment and authoritative state reconciliation; owned by incident management.

Choose thresholds from historical seasonality and a documented risk appetite. A sample threshold such as “alert when usable-Offer rate falls by ten percentage points for 15 minutes” is an illustrative policy, not an industry requirement. Pair rate alerts with absolute counts to avoid paging on a handful of requests. Every alert should identify a responsible team, an evidence dashboard, a runbook and a reversible mitigation.

Worked incident: healthy API, broken customer outcome

Imagine a release that increases shopping API success from 99.3% to 99.7% while introducing a product-assembly error for a subset of multi-sector journeys. Response status monitoring looks better, but the proportion of eligible journeys returning a bookable Offer drops. A commercial-outcome dashboard catches the drop; trace exemplars locate the product-assembly boundary; release-cohort segmentation identifies the change; and a controlled rollback restores the baseline. Investigators then reconcile any in-flight Order attempts and payment authorizations rather than assuming all abandoned sessions were harmless.

The post-incident review should document the detection delay, how many journeys were eligible, what was observed versus estimated, which customers might need recovery, and what regression test would have caught the defect. Avoid asserting exact lost sales unless a credible counterfactual and validated financial evidence exist.

Acceptance tests before rollout

  1. Simulate a 200 response with zero usable Offers; verify the commercial outcome is not counted as success.
  2. Force a timeout after OrderCreate submission; verify the outcome remains unknown until authoritative reconciliation.
  3. Introduce duplicate asynchronous notifications; verify counting is idempotent by business operation.
  4. Drop trace context at one dependency; verify the coverage dashboard detects the broken correlation.
  5. Shift traffic between two channels without changing either channel's performance; verify aggregate movement is explained by mix.
  6. Inject a low-volume partner regression; verify minimum-count logic prevents false certainty without hiding the issue.
  7. Check that sampled traces and log exports contain no raw passenger or payment secrets.
  8. Run a release rollback and demonstrate that the operational dashboard identifies both recovery and remaining unresolved commitments.

What standards say—and what this article recommends

IATA's ONE Order overview describes a single Customer Order reference supporting delivery and settlement; IATA's Shop Order Pay standards programme describes interactions for Offers, Orders, payment and consumption. Neither source mandates the custom KPIs, exposure model or alert thresholds above. OpenTelemetry's general semantic conventions provide technical instrumentation vocabulary, not a normative definition of airline commercial conversion. The outcome taxonomy, runbooks, calculations and test cases here are independent engineering proposals requiring local validation.

For related evidence controls, read Order servicing audit trails and refund orchestration and recovery.

Back to all blogs