04 — Azure / Distributed Systems / Production

Azure Service Bus

Why:

• decouple producer/consumer
• async processing
• absorb bursts
• retry/recovery
• independent scaling

Queue = one logical processing flow
Topic/subscription = multiple subscriber flows

Consumer failure

message → processing fails → retry/redelivery → repeated failure → DLQ → diagnose/reprocess

Duplicate messages

Assume duplicates can happen.
Use:

• idempotency key
• unique constraint
• processed-message table

Key:

at-least-once delivery requires idempotent consumers.

DLQ

For poison/repeatedly failing messages.
Need monitoring + failure reason + safe replay.

Azure Functions

Serverless/event-driven compute.
Good for timers, queues, Service Bus, bursty workloads.

Function vs Web API:
Function = event/serverless model
Web API = long-lived HTTP service, more hosting/control

Retry

Retry transient failures.
Do not blindly retry validation/auth/business-rule failures.
Use max attempts + exponential backoff + jitter + timeout.

Outbox

DB commit succeeds but publish fails:

• write business data + outbox in one DB transaction
• background publisher sends event
• consumer idempotent

Observability

Logs = what happened
Metrics = how much/how often
Traces = where time was spent

Production incident

1. Detect
2. Assess impact
3. Stabilize
4. Investigate
5. Mitigate
6. Restore
7. RCA
8. Prevent recurrence

CI/CD

commit → build → test → checks → package → deploy → smoke test → monitor

Know rollback, health checks, config, secrets, artifact/version.

Zero downtime

Rolling, blue/green, canary.
DB migration: add → compatible deploy → migrate → switch → remove later.

Questions

1. Why Service Bus?
2. Queue vs Topic?
3. Consumer fails?
4. DLQ?
5. Duplicate messages?
6. Idempotency?
7. Function vs Web API?
8. Outbox?
9. Logs vs metrics vs traces?
10. Production API down?