You change your profile picture and the profile page shows it immediately. Then you open the feed and an older post still has the previous picture. A comment may already show the new one, while a notification and a search result do not. The natural question is simple: I changed it, so why is the old picture still here?
Inside a large platform, changing the authoritative profile record is only the first step. The same person can appear in feeds, comments, notifications, search documents, recommendations, messages, cached API responses, and image caches near users around the world. Those surfaces do not all read the same record at the same moment.
one logical writeUser 42 points to a new avatarA small, durable mutation owned by the User Service.many visible representationsMillions of views may still contain old dataEach surface has its own storage, cache, and freshness contract.1. One person, many consistency boundaries
A profile page can query the User Service or a profile projection with a very short freshness target. A feed might use materialized entries created hours ago. Search owns an index. A notification may contain the author summary captured when the notification was created. The image bytes can be cached by a browser and by multiple CDN locations. All of these are valid designs, but they create separate consistency boundaries.
Temporary disagreement can mean that each subsystem is honoring its own contract, not that the profile update failed.
profileCurrent stateUsually reads a fresh profile projection or the authoritative service.
feedMaterialized stateOptimized for predictable reads and fewer dependent calls.
searchIndexed stateUpdated through an independent indexing pipeline.
notificationHistorical snapshotMay intentionally preserve what was rendered when the event occurred.
2. The User Service finishes before the platform converges
The authoritative profile has a narrow ownership boundary. Conceptually, the User Service owns userId, displayName, avatarId, avatarVersion, and profile metadata. When it commits a new avatar reference, its local operation can be complete even though downstream representations still show older data.
3. The image upload is its own workflow
The image bytes normally do not belong in the user database. Uploading, validating, scanning, resizing, encoding, storing, and distributing media have different capacity and failure characteristics from changing a profile row. Keeping those concerns separate also prevents a large upload from holding a profile database transaction open.
A safe workflow does not publish an avatar reference before the required variants are readable. If processing succeeds but the profile commit fails, the new object is unreferenced and can be removed later. If the profile commit succeeds, the platform has a durable pointer to an image that already exists.
Why immutable image references help
Replacing the bytes behind avatar.jpg asks every browser and cache to agree that the same URL now means something different. That is difficult to reason about. A new upload can instead produce a new object such as avatars/42/v43/medium.webp. The profile record moves from version 42 to version 43, while old objects remain valid for the responses that still reference them.
v42Old immutable objectSafe for older snapshots until they expire.profile pointerMoves atomicallyThe current avatar becomes version 43.v43New immutable objectBrowsers and CDN edges cache it under a new key.4. Snapshot versus reference
The most consequential choice is often not the cache technology. It is whether another surface copies profile fields or stores only a user identifier. A materialized feed entry might contain authorName and authorAvatar. Another design keeps only authorId and resolves the author summary during the read.
Snapshots keep the feed fast and available, but copied profile fields can remain stale until refreshed.
| Model | Read path | Profile propagation | Primary risk | Good fit |
|---|---|---|---|---|
| Snapshot | One local lookup | Refresh copied fields later | Visible staleness | Feeds, notifications, historical activity |
| Reference | Resolve profile during read | Current state appears quickly | Latency and dependency pressure | Profile surfaces, small result sets |
| Hybrid | Read snapshot, repair when stale | Version guided lazy refresh | More logic and observability | Large products with mixed freshness needs |
5. Read amplification versus update amplification
Resolving every author during every feed request pushes cost into the read path. Copying author summaries into feed entries pushes cost into profile updates and background repair. The architecture can move the cost, but it rarely makes the cost disappear.
reference modelRead amplificationMore lookups, network calls, connection pressure, cache traffic, and dependency risk on every feed request.
snapshot modelUpdate amplificationMore projection writes, event traffic, invalidation work, reconciliation, and temporary stale data after a profile change.
Avoid fifty profile calls for fifty posts
A feed that makes one User Service call per item creates the classic N plus 1 problem. Latency accumulates, duplicate authors are fetched repeatedly, connection pools fill, and a profile service incident becomes a feed incident. Useful alternatives include batch profile lookup, request scoped deduplication, a local profile cache, and a materialized author summary owned by the feed domain.
6. Why not rewrite every old post?
Consider a popular account with years of posts, comments across thousands of discussions, recommendation candidates, notifications, cached feeds, and millions of followers. Rewriting every representation before acknowledging the profile change would turn one small user mutation into a platform wide transaction.
databaseWrite amplificationMillions of projection rows compete with product traffic.brokerEvent amplificationQueues carry repair work long after the user waits.cacheInvalidation discoveryThe platform may not know every key containing that user.searchIndex churnDocuments are rewritten for cosmetic data.networkRegional trafficReplicated updates cross zones and regions.recoveryPartial completionA failed global rewrite needs checkpoints and reconciliation.Making the user wait for that work would increase latency and reduce availability for little product value. The better boundary is usually clear: confirm the change after the authoritative record commits, then let secondary representations converge according to explicit freshness objectives.
7. Profile changes propagate as events
After the authoritative commit, an event such as AvatarChanged lets independent consumers update at their own pace. The Feed Service may refresh author summaries quickly. Search may accept a longer indexing delay. Historical notifications may keep their original snapshot until expiration.
feedRefresh summarieslag 2ssearchUpdate documentslag 18scacheInvalidate profile keylag 600msrecommendationRefresh candidateslag 7snotificationsKeep or refresh snapshotpolicy basedDifferent lag does not imply lost data. It becomes a problem when a consumer exceeds its product freshness objective or stops converging.
8. Eventual consistency is a business decision
Eventual consistency is useful here because an older avatar is usually a temporary cosmetic defect. The same tolerance would be reckless for wallet balances, payment settlement, authorization, security permissions, or account suspension. Consistency follows the business invariant.
Temporary disagreement can be acceptable when it is bounded and observable.
The read or write boundary must enforce current policy even when projections lag.
9. Cache invalidation is a graph problem
The browser, CDN, API gateway, distributed cache, application process, feed cache, and search index can each hold a representation of the same user. Every layer has its own key structure, expiration policy, and owner. Invalidating profile:42 is easy. Discovering every feed page and notification payload that embeds User 42 may be more expensive than allowing bounded expiration.
Targeted invalidation refreshes known profile keys while embedded historical snapshots converge separately.
Versioning gives consumers another option. A cached author summary can carry avatarVersion: 42. When a nearby projection knows that version 43 exists, it can repair the stale entry lazily instead of starting a massive global invalidation job. This adds metadata and comparison logic, so it is useful only where the reduction in update traffic justifies the complexity.
Popular users and the thundering herd
Invalidating every cache for a globally popular account can cause millions of requests to miss together and fall through to backing services. Staggered expiration, request coalescing, background refresh, controlled invalidation, local caching, and serving stale data while a refresh runs all limit that shock. Popularity can be part of the propagation policy because equal treatment does not always create equal cost.
10. Failure handling starts at the commit boundary
Updating the user row and then publishing AvatarChanged creates a failure window. The database commit can succeed while event publication fails. The user sees success, but no projection learns about version 43. A transactional outbox stores the profile mutation and publication intent in the same local transaction. An independent relay can publish the committed event until it is acknowledged.
The outbox prevents lost intent, but it permits duplicate delivery. Every consumer still needs an idempotent update rule. Applying version 43 twice should leave the same result as applying it once.
A sequence owned by the profile aggregate is safer than assuming wall clock timestamps always arrive with useful precision and ordering.
Operational failure review
The relay retries durable publication intent.
The same version produces one projection state.
The projection rejects the stale event.
Backlog age becomes visible and drains after recovery.
Operators inspect, repair, and replay with an audit trail.
A scheduled comparison repairs state from the source of truth.
11. Deletes, suspension, and moderation need different urgency
Replacing an avatar is cosmetic. Removing an abusive image, suspending an account, changing privacy, or deleting the account can carry safety and legal obligations. Those events may need priority queues, direct policy checks during reads, stronger cache invalidation, or blocking at the image delivery layer.
12. Freshness objectives should match the surface
The product should define what healthy convergence means before the architecture selects timers and queues. The authoritative profile update should feel immediate. Active feed surfaces may need fast convergence. Search can often accept a wider window. Historical cached content may use bounded expiration. These are product objectives, not universal constants.
authoritative_profileImmediate user confirmationSuccess follows a durable profile commit.active_surfacesFast convergenceCurrent feeds and comments refresh promptly.search_projectionMeasured indexing delayFreshness follows search product needs.historical_snapshotsBounded eventual convergenceOld content refreshes lazily or expires.13. Behavior across geographical regions
A profile update may have one write home while reads are served close to users. The owning region commits version 43 and publishes the change. Other regions receive it asynchronously. CDN edges fill the new immutable object independently, and regional projections can show different versions for a short period.
write homeRegion Acommit v43servingRegion Bv42 → v43servingRegion Cv42 → v43Regional ownership and conflict rules matter if profile updates can be accepted in more than one region. Availability does not remove the need for deterministic version ownership.
14. Operate propagation as a product capability
Engineers need to know whether the system is converging before users report it. Useful signals connect the authoritative commit to every downstream surface rather than measuring cache hit rate in isolation.
profile_commit_latencyTime to durably accept the new avatar referenceevent_publish_latencyOutbox age until broker acknowledgementconsumer_lagBacklog age for feed, search, and cache consumersprojection_ageDifference between source and stored profile versioninvalidation_successKnown cache keys refreshed successfullyimage_processing_latencyUpload acceptance through required variantsstale_profile_rateSampled responses serving an older avatar versionreconciliation_backlogDivergent projections waiting for repairTrace context can follow the upload, profile commit, outbox relay, broker, and representative consumers. A trace cannot cover millions of cached feed items, so aggregate version metrics and searchable correlation identifiers remain essential.
15. Spend consistency where the business needs it
Immediate global consistency for cosmetic profile information consumes database writes, cache invalidation traffic, search work, compute, storage IO, and network capacity between regions. It also adds deployment and recovery complexity. A CTO should ask what product value comes from updating every old avatar immediately and whether that value justifies the operational cost.
Prioritize the surface where the user just completed the action.
Use explicit objectives instead of promising instant global repair.
Assume partial failure and make recovery routine.
16. Let the architecture grow with the product
small productResolve profiles directlyA database join, batch lookup, or simple cache may be enough. Keep ownership clear and measure before adding projections.
growing productBatch and cacheAdd immutable image references, profile caches, request deduplication, and asynchronous search updates.
large productProject and reconcileUse materialized summaries, versioned events, regional projections, controlled invalidation, and repair tooling where the workload demands them.
Building the largest possible architecture on day one does not create foresight. It creates ownership and failure modes before the product has earned them. The design should preserve seams for growth while paying complexity only when real traffic, freshness, or availability requirements require it.
Changing a profile picture is easy. Deciding how that new state reaches every place where a person appears is the distributed systems problem. The old picture can remain briefly because the platform is choosing bounded inconsistency over a synchronous global rewrite. When that choice is explicit, measured, recoverable, and aligned with product risk, it is often the correct engineering decision.