EvoCoder – Decompose 2025-10-14 Client Processing Audit into Scoped /codex/ Ticke

codex/STATE.md

@@ -4,7 +4,12 @@ This table is the single source of truth for active and historical tickets. Keep
 
 | Ticket ID | Title | Status | Priority | Owner | PRs | Last Updated |
 |-----------|-------|--------|----------|-------|-----|--------------|
-| TKT-0001 | Audit client-side processing for server migration | Review | P1 | evocoder | — | 2025-10-14 |
+| TKT-0001 | Audit client-side processing for server migration | Done | P1 | evocoder | — | 2025-10-14 |
+| TKT-0002 | Server-side hydrate brain and wave pipelines | Backlog | P0 | evocoder | — | 2025-10-14 |
+| TKT-0003 | Server aggregate collection analytics surfaces | Backlog | P1 | evocoder | — | 2025-10-14 |
+| TKT-0004 | Offload CSV mapping tools to server pipelines | Backlog | P1 | evocoder | — | 2025-10-14 |
+| TKT-0005 | Replace notification polling with server-driven delivery | Backlog | P1 | evocoder | — | 2025-10-14 |
+| TKT-0006 | Centralise media and IPFS upload orchestration | Backlog | P1 | evocoder | — | 2025-10-14 |
 
 ## Usage Guidelines
 

codex/docs/2025-10-14-client-processing-audit.md

@@ -5,7 +5,7 @@ owner: @evocoder
 review_cadence: quarterly
 status: active
 related:
-  tickets: [TKT-0001]
+  tickets: [TKT-0001, TKT-0002, TKT-0003, TKT-0004, TKT-0005, TKT-0006]
   prs: []
 audience: internal-only
 ---
@@ -25,37 +25,47 @@ The team is shifting workload off the browser; this audit catalogues the highest
 - **Why move server-side:** Hydrating the feed requires multiple network calls before content appears, triggering redundant polling and large JSON payloads. Users on slower devices spend time running merge queues and dropout reconciliation, and every tab open duplicates the same work.
 - **Recommended migration:** Render the first page(s) of feed and active wave data in server components that call the REST API with the user's credentials. Stream incremental updates to the client via Server Components or server-sent events, leaving the browser responsible only for UI state and websocket playback. Consolidate eligibility/merge logic into an API layer so clients consume pre-shaped payloads.
 - **Dependencies / blockers:** Requires authenticated fetch support within Next.js `app/` routes and coordination with websocket subscription flows. Ensure telemetry is available server-side for feature gating.
+- **Follow-up ticket:** TKT-0002 – Server-side hydrate brain and wave pipelines.
 
 ### 2. Collection analytics surfaces (MemeLab, 6529 Gradient, timelines)
 - **Current behaviour:** Art collection pages download entire datasets with `fetchAllPages` and then sort/filter in React (`components/memelab/MemeLab.tsx:471-505`, `components/6529Gradient/6529Gradient.tsx:62-96`, `components/the-memes/MemePageTimeline.tsx:19-32`). Sorting, distinct list construction, and chart preparation all run on the client for every visitor.
 - **Why move server-side:** Fetching thousands of records and performing O(n log n) sorts per visitor causes long loading spinners and high memory use, especially on mobile. Because the data changes infrequently, recomputing it per request is wasteful.
 - **Recommended migration:** Shift aggregation to server handlers (Next.js route handlers or edge functions) that return pre-sorted slices and derived metrics. Cache popular queries (e.g., top artists, TDH rankings) and stream results into lightweight server components with pagination to avoid megabyte-sized payloads.
 - **Dependencies / blockers:** Need to design caching/invalidations for rapidly changing metrics (e.g., floor price). Consider background jobs to precompute aggregates.
+- **Follow-up ticket:** TKT-0003 – Server aggregate collection analytics surfaces.
 
 ### 3. CSV mapping tools (Delegation & Consolidation)
 - **Current behaviour:** Tools read user-uploaded CSVs, then download full delegation or consolidation datasets with `fetchAllPages` before merging client-side (`components/mapping-tools/DelegationMappingTool.tsx:20-152`, `components/mapping-tools/ConsolidationMappingTool.tsx:23-175`). The browser parses files, iterates over large arrays, and generates outbound CSVs synchronously.
 - **Why move server-side:** Large CSVs stall the UI and can exceed memory limits, while repeated API pagination from browsers stresses the public API and is hard to secure. The logic also exposes business rules (address matching, fallbacks) that should live server-side.
 - **Recommended migration:** Provide signed upload endpoints that ingest the CSV, perform lookups, and return a processed artifact (or email a download link). Parallelise lookups on the server and throttle API usage centrally. Keep the client limited to upload progress and result display.
 - **Dependencies / blockers:** Requires file-processing infrastructure (queue or worker) and authentication for long-running jobs. Ensure a download UX remains available for power users.
+- **Follow-up ticket:** TKT-0004 – Offload CSV mapping tools to server pipelines.
 
 ### 4. Notifications and activity polling loops
 - **Current behaviour:** Notification drawers and wave activity logs continuously poll REST endpoints using `useInfiniteQuery` and `refetchInterval` timers (`hooks/useNotificationsQuery.tsx:77-147`, `hooks/useWaveActivityLogs.ts:25-127`). Prefetchers bootstrap several pages ahead, so each browser tab maintains parallel polling streams.
 - **Why move server-side:** Polling wastes bandwidth, drains batteries, and scales poorly with concurrent users. Each tab rehydrates and filters identical data even when nothing changed.
 - **Recommended migration:** Transition to server-driven pushes (websocket fan-out or server-sent events) and compute diff logic server-side. Alternatively, expose a consolidated `/stream` endpoint the server component consumes and hydrates into context without repeated polling.
 - **Dependencies / blockers:** Depends on revamping notification infrastructure and ensuring authentication tokens are available to server processes.
+- **Follow-up ticket:** TKT-0005 – Replace notification polling with server-driven delivery.
 
 ### 5. Media and IPFS upload orchestration
 - **Current behaviour:** Large file uploads chunk, retry, and handle MIME inference in the browser (`components/waves/create-wave/services/multiPartUpload.ts:48-195`, `components/ipfs/IPFSService.ts:24-68`). Axios manages PUTs to presigned URLs; errors surface late to the user.
 - **Why move server-side:** Processing large files in-browser ties up memory, makes progress unreliable on throttled networks, and lacks centralized enforcement (virus scanning, quotas). Re-implementing chunk retries in every client platform is brittle.
 - **Recommended migration:** Move chunk management to a signed server endpoint that streams uploads into storage (or uses Upload Part Copy). Expose resumable upload tokens so the client only handles file selection and progress UI. Run validation and content-type detection server-side.
 - **Dependencies / blockers:** Requires storage gateway adjustments and potentially larger server egress budgets. Must protect against uploading untrusted files.
+- **Follow-up ticket:** TKT-0006 – Centralise media and IPFS upload orchestration.
 
 ## Suggested Next Steps
 
-- Define ownership for each migration track and capture sequencing in the roadmap (e.g., feed pipeline first, mapping tools as a separate workstream).
+- Define ownership for each migration track and capture sequencing in the roadmap (TKT-0002 through TKT-0006).
 - Prototype server-side feed hydration in a single route to validate authentication and caching strategy.
 - Instrument current payload sizes and latency to set success metrics for each migration.
 
 ## Backlinks
 
 - TKT-0001 – Audit client-side processing for server migration
+- TKT-0002 – Server-side hydrate brain and wave pipelines
+- TKT-0003 – Server aggregate collection analytics surfaces
+- TKT-0004 – Offload CSV mapping tools to server pipelines
+- TKT-0005 – Replace notification polling with server-driven delivery
+- TKT-0006 – Centralise media and IPFS upload orchestration

codex/tickets/TKT-0001.md

@@ -3,7 +3,7 @@ created: 2025-10-14
 id: TKT-0001
 owner: evocoder
 priority: P1
-status: In-Progress
+status: Done
 title: Audit client-side processing for server migration
 ---
 
@@ -19,17 +19,18 @@ title: Audit client-side processing for server migration
 
 ## Acceptance
 
-- [ ] Comprehensive list of client-side processing suitable for server migration, ranked by user impact.
-- [ ] Document stored under `codex/docs/` with clear action recommendations.
-- [ ] State board reflects latest ticket state and links to output document.
+- [x] Comprehensive list of client-side processing suitable for server migration, ranked by user impact.
+- [x] Document stored under `codex/docs/` with clear action recommendations.
+- [x] State board reflects latest ticket state and links to output document.
 
 ## Links
 
 - Primary PR: _(add when available)_
-- Follow-ups: _(reference additional tickets or TODO items)_
+- Follow-ups: TKT-0002, TKT-0003, TKT-0004, TKT-0005, TKT-0006
 
 ## Log
 
 - 2025-10-14T10:29:22Z – Ticket opened and initial context captured.
 - 2025-10-14T10:37:19Z – Completed client-side processing audit; drafted findings document.
 - 2025-10-14T10:37:39Z – Published audit in codex/docs and updated project board status.
+- 2025-10-14T10:49:39Z – Decomposed audit outcomes into follow-up tickets (TKT-0002–TKT-0006) and closed the audit ticket.

codex/tickets/TKT-0002.md

@@ -0,0 +1,34 @@
+---
+created: 2025-10-14
+id: TKT-0002
+owner: evocoder
+priority: P0
+status: Backlog
+title: Server-side hydrate brain and wave pipelines
+---
+
+## Context
+
+> Infinite queries and merge logic for the brain feed and wave experiences currently execute in the browser, duplicating work across tabs and delaying first contentful render. Moving hydration and merge queues to the server should reduce latency and simplify clients.
+
+## Plan
+
+- [ ] Align with platform team on authenticated server fetch strategy for feed data.
+- [ ] Build server components or route handlers that deliver initial feed and wave payloads.
+- [ ] Stream incremental updates (SSE/websocket) and remove client-side merge/retry queues.
+- [ ] Add telemetry covering server-side fetch latency and payload size regressions.
+
+## Acceptance
+
+- [ ] First-page feed rendering occurs server-side with no redundant client polling before content appears.
+- [ ] Wave merge and dropout reconciliation logic lives in shared server utilities consumed by clients.
+- [ ] Observability dashboards confirm latency and payload size improvements versus the client baseline.
+
+## Links
+
+- Primary PR: _(add when available)_
+- Follow-ups: _(reference additional tickets or TODO items)_
+
+## Log
+
+- 2025-10-14T10:49:39Z – Ticket created from the client processing audit to track server-side feed migration.

codex/tickets/TKT-0003.md

@@ -0,0 +1,34 @@
+---
+created: 2025-10-14
+id: TKT-0003
+owner: evocoder
+priority: P1
+status: Backlog
+title: Server aggregate collection analytics surfaces
+---
+
+## Context
+
+> Collection dashboards (MemeLab, 6529 Gradient, timelines) fetch thousands of records to the browser and perform expensive sorts and aggregations client-side, resulting in long spinners and heavy payloads. Centralising computation server-side enables caching and faster initial paint.
+
+## Plan
+
+- [ ] Catalogue required analytics outputs and normalise contracts across collection surfaces.
+- [ ] Implement server handlers that provide pre-sorted/aggregated datasets with pagination.
+- [ ] Introduce caching and invalidation for volatile metrics (floor price, recent sales, etc.).
+- [ ] Update React components to consume lightweight server payloads and drop redundant client transforms.
+
+## Acceptance
+
+- [ ] Collection pages load using server-provided datasets under agreed latency targets on cold cache.
+- [ ] Cached responses or background jobs avoid downloading full datasets per visitor.
+- [ ] Client bundles no longer include the legacy `fetchAllPages` loops for these surfaces.
+
+## Links
+
+- Primary PR: _(add when available)_
+- Follow-ups: _(reference additional tickets or TODO items)_
+
+## Log
+
+- 2025-10-14T10:49:39Z – Ticket opened from audit findings to scope server-side collection analytics.

codex/tickets/TKT-0004.md

@@ -0,0 +1,34 @@
+---
+created: 2025-10-14
+id: TKT-0004
+owner: evocoder
+priority: P1
+status: Backlog
+title: Offload CSV mapping tools to server pipelines
+---
+
+## Context
+
+> Delegation and consolidation mapping tools parse large CSVs in-browser, then paginate through full datasets with `fetchAllPages`, causing UI stalls, memory pressure, and duplicated business rules. Moving processing server-side will improve reliability and secure long-running jobs.
+
+## Plan
+
+- [ ] Design authenticated upload workflow (signed URLs or direct POST) and queue semantics.
+- [ ] Build server workers that merge CSV inputs with delegation/consolidation datasets and emit results.
+- [ ] Expose download or notification flows for processed artifacts and update client UX accordingly.
+- [ ] Implement throttling/monitoring to protect upstream APIs during batch processing.
+
+## Acceptance
+
+- [ ] CSV ingestion, lookup, and merging execute on the server without blocking the browser.
+- [ ] Client-side tools limit work to upload progress, status polling, and result presentation.
+- [ ] Operational runbooks cover scaling limits, retries, and user support for failed jobs.
+
+## Links
+
+- Primary PR: _(add when available)_
+- Follow-ups: _(reference additional tickets or TODO items)_
+
+## Log
+
+- 2025-10-14T10:49:39Z – Ticket captured to migrate CSV mapping workloads off the client per the audit.

codex/tickets/TKT-0005.md

@@ -0,0 +1,34 @@
+---
+created: 2025-10-14
+id: TKT-0005
+owner: evocoder
+priority: P1
+status: Backlog
+title: Replace notification polling with server-driven delivery
+---
+
+## Context
+
+> Notification drawers and wave activity logs rely on `useInfiniteQuery` polling loops with 5 s timers, leading to duplicated network traffic per tab and wasted battery. Consolidating delivery server-side enables push-based updates and shared diff logic.
+
+## Plan
+
+- [ ] Evaluate websocket vs server-sent events for notification delivery and authentication.
+- [ ] Build consolidated server endpoints or streams that emit diffed notification payloads.
+- [ ] Update client hooks to consume push-driven updates and remove polling timers.
+- [ ] Roll out backoff/health monitoring to catch delivery regressions.
+
+## Acceptance
+
+- [ ] Clients stop issuing periodic REST polls for notifications or wave activity.
+- [ ] Server components or shared stores surface push notifications with correct auth scoping.
+- [ ] Monitoring confirms reduced duplicate traffic and stable delivery latency.
+
+## Links
+
+- Primary PR: _(add when available)_
+- Follow-ups: _(reference additional tickets or TODO items)_
+
+## Log
+
+- 2025-10-14T10:49:39Z – Ticket spawned from audit recommendations to eliminate notification polling.

codex/tickets/TKT-0006.md

@@ -0,0 +1,34 @@
+---
+created: 2025-10-14
+id: TKT-0006
+owner: evocoder
+priority: P1
+status: Backlog
+title: Centralise media and IPFS upload orchestration
+---
+
+## Context
+
+> Large media uploads currently chunk, retry, and infer metadata entirely in the browser via presigned URLs, which is brittle on slow networks and lacks server-side validation. Delegating orchestration to a server gateway will improve reliability and security controls.
+
+## Plan
+
+- [ ] Design a resumable upload service that handles chunking, retries, and content-type detection server-side.
+- [ ] Integrate virus scanning, quota enforcement, and logging into the upload pipeline.
+- [ ] Update client flows to request upload tokens, stream files to the gateway, and surface progress.
+- [ ] Provide migration playbook for existing presigned direct upload callers.
+
+## Acceptance
+
+- [ ] Browser clients delegate chunk management to the server gateway while retaining progress UX.
+- [ ] Server uploads enforce validation (size, MIME, scanning) before persisting artifacts.
+- [ ] Observability dashboards capture success/error rates for the new upload path.
+
+## Links
+
+- Primary PR: _(add when available)_
+- Follow-ups: _(reference additional tickets or TODO items)_
+
+## Log
+
+- 2025-10-14T10:49:39Z – Ticket created to track server-managed media and IPFS upload orchestration.