Firebase Functions Architecture

The Soku backend lives in apps/functions/ and runs as Firebase Cloud Functions on Node.js 22. It handles OAuth for eight social platforms, content publishing and orchestration, a public REST API, webhook processing, media processing, AI features, and scheduled maintenance.

Technology Overview

Technology	Version	Purpose
Node.js	22	Runtime
firebase-functions	6.x	Cloud Functions framework
Firebase Admin SDK	12.x	Firestore, Auth, Storage
Express	5.x	Public API HTTP framework
Stripe	14.0.0	Payment webhook processing
OpenAI SDK	4.56.0	AI transcription and content
Sharp	0.34.5	Image processing and thumbnails
FFmpeg (fluent-ffmpeg + ffmpeg-static)	2.1 / 5.2	Video processing
Pino	9.9	Structured logging
Zod (via @soku/schema)	3.22+	Request/response validation
@google-cloud/tasks	6.2	Cloud Tasks for async publishing
googleapis	144.x	YouTube Data API
got	13.x	HTTP client

Directory Structure

apps/functions/
├── index.js                    # All function exports (entry point)
├── config.js                   # Firebase Admin init, provider credentials, CORS
├── api.js                      # Express app for public API
├── ping.js                     # Health check function
├── adapters/
│   └── gcp/                    # GCP-specific adapters (Firestore helpers)
├── http/
│   ├── handlers/               # Standalone HTTPS function handlers
│   │   ├── cancel-scheduled-post.js
│   │   ├── convert-image-to-video.js
│   │   ├── media-proxy.js
│   │   ├── publish-post.js
│   │   ├── render-og-template.js
│   │   ├── report-ai-usage.js
│   │   └── transcribe-media.js
│   └── routes/                 # Express route modules for public API
│       ├── ai.js
│       ├── api-keys.js
│       ├── media-upload.js
│       ├── posts.js
│       └── templates.js
├── integrations/               # Per-platform OAuth, publish, poll, metrics
│   ├── facebook/
│   ├── instagram/
│   ├── threads/
│   ├── youtube/
│   ├── tiktok/
│   ├── x/
│   ├── linkedin/
│   ├── snapchat/
│   ├── health.js               # Integration health sweep
│   ├── integration-avatar-trigger.js
│   ├── organic-post-trigger.js
│   └── workflow-source-sync.js
├── middleware/                  # Express and function middleware
│   ├── auth.js                 # API key auth, ID token auth, subscription check
│   ├── error-handler.js        # Error classes and Express error handler
│   ├── rate-limit.js           # Tiered rate limiting with Firestore backend
│   └── validation.js           # Request validation middleware
├── orchestrators/              # Publishing orchestration
│   ├── publish-orchestrator.js
│   ├── repost-orchestrator.js
│   ├── clipper-orchestrator.js
│   └── publishing-service.js
├── scheduled/                  # Scheduled (cron) functions
│   ├── daily-metrics-snapshot.js
│   └── rate-limit-cleanup.js
├── services/                   # Business logic services
│   ├── save-media-buffer.js
│   ├── save-media-from-url.js
│   ├── templates/
│   └── transcriptions.js
├── shared/                     # Shared utilities
│   ├── logging.js              # Pino logger factory
│   ├── event-logger.js         # Tracing and span utilities
│   ├── ai/                     # OpenAI integration
│   ├── credits.js              # AI credit tracking
│   ├── http.ts                 # HTTP client helpers
│   ├── ids.ts                  # ID generation
│   ├── integrations.js         # Cross-platform integration helpers
│   ├── integration-schema.js   # Integration document schema
│   ├── internal-auth.js        # Internal service-to-service auth
│   ├── media/                  # Media processing utilities
│   ├── metrics.js              # Metrics collection helpers
│   ├── post-doc.js             # Post document helpers
│   ├── post-processor.js       # Post processing pipeline
│   ├── runs.js                 # Publishing run tracking
│   ├── targets.js              # Platform target resolution
│   ├── token-refresh.js        # OAuth token refresh utilities
│   ├── validate-oauth-state.js # OAuth state validation
│   ├── validate-redirect.js    # Redirect URL validation
│   ├── validate-url.js         # URL validation
│   ├── workflow-account-cleanup.js
│   ├── workflow-dispatcher.js
│   └── workflows.js
├── triggers/                   # Firestore document triggers
│   └── library-thumbnail.js
├── types/                      # TypeScript type definitions
│   └── types.ts
└── webhooks/                   # Webhook handlers
    ├── meta.js                 # Meta (Facebook/Instagram) webhooks
    └── stripe.js               # Stripe webhook handler

Complete Function Exports

Every exported function from index.js is a deployed Cloud Function. They are organized below by category.

Infrastructure

Export	Type	Description
ping	HTTPS	Health check endpoint
api	HTTPS (Express)	Public REST API (/v1/*)
media	HTTPS	Media proxy for serving stored assets

Webhooks

Export	Type	Description
metaWebhook	HTTPS	Meta platform webhook verification and event handling
stripeWebhook	HTTPS	Stripe event processing (subscription changes, payments)

Orchestrators

Export	Type	Description
publishOrchestrator	HTTPS	Receives publish requests and fans out to per-platform publish functions via Cloud Tasks
repostOrchestrator	HTTPS	Handles repost/reshare requests across platforms
clipperOrchestrator	HTTPS	Handles video clipping and cross-platform distribution

Facebook

Export	Type	Description
fbAuthStart	HTTPS	Initiates Facebook OAuth flow
fbAuthCallback	HTTPS	Handles Facebook OAuth callback
publishFacebook	HTTPS	Publishes content to Facebook
fbMetricsCollector	HTTPS	Collects Facebook post metrics
scheduledFacebookMetrics	Scheduled	Periodic Facebook metrics collection
onFacebookAccountCreated	Firestore Trigger	Runs when a Facebook integration is created

Instagram

Export	Type	Description
igAuthStart	HTTPS	Initiates Instagram OAuth flow
igAuthCallback	HTTPS	Handles Instagram OAuth callback
publishInstagram	HTTPS	Publishes content to Instagram
pollInstagram	HTTPS	On-demand poll for Instagram publish status
pollerInstagram	HTTPS	Polling worker for Instagram publish completion
scheduledInstagramPoll	Scheduled	Periodic Instagram publish status polling
onInstagramAccountCreated	Firestore Trigger	Runs when an Instagram integration is created
igDataDeletion	HTTPS	Instagram data deletion callback (GDPR)
igDeauth	HTTPS	Instagram deauthorization callback

Threads

Export	Type	Description
thAuthStart	HTTPS	Initiates Threads OAuth flow
thAuthCallback	HTTPS	Handles Threads OAuth callback
publishThreads	HTTPS	Publishes content to Threads
thInsightsTest	HTTPS	Threads insights testing endpoint
thMetricsCollector	HTTPS	Collects Threads post metrics
scheduledThreadsMetrics	Scheduled	Periodic Threads metrics collection

YouTube

Export	Type	Description
ytAuthStart	HTTPS	Initiates YouTube OAuth flow
ytAuthCallback	HTTPS	Handles YouTube OAuth callback
ytRefreshNow	HTTPS	On-demand YouTube token refresh
ytRefreshSweep	Scheduled	Periodic sweep to refresh expiring YouTube tokens
publishYoutube	HTTPS	Publishes content to YouTube
pollYoutube	HTTPS	On-demand poll for YouTube upload status
pollerYoutube	HTTPS	Polling worker for YouTube upload completion
scheduledYouTubePoll	Scheduled	Periodic YouTube upload status polling
fetchYouTubeCaptions	HTTPS	Fetches captions from a YouTube video

TikTok

Export	Type	Description
ttAuthStart	HTTPS	Initiates TikTok OAuth flow
ttAuthCallback	HTTPS	Handles TikTok OAuth callback
ttRefreshNow	HTTPS	On-demand TikTok token refresh
ttRefreshSweep	Scheduled	Periodic sweep to refresh expiring TikTok tokens
ttCreatorInfo	HTTPS	Fetches TikTok creator info (privacy settings, etc.)
publishTiktok	HTTPS	Publishes content to TikTok
pollTiktok	HTTPS	On-demand poll for TikTok publish status
pollerTiktok	HTTPS	Polling worker for TikTok publish completion
scheduledTikTokPoll	Scheduled	Periodic TikTok publish status polling
onTiktokAccountCreated	Firestore Trigger	Runs when a TikTok integration is created
ttMetricsCollector	HTTPS	Collects TikTok post metrics
scheduledTikTokMetrics	Scheduled	Periodic TikTok metrics collection

X (Twitter)

Export	Type	Description
xAuthStart	HTTPS	Initiates X OAuth 2.0 PKCE flow
xAuthCallback	HTTPS	Handles X OAuth 2.0 callback
xAuth1Start	HTTPS	Initiates X OAuth 1.0a flow (for v1.1 media upload)
xAuth1Callback	HTTPS	Handles X OAuth 1.0a callback
publishX	HTTPS	Publishes content to X
pollX	HTTPS	On-demand poll for X publish status
pollerX	HTTPS	Polling worker for X publish completion
scheduledXPoll	Scheduled	Periodic X publish status polling
onXAccountCreated	Firestore Trigger	Runs when an X integration is created

LinkedIn

Export	Type	Description
liAuthStart	HTTPS	Initiates LinkedIn OAuth flow
liAuthCallback	HTTPS	Handles LinkedIn OAuth callback
publishLinkedin	HTTPS	Publishes content to LinkedIn
onLinkedinAccountCreated	Firestore Trigger	Runs when a LinkedIn integration is created

Snapchat

Export	Type	Description
snapAuthStart	HTTPS	Initiates Snapchat OAuth flow
snapAuthCallback	HTTPS	Handles Snapchat OAuth callback
snapRefreshNow	HTTPS	On-demand Snapchat token refresh
snapRefreshSweep	Scheduled	Periodic sweep to refresh expiring Snapchat tokens
snapPost	HTTPS	Publishes content to Snapchat

Cross-Platform Triggers

Export	Type	Description
onOrganicPostCreated	Firestore Trigger	Fires when a post document is created; handles cross-platform post tracking
onIntegrationAccountAvatarChange	Firestore Trigger	Fires when an integration account's avatar changes; re-hosts the avatar
onAccountDeleted	Firestore Trigger	Cleans up workflows and data when a user account is deleted
onWorkflowWritten	Firestore Trigger	Syncs workflow source account references when a workflow is created or updated
onWorkflowDeleted	Firestore Trigger	Cleans up when a workflow document is deleted
onTemplateWritten	Firestore Trigger	Processes template documents when created or updated
onLibraryAssetUploaded	Firestore Trigger	Generates thumbnails when a library asset is uploaded

Standalone HTTP Handlers

Export	Type	Description
publishPost	HTTPS	Direct publish endpoint (bypasses orchestrator)
transcribeMedia	HTTPS	Transcribes audio/video using OpenAI Whisper
reportAiUsage	HTTPS	Reports AI credit consumption
cancelScheduledPost	HTTPS	Cancels a scheduled post
convertImageToVideo	HTTPS	Converts a static image to a video clip

Scheduled Functions

Export	Type	Description
integrationsHealthSweep	Scheduled	Checks health of all connected integrations across users
scheduledRateLimitCleanup	Scheduled	Purges stale rate limit documents from Firestore
scheduledDailyMetricsSnapshot	Scheduled	Takes daily snapshots of platform metrics

Express API Structure

The public API is an Express 5 application exported as the api Cloud Function. It is served at /v1/* via Firebase Hosting rewrites.

Middleware Stack (Applied in Order)

1. Security headers: Sets X-Content-Type-Options, X-Frame-Options, Referrer-Policy, X-DNS-Prefetch-Control, and Strict-Transport-Security.
2. Body parser: express.json() for JSON request bodies.
3. CORS: Origin allowlist (mysoku.io, www.mysoku.io, stg.mysoku.io, localhost:3000). Sets Access-Control-Allow-Origin only for allowed origins with Vary: Origin.
4. Request ID: Attaches a unique request ID to each request for tracing.
5. Subscription check: Reads the user's subscription status from Firestore and attaches it to req.subscription for downstream tier detection.
6. Rate limiter: Sliding-window rate limiting backed by Firestore transactions. Tier-based limits: default (25/min), authenticated (60/min), premium (100/min), admin (200/min). Each tier includes a burst allowance. Rate limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-RateLimit-Tier) are set on every response.

Route Modules

Mount Point	Module	Description
/v1/posts	routes/posts.js	CRUD for posts via the public API
/v1/media	routes/media-upload.js	Media upload endpoints
/v1/api-keys	routes/api-keys.js	API key management
/v1/ai	routes/ai.js	AI-powered content generation
/v1/templates	routes/templates.js	Template CRUD

A /test endpoint at the root provides a simple health check. Unmatched routes return a 404 via a NotFoundError. The Express error handler is mounted last and formats all errors consistently.

Authentication Middleware

The middleware/auth.js module provides three authentication strategies:

• authenticateApiKey: Reads the soku-api-key header, SHA-256 hashes it, and looks up the hash in the apiKeys Firestore collection. Rejects if missing, invalid, or revoked.
• requireIdToken: Reads Authorization: Bearer <token>, verifies the Firebase ID token via Admin SDK, and attaches req.userId.
• requireActiveSubscription: Checks the user's users/{uid}/subscription/current document for an active or trialing status within the current billing period.

Route modules compose these middleware as needed. Public API routes typically use authenticateApiKey, while user-scoped endpoints use requireIdToken.

OAuth Flow Pattern

All eight platforms follow the same two-function OAuth pattern:

*AuthStart (Initiation)

1. Receives an HTTPS request with the user's Firebase ID token in the Authorization header.
2. Verifies the ID token via Admin SDK to get the uid.
3. Generates a random state string.
4. Writes an oauthStates/{state} document to Firestore containing { uid, provider, returnTo, createdAt }.
5. Constructs the platform-specific authorization URL with the appropriate scopes, callback URL, and state parameter.
6. Returns the authorization URL to the client.

*AuthCallback (Completion)

1. Receives the OAuth redirect from the platform with code and state query parameters.
2. Reads and validates the oauthStates/{state} document from Firestore. Deletes it immediately to prevent reuse.
3. Exchanges the authorization code for access and refresh tokens using the platform's token endpoint.
4. Fetches the user's profile information from the platform API.
5. Writes the tokens, profile metadata, and connection status to users/{uid}/integrations/{provider} in Firestore.
6. Redirects the user back to the returnTo URL (typically the settings page).

Token Refresh

Platforms with expiring tokens (TikTok, YouTube, Snapchat, X) implement two refresh mechanisms:

• *RefreshNow: On-demand refresh triggered by the client when a token is known to be expired.
• *RefreshSweep: Scheduled function that queries all integrations for a given platform, identifies tokens expiring soon, and refreshes them proactively.

The shared/token-refresh.js module provides common token refresh logic used across platforms.

X (Twitter) Special Case

X supports two OAuth flows because the platform requires different auth mechanisms for different API versions:

• OAuth 2.0 PKCE (xAuthStart/xAuthCallback): Used for the v2 API (posting, reading).
• OAuth 1.0a (xAuth1Start/xAuth1Callback): Used for the v1.1 API (media upload). Requires consumer key and consumer secret in addition to per-user tokens.

Orchestrator Pattern

Orchestrators decouple the "what to publish" decision from the "how to publish" execution. They are HTTPS Cloud Functions that receive a publish request and fan out work to per-platform publish functions.

Publish Flow

Client (web app)
  │
  ▼
publishOrchestrator (HTTPS, 2GiB memory)
  │
  ├─ Validates request payload
  ├─ Resolves target platforms from post configuration
  ├─ For each platform:
  │   └─ Enqueues a Cloud Task targeting the platform's publish function
  │      (e.g., publishFacebook, publishInstagram, publishX, etc.)
  ├─ Writes initial run state to Firestore
  └─ Returns immediately with run tracking info
        │
        ▼
  Cloud Tasks execute per-platform publish functions
        │
        ├─ publishFacebook   → Facebook Graph API
        ├─ publishInstagram  → Instagram Content Publishing API
        ├─ publishThreads    → Threads API
        ├─ publishYoutube    → YouTube Data API
        ├─ publishTiktok     → TikTok Content Posting API
        ├─ publishX          → X API v2
        ├─ publishLinkedin   → LinkedIn API
        └─ snapPost          → Snapchat API
              │
              ▼
        Each publish function updates its run state in Firestore

The publishOrchestrator uses the @google-cloud/tasks library to enqueue Cloud Tasks. Each task carries the post data, target account credentials, and a trace context for distributed tracing. The publishing-service.js module contains the shared orchestration logic.

The repostOrchestrator follows the same pattern but handles resharing existing content to additional platforms. The clipperOrchestrator handles video clipping workflows where a long-form video is cut into clips and distributed.

Event Tracing

Orchestrators use shared/event-logger.js to create distributed traces. A trace is created at the start of the orchestrator and passed to each Cloud Task via the request body. Each per-platform publish function restores the trace context and creates child spans, enabling end-to-end visibility across the async publish pipeline.

Polling Pattern

Some platforms (Instagram, YouTube, TikTok, X) do not provide instant confirmation of publish success. For these, a three-tier polling pattern is used:

1. poll*: An on-demand HTTPS function the client can call to check the current status of a publish operation.
2. poller*: A worker function that performs the actual status check against the platform API and updates the run state in Firestore.
3. scheduled*Poll: A scheduled function that runs periodically to poll all pending publish operations for a given platform.

Scheduled Functions

Function	Description
scheduledFacebookMetrics	Collects engagement metrics for recent Facebook posts
scheduledInstagramPoll	Polls pending Instagram publishes for completion
scheduledThreadsMetrics	Collects engagement metrics for recent Threads posts
scheduledYouTubePoll	Polls pending YouTube uploads for processing completion
scheduledTikTokPoll	Polls pending TikTok publishes for completion
scheduledTikTokMetrics	Collects engagement metrics for recent TikTok posts
scheduledXPoll	Polls pending X posts for completion
ytRefreshSweep	Refreshes expiring YouTube tokens
ttRefreshSweep	Refreshes expiring TikTok tokens
snapRefreshSweep	Refreshes expiring Snapchat tokens
integrationsHealthSweep	Validates token health across all connected accounts
scheduledRateLimitCleanup	Deletes rate limit documents older than 24 hours
scheduledDailyMetricsSnapshot	Takes daily aggregate metrics snapshots

Firestore Triggers

Firestore triggers react to document changes and perform side effects:

Function	Trigger	Description
onOrganicPostCreated	Document create	Tracks organic (non-Soku) posts discovered via platform APIs
onIntegrationAccountAvatarChange	Document update	Re-hosts integration account avatars to Soku storage
onAccountDeleted	Document delete	Cleans up workflows, integrations, and data for deleted accounts
onWorkflowWritten	Document write	Syncs workflow source account references
onWorkflowDeleted	Document delete	Cleans up workflow-related data
onTemplateWritten	Document write	Processes and indexes template documents
onLibraryAssetUploaded	Document create	Generates thumbnails for newly uploaded library assets
onFacebookAccountCreated	Document create	Post-connection setup for Facebook accounts
onInstagramAccountCreated	Document create	Post-connection setup for Instagram accounts
onTiktokAccountCreated	Document create	Post-connection setup for TikTok accounts
onXAccountCreated	Document create	Post-connection setup for X accounts
onLinkedinAccountCreated	Document create	Post-connection setup for LinkedIn accounts

Configuration and Initialization

config.js

The configuration module handles three concerns:

1. Firebase Admin initialization: Initializes the Admin SDK once with the project ID and storage bucket from environment variables. Configures Firestore with ignoreUndefinedProperties: true.

2. Provider credentials: The cfg() function returns a lazily-initialized configuration object containing OAuth credentials (app IDs, secrets, callback URLs) for all eight platforms plus the OpenAI API key. Credentials come from environment variables.

3. Configuration validation: Three validation levels based on the environment:

   • None (emulator): No validation, all credentials optional.
   • Basic (development): Warns about missing credentials but continues.
   • Strict (production): Throws if any required credential is missing.

4. CORS: An origin allowlist (mysoku.io, www.mysoku.io, stg.mysoku.io, localhost:3000) is enforced by both the cors() helper (for individual Cloud Functions) and the corsMiddleware (for the Express app).

Logging

All modules use the shared/logging.js Pino logger factory. Each module creates a named logger via createLogger('module-name') for structured, filterable log output.

Security Model

Public API Authentication

The Express API supports two authentication strategies, applied per-route:

• API key authentication: The soku-api-key header is SHA-256 hashed and looked up in the apiKeys Firestore collection. The key document contains the owning userId, allowing requests to be scoped to the correct user.
• Firebase ID token authentication: The Authorization: Bearer <token> header is verified via Admin SDK. Used for user-scoped operations like API key management.

Rate Limiting

Rate limiting uses a sliding-window counter backed by Firestore transactions. Each request's timestamp is added to an array in a rateLimits/{key} document. Timestamps outside the current window are pruned. Four tiers provide escalating limits:

Tier	Max Requests/min	Burst Allowance
Default (unauthenticated)	25	10
Authenticated (API key)	60	30
Premium (active subscription)	100	100
Admin (whitelisted)	200	500

Rate limit response headers are set on every response. The scheduledRateLimitCleanup function purges stale documents older than 24 hours.

Webhook Verification

• Stripe: Verifies webhook signatures using STRIPE_WEBHOOK_SECRET. Does not require user authentication.
• Meta: Handles Facebook/Instagram webhook verification challenges and validates event signatures.
• Instagram GDPR: igDataDeletion and igDeauth handle Instagram's data deletion and deauthorization callbacks.

OAuth State Security

OAuth state parameters are stored in Firestore with the initiating user's uid. On callback, the state document is validated and immediately deleted to prevent replay attacks. Redirect URLs are validated by shared/validate-redirect.js to prevent open redirect vulnerabilities.

CORS

CORS is restricted to a specific origin allowlist. The Access-Control-Allow-Origin header is only set when the request origin matches an allowed domain. The Vary: Origin header ensures proper caching behavior.

Security Headers

The Express API sets the following security headers on all responses:

• X-Content-Type-Options: nosniff
• X-Frame-Options: DENY
• Referrer-Policy: strict-origin-when-cross-origin
• X-DNS-Prefetch-Control: off
• Strict-Transport-Security: max-age=63072000; includeSubDomains