The Umamii Newsletter Ingestion Pipeline: Curation, Costs, and Code-Level Crux
Local restaurants run dynamic weekly events, launch seasonal menus, and host holiday specials. But restaurant managers are far too busy to log into a B2B portal to publish these updates. This friction creates a common problem: consumer apps starve for fresh content while local spots struggle to get the word out.
However, we noticed a massive opportunity. Restaurants already broadcast these updates weekly. They write beautifully structured, highly visual newsletters using platforms like Mailchimp and Substack.
Instead of asking owners to do double entry, we set out to build an automated “Auto-Broadcaster” pipeline. This system pulls weekly newsletters from Gmail, extracts structured event drafts using LLMs, maps them to database venue profiles, and surfaces them in a B2B Partner Portal.
Fully automated publishing is a non-starter; preserving brand standards requires a human-in-the-loop review process.
By surfacing extracted email drafts inside a visual editor, we give admins and owners complete control. They can perfect the copy and preview it exactly as it will look on the consumer app before pushing it live.
Ground-Floor Innovation with Antigravity
Building a pipeline like this requires modeling robust database schemas, writing Express Backend-for-Frontend (BFF) API routes, and designing interactive React components. To hit our launch window, we partnered closely with Antigravity’s autonomous AI coding agents.
During the development cycle, we experienced the tech transition when Antigravity upgraded to version 2.0. The new agent-management-first framework completely changed how we worked.
Instead of waiting for a single agent to handle code edits, tests, and database schema updates sequentially, the new framework allowed us to spin up specialized subagents.
Transitioning to an agent-management-first framework allowed us to iterate on complex portal features in minutes instead of days.
We delegated the database migration and PostgreSQL trigger design to one subagent while another handled the Express API routes. This parallel collaboration cut down manual debugging cycles, letting us move from concept to code-complete in record time.
The Grand Architectural Pivot: Breaking the Monolith
Our first design was a monolithic Gumloop workflow. It fetched emails, parsed their heavy HTML structures, mapped them to our database using fuzzy matching, drafted the copy, and monitored active Slack threads for human approval replies.
It worked, but we quickly hit a massive cost barrier.
Gumloop grants 5,000 free execution tokens per month. Our monolithic workflow consumed over 200 tokens per email, restricting the pipeline to only 25 newsletters a month. Before committing to expensive paid subscriptions, we needed a cost-efficient strategy that scaled.
Our target was to reduce token usage down to 50 tokens per email. We achieved this by completely re-architecting the system and offloading heavy computational and creative tasks out of the ingestion agent.
By offloading brand copywriting and entity matching to our Express backend, we slashed Gumloop token consumption by 75%.
[ Gmail Queue ]
│
▼
┌──────────────────────────────┐
│ Orchestrator Agent (Gumloop)│ ◄── Deduplicates queue
└──────────────────────────────┘
│
├───────────────────────┐
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Subagent 1 │ │ Subagent 2 │ ◄── Parallelized processing
└─────────────┘ └─────────────┘
│ │
└───────────┬───────────┘
▼
┌──────────────────────────────┐
│ Express BFF Proxy API │ ◄── Creative rewriting, entity matching
└──────────────────────────────┘First, we split the monolithic agent into an Orchestrator + Subagent Model using Gemini 3.5 Flash. The Orchestrator fetches the email queue, deduplicates it, and spawns parallel subagents to process each newsletter. Parallelization prevents a single run from getting bogged down and mitigates LLM hallucinations.
Second, we offloaded all brand-voice rewriting, dynamic category matching, and layout configurations from the LLM workflow to our React portal and Express backend.
Finally, we converted the interactive Slack approvals into a state-free, one-way alert. By disabling active thread monitoring, we eliminated massive token overhead.
Hard-Won Technical Hurdles & Engineering Breakthroughs
Building a production-ready ingestion pipeline reveals edge cases that no system design document can predict. Here are the real-world engineering hurdles we solved along the way.
The “Multi-Promotion Split” Database Constraint
Newsletters often contain multiple promotions. For example, a single email from a venue might promote an oyster feature, a graduation dessert platter, and a Father’s Day special.
Our subagent successfully split a single newsletter into three distinct drafts. However, the subsequent database insertions failed immediately.
The cause was a strict database schema design. The news_feed_email_sources table had a UNIQUE constraint on gmail_message_id. Since all three split drafts originated from the same email ID, the database blocked the second and third insertions.
We resolved this by dropping the UNIQUE constraint on gmail_message_id. This simple schema adjustment allows multiple promotions to link back to identical email metadata rows while maintaining absolute relational integrity.
The 2025/2026 Year Anomaly (Temporal Healing)
LLMs notoriously struggle with temporal calculations. When parsing newsletters sent in late 2025 or early 2026, emails frequently referenced events happening “this Friday” or “tomorrow.”
The LLM occasionally resolved these relative dates into the wrong calendar year, setting the event date to 2025 instead of 2026.
Instead of writing complex prompt guidelines that inflated our token usage, we fixed this at the database layer. We deployed a BEFORE INSERT OR UPDATE trigger in PostgreSQL:
CREATE OR REPLACE FUNCTION clean_news_feed_post_years()
RETURNS TRIGGER AS $$
BEGIN
-- Detect and heal accidental 2025 dates to 2026
IF EXTRACT(YEAR FROM NEW.event_date) = 2025 THEN
NEW.event_date := NEW.event_date + INTERVAL '1 year';
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;This trigger programmatically inspects incoming event timestamps, automatically healing accidental 2025 dates to the current active year of 2026.
Slack Kit Markdown Nightmare
We integrated Slack notifications to alert our team when new drafts arrived in the portal. However, the drafts sent to our #news-feed channel frequently failed.
Raw newline characters (\n) generated by the AI were double-escaped during JSON serialization, throwing Slack Block Kit payload errors.
We redesigned the Slack Block Kit notification to use a newline-free modular block layout. By separating metadata, title, and body text into distinct layout blocks, we eliminated the double-escaping issue entirely and created a premium, highly readable alert layout.
Native Python Gmail Parsing Heuristics
Directly feeding raw email MIME or heavy HTML strings to an LLM burns huge amounts of tokens. It also leads to poor extraction, as the LLM pulls social icons, tracking pixels, and company logos instead of high-quality food photography.
We wrote a custom Python helper (email_parser_helper.py) and uploaded it directly into the Gumloop skill’s .zip file structure.
The script parses the Gmail API JSON payload, strips out Mailchimp headers/footers, and runs a ranked negative-keyword heuristic on image alt texts. This ensures we prioritize high-resolution food and venue photos over promotional graphics.
Dynamic Category Management & Safety Rails
To make the system highly flexible, we shifted category management from hardcoded frontend values to database-driven records. Admins can add and remove categories through a visual chip interface in the NewsFeedSettings page.
We normalized category capitalization purely on the frontend UI, converting raw database entries (e.g., VIEW MENU) to clean titles (View Menu). This keeps the database records clean while maintaining design flexibility.
To ensure backend resilience, we added robust try-catch blocks in our Express BFF around JSON.parse operations on the dynamic active_categories database setting:
try {
const settings = JSON.parse(dbResult.active_categories);
return settings;
} catch (error) {
console.error("Failed to parse active categories, falling back to defaults:", error);
return DEFAULT_EVENT_CATEGORIES; // Graceful fallback to 9 standard categories
}If the database values ever become corrupted, the API degrades gracefully to a pre-defined list of nine standard event categories, ensuring absolute system uptime.
Temporal anomalies and text scrubbing are faster, cheaper, and 100% reliable when offloaded from the LLM to PostgreSQL triggers and Python parser heuristics.
Operational Blueprint: Architectural Takeaways
Developing the Auto-Broadcaster pipeline taught us three valuable lessons for building AI-powered B2B software:
- Don’t Let the LLM Calculate: Date calculations, text cleaning, and structural validation are cheaper, faster, and 100% reliable when handled by PostgreSQL triggers, Python scripts, or Express safety rails. Save your LLM tokens for creative synthesis.
- State-Free Parallelization: Keep ingestion agents specialized and state-free. Span subagents in parallel to process multiple items rather than looping inside a single, token-heavy context.
- Visual HITL Preview: Surfacing drafts in a rich, multi-layout B2B editor (Hero, Split, Compact, Text-Only) acts as an essential stress-reliever for content curation, protecting brand standards.
We built an automated pipeline that pulls weekly restaurant newsletters from Gmail, extracts structured event drafts using LLMs, and surfaces them in a B2B portal. By re-architecting the Gumloop workflow into an Orchestrator + Subagent Model and offloading creative and structural tasks to PostgreSQL and our Express BFF, we reduced token costs by 75% while achieving absolute system resilience.
