Senior software engineer, data pipeline
Build and scale the systems that run millions of prompts across six AI engines each week and turn raw answers into data customers can trust.
Engineering
New York or Remote (US)
Full-time
$185k – $225k + equity
About the role
Every number in Cited starts with a prompt run: we send a prompt to an engine, capture the full answer and its citations, and parse it into structured data. You’ll own the pipeline that does this across six engines, several times a week, for every customer. Reliability is the product here, because a failed run or a misread citation means a customer’s Share of Answer is wrong.
What you’ll do
Scale our prompt-execution system to three times its current volume at a lower cost per run
Build parsers that extract citations, brand mentions and recommendations as answer formats change
Design monitoring that catches format changes and failed runs within minutes
Work with the research team to make datasets reproducible and auditable
Join a light on-call rotation, about one week in eight
What you’ll bring
6+ years building production data systems or distributed services
Strong Python or Go, and solid SQL
Experience with queues, workers and backpressure at scale; we use Temporal, Postgres and ClickHouse
A habit of writing things down: design docs, runbooks and clear pull requests
Nice to have
Experience with web crawling or browser automation
Familiarity with LLM APIs and their failure modes
Experience building warehouse exports