Senior software engineer: Applied AI

Integrate with hosted LLM inference (via centralized model gateway infrastructure) for extraction, classification, and agent orchestration workloads; evaluate open-weight models against hosted options for cost and performance tradeoffs. Design and maintain adapters that sync source systems (CMS, event platforms, editorial, research libraries) into a governed dataset without duplicating source-of-record logic. Build and maintain an eval harness: curate golden questions, catch regressions before release, and treat eval results as the gate for shipping. Design MCP tools and API/GraphQL interfaces that separate "fuzzy" retrieval (ranked candidates, confidence scores) from "exact" governed lookups (deterministic, provenance-carrying). Build and extend the pipeline that validates, resolves, and versions governed entities into a serving layer. Design and operate LLM-driven extraction and classification agents that propose structured data for human review rather than auto-publishing unreviewed AI output. Own structured content modeling against a headless CMS, including schema and versioning decisions that other teams depend on. Instrument pipelines and served surfaces for freshness, adoption, and answer-quality telemetry. Participate in and help run the weekly eval review and the biweekly skill-library session, harvesting reusable agent tooling for the team. Mentor engineers being reskilled into applied AI work, particularly around eval design and agentic-coding practices. Contribute governance and vocabulary decisions upstream to org-wide standards where relevant, rather than duplicating them. Desired Candidate Profile 5+ years of software engineering experience, including production API and data-pipeline design. Comfort working with agentic coding tools daily as a core part of the workflow. Hands-on production experience integrating LLMs: Skills, MCP, RAG, structured outputs, and tool/function calling. Strong proficiency in Python across eval tooling, service-level code, and API development (FastAPI or similar), plus working proficiency in TypeScript/Node.js for application integrations. Experience deploying and operating production services in AWS (or equivalent), including containerized workloads and infrastructure-as-code. Security and privacy judgment in AI systems: handling sensitive data appropriately, and designing against failure modes like prompt injection, data leakage through prompts, and unsafe or unattributed model output. Experience operating production systems against latency, reliability, and cost targets, including token-cost management for inference workloads. Evaluation literacy: you can describe an eval you built, what it caught, and how you handled canonical truth, variance, and regression cases. Experience with REST/GraphQL API design. Solid understanding of data pipeline patterns: idempotency, versioning, and staged architectures. Explicitly not required: formal model training or ML research credentials (e.g., pretraining, fine-tuning research). This is an applied systems role; we're looking for builders, not researchers. Strong written communication. You can write a clear design doc, explain a tradeoff to a non-engineer, and document decisions others will build against. Creative problem solver comfortable operating in ambiguity, with a builder's bias toward shipping over ceremony. Nice to have Experience evaluating or benchmarking open-weight models for cost/performance (inference-time evaluation, not training). Experience with cloud-hosted inference services (e.g., Amazon Bedrock, Azure AI/Cognitive Services) and centralized LLM gateways (e.g., LiteLLM). Experience with headless CMS platforms and structured content modeling. Experience in a regulated or compliance-sensitive domain where provenance and auditability matter.