Tech Lead Databricks Data Engineer

Company: Mitre Media
Category: Software Development
Location: USA, Canada, USA timezones

About Mitre Media
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt; padding: 8pt 0pt 12pt 0pt;">Mitre Media is redefining FinTech with AI-driven tools that empower millions of investors. Our portfolio, including Dividend.com and MutualFunds.com, leverages LLMs to deliver novel data insights and visually rich user experiences. For over a decade, we’ve served individual investors, financial advisors, and top asset managers like BlackRock and Vanguard through our premium data, tools, and advertising solutions. If you’re excited by the intersection of big data, AI, and investing, join our lean, entrepreneurial team from anywhere within ±3 hours of Eastern Time.</p>
About the Role
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt; padding: 8pt 0pt 12pt 0pt;">As Tech Lead Data Engineer, you’ll architect and maintain the data backbone powering every feature across our product suite. Reporting to the CTO, you will design Databricks-based ETL pipelines, model complex investment data, and surface low-latency, high-quality datasets for both user-facing features and internal AI/analytics workloads. You’ll collaborate in a remote-first hybrid culture that values in-person “bursts” of collaboration, follow ShapeUp for project planning, and ship pragmatic solutions that favor de-scoping over delays.</p>
Responsibilities
<ul style="">
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 12pt; margin-bottom: 0pt;">Design, implement, and optimize large-scale ETL workflows in Databricks (Apache Spark, Delta Lake, DBT).</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">Develop algorithms that transform raw market data into actionable insights.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">Own data quality and lineage, instituting tests, monitoring, and alerting for mission-critical pipelines.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">Evolve our cloud data platform (AWS & GCP) for scale, performance, and cost efficiency.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 12pt;">Mentor engineers, championing best practices in code reviews, documentation, and DevOps for data.</p>
</li>
</ul>
Required Technical Skills
Data Engineering
<ul style="">
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 12pt; margin-bottom: 0pt;">Programming: Expert in Python plus working knowledge of Scala or Java.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">Databricks & Spark: Hands-on with cluster tuning, job orchestration, and Delta Tables.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">SQL & DBT: Strong analytical SQL, modular data-model design, and CI/CD for transformations.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">Cloud: Production experience on AWS or GCP data services (e.g., S3/GCS, EMR/Dataproc, Glue/Dataflow).</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">ETL & Orchestration: Solid grasp of EL(T) patterns, workflow scheduling, and incremental processing.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 12pt;">Data Modeling: Dimensional and schema-on-read designs for analytics and AI.</p>
</li>
</ul>
AI & Analytics Enablement
<ul style="">
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 12pt; margin-bottom: 0pt;">Building feature stores or inference-ready tables for ML/LLM workflows.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 12pt;">Familiarity with vector databases or embedding pipelines</p>
</li>
</ul>
Nice-to-Have Technical Skills
<ul style="">
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 12pt; margin-bottom: 0pt;">Apache Airflow, Luigi, or Dagster for DAG orchestration.</p>
</li>
<li dir="ltr" style="">
<p dir="ltr" style="line-height: 1.38; background-color: #ffffff; margin-top: 0pt; margin-bottom: 0pt;">Experience with Looker, Tableau, or Stripe’s Vizie…