As a fast-growing startup building a next-generation platform for the gig economy, we are looking for a Data Engineer to enhance our end-to-end data infrastructure to support our AI capabilities. You will be involved in building the data backbone that powers our job matching algorithms and AI features, ensuring high-quality data flows seamlessly from our mobile and web applications into our analytical and AI environments.
Key Responsibilities
Data Engineering & Pipeline Architecture (~70%)
- Scalable ETL/ELT: Design, build, and maintain automated pipelines to collect, clean, and transform data from our Flutter mobile app, web portal, and/or APIs into PostgreSQL, MongoDB, and/or Google BigQuery.
- Hybrid Storage Management: Manage and evolve our data environment, focusing on PostgreSQL for relational data and MongoDB for unstructured conversational data and transcripts.
- AI Data Readiness: Implement data quality checks, monitoring, and alerting to ensure that the data feeding our AI interview and SOP systems is reliable and trustworthy.
- Event Tracking: Facilitate the setup of custom event tracking and user behavior logging to capture the raw data necessary for AI model training.
Data Modeling (~30%)
- Matching Algorithms: Build and iterate on scoring models using vector embeddings and NLP techniques to improve job-worker fit and recommendation quality.
- Vector Search Infrastructure: Implement and optimize vector search capabilities (e.g., using Gemini API Embeddings or Vertex AI) to enable high-speed similarity matching.
- Collaborative ML: Work with our AI Engineer to deploy and monitor machine learning models that predict churn, satisfaction, and platform economics.
Additional Scope (as the data function grows)
- Support data needs for product experimentation and growth initiatives.
- Document data models, pipelines, and business logic for team knowledge sharing.
- Stay up-to-date with new data tools and best practices; suggest improvements to our stack as we scale.
