data platform engineer

توضیحات آگهی

About the Role We re building an internal data and AI platform on top of managed Kubernetes You ll own the data infrastructure layer from ingestion and storage to transformation and serving with a strong focus on security separation between production and analytical workloads and building something the team can maintain long term You ll work closely with data engineers and ML engineers providing the platform they run their work on What You ll Do Design and operate data pipelines batch micro batch and streaming flows Build and maintain a data lakehouse object storage table formats Iceberg Delta Lake and query engines Set up and operate data orchestration scheduling dependency management retries and alerting on failures Implement clear separation between production databases and BI AI analytical workloads read replicas CDC staging layers Own data access control who can read what at what layer with what credentials Deploy and maintain MLOps tooling experiment tracking model registry and notebook environments Build observability into pipelines data quality checks pipeline SLAs and failure alerting Manage secrets and credentials across data systems securely Write and maintain documentation runbooks data flow diagrams and onboarding guides What We re Looking For Must Have Hands on experience building and operating data pipelines batch and or streaming Experience with at least one orchestration tool Airflow Prefect or Dagster Familiarity with data lakehouse concepts object storage MinIO or S3 compatible table formats Iceberg or Delta Lake Experience with CDC or replication patterns to isolate production from analytical workloads Understanding of data level access control and RBAC across storage and query layers Comfortable working on Kubernetes deploying workloads managing configs basic troubleshooting Secrets management experience Vault Sealed Secrets or similar Good to Have Experience with OLAP engines ClickHouse Trino or Spark Kafka or Redpanda for streaming ingestion MLflow or similar for experiment tracking and model registry JupyterHub or multi tenant notebook environments Data quality frameworks Great Expectations Soda or similar Prometheus Grafana for pipeline and infrastructure observability Mindset We Value You think about data flows end to end not just individual components You treat production data as sensitive by default and design access boundaries accordingly You document pipelines and data flows so others can understand and maintain them You re comfortable owning trade offs between simplicity and capability You work with data and ML engineers as an enabler Our Stack Managed Kubernetes on Hamravesh and Sotoon Self hosted data tooling we build what we can t buy as managed services GitOps driven deployments handled by the team you ll consume not own Why This Role Real ownership over the data platform layer Greenfield environment you ll design the architecture from the start Small focused team with high trust تهران تهران میرداماد تمام وقت خودرو45 کارشناس ارشد جنسیت تفاوتی ندارد اینترنت تجارت الکترونیک خدمات آنلاین