AVP, Data Platform Development, LME

HKEX · Shenzhen

Apply on the company website

Location:

CN-Shenzhen-HyQ

Shift:

Standard - 40 Hours (China)

Scheduled Weekly Hours:

Worker Type:

Permanent

Job Summary:

Support the development, enhancement, and day‑to‑day operation of LME technology systems, including trading, market data, and supporting platforms. This entry‑level engineering role focuses on delivering high‑quality software, ensuring system stability, and collaborating with cross‑functional teams across LME Technology.

Job Duties:

Role Summary

Design, build, and operate critical subsystems of the data platform. Take ownership of major components — streaming, batch, storage, or serving — drive them from design to production, and continuously improve their reliability and performance.

Responsibilities

Streaming Infrastructure: Kafka cluster operations (broker tuning, partition rebalancing, monitoring, disaster recovery); manage schema registry and Kafka Connect connectors
Batch & Streaming Compute: Build and optimize Spark batch jobs and Flink/Spark Structured Streaming pipelines; contribute reusable job frameworks and tuning guides
Storage & Lakehouse: Manage Iceberg tables (compaction, snapshot expiration, orphan file cleanup, schema evolution); operate MinIO at scale (lifecycle rules, tiering, performance tuning)
Query & Serving: Deploy and operate Trino clusters (connector config, resource groups, query monitoring); manage StarRocks/ClickHouse clusters (sharding, replication, materialized views)
Orchestration: Build and maintain Airflow/Dagster DAGs for platform operations; extend custom operators and sensors
Platform Observability: Implement monitoring with OpenTelemetry, Prometheus, Grafana, and Loki across all stack layers; build dashboards and alerting rules
Kubernetes Operations: Write Helm charts, manage operator lifecycles, configure resource quotas, node affinity, pod disruption budgets
CI/CD & GitOps: Own ArgoCD application sets, Helm-based deployments, and promotion pipelines from dev to prod
Participate in on-call rotation; write post-mortems and runbooks
Mentor mid-level engineers through pairing, design discussions, and code reviews

Required Skills & Experience

6+ years in data/platform engineering or backend infrastructure
Strong Kubernetes: Helm chart authoring, RBAC, network policies, storage (PV/PVC), operators
Solid Kafka: topic design, consumer group management, offset management, monitoring lag, Kafka Connect, schema registry (Apicurio or Confluent)
Solid Spark: Dataframe/Dataset API, Spark SQL, performance tuning, troubleshooting in production
Working knowledge of Flink or Spark Structured Streaming for real-time pipelines
Practical Iceberg experience: table maintenance, time-travel, catalog integration
Hands-on Trino or Presto: connector configuration, query tuning, resource group management
Experience with an OLAP engine: StarRocks, ClickHouse, or Doris — table design, ingestion pipelines, query optimization
Proficient in Python and either Scala or Java
Solid CI/CD and GitOps: ArgoCD or Flux, Helm, Docker
Airflow or Dagster for pipeline orchestration

Nice to Have

OpenShift-specific: SCC, Routes, ImageStreams, BuildConfigs
dbt project experience for data transformation and modeling
Data quality frameworks: Great Expectations, Soda, or Deequ
Kafka Streams or ksqlDB for stream processing
Exposure to DataHub/Atlas for data discovery and lineage

Company Introduction:

ITD SZ

港交所科技（深圳）有限公司，是2016年12月28日于深圳市前海自贸区成立的外商独资企业。

作为港交所的技术子公司，港交所科技（深圳）有限公司主要是为集团及其附属公司提供计算机软件、计算机硬件、信息系统、云存储、云计算、物联网和计算机网络的开发、技术服务、技术咨询、技术转让；经济信息咨询、企业管理咨询、商务信息咨询、商业信息咨询、信息系统设计、集成、运行维护；数据库管理、大数据分析；以承接服务外包方式提供系统应用管理和维护、信息技术支持管理、数据处理等信息技术和业务流程外包服务。

More jobs at HKEX · Data & Analytics jobs in Cn Shenzhen Hyq