Java Scala Developer_ Apache Spark & SQL
Citigroup · Pune
The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.
Responsibilities:
- Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements, including using script tools and analyzing/interpreting code
- Consult with users, clients, and other technology groups on issues, and recommend programming solutions, install, and support customer exposure systems
- Apply fundamental knowledge of programming languages for design specifications.
- Analyze applications to identify vulnerabilities and security issues, as well as conduct testing and debugging
- Serve as advisor or coach to new or lower level analysts
- Identify problems, analyze information, and make evaluative judgements to recommend and implement solutions
- Resolve issues by identifying and selecting solutions through the applications of acquired technical experience and guided by precedents
- Has the ability to operate with a limited level of direct supervision.
- Can exercise independence of judgement and autonomy.
- Acts as SME to senior stakeholders and /or other team members.
- Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
Qualifications:
- 5- 9 yrs Experienced resource with
- Strong communication skills - both verbal and written
- Strong in Java/Scala development, spring , spring boot, Apache Spark, Spark SQL, shell scripting
- Ability to work in Python is also required. Pyspark experience preferable.
- Strong in SQL programming especially spark SQL
- Demonstrated Data analysis skills namely - data sourcing , processing and insights.
- Tableau experience is good to have but not mandatory
- Expected experience should be minumum 7+ years
- Ability to understand requirements and develop high level and low level design
- Has experience in devops tools like github, Jenkins/lightspeed, harness
- Has experience in using AI tools like github copilot, DevinAI, Cluade code
Experience Level: 5 to 9 years as discussed
Key Skills:
- Java
- Scala/python
- Apache Spark
- Spark SQL
- AI tool usage based development - DevinAI, github copilot etc.
Good to have skills:
- Python
- Unix shell scripting
- Tableau for visualization and reporting
Education:
- Bachelor’s degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
We are seeking an experienced and forward-thinking Senior Big Data Engineer (7+ Years) to join our technology team. In this role, you will be responsible for designing, developing, and optimizing high-performance, large-scale data processing pipelines and regulatory data platforms using Apache Spark, Java/Scala, Spring Boot, and Python/PySpark.
The ideal candidate brings deep expertise in distributed data computing, complex Spark SQL query optimization, data analysis, and modern CI/CD DevOps workflows. You will actively leverage GenAI engineering tools (such as GitHub Copilot and Devin AI) to accelerate delivery, drive code quality, and translate complex business requirements into robust High-Level (HLD) and Low-Level Designs (LLD).
Key Responsibilities
Architect & Develop Data Solutions: Design, implement, and maintain scalable, high-throughput batch and stream data pipelines using Apache Spark, Java/Scala, and PySpark.
End-to-End System Design: Analyze business and regulatory requirements to create comprehensive High-Level Designs (HLD), Low-Level Designs (LLD), data flow diagrams, and interface specifications.
Data Engineering & Analytics: Perform in-depth data analysis across the data lifecycle—including data sourcing, schema evolution, transformation logic, quality validation, and actionable insight generation.
Spark SQL Optimization: Write, tune, and maintain complex Spark SQL queries, partitioned datasets, and transformations for optimal performance over large enterprise datasets.
Microservices & API Integration: Build enterprise-grade services and data ingestion microservices using Spring and Spring Boot.
DevOps & CI/CD Delivery: Implement automated build, test, and deployment workflows using Git/GitHub, Jenkins / Lightspeed, and Harness.
AI-Augmented Engineering: Leverage modern GenAI developer tooling (e.g., GitHub Copilot, Devin AI) to accelerate prototyping, code review, unit testing, and documentation.
Cross-Functional Collaboration: Partner closely with Product Owners, Data Analysts, Business Teams, and Quality Assurance to ensure end-to-end alignment and timely delivery.
Candidate Qualifications & Skill Set
1. Core Technical Skills (Must-Have)
Experience:5-9 years of hands-on software and data engineering experience in enterprise environments.
Distributed Computing & Big Data: Deep practical expertise in Apache Spark architecture (RDDs, DataFrames, Datasets, execution plans, memory management, partitioning, and performance tuning).
Primary Programming Languages: Advanced proficiency in Java and/or Scala, with strong object-oriented and functional programming paradigms.
Frameworks: Extensive hands-on experience building scalable applications using Java and Apache Spark.
Scripting & Python/PySpark: Strong proficiency in Python with practical experience writing data transformations using PySpark.
Data Querying & SQL: Expert-level proficiency in SQL / Spark SQL, window functions, complex joins, indexing, and query plan execution analysis (EXPLAIN).
Data Analysis & Processing: Proven ability to source, cleanse, enrich, reconcile, and validate large datasets to deliver high-integrity reporting and analytical outputs.
2. Architecture & DevOps
Solution Architecture: Proven track record of drafting clear, maintainable HLD and LLD documents, technical specifications, and data modeling artifacts.
Version Control & CI/CD: Hands-on experience with GitHub / Bitbucket, build automation with Jenkins / Lightspeed, and modern continuous delivery platforms like Harness.
3. AI Tools & Modern Engineering Practices
AI-Assisted Development: Experience utilizing AI productivity tools such as GitHub Copilot or Devin AI for test generation, code refactoring, and rapid prototyping.
Engineering Excellence: Strong commitment to test-driven development (TDD), automated unit/integration testing, SonarQube quality gates, and secure coding standards.
4. Preferred / Good-to-Have
Business Intelligence & Visualization: Experience with Tableau or similar BI visualization tools for dashboarding and reporting validation.
Domain Knowledge: Prior exposure to Banking, Financial Services, Liquidity Reporting, Regulatory Data (e.g., NDTL/LCR), or Risk Management.
5. Soft Skills & Professional Attributes
Exceptional Communication: Outstanding verbal and written communication skills; adept at explaining technical concepts to both technical peers and non-technical business stakeholders.
Ownership & Problem Solving: Proactive attitude, strong analytical reasoning, and the ability to drive initiatives from inception to production deployment.
------------------------------------------------------
Job Family Group:
Technology------------------------------------------------------
Job Family:
Applications Development------------------------------------------------------
Time Type:
Full time------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.