We are looking for a Hadoop developer to help lead the effort for onboarding / off boarding as well as data governance and –lineage for our Billing projects. Ideal candidates will have experience with Big Data as well as ETL Technologies. Direct experience with Hadoop / Cloudera is also required.
- Developing and managing data processes to ensure that data is available and usable
- Creation and automation of data pipelines and platforms
- Managing and monitoring data quality via automated testing frameworks (Data Driven Testing, TDD, etc.)
- Develop tools and libraries, and maintain processes for other engineers to access data and write MapReduce programs.
- Write scalable and maintainable ETLs
- Write software to ingest data into Hadoop.
- Build, operate, monitor, and troubleshoot Hadoop infrastructure.
- Aptitude and desire for learning new technologies.
- Responsible for the documentation, design, development, and architecture of Hadoop applications
- Handle the installation, configuration, and supporting of Hadoop
- Converting hard and complex techniques as well as functional requirements into the detailed desi
- Pre-processing of data using Spark, Impala and Hive
- To maintain data security and privacy
- Knowledge on GIT.
Principal Developer->> Consults with internal business groups to provide high-level application software development services or technical support. Provides comprehensive senior-level technical consulting to IT management and senior technical staffs. Evaluates compliance with the organization's technology standards. Works with internal business groups on implementation opportunities, challenges, and requirements of various applications. Analyzes information and provides recommendations to address and resolve business issues for a specific business group. Guides and consults with IT management and technical staffs regarding use of emerging technologies and associated services. Participates in defining corporate implementation and integration strategies of new technologies. Advocates for innovative, creative technology solutions. Contributes to the achievement of area objectives.
Bachelor's degree in computer science engineering or a related discipline, or equivalent work experience required, 10-12 years of experience in software development required, experience in the securities or financial services industry is a plus.
BNY Mellon is an Equal Employment Opportunity/Affirmative Action Employer.
Minorities/Females/Individuals With Disabilities/Protected Veterans.
Our ambition is to build the best global team – one that is representative and inclusive of the diverse talent, clients and communities we work with and serve – and to empower our team to do their best work. We support wellbeing and a balanced life, and offer a range of family-friendly, inclusive employment policies and employee forums.
Primary Location: United States-Pennsylvania-Pittsburgh
Internal Jobcode: 45198
Job: Information Technology
Organization: Operations & Shared Technology-HR13644
Requisition Number: 1915259