Big Data Engineer

About the Company:

Since our founding in 1935, we have been committed to serving local and global communities by being a market leader in Investment Banking, Securities, Investment Management and Wealth Management services. Our belief that capital can work to benefit all of society inspires us to put our clients first, lead with exceptional ideas, hold our business to high ethical standards, and give back to communities around the world through philanthropy and public works. We have a smart casual dress code and operate under a philosophy that balances work with your personal life. Our people’s talent, passion, and expertise is the fuel on which our organization runs, therefore, our people are our greatest asset. Diversity and inclusiveness is a critical component for our success and it is our priority to continue building a firm that values the unique background and identity of every one of our employees, thus enabling our people to bring their full, and best selves to work each day. Teamwork is the essence of our approach, and so are the values of integrity, excellence, and enabling our people to achieve at the highest levels. We invite you to learn more about our commitment to diversity and serving our community.

Our Technology:

Our Technology is the global technology department responsible for the design, development, delivery and support of the technical solutions behind the products and services used by the Company business.

The department is comprised of 10 organizations: Sales, Banking & Corporate-Client Technology, Investment Products & Markets Technology, Client Reporting, Core Processing, Private and International Wealth Management Technology, Technology Integration Office, Enterprise Infrastructure & Production Management, Capital Markets Application & Data Services, Deployment Planning & Release Management, and the Chief Operating Office.

Department Profile:

Wealth Managements Core Platform Services group provides horizontal services to all Wealth management Development Teams. Our mission is to provide stable and scalable infrastructure and technology solutions for entire Wealth Management. We serve as a centralized interface through which Wealth Management teams can obtain infrastructure solutions and project support by working closely with application owners throughout the SDLC process to ensure that established products/services are leveraged, and new requirements are fulfilled.

Position Description:
* This position is for Big data engineer for Company’s Wealth Management

* Framework CoE team at Company’s Alpharetta or New York offices.
* CoE team is responsible to define and govern the data platforms.
* We are looking for colleagues with strong sense of ownership and ability to drive solutions.
* The role is primarily responsible to automate the existing process and bring new ideas and innovation.
* The candidate is expected to code, conduct code reviews, and test framework as needed, along with participating in application architecture and design and other phases of the automation.
* The ideal candidate will be a self-motivated team player committed to delivering on time and should be able to work with or without minimal supervision.

Responsibilities
* Design & Develop new automation framework for ETL processing
* Support existing framework and become technical point of contact for all related teams
* Enhance existing ETL automation framework as per user requirements
* Performance tuning of spark, snowflake ETL jobs
* New technology POC and suitability analysis for Cloud migration
* Process optimization with the help of automation and new utility development
* Work in collaboration for any issues and new features
* Support any batch issue
* Support application team teams with any queries

Required Skills
* 7+ years of Data engineering experience
* Must be strong in UNIX Shell, Python scripting knowledge
* Must be strong in Spark
* Must have strong knowledge of SQL
* Hands-on knowledge on how HDFS/Hive/Impala/Spark works
* Strong in logical reasoning capabilities
* Should have working knowledge of Github, DevOps, CICD/ Enterprise code management tools
* Strong collaboration and communication skills
* Must possess strong team-player skills and should have excellent written and verbal communication skills
* Ability to create and maintain a positive environment of shared success
* Ability to execute and prioritize a tasks and resolve issues without aid from direct manager or project sponsor
* Good to have working experience on snowflake & any data integration tool i.e. informatica cloud

Primary skills:

* Python, Big Data & Apache Spark

Desired skills:
* Snowflake/Azure/AWS any cloud
* IDMC/any ETL tool