Job description
Remote in the following states only: AL, AZ, AR, GA, ID, IN, KS, LA, MO, MT, NC, OH, TN, TX, UT, VA, IA, MS, OK, SC, SD, WY
Senior Data Engineer
Senior Data Engineer
Position Highlights:
- The Senior Data Engineer, as a member of the data engineering team, will provide expertise in design, deployment and maintenance of data engineering framework essential to support the Enterprise data needs within and external to the organization.
- This position is responsible for supporting and advancing delivery of data analytics projects by designing, building and managing data ingestion, transformation and maintenance to deliver accurate and reliable data to stakeholders in cloud-based on on-premise environment.
- The Senior Data Engineer is responsible to continuously improve our data engineering operations capability through innovation and developing self-service automations that simplify and improve the overall customer experience.
- This role combines with technical aspects of data engineering with the business understanding of the systems and processes involved.
Responsibilities:
- Design, develop, deploy and maintain key deliverables understand business need and develop data models which are efficient, effective and scalable
- Assist in selecting the right technologies
- Collaborate with business partners to understand business processes, standard release management
- Build pipelines in AWS platform (Matillion tool) and Informatica
- Collaborate with technical, data quality and business teams on design and development activities
- Update and maintain project, incident and change control systems Build and maintain all required documentation
- Review daily processes for improvements and enhancements
- Monitor existing pipelines for errors and perform root cause analysis to develop solutions
- Act upon opportunities to improve the system process flow, cost, performance, and technical efficiencies
- Ensure proper application, operations, standards, policies, and procedures are followed.
- Be a go to person and subject matter expert to the data engineering team
- Verify code deployments, ensuring the correct build versions have been deployed to new environments.
- Work with business to define ETL strategy
- Conduct design sessions to peer review ETL strategy and data models
- Address production support break-fix in timely manner, keeping IT management and business informed of status and resolutions.
- Review existing pipelines for errors and perform root cause analysis to develop efficient solutions
Credentials and Experience:
- Bachelor’s Degree in Information Systems, Computer Science, Business Administration, or similar field
- Minimum 8 years of experience in implementation of Data engineering initiatives
- Minimum 3 years of cloud experience (AWS preferred)
- Minimum of 3 years of experience working and communicating with cross-functional teams, derive requirements and architect data models, shared datasets
- Expertise in a Data Engineering role, with a focus on building data pipelines or conducting data-intensive analysis on cloud platform (AWS preference) and on-premise environment
- Build and maintain the infrastructure to support ETL processing. Extract data from multiple data sources, such as SQL, NoSQL, Oracle and other platform APIs, and load into a centralized data warehouse/data lake to facilitate unified reporting.
- Experience with software engineering best practices, including but not limited to version control (Git, Subversion, etc), CI/CD (Jenkins, Maven, etc.) automated uite testing, and Dev Ops.
- Experience working with ETL tools like Informatica, SSIS, Matillion
- Experience with one or more AWS and/ or cloud services: EC2, S3, RDS, Snowflake, Lambda
- Apply and mentor junior engineers on advanced data engineering theories, principles and concepts
- Experience working with Spark, Kafka, Airflow
- A strong engineering background and expertise in working with data and databases
- Advanced knowledge of T-SQL, SQL
- Worked on Unix platforms and past experiences in scripting languages (Python)
- Experience in software development environments using Agile/SCRUM, and code
management/versioning - Continuously seek out industry best practices and develop skills to create new capabilities for data engineering
- Ability to understand and work with complex, large enterprise business environments.
Preferred Experience:
- Relevant experience in a research, academic, scientific environment or healthcare provider a plus.
- Working experience in Matillion, Snowflake, Python, S3, Kafka, Github, Informatica, SSIS, Oracle, SQL Server
- Knowledge of emerging technologies
abouteureka.com is the go-to platform for job seekers looking for the best job postings from around the web. With a focus on quality, the platform guarantees that all job postings are from reliable sources and are up-to-date. It also offers a variety of tools to help users find the perfect job for them, such as searching by location and filtering by industry. Furthermore, abouteureka.com provides helpful resources like resume tips and career advice to give job seekers an edge in their search. With its commitment to quality and user-friendliness, abouteureka.com is the ideal place to find your next job.