Data Engineer IV

*****Preference for Hybrid but open to fully remote but they will be required to work 9 AM-5PM5PM EST with the team.

Updated Notes from Engagement Manager:
* Candidates MUST be able to code- there is a coding challenge during the skills assessment. If the candidate is unable to code, they will be rejected.
* Please do not use the top of the rate range unless candidates meet or exceed "must have" requirements.
* Candidates should be able to communicate like a "consultant."
* Candidates with Architect or Manager titles are likely to be rejected. Architects/Managers are not the right fit.
* Needs to be able to talk technical with a nontechnical audience. Candidates MUST be motivated and good communicators.
* Candidates without must haves will not be shortlisted
* Aim for 5-10 years of experience with data and/or software engineering.
* If candidates look like they did not write their own resumes, proceed with caution before submitting.

Duke Energy's Enterprise Data Platforms Team is seeking a Subject Matter Expert to help develop Duke's Data Fabric as an interconnected network of data capabilities and data products designed to deliver data efficiently and at scale. Candidates should have expertise in developing and building data platforms, demonstrating experience with overcoming obstacles and avoiding pitfalls. They should also possess skills in optimizing and automating deliverablesdeliverables to production using the required tech stack. Additionally, candidates should be experienced and adaptable to changing demands and priorities in an Agile development environment.
We are specifically looking for individuals with at least 5+ years of experience in Data Engineering and/or Software Engineering roles who can provide knowledge and support to our existing engineers.
Must have experience with similar platform engineering/management solutions:
* Building/optimizing Data LakeHouseLakeHouse with Open Table formats
* KubernetesKubernetes deployments/cluster administration
* Transitioning on-premise big data platforms to scalablescalable cloud-based platforms like AWS
AWS * Distributed Systems, Microservice Microservicearchitecture, and containers
* Cloud Streaming use cases in Big Data Ecosystems (e.g., EMR,EMR EKS,EKS Hadoop,Hadoop Spark, Hudi,Hudi Kafka/Kinesis)Kinesis
Must have experience with below tech stack:
* Git Hib Hiband Git Hub Actions
* AWS
AWS o IAM
IAM o API APIGateway
o Lambda
o Step Functions
o Lake formation
o EKS EKS& Kubernetes
Kubernetes o Glue: Catalog, ETL,ETL Crawler
o Athena
o Lambda
o S3 S3(Strong foundational foundationalconcepts like object data store vs block data store, encryption/decryption, storage tiers etc)
* Apache Hudi
Hudi * Apache Flink
Flink * PostgreSQL PostgreSQLand SQL
SQL * RDS RDS(Relational Database Services).
* Python
* Java
* Terraform TerraformEnterprise
o Must be able to explain what TF TFis used for
o Understand and explain basic principles (e.g. modules, providers, functions)
o Must be able to write and debug TF
TF Helpful tech stack experience would include:
* Helm
* Kafka and Kafka Schema Registry
* AWS AWSServices: CloudTrail,CloudTrail SNS,SNS SQS,SQS CloudWatch,CloudWatch Step Functions, Aurora, EMR,EMR Redshift,Redshift Iceberg
* Secrets Management Platform: Vault, AWS AWSSecrets manager
Core Responsibilities and Soft Skills
* Provides technical direction, engage team in discussion on how to best guide/build features on key technical aspects and responsible for product tech delivery
* Works closely with the Product Owner and team to align on delivery goals and timing
* Collaborates with architects on key technical decisions for data and overall solution
* Lead design and implementation of data quality check methods
* Ensure data security and permissions solutions, including data encryption, user access controls and logging
* Be able to think unconventionally to find the best way to solve for a defined use case with fuzzy requirements.
* Self-starter mentality
o Willing to do their own research to solve problems and can clearly present findings and engage in conversation on what makes one solution better than another
* Thrive in a fail-fast environment, involving mini PoCs,PoCs and participate in an inspect and adapt process.
* Questioning and Improvement mindset
o Must be ready to ask questions about why something is currently done the way it is and suggest alternative solutions
* Customer facing skills
o Interfacing with stakeholders and other product teams via pairing, troubleshooting support, and debugging issues they encounter with our products

Impellam Group and its brands are equal-opportunity employers committed to diversity and inclusion. All qualified applicants will receive consideration without regard to race, color, religion, gender, sexual orientation, pregnancy or maternity, national origin, age, disability, veteran status, or any other factor determined to be unlawful under applicable law. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application, interview process, pre-employment activity, and the performance of crucial job functions.

If you require additional disability considerations, modifications, or adjustments please let us know by contacting HR-InfoImpellamNA@impellam.com or fill out this form to request accommodations.