NorthHill Technology Resources has a need for an Data Analytics Engineer to support a newly awarded Federal Program with DIA. Qualified candidates must have an active TS/SCI with CI Poly. This role can be located in Annapolis Junction, MD or Colorado Springs, CO.
Data Analytics Engineer – Data Bricks AI / ML, Splunk.
Apply skills in data science, data analytics, and data warehousing to pioneer cloud and data services. Utilize cloud and data technologies to deliver exciting IT & data driven capabilities.
The qualified candidate must possess the following engineering skills and experience supporting a large and complex federal organization:
- Engineer, build and maintain scalable automated data pipelines.
- Support critical data pipelines with a highly scalable distributed architecture - including data ingestion (streaming, events and batch), data integration, data curation.
- Build cloud data solutions and provide domain perspective on storage, big data platform services, serverless architecture, RDBMS, DW/DM, NoSQL databases and security.
- Deploy, automate, maintain and manage cloud-based production system, to ensure the availability, performance, scalability and security of productions systems.
- Conduct full technical discovery, identifying pain points, business and technical requirements, “as is” and “to be” scenarios.
- Design and arrangement of scalable, highly attainable, and fault tolerant big data eco systems on cloud platform.
- Ownership and responsibility for end-to-end design and development, testing, release of key components data lake solution using DataBricks.
- Understand and implement best practices in management of data, including master data, reference data, metadata, data quality and lineage.
- Support regular ad-hoc data querying and analysis to understand customer behaviors.
- Experience with developing and deploying ETL pipelines using Apache Spark
- Experience with distributed computing, such as Apache Spark and/or Databricks
- Experience in SQL and/or Spark SQL
- Experience with code versioning tools and a command of configuration management concepts and tools, CI/CD including DevOps.
- Experience working with Software Development Life Cycle (SDLC), Agile methodologies (Scrum)