Data Bricks Engineer

Atlanta, GA

Data Bricks Engineer Job Description

Job Title: Data Bricks Engineer

Work Location(s):

Atlanta, GA or Birmingham, AL - Hybrid

JOB SUMMARY

The Data Bricks Engineer role will be part of the Marketing Technology Solutions team within the Customer Technology Solutions Organization in Southern Company’s Technology Organization’s Electric Technology Solutions tower. Position responsibilities include Energy Information Exchange (EIX) data scheme versioning and management.

The Data Bricks Engineer will work with stakeholders – both business and Technology Organization to design and develop data-oriented solution architecture involving On-Prem and cloud (Microsoft Azure) technologies that align with and support the business and Technology Organization strategy and standards. This role will also work hands on to implement solutions. Additionally, this role will be responsible for management and alignment of solutions across all technology platforms working closely with developers & infrastructure teams to optimize application code, ensuring reusable services, optimizing performance & resource utilization, planning capacity, and ensuring application/data security.

JOB REQUIREMENTS

Education:

BS/BA required, preferably in Computer Science, Management Information Systems, Engineering or other technology related field.

Experience:

  • Experience with Databricks.

  • Experience with a major cloud platform, preferably Microsoft Azure and services including CosmosDB, Storage Application Services, Azure Data Factory.

  • Experience architecting data-oriented solutions utilizing Cloud (preferably Azure) data architecture platforms and patterns.

  • Experience optimizing cloud PaaS resources to adhere to best practices & reduce cost.

  • Experience with Agile as well as DevOps, CI/CD methodologies.

  • Experience developing complex logical and physical data models.

  • Experience implementing lakehouses with medallion architecture, including data engineering pipelines and data storage layers.

  • Experience with dynamic metadata driven frameworks and large-scale data ingestion patterns.

  • Familiarity with streaming datasets and Databricks delta live tables.

  • Experience with performance tuning techniques in a lakehouse environment, such as partitioning, aggregations, materialized views, etc.

Knowledge/Skills/Attributes:

  • Databricks Platform: Proficiency in Databricks is essential. Should be comfortable working with Databricks notebooks, clusters, and jobs.

  • Data Engineering Tools: Familiarity with tools like Apache Spark, Delta Lake, and MLflow is valuable. These technologies are at the core of Databricks’ offerings.

  • Cloud Platforms: Data Bricks Engineers at Southern Company work with the Azure.

  • Knowledge of cloud PaaS architecture & resources.

  • Knowledge of structured & unstructured data – databases, data warehouses, and data lakes.

  • Knowledge and understanding of software development technologies, methodology and associated tools.

  • Excellent root cause analysis & problem-solving skills

  • Strong technical writing skills preferred.

JOB RESPONSIBILITIES

  • Data Pipeline Development: Design and develop robust data pipelines to ingest, store, and process data from various sources. These pipelines ensure efficient data flow for analytics and reporting.

  • Data Modeling: Create and maintain data models that support data analysis and reporting. These models help organize and structure data for efficient querying and visualization.

  • ETL Processes: Build and maintain Extract, Transform, Load (ETL) processes. These processes move data between systems, ensuring data quality and consistency.

  • Data Lakes: Work on data lakes to optimize storage and retrieval for analytical purposes.

  • Architect, develop, implement, integrate, and/or support cloud technology solutions in a manner that meets or exceeds business partner expectations for quality, cost, and timeliness.

  • Quickly become the team’s SME (Subject Matter Expert) for data architecture & Databricks.

  • Communicate effectively and timely with customers, team members, and management.

  • Build effective relationships with peers, management, and business partners.

  • Ensure required documentation for projects is developed.

  • Adhere to Change Management policies and procedures.

  • Respond/Resolve quickly & effectively to all operational problems.

  • Be a student of the business, taking care to understand the challenges & goals of our company.

  • Exhibit Our Values.

#LI

Southern Company (NYSE: SO) is a leading energy provider serving 9 million residential and commercial customers across the Southeast and beyond through its family of companies. Providing clean, safe, reliable and affordable energy with excellent service is our mission. The company has electric operating companies in three states, natural gas distribution companies in four states, a competitive generation company, a leading distributed energy infrastructure company with national capabilities, a fiber optics network, and telecommunications services. Through an industry-leading commitment to innovation, resilience, and sustainability, we are taking action to meet our customers’ and communities’ needs while advancing our commitment to net zero emissions by 2050. Our uncompromising values ensure we put the needs of those we serve at the center of everything we do and are the key to our sustained success. We are transforming energy into economic, environmental and social progress for tomorrow. Our corporate culture and hiring practices have earned the company national awards and recognition from numerous organizations, including Forbes, Military Times, DiversityInc, Black Enterprise, J.D. Power, Fortune, Human Rights Campaign and more. To learn more, visit www.southerncompany.com.

Southern Company is an equal opportunity employer where an applicant's qualifications are considered without regard to race, color, religion, sex, national origin, age, disability, veteran status, genetic information, sexual orientation, gender identity or expression, or any other basis prohibited by law.

Job Identification: 5872

Job Category: Information Technology

Job Schedule: Full time

Company: Southern Company Services

Apply