Consider a employee dataframe with columns – empID, department, salary. Get minimum, maximum, average salary and employee count for each department. Write it in a single statement.
Sr Data Engineer Interview Questions
2,606 sr data engineer interview questions shared by candidates
They asked SQL and Python questions, the questions were challenging but would have been a fair play if we were given a code playground to test our queries and then submit the final answer.
A. Core Data Engineering Concepts SQL (joins, window functions, performance tuning) Data Modeling (star vs snowflake, normalization) ETL/ELT pipelines (batch vs streaming, orchestration tools like Airflow) B. Apache Spark / PySpark Catalyst Optimizer & Tungsten Narrow vs Wide transformations Joins (broadcast, sort-merge), Skew handling AQE (Adaptive Query Execution) Partitioning, Predicate Pushdown Execution Plan (DAG → Stage → Tasks) Spark UI and Job Debugging SCD Type 2 Implementation in PySpark C. AWS S3, Glue, Athena, Lambda, EMR, Redshift Event-driven design (S3 → EventBridge → Lambda) Security: IAM roles, bucket policies, encryption CI/CD in AWS (CodePipeline, CloudFormation) D. Python Writing modular, reusable code Working with Pandas, Boto3 (for AWS interaction) Exception handling, logging Lambda functions and decorators E. Kafka / Streaming Kafka topic partitioning, consumer groups Offset management Integration with Spark Structured Streaming
Pyspark memory optimization, different types of keys in SQL
Why are long process of release offer letter after interview is done.
Experience, SQL queries based on big data.
Describe your work on ETL tools
L1: core technical discussion for 1 hour( coding, system design, scenario based) L2: Client interviews ( 2 hours, detailed discussion about system design and integration and analytics, KPI) L3: Client interviews ( 2 hours, live project implementation)
Tell us about how you solved a problem you've experienced before. Try to describe it in detail
Fix a stored procedure, OLAP cube modelling
Viewing 1691 - 1700 interview questions
See Interview Questions for Similar Jobs
Data EngineeringSenior Data EngineerBig Data EngineerSenior Big Data EngineerData EngineerSenior Data ConsultantLead Data EngineerData ScientistsData Scientist IiiPrincipal Data EngineerData Mining EngineerBigdata EngineerSr Data AnalystEtl Data Stage DeveloperSr Database EngineerDatabase Administrator IiSenior Data Stage DeveloperEtl Informatica Developer