About project in details. Python Pyspark Aws cloud
Senior Data Engineer Interview Questions
2,606 senior data engineer interview questions shared by candidates
2nd highest salary, broadcast join, catalyst optimizer, salting
Very broad questions of system design around tracks and reports website traffic.
which model to get results from a cube with low latency? what are the models in warehouse? how to use merge statement in which scenario? what motivates you to work? interviewer has some timelines sometimes with good plan sometime have to deliver at gunpoint..etc, how comfortable are you..etc?
What's the hardest data set you've come to work with and why?
They asked SQL and Python questions, the questions were challenging but would have been a fair play if we were given a code playground to test our queries and then submit the final answer.
A. Core Data Engineering Concepts SQL (joins, window functions, performance tuning) Data Modeling (star vs snowflake, normalization) ETL/ELT pipelines (batch vs streaming, orchestration tools like Airflow) B. Apache Spark / PySpark Catalyst Optimizer & Tungsten Narrow vs Wide transformations Joins (broadcast, sort-merge), Skew handling AQE (Adaptive Query Execution) Partitioning, Predicate Pushdown Execution Plan (DAG → Stage → Tasks) Spark UI and Job Debugging SCD Type 2 Implementation in PySpark C. AWS S3, Glue, Athena, Lambda, EMR, Redshift Event-driven design (S3 → EventBridge → Lambda) Security: IAM roles, bucket policies, encryption CI/CD in AWS (CodePipeline, CloudFormation) D. Python Writing modular, reusable code Working with Pandas, Boto3 (for AWS interaction) Exception handling, logging Lambda functions and decorators E. Kafka / Streaming Kafka topic partitioning, consumer groups Offset management Integration with Spark Structured Streaming
Pyspark memory optimization, different types of keys in SQL
Why are long process of release offer letter after interview is done.
Linked list reversal with pointers.
Viewing 1671 - 1680 interview questions