Questions about current research, and things that appeared on my CV. Questions regarding machine learning, statistics, SQL, high-level programming.
Senior Data Engineer Interview Questions
2,606 senior data engineer interview questions shared by candidates
Tell me about the project you're most proud of.
Initial Interview: Cultural and general tech questions Tech interview: General Python technical questions CTO interview: Motivation, experience and professional interests questions
A developer on the team was running the following line in a function for logging purposes, and the job crashed with an "out of memory" exception. The developer says that the cluster has many workers with a lot of memory and disk and still the job crashes. Can you help explain how come to this line makes the job crash with OOM even though the cluster is huge? def someFunc(): for row in df.collect(): print(f'Customerr{row["customer"]} => Paid {row["price"]}')
What feedback did your manager give you
Describe a project you are particularly proud of
How did you design data lake what are the three layers. How did you design using what services. why this services over the other Difference between schema in data warehouse Whats the purpose of this service. How you would store some secret keys and how you would secure the data. what are the methods and how you did it. whats the typical project activities dev,test,prod the cycle.
persist vs cache? what does init do? args vs kargs? json vs parquet? columnar storage? distributed systems? spark dataframe vs pandas dataframe
How do you manage conflict? How do you break down technical issues to non-technical audience? Have you implemented any Data Governance tools?
Spark and Scala, detecting bank fraud
Viewing 1881 - 1890 interview questions