Sep 30, 2023
A great example of inefficient code would be using collect() (using when not needed and also assuming insufficient memory for the driver) or the another common scenario would be the data skewness which is usually the common cause of OOM.
Another scenario could be loading a huge unsplittable file format which could bring up the load on a single executor and will definitely result in OOM.
There would be other scenarios as well but I have just listed down the very common scenarios which would have been better if you could have portrayed it in your article.
I find the article a little bit misleading.