Jul 30, 2022
Great article, Ani. I went through one of the sections in this article where you have considered 2 records as an example. Therefore, would like to understand about deciding the number of partitions required for a dataset. How can I decide how many partitions could be required/consumed by my current dataset? Or is it something related to trial and error? Or is there any way using which I can actually decide the number of partitions required?