Sign in

Cache Management System: Carbon prunes and caches all block/blocklet index information into the driver for normal table to increase the query performance by reducing the number of files which are read. This caching mechanism causes the driver to become a bottleneck in the following ways:

  1. If the cache size becomes huge(70–80% of the driver memory) then there can be excessive GC in the driver which can slow down the query and the driver may even go OutOfMemory.
  2. There will be a hard limit to how much LRU can be saved. In case when the cache is full and it needs…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store