Delta housekeeping initial version

[lorenzorubi-db]

utility on top of discoverx to run Delta Housekeeping across multiple tables

Analysis that provides stats on Delta tables / recommendations for improvements, including:

• stats:size of tables and number of files, timestamps of latest OPTIMIZE & VACUUM operations, stats of OPTIMIZE)
• recommendations on tables that need to be OPTIMIZED/VACUUM'ed
• are tables OPTIMIZED/VACUUM'ed often enough
• tables that have small files / tables for which ZORDER is not being effective

[lorenzorubi-db]

@edurdevic same as PR #95 opened with my user
latest commit takes care of your final comments
thanks!

[lorenzorubi-db]

hi @edurdevic
I still need to review further (and document better) but would like that you take a look so that we agree with the approach
in the end the refactoring was much bigger to what I expected... anyhow now apply gives back a single dataframe with 3 boolean columns:

• rec_optimize with rows that need action with OPTIMIZE
• rec_vacuum analogous for VACUUM
• rec_misc other recommendations

plus 3 string columns with the reasons for each
thanks!

[lorenzorubi-db]

@edurdevic ready to review, thanks

[lorenzorubi-db]

@edurdevic pls take another look, thanks

[edurdevic]

LGTM

[david-tempelmann]

Looks good!