[REVIEW] Add MultiIndex support for Dataframes and Series

[thomcom]

This PR adds the MultiIndex class to cudf. MultiIndex (MI) is used for slicing and manipulating dataframes in a higher dimensionality than 2d. It is of particular importance for groupby, which uses it in an index for and a column form.

This PR creates the class and adds 90% of the public methods from Pandas to the MultiIndex. The core functionality (MI codes) is implemented using cudf dataframes, as one code row must be created for each row or column in the dataframe the index is attached to.

I've left a few important tasks incomplete, as the basic functionality and groupby support is all available now for 0.7. The MI work checklist is below:

MultiIndex checklist

• Create MultiIndex with levels/codes args
• Do MI validation like Pandas
• MultiIndex datatypes:
  • int
  • float
  • string
• MultiIndex as Index
  • Set MultiIndex as DataFrame index
  • Set MultiIndex as Series index
  • MultiIndex .loc row access tuples
  • .index Shape test
  • .index Find indices
  • .index Rebuild index
• StringIndex
• MultiIndex iterator
  • getitem
  • take
• MultiColumn
  • Set MultiIndex as DataFrame columns
  • .columns Shape test
  • .columns Find indices
  • MultiIndex column access tuples
  • .columns Rebuild index
• Convenience methods
  • from_tuples
  • from_dataframe
  • from_product
• Backwards compatibility with Pandas 0.23.4
• MultiIndex slicing
  • None / Wildcards
  • Range / validity_mask0 to validity_mask1
• Deep copy of MI Members
• groupby level into MultiIndex support
• Set item via tuple and add to levels/codes objects

This will fix #483
Fixes #1337
Fixes rapidsai/dask-cudf#191
Fixes rapidsai/dask-cudf#125
Fixes rapidsai/dask-cudf#132

[raydouglass]

rerun tests

[kkraus14]

@thomcom given we're still working on the correct approach for .loc on normal indexes, don't worry about it for MultiIndexes yet 😄, otherwise this is looking pretty good thus far.

[thomcom]

I'm writing the first implementation dumb/slow/naive, based on the assumption that we can use a libcudf gather call for the efficient method in the near future. Once I have 100% test passing we can talk about the more efficient solution and make bindings for gather into cudf. Sound good? More progress coming today.

[kkraus14]

I'm writing the first implementation dumb/slow/naive, based on the assumption that we can use a libcudf gather call for the efficient method in the near future. Once I have 100% test passing we can talk about the more efficient solution and make bindings for gather into cudf. Sound good? More progress coming today.

Sounds perfect, thanks for leading the charge on this!

[thomcom]

Tests aren't passing for obscure circular dependency problems (I think), but I'm able to continue development and run tests locally without any issues. I'm asking for comments at this time. Before the end of the 0.6 dev cycle I intend to add proper multiindex output to groupby results and I hope to add slicing. Slicing is most likely to get bumped.

[kkraus14]

Mostly formatting changes and need an offline code review of some of the groupby code, but otherwise this looks amazing. Great work @thomcom!