Splitting hudi-common classes into common and core

[vinothchandar]

All , the hudi-common package has grown in scope. and I have spent sometime with #19193 introducing a new core subpackage here to house all the core Hudi timeline, filesystem view and the consistency/transactional stuff, as well as key I/O. I would like common to be mostly holding common models, common base infrastructure (functions, expressions, schema) used by everyother module.

Eventually, I want to split off hudi-common and hudi-core as separate modules where something like hudi-io can depend on hudi-common. Today due to this mixing of classes with different scope, the dependency is other way around and we are keeping strictly I/O related classes in hudi-common without a choice.

End state looks like this, with proper homing of classes.

hudi-common. // all base models and classes and utils , e.g schema, enginecontext, expression, filegroup, fileslice. ...
   |============> hudi-io  // all core I/O classes, lower level  (ideally including reading/writing files, but can't have hadoop deps) 
       |================> hudi-core // everything around txns, timeline, metadata, filesystemview, hudi specific reader/writers (single  read/write handle level like filegroupreader)
   |============> hudi-hadoop-common // how it is today

Thoughts/suggestions welcome!

[vinothchandar]

I have completed the clean up here. #19195

love to discuss the split here more, before I get started.

[vinothchandar]

Opened a draft PR here.

#19214

Will bring this up in the weekly sync as well

[vinothchandar]

Impact: compile time deps on hudi-common will now need hudi-core as well.

[nsivabalan]

+1 logically the new hierarchy makes sense.

[vinothchandar]

Proceeding with the changes. Thanks for chiming in here and also the dev weekly.