Big Oracle tables are too big

[nikodemas]

Impact of the new feature
DBS on Oracle and CMS Monitoring ?

Is your feature request related to a problem? Please describe.
We are trying to move our Sqoop dumps to Spark as Sqoop is officially retired by the Apache. However, Spark underperforms compared to Sqoop on big tables that are not partitioned and DBS has a few of those (FILES, FILE_LUMIS and FILES_PARENTS) and overall it takes a very long time to get any results from these tables.

Describe the solution you'd like
Make biggest tables on Oracle partitioned. Or maybe create a better distributed index?

Describe alternatives you've considered
We are currently staying with our sqoop solution, but eventually we need to migrate. We have considered to open more concurrent sessions to the OracleDB on our spark jobs, but that puts too much load on the database.

Related issues:

• DBS R&D for large tables #84

FYI @Carlosbogo @leggerf

[todor-ivanov]

I am linking this issue to the previous R&D on the topic done by @vkuznet and tracked here:

• DBS R&D for large tables #84

NOTE: Not closing any of them both as of now, until further actions are taken.