Disk space usage of pipelines #157
Description
ocaml-docs-ci uses a concept of Epochs to track sets of package documentation that interlink and share common dependencies. Each Epoch is self-contained and can grow to 100G or more for production.
When they are no longer used Epochs can be removed and the disk space reclaimed. Outside of epochs two directories compile and prep also take up a lot of disk space. But there is no clean way of reclaiming the un-used diskspace from them. For example on live docs.ci.ocaml.org they look like:
$ ls -l
total 940
drwxr-xr-x 4 localadmin localadmin 4096 Oct 12 17:50 compile
-rw-r--r-- 1 root root 459760 Jan 3 2023 content.current
-rw-r--r-- 1 root root 450418 Jan 3 2023 content.live
drwxr-xr-x 3 localadmin localadmin 4096 Aug 11 10:58 epoch-51470fc39ff110dfccafc8f0f1d882ec
drwxr-xr-x 3 localadmin localadmin 4096 Oct 12 17:50 epoch-5daeecab2ad7a2d07a12742d4cc0ab6f
drwxr-xr-x 3 localadmin localadmin 4096 Aug 11 12:23 epoch-804419edbd92ec13bc06b69d15132959
drwxr-xr-x 3 localadmin localadmin 4096 Oct 12 17:51 epoch-ae8bf595b8594945ee40c58377e03730
drwxr-xr-x 3 localadmin localadmin 4096 Jul 20 09:44 epoch-c96e56dbd72372ea5f56e27e667243ea
lrwxrwxrwx 1 root root 44 Aug 16 05:48 html-current -> /data/epoch-ae8bf595b8594945ee40c58377e03730
-rw-r--r-- 1 root root 1365 Aug 16 05:48 html-current.log
lrwxrwxrwx 1 root root 44 Sep 7 13:56 html-live -> /data/epoch-ae8bf595b8594945ee40c58377e03730
-rw-r--r-- 1 root root 1028 Sep 7 13:56 html-live.log
drwxr-xr-x 2 root root 4096 Aug 25 2022 linked
lrwxrwxrwx 1 root root 44 Aug 16 05:48 linked-current -> /data/epoch-5daeecab2ad7a2d07a12742d4cc0ab6f
-rw-r--r-- 1 root root 1690 Aug 16 05:48 linked-current.log
lrwxrwxrwx 1 root root 44 Sep 7 13:56 linked-live -> /data/epoch-5daeecab2ad7a2d07a12742d4cc0ab6f
-rw-r--r-- 1 root root 585 Sep 7 13:56 linked-live.log
lrwxrwxrwx 1 root root 9 Jul 21 2021 live -> html-live
drwxr-xr-x 3 localadmin localadmin 4096 Oct 12 17:45 prep
$ du -skhc *
159G compile
452K content.current
440K content.live
94G epoch-51470fc39ff110dfccafc8f0f1d882ec
28G epoch-5daeecab2ad7a2d07a12742d4cc0ab6f
59G epoch-804419edbd92ec13bc06b69d15132959
47G epoch-ae8bf595b8594945ee40c58377e03730
103G epoch-c96e56dbd72372ea5f56e27e667243ea
0 html-current
4.0K html-current.log
0 html-live
4.0K html-live.log
4.0K linked
0 linked-current
4.0K linked-current.log
0 linked-live
4.0K linked-live.log
0 live
551G prep
1.1T totalBoth directories appear to contain universes but they aren't linked to a specific Epoch and can't be easily pruned.
Further investigation is required to work out how we can reclaim disk space or organise these directories into epochs so they can be reclaimed.