Infinite Indexing Problem - indexing wrong mounts in Linux which will take infinite time #3993
Description
Description
I am a beginner with opengrok. I was following all the instructions given on https://github.com/oracle/opengrok/wiki/How-to-setup-OpenGrok to have my opengrok instance running. Earlier, when I was running my instance, it worked fine and as per expectations. But, suddenly when I try to re-index my sources it runs infinitely as it tries to scan other mounts than specified in the -s flag, which is supposed to take infinitely long time. I am not sure what is wrong here. When I try to stop the re-indexing by Ctrl+C in terminal, it doesn't even exit and when I try to kill my terminal instance, the command goes into a state.
Opengrok: 1.7.33
JDK: 11.0.8
OS: RHEL 7
Tomcat: 10.0.22
To Reproduce
Steps to reproduce the behavior: Followed the steps given in the documentation, nothing special. Ran intially smoothly, but doesn't index now.
Opengrok Logs
2022-07-15 17:13:45.723+0530 FINE t1 Executor.registerErrorHandler: Installing default uncaught exception handler
2022-07-15 17:13:45.884+0530 INFO t1 Indexer.parseOptions: Indexer options: [-c, /path/to/ctags, -s, /path/to/opengrok/src, -d, /path/to/opengrok/data, -H, -P, -S, -G, -W, /path/to/opengrok/etc/configuration.xml, -U, http://localhost:8080/source]
2022-07-15 17:13:45.927+0530 FINE t1 HostUtil.isReachable: URI http://localhost:8080/source is reachable via localhost/127.0.0.1
2022-07-15 17:13:46.194+0530 FINER t1 ReflectionHelper$2.run: Unable to load class org.osgi.framework.BundleReference using the current class loader.
///////////////////////////////////////////////////////////////////////
<Unwanted traversal of various files not mentioned anywhere like my .bashrc, and other paths not intentional>
///////////////////////////////////////////////////////////////////////
2022-07-15 17:13:46.939+0530 FINER t1 HistoryGuru.addRepositories: Discovered repositories: []
2022-07-15 17:13:46.939+0530 INFO t1 Statistics.logIt: Done scanning for repositories, found 0 repositories (took 17 ms)
2022-07-15 17:13:46.940+0530 INFO t1 Indexer.prepareIndexer: Generating history cache for all repositories ...
2022-07-15 17:13:46.940+0530 WARNING t1 HistoryGuru.createCacheReal: History cache is enabled however the list of repositories is empty. Either specify the repositories in configuration or let the indexer scan them.
2022-07-15 17:13:46.940+0530 INFO t1 Indexer.prepareIndexer: Done generating history cache
2022-07-15 17:13:46.943+0530 INFO t1 Indexer.doIndexerExecution: Starting indexing
2022-07-15 17:13:47.004+0530 INFO t1 Indexer.doIndexerExecution: Waiting for the executors to finish
The log stops here and now it runs infinitely. Obviously because its is trying to scan everythig in linux server.
strace.out
11480 open("/sys/<unwanted-path>", O_RDONLY <unfinished ...>
11474 <... gettid resumed>) = 11474
11502 <... mmap resumed>) = 0x7ff0cc000000
11496 <... gettid resumed>) = 11496
11486 gettid( <unfinished ...>
11481 <... gettid resumed>) = 11481
11480 <... open resumed>) = 78</sys/<unwanted-path>>
11502 mprotect(0x7ff0cc000000, 135168, PROT_READ|PROT_WRITE <unfinished ...>
11496 gettid( <unfinished ...>
11486 <... gettid resumed>) = 11486
11481 gettid( <unfinished ...>
11480 fstat(78</sys/<unwanted-path>>, <unfinished ...>
11474 gettid( <unfinished ...>
11502 <... mprotect resumed>) = 0
11496 <... gettid resumed>) = 11496
11486 gettid( <unfinished ...>
11481 <... gettid resumed>) = 11481
11480 <... fstat resumed>{st_mode=S_IFREG|0444, st_size=0, ...}) = 0
11474 <... gettid resumed>) = 11474
11502 sched_getaffinity(11502, 32, <unfinished ...>
11486 <... gettid resumed>) = 11486
11481 gettid( <unfinished ...>
11480 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0 <unfinished ...>
11502 <... sched_getaffinity resumed>0x7ff0cc0008c0) = -1 EINVAL (Invalid argument)
11496 gettid( <unfinished ...>
11481 <... gettid resumed>) = 11481
11474 gettid( <unfinished ...>
11502 sched_getaffinity(11502, 64, <unfinished ...>
11496 <... gettid resumed>) = 11496
11486 gettid( <unfinished ...>
11481 gettid( <unfinished ...>
11480 <... mmap resumed>) = 0x7ff17b7d7000
11474 <... gettid resumed>) = 11474
11502 <... sched_getaffinity resumed>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]) = 64
11496 gettid( <unfinished ...>
11486 <... gettid resumed>) = 11486
11481 <... gettid resumed>) = 11481
11480 read(78</sys/<unwanted-path>>, <unfinished ...>
11502 sched_getaffinity(11502, 32, <unfinished ...>
11496 <... gettid resumed>) = 11496
11481 gettid( <unfinished ...>
11474 gettid( <unfinished ...>
11502 <... sched_getaffinity resumed>0x7ff0cc000910) = -1 EINVAL (Invalid argument)
11496 gettid( <unfinished ...>
11481 <... gettid resumed>) = 11481
11480 <... read resumed>"13893746688\n", 4096) = 12
11474 <... gettid resumed>) = 11474
11502 sched_getaffinity(11502, 64, <unfinished ...>
11496 <... gettid resumed>) = 11496
11481 gettid( <unfinished ...>
11480 close(78</sys/<unwanted-path>> <unfinished ...>
11502 <... sched_getaffinity resumed>[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]) = 64
11481 <... gettid resumed>) = 11481
11496 gettid( <unfinished ...>
Above shows I think that it tries to scan unintentional mounts.
Command Run
java \
-Djava.util.logging.config.file=/path/to/opengrok/etc/logging.properties \
-jar /path/to/opengrok/dist/lib/opengrok.jar \
-c /path/to/bin/ctags \
-s /path/to/opengrok/src -d /path/to/opengrok/data -H -P -S -G \
-W /path/to/opengrok/etc/configuration.xml -U http://localhost:8080/source
Please help to resolve this issue if its a bug. Thanks!