Resource usage

[kavemang]

@BlessedRebuS For the time being I have restricted krawl to .6 of one core, with the new features it seems to use a full core of my 8 core i5 fairly constantly

[BlessedRebuS]

@ptarrant what was your usage in the container for the new features?

[kavemang]

on an i5 @ 4ghz it was using 100% of one core, not constantly but maybe about 70% of the time I think, for now I restricted it with docker to use 60% of one core max

[BlessedRebuS]

We could open an issue to improve performance. Keep in mind that clicking the Analyze link repeatedly can consume more CPU, since it runs statistical computations behind the scenes. Also, we’re planning to revisit and refine the scoring system in the future to make it more accurate.

[kavemang]

@BlessedRebuS it looks like the dev branch reanalyzes all the IPs every minute without being prompted, id recommend this only get triggered if there is new access activity on krawl and perhaps have the re-analysis be restricted to the IPs with new access activity

while this 1 minute cycle runes over and over the container uses about 500mb of ram

[ptarrant]

I'll look into it. We don't want to trigger the job every hit, because with bots hitting it constantly that wouldn't be as efficient. We could do a quick poll to see the last access date and exit the job quickly if there hasn't been access in the last 5 minutes. We could also back off the task itself and set it to every 5 minutes. However the check of last access date would most likely be the best way to do it. Memory usage is something we are about to start working on as well. We need to update a good bit of logic to lower the usage but it's doable for sure.

…

On Thu, Jan 15, 2026, 12:17 PM kavemang ***@***.***> wrote: @BlessedRebuS <https://github.com/BlessedRebuS> it looks like the dev branch reanalyzes all the IPs every minute without being prompted, id recommend this only get triggered if there is new access activity on krawl and perhaps have the re-analysis be restricted to the IPs with new access activity while this 1 minute cycle runes over and over the container uses about 500mb of ram — Reply to this email directly, view it on GitHub <#40 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALLPCWAEEQWXARZVPIB3P34G7KUFAVCNFSM6AAAAACRXPVGFWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNJQHE4TSMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[BlessedRebuS]

@kavemang we had a BIG update with memory cleanup, IP request threshold, dashboard additions, etc...
We are planning to release the new stable main version soon

[kavemang]

the latest dev branch is down from 40% of allotted cpu to 1% and from 500MB ram to ~40MB !

[ptarrant]

The demo page stats are looking much better as well.