Maven Mirror - 404 initially, succeeds after a while

[rdehuyss]

What happened?

Hi,

First of all, thanks again for this great product!

We recently have an issue where we setup a new reposilite instance that mirrors maven central. I noticed that when the jar is not in the local file system storage, we first get a 404. Only after a couple of minutes and after retrying multiple times, we get the proxied jar.

This is reposilite version 3.5.28. See screenshots below:

First try

Sixth try (after some time)

I had issues for each jar from the downloads.

Reposilite version

3.x

Relevant log output

[dzikoysk]

I'm noticing something quite similar lately and I think this might be related to Maven Central itself. I have a feeling they're over aggressively rejecting requests from some hosts...? I can't reproduce this locally as it always works via direct link, but mirrored Maven Central or even direct downloads of artifacts from Maven Central on GitHub Actions requires multiple retries.

[mmoqui]

I have also a similar behavior. But, by consulting the logs, it seems Reposilite received 429 as HTTP response code from Maven Central (repo1.maven.org). So, it requests the others external repos for which it serves also as a proxy/mirror but those repos don't have the requested artifacts, hence the eventually 404 HTTP error code

[dzikoysk]

That would confirm the comment from there that Maven Central reduced threshold for rate limits:

Hi there! Any updates here?
It seems Maven Central has tightened rate limits.
Reposilite could be a great option to handle this!
I would like to help here, but my Java knowledge is pretty weak.

~ #2221 (comment)

[dzikoysk]

Several improvements will land in 3.6.0:

• Release 3.6.0 #2620

As I said, I don't think it is a problem on our side, but those should simply help us workaround it.

[rdehuyss]

Thanks so much!

[dzikoysk]

3.6.0 is officially released, feel free to enable the new resolution cache and let me know if it helps.

• https://reposilite.com/guide/mirrors#proxying-maven-central

Of course it is in-memory cache, so it requires some time to fill with the data.

By the way, it looks like they also confirm our observations:

• https://central.sonatype.org/faq/429-error/

I'll plan some additional improvements on top of what we've got in 3.6.0 to make this experience smoother, let's see how much we can get out of it.