Regex-support for dnsmasq

@Somebodyisnobody commented on Mar 27, 2020, 2:19 PM UTC:

PiHole FTL and dnsmasq does support regex. We could shrink the file size save ourselves the www. entries. Are there any discussions at the moment about it?

This issue was moved by funilrys from mitchellkrogza/Ultimate.Hosts.Blacklist#564.

[funilrys]

Hi @Somebodyisnobody,
Interesting. Actually not but we could implement it in near the future!

---

Hey @dnmTX @smed79 @mitchellkrogza @Ultimate-Hosts-Blacklist/contributors what do you think of this feature request?

Stay safe and healthy!

[spirillen]

Just to give a little feedback on what you are asking for.

What you'll need to achieve this is minimum one list which hold all domain fir which you are able to blacklist by a wildcard, I did start this once, also to be working for this list (forgot where it is) but that will be quit some work (days of work), on the other hand, you could use any existing wildcard list such as mine https://github.com/mypdns/matrix/tree/master/source, but as I also know you are not "prepared" to use all of them as is I would suggest you hold those against a whitelist before running the process this could safe you a lot of time, and at the same time you could help contributing to those added into my project, we would all then be benefiting from each others time and knowledge.

[Somebodyisnobody]

What you'll need to achieve this is minimum one list which hold all domain fir which you are able to blacklist by a wildcard

So you mean that wildcard blocking would not give the possibility to exempt some subdomains? And that we have no index of active subdomains if we want to block only 99 of 100 subdomains of example.com? You fear that the last of 100 would be also blocked? I don't want block in wildcard style, I want to use regex. And for hermless sites like newspaperexample.com only ads.newspaperexample.com could be blocked without using regex.

I think for something like this:
(ads|advertising|tracking|mining).newspaperexample.com
instead of:

ads.newspaperexample.com
advertising.newspaperexample.com
tracking.newspaperexample.com
mining.newspaperexample.com

newspaperexample.com itself would still be callable.

When I understand it right @funilrys plans to redo the backend with a shadowdatabase. A central database where lists are indexed, sorted and where black- and whitelister can add/remove entries and where in the end a new list is generated from the database. In the last step, the generation could create such a regex-list:
database:

id	domain	source	cur_revision	ro_locked_by_admin
1	ads.newspaperexample.com	1	a518b8fb-83fc-4c17-86a2-6791c82e691a
2	advertising.newspaperexample.com	1	d1964bad-b252-4410-8416-402174341b8d
3	tracking.newspaperexample.com	3	d1964bad-b252-4410-8416-402174341b8d
4	mining.newspaperexample.com	2	3d631393-cccc-47cb-9c5e-2cf0e23566a3
5	www.badsite.com	17	a6250a84-3eac-42e1-835a-32fc52a24bc5	1
6	badsite.com	17	fb298064-28d8-43cb-8a98-dcdad0489f91	1

generation result:

^(ads|advertising|tracking|mining).newspaperexample.com$
^(www\.)?badsite.com$

or *.badsite.com depending how we interpret the target of blocking www.badsite.com and badsite.com

So if I understand you right then I don't see the logic problem for automatic generation of such a list.

[spirillen]

I think for something like this:
(ads|advertising|tracking|mining).newspaperexample.com

That's one it new to me in conjunction with dnsmasq 😃 but clearly improvement

When I understand it right @funilrys plans to redo the backend with a shadowdatabase. A central database where lists are indexed

Yes, you can read a bit more about that here: https://www.mypdns.org/project/view/15/, and please do feel free to come up with any idea, as you usually have some good once 😃

generation result:

^(ads|advertising|tracking|mining).newspaperexample.com$
^(www\.)?badsite.com$

But what about example google-analytics.com, 207.net or 2o7.net would you append all subdomains to them as well? or would you rather just wildcard them out?

In either way, you would need a list (DB) which hold all exception (WhiteList) and what should just be using the wildcard feature *.google-analytics.com

[rusty-snake]

I think for something like this:
(ads|advertising|tracking|mining).newspaperexample.com

That's one it new to me in conjunction with dnsmasq smiley but clearly improvement

Because dnsmasq (upstream) has no regex support. Only patched versions like this.

[Somebodyisnobody]

Because dnsmasq (upstream) has no regex support. Only patched versions like this.

Oh really? You're blowing up my dreams... But at least Pihole supports regex: https://docs.pi-hole.net/ftldns/regex/tutorial/

But what about example google-analytics.com, 207.net or 2o7.net would you append all subdomains to them as well? or would you rather just wildcard them out?

Depends on how we're defining for example google-analytics.com. If there's no subdomain worth to be whitelisted I would say *.google-analytics.com or in regex-form ^\w*\.google-analytics\.com$. In doubt we always have a list of bad subdomains otherwise and can do ^(...|...|...).google-analytics.com$