Brainstorming: on_result plugin / https_rewrite plugin

Different plugin ideas require to filter / modify the results according to the host of the URL.

Usage examples:

• the https_rewrite plugin, and the current implementation fails to do so: https://github.com/asciimoo/searx/tree/master/searx/plugins/https_rules are outdated, and the number of rules is a fraction of the actual list to avoid an overhead. Here there is one lookup per .xml files: https://github.com/asciimoo/searx/blob/6e37255e59693d7a3768491b2b8d06441051131f/searx/plugins/https_rewrite.py#L140
• Add support for Youtube and Twitter links rewrite PR #1726 rewrite some URL too.
• feature request: private user blocklists/blacklists #2001 asks to remove some domain names from the results.

Currently, on_result plugin scan for each URLs and process them. What if we provide provide each plugin register for a list of domain.

Some technical solutions to create a static list of domains:

• a trie. (This issue superseed this one https://github.com/asciimoo/searx/issues/936 )
• one hash table per label in the domain name: https://github.com/sethmlarson/hstspreload

According to https://www.eff.org/https-everywhere/rulesets#wildcard-targets the wildcard in HTTPS everywhere can be either:

• *.eff.org
• www.eff.*

A solution: two lookups in two different structures, one in reversed order (org -> eff -> *), one in normal order (www -> eff -> *).

Prototype: https://gist.github.com/dalf/26462c1627c8b00fdfacdd55525b08b6

The idea is provide:

• a format to store the rules (either a binary or plain text format), with a module to create this output format.
• a plugin can register this structure: the searx core calls the plugin where there is a match.
• it is difficult to merge all trie without a lot of repetition, some most probably one "double trie" per plugin; but in simple cases I guess it can be done.

Note about the issue #2001:

• the choice of domains to remove is a preference, then the issue is not solved.
• if we can make some maintained list, then the issue is solved.