Using tags causes different (and poor) search results

So I have an application searching with an extra category select as option. Whenever it's enabled, results are not found that are found when the tag isn't applied, even though the products clearly did get correctly assigned the tag, since a shorter query with the same tag does find the items.

For example, all the initial 3 searches include the alfa romeo 166 as a result, however once both the tag (using the "category" selector) and the search query are for "alfa romeo", it will not find any results, while logically you would expect it to be found given the previous 3 results.