ENXF NET
Administrator
Staff member
Administrator
Moderator
+Lifetime VIP+
S.V.I.P.S Member
S.V.I.P Member
V.I.P Member
Collaborate
Registered
- Joined
- Nov 13, 2018
- Messages
- 34,419
- Points
- 823
Various robots ignore both the Disallow: /misc/cookies directive in robots.txt and the noindex directive used in the misc_cookies template, and still follow the generated cookie consent links.
One example is the robot identified by XenForo as Meta External Agent, but the same behavior can also be observed with other crawlers and bots.
This results in a significant number of unnecessary requests to /misc/cookies, which XenForo correctly handles and rejects when the required...
Read more
Continue reading...
One example is the robot identified by XenForo as Meta External Agent, but the same behavior can also be observed with other crawlers and bots.
This results in a significant number of unnecessary requests to /misc/cookies, which XenForo correctly handles and rejects when the required...
Read more
Continue reading...