filibot is a specialized web crawler designed to access and analyze the SEO signals across websites on the internet. Most likely your website was requested by filibot when someone is manually using one of Fili's free SEO tools to analyze the SEO signals of your site.
You can identify filibot by any of the following user agent strings:
filibot (+https://filibot.com)
or
Mozilla/5.0 (compatible; filibot; +https://filibot.com)
or
Mozilla/5.0 (compatible; filibot/1.0; +https://filibot.com)
The primary purpose of filibot is to crawl and analyze the SEO signals of your website.
filibot is designed to access individual websites mostly upon user request. Which means that most websites will receive traffic from filibot extremely infrequently, if at all.
SEO Tools which may be using filibot include, but are not limited to:
Note: It's possible that this timeframe may be more or less frequent in some edge cases. filibot is engineered to minimize its impact and should not pose a burden on most sites. However, due to network delays it's possible that the rate will appear to be slightly higher over short periods.
filibot respects robots.txt directives. If robots.txt directives don't mention filibot specifically but do mention Googlebot, then filibot will follow Googlebot directives. You can block filibot using the following methods:
Add the following lines to your robots.txt file:
User-agent: filibot Disallow: /
Important: This method will not prevent filibot from accessing the robots.txt file itself, as defined in section 2.2.2 of RFC 9309.
Alternatively, block the following IP addresses at the server/firewall level:
Important: Blocking using robots.txt is the recommeded and most reliable method, because these IP addresses may change over time. New addresses may be added, old ones may be removed and temporary IP addresses may be used.
If you have any questions or concerns regarding filibot, please reach out at filibot@filibot.com.