What is wget?
Unknown operator · script · class automation
wget is a script crawler.
Verdict: Charge it or block it. It did not say who it was.
On the mugshot wall: Unidentified: no operator, no documentation
Profile
Compiled by Crawler Atlas from the operator's published documentation. Not yet declared by the operator. JSON
| Operator | Unknown |
|---|---|
| Purpose | Scripts and scrapers. Bare HTTP libraries, scraping frameworks, headless browsers. Nobody identified themselves; paid by default. |
| User-agent contains | wget/ |
| robots.txt | No documented token. Block at the edge or by user-agent match. |
| How to verify | No published verification method. Treat the name as a claim, not proof. |
| Should you block it? | Charge it or block it. It did not say who it was. |
| Does it pay? | No. It does not carry a wallet and cannot answer a 402. A site that charges simply serves it nothing. |
| Docs | None published |
Observed on the network
Anonymous daily counts from sites running Pay-per-Crawl that opted in, plus the ForgeMesh fleet. Last 7 days. Small sample: 1 sites reporting.
No sightings reported yet. wget appears here the day an opted-in site sees it.
Do something about it
- See it in your own logs, verify it, and charge, allow or block it by name with the free Pay-per-Crawl plugin (WordPress, Python, JavaScript, Go, PHP).
- Run this crawler? Publish a crawler-profile.json on your domain and this page shows your declared profile with a verified badge.
- Machine-readable twin: /bots/wget/profile.json (free in a browser). Machines: POST
{"slug":"wget"}to x402.forgemesh.io/bot-profiles, $0.005 per call over x402.
Also in scripts and scrapers
Crawler Atlas is maintained by ForgeMesh. Names and marks belong to their operators and are used to identify their crawlers.