{
    "mode": "perldoc",
    "parameter": "LWP::RobotUA",
    "section": "",
    "url": "https://www.chedong.com/phpMan.php/perldoc/LWP%3A%3ARobotUA/json",
    "generated": "2026-06-12T11:44:51Z",
    "synopsis": "use LWP::RobotUA;\nmy $ua = LWP::RobotUA->new('my-robot/0.1', 'me@foo.com');\n$ua->delay(10);  # be very nice -- max one hit every ten minutes!\n...\n# Then just use it just like a normal LWP::UserAgent:\nmy $response = $ua->get('http://whatever.int/...');\n...",
    "sections": {
        "NAME": {
            "content": "LWP::RobotUA - a class for well-behaved Web robots\n",
            "subsections": []
        },
        "SYNOPSIS": {
            "content": "use LWP::RobotUA;\nmy $ua = LWP::RobotUA->new('my-robot/0.1', 'me@foo.com');\n$ua->delay(10);  # be very nice -- max one hit every ten minutes!\n...\n\n# Then just use it just like a normal LWP::UserAgent:\nmy $response = $ua->get('http://whatever.int/...');\n...\n",
            "subsections": []
        },
        "DESCRIPTION": {
            "content": "This class implements a user agent that is suitable for robot applications. Robots should be\nnice to the servers they visit. They should consult the /robots.txt file to ensure that they are\nwelcomed and they should not make requests too frequently.\n\nBut before you consider writing a robot, take a look at <http://www.robotstxt.org/>.\n\nWhen you use an *LWP::RobotUA* object as your user agent, then you do not really have to think\nabout these things yourself; \"robots.txt\" files are automatically consulted and obeyed, the\nserver isn't queried too rapidly, and so on. Just send requests as you do when you are using a\nnormal *LWP::UserAgent* object (using \"$ua->get(...)\", \"$ua->head(...)\", \"$ua->request(...)\",\netc.), and this special agent will make sure you are nice.\n",
            "subsections": []
        },
        "METHODS": {
            "content": "The LWP::RobotUA is a sub-class of LWP::UserAgent and implements the same methods. In addition\nthe following methods are provided:\n\nnew\nmy $ua = LWP::RobotUA->new( %options )\nmy $ua = LWP::RobotUA->new( $agent, $from )\nmy $ua = LWP::RobotUA->new( $agent, $from, $rules )\n\nThe LWP::UserAgent options \"agent\" and \"from\" are mandatory. The options \"delay\", \"usesleep\"\nand \"rules\" initialize attributes private to the RobotUA. If \"rules\" are not provided, then\nWWW::RobotRules is instantiated providing an internal database of robots.txt.\n\nIt is also possible to just pass the value of \"agent\", \"from\" and optionally \"rules\" as plain\npositional arguments.\n\ndelay\nmy $delay = $ua->delay;\n$ua->delay( $minutes );\n\nGet/set the minimum delay between requests to the same server, in *minutes*. The default is 1\nminute. Note that this number doesn't have to be an integer; for example, this sets the delay to\n10 seconds:\n\n$ua->delay(10/60);\n\nusesleep\nmy $bool = $ua->usesleep;\n$ua->usesleep( $boolean );\n\nGet/set a value indicating whether the UA should \"sleep\" in LWP::RobotUA if requests arrive too\nfast, defined as \"$ua->delay\" minutes not passed since last request to the given server. The\ndefault is true. If this value is false then an internal \"SERVICEUNAVAILABLE\" response will be\ngenerated. It will have a \"Retry-After\" header that indicates when it is OK to send another\nrequest to this server.\n\nrules\nmy $rules = $ua->rules;\n$ua->rules( $rules );\n\nSet/get which *WWW::RobotRules* object to use.\n\nnovisits\nmy $num = $ua->novisits( $netloc )\n\nReturns the number of documents fetched from this server host. Yeah I know, this method should\nprobably have been named \"numvisits\" or something like that. :-(\n\nhostwait\nmy $num = $ua->hostwait( $netloc )\n\nReturns the number of *seconds* (from now) you must wait before you can make a new request to\nthis host.\n\nasstring\nmy $string = $ua->asstring;\n\nReturns a string that describes the state of the UA. Mainly useful for debugging.\n",
            "subsections": []
        },
        "SEE ALSO": {
            "content": "LWP::UserAgent, WWW::RobotRules\n",
            "subsections": []
        },
        "COPYRIGHT": {
            "content": "Copyright 1996-2004 Gisle Aas.\n\nThis library is free software; you can redistribute it and/or modify it under the same terms as\nPerl itself.\n",
            "subsections": []
        }
    },
    "summary": "LWP::RobotUA - a class for well-behaved Web robots",
    "flags": [],
    "examples": [],
    "see_also": []
}