{
    "content": [
        {
            "type": "text",
            "text": "# libwww::lwpcook (perldoc)\n\n## NAME\n\nlwpcook - The libwww-perl cookbook\n\n## DESCRIPTION\n\nThis document contain some examples that show typical usage of the libwww-perl library. You\nshould consult the documentation for the individual modules for more detail.\n\n## Sections\n\n- **NAME**\n- **DESCRIPTION**\n- **GET**\n- **HEAD**\n- **POST**\n- **PROXIES**\n- **ACCESS TO PROTECTED DOCUMENTS** (1 subsections)\n- **COOKIES**\n- **HTTPS**\n- **MIRRORING**\n- **LARGE DOCUMENTS**\n- **COPYRIGHT**\n\nUse structuredContent.sections for detailed options, examples, and full documentation.\n"
        }
    ],
    "structuredContent": {
        "command": "libwww::lwpcook",
        "section": "",
        "mode": "perldoc",
        "summary": "lwpcook - The libwww-perl cookbook",
        "synopsis": null,
        "tldr_summary": null,
        "tldr_examples": [],
        "tldr_source": null,
        "flags": [],
        "examples": [],
        "see_also": [],
        "section_outline": [
            {
                "name": "NAME",
                "lines": 2,
                "subsections": []
            },
            {
                "name": "DESCRIPTION",
                "lines": 6,
                "subsections": []
            },
            {
                "name": "GET",
                "lines": 48,
                "subsections": []
            },
            {
                "name": "HEAD",
                "lines": 16,
                "subsections": []
            },
            {
                "name": "POST",
                "lines": 30,
                "subsections": []
            },
            {
                "name": "PROXIES",
                "lines": 35,
                "subsections": []
            },
            {
                "name": "ACCESS TO PROTECTED DOCUMENTS",
                "lines": 9,
                "subsections": [
                    {
                        "name": "get_basic_credentials",
                        "lines": 1
                    }
                ]
            },
            {
                "name": "COOKIES",
                "lines": 16,
                "subsections": []
            },
            {
                "name": "HTTPS",
                "lines": 20,
                "subsections": []
            },
            {
                "name": "MIRRORING",
                "lines": 22,
                "subsections": []
            },
            {
                "name": "LARGE DOCUMENTS",
                "lines": 45,
                "subsections": []
            },
            {
                "name": "COPYRIGHT",
                "lines": 5,
                "subsections": []
            }
        ],
        "sections": {
            "NAME": {
                "content": "lwpcook - The libwww-perl cookbook\n",
                "subsections": []
            },
            "DESCRIPTION": {
                "content": "This document contain some examples that show typical usage of the libwww-perl library. You\nshould consult the documentation for the individual modules for more detail.\n\nAll examples should be runnable programs. You can, in most cases, test the code sections by\npiping the program text directly to perl.\n",
                "subsections": []
            },
            "GET": {
                "content": "It is very easy to use this library to just fetch documents from the net. The LWP::Simple module\nprovides the get() function that return the document specified by its URL argument:\n\nuse LWP::Simple;\n$doc = get 'http://search.cpan.org/dist/libwww-perl/';\n\nor, as a perl one-liner using the getprint() function:\n\nperl -MLWP::Simple -e 'getprint \"http://search.cpan.org/dist/libwww-perl/\"'\n\nor, how about fetching the latest perl by running this command:\n\nperl -MLWP::Simple -e '\ngetstore \"ftp://ftp.sunet.se/pub/lang/perl/CPAN/src/latest.tar.gz\",\n\"perl.tar.gz\"'\n\nYou will probably first want to find a CPAN site closer to you by running something like the\nfollowing command:\n\nperl -MLWP::Simple -e 'getprint \"http://www.cpan.org/SITES.html\"'\n\nEnough of this simple stuff! The LWP object oriented interface gives you more control over the\nrequest sent to the server. Using this interface you have full control over headers sent and how\nyou want to handle the response returned.\n\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n$ua->agent(\"$0/0.1 \" . $ua->agent);\n# $ua->agent(\"Mozilla/8.0\") # pretend we are very capable browser\n\n$req = HTTP::Request->new(\nGET => 'http://search.cpan.org/dist/libwww-perl/');\n$req->header('Accept' => 'text/html');\n\n# send request\n$res = $ua->request($req);\n\n# check the outcome\nif ($res->issuccess) {\nprint $res->decodedcontent;\n}\nelse {\nprint \"Error: \" . $res->statusline . \"\\n\";\n}\n\nThe lwp-request program (alias GET) that is distributed with the library can also be used to\nfetch documents from WWW servers.\n",
                "subsections": []
            },
            "HEAD": {
                "content": "If you just want to check if a document is present (i.e. the URL is valid) try to run code that\nlooks like this:\n\nuse LWP::Simple;\n\nif (head($url)) {\n# ok document exists\n}\n\nThe head() function really returns a list of meta-information about the document. The first\nthree values of the list returned are the document type, the size of the document, and the age\nof the document.\n\nMore control over the request or access to all header values returned require that you use the\nobject oriented interface described for GET above. Just s/GET/HEAD/g.\n",
                "subsections": []
            },
            "POST": {
                "content": "There is no simple procedural interface for posting data to a WWW server. You must use the\nobject oriented interface for this. The most common POST operation is to access a WWW form\napplication:\n\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n\nmy $req = HTTP::Request->new(\nPOST => 'https://rt.cpan.org/Public/Dist/Display.html');\n$req->contenttype('application/x-www-form-urlencoded');\n$req->content('Status=Active&Name=libwww-perl');\n\nmy $res = $ua->request($req);\nprint $res->asstring;\n\nLazy people use the HTTP::Request::Common module to set up a suitable POST request message (it\nhandles all the escaping issues) and has a suitable default for the contenttype:\n\nuse HTTP::Request::Common qw(POST);\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n\nmy $req = POST 'https://rt.cpan.org/Public/Dist/Display.html',\n[ Status => 'Active', Name => 'libwww-perl' ];\n\nprint $ua->request($req)->asstring;\n\nThe lwp-request program (alias POST) that is distributed with the library can also be used for\nposting data.\n",
                "subsections": []
            },
            "PROXIES": {
                "content": "Some sites use proxies to go through fire wall machines, or just as cache in order to improve\nperformance. Proxies can also be used for accessing resources through protocols not supported\ndirectly (or supported badly :-) by the libwww-perl library.\n\nYou should initialize your proxy setting before you start sending requests:\n\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n$ua->envproxy; # initialize from environment variables\n# or\n$ua->proxy(ftp  => 'http://proxy.myorg.com');\n$ua->proxy(wais => 'http://proxy.myorg.com');\n$ua->noproxy(qw(no se fi));\n\nmy $req = HTTP::Request->new(GET => 'wais://xxx.com/');\nprint $ua->request($req)->asstring;\n\nThe LWP::Simple interface will call envproxy() for you automatically. Applications that use the\n$ua->envproxy() method will normally not use the $ua->proxy() and $ua->noproxy() methods.\n\nSome proxies also require that you send it a username/password in order to let requests through.\nYou should be able to add the required header, with something like this:\n\nuse LWP::UserAgent;\n\n$ua = LWP::UserAgent->new;\n$ua->proxy(['http', 'ftp'] => 'http://username:password@proxy.myorg.com');\n\n$req = HTTP::Request->new('GET',\"http://www.perl.com\");\n\n$res = $ua->request($req);\nprint $res->decodedcontent if $res->issuccess;\n\nReplace \"proxy.myorg.com\", \"username\" and \"password\" with something suitable for your site.\n",
                "subsections": []
            },
            "ACCESS TO PROTECTED DOCUMENTS": {
                "content": "Documents protected by basic authorization can easily be accessed like this:\n\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n$req = HTTP::Request->new(GET => 'http://www.linpro.no/secret/');\n$req->authorizationbasic('aas', 'mypassword');\nprint $ua->request($req)->asstring;\n\nThe other alternative is to provide a subclass of *LWP::UserAgent* that overrides the",
                "subsections": [
                    {
                        "name": "get_basic_credentials",
                        "content": ""
                    }
                ]
            },
            "COOKIES": {
                "content": "Some sites like to play games with cookies. By default LWP ignores cookies provided by the\nservers it visits. LWP will collect cookies and respond to cookie requests if you set up a\ncookie jar. LWP doesn't provide a cookie jar itself, but if you install HTTP::CookieJar::LWP, it\ncan be used like this:\n\nuse LWP::UserAgent;\nuse HTTP::CookieJar::LWP;\n\n$ua = LWP::UserAgent->new(\ncookiejar => HTTP::CookieJar::LWP->new,\n);\n\n# and then send requests just as you used to do\n$res = $ua->request(HTTP::Request->new(GET => \"http://no.yahoo.com/\"));\nprint $res->statusline, \"\\n\";\n",
                "subsections": []
            },
            "HTTPS": {
                "content": "URLs with https scheme are accessed in exactly the same way as with http scheme, provided that\nan SSL interface module for LWP has been properly installed (see the README.SSL file found in\nthe libwww-perl distribution for more details). If no SSL interface is installed for LWP to use,\nthen you will get \"501 Protocol scheme 'https' is not supported\" errors when accessing such\nURLs.\n\nHere's an example of fetching and printing a WWW page using SSL:\n\nuse LWP::UserAgent;\n\nmy $ua = LWP::UserAgent->new;\nmy $req = HTTP::Request->new(GET => 'https://www.helsinki.fi/');\nmy $res = $ua->request($req);\nif ($res->issuccess) {\nprint $res->asstring;\n}\nelse {\nprint \"Failed: \", $res->statusline, \"\\n\";\n}\n",
                "subsections": []
            },
            "MIRRORING": {
                "content": "If you want to mirror documents from a WWW server, then try to run code similar to this at\nregular intervals:\n\nuse LWP::Simple;\n\n%mirrors = (\n'http://www.sn.no/'                       => 'sn.html',\n'http://www.perl.com/'                    => 'perl.html',\n'http://search.cpan.org/distlibwww-perl/' => 'lwp.html',\n'gopher://gopher.sn.no/'                  => 'gopher.html',\n);\n\nwhile (($url, $localfile) = each(%mirrors)) {\nmirror($url, $localfile);\n}\n\nOr, as a perl one-liner:\n\nperl -MLWP::Simple -e 'mirror(\"http://www.perl.com/\", \"perl.html\")';\n\nThe document will not be transferred unless it has been updated.\n",
                "subsections": []
            },
            "LARGE DOCUMENTS": {
                "content": "If the document you want to fetch is too large to be kept in memory, then you have two\nalternatives. You can instruct the library to write the document content to a file (second\n$ua->request() argument is a file name):\n\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n\nmy $req = HTTP::Request->new(GET =>\n'http://www.cpan.org/CPAN/authors/id/O/OA/OALDERS/libwww-perl-6.26.tar.gz');\n$res = $ua->request($req, \"libwww-perl.tar.gz\");\nif ($res->issuccess) {\nprint \"ok\\n\";\n}\nelse {\nprint $res->statusline, \"\\n\";\n}\n\nOr you can process the document as it arrives (second $ua->request() argument is a code\nreference):\n\nuse LWP::UserAgent;\n$ua = LWP::UserAgent->new;\n$URL = 'ftp://ftp.isc.org/pub/rfc/rfc-index.txt';\n\nmy $expectedlength;\nmy $bytesreceived = 0;\nmy $res =\n$ua->request(HTTP::Request->new(GET => $URL),\nsub {\nmy($chunk, $res) = @;\n$bytesreceived += length($chunk);\nunless (defined $expectedlength) {\n$expectedlength = $res->contentlength || 0;\n}\nif ($expectedlength) {\nprintf STDERR \"%d%% - \",\n100 * $bytesreceived / $expectedlength;\n}\nprint STDERR \"$bytesreceived bytes received\\n\";\n\n# XXX Should really do something with the chunk itself\n# print $chunk;\n});\nprint $res->statusline, \"\\n\";\n",
                "subsections": []
            },
            "COPYRIGHT": {
                "content": "Copyright 1996-2001, Gisle Aas\n\nThis library is free software; you can redistribute it and/or modify it under the same terms as\nPerl itself.\n",
                "subsections": []
            }
        }
    }
}