pydoc > bs4

man perldoc info pydoc3 ri search

📖 NAME

bs4 – Beautiful Soup Elixir and Tonic – “The Screen-Scraper’s Friend”.

🚀 Quick Reference

Use Case	Command	Description
Parse HTML	soup = BeautifulSoup(html, 'html.parser')	Parse an HTML string into a BeautifulSoup object.
Parse XML	soup = BeautifulSoup(xml, 'lxml-xml')	Parse an XML string using lxml.
Find all tags	soup.find_all('a')	Find all occurrences of a tag.
Find first tag	soup.find('a')	Find the first occurrence of a tag.
CSS selector	soup.select('.class')	Select elements by CSS selector.
Extract text	soup.get_text()	Get all text from the document.
Get attribute	tag['href']	Get the value of an attribute.
Set attribute	tag['href'] = 'new_url'	Set the value of an attribute.
Navigate tree	tag.parent	Access the parent of a tag.
Iterate children	tag.children	Iterate over direct children.
Pretty print	soup.prettify()	Return indented HTML/XML string.
Modify tag	new_tag = soup.new_tag('div')	Create a new tag and insert it.

📝 DESCRIPTION

http://www.crummy.com/software/BeautifulSoup/

Beautiful Soup uses a pluggable XML or HTML parser to parse a (possibly invalid) document into a tree representation. Beautiful Soup provides methods and Pythonic idioms that make it easy to navigate, search, and modify the parse tree.

Beautiful Soup works with Python 3.6 and up. It works better if lxml and/or html5lib is installed.

For more than you ever wanted to know about Beautiful Soup, see the documentation: http://www.crummy.com/software/BeautifulSoup/bs4/doc/

📦 PACKAGE CONTENTS

• builder (package)
• css
• dammit
• diagnose
• element
• formatter
• tests (package)

🏛️ CLASSES

bs4.element.Tag(bs4.element.PageElement)
BeautifulSoup

🏛️ class BeautifulSoup(bs4.element.Tag)

Constructor: BeautifulSoup(markup='', features=None, builder=None, parse_only=None, from_encoding=None, exclude_encodings=None, element_classes=None, **kwargs)

A data structure representing a parsed HTML or XML document. Most methods are inherited from PageElement or Tag.

📌 Methods defined here

• __getstate__(self)
• __init__(self, markup='', features=None, builder=None, parse_only=None, from_encoding=None, exclude_encodings=None, element_classes=None, **kwargs)
  Constructor.
  • 📄 markup – A string or file-like object representing markup to be parsed.
  • ⚙️ features – Desired parser features (e.g. "lxml", "html.parser", "html5lib") or markup type ("html", "xml").
  • 🔧 builder – A custom TreeBuilder instance.
  • 🔍 parse_only – A SoupStrainer to limit parsing to parts of the document.
  • 🔤 from_encoding – String indicating the document encoding if Beautiful Soup guesses wrongly.
  • 🚫 exclude_encodings – List of encodings known to be wrong.
  • 🧩 element_classes – Dictionary mapping BeautifulSoup classes to custom subclasses.
  • 💡 **kwargs – Propagated to TreeBuilder constructor; BS3 backwards compatibility args are ignored with a warning.
• __setstate__(self, state)
• decode(self, pretty_print=False, eventual_encoding='utf-8', formatter='minimal', iterator=None)
  Returns a string or Unicode representation of the parse tree as HTML/XML.
  • 🎨 pretty_print – If True, indentation is used for readability.
  • 📝 eventual_encoding – The encoding of the final document; if None, returns Unicode string.
• endData(self, containerClass=None)
  Called by the TreeBuilder when the end of a data segment occurs.
• handle_data(self, data)
  Called by the tree builder when a chunk of textual data is encountered.
• handle_endtag(self, name, nsprefix=None)
  Called by the tree builder when an ending tag is encountered.
• handle_starttag(self, name, namespace, nsprefix, attrs, sourceline=None, sourcepos=None, namespaces=None)
  Called by the tree builder when a new tag is encountered. If returns None, the tag was rejected by an active SoupStrainer.
• insert_after(self, *args)
  Part of PageElement API, but not implemented for BeautifulSoup.
• insert_before(self, *args)
  Part of PageElement API, but not implemented for BeautifulSoup.
• new_string(self, s, subclass=None)
  Create a new NavigableString associated with this BeautifulSoup object.
• new_tag(self, name, namespace=None, nsprefix=None, attrs={}, sourceline=None, sourcepos=None, **kwattrs)
  Create a new Tag associated with this BeautifulSoup object.
  • 🏷️ name – The name of the new Tag.
  • 🌐 namespace – XML namespace URI.
  • 🔖 prefix – XML namespace prefix.
  • 📋 attrs – Dictionary of attribute values; can be used instead of kwattrs for reserved words like 'class'.
• object_was_parsed(self, o, parent=None, most_recent_element=None)
  Called by the TreeBuilder to integrate an object into the parse tree.
• popTag(self)
  Internal method called when a tag is closed.
• pushTag(self, tag)
  Internal method called when a tag is opened.
• reset(self)
  Reset this object to a state as though it had never parsed any markup.
• string_container(self, base_class=None)

📊 Data and other attributes defined here

• ASCII_SPACES = ' \n\t\x0c\r'
• DEFAULT_BUILDER_FEATURES = ['html', 'fast']
• NO_PARSER_SPECIFIED_WARNING = 'No parser was explicitly specified, so ...'
• ROOT_TAG_NAME = '[document]'

📜 Methods inherited from bs4.element.Tag

• __bool__(self) – A tag is non‑None even if it has no contents.
• __call__(self, *args, **kwargs) – Calling a Tag like a function is the same as calling its find_all() method.
• __contains__(self, x)
• __copy__(self) – Always a deep copy (children can only have one parent).
• __deepcopy__(self, memo, recursive=True)
• __delitem__(self, key) – Deletes all 'key' attributes.
• __eq__(self, other) – True iff same name, attributes, and contents recursively.
• __getattr__(self, tag) – Same as tag.find(name="subtag").
• __getitem__(self, key) – Returns attribute value; raises exception if missing.
• __hash__(self)
• __iter__(self) – Iterates over contents.
• __len__(self) – Length of contents list.
• __ne__(self, other)
• __repr__ = __unicode__(self)
• __setitem__(self, key, value) – Sets attribute value.
• __str__ = __unicode__(self)
• __unicode__(self) – Renders as Unicode string.
• clear(self, decompose=False) – Wipes out all children via extract(); if decompose is True, uses decompose().
• decode_contents(self, indent_level=None, eventual_encoding='utf-8', formatter='minimal') – Renders contents as Unicode string.
• decompose(self) – Recursively destroys this element and its children.
• encode(self, encoding='utf-8', indent_level=None, formatter='minimal', errors='xmlcharrefreplace') – Render as bytestring.
• encode_contents(self, indent_level=None, encoding='utf-8', formatter='minimal') – Render contents as bytestring.
• find(self, name=None, attrs={}, recursive=True, string=None, **kwargs) – Find first PageElement matching criteria.
• find_all(self, name=None, attrs={}, recursive=True, string=None, limit=None, **kwargs) – Find all matching PageElements; returns ResultSet.
• findAll = find_all
• findChild = find
• findChildren = find_all
• get(self, key, default=None) – Returns attribute value or default.
• get_attribute_list(self, key, default=None) – Same as get() but always returns a list.
• has_attr(self, key) – Does this element have the attribute?
• has_key(self, key) – Deprecated.
• index(self, element) – Find index of a child by identity.
• prettify(self, encoding=None, formatter='minimal') – Pretty‑print as string.
• select(self, selector, namespaces=None, limit=None, **kwargs) – Perform CSS selection; returns ResultSet of Tags.
• select_one(self, selector, namespaces=None, **kwargs) – Perform CSS selection; returns first Tag.
• smooth(self) – Consolidate consecutive strings.

🔍 Readonly properties inherited from bs4.element.Tag

• children – Iterate over direct children.
• css – Interface to CSS selector API.
• descendants – Iterate over all children breadth‑first.
• isSelfClosing – Is this an empty‑element (self‑closing) tag?
• is_empty_element – Same as isSelfClosing.
• self_and_descendants – Iterate over this element and its children breadth‑first.
• strings – Yield all strings (with optional strip and types).

🔧 Data descriptors inherited from bs4.element.Tag

• parserClass
• string – Convenience property to get the single string within this element.

🧩 Data and other attributes inherited from bs4.element.Tag

• DEFAULT_INTERESTING_STRING_TYPES
• EMPTY_ELEMENT_EVENT
• END_ELEMENT_EVENT
• START_ELEMENT_EVENT
• STRING_ELEMENT_EVENT

📜 Methods inherited from bs4.element.PageElement

• append(self, tag) – Appends a PageElement to contents.
• extend(self, tags) – Appends a list of PageElements (or a Tag's contents if a single Tag is given).
• extract(self, _self_index=None) – Destructively removes this element from the tree; returns self.
• fetchNextSiblings = find_next_siblings
• fetchParents = find_parents
• fetchPrevious = find_all_previous
• fetchPreviousSiblings = find_previous_siblings
• findAllNext = find_all_next
• findAllPrevious = find_all_previous
• findNext = find_next
• findNextSibling = find_next_sibling
• findNextSiblings = find_next_siblings
• findParent = find_parent
• findParents = find_parents
• findPrevious = find_previous
• findPreviousSibling = find_previous_sibling
• findPreviousSiblings = find_previous_siblings
• find_all_next(self, name=None, attrs={}, string=None, limit=None, **kwargs) – Find all matching PageElements later in the document.
• find_all_previous(self, name=None, attrs={}, string=None, limit=None, **kwargs) – Find all matching PageElements earlier in the document.
• find_next(self, name=None, attrs={}, string=None, **kwargs) – Find first matching PageElement later in the document.
• find_next_sibling(self, name=None, attrs={}, string=None, **kwargs) – Find closest later sibling matching criteria.
• find_next_siblings(self, name=None, attrs={}, string=None, limit=None, **kwargs) – Find all later siblings matching criteria.
• find_parent(self, name=None, attrs={}, **kwargs) – Find closest parent matching criteria.
• find_parents(self, name=None, attrs={}, limit=None, **kwargs) – Find all parents matching criteria.
• find_previous(self, name=None, attrs={}, string=None, **kwargs) – Find first matching PageElement earlier in the document.
• find_previous_sibling(self, name=None, attrs={}, string=None, **kwargs) – Find closest earlier sibling matching criteria.
• find_previous_siblings(self, name=None, attrs={}, string=None, limit=None, **kwargs) – Find all earlier siblings matching criteria.
• format_string(self, s, formatter) – Format string using given formatter.
• formatter_for_name(self, formatter) – Look up or create a Formatter.
• getText = get_text
• get_text(self, separator='', strip=False, types=<object object at 0x7f375cc44cb0>) – Concatenate all child strings using separator.
• insert(self, position, new_child) – Insert a PageElement at a given position.
• replaceWith = replace_with
• replaceWithChildren = unwrap
• replace_with(self, *args) – Replace this element with one or more PageElements.
• replace_with_children = unwrap
• setup(self, parent=None, previous_element=None, next_element=None, previous_sibling=None, next_sibling=None) – Sets initial relations between elements.
• unwrap(self) – Replace this element with its contents.
• wrap(self, wrap_inside) – Wrap this element inside another one.

🔍 Readonly properties inherited from bs4.element.PageElement

• decomposed – Check whether element has been decomposed.
• next – The PageElement parsed just after this one.
• next_elements – All PageElements parsed after this one.
• next_siblings – All later siblings.
• parents – All parents.
• previous – The PageElement parsed just before this one.
• previous_elements – All PageElements parsed before this one.
• previous_siblings – All earlier siblings.
• stripped_strings – Yield all strings, stripped.
• text – Get all child strings concatenated (with separator, strip, types).

🔧 Data descriptors inherited from bs4.element.PageElement

• __dict__
• __weakref__
• nextSibling
• previousSibling

🧩 Data and other attributes inherited from bs4.element.PageElement

• default = <object object>
• known_xml = None

📊 DATA

• __all__ = ['BeautifulSoup']
• __copyright__ = 'Copyright (c) 2004-2024 Leonard Richardson'
• __license__ = 'MIT'

🔖 VERSION

4.12.3

👤 AUTHOR

Leonard Richardson (leonardr AT segfault.org)

📁 FILE

/home/chedong/.local/lib/python3.10/site-packages/bs4/__init__.py

📖 NAME 🚀 Quick Reference 📝 DESCRIPTION 📦 PACKAGE CONTENTS 🏛️ CLASSES 📊 DATA 🔖 VERSION 👤 AUTHOR 📁 FILE