SIGNED 2000NIKE AIR JORDAN 1 HIGH BANNED 1985 ROOKIE RETRO SHOES AUTOGRAPH UDA SIGNED IN PRESENCE OF UPPER DECK REPRESENTATIVES Iframe=requests.get(Nike_shoe.find(id='desc_ifr'))Ĭustom_description = soup(iframe.text, 'html5lib') p1xel his answer can be implemented as follows: source=requests.get('') Also I am not familiar with this package, so perhaps I am doing something wrong.īoth answers below Kesely & give correct results. I have also tried to use Requests-HTML package which should have full JavaScript support: from requests_html import HTMLSessionīut unfortunately, I was still not able to retrieve this data. I have tried to parse the source.text as lxml, html.parser, html5lib and xml. When I scan through the Nike_shoe soup, this text is not present.

This description is part of the following HTML structure: This can be found a bit lower on the eBay page. The description part I am trying to filter contains, among other lines, the following excerpt:

The example I am working on is as follows: from bs4 import BeautifulSoup as soup I would prefer to do it with requests and BS4, as the script is almost ready, and scraping through for example Selenium is much slower. This seems to be an event html, which in a browser is loaded automatically, but not with requests and BS4. Unfortunately, the only thing missing is the custom item description. I have collected all aspects such as price, shipping title etc. For every item I want to collect all data including the custom description to build up a database. I am trying to collect data from shoes on eBay.

Ebay item description html templates