feedparser

i am using the feedparser library in python https://github.com/kurtmckee/feedparser/ which basically takes an RSS url and standardizes it to a reasonable extent. But I have noticed that different websites still get parsed slightly differently. For example look at how https://beincrypto.com/feed/ has a long description (containing actual HTML) inside but this website https://www.coindesk.com/arc/outboundfeeds/rss/ completely cuts the description out. I have about 50 such websites and they all have slight variations. So you are saying that in addition to storing parsed data (title, summary, content, author, pubdate, link, guid) that I currently store, I should also add an xml column and store the raw <item></item> from each url till I get a good hang of how each site differs?

Many people have given their "why," but I'll give another "how". castget¹ is a simple tool that does exactly what you want, just add a feed URL and it pulls all the episodes. It has a nice catchup mode so you can add feeds that you've listened to elsewhere without pulling all the old episodes too. hpodder² is another tool I've used in the past, but I can't remember why I switched. Depending on how much you like cobbling together your own solution html-xml-utils³ makes it incredibly easy to script a solution. For example, "curl feed | hxselect -c -s '\n' 'enclosure::attr(url)'" would list just the URLs. feedparser⁴ is a battle tested solution for processing feeds if you'd prefer a proper parsing solution over a hacky shell script(it will correctly handle different formats without any extra work for example). ¹ https://castget.johndal.c