trafilatura merges unrelated text snippets when extracting content from list/index pages

[cikay]

trafilatura merges unrelated text snippets when extracting content from list/index pages. In the following I extract list of news articles it merge all unrelated articles title and summary. I expect it to return an empty string or None instead.

>>> from trafilatura import fetch_url, extract
>>> url = "https://www.bbc.com/innovation"
>>> downloaded = fetch_url(url)
>>> result = extract(downloaded)
>>> result
"Innovation\nHealth Decoded\nElectrolyte-packed drinks are said to optimise health and hydration. But do we really need them?\n7 August 2025Explore More\nThe health benefits of drinking matcha tea\n24 Jul 2025Health Decoded\nWhy walking backwards can be good for your health\n10 Jul 2025Health Decoded\nThe hidden calories we consume without knowing\n26 Jun 2025Health Decoded\nSimple tips for cutting back on ultra-processed foods\n12 Jun 2025Health Decoded\nAn immersive therapy that can improve your sleep\n29 May 2025Health Decoded\nEight simple steps to help you live longer\n15 May 2025Health Decoded\nHow your 'second brain' changes the way you think\n1 May 2025Health Decoded\nWhy you might be younger than you think you are\n16 Apr 2025Health Decoded\nThe 'healthier' sweetener that could replace sugar\n3 Apr 2025Health Decoded\nA powerful way to improve your fitness\n20 Mar 2025Health Decoded\nWhy drinking coffee can benefit your health\n6 Mar 2025Health Decoded\nHow to change your personality traits with intention\n20 Feb 2025Health Decoded\nHow you can think yourself well\n6 Feb 2025Health Decoded\nWhy being a 'loner' could be good for you\n23 Jan 2025Health Decoded\nAn eating habit that helps you maintain a healthy weight\n9 Jan 2025Health Decoded\nA simple practice that is good for stress\n19 Dec 2024Health Decoded\nHow immersion in nature can make you feel calmer\n5 Dec 2024Health Decoded\nThe benefits of being a 'weekend warrior'\n21 Nov 2024Health Decoded\nWhy friendships are key to living longer\n7 Nov 2024Health Decoded\nHow to reset your brain with your breath\n10 Oct 2024Health Decoded\nThe vegetable that can protect you from cancer\n26 Sep 2024Health Decoded\nA simple and tasty way to reduce wrinkles and keep your skin healthy\n12 Sep 2024Health Decoded\nWhy improving your nutrition will help you think better\n29 Aug 2024Health Decoded\nThe surprising benefits of self-soothing touch\n15 Aug 2024Health Decoded\nHow your gut microbes shape your health\n25 Jul 2024Health Decoded\nIs cold water swimming good for you?\n12 Jul 2024Science & Health\nWhy women 'need less exercise than men'\n28 Jun 2024Science & Health\nHow 'tapping therapy' can help calm your mind\n14 Jun 2024Health Decoded\nCan this mineral help reduce anxiety?\n1 Jun 2024Science & Health"
>>>