An open-source entry in TopGit's GitHub warehouse: scrapinghub/article-extraction-benchmark, 378 stars, Python. Article extraction benchmark: dataset and evaluation scripts
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
Article extraction benchmark: open-source libraries and commercial services
We evaluate the quality of article body
extraction for commercial services
Zyte Automatic Extraction (ours) <https://www.zyte.com/data-types/news-scraping-api/>,
Diffbot <https://www.diffbot.com/>
and open-source libraries
newspaper4k <https://github.com/AndyTheFactory/newspaper4k>,
readability-lxml <https://github.com/buriy/python-readability>,
dragnet <https://github.com/dragnet-org/dragnet>,
boilerpipe <https://github.com/misja/python-boilerpipe>,
html-text <https://github.com/TeamHG-Memex/html-text>,
trafilatura <https://github.com/adbar/trafilatura>,
go-trafilatura <https://github.com/markusmobius/go-trafilatura>,
go-readability <https://github.com/go-shiori/go-readability>,
readeck/go-readability <https://codeberg.org/readeck/go-readability>,
Readability.js <https://github.com/mozilla/readability>,
Go-DomDistiller <https://github.com/markusmobius/go-domdistiller>.
news-please <https://github.com/fhamborg/news-please>.
Goose3 <https://github.com/goose3/goose3>,
inscriptis <https://github.com/weblyzard/inscriptis>,
html2text <https://github.com/Alir3z4/html2text>,
jusText <https://github.com/miso-belica/jusText>,
BeautifulSoup <https://www.crummy.com/software/BeautifulSoup/bs4/doc/>.
Rust crates:
august <https://crates.io/crates/august>,
boilerpipe <https://crates.io/crates/boilerpipe>,
dom_smoothie <https://crates.io/crates/dom_smoothie>,
fast_html2md <https://crates.io/crates/fast_html2md>,
htmd <https://crates.io/crates/htmd>,
html2md-rs <https://crates.io/crates/html2md-rs>,
html2text <https://crates.io/crates/html2text>,
llm_readability <https://crates.io/crates/llm_readability>,
mdka <https://crates.io/crates/mdka>,
nanohtml2text <https://crates.io/crates/nanohtml2text>,
readability <https://crates.io/crates/readability>,
readable-readability <https://crates.io/crates/readable-readability>,
readabilityrs <https://crates.io/crates/readabilityrs>,
rs_trafilatura <https://github.com/Murrough-Foley/rs-trafilatura>_.
We release evaluation datasets and scripts,
and provide more details in a whitepaper.
Article extraction is a task of extracting certain fields of an article
(e.g. news or blog post), such as headline, article body, publication date,
authors, etc. Article extraction systems must work on any web-site.
Here we evaluate only the article body field, as this is one of the most important fields
and one of the hardest to get right.
Below you can find more details about the packages and result reproduction.
More details
More details are available:
In the whitepaper at https://www.zyte.com/whitepaper-ebook/in-depth-analysis-and-evaluation-on-the-quality-of-article-body-extraction/
In a technical report attached to the v1.0.0 release at
https://github.com/scrapinghub/article-extraction-benchmark/releases/tag/v1.0.0
Installation
Clone this repo, and use Python 3.6+.
Evaluation does not require any dependencies.
Dependencies listed in requirements.txt are only for re-generating
output files for open-source article extraction libraries.
See below for their installation details.
Data
JSON data format: a dictionary which maps item ids to dictionaries,
with the following fields:
articleBody: text of the article
url: page url (optional)
All files should have the same keys.
Ground truth is in ground-truth.json,
predictions from different systems is in output/*.json files.
Prediction files in output/*.json may optionally be wrapped to include the
system/library version used to generate them::
HTML files are in html folder. They were fetched with Splash headless
browser with JS disabled by default. They are gzip-compressed and utf-8 encoded.
Screenshots of all pages are not in the repo, they are available on github
in the "Releases" section: https://github.com/scrapinghub/article-extraction-benchmark/releases
Open-source libraries
In addition to benchmarking AutoExtract and Diffbot services, we also benchmark several
open-source libraries that work directly on HTML files without a need for rendering
or external resources:
inscriptis: https://github.com/weblyzard/inscriptis -
converts HTML to text with a particular emphasis on nested tables
html2text: https://github.com/Alir3z4/html2text -
converts HTML pages to Markup language
jusText: https://github.com/miso-belica/jusText -
Heuristic based boilerplate removal tool
BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/bs4/doc/ -
Python library for pulling data out of HTML and XML files.
Rust crates
- august (MIT): https://crates.io/crates/august
- boilerpipe (MIT): https://crates.io/crates/boilerpipe
- dom_smoothie (MIT): https://crates.io/crates/dom_smoothie
- fast_html2md (MIT): https://crates.io/crates/fast_html2md
- htmd (Apache-2.0): https://crates.io/crates/htmd
- html2md-rs (MIT): https://crates.io/crates/html2md-rs
- html2text (MIT): https://crates.io/crates/html2text
- llm_readability (MIT): https://crates.io/crates/llm_readability
- mdka (Apache-2.0): https://crates.io/crates/mdka
- nanohtml2text (MIT): https://crates.io/crates/nanohtml2text
- readability (MIT): https://crates.io/crates/readability
- readable-readability (MIT): https://crates.io/crates/readable-readability
- readabilityrs (Apache-2.0): https://crates.io/crates/readabilityrs
- rs_trafilatura (MIT OR Apache-2.0): https://github.com/Murrough-Foley/rs-trafilatura
Output from these libraries is already present in the repo in ``output/*.json`` files.
They were generated with ``extractors/run_*.py`` files.
You can re-generate output JSON files with:
python3 -m venv ./venv
source ./venv/bin/activate
make run-all
This will install Python dependencies from ``requirements.txt`` into a
`virtual environment <https://docs.python.org/3/library/venv.html>`_
Evaluation
----------
For evaluation, run::
python3 evaluate.py
We report precision, recall, F1, accuracy and their standard deviation estimated with bootstrap.
Please refer to the technical report for more details.
License
-------
License is MIT.
Does scrapinghub/article-extraction-benchmark have a project website?
No homepage URL was recorded for scrapinghub/article-extraction-benchmark in TopGit's last sync. The README tab above frequently contains screenshots and demo links, or check the repository description on GitHub.
Does scrapinghub/article-extraction-benchmark have any tags?
TopGit's last sync did not record any GitHub topics for scrapinghub/article-extraction-benchmark. GitHub topics appear in the right sidebar of a repository page; that's the authoritative place to check.
How active is development on scrapinghub/article-extraction-benchmark?
The most recent commit recorded on scrapinghub/article-extraction-benchmark was 5 months ago, based on the GitHub push timestamp. The repository has 37 forks — one of the better signals of community interest.
Is scrapinghub/article-extraction-benchmark open source?
Yes — scrapinghub/article-extraction-benchmark ships under the MIT license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/scrapinghub/article-extraction-benchmark.
What license does scrapinghub/article-extraction-benchmark use?
scrapinghub/article-extraction-benchmark is released under the MIT license. Always verify the LICENSE file directly on GitHub for the authoritative terms — license strings can be edited out of sync with a project's actual stance.
Where do I read more about scrapinghub/article-extraction-benchmark?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/scrapinghub/article-extraction-benchmark is the definitive source.
Read full README in the tab above.
Curious whether article-extraction-benchmark is right for you?
Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about article-extraction-benchmark.