Loading...
Access content and metadata from all Wikimedia projects including Wikipedia, Wiktionary, Wikiquote, and Commons. Used in data pipelines for multilingual text corpus construction, knowledge graph enrichment, page view analytics, and building NLP training datasets from structured encyclopaedic content in Python.
Engineers use the Wikimedia REST API and `mwparserfromhell` to extract clean article text, stripping wiki markup. The PageViews API provides article traffic data, while the MediaWiki Action API supports bulk content extraction for large-scale NLP datasets.
Wikimedia content is foundational for training and grounding AI systems. Wikipedia article text is in virtually every major LLM's training corpus. For RAG, index Wikipedia sections with an embedding model and retrieve them to ground LLM responses on factual questions with cited, verifiable sources.
# pip install requests
import requests
S = requests.Session()
resp = S.get("https://en.wikipedia.org/w/api.php", params={
"action": "query", "prop": "extracts", "exintro": True,
"titles": "Data_engineering", "format": "json"
})
pages = resp.json()["query"]["pages"]
page = next(iter(pages.values()))
print(page["extract"][:500])Official dataset source
More datasets used by Python data engineers.
Retrieve Wikipedia article content, summaries, page views, links, categories, and search results programmatically. Commonly used in NLP pipelines for training data collection, knowledge graph construction, entity resolution, and enriching datasets with encyclopedic context using the wikipedia-api Python library.
Provides structured data about Breaking Bad characters, episodes, quotes, and deaths. A clean, well-documented REST API commonly used to practise JSON ingestion, relational data modelling, and building small ETL pipelines in Python before working with larger production data sources.
A lightweight REST API that returns random facts and trivia about cats. Useful for learning API integration, testing HTTP client libraries in Python, and building practice ETL pipelines before connecting to more complex data sources.