Access NYT articles, book reviews, bestseller lists, movie reviews, event listings, and archive content from 1851 onwards. Used in data engineering for building news analytics pipelines, constructing NLP training corpora, and ingesting structured editorial metadata into data warehouses using Python.
Engineers query the NYT Article Search API using `requests` with date ranges and keyword filters, paginating through results and storing article metadata (headline, abstract, section, URL) in Elasticsearch or PostgreSQL for full-text search applications.
NYT article archives are premium training data for news summarization and editorial analysis models. RAG pipelines retrieve relevant articles to give LLMs factual grounding for questions about historical events. The Books API also powers AI-driven reading recommendation systems.
# pip install requests
import requests
resp = requests.get(
"https://api.nytimes.com/svc/search/v2/articlesearch.json",
params={"q": "data engineering", "api-key": "YOUR_API_KEY"}
)
for doc in resp.json()["response"]["docs"][:3]:
print(doc["headline"]["main"])Official dataset source
More datasets used by Python data engineers.
Access news articles, headlines, and metadata from over 80,000 sources worldwide, searchable by keyword, source, language, and date. Used in data engineering for media monitoring pipelines, NLP training data collection, topic trend analysis, and real-time news feed ingestion into data warehouses.
Retrieve Wikipedia article content, summaries, page views, links, categories, and search results programmatically. Commonly used in NLP pipelines for training data collection, knowledge graph construction, entity resolution, and enriching datasets with encyclopedic context using the wikipedia-api Python library.
Access content and metadata from all Wikimedia projects including Wikipedia, Wiktionary, Wikiquote, and Commons. Used in data pipelines for multilingual text corpus construction, knowledge graph enrichment, page view analytics, and building NLP training datasets from structured encyclopaedic content in Python.