Data Profiler

About Data Profiler

A Python library by Capital One designed to make data analysis, monitoring, and sensitive data detection easy. Data Profiler automatically identifies data types, statistical patterns, and PII across structured and unstructured datasets.

Key Features

• 1Open-source library from Capital One for profiling structured datasets
• 2Detects column data types, statistics, and PII automatically
• 3Supports CSV, JSON, Parquet, and database sources
• 4Generates HTML or JSON reports summarizing dataset characteristics
• 5Labeler component identifies 30+ types of sensitive data fields

How Python Data Engineers Use Data Profiler

Python data engineers use the DataProfiler library to auto-generate statistical profiles and PII detection reports for new datasets before ingesting them into the pipeline. Calling `dp.Data(file_path)` and `dp.Profiler(data).report()` returns column-level statistics and sensitive data labels — informing masking decisions and validation rule design before production pipeline code is written.