User Guide

Skrub is a Python library that facilitates machine learning with tabular data (dataframes, such as pandas and polars) using a scikit-learn-compatible API.

Use the sections below to navigate the guide. For a quickstart example, try Getting Started. For runnable code, see the Example gallery. For class and function details, see the API Reference.

• Exploring a Dataframe
  • Exploring dataframes interactively with the TableReport
    • A demo of the TableReport
  • Finding Correlated Columns in a DataFrame
• Wrangling data with good defaults
  • Cleaner: sanitizing a dataframe
    • Converting numeric dtypes to float32 with the Cleaner
  • Transforming a table into an array of numeric features: TableVectorizer
    • Numeric strings and categorical encoding
  • Building robust ML baselines with tabular_pipeline()
  • Transforming selected columns with ApplyToCols
    • Dealing with columns that cannot be handled by a transformer
    • Advanced usage of ApplyToCols
• Column-level feature extraction
  • Encoding string and text columns as numeric features
    • Comparing the categorical encoders included in skrub
  • Handling datetimes: parsing from strings and encoding as numbers
    • Parsing Datetime Strings with ToDatetime
    • Encoding and Feature Engineering with DatetimeEncoder
  • Robust scaling of numeric features using SquashingScaler
  • Advanced columnwise operations
    • The single column transformer
    • Rejection handling with ApplyToCols and core.RejectColumn
• Multi-column operations
  • Removing unneeded columns with DropUninformative and Cleaner
  • Dropping columns with many missing values
  • Applying DropUninformative only to a subset of columns
  • Skrub Selectors, for selecting columns in a dataframe
    • Introduction to selectors
    • Type of selectors
    • Combining selectors
    • Visualizing a selector
    • Using selectors with other skrub transformers
  • Selecting based on dtype or data properties
  • Categories of selectors
    • Selectors based on column data types
    • Selectors based on column content and properties
    • Selectors based on column names
  • filter() and filter_names() to select with user-defined criteria
    • Example of custom criteria in filter(): selecting columns with outliers
  • Select columns with null values
    • Example: Selecting columns by null percentage with has_nulls()
• Complex multi-table pipelines with Data Ops
  • Data Ops basic concepts
    • Basics of DataOps: the DataOps plan, variables, and learners
    • Building a simple DataOps plan
    • Using previews for easier development and debugging
    • DataOps allow direct access to methods of the underlying data
    • Control flow in DataOps: eager and deferred evaluation
    • How do skrub Data Ops differ from the alternatives?
  • Building a complex pipeline with the skrub Data Ops
    • Applying machine-learning estimators
    • Applying different transformers using skrub selectors and DataOps
    • Documenting the DataOps plan with node names and descriptions
    • Evaluating and debugging the DataOps plan with .skb.full_report()
    • Using only a part of a DataOps plan
    • Subsampling data for easier development and debugging
  • Tuning and validating Skrub DataOps plans
    • Tuning and validating skrub DataOps plans
    • Improving the confidence in our score through cross-validation
    • Splitting the data in train and test sets
    • Passing additional arguments to the splitter
    • Using the skrub choose_* functions to tune hyperparameters
    • Feature selection with skrub SelectCols and DropCols
    • Validating hyperparameter search with nested cross-validation
    • Going beyond estimator hyperparameters: nesting choices and choosing pipelines
    • Linking choices depending on other choices
    • Exporting the DataOps plan as a learner and reusing it
    • Tuning DataOps with Optuna
• Configuration and dataset utilities
  • Customizing the global configuration
    • Configuration parameters
  • Deduplicating categorical data with deduplicate()
    • Deduplicating values in a dataframe
  • Working with the example datasets provided by skrub
    • Modifying the download location of skrub datasets
• Joining Dataframes
  • Assembling: joining multiple tables
    • Joining external tables for machine learning
    • Fuzzy joining tables