User Guide

Skrub is a Python library that facilitates machine learning with tabular data (dataframes, such as pandas and polars) using a scikit-learn-compatible API.

Use the sections below to navigate the guide. For a quickstart example, try Getting Started. For runnable code, see the Example gallery. For class and function details, see the API Reference. For common use cases and how to address them, see the How-to guides.

• Getting Started with skrub
  • Preliminary exploration with the TableReport
  • Sanitizing data with the Cleaner
  • Easily building a strong baseline for tabular machine learning
  • Encoding any data as numerical features
  • Advanced use cases
  • Next steps
• Exploring a Dataframe
  • Exploring dataframes interactively with the TableReport
    • A demo of the TableReport
• Wrangling data with good defaults
  • Cleaner: sanitizing a dataframe
    • Parsing numeric-looking strings with the Cleaner
    • Downcasting float dtypes to float32 with the Cleaner
  • Transforming a table into an array of numeric features: TableVectorizer
    • Numeric strings and categorical encoding
  • Building robust ML baselines with tabular_pipeline()
  • The logic used by the tabular pipeline is quite simple
  • Extending the pipeline with the .steps attribute
  • Using a pipeline as the estimator
  • Transforming selected columns with ApplyToCols
    • Dealing with columns that cannot be handled by a transformer
    • Advanced usage of ApplyToCols
• Column-level feature extraction
  • Encoding string and text columns as numeric features
    • Choosing the right encoder for the job
  • Handling datetimes: parsing from strings and encoding as numbers
    • Parsing Datetime Strings with ToDatetime
    • Encoding and Feature Engineering with DatetimeEncoder
  • Robust scaling of numeric features using SquashingScaler
  • Advanced columnwise operations
    • The single column transformer
    • Rejection handling with ApplyToCols and core.RejectColumn
• Multi-column operations
  • Removing unneeded columns with DropUninformative and Cleaner
  • Dropping columns with many missing values
  • Applying DropUninformative only to a subset of columns
  • Skrub Selectors, for selecting columns in a dataframe
    • Introduction to selectors
    • Type of selectors
    • Combining selectors
    • Visualizing a selector
    • Using selectors with other skrub transformers
  • Selecting based on dtype or data properties
  • Categories of selectors
    • Selectors based on column data types
    • Selectors based on column content and properties
    • Selectors based on column names
  • filter() and filter_names() to select with user-defined criteria
    • Example of custom criteria in filter(): selecting columns with outliers
  • Select columns with null values
    • Example: Selecting columns by null percentage with has_nulls()
• Complex multi-table pipelines with Data Ops
  • Data Ops basic concepts
    • Basics of DataOps: the DataOps plan, variables, and learners
    • Building a simple DataOps plan
    • Tutorial: Using Data Ops to build a machine-learning pipeline
    • Using previews for easier development and debugging
    • DataOps allow direct access to methods of the underlying data
    • Control flow in DataOps: eager and deferred evaluation
    • How do skrub Data Ops differ from the alternatives?
  • Building a complex pipeline with the skrub Data Ops
    • Applying machine-learning estimators
    • Applying different transformers using skrub selectors and DataOps
    • Documenting the DataOps plan with node names and descriptions
    • Evaluating and debugging the DataOps plan with .skb.full_report()
    • Using only a part of a DataOps plan
    • Subsampling data for easier development and debugging
  • Tuning and validating skrub DataOps plans
    • Tuning and validating skrub DataOps plans
    • Improving the confidence in our score through cross-validation
    • Splitting the data in train and test sets
    • Passing additional arguments to the splitter
    • Passing additional arguments to the scorer
    • Using the skrub choose_* functions to tune hyperparameters
    • Feature selection with skrub SelectCols and DropCols
    • Validating hyperparameter search with nested cross-validation
    • Going beyond estimator hyperparameters: nesting choices and choosing pipelines
    • Linking choices depending on other choices
    • Exporting the DataOps plan as a learner and reusing it
    • Tuning DataOps with Optuna
• Joining Dataframes
  • Assembling: joining multiple tables
    • Joining external tables for machine learning
    • Fuzzy joining tables