:orphan:

Examples
===========



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This guide showcases the features of skrub, an open-source package that aims at bridging the gap between tabular data sources and machine-learning models.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_00_getting_started_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_00_getting_started.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Getting Started</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example shows how to transform a rich dataframe with columns of various types into a numerical matrix on which machine-learning algorithms can be applied. We study the case of predicting wages using the employee salaries dataset.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_01_encodings_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_01_encodings.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Encoding: from a dataframe to a numerical matrix for machine learning</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we explore the performance of string and categorical encoders available in skrub.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_02_text_with_string_encoders_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_02_text_with_string_encoders.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Various string encoders: a sentiment analysis example</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we illustrate how to better integrate datetime features in machine learning models with the |DatetimeEncoder|.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_03_datetime_encoder_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_03_datetime_encoder.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Handling datetime features with the DatetimeEncoder</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we show how to combine data from different sources, with a vocabulary not well normalized.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_04_fuzzy_joining_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_04_fuzzy_joining.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Fuzzy joining dirty tables with the Joiner</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Real world datasets often come with misspellings, for instance in manually inputted categorical variables. Such misspelling break data analysis steps that require exact matching, such as a GROUP BY operation.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_05_deduplication_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_05_deduplication.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Deduplicating misspelled categories</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="When the data comprises common entities (cities, companies or famous people), bringing new information assembled from external sources may be the key to improving the analysis.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_06_ken_embeddings_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_06_ken_embeddings.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Wikipedia embeddings to enrich the data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Joining tables may be difficult if one entry on one side does not have an exact match on the other side.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_07_multiple_key_join_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_07_multiple_key_join.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Spatial join for flight data: Joining across multiple columns</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Many problems involve tables whose entities have a one-to-many relationship. To simplify aggregate-then-join operations for machine learning, we can include the |AggJoiner| in our pipeline.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_08_join_aggregation_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_08_join_aggregation.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">AggJoiner on a credit fraud dataset</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="We illustrate the InterpolationJoiner, which is a type of join where values from the second table are inferred with machine-learning, rather than looked up in the table. It is useful when exact matches are not available but we have rows that are close enough to make an educated guess -- in this sense it is a generalization of a fuzzy_join.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_09_interpolation_join_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_09_interpolation_join.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Interpolation join: infer missing rows when joining two tables</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In previous examples, we saw how skrub provides powerful abstractions like TableVectorizer and tabular_pipeline to create pipelines.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_10_apply_on_cols_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_10_apply_on_cols.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Hands-On with Column Selection and Transformers</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The following example illustrates the use of the SquashingScaler, a transformer that can rescale and squash numerical features to a range that works well with neural networks and perhaps also other related models. Its basic idea is to rescale the features based on quantile statistics (to be robust to outliers), and then perform a smooth squashing function to limit the outputs to a pre-defined range. This transform has been found to even work well when applied to one-hot encoded features.">

.. only:: html

  .. image:: /auto_examples/images/thumb/sphx_glr_11_squashing_scaler_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_11_squashing_scaler.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">SquashingScaler: Robust numerical preprocessing for neural networks</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/00_getting_started
   /auto_examples/01_encodings
   /auto_examples/02_text_with_string_encoders
   /auto_examples/03_datetime_encoder
   /auto_examples/04_fuzzy_joining
   /auto_examples/05_deduplication
   /auto_examples/06_ken_embeddings
   /auto_examples/07_multiple_key_join
   /auto_examples/08_join_aggregation
   /auto_examples/09_interpolation_join
   /auto_examples/10_apply_on_cols
   /auto_examples/11_squashing_scaler


Skrub DataOps
=================



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we show how we can use Skrub&#x27;s userguide_data_ops to build a machine learning pipeline that records all the operations involved in pre-processing data and training a model. We will also show how to save the model, load it back, and then use it to make predictions on new, unseen data.">

.. only:: html

  .. image:: /auto_examples/data_ops/images/thumb/sphx_glr_10_data_ops_intro_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_data_ops_10_data_ops_intro.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Introduction to machine-learning pipelines with skrub DataOps</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we show how to build a DataOps plan to handle pre-processing, validation and hyperparameter tuning of a dataset with multiple tables.">

.. only:: html

  .. image:: /auto_examples/data_ops/images/thumb/sphx_glr_11_multiple_tables_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_data_ops_11_multiple_tables.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Multiples tables: building machine learning pipelines with DataOps</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="A machine-learning pipeline typically contains some values or choices which may influence its prediction performance, such as hyperparameters (e.g. the regularization parameter alpha of a RidgeClassifier, the learning_rate of a HistGradientBoostingClassifier), which estimator to use (e.g. RidgeClassifier or HistGradientBoostingClassifier), or which steps to include (e.g. should we join a table to bring additional information or not).">

.. only:: html

  .. image:: /auto_examples/data_ops/images/thumb/sphx_glr_12_choices_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_data_ops_12_choices.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Hyperparameter tuning with DataOps</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we show how to use DataOp.skb.subsample to speed-up interactive construction of a skrub DataOps plan by computing previews on a subsampled version of the original data.">

.. only:: html

  .. image:: /auto_examples/data_ops/images/thumb/sphx_glr_13_subsampling_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_data_ops_13_subsampling.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Subsampling for faster development</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Usecase: developing locally, and avoiding to repeat code in production">

.. only:: html

  .. image:: /auto_examples/data_ops/images/thumb/sphx_glr_14_use_case_thumb.svg
    :alt:

  :ref:`sphx_glr_auto_examples_data_ops_14_use_case.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Usecase: developing locally, and avoiding to repeat code in production</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:
   :includehidden:


   /auto_examples/data_ops/index.rst



.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_