Contributing to skrub#

First off, thanks for taking the time to contribute!

Below are some guidelines to help you get started.

Have a question?#

If you have any questions, feel free to reach out:

What to know before you begin#

To understand the purpose and goals behind skrub, please read our vision statement.

If you’re interested in the research behind skrub, we encourage you to explore these papers:

How can I contribute?#

Reporting bugs#

Using the library is the best way to discover bugs and limitations. If you find one, please:

  1. Check if an issue already exists by searching the GitHub issues

    • If open, leave a đź‘Ť on the original message to signal that others are affected.

    • If closed, check for one of the following:

      • A merged pull request may indicate the bug is fixed. Update your skrub version or note if the fix is pending a release.

      • A wontfix label or reasoning may be provided if the issue was closed without a fix.

  2. If the issue does not exist, create a new one.

How to submit a bug report?#

To help us resolve the issue quickly, please include:

  • A clear and descriptive title.

  • A summary of the expected result.

  • Any additional details where the bug might occur or doesn’t occur unexpectedly.

  • A code snippet that reproduces the issue, if applicable.

  • Version information for Python, skrub, and relevant dependencies (e.g., scikit-learn, numpy, pandas).

Suggesting enhancements#

If you have an idea for improving skrub, whether it’s a small fix or a new feature, first:

How to submit an enhancement proposal?#

When proposing an enhancement:

  • Use a clear and descriptive title.

  • Explain the goal of the enhancement.

  • Provide a detailed step-by-step description of the proposed change.

  • Link to any relevant resources that may support the enhancement.

If the enhancement proposal is validated#

Once your enhancement proposal is approved, let the maintainers know the following:

  • If you will write the code and submit a Pull Request (PR): Contributing the feature yourself is the quickest way to see it implemented. We’re here to guide you through the process if needed! To get started, refer to the section Writing your first Pull Request.

  • If you won’t be writing the code: A developer can then take over the implementation. However, please note that we cannot guarantee how long it will take for the feature to be added.

If the enhancement is refused#

Although many ideas are great, not all will align with the objectives of skrub.

If your enhancement is not accepted, consider implementing it as a separate package that builds on top of skrub!

We would love to see your work, and in some cases, we might even feature your package in the official repository.

Writing your first Pull Request#

Preparing the ground#

Before writing any code, ensure you have created an issue discussing the proposed changes with the maintainers. See the relevant sections above on how to do this.

Setting up the environment#

To contribute, you will first have to run through some steps:

  • Set up your environment by forking the repository (Github doc on forking and cloning).

  • Create and activate a new virtual environment:

    • With venv, create the env with python -m venv env_skrub and then activate it with source env_skrub/bin/activate.

    • With conda, create the env with conda new -n env_skrub and activate it with conda activate env_skrub.

    • While at the root of your local copy of skrub and within the new env, install the required development dependencies by running pip install --editable ".[dev, lint, test, doc]".

  • Run pre-commit install to activate some checks that will run every time you do a git commit (mostly, formatting checks).

If you want to make sure that everything runs properly, you can run all the tests with the command pytest -s skrub/tests; note that this may take a long time. Some tests may raise warnings such as:

UserWarning: Only pandas and polars DataFrames are supported, but input is a Numpy array. Please convert Numpy arrays to DataFrames before passing them to skrub transformers. Converting to pandas DataFrame with columns ['0', '1', …].
  warnings.warn(

This is expected, and you may proceed with the next steps without worrying about them. However, no tests should fail at this point: if they do fail, then let us know.

Now that the development environment is ready, you may start working on the new issue by creating a new branch:

git checkout -b my-branch-name-eg-fix-issue-123
# make some changes
git add ./the/file-i-changed
git commit -m "my message"
git push --set-upstream origin my-branch-name-eg-fix-issue-123

At this point, if you visit again the pull requests page github should show a banner asking if you want to open a pull request from your new branch.

Implementation Guidelines#

When contributing, keep these project goals in mind:

  • Pure Python code: Avoid using binary extensions, Cython, or other compiled languages.

  • Production-friendly code:

    • Target the widest possible range of Python versions and dependencies.

    • Minimize the use of external dependencies.

    • Ensure backward compatibility as much as possible.

  • Performance over readability: Optimized code may be less readable, so please include clear and detailed comments. Refer to this best practice guide.

  • Explicit variable/function names: Use descriptive, verbose names for clarity.

  • Document public API components:

    • Document all public functions, methods, variables, and class signatures.

    • The public API refers to all components available for import and use by library users. Anything that doesn’t begin with an underscore is considered part of the public API.

Testing the code#

Tests for files in a given folder should be located in a sub-folder named tests: tests for Skrub objects are located in skrub/tests/, tests for the dataframe API are in skrub/_dataframe/tests/ and so on.

Tests should check all functionalities of the code that you are going to add. If needed, additional tests should be added to verify that other objects behave correctly.

Consider an example: your contribution is for the AmazingTransformer, whose code is in skrub/_amazing_transformer.py. The AmazingTransformer is added as one of the default transformers for TableVectorizer.

As such, you should add a new file testing the functionality of AmazingTransformer in skrub/tests/test_amazing_transformer.py, and update the file skrub/tests/test_table_vectorizer.py so that it takes into account the new transformer.

Additionally, you might have updated the internal dataframe API in skrub/_dataframe/_common.py with a new function, amazing_function. In this case, you should also update skrub/_dataframe/tests/test_common.py to add a test for the amazing_function.

Run each updated test file using pytest ([pytest docs](https://docs.pytest.org/en/stable/)):

pytest -vsl skrub/tests/test_amazing_transformer.py \
skrub/_dataframe/tests/test_common.py \
skrub/_dataframe/tests/test_table_vectorizer.py

The -vsl flag provides more information when running the tests.

It is also possible to run a specific test, or set of tests using the commands pytest the_file.py::the_test, or pytest the_file.py -k 'test_name_pattern'. This is helpful to avoid having to run all the tests.

If you work on Windows, you might have some issues with the working directory if you use pytest, while python -m pytest ... should be more robust.

Once you are satisfied with your changes, you can run all the tests to make sure that your change did not break code elsewhere:

pytest -s skrub/tests

Finally, sync your changes with the remote repository and wait for CI to run.

Checking coverage on the local machine#

Checking coverage is one of the operations that is performed after submitting the code. As this operation may take a long time online, it is possible to check whether the code coverage is high enough on your local machine.

Run your tests with the --cov and --cov-report arguments:

pytest -vsl skrub/tests/test_amazing_transformer.py --cov=skrub --cov-report=html

This will create the folder htmlcov: by opening htmlcov/index.html it is possible to check what lines are covered in each file.

Updating doctests#

If you alter the default behavior of an object, then this might affect the docstrings. Check for possible problems by running

pytest skrub/path/to/file

Formatting and pre-commit checks#

Formatting the code well helps with code development and maintenance, which why is skrub requires that all commits follow a specific set of formatting rules to ensure code quality.

Luckily, these checks are performed automatically by the pre-commit tool (pre-commit docs) before any commit can be pushed. Something worth noting is that if the pre-commit hooks format some files, the commit will be canceled: you will have to stage the changes made by pre-commit and commit again.

Submitting your code#

Once you have pushed your commits to your remote repository, you can submit a PR by clicking the “Compare & pull request” button on GitHub, targeting the skrub repository.

Continuous Integration (CI)#

After creating your PR, CI tools will run proceed to run all the tests on all configurations supported by skrub.

  • Github Actions: Used for testing skrub across various platforms (Linux, macOS, Windows) and dependencies.

  • CircleCI: Builds and verifies the project documentation.

If any of the following markers appears in the commit message, the following actions are taken.

Commit Message Marker

Action Taken by CI

[ci skip]

CI is skipped completely

[skip ci]

CI is skipped completely

[skip github]

CI is skipped completely

[deps nightly]

CI is run with the nightly builds of dependencies

[doc skip]

Docs are not built

[doc quick]

Docs built, but excludes example gallery plots

[doc build]

Docs built including example gallery plots (longer)

Note that by default the documentation is built, but only the examples that are directly modified by the pull request are executed.

CI is testing all possible configurations supported by skrub, so tests may fail with configurations different from what you are developing with. If this is the case, it is possible to run the tests in the environment that is failing by using pixi. For example if the env is ci-py309-min-optional-deps, it is possible to replicate it using the following command:

pixi run -e ci-py309-min-optional-deps  pytest skrub/tests/path/to/test

This command downloads the specific environment on the machine, so you can test it locally and apply fixes, or have a clearer idea of where the code is failing to discuss with the maintainers.

Finally, if the remote repository was changed, you might need to run

pre-commit run --all-files to make sure that the formatting is correct.

Integration#

Community consensus is key in the integration process. Expect a minimum of 1 to 3 reviews depending on the size of the change before we consider merging the PR.

Building the documentation#

Before submitting your pull request, ensure that your modifications haven’t introduced any new Sphinx warnings by building the documentation locally and addressing any issues.

First, make sure you have properly installed the development version of skrub. You can follow the Install > “From source” section, if needed.

Building the documentation requires installing some additional packages:

cd skrub
pip install '.[doc]'

To build the documentation, you need to be in the doc folder:

cd doc

To generate the full documentation, including the example gallery, run the following command:

make html

The documentation will be generated in the _build/html/ directory and are viewable in a web browser, for instance by opening the local _build/html/index.html file.

Running all the examples can take a while, so if you only want to generate specific examples, you can use the following command with a regex pattern:

make html EXAMPLES_PATTERN=your_regex_goes_here make html

This is especially helpful when you’re only modifying or checking a few examples.