conda-forge/staged-recipes

About

This repo is a holding area for recipes destined for a conda-forge feedstock repo.

Keep reading to learn about getting started, preparing your local environment, generating recipes for Python/R packages, linting, and building a package.

To find out more about conda-forge, see conda-smithy.

Getting started

1. Fork this repository.

2. Make a new branch from main for your package's recipe.

3. Make a new folder in recipes for your package, and start a v1 recipe.yaml.

   Recipe format policy: New recipes should use the v1 format. The v0 meta.yaml format is deprecated for new recipes and should only be used in exceptional circumstances where v1 cannot support the recipe. Explain the exception in the pull request.

   For more information:

   • generate a recipe
   • read the v1 example recipe
   • the v0 example recipe is deprecated and retained only for exceptional cases
   • read the FAQ
   • search for examples on GitHub
   • visit our documentation

4. (recommended) Try to build the feedstock locally.

5. Open a pull request, paying attention to the checklist. Building of your package will be tested on Linux, macOS, and Windows.

6. Ask for review or help by @-mentioning the appropriate review teams (or using the bot command) in a pull request comment

7. When your pull request is reviewed and merged:

   • a new "feedstock" repository is created in the GitHub conda-forge organization
     • If this is your first recipe, you will receive an email about steps to accept an invitation to a new GitHub group.
   • a build of your package is triggered
   • the package is uploaded to conda-forge

Feedstock conversion status

PRs are converted to feedstocks in two steps.

1. PRs get put into a merge queue for testing against the latest CI scripts. Your PR may fail in the merge queue in which case it needs to be remerged. Be sure to update the PR with the latest commits from main on the upstream repo before requesting the PR be merged again.

2. Recipes merged into the main branch are turned into feedstocks via the create feedstocks CI job on the conda-forge/admin-requests repo.

   Failures with the above job are often caused by API rate limits from the various services used by conda-forge. This can result in empty feedstock repositories and will resolve itself automatically. If the issue persists, support can be found on Zulip.

Local debugging

While all of the above steps eventually need to work in CI for a recipe to be merged, building locally is a good way to learn more about conda-forge, and make better use of donated, community resources.

Building with build-locally.py

The script build-locally.py will guide you through the local debugging process. This script will then launch the platform-specific scripts, and the resulting artifacts will be available under build_artifacts in the repository directory.

On Linux, everything runs in a Docker container. The staged-recipes directory is mounted as a volume.

On macOS and Windows, some environment variables control where files are kept:

• MINIFORGE_HOME: Where the build tools will be installed. Defaults to ~/Miniforge3.
• CONDA_BLD_PATH: Where the build artifacts will be kept. Defaults to ~/Miniforge3/conda-bld on macOS and C:\bld on Windows.

build-locally.py can be run with any recent Python, or via a pixi task.

Building with pixi

$CONDA_EXE

If you have never used conda-forge before, you may need a conda-compatible package manager, such as conda, mamba, or micromamba. $CONDA_EXE, the "well-known" environment variable is used for "a conda package manager in an activated POSIX shell session," is used throughout this document.

On Windows, this environment variable would be %CONDA_EXE%.

If a compatible $CONDA_EXE is not found, the build-locally.py script may download micromamba: as a single file static binary it isn't installed, per se, and can be used to create other environments.

miniforge

For a more traditional installation, downloading, installing, and activating a miniforge installer provides conda and mamba, with conda-forge as the default source of packages. If found, build-locally.py will use this instead of downloading micromamba.

pixi

pixi is a workspace-based environment and task runner optimized for conda packaging. Several of the local workflows and their dependencies described below are captured in pixi.toml. Install pixi via $CONDA_EXE:

$CONDA_EXE install -c conda-forge pixi

... or one of the documented approaches.

See the available tasks with pixi task list, or get started building with pixi.

Generating recipes with grayskull

grayskull can generate recipes from Python packages on PyPI or R packages on CRAN. The user should review the recipe generated, especially the license and dependencies. Always pass --use-v1-format when invoking grayskull for a new recipe, as shown below.

Linting recipes with conda-smithy

The conda-smithy package provides helpful linters that can save CI resources by catching known issues up-front.

FAQ

1. How do I start editing the recipe?

Look at one of these examples in this repository and modify it as necessary.

Follow the order of the sections in the example recipe. If you make a copy of example recipe, please remove the example's explainer comments from your recipe. Add your own comments to the recipe and build scripts to explain unusual build behavior or recipe options.

If there are details you are not sure about please open a pull request. The conda-forge team will be better able to answer questions about a failing build.

2. How do I populate the hash field?

If your package is on PyPI, you can get the sha256 hash from your package's page on PyPI; look for the SHA256 link next to the download link for your package.

You can also generate a hash from the command line on Linux (and Mac if you install the necessary tools below). If you go this route, the sha256 hash is preferable to the md5 hash.

To generate the md5 hash: md5 your_sdist.tar.gz

To generate the sha256 hash: openssl sha256 your_sdist.tar.gz

You may need the openssl package, available on conda-forge: conda install openssl -c conda-forge

3. How do I exclude a platform?

Use the skip key in the build section along with a selector:

• v1 recipe.yaml

  build:
    skip: win
  

  a full description of selectors is in the rattler-build docs

• Deprecated v0 meta.yaml (only for exceptional cases)

  build:
    skip: true # [win]
  

  A full description of selectors is in the conda docs.

If the package can otherwise be noarch you can also skip it by using virtual packages.

Note: As the package will always be built on linux, it needs to be at least available on there.

4. What does the build: 0 entry mean?

The build number is used when the source code for the package has not changed but you need to make a new build. For example, if one of the dependencies of the package was not properly specified the first time you build a package, then when you fix the dependency and rebuild the package you should increase the build number.

When the package version changes you should reset the build number to 0.

5. Do I have to import all of my unit tests into the recipe's test field?

No, you do not. The main purpose of the test section is to test whether this conda package was built and installed correctly (not whether the upstream package contains bugs).

6. Do all of my package's dependencies have to be in conda(-forge) already?

Short answer: yes. Long answer: In principle, as long as your dependencies are in at least one of your user's conda channels they will be able to install your package. In practice, that is difficult to manage, and we strive to get all dependencies built in conda-forge.

7. When or why do I need to use {{ PYTHON }} -m pip install . -vv?

This should be the default install line for most Python packages. This is preferable to python setup.py because it handles metadata in a conda-friendlier way.

8. Do I need bld.bat and/or build.sh?

In many cases, no. Python packages almost never need it. If the build can be done with one line you can put it in the script line of the build section.

If you would like help with .sh best practices, see more information about linting with conda-smithy and shellcheck.

9. What does being a conda-forge feedstock maintainer entail?

The maintainers "job" is to:

• keep the feedstock updated by merging maintenance PRs from conda-forge's bots;
• keep the package updated by bumping the version whenever there is a new release;
• answer questions about the package on the feedstock issue tracker.

10. Why are there recipes already in the recipes directory? Should I do something about it?

When a PR of recipe(s) is ready to go, it is merged into main. This will trigger a CI build specially designed to convert the recipe(s). However, for any number of reasons the recipe(s) may not be converted right away. In the interim, the recipe(s) will remain in main until they can be converted. There is no action required on the part of recipe contributors to resolve this. Also it should have no impact on any other PRs being proposed. If these recipe(s) pending conversion do cause issues for your submission, please ping conda-forge/core for help.

11. Some checks failed, but it wasn't my recipe! How do I trigger a rebuild?

Sometimes, some of the CI tools' builds fail due to no error within your recipe. If that happens, you can trigger a rebuild by re-creating the last commit and force pushing it to your branch:

# edit your last commit, giving it a new time stamp and hash
# (you can just leave the message as it is)
git commit --amend
# push to github, overwriting your branch
git push -f

If the problem was due to scripts in the staged-recipes repository, you may be asked to "rebase" once these are fixed. To do so, run:

# If you didn't add a remote for conda-forge/staged-recipes yet, also run
# these lines:
# git remote add upstream https://github.com/conda-forge/staged-recipes.git
# git fetch --all
git rebase upstream/main
git push -f

12. My pull request passes all checks, but hasn't received any attention. How do I call attention to my PR? What is the customary amount of time to wait?

Thank you very much for putting in this recipe PR!

This repository is very active, so if you need help with a PR, please let the right people know.

Review teams

There are language-specific teams for reviewing recipes.

Once the PR is ready for review, please mention one of the teams above in a new comment. i.e. @conda-forge/help-some-language, ready for review! Then, a bot will label the PR as 'review-requested'.

Review Bot Command

Due to GitHub limitations, first time contributors to conda-forge are unable to ping conda-forge teams directly, but you can ask a bot to ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team. You can also consider asking on our Zulip chat if your recipe isn't reviewed promptly.

All apologies in advance if your recipe PR does not receive prompt attention. This is a high volume repository and the reviewers are volunteers. Review times vary depending on the number of reviewers on a given language team and may be days or weeks. We are always looking for more staged-recipe reviewers. If you are interested in volunteering, please contact a member of @conda-forge/core. We'd love to have your help!

13. Is there a changelog for this repository?

There's no changelog file, but the following git command gives a good overview of the recent changes in the repository:

$ git log --merges -- ':!recipes'