path: root/doc/RELEASE_WALKTHROUGH.rst.txt

This file contains a walkthrough of the NumPy 1.19.0 release on Linux, modified
for building on azure and uploading to anaconda.org
The commands can be copied into the command line, but be sure to
replace 1.19.0 by the correct version.

This should be read together with the general directions in `releasing`.


Release Preparation
===================

Backport Pull Requests
----------------------

Changes that have been marked for this release must be backported to the
maintenance/1.19.x branch.


Update Release documentation
----------------------------

The file ``doc/changelog/1.19.0-changelog.rst`` should be updated to reflect
the final list of changes and contributors. This text can be generated by::

    $ python tools/changelog.py $GITHUB v1.18.0..maintenance/1.19.x > doc/changelog/1.19.0-changelog.rst

where ``GITHUB`` contains your github access token. This text may also be
appended to ``doc/release/1.19.0-notes.rst`` for patch release, though not for
new releases like ``1.19.0``, as the changelogs for ``*.0`` releases tend to be
excessively long. The ``doc/source/release.rst`` file should also be updated
with a link to the new release notes. These changes should be committed to the
maintenance branch, and later will be forward ported to master. The changelog
should be reviewed for name duplicates or short names and the ``.mailmap`` file
updated if needed.


Finish the Release Note
-----------------------

.. note:

  This has changed now that we use ``towncrier``. See the instructions for
  creating the release note in ``doc/release/upcoming_changes/README.rst``.

Fill out the release note ``doc/release/1.19.0-notes.rst`` calling out
significant changes.


Release  Walkthrough
====================

Note that in the code snippets below, ``upstream`` refers to the root repository on
github and ``origin`` to a fork in your personal account. You may need to make adjustments
if you have not forked the repository but simply cloned it locally. You can
also edit ``.git/config`` and add ``upstream`` if it isn't already present.

Prepare the release commit
--------------------------

Checkout the branch for the release, make sure it is up to date, and clean the
repository::

    $ git checkout maintenance/1.19.x
    $ git pull upstream maintenance/1.19.x
    $ git submodule update
    $ git clean -xdfq

Edit pavement.py and setup.py as detailed in HOWTO_RELEASE::

    $ gvim pavement.py setup.py  # Generally only setup.py needs updating
    $ git commit -a -m"REL: NumPy 1.19.0 release."

Sanity check::

    $ python3 runtests.py -m "full"

Push this release directly onto the end of the maintenance branch. This
requires write permission to the numpy repository::

    $ git push upstream HEAD


Build source releases
---------------------

Paver is used to build the source releases. It will create the ``release`` and
``release/installers`` directories and put the ``*.zip`` and ``*.tar.gz``
source releases in the latter. ::

    $ python3 -m cython --version  # check for correct cython version
    $ paver sdist  # sdist will do a git clean -xdfq, so we omit that


Build wheels
------------

Trigger the wheels build by pointing the numpy-wheels repository at this
commit. This can take up to an hour. The numpy-wheels repository is cloned from
`<https://github.com/MacPython/numpy-wheels>`_. If this is the first release in
a series, start with a pull as the repo may have been accessed and changed by
someone else, then create a new branch for the series. If the branch already
exists skip this::

    $ cd ../numpy-wheels
    $ git co master
    $ git pull upstream master
    $ git branch v1.19.x

Checkout the new branch and edit the ``azure-pipelines.yml`` and
``.travis.yml`` files to make sure they have the correct version, and put in
the commit hash for the ``REL`` commit created above for ``BUILD_COMMIT``. The
``azure/posix.yml`` and ``.travis.yml`` files may also need the Cython versions
updated to keep up with Python releases, but generally just do::

    $ git checkout v1.19.x
    $ gvim azure-pipelines .travis.yml
    $ git commit -a -m"NumPy 1.19.0 release."
    $ git push upstream HEAD

Now wait. If you get nervous at the amount of time taken -- the builds can take
a while -- you can check the build progress by following the links
provided at `<https://github.com/MacPython/numpy-wheels>`_ to check the
build status. Check if all the needed wheels have been built and
uploaded to the staging repository before proceeding.

Note that sometimes builds, like tests, fail for unrelated reasons and you will
need to rerun them. You will need to be logged in under 'numpy' to do this
on azure.

Download wheels
---------------

When the wheels have all been successfully built and staged, download them from the
Anaconda staging directory using the ``tools/download-wheels.py`` script::

    $ cd ../numpy
    $ python3 tools/download-wheels.py 1.19.0


Generate the README files
-------------------------

This needs to be done after all installers are downloaded, but before the pavement
file is updated for continued development::

    $ paver write_release


Tag the release
---------------

Once the wheels have been built and downloaded without errors tag the ``REL`` commit, signing
it with your gpg key::

    $ git tag -s -m"NumPy 1.19.0 release" v1.19.0

You should upload your public gpg key to github, so that the tag will appear
"verified" there.

Check that the files in ``release/installers`` have the correct versions, then
push the tag upstream::

    $ git push upstream v1.19.0

We wait until this point to push the tag because it is public and should not
be changed after it has been pushed.


Reset the maintenance branch into a development state
-----------------------------------------------------

Add another ``REL`` commit to the numpy maintenance branch, which resets the
``ISREALEASED`` flag to ``False`` and increments the version counter::

    $ gvim pavement.py setup.py

Create release notes for next release and edit them to set the version::

    $ cp doc/source/release/template.rst doc/source/release/1.19.1-notes.rst
    $ gvim doc/source/release/1.19.1-notes.rst
    $ git add doc/source/release/1.19.1-notes.rst

Add new release notes to the documentation release list::

    $ gvim doc/source/release.rst

Commit the result::

    $ git commit -a -m"REL: prepare 1.19.x for further development"
    $ git push upstream HEAD


Upload to PyPI
--------------

Upload to PyPI using ``twine``. A recent version of ``twine`` of is needed
after recent PyPI changes, version ``3.1.1`` was used here::

    $ cd ../numpy
    $ twine upload release/installers/*.whl
    $ twine upload release/installers/numpy-1.19.0.zip  # Upload last.

If one of the commands breaks in the middle, you may need to selectively upload
the remaining files because PyPI does not allow the same file to be uploaded
twice. The source file should be uploaded last to avoid synchronization
problems that might occur if pip users access the files while this is in
process. Note that PyPI only allows a single source distribution, here we have
chosen the zip archive.


Upload files to github
----------------------

Go to `<https://github.com/numpy/numpy/releases>`_, there should be a ``v1.19.0
tag``, click on it and hit the edit button for that tag. There are two ways to
add files, using an editable text window and as binary uploads. Cut and paste
the ``release/README.md`` file contents into the text window. You will probably
need to make some edits to get it to look right. Then

- Upload ``release/installers/numpy-1.19.0.tar.gz`` as a binary file.
- Upload ``release/installers/numpy-1.19.0.zip`` as a binary file.
- Upload ``release/README.rst`` as a binary file.
- Upload ``doc/changelog/1.19.0-changelog.rst`` as a binary file.
- Check the pre-release button if this is a pre-releases.
- Hit the ``{Publish,Update} release`` button at the bottom.


Upload documents to numpy.org
-----------------------------

This step is only needed for final releases and can be skipped for
pre-releases. ``make merge-doc`` clones the ``numpy/doc`` repo into
``doc/build/merge`` and updates it with the new documentation::

    $ pushd doc
    $ make dist
    $ make merge-doc
    $ popd

If the release series is a new one, you will need to add a new section to the
``doc/build/merge/index.html`` front page just after the "insert here" comment::

    $ gvim doc/build/merge/index.html +/'insert here'

Otherwise, only the ``zip`` and ``pdf`` links should be updated with the
new tag name::

    $ gvim doc/build/merge/index.html +/'tag v1.19'

You can "test run" the new documentation in a browser to make sure the links
work::

    $ firefox doc/build/merge/index.html

Once everything seems satisfactory, commit and upload the changes::

    $ pushd doc/build/merge
    $ git commit -am"Add documentation for v1.19.0"
    $ git push
    $ popd

Announce the release on scipy.org
---------------------------------

This assumes that you have forked `<https://github.com/scipy/scipy.org>`_::

    $ cd ../scipy.org
    $ git checkout master
    $ git pull upstream master
    $ git checkout -b numpy-1.19.0
    $ gvim www/index.rst # edit the News section
    $ git commit -a
    $ git push origin HEAD

Now go to your fork and make a pull request for the branch.


Announce to mailing lists
-------------------------

The release should be announced on the numpy-discussion, scipy-devel,
scipy-user, and python-announce-list mailing lists. Look at previous
announcements for the basic template. The contributor and PR lists are the same
as generated for the release notes above. If you crosspost, make sure that
python-announce-list is BCC so that replies will not be sent to that list.


Post-Release Tasks
------------------

Checkout master and forward port the documentation changes::

    $ git checkout -b post-1.19.0-release-update
    $ git checkout maintenance/1.19.x doc/source/release/1.19.0-notes.rst
    $ git checkout maintenance/1.19.x doc/changelog/1.19.0-changelog.rst
    $ git checkout maintenance/1.19.x .mailmap  # only if updated for release.
    $ gvim doc/source/release.rst  # Add link to new notes
    $ git add doc/changelog/1.19.0-changelog.rst doc/source/release/1.19.0-notes.rst
    $ git status  # check status before commit
    $ git commit -a -m"REL: Update master after 1.19.0 release."
    $ git push origin HEAD

Go to github and make a PR.