From 0e7b0dfda0188d04e979c4b7a64337d4cf83fa39 Mon Sep 17 00:00:00 2001 From: Mukulika <60316606+Mukulikaa@users.noreply.github.com> Date: Thu, 12 Aug 2021 13:24:32 +0530 Subject: DOC: Reorganized the documentation contribution docs (#19645) * DOC: Reorganized documentation contribution docs 1. Removed Documentation conventions page 2. Merged A guide to NumPy documentation into How to contribute to NumPy documentation 3. Moved Building NumPy reference and docs to Development section 4. Removed the Documentation index stub --- doc/source/_templates/indexcontent.html | 7 +- doc/source/conf.py | 1 + doc/source/dev/howto-docs.rst | 63 ++++++++++++++++ doc/source/dev/howto_build_docs.rst | 126 ++++++++++++++++++++++++++++++++ doc/source/dev/index.rst | 1 + doc/source/doc_conventions.rst | 23 ------ doc/source/docs/howto_build_docs.rst | 126 -------------------------------- doc/source/docs/howto_document.rst | 75 ------------------- doc/source/docs/index.rst | 11 --- doc/source/user/absolute_beginners.rst | 7 +- doc/source/user/index.rst | 2 - 11 files changed, 199 insertions(+), 243 deletions(-) create mode 100644 doc/source/dev/howto_build_docs.rst delete mode 100644 doc/source/doc_conventions.rst delete mode 100644 doc/source/docs/howto_build_docs.rst delete mode 100644 doc/source/docs/howto_document.rst delete mode 100644 doc/source/docs/index.rst (limited to 'doc/source') diff --git a/doc/source/_templates/indexcontent.html b/doc/source/_templates/indexcontent.html index 4cbe7d4d5..450cb1e2e 100644 --- a/doc/source/_templates/indexcontent.html +++ b/doc/source/_templates/indexcontent.html @@ -41,8 +41,8 @@ Contributing to NumPy

Under-the-hood docs
Specialized, in-depth documentation

-

A guide to NumPy documentation
-

Building the NumPy API and reference docs
+

How to contribute to the NumPy documentation
+

Building the NumPy API and reference docs

Benchmarking
benchmarking NumPy

NumPy Enhancement Proposals
@@ -55,7 +55,6 @@

Reporting bugs

Release notes

-

Document conventions

License of NumPy

@@ -70,7 +69,7 @@

The preferred way to update the documentation is by submitting a pull - request on GitHub (see the Documentation index). + request on GitHub (see the Documentation guide). Please help us to further improve the NumPy documentation!

{% endblock %} diff --git a/doc/source/conf.py b/doc/source/conf.py index 587307cf9..41b5cee25 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -300,6 +300,7 @@ intersphinx_mapping = { 'scipy-lecture-notes': ('https://scipy-lectures.org', None), 'pytest': ('https://docs.pytest.org/en/stable', None), 'numpy-tutorials': ('https://numpy.org/numpy-tutorials', None), + 'numpydoc': ('https://numpydoc.readthedocs.io/en/latest', None), } diff --git a/doc/source/dev/howto-docs.rst b/doc/source/dev/howto-docs.rst index 9354357e8..3156d3452 100644 --- a/doc/source/dev/howto-docs.rst +++ b/doc/source/dev/howto-docs.rst @@ -153,6 +153,69 @@ if you write a tutorial on your blog, create a YouTube video, or answer question on Stack Overflow and other sites. +.. _howto-document: + +******************* +Documentation style +******************* + +.. _userdoc_guide: + +User documentation +================== + +- In general, we follow the + `Google developer documentation style guide `_ + for the User Guide. + +- NumPy style governs cases where: + + - Google has no guidance, or + - We prefer not to use the Google style + + Our current rules: + + - We pluralize *index* as *indices* rather than + `indexes `_, + following the precedent of :func:`numpy.indices`. + + - For consistency we also pluralize *matrix* as *matrices*. + +- Grammatical issues inadequately addressed by the NumPy or Google rules are + decided by the section on "Grammar and Usage" in the most recent edition of + the `Chicago Manual of Style + `_. + +- We welcome being + `alerted `_ to cases + we should add to the NumPy style rules. + +.. _docstring_intro: + +Docstrings +========== + +When using `Sphinx `_ in combination with the +NumPy conventions, you should use the ``numpydoc`` extension so that your +docstrings will be handled correctly. For example, Sphinx will extract the +``Parameters`` section from your docstring and convert it into a field +list. Using ``numpydoc`` will also avoid the reStructuredText errors produced +by plain Sphinx when it encounters NumPy docstring conventions like +section headers (e.g. ``-------------``) that sphinx does not expect to +find in docstrings. + +It is available from: + +* `numpydoc on PyPI `_ +* `numpydoc on GitHub `_ + +Note that for documentation within NumPy, it is not necessary to do +``import numpy as np`` at the beginning of an example. + +Please use the ``numpydoc`` :ref:`formatting standard ` as +shown in their :ref:`example `. + + ********************* Documentation reading ********************* diff --git a/doc/source/dev/howto_build_docs.rst b/doc/source/dev/howto_build_docs.rst new file mode 100644 index 000000000..884cf7935 --- /dev/null +++ b/doc/source/dev/howto_build_docs.rst @@ -0,0 +1,126 @@ +.. _howto-build-docs: + +========================================= +Building the NumPy API and reference docs +========================================= + +If you only want to get the documentation, note that pre-built +versions can be found at + + https://numpy.org/doc/ + +in several different formats. + +Development environments +------------------------ + +Before proceeding further it should be noted that the documentation is built with the ``make`` tool, +which is not natively available on Windows. MacOS or Linux users can jump +to :ref:`how-todoc.prerequisites`. It is recommended for Windows users to set up their development +environment on :ref:`Gitpod ` or `Windows Subsystem +for Linux (WSL) `_. WSL is a good option +for a persistent local set-up. + +Gitpod +^^^^^^ +Gitpod is an open-source platform that automatically creates the correct development environment right +in your browser, reducing the need to install local development environments and deal with +incompatible dependencies. + +If you have good internet connectivity and want a temporary set-up, +it is often faster to build with Gitpod. Here are the in-depth instructions for +:ref:`building NumPy with Gitpod `. + + +.. _how-todoc.prerequisites: + +Prerequisites +------------- + +Building the NumPy documentation and API reference requires the following: + +NumPy +^^^^^ + +Since large parts of the main documentation are obtained from NumPy via +``import numpy`` and examining the docstrings, you will need to first +:ref:`build ` and install it so that the correct version is imported. +NumPy has to be re-built and re-installed every time you fetch the latest version of the +repository, before generating the documentation. This ensures that the NumPy version and +the git repository version are in sync. + +Note that you can e.g. install NumPy to a temporary location and set +the PYTHONPATH environment variable appropriately. +Alternatively, if using Python virtual environments (via e.g. ``conda``, +``virtualenv`` or the ``venv`` module), installing NumPy into a +new virtual environment is recommended. + +Dependencies +^^^^^^^^^^^^ + +All of the necessary dependencies for building the NumPy docs can be installed +with:: + + pip install -r doc_requirements.txt + +We currently use Sphinx_ for generating the API and reference +documentation for NumPy. In addition, building the documentation requires +the Sphinx extension `plot_directive`, which is shipped with +:doc:`Matplotlib `. We also use numpydoc_ to render docstrings in +the generated API documentation. :doc:`SciPy ` +is installed since some parts of the documentation require SciPy functions. + +Submodules +^^^^^^^^^^ + +If you obtained NumPy via git, also get the git submodules that contain +additional parts required for building the documentation:: + + git submodule update --init + +.. _Sphinx: http://www.sphinx-doc.org/ +.. _numpydoc: https://numpydoc.readthedocs.io/en/latest/index.html + +Instructions +------------ + +Now you are ready to generate the docs, so write:: + + cd doc + make html + +If all goes well, this will generate a +``build/html`` subdirectory in the ``/doc`` directory, containing the built documentation. If +you get a message about ``installed numpy != current repo git version``, you must +either override the check by setting ``GITVER`` or re-install NumPy. + +If you have built NumPy into a virtual environment and get an error +that says ``numpy not found, cannot build documentation without...``, +you need to override the makefile ``PYTHON`` variable at the command +line, so instead of writing ``make html`` write:: + + make PYTHON=python html + +To build the PDF documentation, do instead:: + + make latex + make -C build/latex all-pdf + +You will need to have LaTeX_ installed for this, inclusive of support for +Greek letters. For example, on Ubuntu xenial ``texlive-lang-greek`` and +``cm-super`` are needed. Also, ``latexmk`` is needed on non-Windows systems. + +Instead of the above, you can also do:: + + make dist + +which will rebuild NumPy, install it to a temporary location, and +build the documentation in all formats. This will most likely again +only work on Unix platforms. + +The documentation for NumPy distributed at https://numpy.org/doc in html and +pdf format is also built with ``make dist``. See `HOWTO RELEASE`_ for details +on how to update https://numpy.org/doc. + +.. _LaTeX: https://www.latex-project.org/ +.. _HOWTO RELEASE: https://github.com/numpy/numpy/blob/main/doc/HOWTO_RELEASE.rst.txt diff --git a/doc/source/dev/index.rst b/doc/source/dev/index.rst index b9347a58c..a8c969267 100644 --- a/doc/source/dev/index.rst +++ b/doc/source/dev/index.rst @@ -257,6 +257,7 @@ The rest of the story Git Basics development_environment development_gitpod + howto_build_docs development_workflow development_advanced_debugging reviewer_guidelines diff --git a/doc/source/doc_conventions.rst b/doc/source/doc_conventions.rst deleted file mode 100644 index e2bc419d1..000000000 --- a/doc/source/doc_conventions.rst +++ /dev/null @@ -1,23 +0,0 @@ -.. _documentation_conventions: - -############################################################################## -Documentation conventions -############################################################################## - -- Names that look like :func:`numpy.array` are links to detailed - documentation. - -- Examples often include the Python prompt ``>>>``. This is not part of the - code and will cause an error if typed or pasted into the Python - shell. It can be safely typed or pasted into the IPython shell; the ``>>>`` - is ignored. - -- Examples often use ``np`` as an alias for ``numpy``; that is, they assume - you've run:: - - >>> import numpy as np - -- If you're a code contributor writing a docstring, see :ref:`docstring_intro`. - -- If you're a writer contributing ordinary (non-docstring) documentation, see - :ref:`userdoc_guide`. diff --git a/doc/source/docs/howto_build_docs.rst b/doc/source/docs/howto_build_docs.rst deleted file mode 100644 index 884cf7935..000000000 --- a/doc/source/docs/howto_build_docs.rst +++ /dev/null @@ -1,126 +0,0 @@ -.. _howto-build-docs: - -========================================= -Building the NumPy API and reference docs -========================================= - -If you only want to get the documentation, note that pre-built -versions can be found at - - https://numpy.org/doc/ - -in several different formats. - -Development environments ------------------------- - -Before proceeding further it should be noted that the documentation is built with the ``make`` tool, -which is not natively available on Windows. MacOS or Linux users can jump -to :ref:`how-todoc.prerequisites`. It is recommended for Windows users to set up their development -environment on :ref:`Gitpod ` or `Windows Subsystem -for Linux (WSL) `_. WSL is a good option -for a persistent local set-up. - -Gitpod -^^^^^^ -Gitpod is an open-source platform that automatically creates the correct development environment right -in your browser, reducing the need to install local development environments and deal with -incompatible dependencies. - -If you have good internet connectivity and want a temporary set-up, -it is often faster to build with Gitpod. Here are the in-depth instructions for -:ref:`building NumPy with Gitpod `. - - -.. _how-todoc.prerequisites: - -Prerequisites -------------- - -Building the NumPy documentation and API reference requires the following: - -NumPy -^^^^^ - -Since large parts of the main documentation are obtained from NumPy via -``import numpy`` and examining the docstrings, you will need to first -:ref:`build ` and install it so that the correct version is imported. -NumPy has to be re-built and re-installed every time you fetch the latest version of the -repository, before generating the documentation. This ensures that the NumPy version and -the git repository version are in sync. - -Note that you can e.g. install NumPy to a temporary location and set -the PYTHONPATH environment variable appropriately. -Alternatively, if using Python virtual environments (via e.g. ``conda``, -``virtualenv`` or the ``venv`` module), installing NumPy into a -new virtual environment is recommended. - -Dependencies -^^^^^^^^^^^^ - -All of the necessary dependencies for building the NumPy docs can be installed -with:: - - pip install -r doc_requirements.txt - -We currently use Sphinx_ for generating the API and reference -documentation for NumPy. In addition, building the documentation requires -the Sphinx extension `plot_directive`, which is shipped with -:doc:`Matplotlib `. We also use numpydoc_ to render docstrings in -the generated API documentation. :doc:`SciPy ` -is installed since some parts of the documentation require SciPy functions. - -Submodules -^^^^^^^^^^ - -If you obtained NumPy via git, also get the git submodules that contain -additional parts required for building the documentation:: - - git submodule update --init - -.. _Sphinx: http://www.sphinx-doc.org/ -.. _numpydoc: https://numpydoc.readthedocs.io/en/latest/index.html - -Instructions ------------- - -Now you are ready to generate the docs, so write:: - - cd doc - make html - -If all goes well, this will generate a -``build/html`` subdirectory in the ``/doc`` directory, containing the built documentation. If -you get a message about ``installed numpy != current repo git version``, you must -either override the check by setting ``GITVER`` or re-install NumPy. - -If you have built NumPy into a virtual environment and get an error -that says ``numpy not found, cannot build documentation without...``, -you need to override the makefile ``PYTHON`` variable at the command -line, so instead of writing ``make html`` write:: - - make PYTHON=python html - -To build the PDF documentation, do instead:: - - make latex - make -C build/latex all-pdf - -You will need to have LaTeX_ installed for this, inclusive of support for -Greek letters. For example, on Ubuntu xenial ``texlive-lang-greek`` and -``cm-super`` are needed. Also, ``latexmk`` is needed on non-Windows systems. - -Instead of the above, you can also do:: - - make dist - -which will rebuild NumPy, install it to a temporary location, and -build the documentation in all formats. This will most likely again -only work on Unix platforms. - -The documentation for NumPy distributed at https://numpy.org/doc in html and -pdf format is also built with ``make dist``. See `HOWTO RELEASE`_ for details -on how to update https://numpy.org/doc. - -.. _LaTeX: https://www.latex-project.org/ -.. _HOWTO RELEASE: https://github.com/numpy/numpy/blob/main/doc/HOWTO_RELEASE.rst.txt diff --git a/doc/source/docs/howto_document.rst b/doc/source/docs/howto_document.rst deleted file mode 100644 index ff726c67c..000000000 --- a/doc/source/docs/howto_document.rst +++ /dev/null @@ -1,75 +0,0 @@ -.. _howto-document: - - -A Guide to NumPy Documentation -============================== - -.. _userdoc_guide: - -User documentation -****************** -- In general, we follow the - `Google developer documentation style guide `_. - -- NumPy style governs cases where: - - - Google has no guidance, or - - We prefer not to use the Google style - - Our current rules: - - - We pluralize *index* as *indices* rather than - `indexes `_, - following the precedent of :func:`numpy.indices`. - - - For consistency we also pluralize *matrix* as *matrices*. - -- Grammatical issues inadequately addressed by the NumPy or Google rules are - decided by the section on "Grammar and Usage" in the most recent edition of - the `Chicago Manual of Style - `_. - -- We welcome being - `alerted `_ to cases - we should add to the NumPy style rules. - - - -.. _docstring_intro: - -Docstrings -********** - -When using `Sphinx `__ in combination with the -numpy conventions, you should use the ``numpydoc`` extension so that your -docstrings will be handled correctly. For example, Sphinx will extract the -``Parameters`` section from your docstring and convert it into a field -list. Using ``numpydoc`` will also avoid the reStructuredText errors produced -by plain Sphinx when it encounters numpy docstring conventions like -section headers (e.g. ``-------------``) that sphinx does not expect to -find in docstrings. - -Some features described in this document require a recent version of -``numpydoc``. For example, the **Yields** section was added in -``numpydoc`` 0.6. - -It is available from: - -* `numpydoc on PyPI `_ -* `numpydoc on GitHub `_ - -Note that for documentation within numpy, it is not necessary to do -``import numpy as np`` at the beginning of an example. However, some -sub-modules, such as ``fft``, are not imported by default, and you have to -include them explicitly:: - - import numpy.fft - -after which you may use it:: - - np.fft.fft2(...) - -Please use the numpydoc `formatting standard`_ as shown in their example_ - -.. _`formatting standard`: https://numpydoc.readthedocs.io/en/latest/format.html -.. _example: https://numpydoc.readthedocs.io/en/latest/example.html diff --git a/doc/source/docs/index.rst b/doc/source/docs/index.rst deleted file mode 100644 index 7d8b1bcb4..000000000 --- a/doc/source/docs/index.rst +++ /dev/null @@ -1,11 +0,0 @@ -.. _documentation: - -NumPy's Documentation -===================== - -.. toctree:: - :maxdepth: 2 - - howto_document - howto_build_docs - diff --git a/doc/source/user/absolute_beginners.rst b/doc/source/user/absolute_beginners.rst index 499e9ec5c..bb570f622 100644 --- a/doc/source/user/absolute_beginners.rst +++ b/doc/source/user/absolute_beginners.rst @@ -83,9 +83,12 @@ If you aren't familiar with this style, it's very easy to understand. If you see ``>>>``, you're looking at **input**, or the code that you would enter. Everything that doesn't have ``>>>`` in front of it is **output**, or the results of running your code. This is the style -you see when you run ``python`` on the command line, but if you're using IPython, you might see a different style. +you see when you run ``python`` on the command line, but if you're using +IPython, you might see a different style. Note that it is not part of the +code and will cause an error if typed or pasted into the Python +shell. It can be safely typed or pasted into the IPython shell; the ``>>>`` +is ignored. -For more information, see :ref:`documentation_conventions`. What’s the difference between a Python list and a NumPy array? -------------------------------------------------------------- diff --git a/doc/source/user/index.rst b/doc/source/user/index.rst index b3ecc3a95..e5c51351e 100644 --- a/doc/source/user/index.rst +++ b/doc/source/user/index.rst @@ -38,8 +38,6 @@ details are found in :ref:`reference`. ../f2py/index ../glossary ../dev/underthehood - ../docs/index ../bugs ../release - ../doc_conventions ../license -- cgit v1.2.1 From 4a1c0bf0745b8a6c7c920d579bd0b2e5eab12f8a Mon Sep 17 00:00:00 2001 From: Mukulika <60316606+Mukulikaa@users.noreply.github.com> Date: Thu, 12 Aug 2021 17:49:19 +0530 Subject: DOC: Created fundamentals doc for explanations in `ufunc` reference doc (#19516) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Created fundamentals doc for explanations in ufunc reference doc * Move explanatory material on ufuncs and broadcasting from the reference guide to the ufunc fundamentals and/or broadcasting document in the user guide * Added examples for ufunc method arguments * Reorganized broadcasting duplicate content from ufuncs doc Co-authored-by: Melissa Weber Mendonça --- doc/source/reference/ufuncs.rst | 567 ++++++++------------------------ doc/source/user/basics.broadcasting.rst | 28 ++ doc/source/user/basics.rst | 1 + doc/source/user/basics.ufuncs.rst | 313 ++++++++++++++++++ 4 files changed, 471 insertions(+), 438 deletions(-) create mode 100644 doc/source/user/basics.ufuncs.rst (limited to 'doc/source') diff --git a/doc/source/reference/ufuncs.rst b/doc/source/reference/ufuncs.rst index 3eae4e159..b832dad04 100644 --- a/doc/source/reference/ufuncs.rst +++ b/doc/source/reference/ufuncs.rst @@ -8,292 +8,16 @@ Universal functions (:class:`ufunc`) ************************************ -.. note: XXX: section might need to be made more reference-guideish... - -.. index: ufunc, universal function, arithmetic, operation +.. seealso:: :ref:`ufuncs-basics` A universal function (or :term:`ufunc` for short) is a function that -operates on :class:`ndarrays ` in an element-by-element fashion, +operates on :class:`ndarrays ` in an element-by-element fashion, supporting :ref:`array broadcasting `, :ref:`type casting `, and several other standard features. That -is, a ufunc is a ":term:`vectorized `" wrapper for a function that -takes a fixed number of specific inputs and produces a fixed number of -specific outputs. - -In NumPy, universal functions are instances of the -:class:`numpy.ufunc` class. Many of the built-in functions are -implemented in compiled C code. The basic ufuncs operate on scalars, but -there is also a generalized kind for which the basic elements are sub-arrays -(vectors, matrices, etc.), and broadcasting is done over other dimensions. -One can also produce custom :class:`ufunc` instances using the -:func:`frompyfunc` factory function. - - -.. _ufuncs.broadcasting: - -Broadcasting -============ - -.. index:: broadcasting - -Each universal function takes array inputs and produces array outputs -by performing the core function element-wise on the inputs (where an -element is generally a scalar, but can be a vector or higher-order -sub-array for generalized ufuncs). Standard -broadcasting rules are applied so that inputs not sharing exactly the -same shapes can still be usefully operated on. Broadcasting can be -understood by four rules: - -1. All input arrays with :attr:`ndim ` smaller than the - input array of largest :attr:`ndim `, have 1's - prepended to their shapes. - -2. The size in each dimension of the output shape is the maximum of all - the input sizes in that dimension. - -3. An input can be used in the calculation if its size in a particular - dimension either matches the output size in that dimension, or has - value exactly 1. - -4. If an input has a dimension size of 1 in its shape, the first data - entry in that dimension will be used for all calculations along - that dimension. In other words, the stepping machinery of the - :term:`ufunc` will simply not step along that dimension (the - :ref:`stride ` will be 0 for that dimension). - -Broadcasting is used throughout NumPy to decide how to handle -disparately shaped arrays; for example, all arithmetic operations (``+``, -``-``, ``*``, ...) between :class:`ndarrays ` broadcast the -arrays before operation. - -.. _arrays.broadcasting.broadcastable: - -.. index:: broadcastable - -A set of arrays is called "broadcastable" to the same shape if -the above rules produce a valid result, *i.e.*, one of the following -is true: - -1. The arrays all have exactly the same shape. - -2. The arrays all have the same number of dimensions and the length of - each dimensions is either a common length or 1. - -3. The arrays that have too few dimensions can have their shapes prepended - with a dimension of length 1 to satisfy property 2. - -.. admonition:: Example - - If ``a.shape`` is (5,1), ``b.shape`` is (1,6), ``c.shape`` is (6,) - and ``d.shape`` is () so that *d* is a scalar, then *a*, *b*, *c*, - and *d* are all broadcastable to dimension (5,6); and - - - *a* acts like a (5,6) array where ``a[:,0]`` is broadcast to the other - columns, - - - *b* acts like a (5,6) array where ``b[0,:]`` is broadcast - to the other rows, - - - *c* acts like a (1,6) array and therefore like a (5,6) array - where ``c[:]`` is broadcast to every row, and finally, - - - *d* acts like a (5,6) array where the single value is repeated. - - -.. _ufuncs-output-type: - -Output type determination -========================= - -The output of the ufunc (and its methods) is not necessarily an -:class:`ndarray`, if all input arguments are not :class:`ndarrays `. -Indeed, if any input defines an :obj:`~class.__array_ufunc__` method, -control will be passed completely to that function, i.e., the ufunc is -:ref:`overridden `. - -If none of the inputs overrides the ufunc, then -all output arrays will be passed to the :obj:`~class.__array_prepare__` and -:obj:`~class.__array_wrap__` methods of the input (besides -:class:`ndarrays `, and scalars) that defines it **and** has -the highest :obj:`~class.__array_priority__` of any other input to the -universal function. The default :obj:`~class.__array_priority__` of the -ndarray is 0.0, and the default :obj:`~class.__array_priority__` of a subtype -is 0.0. Matrices have :obj:`~class.__array_priority__` equal to 10.0. - -All ufuncs can also take output arguments. If necessary, output will -be cast to the data-type(s) of the provided output array(s). If a class -with an :obj:`~class.__array__` method is used for the output, results will be -written to the object returned by :obj:`~class.__array__`. Then, if the class -also has an :obj:`~class.__array_prepare__` method, it is called so metadata -may be determined based on the context of the ufunc (the context -consisting of the ufunc itself, the arguments passed to the ufunc, and -the ufunc domain.) The array object returned by -:obj:`~class.__array_prepare__` is passed to the ufunc for computation. -Finally, if the class also has an :obj:`~class.__array_wrap__` method, the returned -:class:`ndarray` result will be passed to that method just before -passing control back to the caller. - -Use of internal buffers -======================= - -.. index:: buffers - -Internally, buffers are used for misaligned data, swapped data, and -data that has to be converted from one data type to another. The size -of internal buffers is settable on a per-thread basis. There can -be up to :math:`2 (n_{\mathrm{inputs}} + n_{\mathrm{outputs}})` -buffers of the specified size created to handle the data from all the -inputs and outputs of a ufunc. The default size of a buffer is -10,000 elements. Whenever buffer-based calculation would be needed, -but all input arrays are smaller than the buffer size, those -misbehaved or incorrectly-typed arrays will be copied before the -calculation proceeds. Adjusting the size of the buffer may therefore -alter the speed at which ufunc calculations of various sorts are -completed. A simple interface for setting this variable is accessible -using the function - -.. autosummary:: - :toctree: generated/ - - setbufsize - - -Error handling -============== - -.. index:: error handling - -Universal functions can trip special floating-point status registers -in your hardware (such as divide-by-zero). If available on your -platform, these registers will be regularly checked during -calculation. Error handling is controlled on a per-thread basis, -and can be configured using the functions - -.. autosummary:: - :toctree: generated/ - - seterr - seterrcall - -.. _ufuncs.casting: - -Casting Rules -============= - -.. index:: - pair: ufunc; casting rules - -.. note:: - - In NumPy 1.6.0, a type promotion API was created to encapsulate the - mechanism for determining output types. See the functions - :func:`result_type`, :func:`promote_types`, and - :func:`min_scalar_type` for more details. - -At the core of every ufunc is a one-dimensional strided loop that -implements the actual function for a specific type combination. When a -ufunc is created, it is given a static list of inner loops and a -corresponding list of type signatures over which the ufunc operates. -The ufunc machinery uses this list to determine which inner loop to -use for a particular case. You can inspect the :attr:`.types -` attribute for a particular ufunc to see which type -combinations have a defined inner loop and which output type they -produce (:ref:`character codes ` are used -in said output for brevity). - -Casting must be done on one or more of the inputs whenever the ufunc -does not have a core loop implementation for the input types provided. -If an implementation for the input types cannot be found, then the -algorithm searches for an implementation with a type signature to -which all of the inputs can be cast "safely." The first one it finds -in its internal list of loops is selected and performed, after all -necessary type casting. Recall that internal copies during ufuncs (even -for casting) are limited to the size of an internal buffer (which is user -settable). - -.. note:: - - Universal functions in NumPy are flexible enough to have mixed type - signatures. Thus, for example, a universal function could be defined - that works with floating-point and integer values. See :func:`ldexp` - for an example. - -By the above description, the casting rules are essentially -implemented by the question of when a data type can be cast "safely" -to another data type. The answer to this question can be determined in -Python with a function call: :func:`can_cast(fromtype, totype) -`. The Figure below shows the results of this call for -the 24 internally supported types on the author's 64-bit system. You -can generate this table for your system with the code given in the Figure. - -.. admonition:: Figure - - Code segment showing the "can cast safely" table for a 64-bit system. - Generally the output depends on the system; your system might result in - a different table. - - >>> mark = {False: ' -', True: ' Y'} - >>> def print_table(ntypes): - ... print('X ' + ' '.join(ntypes)) - ... for row in ntypes: - ... print(row, end='') - ... for col in ntypes: - ... print(mark[np.can_cast(row, col)], end='') - ... print() - ... - >>> print_table(np.typecodes['All']) - X ? b h i l q p B H I L Q P e f d g F D G S U V O M m - ? Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y - Y - b - Y Y Y Y Y Y - - - - - - Y Y Y Y Y Y Y Y Y Y Y - Y - h - - Y Y Y Y Y - - - - - - - Y Y Y Y Y Y Y Y Y Y - Y - i - - - Y Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y - l - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y - q - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y - p - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y - B - - Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y - Y - H - - - Y Y Y Y - Y Y Y Y Y - Y Y Y Y Y Y Y Y Y Y - Y - I - - - - Y Y Y - - Y Y Y Y - - Y Y - Y Y Y Y Y Y - Y - L - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - - Q - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - - P - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - - e - - - - - - - - - - - - - Y Y Y Y Y Y Y Y Y Y Y - - - f - - - - - - - - - - - - - - Y Y Y Y Y Y Y Y Y Y - - - d - - - - - - - - - - - - - - - Y Y - Y Y Y Y Y Y - - - g - - - - - - - - - - - - - - - - Y - - Y Y Y Y Y - - - F - - - - - - - - - - - - - - - - - Y Y Y Y Y Y Y - - - D - - - - - - - - - - - - - - - - - - Y Y Y Y Y Y - - - G - - - - - - - - - - - - - - - - - - - Y Y Y Y Y - - - S - - - - - - - - - - - - - - - - - - - - Y Y Y Y - - - U - - - - - - - - - - - - - - - - - - - - - Y Y Y - - - V - - - - - - - - - - - - - - - - - - - - - - Y Y - - - O - - - - - - - - - - - - - - - - - - - - - - - Y - - - M - - - - - - - - - - - - - - - - - - - - - - Y Y Y - - m - - - - - - - - - - - - - - - - - - - - - - Y Y - Y - -You should note that, while included in the table for completeness, -the 'S', 'U', and 'V' types cannot be operated on by ufuncs. Also, -note that on a 32-bit system the integer types may have different -sizes, resulting in a slightly altered table. - -Mixed scalar-array operations use a different set of casting rules -that ensure that a scalar cannot "upcast" an array unless the scalar is -of a fundamentally different kind of data (*i.e.*, under a different -hierarchy in the data-type hierarchy) than the array. This rule -enables you to use scalar constants in your code (which, as Python -types, are interpreted accordingly in ufuncs) without worrying about -whether the precision of the scalar constant will cause upcasting on -your large (small precision) array. - - -.. _ufuncs.overrides: - -Overriding Ufunc behavior -========================= - -Classes (including ndarray subclasses) can override how ufuncs act on -them by defining certain special methods. For details, see -:ref:`arrays.classes`. - +is, a ufunc is a ":term:`vectorized `" wrapper for a function +that takes a fixed number of specific inputs and produces a fixed number of +specific outputs. For detailed information on universal functions, see +:ref:`ufuncs-basics`. :class:`ufunc` ============== @@ -314,171 +38,171 @@ advanced usage and will not typically be used. .. index:: pair: ufunc; keyword arguments -*out* +.. rubric:: *out* - .. versionadded:: 1.6 +.. versionadded:: 1.6 - The first output can be provided as either a positional or a keyword - parameter. Keyword 'out' arguments are incompatible with positional - ones. +The first output can be provided as either a positional or a keyword +parameter. Keyword 'out' arguments are incompatible with positional +ones. - .. versionadded:: 1.10 +.. versionadded:: 1.10 - The 'out' keyword argument is expected to be a tuple with one entry per - output (which can be None for arrays to be allocated by the ufunc). - For ufuncs with a single output, passing a single array (instead of a - tuple holding a single array) is also valid. +The 'out' keyword argument is expected to be a tuple with one entry per +output (which can be None for arrays to be allocated by the ufunc). +For ufuncs with a single output, passing a single array (instead of a +tuple holding a single array) is also valid. - Passing a single array in the 'out' keyword argument to a ufunc with - multiple outputs is deprecated, and will raise a warning in numpy 1.10, - and an error in a future release. +Passing a single array in the 'out' keyword argument to a ufunc with +multiple outputs is deprecated, and will raise a warning in numpy 1.10, +and an error in a future release. - If 'out' is None (the default), a uninitialized return array is created. - The output array is then filled with the results of the ufunc in the places - that the broadcast 'where' is True. If 'where' is the scalar True (the - default), then this corresponds to the entire output being filled. - Note that outputs not explicitly filled are left with their - uninitialized values. +If 'out' is None (the default), a uninitialized return array is created. +The output array is then filled with the results of the ufunc in the places +that the broadcast 'where' is True. If 'where' is the scalar True (the +default), then this corresponds to the entire output being filled. +Note that outputs not explicitly filled are left with their +uninitialized values. - .. versionadded:: 1.13 +.. versionadded:: 1.13 - Operations where ufunc input and output operands have memory overlap are - defined to be the same as for equivalent operations where there - is no memory overlap. Operations affected make temporary copies - as needed to eliminate data dependency. As detecting these cases - is computationally expensive, a heuristic is used, which may in rare - cases result in needless temporary copies. For operations where the - data dependency is simple enough for the heuristic to analyze, - temporary copies will not be made even if the arrays overlap, if it - can be deduced copies are not necessary. As an example, - ``np.add(a, b, out=a)`` will not involve copies. +Operations where ufunc input and output operands have memory overlap are +defined to be the same as for equivalent operations where there +is no memory overlap. Operations affected make temporary copies +as needed to eliminate data dependency. As detecting these cases +is computationally expensive, a heuristic is used, which may in rare +cases result in needless temporary copies. For operations where the +data dependency is simple enough for the heuristic to analyze, +temporary copies will not be made even if the arrays overlap, if it +can be deduced copies are not necessary. As an example, +``np.add(a, b, out=a)`` will not involve copies. -*where* +.. rubric:: *where* - .. versionadded:: 1.7 +.. versionadded:: 1.7 - Accepts a boolean array which is broadcast together with the operands. - Values of True indicate to calculate the ufunc at that position, values - of False indicate to leave the value in the output alone. This argument - cannot be used for generalized ufuncs as those take non-scalar input. +Accepts a boolean array which is broadcast together with the operands. +Values of True indicate to calculate the ufunc at that position, values +of False indicate to leave the value in the output alone. This argument +cannot be used for generalized ufuncs as those take non-scalar input. - Note that if an uninitialized return array is created, values of False - will leave those values **uninitialized**. +Note that if an uninitialized return array is created, values of False +will leave those values **uninitialized**. -*axes* +.. rubric:: *axes* - .. versionadded:: 1.15 +.. versionadded:: 1.15 - A list of tuples with indices of axes a generalized ufunc should operate - on. For instance, for a signature of ``(i,j),(j,k)->(i,k)`` appropriate - for matrix multiplication, the base elements are two-dimensional matrices - and these are taken to be stored in the two last axes of each argument. - The corresponding axes keyword would be ``[(-2, -1), (-2, -1), (-2, -1)]``. - For simplicity, for generalized ufuncs that operate on 1-dimensional arrays - (vectors), a single integer is accepted instead of a single-element tuple, - and for generalized ufuncs for which all outputs are scalars, the output - tuples can be omitted. +A list of tuples with indices of axes a generalized ufunc should operate +on. For instance, for a signature of ``(i,j),(j,k)->(i,k)`` appropriate +for matrix multiplication, the base elements are two-dimensional matrices +and these are taken to be stored in the two last axes of each argument. +The corresponding axes keyword would be ``[(-2, -1), (-2, -1), (-2, -1)]``. +For simplicity, for generalized ufuncs that operate on 1-dimensional arrays +(vectors), a single integer is accepted instead of a single-element tuple, +and for generalized ufuncs for which all outputs are scalars, the output +tuples can be omitted. -*axis* +.. rubric:: *axis* - .. versionadded:: 1.15 +.. versionadded:: 1.15 - A single axis over which a generalized ufunc should operate. This is a - short-cut for ufuncs that operate over a single, shared core dimension, - equivalent to passing in ``axes`` with entries of ``(axis,)`` for each - single-core-dimension argument and ``()`` for all others. For instance, - for a signature ``(i),(i)->()``, it is equivalent to passing in - ``axes=[(axis,), (axis,), ()]``. +A single axis over which a generalized ufunc should operate. This is a +short-cut for ufuncs that operate over a single, shared core dimension, +equivalent to passing in ``axes`` with entries of ``(axis,)`` for each +single-core-dimension argument and ``()`` for all others. For instance, +for a signature ``(i),(i)->()``, it is equivalent to passing in +``axes=[(axis,), (axis,), ()]``. -*keepdims* +.. rubric:: *keepdims* - .. versionadded:: 1.15 +.. versionadded:: 1.15 - If this is set to `True`, axes which are reduced over will be left in the - result as a dimension with size one, so that the result will broadcast - correctly against the inputs. This option can only be used for generalized - ufuncs that operate on inputs that all have the same number of core - dimensions and with outputs that have no core dimensions, i.e., with - signatures like ``(i),(i)->()`` or ``(m,m)->()``. If used, the location of - the dimensions in the output can be controlled with ``axes`` and ``axis``. +If this is set to `True`, axes which are reduced over will be left in the +result as a dimension with size one, so that the result will broadcast +correctly against the inputs. This option can only be used for generalized +ufuncs that operate on inputs that all have the same number of core +dimensions and with outputs that have no core dimensions, i.e., with +signatures like ``(i),(i)->()`` or ``(m,m)->()``. If used, the location of +the dimensions in the output can be controlled with ``axes`` and ``axis``. -*casting* +.. rubric:: *casting* - .. versionadded:: 1.6 +.. versionadded:: 1.6 - May be 'no', 'equiv', 'safe', 'same_kind', or 'unsafe'. - See :func:`can_cast` for explanations of the parameter values. +May be 'no', 'equiv', 'safe', 'same_kind', or 'unsafe'. +See :func:`can_cast` for explanations of the parameter values. - Provides a policy for what kind of casting is permitted. For compatibility - with previous versions of NumPy, this defaults to 'unsafe' for numpy < 1.7. - In numpy 1.7 a transition to 'same_kind' was begun where ufuncs produce a - DeprecationWarning for calls which are allowed under the 'unsafe' - rules, but not under the 'same_kind' rules. From numpy 1.10 and - onwards, the default is 'same_kind'. +Provides a policy for what kind of casting is permitted. For compatibility +with previous versions of NumPy, this defaults to 'unsafe' for numpy < 1.7. +In numpy 1.7 a transition to 'same_kind' was begun where ufuncs produce a +DeprecationWarning for calls which are allowed under the 'unsafe' +rules, but not under the 'same_kind' rules. From numpy 1.10 and +onwards, the default is 'same_kind'. -*order* +.. rubric:: *order* - .. versionadded:: 1.6 +.. versionadded:: 1.6 - Specifies the calculation iteration order/memory layout of the output array. - Defaults to 'K'. 'C' means the output should be C-contiguous, 'F' means - F-contiguous, 'A' means F-contiguous if the inputs are F-contiguous and - not also not C-contiguous, C-contiguous otherwise, and 'K' means to match - the element ordering of the inputs as closely as possible. +Specifies the calculation iteration order/memory layout of the output array. +Defaults to 'K'. 'C' means the output should be C-contiguous, 'F' means +F-contiguous, 'A' means F-contiguous if the inputs are F-contiguous and +not also not C-contiguous, C-contiguous otherwise, and 'K' means to match +the element ordering of the inputs as closely as possible. -*dtype* +.. rubric:: *dtype* - .. versionadded:: 1.6 +.. versionadded:: 1.6 - Overrides the DType of the output arrays the same way as the *signature*. - This should ensure a matching precision of the calculation. The exact - calculation DTypes chosen may depend on the ufunc and the inputs may be - cast to this DType to perform the calculation. +Overrides the DType of the output arrays the same way as the *signature*. +This should ensure a matching precision of the calculation. The exact +calculation DTypes chosen may depend on the ufunc and the inputs may be +cast to this DType to perform the calculation. -*subok* +.. rubric:: *subok* - .. versionadded:: 1.6 +.. versionadded:: 1.6 - Defaults to true. If set to false, the output will always be a strict - array, not a subtype. +Defaults to true. If set to false, the output will always be a strict +array, not a subtype. -*signature* +.. rubric:: *signature* - Either a Dtype, a tuple of DTypes, or a special signature string - indicating the input and output types of a ufunc. +Either a Dtype, a tuple of DTypes, or a special signature string +indicating the input and output types of a ufunc. - This argument allows the user to specify exact DTypes to be used for the - calculation. Casting will be used as necessary. The actual DType of the - input arrays is not considered unless ``signature`` is ``None`` for - that array. +This argument allows the user to specify exact DTypes to be used for the +calculation. Casting will be used as necessary. The actual DType of the +input arrays is not considered unless ``signature`` is ``None`` for +that array. - When all DTypes are fixed, a specific loop is chosen or an error raised - if no matching loop exists. - If some DTypes are not specified and left ``None``, the behaviour may - depend on the ufunc. - At this time, a list of available signatures is provided by the **types** - attribute of the ufunc. (This list may be missing DTypes not defined - by NumPy.) +When all DTypes are fixed, a specific loop is chosen or an error raised +if no matching loop exists. +If some DTypes are not specified and left ``None``, the behaviour may +depend on the ufunc. +At this time, a list of available signatures is provided by the **types** +attribute of the ufunc. (This list may be missing DTypes not defined +by NumPy.) - The ``signature`` only specifies the DType class/type. For example, it - can specifiy that the operation should be ``datetime64`` or ``float64`` - operation. It does not specify the ``datetime64`` time-unit or the - ``float64`` byte-order. +The ``signature`` only specifies the DType class/type. For example, it +can specifiy that the operation should be ``datetime64`` or ``float64`` +operation. It does not specify the ``datetime64`` time-unit or the +``float64`` byte-order. - For backwards compatibility this argument can also be provided as *sig*, - although the long form is preferred. Note that this should not be - confused with the generalized ufunc :ref:`signature ` - that is stored in the **signature** attribute of the of the ufunc object. +For backwards compatibility this argument can also be provided as *sig*, +although the long form is preferred. Note that this should not be +confused with the generalized ufunc :ref:`signature ` +that is stored in the **signature** attribute of the of the ufunc object. -*extobj* +.. rubric:: *extobj* - a list of length 3 specifying the ufunc buffer-size, the error - mode integer, and the error call-back function. Normally, these - values are looked up in a thread-specific dictionary. Passing them - here circumvents that look up and uses the low-level specification - provided for the error mode. This may be useful, for example, as - an optimization for calculations requiring many ufunc calls on - small arrays in a loop. +A list of length 3 specifying the ufunc buffer-size, the error +mode integer, and the error call-back function. Normally, these +values are looked up in a thread-specific dictionary. Passing them +here circumvents that look up and uses the low-level specification +provided for the error mode. This may be useful, for example, as +an optimization for calculations requiring many ufunc calls on +small arrays in a loop. @@ -517,39 +241,6 @@ possess. None of the attributes can be set. Methods ------- -All ufuncs have four methods. However, these methods only make sense on scalar -ufuncs that take two input arguments and return one output argument. -Attempting to call these methods on other ufuncs will cause a -:exc:`ValueError`. The reduce-like methods all take an *axis* keyword, a *dtype* -keyword, and an *out* keyword, and the arrays must all have dimension >= 1. -The *axis* keyword specifies the axis of the array over which the reduction -will take place (with negative values counting backwards). Generally, it is an -integer, though for :meth:`ufunc.reduce`, it can also be a tuple of `int` to -reduce over several axes at once, or None, to reduce over all axes. -The *dtype* keyword allows you to manage a very common problem that arises -when naively using :meth:`ufunc.reduce`. Sometimes you may -have an array of a certain data type and wish to add up all of its -elements, but the result does not fit into the data type of the -array. This commonly happens if you have an array of single-byte -integers. The *dtype* keyword allows you to alter the data type over which -the reduction takes place (and therefore the type of the output). Thus, -you can ensure that the output is a data type with precision large enough -to handle your output. The responsibility of altering the reduce type is -mostly up to you. There is one exception: if no *dtype* is given for a -reduction on the "add" or "multiply" operations, then if the input type is -an integer (or Boolean) data-type and smaller than the size of the -:class:`int_` data type, it will be internally upcast to the :class:`int_` -(or :class:`uint`) data-type. Finally, the *out* keyword allows you to provide -an output array (for single-output ufuncs, which are currently the only ones -supported; for future extension, however, a tuple with a single argument -can be passed in). If *out* is given, the *dtype* argument is ignored. - -Ufuncs also have a fifth method that allows in place operations to be -performed using fancy indexing. No buffering is used on the dimensions where -fancy indexing is used, so the fancy index can list an item more than once and -the operation will be performed on the result of the previous operation for -that item. - .. index:: pair: ufunc; methods diff --git a/doc/source/user/basics.broadcasting.rst b/doc/source/user/basics.broadcasting.rst index c31fc98bd..5a252122f 100644 --- a/doc/source/user/basics.broadcasting.rst +++ b/doc/source/user/basics.broadcasting.rst @@ -61,6 +61,8 @@ The code in the second example is more efficient than that in the first because broadcasting moves less memory around during the multiplication (``b`` is a scalar rather than an array). +.. _general-broadcasting-rules: + General Broadcasting Rules ========================== When operating on two arrays, NumPy compares their shapes element-wise. @@ -98,6 +100,32 @@ operation:: B (3d array): 7 x 1 x 5 Result (4d array): 8 x 7 x 6 x 5 + +.. _arrays.broadcasting.broadcastable: + +Broadcastable arrays +==================== + +.. index:: broadcastable + +A set of arrays is called "broadcastable" to the same shape if +the above rules produce a valid result. + +For example, if ``a.shape`` is (5,1), ``b.shape`` is (1,6), ``c.shape`` is (6,) +and ``d.shape`` is () so that *d* is a scalar, then *a*, *b*, *c*, +and *d* are all broadcastable to dimension (5,6); and + +- *a* acts like a (5,6) array where ``a[:,0]`` is broadcast to the other + columns, + +- *b* acts like a (5,6) array where ``b[0,:]`` is broadcast + to the other rows, + +- *c* acts like a (1,6) array and therefore like a (5,6) array + where ``c[:]`` is broadcast to every row, and finally, + +- *d* acts like a (5,6) array where the single value is repeated. + Here are some more examples:: A (2d array): 5 x 4 diff --git a/doc/source/user/basics.rst b/doc/source/user/basics.rst index 66f3f9ee9..bcd51d983 100644 --- a/doc/source/user/basics.rst +++ b/doc/source/user/basics.rst @@ -18,3 +18,4 @@ fundamental NumPy ideas and philosophy. basics.rec basics.dispatch basics.subclassing + basics.ufuncs diff --git a/doc/source/user/basics.ufuncs.rst b/doc/source/user/basics.ufuncs.rst new file mode 100644 index 000000000..083e31f70 --- /dev/null +++ b/doc/source/user/basics.ufuncs.rst @@ -0,0 +1,313 @@ +.. sectionauthor:: adapted from "Guide to NumPy" by Travis E. Oliphant + +.. _ufuncs-basics: + +******************************************** +Universal functions (:class:`.ufunc`) basics +******************************************** + +.. seealso:: :ref:`ufuncs` + +.. index: ufunc, universal function, arithmetic, operation + +A universal function (or :term:`ufunc` for short) is a function that +operates on :class:`ndarrays ` in an element-by-element fashion, +supporting :ref:`array broadcasting `, :ref:`type +casting `, and several other standard features. That +is, a ufunc is a ":term:`vectorized `" wrapper for a function +that takes a fixed number of specific inputs and produces a fixed number of +specific outputs. + +In NumPy, universal functions are instances of the +:class:`numpy.ufunc` class. Many of the built-in functions are +implemented in compiled C code. The basic ufuncs operate on scalars, but +there is also a generalized kind for which the basic elements are sub-arrays +(vectors, matrices, etc.), and broadcasting is done over other dimensions. +The simplest example is the addition operator:: + + >>> np.array([0,2,3,4]) + np.array([1,1,-1,2]) + array([1, 3, 2, 6]) + +One can also produce custom :class:`numpy.ufunc` instances using the +:func:`numpy.frompyfunc` factory function. + + +Ufunc methods +============= + +All ufuncs have four methods. They can be found at +:ref:`ufuncs.methods`. However, these methods only make sense on scalar +ufuncs that take two input arguments and return one output argument. +Attempting to call these methods on other ufuncs will cause a +:exc:`ValueError`. + +The reduce-like methods all take an *axis* keyword, a *dtype* +keyword, and an *out* keyword, and the arrays must all have dimension >= 1. +The *axis* keyword specifies the axis of the array over which the reduction +will take place (with negative values counting backwards). Generally, it is an +integer, though for :meth:`numpy.ufunc.reduce`, it can also be a tuple of +``int`` to reduce over several axes at once, or ``None``, to reduce over all +axes. For example:: + + >>> x = np.arange(9).reshape(3,3) + >>> x + array([[0, 1, 2], + [3, 4, 5], + [6, 7, 8]]) + >>> np.add.reduce(x, 1) + array([ 3, 12, 21]) + >>> np.add.reduce(x, (0, 1)) + 36 + +The *dtype* keyword allows you to manage a very common problem that arises +when naively using :meth:`.ufunc.reduce`. Sometimes you may +have an array of a certain data type and wish to add up all of its +elements, but the result does not fit into the data type of the +array. This commonly happens if you have an array of single-byte +integers. The *dtype* keyword allows you to alter the data type over which +the reduction takes place (and therefore the type of the output). Thus, +you can ensure that the output is a data type with precision large enough +to handle your output. The responsibility of altering the reduce type is +mostly up to you. There is one exception: if no *dtype* is given for a +reduction on the "add" or "multiply" operations, then if the input type is +an integer (or Boolean) data-type and smaller than the size of the +:class:`numpy.int_` data type, it will be internally upcast to the :class:`.int_` +(or :class:`numpy.uint`) data-type. In the previous example:: + + >>> x.dtype + dtype('int64') + >>> np.multiply.reduce(x, dtype=float) + array([ 0., 28., 80.]) + +Finally, the *out* keyword allows you to +provide an output array (for single-output ufuncs, which are currently the only +ones supported; for future extension, however, a tuple with a single argument +can be passed in). If *out* is given, the *dtype* argument is ignored. +Considering ``x`` from the previous example:: + + >>> y = np.zeros(3, dtype=int) + >>> y + array([0, 0, 0]) + >>> np.multiply.reduce(x, dtype=float, out=y) + array([ 0, 28, 80]) # dtype argument is ignored + +Ufuncs also have a fifth method, :func:`numpy.ufunc.at`, that allows in place +operations to be performed using advanced indexing. No +:ref:`buffering ` is used on the dimensions where +advanced indexing is used, so the advanced index can +list an item more than once and the operation will be performed on the result +of the previous operation for that item. + + +.. _ufuncs-output-type: + +Output type determination +========================= + +The output of the ufunc (and its methods) is not necessarily an +:class:`ndarray `, if all input arguments are not +:class:`ndarrays `. Indeed, if any input defines an +:obj:`~.class.__array_ufunc__` method, +control will be passed completely to that function, i.e., the ufunc is +:ref:`overridden `. + +If none of the inputs overrides the ufunc, then +all output arrays will be passed to the +:obj:`~.class.__array_prepare__` and +:obj:`~.class.__array_wrap__` methods of the input (besides +:class:`ndarrays <.ndarray>`, and scalars) that defines it **and** has +the highest :obj:`~.class.__array_priority__` +of any other input to the universal function. The default +:obj:`~.class.__array_priority__` of the +ndarray is 0.0, and the default :obj:`~.class.__array_priority__` of a subtype +is 0.0. Matrices have :obj:`~.class.__array_priority__` equal to 10.0. + +All ufuncs can also take output arguments. If necessary, output will +be cast to the data-type(s) of the provided output array(s). If a class +with an :obj:`~.class.__array__` method is used for the output, +results will be written to the object returned by :obj:`~.class.__array__`. +Then, if the class also has an :obj:`~.class.__array_prepare__` method, it is +called so metadata may be determined based on the context of the ufunc (the +context consisting of the ufunc itself, the arguments passed to the ufunc, and +the ufunc domain.) The array object returned by +:obj:`~.class.__array_prepare__` is passed to the ufunc for computation. +Finally, if the class also has an :obj:`~.class.__array_wrap__` method, the +returned :class:`.ndarray` result will be passed to that method just before +passing control back to the caller. + +.. _ufuncs.broadcasting: + +Broadcasting +============ + +.. seealso:: :doc:`Broadcasting basics ` + +.. index:: broadcasting + +Each universal function takes array inputs and produces array outputs +by performing the core function element-wise on the inputs (where an +element is generally a scalar, but can be a vector or higher-order +sub-array for generalized ufuncs). Standard +:ref:`broadcasting rules ` are applied +so that inputs not sharing exactly the +same shapes can still be usefully operated on. + +By these rules, if an input has a dimension size of 1 in its shape, the +first data entry in that dimension will be used for all calculations along +that dimension. In other words, the stepping machinery of the +:term:`ufunc` will simply not step along that dimension (the +:ref:`stride ` will be 0 for that dimension). + + +.. _ufuncs.casting: + +Type casting rules +================== + +.. index:: + pair: ufunc; casting rules + +.. note:: + + In NumPy 1.6.0, a type promotion API was created to encapsulate the + mechanism for determining output types. See the functions + :func:`numpy.result_type`, :func:`numpy.promote_types`, and + :func:`numpy.min_scalar_type` for more details. + +At the core of every ufunc is a one-dimensional strided loop that +implements the actual function for a specific type combination. When a +ufunc is created, it is given a static list of inner loops and a +corresponding list of type signatures over which the ufunc operates. +The ufunc machinery uses this list to determine which inner loop to +use for a particular case. You can inspect the :attr:`.types +<.ufunc.types>` attribute for a particular ufunc to see which type +combinations have a defined inner loop and which output type they +produce (:ref:`character codes ` are used +in said output for brevity). + +Casting must be done on one or more of the inputs whenever the ufunc +does not have a core loop implementation for the input types provided. +If an implementation for the input types cannot be found, then the +algorithm searches for an implementation with a type signature to +which all of the inputs can be cast "safely." The first one it finds +in its internal list of loops is selected and performed, after all +necessary type casting. Recall that internal copies during ufuncs (even +for casting) are limited to the size of an internal buffer (which is user +settable). + +.. note:: + + Universal functions in NumPy are flexible enough to have mixed type + signatures. Thus, for example, a universal function could be defined + that works with floating-point and integer values. See + :func:`numpy.ldexp` for an example. + +By the above description, the casting rules are essentially +implemented by the question of when a data type can be cast "safely" +to another data type. The answer to this question can be determined in +Python with a function call: :func:`can_cast(fromtype, totype) +`. The example below shows the results of this call for +the 24 internally supported types on the author's 64-bit system. You +can generate this table for your system with the code given in the example. + +.. rubric:: Example + +Code segment showing the "can cast safely" table for a 64-bit system. +Generally the output depends on the system; your system might result in +a different table. + +>>> mark = {False: ' -', True: ' Y'} +>>> def print_table(ntypes): +... print('X ' + ' '.join(ntypes)) +... for row in ntypes: +... print(row, end='') +... for col in ntypes: +... print(mark[np.can_cast(row, col)], end='') +... print() +... +>>> print_table(np.typecodes['All']) +X ? b h i l q p B H I L Q P e f d g F D G S U V O M m +? Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y - Y +b - Y Y Y Y Y Y - - - - - - Y Y Y Y Y Y Y Y Y Y Y - Y +h - - Y Y Y Y Y - - - - - - - Y Y Y Y Y Y Y Y Y Y - Y +i - - - Y Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y +l - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y +q - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y +p - - - - Y Y Y - - - - - - - - Y Y - Y Y Y Y Y Y - Y +B - - Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y - Y +H - - - Y Y Y Y - Y Y Y Y Y - Y Y Y Y Y Y Y Y Y Y - Y +I - - - - Y Y Y - - Y Y Y Y - - Y Y - Y Y Y Y Y Y - Y +L - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - +Q - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - +P - - - - - - - - - - Y Y Y - - Y Y - Y Y Y Y Y Y - - +e - - - - - - - - - - - - - Y Y Y Y Y Y Y Y Y Y Y - - +f - - - - - - - - - - - - - - Y Y Y Y Y Y Y Y Y Y - - +d - - - - - - - - - - - - - - - Y Y - Y Y Y Y Y Y - - +g - - - - - - - - - - - - - - - - Y - - Y Y Y Y Y - - +F - - - - - - - - - - - - - - - - - Y Y Y Y Y Y Y - - +D - - - - - - - - - - - - - - - - - - Y Y Y Y Y Y - - +G - - - - - - - - - - - - - - - - - - - Y Y Y Y Y - - +S - - - - - - - - - - - - - - - - - - - - Y Y Y Y - - +U - - - - - - - - - - - - - - - - - - - - - Y Y Y - - +V - - - - - - - - - - - - - - - - - - - - - - Y Y - - +O - - - - - - - - - - - - - - - - - - - - - - - Y - - +M - - - - - - - - - - - - - - - - - - - - - - Y Y Y - +m - - - - - - - - - - - - - - - - - - - - - - Y Y - Y + +You should note that, while included in the table for completeness, +the 'S', 'U', and 'V' types cannot be operated on by ufuncs. Also, +note that on a 32-bit system the integer types may have different +sizes, resulting in a slightly altered table. + +Mixed scalar-array operations use a different set of casting rules +that ensure that a scalar cannot "upcast" an array unless the scalar is +of a fundamentally different kind of data (i.e., under a different +hierarchy in the data-type hierarchy) than the array. This rule +enables you to use scalar constants in your code (which, as Python +types, are interpreted accordingly in ufuncs) without worrying about +whether the precision of the scalar constant will cause upcasting on +your large (small precision) array. + +.. _use-of-internal-buffers: + +Use of internal buffers +======================= + +.. index:: buffers + +Internally, buffers are used for misaligned data, swapped data, and +data that has to be converted from one data type to another. The size +of internal buffers is settable on a per-thread basis. There can +be up to :math:`2 (n_{\mathrm{inputs}} + n_{\mathrm{outputs}})` +buffers of the specified size created to handle the data from all the +inputs and outputs of a ufunc. The default size of a buffer is +10,000 elements. Whenever buffer-based calculation would be needed, +but all input arrays are smaller than the buffer size, those +misbehaved or incorrectly-typed arrays will be copied before the +calculation proceeds. Adjusting the size of the buffer may therefore +alter the speed at which ufunc calculations of various sorts are +completed. A simple interface for setting this variable is accessible +using the function :func:`numpy.setbufsize`. + + +Error handling +============== + +.. index:: error handling + +Universal functions can trip special floating-point status registers +in your hardware (such as divide-by-zero). If available on your +platform, these registers will be regularly checked during +calculation. Error handling is controlled on a per-thread basis, +and can be configured using the functions :func:`numpy.seterr` and +:func:`numpy.seterrcall`. + + +.. _ufuncs.overrides: + +Overriding ufunc behavior +========================= + +Classes (including ndarray subclasses) can override how ufuncs act on +them by defining certain special methods. For details, see +:ref:`arrays.classes`. -- cgit v1.2.1 From e83c48615fc5b2c8eb647521b4498824d2a8db5b Mon Sep 17 00:00:00 2001 From: Charles Harris Date: Sun, 15 Aug 2021 14:47:03 -0600 Subject: REL: Update main after 1.21.2 release. [skip ci] --- doc/source/release.rst | 1 + doc/source/release/1.21.2-notes.rst | 59 +++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) create mode 100644 doc/source/release/1.21.2-notes.rst (limited to 'doc/source') diff --git a/doc/source/release.rst b/doc/source/release.rst index 484181a4f..e9057a531 100644 --- a/doc/source/release.rst +++ b/doc/source/release.rst @@ -6,6 +6,7 @@ Release Notes :maxdepth: 3 1.22.0 + 1.21.2 1.21.1 1.21.0 1.20.3 diff --git a/doc/source/release/1.21.2-notes.rst b/doc/source/release/1.21.2-notes.rst new file mode 100644 index 000000000..bc17c069e --- /dev/null +++ b/doc/source/release/1.21.2-notes.rst @@ -0,0 +1,59 @@ +.. currentmodule:: numpy + +========================== +NumPy 1.21.2 Release Notes +========================== + +The NumPy 1.21.2 is a maintenance release that fixes bugs discovered after +1.21.1. It also provides 64 bit manylinux Python 3.10.0rc1 wheels for +downstream testing. Note that Python 3.10 is not yet final. It also has +preliminary support for Windows on ARM64, but there is no OpenBLAS for that +platform and no wheels are available. + +The Python versions supported for this release are 3.7-3.9. The 1.21.x series +is compatible with Python 3.10.0rc1 and Python 3.10 will be officially +supported after it is released. The previous problems with gcc-11.1 have been +fixed by gcc-11.2, check your version if you are using gcc-11. + + +Contributors +============ + +A total of 10 people contributed to this release. People with a "+" by their +names contributed a patch for the first time. + +* Bas van Beek +* Carl Johnsen + +* Charles Harris +* Gwyn Ciesla + +* Matthieu Dartiailh +* Matti Picus +* Niyas Sait + +* Ralf Gommers +* Sayed Adel +* Sebastian Berg + + +Pull requests merged +==================== + +A total of 18 pull requests were merged for this release. + +* `#19497 `__: MAINT: set Python version for 1.21.x to ``<3.11`` +* `#19533 `__: BUG: Fix an issue wherein importing ``numpy.typing`` could raise +* `#19646 `__: MAINT: Update Cython version for Python 3.10. +* `#19648 `__: TST: Bump the python 3.10 test version from beta4 to rc1 +* `#19651 `__: TST: avoid distutils.sysconfig in runtests.py +* `#19652 `__: MAINT: add missing dunder method to nditer type hints +* `#19656 `__: BLD, SIMD: Fix testing extra checks when ``-Werror`` isn't applicable... +* `#19657 `__: BUG: Remove logical object ufuncs with bool output +* `#19658 `__: MAINT: Include .coveragerc in source distributions to support... +* `#19659 `__: BUG: Fix bad write in masked iterator output copy paths +* `#19660 `__: ENH: Add support for windows on arm targets +* `#19661 `__: BUG: add base to templated arguments for platlib +* `#19662 `__: BUG,DEP: Non-default UFunc signature/dtype usage should be deprecated +* `#19666 `__: MAINT: Add Python 3.10 to supported versions. +* `#19668 `__: TST,BUG: Sanitize path-separators when running ``runtest.py`` +* `#19671 `__: BLD: load extra flags when checking for libflame +* `#19676 `__: BLD: update circleCI docker image +* `#19677 `__: REL: Prepare for 1.21.2 release. -- cgit v1.2.1 From 9e1c34eda4569a96fba07ed56fb971dedbb438fa Mon Sep 17 00:00:00 2001 From: Antony Lee Date: Tue, 17 Aug 2021 20:30:32 +0200 Subject: DOC: Fix trivial doc typo. (#19692) --- doc/source/reference/c-api/types-and-structures.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/reference/c-api/types-and-structures.rst b/doc/source/reference/c-api/types-and-structures.rst index 54a1e09e1..39a17cc72 100644 --- a/doc/source/reference/c-api/types-and-structures.rst +++ b/doc/source/reference/c-api/types-and-structures.rst @@ -225,7 +225,7 @@ PyArrayDescr_Type and PyArray_Descr compatibility: - Never declare a non-pointer instance of the struct - - Never perform pointer arithmatic + - Never perform pointer arithmetic - Never use ``sizof(PyArray_Descr)`` It has the following structure: -- cgit v1.2.1 From c09a6d010cd55f047a0010374d8e750fecb0483a Mon Sep 17 00:00:00 2001 From: yan Date: Wed, 18 Aug 2021 15:47:32 +0800 Subject: Grammatical errors: B = np.eye((2, 2)) It should be B = np.eye(2, 2) Signed-off-by: yan --- doc/source/user/basics.creation.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.creation.rst b/doc/source/user/basics.creation.rst index ccd6de184..b6f46a783 100644 --- a/doc/source/user/basics.creation.rst +++ b/doc/source/user/basics.creation.rst @@ -296,7 +296,7 @@ There are a number of routines to join existing arrays e.g. :func:`numpy.vstack` arrays into a 4-by-4 array using ``block``:: >>> A = np.ones((2, 2)) - >>> B = np.eye((2, 2)) + >>> B = np.eye(2, 2) >>> C = np.zeros((2, 2)) >>> D = np.diag((-3, -4)) >>> np.block([[A, B], -- cgit v1.2.1 From 31803caed1aec021778d79fcf5fafc516becd3a6 Mon Sep 17 00:00:00 2001 From: Mukulika Date: Mon, 5 Jul 2021 17:54:33 +0530 Subject: DOC: Rearranged parts of the Indexing docs to consolidate content --- doc/source/reference/arrays.indexing.rst | 567 ++++++++++++++++++++----------- doc/source/user/basics.indexing.rst | 383 +++------------------ 2 files changed, 404 insertions(+), 546 deletions(-) (limited to 'doc/source') diff --git a/doc/source/reference/arrays.indexing.rst b/doc/source/reference/arrays.indexing.rst index 9f82875ea..58b87dd7f 100644 --- a/doc/source/reference/arrays.indexing.rst +++ b/doc/source/reference/arrays.indexing.rst @@ -8,7 +8,9 @@ Indexing .. seealso:: - :ref:`Indexing basics ` + :ref:`basics.indexing` + + :ref:`Indexing routines ` .. sectionauthor:: adapted from "Guide to NumPy" by Travis E. Oliphant @@ -18,17 +20,76 @@ Indexing :class:`ndarrays ` can be indexed using the standard Python ``x[obj]`` syntax, where *x* is the array and *obj* the selection. -There are three kinds of indexing available: field access, basic -slicing, advanced indexing. Which one occurs depends on *obj*. +There are four kinds of indexing available depending on *obj*: +single element indexing, basic slicing, advanced indexing and field access. + +Most of the following examples show the use of indexing when +referencing data in an array. The examples work just as well +when assigning to an array. See :ref:`assigning-values-to-indexed-arrays` for +specific examples and explanations on how assignments work. + +Note that in Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to +``x[exp1, exp2, ..., expN]``; the latter is just syntactic sugar +for the former. + +.. _single-element-indexing: + +Single element indexing +----------------------- + +Single element indexing works +exactly like that for other standard Python sequences. It is 0-based, +and accepts negative indices for indexing from the end of the array. :: + + >>> x = np.arange(10) + >>> x[2] + 2 + >>> x[-2] + 8 + +It is not necessary to +separate each dimension's index into its own set of square brackets. :: + + >>> x.shape = (2,5) # now x is 2-dimensional + >>> x[1,3] + 8 + >>> x[1,-1] + 9 + +Note that if one indexes a multidimensional array with fewer indices +than dimensions, one gets a subdimensional array. For example: :: + + >>> x[0] + array([0, 1, 2, 3, 4]) + +That is, each index specified selects the array corresponding to the +rest of the dimensions selected. In the above example, choosing 0 +means that the remaining dimension of length 5 is being left unspecified, +and that what is returned is an array of that dimensionality and size. +It must be noted that the returned array is not a copy of the original, +but points to the same values in memory as does the original array. +In this case, the 1-D array at the first position (0) is returned. +So using a single index on the returned array, results in a single +element being returned. That is: :: + + >>> x[0][2] + 2 + +So note that ``x[0,2] = x[0][2]`` though the second case is more +inefficient as a new temporary array is created after the first index +that is subsequently indexed by 2. .. note:: - In Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to - ``x[exp1, exp2, ..., expN]``; the latter is just syntactic sugar - for the former. + NumPy uses C-order indexing. That means that the last + index usually represents the most rapidly changing memory location, + unlike Fortran or IDL, where the first index represents the most + rapidly changing location in memory. This difference represents a + great potential for confusion. +.. _basic-slicing-and-indexing: -Basic Slicing and Indexing +Basic slicing and indexing -------------------------- Basic slicing extends Python's basic concept of slicing to N @@ -67,7 +128,7 @@ of the original array. .. note:: NumPy slicing creates a :term:`view` instead of a copy as in the case of - builtin Python sequences such as string, tuple and list. + built-in Python sequences such as string, tuple and list. Care must be taken when extracting a small portion from a large array which becomes useless after the extraction, because the small portion extracted contains a reference @@ -85,9 +146,8 @@ concepts to remember include: index values *i*, *i + k*, ..., *i + (m - 1) k* where :math:`m = q + (r\neq0)` and *q* and *r* are the quotient and remainder obtained by dividing *j - i* by *k*: *j - i = q k + r*, so that - *i + (m - 1) k < j*. - - .. admonition:: Example + *i + (m - 1) k < j*. + For example:: >>> x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) >>> x[1:7:2] @@ -96,8 +156,7 @@ concepts to remember include: - Negative *i* and *j* are interpreted as *n + i* and *n + j* where *n* is the number of elements in the corresponding dimension. Negative *k* makes stepping go towards smaller indices. - - .. admonition:: Example + From the above example:: >>> x[-2:10] array([8, 9]) @@ -110,16 +169,14 @@ concepts to remember include: and *-n-1* for *k < 0* . If *k* is not given it defaults to 1. Note that ``::`` is the same as ``:`` and means select all indices along this axis. - - .. admonition:: Example + From the above example:: >>> x[5:] array([5, 6, 7, 8, 9]) - If the number of objects in the selection tuple is less than *N*, then ``:`` is assumed for any subsequent dimensions. - - .. admonition:: Example + For example:: >>> x = np.array([[[1],[2],[3]], [[4],[5],[6]]]) >>> x.shape @@ -129,27 +186,6 @@ concepts to remember include: [5], [6]]]) -- :py:data:`Ellipsis` expands to the number of ``:`` objects needed for the - selection tuple to index all dimensions. In most cases, this means that - length of the expanded selection tuple is ``x.ndim``. There may only be a - single ellipsis present. - - .. admonition:: Example - - >>> x[...,0] - array([[1, 2, 3], - [4, 5, 6]]) - -- Each :const:`newaxis` object in the selection tuple serves to expand - the dimensions of the resulting selection by one unit-length - dimension. The added dimension is the position of the :const:`newaxis` - object in the selection tuple. - - .. admonition:: Example - - >>> x[:,np.newaxis,:,:].shape - (2, 1, 3, 1) - - An integer, *i*, returns the same values as ``i:i+1`` **except** the dimensionality of the returned object is reduced by 1. In particular, a selection tuple with the *p*-th @@ -178,31 +214,71 @@ concepts to remember include: ``x[obj] = value`` must be (broadcastable) to the same shape as ``x[obj]``. +- A slicing tuple can always be constructed as *obj* + and used in the ``x[obj]`` notation. Slice objects can be used in + the construction in place of the ``[start:stop:step]`` + notation. For example, ``x[1:10:5,::-1]`` can also be implemented + as ``obj = (slice(1,10,5), slice(None,None,-1)); x[obj]`` . This + can be useful for constructing generic code that works on arrays + of arbitrary dimensions. See :ref:`dealing-with-variable-indices` + for more information. + .. index:: pair: ndarray; view -.. note:: +Structural indexing tools +^^^^^^^^^^^^^^^^^^^^^^^^^ +There are some tools to facilitate the easy matching of array shapes with +expressions and in assignments. + +:py:data:`Ellipsis` expands to the number of ``:`` objects needed for the +selection tuple to index all dimensions. In most cases, this means that the +length of the expanded selection tuple is ``x.ndim``. There may only be a +single ellipsis present. +From the above example:: + + >>> x[...,0] + array([[1, 2, 3], + [4, 5, 6]]) + +This is equivalent to:: - Remember that a slicing tuple can always be constructed as *obj* - and used in the ``x[obj]`` notation. Slice objects can be used in - the construction in place of the ``[start:stop:step]`` - notation. For example, ``x[1:10:5,::-1]`` can also be implemented - as ``obj = (slice(1,10,5), slice(None,None,-1)); x[obj]`` . This - can be useful for constructing generic code that works on arrays - of arbitrary dimension. + >>> x[:,:,0] + array([[1, 2, 3], + [4, 5, 6]]) -.. data:: newaxis - :noindex: +Each :const:`newaxis` object in the selection tuple serves to expand +the dimensions of the resulting selection by one unit-length +dimension. The added dimension is the position of the :const:`newaxis` +object in the selection tuple. :const:`newaxis` is an alias for +'None', and 'None' can be used in place of this with the same result. +From the above example:: + + >>> x[:,np.newaxis,:,:].shape + (2, 1, 3, 1) + >>> x[:,None,:,:].shape + (2, 1, 3, 1) + +This can be handy to combine two +arrays in a way that otherwise would require explicitly reshaping +operations. For example:: + + >>> x = np.arange(5) + >>> x[:,np.newaxis] + x[np.newaxis,:] + array([[0, 1, 2, 3, 4], + [1, 2, 3, 4, 5], + [2, 3, 4, 5, 6], + [3, 4, 5, 6, 7], + [4, 5, 6, 7, 8]]) - The :const:`newaxis` object can be used in all slicing operations to - create an axis of length one. :const:`newaxis` is an alias for - 'None', and 'None' can be used in place of this with the same result. .. _advanced-indexing: -Advanced Indexing +Advanced indexing ----------------- +.. seealso:: :ref:`basics.broadcasting` + Advanced indexing is triggered when the selection object, *obj*, is a non-tuple sequence object, an :class:`ndarray` (of data type integer or bool), or a tuple with at least one sequence object or ndarray (of data type @@ -229,29 +305,40 @@ Integer array indexing Integer array indexing allows selection of arbitrary items in the array based on their *N*-dimensional index. Each integer array represents a number -of indexes into that dimension. +of indices into that dimension. + +Negative values are permitted in the index arrays and work as they do with +single indices or slices. If the index values are out of bounds then an +``IndexError`` is thrown:: + + >>> x = np.array([[1, 2], [3, 4], [5, 6]]) + >>> x[np.array([1, -1])] + array([[3, 4], + [5, 6]]) + >>> x[np.array([3, 4])] + IndexError: index 3 is out of bounds for axis 0 with size 3 -Purely integer array indexing -""""""""""""""""""""""""""""" -When the index consists of as many integer arrays as the array being indexed -has dimensions, the indexing is straight forward, but different from slicing. +When the index consists of as many integer arrays as dimensions of the array +being indexed, the indexing is straightforward, but different from slicing. -Advanced indexes always are :ref:`broadcast` and +Advanced indices always are :ref:`broadcast` and iterated as *one*:: result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M], ..., ind_N[i_1, ..., i_M]] -Note that the result shape is identical to the (broadcast) indexing array -shapes ``ind_1, ..., ind_N``. +Note that the resulting shape is identical to the (broadcast) indexing array +shapes ``ind_1, ..., ind_N``. If the indices cannot be broadcast to the +same shape, an exception ``IndexError: shape mismatch: indexing arrays could +not be broadcast together with shapes...`` is raised. -.. admonition:: Example +.. rubric:: Example - From each row, a specific element should be selected. The row index is just - ``[0, 1, 2]`` and the column index specifies the element to choose for the - corresponding row, here ``[0, 1, 0]``. Using both together the task - can be solved using advanced indexing: +From each row, a specific element should be selected. The row index is just +``[0, 1, 2]`` and the column index specifies the element to choose for the +corresponding row, here ``[0, 1, 0]``. Using both together the task +can be solved using advanced indexing:: >>> x = np.array([[1, 2], [3, 4], [5, 6]]) >>> x[[0, 1, 2], [0, 1, 0]] @@ -261,13 +348,13 @@ To achieve a behaviour similar to the basic slicing above, broadcasting can be used. The function :func:`ix_` can help with this broadcasting. This is best understood with an example. -.. admonition:: Example +.. rubric:: Example - From a 4x3 array the corner elements should be selected using advanced - indexing. Thus all elements for which the column is one of ``[0, 2]`` and - the row is one of ``[0, 3]`` need to be selected. To use advanced indexing - one needs to select all elements *explicitly*. Using the method explained - previously one could write: +From a 4x3 array the corner elements should be selected using advanced +indexing. Thus all elements for which the column is one of ``[0, 2]`` and +the row is one of ``[0, 3]`` need to be selected. To use advanced indexing +one needs to select all elements *explicitly*. Using the method explained +previously one could write:: >>> x = np.array([[ 0, 1, 2], ... [ 3, 4, 5], @@ -281,9 +368,9 @@ understood with an example. array([[ 0, 2], [ 9, 11]]) - However, since the indexing arrays above just repeat themselves, - broadcasting can be used (compare operations such as - ``rows[:, np.newaxis] + columns``) to simplify this: +However, since the indexing arrays above just repeat themselves, +broadcasting can be used (compare operations such as +``rows[:, np.newaxis] + columns``) to simplify this:: >>> rows = np.array([0, 3], dtype=np.intp) >>> columns = np.array([0, 2], dtype=np.intp) @@ -294,88 +381,42 @@ understood with an example. array([[ 0, 2], [ 9, 11]]) - This broadcasting can also be achieved using the function :func:`ix_`: +This broadcasting can also be achieved using the function :func:`ix_`: >>> x[np.ix_(rows, columns)] array([[ 0, 2], [ 9, 11]]) - Note that without the ``np.ix_`` call, only the diagonal elements would - be selected, as was used in the previous example. This difference is the - most important thing to remember about indexing with multiple advanced - indexes. - -.. _combining-advanced-and-basic-indexing: - -Combining advanced and basic indexing -""""""""""""""""""""""""""""""""""""" - -When there is at least one slice (``:``), ellipsis (``...``) or :const:`newaxis` -in the index (or the array has more dimensions than there are advanced indexes), -then the behaviour can be more complicated. It is like concatenating the -indexing result for each advanced index element - -In the simplest case, there is only a *single* advanced index. A single -advanced index can for example replace a slice and the result array will be -the same, however, it is a copy and may have a different memory layout. -A slice is preferable when it is possible. - -.. admonition:: Example - - >>> x[1:2, 1:3] - array([[4, 5]]) - >>> x[1:2, [1, 2]] - array([[4, 5]]) +Note that without the ``np.ix_`` call, only the diagonal elements would +be selected, as was used in the previous example. This difference is the +most important thing to remember about indexing with multiple advanced +indices. -The easiest way to understand the situation may be to think in -terms of the result shape. There are two parts to the indexing operation, -the subspace defined by the basic indexing (excluding integers) and the -subspace from the advanced indexing part. Two cases of index combination -need to be distinguished: - -* The advanced indexes are separated by a slice, :py:data:`Ellipsis` or :const:`newaxis`. - For example ``x[arr1, :, arr2]``. -* The advanced indexes are all next to each other. - For example ``x[..., arr1, arr2, :]`` but *not* ``x[arr1, :, 1]`` - since ``1`` is an advanced index in this regard. - -In the first case, the dimensions resulting from the advanced indexing -operation come first in the result array, and the subspace dimensions after -that. -In the second case, the dimensions from the advanced indexing operations -are inserted into the result array at the same spot as they were in the -initial array (the latter logic is what makes simple advanced indexing -behave just like slicing). +.. rubric:: Example -.. admonition:: Example +The broadcasting mechanism permits index arrays to be combined with +scalars for other indices. The effect is that the scalar value is used +for all the corresponding values of the index arrays:: - Suppose ``x.shape`` is (10,20,30) and ``ind`` is a (2,3,4)-shaped - indexing :class:`intp` array, then ``result = x[...,ind,:]`` has - shape (10,2,3,4,30) because the (20,)-shaped subspace has been - replaced with a (2,3,4)-shaped broadcasted indexing subspace. If - we let *i, j, k* loop over the (2,3,4)-shaped subspace then - ``result[...,i,j,k,:] = x[...,ind[i,j,k],:]``. This example - produces the same result as :meth:`x.take(ind, axis=-2) `. + >>> x = np.arange(35).reshape(5,7) + >>> x[np.array([0,2,4]), 1] + array([ 1, 15, 29]) -.. admonition:: Example +.. rubric:: Example - Let ``x.shape`` be (10,20,30,40,50) and suppose ``ind_1`` - and ``ind_2`` can be broadcast to the shape (2,3,4). Then - ``x[:,ind_1,ind_2]`` has shape (10,2,3,4,40,50) because the - (20,30)-shaped subspace from X has been replaced with the - (2,3,4) subspace from the indices. However, - ``x[:,ind_1,:,ind_2]`` has shape (2,3,4,10,30,50) because there - is no unambiguous place to drop in the indexing subspace, thus - it is tacked-on to the beginning. It is always possible to use - :meth:`.transpose() ` to move the subspace - anywhere desired. Note that this example cannot be replicated - using :func:`take`. +A real-life example of where advanced indexing may be useful is for a color +lookup table where we want to map the values of an image into RGB triples for +display. The lookup table could have a shape (nlookup, 3). Indexing +such an array with an image with shape (ny, nx) with dtype=np.uint8 +(or any integer type so long as values are with the bounds of the +lookup table) will result in an array of shape (ny, nx, 3) where a +triple of RGB values is associated with each pixel location. Boolean array indexing ^^^^^^^^^^^^^^^^^^^^^^ -This advanced indexing occurs when obj is an array object of Boolean +This advanced indexing occurs when *obj* is an array object of Boolean type, such as may be returned from comparison operators. A single boolean index array is practically identical to ``x[obj.nonzero()]`` where, as described above, :meth:`obj.nonzero() ` returns a @@ -390,17 +431,15 @@ C-style. If *obj* has :py:data:`True` values at entries that are outside of the bounds of *x*, then an index error will be raised. If *obj* is smaller than *x* it is identical to filling it with :py:data:`False`. -.. admonition:: Example - - A common use case for this is filtering for desired element values. - For example one may wish to select all entries from an array which - are not NaN: +A common use case for this is filtering for desired element values. +For example, one may wish to select all entries from an array which +are not NaN:: >>> x = np.array([[1., 2.], [np.nan, 3.], [np.nan, np.nan]]) >>> x[~np.isnan(x)] array([1., 2., 3.]) - Or wish to add a constant to all negative elements: +Or wish to add a constant to all negative elements:: >>> x = np.array([1., -1., -2., 3]) >>> x[x < 0] += 20 @@ -418,9 +457,9 @@ this is straight forward. Care must only be taken to make sure that the boolean index has *exactly* as many dimensions as it is supposed to work with. -.. admonition:: Example +.. rubric:: Example - From an array, select all rows which sum up to less or equal two: +From an array, select all rows which sum up to less or equal two:: >>> x = np.array([[0, 1], [1, 1], [2, 2]]) >>> rowsum = x.sum(-1) @@ -434,12 +473,12 @@ indexing array can best be understood with the :meth:`obj.nonzero() ` analogy. The function :func:`ix_` also supports boolean arrays and will work without any surprises. -.. admonition:: Example +.. rubric:: Example - Use boolean indexing to select all rows adding up to an even - number. At the same time columns 0 and 2 should be selected with an - advanced integer index. Using the :func:`ix_` function this can be done - with: +Use boolean indexing to select all rows adding up to an even +number. At the same time columns 0 and 2 should be selected with an +advanced integer index. Using the :func:`ix_` function this can be done +with:: >>> x = np.array([[ 0, 1, 2], ... [ 3, 4, 5], @@ -453,64 +492,145 @@ also supports boolean arrays and will work without any surprises. array([[ 3, 5], [ 9, 11]]) - Without the ``np.ix_`` call, only the diagonal elements would be - selected. +Without the ``np.ix_`` call, only the diagonal elements would be +selected. - Or without ``np.ix_`` (compare the integer array examples): +Or without ``np.ix_`` (compare the integer array examples):: >>> rows = rows.nonzero()[0] >>> x[rows[:, np.newaxis], columns] array([[ 3, 5], [ 9, 11]]) -Detailed notes --------------- +.. rubric:: Example -These are some detailed notes, which are not of importance for day to day -indexing (in no particular order): +If x has more dimensions than b then the result will be multi-dimensional:: -* The native NumPy indexing type is ``intp`` and may differ from the - default integer array type. ``intp`` is the smallest data type - sufficient to safely index any array; for advanced indexing it may be - faster than other types. -* For advanced assignments, there is in general no guarantee for the - iteration order. This means that if an element is set more than once, - it is not possible to predict the final result. -* An empty (tuple) index is a full scalar index into a zero dimensional array. - ``x[()]`` returns a *scalar* if ``x`` is zero dimensional and a view - otherwise. On the other hand ``x[...]`` always returns a view. -* If a zero dimensional array is present in the index *and* it is a full - integer index the result will be a *scalar* and not a zero dimensional array. - (Advanced indexing is not triggered.) -* When an ellipsis (``...``) is present but has no size (i.e. replaces zero - ``:``) the result will still always be an array. A view if no advanced index - is present, otherwise a copy. -* the ``nonzero`` equivalence for Boolean arrays does not hold for zero - dimensional boolean arrays. -* When the result of an advanced indexing operation has no elements but an - individual index is out of bounds, whether or not an ``IndexError`` is - raised is undefined (e.g. ``x[[], [123]]`` with ``123`` being out of bounds). -* When a *casting* error occurs during assignment (for example updating a - numerical array using a sequence of strings), the array being assigned - to may end up in an unpredictable partially updated state. - However, if any other error (such as an out of bounds index) occurs, the - array will remain unchanged. -* The memory layout of an advanced indexing result is optimized for each - indexing operation and no particular memory order can be assumed. -* When using a subclass (especially one which manipulates its shape), the - default ``ndarray.__setitem__`` behaviour will call ``__getitem__`` for - *basic* indexing but not for *advanced* indexing. For such a subclass it may - be preferable to call ``ndarray.__setitem__`` with a *base class* ndarray - view on the data. This *must* be done if the subclasses ``__getitem__`` does - not return views. + >>> x = np.arange(35).reshape(5,7) + >>> b = x>20 + >>> b[:,5] + array([False, False, False, True, True]) + >>> x[b[:,5]] + array([[21, 22, 23, 24, 25, 26, 27], + [28, 29, 30, 31, 32, 33, 34]]) -.. _arrays.indexing.fields: +Here the 4th and 5th rows are selected from the indexed array and +combined to make a 2-D array. + +.. rubric:: Example +Using a 2-D boolean array of shape (2,3) +with four True elements to select rows from a 3-D array of shape +(2,3,5) results in a 2-D result of shape (4,5):: -Field Access + >>> x = np.arange(30).reshape(2,3,5) + >>> x + array([[[ 0, 1, 2, 3, 4], + [ 5, 6, 7, 8, 9], + [10, 11, 12, 13, 14]], + [[15, 16, 17, 18, 19], + [20, 21, 22, 23, 24], + [25, 26, 27, 28, 29]]]) + >>> b = np.array([[True, True, False], [False, True, True]]) + >>> x[b] + array([[ 0, 1, 2, 3, 4], + [ 5, 6, 7, 8, 9], + [20, 21, 22, 23, 24], + [25, 26, 27, 28, 29]]) + + +.. _combining-advanced-and-basic-indexing: + +Combining advanced and basic indexing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When there is at least one slice (``:``), ellipsis (``...``) or :const:`newaxis` +in the index (or the array has more dimensions than there are advanced indices), +then the behaviour can be more complicated. It is like concatenating the +indexing result for each advanced index element. + +In the simplest case, there is only a *single* advanced index. A single +advanced index can for example replace a slice and the result array will be +the same, however, it is a copy and may have a different memory layout. +A slice is preferable when it is possible. +For example:: + + >>> x = np.array([[ 0, 1, 2], + ... [ 3, 4, 5], + ... [ 6, 7, 8], + ... [ 9, 10, 11]]) + >>> x[1:2, 1:3] + array([[4, 5]]) + >>> x[1:2, [1, 2]] + array([[4, 5]]) + +The easiest way to understand the situation may be to think in +terms of the resulting shape. There are two parts to the indexing operation, +the subspace defined by the basic indexing (excluding integers) and the +subspace from the advanced indexing part. Two cases of index combination +need to be distinguished: + +* The advanced indices are separated by a slice, :py:data:`Ellipsis` or + :const:`newaxis`. For example ``x[arr1, :, arr2]``. +* The advanced indices are all next to each other. + For example ``x[..., arr1, arr2, :]`` but *not* ``x[arr1, :, 1]`` + since ``1`` is an advanced index in this regard. + +In the first case, the dimensions resulting from the advanced indexing +operation come first in the result array, and the subspace dimensions after +that. +In the second case, the dimensions from the advanced indexing operations +are inserted into the result array at the same spot as they were in the +initial array (the latter logic is what makes simple advanced indexing +behave just like slicing). + +.. rubric:: Example + +Suppose ``x.shape`` is (10,20,30) and ``ind`` is a (2,3,4)-shaped +indexing :class:`intp` array, then ``result = x[...,ind,:]`` has +shape (10,2,3,4,30) because the (20,)-shaped subspace has been +replaced with a (2,3,4)-shaped broadcasted indexing subspace. If +we let *i, j, k* loop over the (2,3,4)-shaped subspace then +``result[...,i,j,k,:] = x[...,ind[i,j,k],:]``. This example +produces the same result as :meth:`x.take(ind, axis=-2) `. + +.. rubric:: Example + +Let ``x.shape`` be (10,20,30,40,50) and suppose ``ind_1`` +and ``ind_2`` can be broadcast to the shape (2,3,4). Then +``x[:,ind_1,ind_2]`` has shape (10,2,3,4,40,50) because the +(20,30)-shaped subspace from X has been replaced with the +(2,3,4) subspace from the indices. However, +``x[:,ind_1,:,ind_2]`` has shape (2,3,4,10,30,50) because there +is no unambiguous place to drop in the indexing subspace, thus +it is tacked-on to the beginning. It is always possible to use +:meth:`.transpose() ` to move the subspace +anywhere desired. Note that this example cannot be replicated +using :func:`take`. + +.. rubric:: Example + +Slicing can be combined with broadcasted boolean indices:: + + >>> b = y > 20 + >>> b + array([[False, False, False, False, False, False, False], + [False, False, False, False, False, False, False], + [False, False, False, False, False, False, False], + [ True, True, True, True, True, True, True], + [ True, True, True, True, True, True, True]]) + >>> y[b[:,5],1:3] + array([[22, 23], + [29, 30]]) + + +.. _arrays.indexing.fields: + +Field access ------------- -.. seealso:: :ref:`arrays.dtypes`, :ref:`arrays.scalars` +.. seealso:: :ref:`arrays.dtypes`, :ref:`arrays.scalars`, + :ref:`structured_arrays` If the :class:`ndarray` object is a structured array the :term:`fields ` of the array can be accessed by indexing the array with strings, @@ -519,19 +639,18 @@ dictionary-like. Indexing ``x['field-name']`` returns a new :term:`view` to the array, which is of the same shape as *x* (except when the field is a sub-array) but of data type ``x.dtype['field-name']`` and contains -only the part of the data in the specified field. Also +only the part of the data in the specified field. Also, :ref:`record array ` scalars can be "indexed" this way. Indexing into a structured array can also be done with a list of field names, -*e.g.* ``x[['field-name1','field-name2']]``. As of NumPy 1.16 this returns a -view containing only those fields. In older versions of numpy it returned a +e.g. ``x[['field-name1','field-name2']]``. As of NumPy 1.16, this returns a +view containing only those fields. In older versions of NumPy, it returned a copy. See the user guide section on :ref:`structured_arrays` for more information on multifield indexing. If the accessed field is a sub-array, the dimensions of the sub-array are appended to the shape of the result. - -.. admonition:: Example +For example:: >>> x = np.zeros((2,2), dtype=[('a', np.int32), ('b', np.float64, (3,3))]) >>> x['a'].shape @@ -543,6 +662,7 @@ are appended to the shape of the result. >>> x['b'].dtype dtype('float64') +.. _flat-iterator-indexing: Flat Iterator indexing ---------------------- @@ -559,3 +679,44 @@ returned array is therefore the shape of the integer indexing object. .. index:: single: indexing single: ndarray + +Detailed notes +-------------- + +These are some detailed notes, which are not of importance for day to day +indexing (in no particular order): + +* The native NumPy indexing type is ``intp`` and may differ from the + default integer array type. ``intp`` is the smallest data type + sufficient to safely index any array; for advanced indexing it may be + faster than other types. +* For advanced assignments, there is in general no guarantee for the + iteration order. This means that if an element is set more than once, + it is not possible to predict the final result. +* An empty (tuple) index is a full scalar index into a zero-dimensional array. + ``x[()]`` returns a *scalar* if ``x`` is zero-dimensional and a view + otherwise. On the other hand, ``x[...]`` always returns a view. +* If a zero-dimensional array is present in the index *and* it is a full + integer index the result will be a *scalar* and not a zero-dimensional array. + (Advanced indexing is not triggered.) +* When an ellipsis (``...``) is present but has no size (i.e. replaces zero + ``:``) the result will still always be an array. A view if no advanced index + is present, otherwise a copy. +* The ``nonzero`` equivalence for Boolean arrays does not hold for zero + dimensional boolean arrays. +* When the result of an advanced indexing operation has no elements but an + individual index is out of bounds, whether or not an ``IndexError`` is + raised is undefined (e.g. ``x[[], [123]]`` with ``123`` being out of bounds). +* When a *casting* error occurs during assignment (for example updating a + numerical array using a sequence of strings), the array being assigned + to may end up in an unpredictable partially updated state. + However, if any other error (such as an out of bounds index) occurs, the + array will remain unchanged. +* The memory layout of an advanced indexing result is optimized for each + indexing operation and no particular memory order can be assumed. +* When using a subclass (especially one which manipulates its shape), the + default ``ndarray.__setitem__`` behaviour will call ``__getitem__`` for + *basic* indexing but not for *advanced* indexing. For such a subclass it may + be preferable to call ``ndarray.__setitem__`` with a *base class* ndarray + view on the data. This *must* be done if the subclasses ``__getitem__`` does + not return views. \ No newline at end of file diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index 7ee61b130..66d35abee 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -1,8 +1,8 @@ .. _basics.indexing: -******** -Indexing -******** +*************** +Indexing basics +*************** .. seealso:: @@ -14,355 +14,50 @@ Array indexing refers to any use of the square brackets ([]) to index array values. There are many options to indexing, which give NumPy indexing great power, but with power comes some complexity and the potential for confusion. This section is just an overview of the -various options and issues related to indexing. Aside from single -element indexing, the details on most of these options are to be -found in related sections. - -Assignment vs referencing -========================= - -Most of the following examples show the use of indexing when -referencing data in an array. The examples work just as well -when assigning to an array. See the section at the end for -specific examples and explanations on how assignments work. - -Single element indexing -======================= - -Single element indexing for a 1-D array is what one expects. It work -exactly like that for other standard Python sequences. It is 0-based, -and accepts negative indices for indexing from the end of the array. :: - - >>> x = np.arange(10) - >>> x[2] - 2 - >>> x[-2] - 8 - -Unlike lists and tuples, NumPy arrays support multidimensional indexing -for multidimensional arrays. That means that it is not necessary to -separate each dimension's index into its own set of square brackets. :: - - >>> x.shape = (2,5) # now x is 2-dimensional - >>> x[1,3] - 8 - >>> x[1,-1] - 9 - -Note that if one indexes a multidimensional array with fewer indices -than dimensions, one gets a subdimensional array. For example: :: - - >>> x[0] - array([0, 1, 2, 3, 4]) - -That is, each index specified selects the array corresponding to the -rest of the dimensions selected. In the above example, choosing 0 -means that the remaining dimension of length 5 is being left unspecified, -and that what is returned is an array of that dimensionality and size. -It must be noted that the returned array is not a copy of the original, -but points to the same values in memory as does the original array. -In this case, the 1-D array at the first position (0) is returned. -So using a single index on the returned array, results in a single -element being returned. That is: :: - - >>> x[0][2] - 2 - -So note that ``x[0,2] = x[0][2]`` though the second case is more -inefficient as a new temporary array is created after the first index -that is subsequently indexed by 2. - -Note to those used to IDL or Fortran memory order as it relates to -indexing. NumPy uses C-order indexing. That means that the last -index usually represents the most rapidly changing memory location, -unlike Fortran or IDL, where the first index represents the most -rapidly changing location in memory. This difference represents a -great potential for confusion. +various options and issues related to indexing. Further details can be +found in the relevant sections of :ref:`arrays.indexing`. -Other indexing options -====================== + +Basic indexing +============== + +Basic indexing will be triggered through single element indexing or +slicing and striding. + +Single element indexing works +exactly like that for other standard Python sequences but has been expanded +to support multidimensional indexing +for multidimensional NumPy arrays. For more details and examples, see +:ref:`single-element-indexing`. It is possible to slice and stride arrays to extract arrays of the same number of dimensions, but of different sizes than the original. -The slicing and striding works exactly the same way it does for lists +The slicing and striding work exactly the same way it does for lists and tuples except that they can be applied to multiple dimensions as -well. A few examples illustrates best: :: +well. See :ref:`basic-slicing-and-indexing` for more information and examples. - >>> x = np.arange(10) - >>> x[2:5] - array([2, 3, 4]) - >>> x[:-7] - array([0, 1, 2]) - >>> x[1:7:2] - array([1, 3, 5]) - >>> y = np.arange(35).reshape(5,7) - >>> y[1:5:2,::3] - array([[ 7, 10, 13], - [21, 24, 27]]) - -Note that slices of arrays do not copy the internal array data but -only produce new views of the original data. This is different from -list or tuple slicing and an explicit ``copy()`` is recommended if -the original data is not required anymore. - -It is possible to index arrays with other arrays for the purposes of -selecting lists of values out of arrays into new arrays. There are + +Advanced indexing +================= + +Arrays can be indexed with other arrays to +select lists of values out of arrays into new arrays. There are two different ways of accomplishing this. One uses one or more arrays of index values. The other involves giving a boolean array of the proper shape to indicate the values to be selected. Index arrays are a very -powerful tool that allow one to avoid looping over individual elements in -arrays and thus greatly improve performance. +powerful tool that allows one to avoid looping over individual elements in +arrays and thus greatly improve performance. See :ref:`advanced-indexing` +for more information and examples. -It is possible to use special features to effectively increase the -number of dimensions in an array through indexing so the resulting -array acquires the shape needed for use in an expression or with a -specific function. -Index arrays -============ +Other indexing options +====================== -NumPy arrays may be indexed with other arrays (or any other sequence- -like object that can be converted to an array, such as lists, with the -exception of tuples; see the end of this document for why this is). The -use of index arrays ranges from simple, straightforward cases to -complex, hard-to-understand cases. For all cases of index arrays, what -is returned is a copy of the original data, not a view as one gets for -slices. +:ref:`arrays.indexing.fields`, :ref:`flat-iterator-indexing` are a couple +of other indexing options. -Index arrays must be of integer type. Each value in the array indicates -which value in the array to use in place of the index. To illustrate: :: - >>> x = np.arange(10,1,-1) - >>> x - array([10, 9, 8, 7, 6, 5, 4, 3, 2]) - >>> x[np.array([3, 3, 1, 8])] - array([7, 7, 9, 2]) - - -The index array consisting of the values 3, 3, 1 and 8 correspondingly -create an array of length 4 (same as the index array) where each index -is replaced by the value the index array has in the array being indexed. - -Negative values are permitted and work as they do with single indices -or slices: :: - - >>> x[np.array([3,3,-3,8])] - array([7, 7, 4, 2]) - -It is an error to have index values out of bounds: :: - - >>> x[np.array([3, 3, 20, 8])] - : index 20 out of bounds 0<=index<9 - -Generally speaking, what is returned when index arrays are used is -an array with the same shape as the index array, but with the type -and values of the array being indexed. As an example, we can use a -multidimensional index array instead: :: - - >>> x[np.array([[1,1],[2,3]])] - array([[9, 9], - [8, 7]]) - -Indexing Multi-dimensional arrays -================================= - -Things become more complex when multidimensional arrays are indexed, -particularly with multidimensional index arrays. These tend to be -more unusual uses, but they are permitted, and they are useful for some -problems. We'll start with the simplest multidimensional case (using -the array y from the previous examples): :: - - >>> y[np.array([0,2,4]), np.array([0,1,2])] - array([ 0, 15, 30]) - -In this case, if the index arrays have a matching shape, and there is -an index array for each dimension of the array being indexed, the -resultant array has the same shape as the index arrays, and the values -correspond to the index set for each position in the index arrays. In -this example, the first index value is 0 for both index arrays, and -thus the first value of the resultant array is y[0,0]. The next value -is y[2,1], and the last is y[4,2]. - -If the index arrays do not have the same shape, there is an attempt to -broadcast them to the same shape. If they cannot be broadcast to the -same shape, an exception is raised: :: - - >>> y[np.array([0,2,4]), np.array([0,1])] - : shape mismatch: objects cannot be - broadcast to a single shape - -The broadcasting mechanism permits index arrays to be combined with -scalars for other indices. The effect is that the scalar value is used -for all the corresponding values of the index arrays: :: - - >>> y[np.array([0,2,4]), 1] - array([ 1, 15, 29]) - -Jumping to the next level of complexity, it is possible to only -partially index an array with index arrays. It takes a bit of thought -to understand what happens in such cases. For example if we just use -one index array with y: :: - - >>> y[np.array([0,2,4])] - array([[ 0, 1, 2, 3, 4, 5, 6], - [14, 15, 16, 17, 18, 19, 20], - [28, 29, 30, 31, 32, 33, 34]]) - -What results is the construction of a new array where each value of -the index array selects one row from the array being indexed and the -resultant array has the resulting shape (number of index elements, -size of row). - -An example of where this may be useful is for a color lookup table -where we want to map the values of an image into RGB triples for -display. The lookup table could have a shape (nlookup, 3). Indexing -such an array with an image with shape (ny, nx) with dtype=np.uint8 -(or any integer type so long as values are with the bounds of the -lookup table) will result in an array of shape (ny, nx, 3) where a -triple of RGB values is associated with each pixel location. - -In general, the shape of the resultant array will be the concatenation -of the shape of the index array (or the shape that all the index arrays -were broadcast to) with the shape of any unused dimensions (those not -indexed) in the array being indexed. - -Boolean or "mask" index arrays -============================== - -Boolean arrays used as indices are treated in a different manner -entirely than index arrays. Boolean arrays must be of the same shape -as the initial dimensions of the array being indexed. In the -most straightforward case, the boolean array has the same shape: :: - - >>> b = y>20 - >>> y[b] - array([21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34]) - -Unlike in the case of integer index arrays, in the boolean case, the -result is a 1-D array containing all the elements in the indexed array -corresponding to all the true elements in the boolean array. The -elements in the indexed array are always iterated and returned in -:term:`row-major` (C-style) order. The result is also identical to -``y[np.nonzero(b)]``. As with index arrays, what is returned is a copy -of the data, not a view as one gets with slices. - -The result will be multidimensional if y has more dimensions than b. -For example: :: - - >>> b[:,5] # use a 1-D boolean whose first dim agrees with the first dim of y - array([False, False, False, True, True]) - >>> y[b[:,5]] - array([[21, 22, 23, 24, 25, 26, 27], - [28, 29, 30, 31, 32, 33, 34]]) - -Here the 4th and 5th rows are selected from the indexed array and -combined to make a 2-D array. - -In general, when the boolean array has fewer dimensions than the array -being indexed, this is equivalent to y[b, ...], which means -y is indexed by b followed by as many : as are needed to fill -out the rank of y. -Thus the shape of the result is one dimension containing the number -of True elements of the boolean array, followed by the remaining -dimensions of the array being indexed. - -For example, using a 2-D boolean array of shape (2,3) -with four True elements to select rows from a 3-D array of shape -(2,3,5) results in a 2-D result of shape (4,5): :: - - >>> x = np.arange(30).reshape(2,3,5) - >>> x - array([[[ 0, 1, 2, 3, 4], - [ 5, 6, 7, 8, 9], - [10, 11, 12, 13, 14]], - [[15, 16, 17, 18, 19], - [20, 21, 22, 23, 24], - [25, 26, 27, 28, 29]]]) - >>> b = np.array([[True, True, False], [False, True, True]]) - >>> x[b] - array([[ 0, 1, 2, 3, 4], - [ 5, 6, 7, 8, 9], - [20, 21, 22, 23, 24], - [25, 26, 27, 28, 29]]) - -For further details, consult the NumPy reference documentation on array indexing. - -Combining index arrays with slices -================================== - -Index arrays may be combined with slices. For example: :: - - >>> y[np.array([0, 2, 4]), 1:3] - array([[ 1, 2], - [15, 16], - [29, 30]]) - -In effect, the slice and index array operation are independent. -The slice operation extracts columns with index 1 and 2, -(i.e. the 2nd and 3rd columns), -followed by the index array operation which extracts rows with -index 0, 2 and 4 (i.e the first, third and fifth rows). - -This is equivalent to:: - - >>> y[:, 1:3][np.array([0, 2, 4]), :] - array([[ 1, 2], - [15, 16], - [29, 30]]) - -Likewise, slicing can be combined with broadcasted boolean indices: :: - - >>> b = y > 20 - >>> b - array([[False, False, False, False, False, False, False], - [False, False, False, False, False, False, False], - [False, False, False, False, False, False, False], - [ True, True, True, True, True, True, True], - [ True, True, True, True, True, True, True]]) - >>> y[b[:,5],1:3] - array([[22, 23], - [29, 30]]) - -Structural indexing tools -========================= - -To facilitate easy matching of array shapes with expressions and in -assignments, the np.newaxis object can be used within array indices -to add new dimensions with a size of 1. For example: :: - - >>> y.shape - (5, 7) - >>> y[:,np.newaxis,:].shape - (5, 1, 7) - -Note that there are no new elements in the array, just that the -dimensionality is increased. This can be handy to combine two -arrays in a way that otherwise would require explicitly reshaping -operations. For example: :: - - >>> x = np.arange(5) - >>> x[:,np.newaxis] + x[np.newaxis,:] - array([[0, 1, 2, 3, 4], - [1, 2, 3, 4, 5], - [2, 3, 4, 5, 6], - [3, 4, 5, 6, 7], - [4, 5, 6, 7, 8]]) - -The ellipsis syntax maybe used to indicate selecting in full any -remaining unspecified dimensions. For example: :: - - >>> z = np.arange(81).reshape(3,3,3,3) - >>> z[1,...,2] - array([[29, 32, 35], - [38, 41, 44], - [47, 50, 53]]) - -This is equivalent to: :: - - >>> z[1,:,:,2] - array([[29, 32, 35], - [38, 41, 44], - [47, 50, 53]]) +.. _assigning-values-to-indexed-arrays: Assigning values to indexed arrays ================================== @@ -405,17 +100,19 @@ example is often surprising to people: :: array([ 0, 11, 20, 31, 40]) Where people expect that the 1st location will be incremented by 3. -In fact, it will only be incremented by 1. The reason is because +In fact, it will only be incremented by 1. The reason is that a new array is extracted from the original (as a temporary) containing the values at 1, 1, 3, 1, then the value 1 is added to the temporary, and then the temporary is assigned back to the original array. Thus the value of the array at x[1]+1 is assigned to x[1] three times, rather than being incremented 3 times. +.. _dealing-with-variable-indices: + Dealing with variable numbers of indices within programs ======================================================== -The index syntax is very powerful but limiting when dealing with +The indexing syntax is very powerful but limiting when dealing with a variable number of indices. For example, if you want to write a function that can handle arguments with various numbers of dimensions without having to write special case code for each @@ -447,9 +144,9 @@ object: :: [37, 40, 43], [46, 49, 52]]) -For this reason it is possible to use the output from the np.nonzero() -function directly as an index since it always returns a tuple of index -arrays. +For this reason, it is possible to use the output from the +:meth:`np.nonzero() ` function directly as an index since +it always returns a tuple of index arrays. Because the special treatment of tuples, they are not automatically converted to an array as a list would be. As an example: :: -- cgit v1.2.1 From a69e225446ba5d1232506c47e72d796dde621edf Mon Sep 17 00:00:00 2001 From: Mukulika Date: Wed, 14 Jul 2021 14:53:34 +0530 Subject: DOC: Exchanged the contents of Indexing basics and Indexing ref docs --- doc/source/reference/arrays.indexing.rst | 730 ++---------------------------- doc/source/user/basics.indexing.rst | 739 +++++++++++++++++++++++++++++-- 2 files changed, 735 insertions(+), 734 deletions(-) (limited to 'doc/source') diff --git a/doc/source/reference/arrays.indexing.rst b/doc/source/reference/arrays.indexing.rst index 58b87dd7f..cba028baa 100644 --- a/doc/source/reference/arrays.indexing.rst +++ b/doc/source/reference/arrays.indexing.rst @@ -3,8 +3,9 @@ .. _arrays.indexing: +******** Indexing -======== +******** .. seealso:: @@ -12,711 +13,48 @@ Indexing :ref:`Indexing routines ` -.. sectionauthor:: adapted from "Guide to NumPy" by Travis E. Oliphant +Array indexing refers to any use of the square brackets ([]) to index +array values. There are many options to indexing, which give NumPy +indexing great power, but with power comes some complexity and the +potential for confusion. This section is just an overview of the +various options and issues related to indexing. Further details can be +found in the relevant sections of :ref:`basics.indexing`. -.. currentmodule:: numpy -.. index:: indexing, slicing +Basic indexing +============== -:class:`ndarrays ` can be indexed using the standard Python -``x[obj]`` syntax, where *x* is the array and *obj* the selection. -There are four kinds of indexing available depending on *obj*: -single element indexing, basic slicing, advanced indexing and field access. - -Most of the following examples show the use of indexing when -referencing data in an array. The examples work just as well -when assigning to an array. See :ref:`assigning-values-to-indexed-arrays` for -specific examples and explanations on how assignments work. - -Note that in Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to -``x[exp1, exp2, ..., expN]``; the latter is just syntactic sugar -for the former. - -.. _single-element-indexing: - -Single element indexing ------------------------ +Basic indexing will be triggered through single element indexing or +slicing and striding. Single element indexing works -exactly like that for other standard Python sequences. It is 0-based, -and accepts negative indices for indexing from the end of the array. :: - - >>> x = np.arange(10) - >>> x[2] - 2 - >>> x[-2] - 8 - -It is not necessary to -separate each dimension's index into its own set of square brackets. :: - - >>> x.shape = (2,5) # now x is 2-dimensional - >>> x[1,3] - 8 - >>> x[1,-1] - 9 - -Note that if one indexes a multidimensional array with fewer indices -than dimensions, one gets a subdimensional array. For example: :: - - >>> x[0] - array([0, 1, 2, 3, 4]) - -That is, each index specified selects the array corresponding to the -rest of the dimensions selected. In the above example, choosing 0 -means that the remaining dimension of length 5 is being left unspecified, -and that what is returned is an array of that dimensionality and size. -It must be noted that the returned array is not a copy of the original, -but points to the same values in memory as does the original array. -In this case, the 1-D array at the first position (0) is returned. -So using a single index on the returned array, results in a single -element being returned. That is: :: - - >>> x[0][2] - 2 - -So note that ``x[0,2] = x[0][2]`` though the second case is more -inefficient as a new temporary array is created after the first index -that is subsequently indexed by 2. - -.. note:: - - NumPy uses C-order indexing. That means that the last - index usually represents the most rapidly changing memory location, - unlike Fortran or IDL, where the first index represents the most - rapidly changing location in memory. This difference represents a - great potential for confusion. - -.. _basic-slicing-and-indexing: - -Basic slicing and indexing --------------------------- - -Basic slicing extends Python's basic concept of slicing to N -dimensions. Basic slicing occurs when *obj* is a :class:`slice` object -(constructed by ``start:stop:step`` notation inside of brackets), an -integer, or a tuple of slice objects and integers. :py:data:`Ellipsis` -and :const:`newaxis` objects can be interspersed with these as -well. - -.. deprecated:: 1.15.0 - - In order to remain backward compatible with a common usage in - Numeric, basic slicing is also initiated if the selection object is - any non-ndarray and non-tuple sequence (such as a :class:`list`) containing - :class:`slice` objects, the :py:data:`Ellipsis` object, or the :const:`newaxis` - object, but not for integer arrays or other embedded sequences. - -.. index:: - triple: ndarray; special methods; getitem - triple: ndarray; special methods; setitem - single: ellipsis - single: newaxis - -The simplest case of indexing with *N* integers returns an :ref:`array -scalar ` representing the corresponding item. As in -Python, all indices are zero-based: for the *i*-th index :math:`n_i`, -the valid range is :math:`0 \le n_i < d_i` where :math:`d_i` is the -*i*-th element of the shape of the array. Negative indices are -interpreted as counting from the end of the array (*i.e.*, if -:math:`n_i < 0`, it means :math:`n_i + d_i`). - - -All arrays generated by basic slicing are always :term:`views ` -of the original array. - -.. note:: - - NumPy slicing creates a :term:`view` instead of a copy as in the case of - built-in Python sequences such as string, tuple and list. - Care must be taken when extracting - a small portion from a large array which becomes useless after the - extraction, because the small portion extracted contains a reference - to the large original array whose memory will not be released until - all arrays derived from it are garbage-collected. In such cases an - explicit ``copy()`` is recommended. - -The standard rules of sequence slicing apply to basic slicing on a -per-dimension basis (including using a step index). Some useful -concepts to remember include: - -- The basic slice syntax is ``i:j:k`` where *i* is the starting index, - *j* is the stopping index, and *k* is the step (:math:`k\neq0`). - This selects the *m* elements (in the corresponding dimension) with - index values *i*, *i + k*, ..., *i + (m - 1) k* where - :math:`m = q + (r\neq0)` and *q* and *r* are the quotient and remainder - obtained by dividing *j - i* by *k*: *j - i = q k + r*, so that - *i + (m - 1) k < j*. - For example:: - - >>> x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) - >>> x[1:7:2] - array([1, 3, 5]) - -- Negative *i* and *j* are interpreted as *n + i* and *n + j* where - *n* is the number of elements in the corresponding dimension. - Negative *k* makes stepping go towards smaller indices. - From the above example:: - - >>> x[-2:10] - array([8, 9]) - >>> x[-3:3:-1] - array([7, 6, 5, 4]) - -- Assume *n* is the number of elements in the dimension being - sliced. Then, if *i* is not given it defaults to 0 for *k > 0* and - *n - 1* for *k < 0* . If *j* is not given it defaults to *n* for *k > 0* - and *-n-1* for *k < 0* . If *k* is not given it defaults to 1. Note that - ``::`` is the same as ``:`` and means select all indices along this - axis. - From the above example:: - - >>> x[5:] - array([5, 6, 7, 8, 9]) - -- If the number of objects in the selection tuple is less than - *N*, then ``:`` is assumed for any subsequent dimensions. - For example:: - - >>> x = np.array([[[1],[2],[3]], [[4],[5],[6]]]) - >>> x.shape - (2, 3, 1) - >>> x[1:2] - array([[[4], - [5], - [6]]]) - -- An integer, *i*, returns the same values as ``i:i+1`` - **except** the dimensionality of the returned object is reduced by - 1. In particular, a selection tuple with the *p*-th - element an integer (and all other entries ``:``) returns the - corresponding sub-array with dimension *N - 1*. If *N = 1* - then the returned object is an array scalar. These objects are - explained in :ref:`arrays.scalars`. +exactly like that for other standard Python sequences but has been expanded +to support multidimensional indexing +for multidimensional NumPy arrays. For more details and examples, see +:ref:`single-element-indexing`. -- If the selection tuple has all entries ``:`` except the - *p*-th entry which is a slice object ``i:j:k``, - then the returned array has dimension *N* formed by - concatenating the sub-arrays returned by integer indexing of - elements *i*, *i+k*, ..., *i + (m - 1) k < j*, +It is possible to slice and stride arrays to extract arrays of the +same number of dimensions, but of different sizes than the original. +The slicing and striding work exactly the same way it does for lists +and tuples except that they can be applied to multiple dimensions as +well. See :ref:`basic-slicing-and-indexing` for more information and examples. -- Basic slicing with more than one non-``:`` entry in the slicing - tuple, acts like repeated application of slicing using a single - non-``:`` entry, where the non-``:`` entries are successively taken - (with all other non-``:`` entries replaced by ``:``). Thus, - ``x[ind1,...,ind2,:]`` acts like ``x[ind1][...,ind2,:]`` under basic - slicing. - - .. warning:: The above is **not** true for advanced indexing. - -- You may use slicing to set values in the array, but (unlike lists) you - can never grow the array. The size of the value to be set in - ``x[obj] = value`` must be (broadcastable) to the same shape as - ``x[obj]``. - -- A slicing tuple can always be constructed as *obj* - and used in the ``x[obj]`` notation. Slice objects can be used in - the construction in place of the ``[start:stop:step]`` - notation. For example, ``x[1:10:5,::-1]`` can also be implemented - as ``obj = (slice(1,10,5), slice(None,None,-1)); x[obj]`` . This - can be useful for constructing generic code that works on arrays - of arbitrary dimensions. See :ref:`dealing-with-variable-indices` - for more information. - -.. index:: - pair: ndarray; view - -Structural indexing tools -^^^^^^^^^^^^^^^^^^^^^^^^^ -There are some tools to facilitate the easy matching of array shapes with -expressions and in assignments. - -:py:data:`Ellipsis` expands to the number of ``:`` objects needed for the -selection tuple to index all dimensions. In most cases, this means that the -length of the expanded selection tuple is ``x.ndim``. There may only be a -single ellipsis present. -From the above example:: - - >>> x[...,0] - array([[1, 2, 3], - [4, 5, 6]]) - -This is equivalent to:: - - >>> x[:,:,0] - array([[1, 2, 3], - [4, 5, 6]]) - -Each :const:`newaxis` object in the selection tuple serves to expand -the dimensions of the resulting selection by one unit-length -dimension. The added dimension is the position of the :const:`newaxis` -object in the selection tuple. :const:`newaxis` is an alias for -'None', and 'None' can be used in place of this with the same result. -From the above example:: - - >>> x[:,np.newaxis,:,:].shape - (2, 1, 3, 1) - >>> x[:,None,:,:].shape - (2, 1, 3, 1) - -This can be handy to combine two -arrays in a way that otherwise would require explicitly reshaping -operations. For example:: - - >>> x = np.arange(5) - >>> x[:,np.newaxis] + x[np.newaxis,:] - array([[0, 1, 2, 3, 4], - [1, 2, 3, 4, 5], - [2, 3, 4, 5, 6], - [3, 4, 5, 6, 7], - [4, 5, 6, 7, 8]]) - - -.. _advanced-indexing: Advanced indexing ------------------ - -.. seealso:: :ref:`basics.broadcasting` - -Advanced indexing is triggered when the selection object, *obj*, is a -non-tuple sequence object, an :class:`ndarray` (of data type integer or bool), -or a tuple with at least one sequence object or ndarray (of data type -integer or bool). There are two types of advanced indexing: integer -and Boolean. - -Advanced indexing always returns a *copy* of the data (contrast with -basic slicing that returns a :term:`view`). - -.. warning:: - - The definition of advanced indexing means that ``x[(1,2,3),]`` is - fundamentally different than ``x[(1,2,3)]``. The latter is - equivalent to ``x[1,2,3]`` which will trigger basic selection while - the former will trigger advanced indexing. Be sure to understand - why this occurs. - - Also recognize that ``x[[1,2,3]]`` will trigger advanced indexing, - whereas due to the deprecated Numeric compatibility mentioned above, - ``x[[1,2,slice(None)]]`` will trigger basic slicing. - -Integer array indexing -^^^^^^^^^^^^^^^^^^^^^^ - -Integer array indexing allows selection of arbitrary items in the array -based on their *N*-dimensional index. Each integer array represents a number -of indices into that dimension. - -Negative values are permitted in the index arrays and work as they do with -single indices or slices. If the index values are out of bounds then an -``IndexError`` is thrown:: - - >>> x = np.array([[1, 2], [3, 4], [5, 6]]) - >>> x[np.array([1, -1])] - array([[3, 4], - [5, 6]]) - >>> x[np.array([3, 4])] - IndexError: index 3 is out of bounds for axis 0 with size 3 - - -When the index consists of as many integer arrays as dimensions of the array -being indexed, the indexing is straightforward, but different from slicing. - -Advanced indices always are :ref:`broadcast` and -iterated as *one*:: - - result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M], - ..., ind_N[i_1, ..., i_M]] - -Note that the resulting shape is identical to the (broadcast) indexing array -shapes ``ind_1, ..., ind_N``. If the indices cannot be broadcast to the -same shape, an exception ``IndexError: shape mismatch: indexing arrays could -not be broadcast together with shapes...`` is raised. - -.. rubric:: Example - -From each row, a specific element should be selected. The row index is just -``[0, 1, 2]`` and the column index specifies the element to choose for the -corresponding row, here ``[0, 1, 0]``. Using both together the task -can be solved using advanced indexing:: - - >>> x = np.array([[1, 2], [3, 4], [5, 6]]) - >>> x[[0, 1, 2], [0, 1, 0]] - array([1, 4, 5]) - -To achieve a behaviour similar to the basic slicing above, broadcasting can be -used. The function :func:`ix_` can help with this broadcasting. This is best -understood with an example. - -.. rubric:: Example - -From a 4x3 array the corner elements should be selected using advanced -indexing. Thus all elements for which the column is one of ``[0, 2]`` and -the row is one of ``[0, 3]`` need to be selected. To use advanced indexing -one needs to select all elements *explicitly*. Using the method explained -previously one could write:: - - >>> x = np.array([[ 0, 1, 2], - ... [ 3, 4, 5], - ... [ 6, 7, 8], - ... [ 9, 10, 11]]) - >>> rows = np.array([[0, 0], - ... [3, 3]], dtype=np.intp) - >>> columns = np.array([[0, 2], - ... [0, 2]], dtype=np.intp) - >>> x[rows, columns] - array([[ 0, 2], - [ 9, 11]]) - -However, since the indexing arrays above just repeat themselves, -broadcasting can be used (compare operations such as -``rows[:, np.newaxis] + columns``) to simplify this:: - - >>> rows = np.array([0, 3], dtype=np.intp) - >>> columns = np.array([0, 2], dtype=np.intp) - >>> rows[:, np.newaxis] - array([[0], - [3]]) - >>> x[rows[:, np.newaxis], columns] - array([[ 0, 2], - [ 9, 11]]) - -This broadcasting can also be achieved using the function :func:`ix_`: - - >>> x[np.ix_(rows, columns)] - array([[ 0, 2], - [ 9, 11]]) - -Note that without the ``np.ix_`` call, only the diagonal elements would -be selected, as was used in the previous example. This difference is the -most important thing to remember about indexing with multiple advanced -indices. - -.. rubric:: Example - -The broadcasting mechanism permits index arrays to be combined with -scalars for other indices. The effect is that the scalar value is used -for all the corresponding values of the index arrays:: - - >>> x = np.arange(35).reshape(5,7) - >>> x[np.array([0,2,4]), 1] - array([ 1, 15, 29]) - -.. rubric:: Example - -A real-life example of where advanced indexing may be useful is for a color -lookup table where we want to map the values of an image into RGB triples for -display. The lookup table could have a shape (nlookup, 3). Indexing -such an array with an image with shape (ny, nx) with dtype=np.uint8 -(or any integer type so long as values are with the bounds of the -lookup table) will result in an array of shape (ny, nx, 3) where a -triple of RGB values is associated with each pixel location. - - -Boolean array indexing -^^^^^^^^^^^^^^^^^^^^^^ - -This advanced indexing occurs when *obj* is an array object of Boolean -type, such as may be returned from comparison operators. A single -boolean index array is practically identical to ``x[obj.nonzero()]`` where, -as described above, :meth:`obj.nonzero() ` returns a -tuple (of length :attr:`obj.ndim `) of integer index -arrays showing the :py:data:`True` elements of *obj*. However, it is -faster when ``obj.shape == x.shape``. - -If ``obj.ndim == x.ndim``, ``x[obj]`` returns a 1-dimensional array -filled with the elements of *x* corresponding to the :py:data:`True` -values of *obj*. The search order will be :term:`row-major`, -C-style. If *obj* has :py:data:`True` values at entries that are outside -of the bounds of *x*, then an index error will be raised. If *obj* is -smaller than *x* it is identical to filling it with :py:data:`False`. - -A common use case for this is filtering for desired element values. -For example, one may wish to select all entries from an array which -are not NaN:: - - >>> x = np.array([[1., 2.], [np.nan, 3.], [np.nan, np.nan]]) - >>> x[~np.isnan(x)] - array([1., 2., 3.]) - -Or wish to add a constant to all negative elements:: - - >>> x = np.array([1., -1., -2., 3]) - >>> x[x < 0] += 20 - >>> x - array([ 1., 19., 18., 3.]) - -In general if an index includes a Boolean array, the result will be -identical to inserting ``obj.nonzero()`` into the same position -and using the integer array indexing mechanism described above. -``x[ind_1, boolean_array, ind_2]`` is equivalent to -``x[(ind_1,) + boolean_array.nonzero() + (ind_2,)]``. - -If there is only one Boolean array and no integer indexing array present, -this is straight forward. Care must only be taken to make sure that the -boolean index has *exactly* as many dimensions as it is supposed to work -with. - -.. rubric:: Example - -From an array, select all rows which sum up to less or equal two:: - - >>> x = np.array([[0, 1], [1, 1], [2, 2]]) - >>> rowsum = x.sum(-1) - >>> x[rowsum <= 2, :] - array([[0, 1], - [1, 1]]) - - -Combining multiple Boolean indexing arrays or a Boolean with an integer -indexing array can best be understood with the -:meth:`obj.nonzero() ` analogy. The function :func:`ix_` -also supports boolean arrays and will work without any surprises. - -.. rubric:: Example - -Use boolean indexing to select all rows adding up to an even -number. At the same time columns 0 and 2 should be selected with an -advanced integer index. Using the :func:`ix_` function this can be done -with:: - - >>> x = np.array([[ 0, 1, 2], - ... [ 3, 4, 5], - ... [ 6, 7, 8], - ... [ 9, 10, 11]]) - >>> rows = (x.sum(-1) % 2) == 0 - >>> rows - array([False, True, False, True]) - >>> columns = [0, 2] - >>> x[np.ix_(rows, columns)] - array([[ 3, 5], - [ 9, 11]]) - -Without the ``np.ix_`` call, only the diagonal elements would be -selected. - -Or without ``np.ix_`` (compare the integer array examples):: - - >>> rows = rows.nonzero()[0] - >>> x[rows[:, np.newaxis], columns] - array([[ 3, 5], - [ 9, 11]]) - -.. rubric:: Example - -If x has more dimensions than b then the result will be multi-dimensional:: - - >>> x = np.arange(35).reshape(5,7) - >>> b = x>20 - >>> b[:,5] - array([False, False, False, True, True]) - >>> x[b[:,5]] - array([[21, 22, 23, 24, 25, 26, 27], - [28, 29, 30, 31, 32, 33, 34]]) - -Here the 4th and 5th rows are selected from the indexed array and -combined to make a 2-D array. - -.. rubric:: Example - -Using a 2-D boolean array of shape (2,3) -with four True elements to select rows from a 3-D array of shape -(2,3,5) results in a 2-D result of shape (4,5):: - - >>> x = np.arange(30).reshape(2,3,5) - >>> x - array([[[ 0, 1, 2, 3, 4], - [ 5, 6, 7, 8, 9], - [10, 11, 12, 13, 14]], - [[15, 16, 17, 18, 19], - [20, 21, 22, 23, 24], - [25, 26, 27, 28, 29]]]) - >>> b = np.array([[True, True, False], [False, True, True]]) - >>> x[b] - array([[ 0, 1, 2, 3, 4], - [ 5, 6, 7, 8, 9], - [20, 21, 22, 23, 24], - [25, 26, 27, 28, 29]]) - - -.. _combining-advanced-and-basic-indexing: - -Combining advanced and basic indexing -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When there is at least one slice (``:``), ellipsis (``...``) or :const:`newaxis` -in the index (or the array has more dimensions than there are advanced indices), -then the behaviour can be more complicated. It is like concatenating the -indexing result for each advanced index element. - -In the simplest case, there is only a *single* advanced index. A single -advanced index can for example replace a slice and the result array will be -the same, however, it is a copy and may have a different memory layout. -A slice is preferable when it is possible. -For example:: - - >>> x = np.array([[ 0, 1, 2], - ... [ 3, 4, 5], - ... [ 6, 7, 8], - ... [ 9, 10, 11]]) - >>> x[1:2, 1:3] - array([[4, 5]]) - >>> x[1:2, [1, 2]] - array([[4, 5]]) - -The easiest way to understand the situation may be to think in -terms of the resulting shape. There are two parts to the indexing operation, -the subspace defined by the basic indexing (excluding integers) and the -subspace from the advanced indexing part. Two cases of index combination -need to be distinguished: - -* The advanced indices are separated by a slice, :py:data:`Ellipsis` or - :const:`newaxis`. For example ``x[arr1, :, arr2]``. -* The advanced indices are all next to each other. - For example ``x[..., arr1, arr2, :]`` but *not* ``x[arr1, :, 1]`` - since ``1`` is an advanced index in this regard. - -In the first case, the dimensions resulting from the advanced indexing -operation come first in the result array, and the subspace dimensions after -that. -In the second case, the dimensions from the advanced indexing operations -are inserted into the result array at the same spot as they were in the -initial array (the latter logic is what makes simple advanced indexing -behave just like slicing). - -.. rubric:: Example - -Suppose ``x.shape`` is (10,20,30) and ``ind`` is a (2,3,4)-shaped -indexing :class:`intp` array, then ``result = x[...,ind,:]`` has -shape (10,2,3,4,30) because the (20,)-shaped subspace has been -replaced with a (2,3,4)-shaped broadcasted indexing subspace. If -we let *i, j, k* loop over the (2,3,4)-shaped subspace then -``result[...,i,j,k,:] = x[...,ind[i,j,k],:]``. This example -produces the same result as :meth:`x.take(ind, axis=-2) `. - -.. rubric:: Example - -Let ``x.shape`` be (10,20,30,40,50) and suppose ``ind_1`` -and ``ind_2`` can be broadcast to the shape (2,3,4). Then -``x[:,ind_1,ind_2]`` has shape (10,2,3,4,40,50) because the -(20,30)-shaped subspace from X has been replaced with the -(2,3,4) subspace from the indices. However, -``x[:,ind_1,:,ind_2]`` has shape (2,3,4,10,30,50) because there -is no unambiguous place to drop in the indexing subspace, thus -it is tacked-on to the beginning. It is always possible to use -:meth:`.transpose() ` to move the subspace -anywhere desired. Note that this example cannot be replicated -using :func:`take`. - -.. rubric:: Example - -Slicing can be combined with broadcasted boolean indices:: - - >>> b = y > 20 - >>> b - array([[False, False, False, False, False, False, False], - [False, False, False, False, False, False, False], - [False, False, False, False, False, False, False], - [ True, True, True, True, True, True, True], - [ True, True, True, True, True, True, True]]) - >>> y[b[:,5],1:3] - array([[22, 23], - [29, 30]]) - - -.. _arrays.indexing.fields: - -Field access -------------- - -.. seealso:: :ref:`arrays.dtypes`, :ref:`arrays.scalars`, - :ref:`structured_arrays` - -If the :class:`ndarray` object is a structured array the :term:`fields ` -of the array can be accessed by indexing the array with strings, -dictionary-like. - -Indexing ``x['field-name']`` returns a new :term:`view` to the array, -which is of the same shape as *x* (except when the field is a -sub-array) but of data type ``x.dtype['field-name']`` and contains -only the part of the data in the specified field. Also, -:ref:`record array ` scalars can be "indexed" this way. - -Indexing into a structured array can also be done with a list of field names, -e.g. ``x[['field-name1','field-name2']]``. As of NumPy 1.16, this returns a -view containing only those fields. In older versions of NumPy, it returned a -copy. See the user guide section on :ref:`structured_arrays` for more -information on multifield indexing. - -If the accessed field is a sub-array, the dimensions of the sub-array -are appended to the shape of the result. -For example:: - - >>> x = np.zeros((2,2), dtype=[('a', np.int32), ('b', np.float64, (3,3))]) - >>> x['a'].shape - (2, 2) - >>> x['a'].dtype - dtype('int32') - >>> x['b'].shape - (2, 2, 3, 3) - >>> x['b'].dtype - dtype('float64') - -.. _flat-iterator-indexing: - -Flat Iterator indexing ----------------------- - -:attr:`x.flat ` returns an iterator that will iterate -over the entire array (in C-contiguous style with the last index -varying the fastest). This iterator object can also be indexed using -basic slicing or advanced indexing as long as the selection object is -not a tuple. This should be clear from the fact that :attr:`x.flat -` is a 1-dimensional view. It can be used for integer -indexing with 1-dimensional C-style-flat indices. The shape of any -returned array is therefore the shape of the integer indexing object. +================= -.. index:: - single: indexing - single: ndarray +Arrays can be indexed with other arrays to +select lists of values out of arrays into new arrays. There are +two different ways of accomplishing this. One uses one or more arrays +of index values. The other involves giving a boolean array of the proper +shape to indicate the values to be selected. Index arrays are a very +powerful tool that allows one to avoid looping over individual elements in +arrays and thus greatly improve performance. See :ref:`advanced-indexing` +for more information and examples. -Detailed notes --------------- -These are some detailed notes, which are not of importance for day to day -indexing (in no particular order): +Other indexing options +====================== -* The native NumPy indexing type is ``intp`` and may differ from the - default integer array type. ``intp`` is the smallest data type - sufficient to safely index any array; for advanced indexing it may be - faster than other types. -* For advanced assignments, there is in general no guarantee for the - iteration order. This means that if an element is set more than once, - it is not possible to predict the final result. -* An empty (tuple) index is a full scalar index into a zero-dimensional array. - ``x[()]`` returns a *scalar* if ``x`` is zero-dimensional and a view - otherwise. On the other hand, ``x[...]`` always returns a view. -* If a zero-dimensional array is present in the index *and* it is a full - integer index the result will be a *scalar* and not a zero-dimensional array. - (Advanced indexing is not triggered.) -* When an ellipsis (``...``) is present but has no size (i.e. replaces zero - ``:``) the result will still always be an array. A view if no advanced index - is present, otherwise a copy. -* The ``nonzero`` equivalence for Boolean arrays does not hold for zero - dimensional boolean arrays. -* When the result of an advanced indexing operation has no elements but an - individual index is out of bounds, whether or not an ``IndexError`` is - raised is undefined (e.g. ``x[[], [123]]`` with ``123`` being out of bounds). -* When a *casting* error occurs during assignment (for example updating a - numerical array using a sequence of strings), the array being assigned - to may end up in an unpredictable partially updated state. - However, if any other error (such as an out of bounds index) occurs, the - array will remain unchanged. -* The memory layout of an advanced indexing result is optimized for each - indexing operation and no particular memory order can be assumed. -* When using a subclass (especially one which manipulates its shape), the - default ``ndarray.__setitem__`` behaviour will call ``__getitem__`` for - *basic* indexing but not for *advanced* indexing. For such a subclass it may - be preferable to call ``ndarray.__setitem__`` with a *base class* ndarray - view on the data. This *must* be done if the subclasses ``__getitem__`` does - not return views. \ No newline at end of file +:ref:`arrays.indexing.fields`, :ref:`flat-iterator-indexing` are a couple +of other indexing options. \ No newline at end of file diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index 66d35abee..78183d945 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -6,61 +6,684 @@ Indexing basics .. seealso:: - :ref:`Indexing ` + :ref:`arrays.indexing` :ref:`Indexing routines ` -Array indexing refers to any use of the square brackets ([]) to index -array values. There are many options to indexing, which give NumPy -indexing great power, but with power comes some complexity and the -potential for confusion. This section is just an overview of the -various options and issues related to indexing. Further details can be -found in the relevant sections of :ref:`arrays.indexing`. +.. sectionauthor:: adapted from "Guide to NumPy" by Travis E. Oliphant +.. currentmodule:: numpy -Basic indexing -============== +.. index:: indexing, slicing -Basic indexing will be triggered through single element indexing or -slicing and striding. +:class:`ndarrays ` can be indexed using the standard Python +``x[obj]`` syntax, where *x* is the array and *obj* the selection. +There are four kinds of indexing available depending on *obj*: +single element indexing, basic slicing, advanced indexing and field access. -Single element indexing works -exactly like that for other standard Python sequences but has been expanded -to support multidimensional indexing -for multidimensional NumPy arrays. For more details and examples, see -:ref:`single-element-indexing`. - -It is possible to slice and stride arrays to extract arrays of the -same number of dimensions, but of different sizes than the original. -The slicing and striding work exactly the same way it does for lists -and tuples except that they can be applied to multiple dimensions as -well. See :ref:`basic-slicing-and-indexing` for more information and examples. +Most of the following examples show the use of indexing when +referencing data in an array. The examples work just as well +when assigning to an array. See :ref:`assigning-values-to-indexed-arrays` for +specific examples and explanations on how assignments work. +Note that in Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to +``x[exp1, exp2, ..., expN]``; the latter is just syntactic sugar +for the former. -Advanced indexing -================= +.. _single-element-indexing: -Arrays can be indexed with other arrays to -select lists of values out of arrays into new arrays. There are -two different ways of accomplishing this. One uses one or more arrays -of index values. The other involves giving a boolean array of the proper -shape to indicate the values to be selected. Index arrays are a very -powerful tool that allows one to avoid looping over individual elements in -arrays and thus greatly improve performance. See :ref:`advanced-indexing` -for more information and examples. +Single element indexing +----------------------- +Single element indexing works +exactly like that for other standard Python sequences. It is 0-based, +and accepts negative indices for indexing from the end of the array. :: + + >>> x = np.arange(10) + >>> x[2] + 2 + >>> x[-2] + 8 + +It is not necessary to +separate each dimension's index into its own set of square brackets. :: + + >>> x.shape = (2,5) # now x is 2-dimensional + >>> x[1,3] + 8 + >>> x[1,-1] + 9 + +Note that if one indexes a multidimensional array with fewer indices +than dimensions, one gets a subdimensional array. For example: :: + + >>> x[0] + array([0, 1, 2, 3, 4]) + +That is, each index specified selects the array corresponding to the +rest of the dimensions selected. In the above example, choosing 0 +means that the remaining dimension of length 5 is being left unspecified, +and that what is returned is an array of that dimensionality and size. +It must be noted that the returned array is not a copy of the original, +but points to the same values in memory as does the original array. +In this case, the 1-D array at the first position (0) is returned. +So using a single index on the returned array, results in a single +element being returned. That is: :: + + >>> x[0][2] + 2 + +So note that ``x[0,2] = x[0][2]`` though the second case is more +inefficient as a new temporary array is created after the first index +that is subsequently indexed by 2. + +.. note:: + + NumPy uses C-order indexing. That means that the last + index usually represents the most rapidly changing memory location, + unlike Fortran or IDL, where the first index represents the most + rapidly changing location in memory. This difference represents a + great potential for confusion. + +.. _basic-slicing-and-indexing: + +Basic slicing and indexing +-------------------------- + +Basic slicing extends Python's basic concept of slicing to N +dimensions. Basic slicing occurs when *obj* is a :class:`slice` object +(constructed by ``start:stop:step`` notation inside of brackets), an +integer, or a tuple of slice objects and integers. :py:data:`Ellipsis` +and :const:`newaxis` objects can be interspersed with these as +well. + +.. deprecated:: 1.15.0 + + In order to remain backward compatible with a common usage in + Numeric, basic slicing is also initiated if the selection object is + any non-ndarray and non-tuple sequence (such as a :class:`list`) containing + :class:`slice` objects, the :py:data:`Ellipsis` object, or the :const:`newaxis` + object, but not for integer arrays or other embedded sequences. + +.. index:: + triple: ndarray; special methods; getitem + triple: ndarray; special methods; setitem + single: ellipsis + single: newaxis + +The simplest case of indexing with *N* integers returns an :ref:`array +scalar ` representing the corresponding item. As in +Python, all indices are zero-based: for the *i*-th index :math:`n_i`, +the valid range is :math:`0 \le n_i < d_i` where :math:`d_i` is the +*i*-th element of the shape of the array. Negative indices are +interpreted as counting from the end of the array (*i.e.*, if +:math:`n_i < 0`, it means :math:`n_i + d_i`). + + +All arrays generated by basic slicing are always :term:`views ` +of the original array. + +.. note:: + + NumPy slicing creates a :term:`view` instead of a copy as in the case of + built-in Python sequences such as string, tuple and list. + Care must be taken when extracting + a small portion from a large array which becomes useless after the + extraction, because the small portion extracted contains a reference + to the large original array whose memory will not be released until + all arrays derived from it are garbage-collected. In such cases an + explicit ``copy()`` is recommended. + +The standard rules of sequence slicing apply to basic slicing on a +per-dimension basis (including using a step index). Some useful +concepts to remember include: + +- The basic slice syntax is ``i:j:k`` where *i* is the starting index, + *j* is the stopping index, and *k* is the step (:math:`k\neq0`). + This selects the *m* elements (in the corresponding dimension) with + index values *i*, *i + k*, ..., *i + (m - 1) k* where + :math:`m = q + (r\neq0)` and *q* and *r* are the quotient and remainder + obtained by dividing *j - i* by *k*: *j - i = q k + r*, so that + *i + (m - 1) k < j*. + For example:: + + >>> x = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) + >>> x[1:7:2] + array([1, 3, 5]) + +- Negative *i* and *j* are interpreted as *n + i* and *n + j* where + *n* is the number of elements in the corresponding dimension. + Negative *k* makes stepping go towards smaller indices. + From the above example:: + + >>> x[-2:10] + array([8, 9]) + >>> x[-3:3:-1] + array([7, 6, 5, 4]) + +- Assume *n* is the number of elements in the dimension being + sliced. Then, if *i* is not given it defaults to 0 for *k > 0* and + *n - 1* for *k < 0* . If *j* is not given it defaults to *n* for *k > 0* + and *-n-1* for *k < 0* . If *k* is not given it defaults to 1. Note that + ``::`` is the same as ``:`` and means select all indices along this + axis. + From the above example:: + + >>> x[5:] + array([5, 6, 7, 8, 9]) + +- If the number of objects in the selection tuple is less than + *N*, then ``:`` is assumed for any subsequent dimensions. + For example:: + + >>> x = np.array([[[1],[2],[3]], [[4],[5],[6]]]) + >>> x.shape + (2, 3, 1) + >>> x[1:2] + array([[[4], + [5], + [6]]]) + +- An integer, *i*, returns the same values as ``i:i+1`` + **except** the dimensionality of the returned object is reduced by + 1. In particular, a selection tuple with the *p*-th + element an integer (and all other entries ``:``) returns the + corresponding sub-array with dimension *N - 1*. If *N = 1* + then the returned object is an array scalar. These objects are + explained in :ref:`arrays.scalars`. + +- If the selection tuple has all entries ``:`` except the + *p*-th entry which is a slice object ``i:j:k``, + then the returned array has dimension *N* formed by + concatenating the sub-arrays returned by integer indexing of + elements *i*, *i+k*, ..., *i + (m - 1) k < j*, + +- Basic slicing with more than one non-``:`` entry in the slicing + tuple, acts like repeated application of slicing using a single + non-``:`` entry, where the non-``:`` entries are successively taken + (with all other non-``:`` entries replaced by ``:``). Thus, + ``x[ind1,...,ind2,:]`` acts like ``x[ind1][...,ind2,:]`` under basic + slicing. + + .. warning:: The above is **not** true for advanced indexing. + +- You may use slicing to set values in the array, but (unlike lists) you + can never grow the array. The size of the value to be set in + ``x[obj] = value`` must be (broadcastable) to the same shape as + ``x[obj]``. + +- A slicing tuple can always be constructed as *obj* + and used in the ``x[obj]`` notation. Slice objects can be used in + the construction in place of the ``[start:stop:step]`` + notation. For example, ``x[1:10:5,::-1]`` can also be implemented + as ``obj = (slice(1,10,5), slice(None,None,-1)); x[obj]`` . This + can be useful for constructing generic code that works on arrays + of arbitrary dimensions. See :ref:`dealing-with-variable-indices` + for more information. + +.. index:: + pair: ndarray; view + +Structural indexing tools +^^^^^^^^^^^^^^^^^^^^^^^^^ +There are some tools to facilitate the easy matching of array shapes with +expressions and in assignments. + +:py:data:`Ellipsis` expands to the number of ``:`` objects needed for the +selection tuple to index all dimensions. In most cases, this means that the +length of the expanded selection tuple is ``x.ndim``. There may only be a +single ellipsis present. +From the above example:: + + >>> x[...,0] + array([[1, 2, 3], + [4, 5, 6]]) + +This is equivalent to:: + + >>> x[:,:,0] + array([[1, 2, 3], + [4, 5, 6]]) + +Each :const:`newaxis` object in the selection tuple serves to expand +the dimensions of the resulting selection by one unit-length +dimension. The added dimension is the position of the :const:`newaxis` +object in the selection tuple. :const:`newaxis` is an alias for +'None', and 'None' can be used in place of this with the same result. +From the above example:: + + >>> x[:,np.newaxis,:,:].shape + (2, 1, 3, 1) + >>> x[:,None,:,:].shape + (2, 1, 3, 1) + +This can be handy to combine two +arrays in a way that otherwise would require explicitly reshaping +operations. For example:: + + >>> x = np.arange(5) + >>> x[:,np.newaxis] + x[np.newaxis,:] + array([[0, 1, 2, 3, 4], + [1, 2, 3, 4, 5], + [2, 3, 4, 5, 6], + [3, 4, 5, 6, 7], + [4, 5, 6, 7, 8]]) + + +.. _advanced-indexing: -Other indexing options -====================== +Advanced indexing +----------------- + +.. seealso:: :ref:`basics.broadcasting` + +Advanced indexing is triggered when the selection object, *obj*, is a +non-tuple sequence object, an :class:`ndarray` (of data type integer or bool), +or a tuple with at least one sequence object or ndarray (of data type +integer or bool). There are two types of advanced indexing: integer +and Boolean. + +Advanced indexing always returns a *copy* of the data (contrast with +basic slicing that returns a :term:`view`). + +.. warning:: + + The definition of advanced indexing means that ``x[(1,2,3),]`` is + fundamentally different than ``x[(1,2,3)]``. The latter is + equivalent to ``x[1,2,3]`` which will trigger basic selection while + the former will trigger advanced indexing. Be sure to understand + why this occurs. + + Also recognize that ``x[[1,2,3]]`` will trigger advanced indexing, + whereas due to the deprecated Numeric compatibility mentioned above, + ``x[[1,2,slice(None)]]`` will trigger basic slicing. + +Integer array indexing +^^^^^^^^^^^^^^^^^^^^^^ + +Integer array indexing allows selection of arbitrary items in the array +based on their *N*-dimensional index. Each integer array represents a number +of indices into that dimension. + +Negative values are permitted in the index arrays and work as they do with +single indices or slices. If the index values are out of bounds then an +``IndexError`` is thrown:: + + >>> x = np.array([[1, 2], [3, 4], [5, 6]]) + >>> x[np.array([1, -1])] + array([[3, 4], + [5, 6]]) + >>> x[np.array([3, 4])] + IndexError: index 3 is out of bounds for axis 0 with size 3 -:ref:`arrays.indexing.fields`, :ref:`flat-iterator-indexing` are a couple -of other indexing options. + +When the index consists of as many integer arrays as dimensions of the array +being indexed, the indexing is straightforward, but different from slicing. + +Advanced indices always are :ref:`broadcast` and +iterated as *one*:: + + result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M], + ..., ind_N[i_1, ..., i_M]] + +Note that the resulting shape is identical to the (broadcast) indexing array +shapes ``ind_1, ..., ind_N``. If the indices cannot be broadcast to the +same shape, an exception ``IndexError: shape mismatch: indexing arrays could +not be broadcast together with shapes...`` is raised. + +.. rubric:: Example + +From each row, a specific element should be selected. The row index is just +``[0, 1, 2]`` and the column index specifies the element to choose for the +corresponding row, here ``[0, 1, 0]``. Using both together the task +can be solved using advanced indexing:: + + >>> x = np.array([[1, 2], [3, 4], [5, 6]]) + >>> x[[0, 1, 2], [0, 1, 0]] + array([1, 4, 5]) + +To achieve a behaviour similar to the basic slicing above, broadcasting can be +used. The function :func:`ix_` can help with this broadcasting. This is best +understood with an example. + +.. rubric:: Example + +From a 4x3 array the corner elements should be selected using advanced +indexing. Thus all elements for which the column is one of ``[0, 2]`` and +the row is one of ``[0, 3]`` need to be selected. To use advanced indexing +one needs to select all elements *explicitly*. Using the method explained +previously one could write:: + + >>> x = np.array([[ 0, 1, 2], + ... [ 3, 4, 5], + ... [ 6, 7, 8], + ... [ 9, 10, 11]]) + >>> rows = np.array([[0, 0], + ... [3, 3]], dtype=np.intp) + >>> columns = np.array([[0, 2], + ... [0, 2]], dtype=np.intp) + >>> x[rows, columns] + array([[ 0, 2], + [ 9, 11]]) + +However, since the indexing arrays above just repeat themselves, +broadcasting can be used (compare operations such as +``rows[:, np.newaxis] + columns``) to simplify this:: + + >>> rows = np.array([0, 3], dtype=np.intp) + >>> columns = np.array([0, 2], dtype=np.intp) + >>> rows[:, np.newaxis] + array([[0], + [3]]) + >>> x[rows[:, np.newaxis], columns] + array([[ 0, 2], + [ 9, 11]]) + +This broadcasting can also be achieved using the function :func:`ix_`: + + >>> x[np.ix_(rows, columns)] + array([[ 0, 2], + [ 9, 11]]) + +Note that without the ``np.ix_`` call, only the diagonal elements would +be selected, as was used in the previous example. This difference is the +most important thing to remember about indexing with multiple advanced +indices. + +.. rubric:: Example + +The broadcasting mechanism permits index arrays to be combined with +scalars for other indices. The effect is that the scalar value is used +for all the corresponding values of the index arrays:: + + >>> x = np.arange(35).reshape(5,7) + >>> x[np.array([0,2,4]), 1] + array([ 1, 15, 29]) + +.. rubric:: Example + +A real-life example of where advanced indexing may be useful is for a color +lookup table where we want to map the values of an image into RGB triples for +display. The lookup table could have a shape (nlookup, 3). Indexing +such an array with an image with shape (ny, nx) with dtype=np.uint8 +(or any integer type so long as values are with the bounds of the +lookup table) will result in an array of shape (ny, nx, 3) where a +triple of RGB values is associated with each pixel location. + + +Boolean array indexing +^^^^^^^^^^^^^^^^^^^^^^ + +This advanced indexing occurs when *obj* is an array object of Boolean +type, such as may be returned from comparison operators. A single +boolean index array is practically identical to ``x[obj.nonzero()]`` where, +as described above, :meth:`obj.nonzero() ` returns a +tuple (of length :attr:`obj.ndim `) of integer index +arrays showing the :py:data:`True` elements of *obj*. However, it is +faster when ``obj.shape == x.shape``. + +If ``obj.ndim == x.ndim``, ``x[obj]`` returns a 1-dimensional array +filled with the elements of *x* corresponding to the :py:data:`True` +values of *obj*. The search order will be :term:`row-major`, +C-style. If *obj* has :py:data:`True` values at entries that are outside +of the bounds of *x*, then an index error will be raised. If *obj* is +smaller than *x* it is identical to filling it with :py:data:`False`. + +A common use case for this is filtering for desired element values. +For example, one may wish to select all entries from an array which +are not NaN:: + + >>> x = np.array([[1., 2.], [np.nan, 3.], [np.nan, np.nan]]) + >>> x[~np.isnan(x)] + array([1., 2., 3.]) + +Or wish to add a constant to all negative elements:: + + >>> x = np.array([1., -1., -2., 3]) + >>> x[x < 0] += 20 + >>> x + array([ 1., 19., 18., 3.]) + +In general if an index includes a Boolean array, the result will be +identical to inserting ``obj.nonzero()`` into the same position +and using the integer array indexing mechanism described above. +``x[ind_1, boolean_array, ind_2]`` is equivalent to +``x[(ind_1,) + boolean_array.nonzero() + (ind_2,)]``. + +If there is only one Boolean array and no integer indexing array present, +this is straight forward. Care must only be taken to make sure that the +boolean index has *exactly* as many dimensions as it is supposed to work +with. + +.. rubric:: Example + +From an array, select all rows which sum up to less or equal two:: + + >>> x = np.array([[0, 1], [1, 1], [2, 2]]) + >>> rowsum = x.sum(-1) + >>> x[rowsum <= 2, :] + array([[0, 1], + [1, 1]]) + + +Combining multiple Boolean indexing arrays or a Boolean with an integer +indexing array can best be understood with the +:meth:`obj.nonzero() ` analogy. The function :func:`ix_` +also supports boolean arrays and will work without any surprises. + +.. rubric:: Example + +Use boolean indexing to select all rows adding up to an even +number. At the same time columns 0 and 2 should be selected with an +advanced integer index. Using the :func:`ix_` function this can be done +with:: + + >>> x = np.array([[ 0, 1, 2], + ... [ 3, 4, 5], + ... [ 6, 7, 8], + ... [ 9, 10, 11]]) + >>> rows = (x.sum(-1) % 2) == 0 + >>> rows + array([False, True, False, True]) + >>> columns = [0, 2] + >>> x[np.ix_(rows, columns)] + array([[ 3, 5], + [ 9, 11]]) + +Without the ``np.ix_`` call, only the diagonal elements would be +selected. + +Or without ``np.ix_`` (compare the integer array examples):: + + >>> rows = rows.nonzero()[0] + >>> x[rows[:, np.newaxis], columns] + array([[ 3, 5], + [ 9, 11]]) + +.. rubric:: Example + +If x has more dimensions than b then the result will be multi-dimensional:: + + >>> x = np.arange(35).reshape(5,7) + >>> b = x>20 + >>> b[:,5] + array([False, False, False, True, True]) + >>> x[b[:,5]] + array([[21, 22, 23, 24, 25, 26, 27], + [28, 29, 30, 31, 32, 33, 34]]) + +Here the 4th and 5th rows are selected from the indexed array and +combined to make a 2-D array. + +.. rubric:: Example + +Using a 2-D boolean array of shape (2,3) +with four True elements to select rows from a 3-D array of shape +(2,3,5) results in a 2-D result of shape (4,5):: + + >>> x = np.arange(30).reshape(2,3,5) + >>> x + array([[[ 0, 1, 2, 3, 4], + [ 5, 6, 7, 8, 9], + [10, 11, 12, 13, 14]], + [[15, 16, 17, 18, 19], + [20, 21, 22, 23, 24], + [25, 26, 27, 28, 29]]]) + >>> b = np.array([[True, True, False], [False, True, True]]) + >>> x[b] + array([[ 0, 1, 2, 3, 4], + [ 5, 6, 7, 8, 9], + [20, 21, 22, 23, 24], + [25, 26, 27, 28, 29]]) + + +.. _combining-advanced-and-basic-indexing: + +Combining advanced and basic indexing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When there is at least one slice (``:``), ellipsis (``...``) or :const:`newaxis` +in the index (or the array has more dimensions than there are advanced indices), +then the behaviour can be more complicated. It is like concatenating the +indexing result for each advanced index element. + +In the simplest case, there is only a *single* advanced index. A single +advanced index can for example replace a slice and the result array will be +the same, however, it is a copy and may have a different memory layout. +A slice is preferable when it is possible. +For example:: + + >>> x = np.array([[ 0, 1, 2], + ... [ 3, 4, 5], + ... [ 6, 7, 8], + ... [ 9, 10, 11]]) + >>> x[1:2, 1:3] + array([[4, 5]]) + >>> x[1:2, [1, 2]] + array([[4, 5]]) + +The easiest way to understand the situation may be to think in +terms of the resulting shape. There are two parts to the indexing operation, +the subspace defined by the basic indexing (excluding integers) and the +subspace from the advanced indexing part. Two cases of index combination +need to be distinguished: + +* The advanced indices are separated by a slice, :py:data:`Ellipsis` or + :const:`newaxis`. For example ``x[arr1, :, arr2]``. +* The advanced indices are all next to each other. + For example ``x[..., arr1, arr2, :]`` but *not* ``x[arr1, :, 1]`` + since ``1`` is an advanced index in this regard. + +In the first case, the dimensions resulting from the advanced indexing +operation come first in the result array, and the subspace dimensions after +that. +In the second case, the dimensions from the advanced indexing operations +are inserted into the result array at the same spot as they were in the +initial array (the latter logic is what makes simple advanced indexing +behave just like slicing). + +.. rubric:: Example + +Suppose ``x.shape`` is (10,20,30) and ``ind`` is a (2,3,4)-shaped +indexing :class:`intp` array, then ``result = x[...,ind,:]`` has +shape (10,2,3,4,30) because the (20,)-shaped subspace has been +replaced with a (2,3,4)-shaped broadcasted indexing subspace. If +we let *i, j, k* loop over the (2,3,4)-shaped subspace then +``result[...,i,j,k,:] = x[...,ind[i,j,k],:]``. This example +produces the same result as :meth:`x.take(ind, axis=-2) `. + +.. rubric:: Example + +Let ``x.shape`` be (10,20,30,40,50) and suppose ``ind_1`` +and ``ind_2`` can be broadcast to the shape (2,3,4). Then +``x[:,ind_1,ind_2]`` has shape (10,2,3,4,40,50) because the +(20,30)-shaped subspace from X has been replaced with the +(2,3,4) subspace from the indices. However, +``x[:,ind_1,:,ind_2]`` has shape (2,3,4,10,30,50) because there +is no unambiguous place to drop in the indexing subspace, thus +it is tacked-on to the beginning. It is always possible to use +:meth:`.transpose() ` to move the subspace +anywhere desired. Note that this example cannot be replicated +using :func:`take`. + +.. rubric:: Example + +Slicing can be combined with broadcasted boolean indices:: + + >>> x = np.arange(35).reshape(5,7) + >>> b = x > 20 + >>> b + array([[False, False, False, False, False, False, False], + [False, False, False, False, False, False, False], + [False, False, False, False, False, False, False], + [ True, True, True, True, True, True, True], + [ True, True, True, True, True, True, True]]) + >>> x[b[:,5],1:3] + array([[22, 23], + [29, 30]]) + + +.. _arrays.indexing.fields: + +Field access +------------- + +.. seealso:: :ref:`arrays.dtypes`, :ref:`arrays.scalars`, + :ref:`structured_arrays` + +If the :class:`ndarray` object is a structured array the :term:`fields ` +of the array can be accessed by indexing the array with strings, +dictionary-like. + +Indexing ``x['field-name']`` returns a new :term:`view` to the array, +which is of the same shape as *x* (except when the field is a +sub-array) but of data type ``x.dtype['field-name']`` and contains +only the part of the data in the specified field. Also, +:ref:`record array ` scalars can be "indexed" this way. + +Indexing into a structured array can also be done with a list of field names, +e.g. ``x[['field-name1','field-name2']]``. As of NumPy 1.16, this returns a +view containing only those fields. In older versions of NumPy, it returned a +copy. See the user guide section on :ref:`structured_arrays` for more +information on multifield indexing. + +If the accessed field is a sub-array, the dimensions of the sub-array +are appended to the shape of the result. +For example:: + + >>> x = np.zeros((2,2), dtype=[('a', np.int32), ('b', np.float64, (3,3))]) + >>> x['a'].shape + (2, 2) + >>> x['a'].dtype + dtype('int32') + >>> x['b'].shape + (2, 2, 3, 3) + >>> x['b'].dtype + dtype('float64') + +.. _flat-iterator-indexing: + +Flat Iterator indexing +---------------------- + +:attr:`x.flat ` returns an iterator that will iterate +over the entire array (in C-contiguous style with the last index +varying the fastest). This iterator object can also be indexed using +basic slicing or advanced indexing as long as the selection object is +not a tuple. This should be clear from the fact that :attr:`x.flat +` is a 1-dimensional view. It can be used for integer +indexing with 1-dimensional C-style-flat indices. The shape of any +returned array is therefore the shape of the integer indexing object. + +.. index:: + single: indexing + single: ndarray .. _assigning-values-to-indexed-arrays: Assigning values to indexed arrays -================================== +---------------------------------- As mentioned, one can select a subset of an array to assign to using a single index, slices, and index and mask arrays. The value being @@ -110,7 +733,7 @@ rather than being incremented 3 times. .. _dealing-with-variable-indices: Dealing with variable numbers of indices within programs -======================================================== +-------------------------------------------------------- The indexing syntax is very powerful but limiting when dealing with a variable number of indices. For example, if you want to write @@ -158,3 +781,43 @@ converted to an array as a list would be. As an example: :: 40 +Detailed notes +-------------- + +These are some detailed notes, which are not of importance for day to day +indexing (in no particular order): + +* The native NumPy indexing type is ``intp`` and may differ from the + default integer array type. ``intp`` is the smallest data type + sufficient to safely index any array; for advanced indexing it may be + faster than other types. +* For advanced assignments, there is in general no guarantee for the + iteration order. This means that if an element is set more than once, + it is not possible to predict the final result. +* An empty (tuple) index is a full scalar index into a zero-dimensional array. + ``x[()]`` returns a *scalar* if ``x`` is zero-dimensional and a view + otherwise. On the other hand, ``x[...]`` always returns a view. +* If a zero-dimensional array is present in the index *and* it is a full + integer index the result will be a *scalar* and not a zero-dimensional array. + (Advanced indexing is not triggered.) +* When an ellipsis (``...``) is present but has no size (i.e. replaces zero + ``:``) the result will still always be an array. A view if no advanced index + is present, otherwise a copy. +* The ``nonzero`` equivalence for Boolean arrays does not hold for zero + dimensional boolean arrays. +* When the result of an advanced indexing operation has no elements but an + individual index is out of bounds, whether or not an ``IndexError`` is + raised is undefined (e.g. ``x[[], [123]]`` with ``123`` being out of bounds). +* When a *casting* error occurs during assignment (for example updating a + numerical array using a sequence of strings), the array being assigned + to may end up in an unpredictable partially updated state. + However, if any other error (such as an out of bounds index) occurs, the + array will remain unchanged. +* The memory layout of an advanced indexing result is optimized for each + indexing operation and no particular memory order can be assumed. +* When using a subclass (especially one which manipulates its shape), the + default ``ndarray.__setitem__`` behaviour will call ``__getitem__`` for + *basic* indexing but not for *advanced* indexing. For such a subclass it may + be preferable to call ``ndarray.__setitem__`` with a *base class* ndarray + view on the data. This *must* be done if the subclasses ``__getitem__`` does + not return views. -- cgit v1.2.1 From a3c72d61587e5896f39c86d4696a7cd449c3ca7b Mon Sep 17 00:00:00 2001 From: Mukulika Date: Wed, 18 Aug 2021 19:53:47 +0530 Subject: DOC: Replaced indexing stub doc with routines --- doc/source/reference/arrays.indexing.rst | 122 ++++++++++++++++------------- doc/source/reference/routines.indexing.rst | 69 ---------------- doc/source/reference/routines.rst | 1 - doc/source/user/basics.indexing.rst | 8 +- 4 files changed, 69 insertions(+), 131 deletions(-) delete mode 100644 doc/source/reference/routines.indexing.rst (limited to 'doc/source') diff --git a/doc/source/reference/arrays.indexing.rst b/doc/source/reference/arrays.indexing.rst index cba028baa..100d22e02 100644 --- a/doc/source/reference/arrays.indexing.rst +++ b/doc/source/reference/arrays.indexing.rst @@ -1,60 +1,70 @@ -.. for doctests - >>> import numpy as np - +.. _routines.indexing: .. _arrays.indexing: -******** -Indexing -******** - -.. seealso:: - - :ref:`basics.indexing` - - :ref:`Indexing routines ` - -Array indexing refers to any use of the square brackets ([]) to index -array values. There are many options to indexing, which give NumPy -indexing great power, but with power comes some complexity and the -potential for confusion. This section is just an overview of the -various options and issues related to indexing. Further details can be -found in the relevant sections of :ref:`basics.indexing`. - - -Basic indexing -============== - -Basic indexing will be triggered through single element indexing or -slicing and striding. - -Single element indexing works -exactly like that for other standard Python sequences but has been expanded -to support multidimensional indexing -for multidimensional NumPy arrays. For more details and examples, see -:ref:`single-element-indexing`. - -It is possible to slice and stride arrays to extract arrays of the -same number of dimensions, but of different sizes than the original. -The slicing and striding work exactly the same way it does for lists -and tuples except that they can be applied to multiple dimensions as -well. See :ref:`basic-slicing-and-indexing` for more information and examples. - - -Advanced indexing +Indexing routines ================= -Arrays can be indexed with other arrays to -select lists of values out of arrays into new arrays. There are -two different ways of accomplishing this. One uses one or more arrays -of index values. The other involves giving a boolean array of the proper -shape to indicate the values to be selected. Index arrays are a very -powerful tool that allows one to avoid looping over individual elements in -arrays and thus greatly improve performance. See :ref:`advanced-indexing` -for more information and examples. - - -Other indexing options -====================== - -:ref:`arrays.indexing.fields`, :ref:`flat-iterator-indexing` are a couple -of other indexing options. \ No newline at end of file +.. seealso:: :ref:`basics.indexing` + +.. currentmodule:: numpy + +Generating index arrays +----------------------- +.. autosummary:: + :toctree: generated/ + + c_ + r_ + s_ + nonzero + where + indices + ix_ + ogrid + ravel_multi_index + unravel_index + diag_indices + diag_indices_from + mask_indices + tril_indices + tril_indices_from + triu_indices + triu_indices_from + +Indexing-like operations +------------------------ +.. autosummary:: + :toctree: generated/ + + take + take_along_axis + choose + compress + diag + diagonal + select + lib.stride_tricks.sliding_window_view + lib.stride_tricks.as_strided + +Inserting data into arrays +-------------------------- +.. autosummary:: + :toctree: generated/ + + place + put + put_along_axis + putmask + fill_diagonal + +Iterating over arrays +--------------------- +.. autosummary:: + :toctree: generated/ + + nditer + ndenumerate + ndindex + nested_iters + flatiter + lib.Arrayterator diff --git a/doc/source/reference/routines.indexing.rst b/doc/source/reference/routines.indexing.rst deleted file mode 100644 index eebbf4989..000000000 --- a/doc/source/reference/routines.indexing.rst +++ /dev/null @@ -1,69 +0,0 @@ -.. _routines.indexing: - -Indexing routines -================= - -.. seealso:: :ref:`Indexing ` - -.. currentmodule:: numpy - -Generating index arrays ------------------------ -.. autosummary:: - :toctree: generated/ - - c_ - r_ - s_ - nonzero - where - indices - ix_ - ogrid - ravel_multi_index - unravel_index - diag_indices - diag_indices_from - mask_indices - tril_indices - tril_indices_from - triu_indices - triu_indices_from - -Indexing-like operations ------------------------- -.. autosummary:: - :toctree: generated/ - - take - take_along_axis - choose - compress - diag - diagonal - select - lib.stride_tricks.sliding_window_view - lib.stride_tricks.as_strided - -Inserting data into arrays --------------------------- -.. autosummary:: - :toctree: generated/ - - place - put - put_along_axis - putmask - fill_diagonal - -Iterating over arrays ---------------------- -.. autosummary:: - :toctree: generated/ - - nditer - ndenumerate - ndindex - nested_iters - flatiter - lib.Arrayterator diff --git a/doc/source/reference/routines.rst b/doc/source/reference/routines.rst index 5d6a823b7..593d017cc 100644 --- a/doc/source/reference/routines.rst +++ b/doc/source/reference/routines.rst @@ -30,7 +30,6 @@ indentation. routines.fft routines.functional routines.help - routines.indexing routines.io routines.linalg routines.logic diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index 78183d945..4f1226f90 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -1,13 +1,11 @@ .. _basics.indexing: -*************** -Indexing basics -*************** +**************************************** +Indexing on :class:`ndarrays <.ndarray>` +**************************************** .. seealso:: - :ref:`arrays.indexing` - :ref:`Indexing routines ` .. sectionauthor:: adapted from "Guide to NumPy" by Travis E. Oliphant -- cgit v1.2.1 From 2f3a99907cbe78fccc72fe80e83408ea8bde84ec Mon Sep 17 00:00:00 2001 From: Mukulika Date: Fri, 20 Aug 2021 11:54:15 +0530 Subject: DOC: Reformatted examples --- doc/source/user/basics.indexing.rst | 103 ++++++++++++++++++------------------ 1 file changed, 52 insertions(+), 51 deletions(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index 4f1226f90..79108bb20 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -46,10 +46,10 @@ and accepts negative indices for indexing from the end of the array. :: It is not necessary to separate each dimension's index into its own set of square brackets. :: - >>> x.shape = (2,5) # now x is 2-dimensional - >>> x[1,3] + >>> x.shape = (2, 5) # now x is 2-dimensional + >>> x[1, 3] 8 - >>> x[1,-1] + >>> x[1, -1] 9 Note that if one indexes a multidimensional array with fewer indices @@ -62,8 +62,9 @@ That is, each index specified selects the array corresponding to the rest of the dimensions selected. In the above example, choosing 0 means that the remaining dimension of length 5 is being left unspecified, and that what is returned is an array of that dimensionality and size. -It must be noted that the returned array is not a copy of the original, -but points to the same values in memory as does the original array. +It must be noted that the returned array is a :term:`view`, i.e., it is not a +copy of the original, but points to the same values in memory as does the +original array. In this case, the 1-D array at the first position (0) is returned. So using a single index on the returned array, results in a single element being returned. That is: :: @@ -71,7 +72,7 @@ element being returned. That is: :: >>> x[0][2] 2 -So note that ``x[0,2] = x[0][2]`` though the second case is more +So note that ``x[0, 2] = x[0][2]`` though the second case is more inefficient as a new temporary array is created after the first index that is subsequently indexed by 2. @@ -200,7 +201,7 @@ concepts to remember include: tuple, acts like repeated application of slicing using a single non-``:`` entry, where the non-``:`` entries are successively taken (with all other non-``:`` entries replaced by ``:``). Thus, - ``x[ind1,...,ind2,:]`` acts like ``x[ind1][...,ind2,:]`` under basic + ``x[ind1, ..., ind2,:]`` acts like ``x[ind1][..., ind2, :]`` under basic slicing. .. warning:: The above is **not** true for advanced indexing. @@ -213,8 +214,8 @@ concepts to remember include: - A slicing tuple can always be constructed as *obj* and used in the ``x[obj]`` notation. Slice objects can be used in the construction in place of the ``[start:stop:step]`` - notation. For example, ``x[1:10:5,::-1]`` can also be implemented - as ``obj = (slice(1,10,5), slice(None,None,-1)); x[obj]`` . This + notation. For example, ``x[1:10:5, ::-1]`` can also be implemented + as ``obj = (slice(1, 10, 5), slice(None, None, -1)); x[obj]`` . This can be useful for constructing generic code that works on arrays of arbitrary dimensions. See :ref:`dealing-with-variable-indices` for more information. @@ -233,13 +234,13 @@ length of the expanded selection tuple is ``x.ndim``. There may only be a single ellipsis present. From the above example:: - >>> x[...,0] + >>> x[..., 0] array([[1, 2, 3], [4, 5, 6]]) This is equivalent to:: - >>> x[:,:,0] + >>> x[:, :, 0] array([[1, 2, 3], [4, 5, 6]]) @@ -250,9 +251,9 @@ object in the selection tuple. :const:`newaxis` is an alias for 'None', and 'None' can be used in place of this with the same result. From the above example:: - >>> x[:,np.newaxis,:,:].shape + >>> x[:, np.newaxis, :, :].shape (2, 1, 3, 1) - >>> x[:,None,:,:].shape + >>> x[:, None, :, :].shape (2, 1, 3, 1) This can be handy to combine two @@ -260,7 +261,7 @@ arrays in a way that otherwise would require explicitly reshaping operations. For example:: >>> x = np.arange(5) - >>> x[:,np.newaxis] + x[np.newaxis,:] + >>> x[:, np.newaxis] + x[np.newaxis, :] array([[0, 1, 2, 3, 4], [1, 2, 3, 4, 5], [2, 3, 4, 5, 6], @@ -286,15 +287,15 @@ basic slicing that returns a :term:`view`). .. warning:: - The definition of advanced indexing means that ``x[(1,2,3),]`` is - fundamentally different than ``x[(1,2,3)]``. The latter is - equivalent to ``x[1,2,3]`` which will trigger basic selection while + The definition of advanced indexing means that ``x[(1, 2, 3),]`` is + fundamentally different than ``x[(1, 2, 3)]``. The latter is + equivalent to ``x[1, 2, 3]`` which will trigger basic selection while the former will trigger advanced indexing. Be sure to understand why this occurs. - Also recognize that ``x[[1,2,3]]`` will trigger advanced indexing, + Also recognize that ``x[[1, 2, 3]]`` will trigger advanced indexing, whereas due to the deprecated Numeric compatibility mentioned above, - ``x[[1,2,slice(None)]]`` will trigger basic slicing. + ``x[[1, 2, slice(None)]]`` will trigger basic slicing. Integer array indexing ^^^^^^^^^^^^^^^^^^^^^^ @@ -394,8 +395,8 @@ The broadcasting mechanism permits index arrays to be combined with scalars for other indices. The effect is that the scalar value is used for all the corresponding values of the index arrays:: - >>> x = np.arange(35).reshape(5,7) - >>> x[np.array([0,2,4]), 1] + >>> x = np.arange(35).reshape(5, 7) + >>> x[np.array([0, 2, 4]), 1] array([ 1, 15, 29]) .. rubric:: Example @@ -440,7 +441,7 @@ Or wish to add a constant to all negative elements:: >>> x = np.array([1., -1., -2., 3]) >>> x[x < 0] += 20 >>> x - array([ 1., 19., 18., 3.]) + array([1., 19., 18., 3.]) In general if an index includes a Boolean array, the result will be identical to inserting ``obj.nonzero()`` into the same position @@ -502,11 +503,11 @@ Or without ``np.ix_`` (compare the integer array examples):: If x has more dimensions than b then the result will be multi-dimensional:: - >>> x = np.arange(35).reshape(5,7) - >>> b = x>20 - >>> b[:,5] + >>> x = np.arange(35).reshape(5, 7) + >>> b = x > 20 + >>> b[:, 5] array([False, False, False, True, True]) - >>> x[b[:,5]] + >>> x[b[:, 5]] array([[21, 22, 23, 24, 25, 26, 27], [28, 29, 30, 31, 32, 33, 34]]) @@ -515,11 +516,11 @@ combined to make a 2-D array. .. rubric:: Example -Using a 2-D boolean array of shape (2,3) +Using a 2-D boolean array of shape (2, 3) with four True elements to select rows from a 3-D array of shape -(2,3,5) results in a 2-D result of shape (4,5):: +(2, 3, 5) results in a 2-D result of shape (4, 5):: - >>> x = np.arange(30).reshape(2,3,5) + >>> x = np.arange(30).reshape(2, 3, 5) >>> x array([[[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], @@ -582,22 +583,22 @@ behave just like slicing). .. rubric:: Example -Suppose ``x.shape`` is (10,20,30) and ``ind`` is a (2,3,4)-shaped -indexing :class:`intp` array, then ``result = x[...,ind,:]`` has -shape (10,2,3,4,30) because the (20,)-shaped subspace has been -replaced with a (2,3,4)-shaped broadcasted indexing subspace. If -we let *i, j, k* loop over the (2,3,4)-shaped subspace then -``result[...,i,j,k,:] = x[...,ind[i,j,k],:]``. This example +Suppose ``x.shape`` is (10, 20, 30) and ``ind`` is a (2, 3, 4)-shaped +indexing :class:`intp` array, then ``result = x[..., ind, :]`` has +shape (10, 2, 3, 4, 30) because the (20,)-shaped subspace has been +replaced with a (2, 3, 4)-shaped broadcasted indexing subspace. If +we let *i, j, k* loop over the (2, 3, 4)-shaped subspace then +``result[..., i, j, k, :] = x[..., ind[i, j, k], :]``. This example produces the same result as :meth:`x.take(ind, axis=-2) `. .. rubric:: Example -Let ``x.shape`` be (10,20,30,40,50) and suppose ``ind_1`` -and ``ind_2`` can be broadcast to the shape (2,3,4). Then -``x[:,ind_1,ind_2]`` has shape (10,2,3,4,40,50) because the -(20,30)-shaped subspace from X has been replaced with the -(2,3,4) subspace from the indices. However, -``x[:,ind_1,:,ind_2]`` has shape (2,3,4,10,30,50) because there +Let ``x.shape`` be (10, 20, 30, 40, 50) and suppose ``ind_1`` +and ``ind_2`` can be broadcast to the shape (2, 3, 4). Then +``x[:, ind_1, ind_2]`` has shape (10, 2, 3, 4, 40, 50) because the +(20, 30)-shaped subspace from X has been replaced with the +(2, 3, 4) subspace from the indices. However, +``x[:, ind_1, :, ind_2]`` has shape (2, 3, 4, 10, 30, 50) because there is no unambiguous place to drop in the indexing subspace, thus it is tacked-on to the beginning. It is always possible to use :meth:`.transpose() ` to move the subspace @@ -608,7 +609,7 @@ using :func:`take`. Slicing can be combined with broadcasted boolean indices:: - >>> x = np.arange(35).reshape(5,7) + >>> x = np.arange(35).reshape(5, 7) >>> b = x > 20 >>> b array([[False, False, False, False, False, False, False], @@ -616,7 +617,7 @@ Slicing can be combined with broadcasted boolean indices:: [False, False, False, False, False, False, False], [ True, True, True, True, True, True, True], [ True, True, True, True, True, True, True]]) - >>> x[b[:,5],1:3] + >>> x[b[:, 5], 1:3] array([[22, 23], [29, 30]]) @@ -640,7 +641,7 @@ only the part of the data in the specified field. Also, :ref:`record array ` scalars can be "indexed" this way. Indexing into a structured array can also be done with a list of field names, -e.g. ``x[['field-name1','field-name2']]``. As of NumPy 1.16, this returns a +e.g. ``x[['field-name1', 'field-name2']]``. As of NumPy 1.16, this returns a view containing only those fields. In older versions of NumPy, it returned a copy. See the user guide section on :ref:`structured_arrays` for more information on multifield indexing. @@ -649,7 +650,7 @@ If the accessed field is a sub-array, the dimensions of the sub-array are appended to the shape of the result. For example:: - >>> x = np.zeros((2,2), dtype=[('a', np.int32), ('b', np.float64, (3,3))]) + >>> x = np.zeros((2, 2), dtype=[('a', np.int32), ('b', np.float64, (3, 3))]) >>> x['a'].shape (2, 2) >>> x['a'].dtype @@ -725,7 +726,7 @@ In fact, it will only be incremented by 1. The reason is that a new array is extracted from the original (as a temporary) containing the values at 1, 1, 3, 1, then the value 1 is added to the temporary, and then the temporary is assigned back to the original array. Thus -the value of the array at x[1]+1 is assigned to x[1] three times, +the value of the array at x[1] + 1 is assigned to x[1] three times, rather than being incremented 3 times. .. _dealing-with-variable-indices: @@ -742,7 +743,7 @@ supplies to the index a tuple, the tuple will be interpreted as a list of indices. For example (using the previous definition for the array z): :: - >>> indices = (1,1,1,1) + >>> indices = (1, 1, 1, 1) >>> z[indices] 40 @@ -752,14 +753,14 @@ and then use these within an index. Slices can be specified within programs by using the slice() function in Python. For example: :: - >>> indices = (1,1,1,slice(0,2)) # same as [1,1,1,0:2] + >>> indices = (1, 1, 1, slice(0, 2)) # same as [1, 1, 1, 0:2] >>> z[indices] array([39, 40]) Likewise, ellipsis can be specified by code by using the Ellipsis object: :: - >>> indices = (1, Ellipsis, 1) # same as [1,...,1] + >>> indices = (1, Ellipsis, 1) # same as [1, ..., 1] >>> z[indices] array([[28, 31, 34], [37, 40, 43], @@ -772,10 +773,10 @@ it always returns a tuple of index arrays. Because the special treatment of tuples, they are not automatically converted to an array as a list would be. As an example: :: - >>> z[[1,1,1,1]] # produces a large array + >>> z[[1, 1, 1, 1]] # produces a large array array([[[[27, 28, 29], [30, 31, 32], ... - >>> z[(1,1,1,1)] # returns a single value + >>> z[(1, 1, 1, 1)] # returns a single value 40 -- cgit v1.2.1 From d9fba62d64bcbb574c6df415faf88765cdced842 Mon Sep 17 00:00:00 2001 From: Derek Huang Date: Sat, 21 Aug 2021 17:29:19 -0400 Subject: fix basics.creation.rst to address issue 19726 --- doc/source/user/basics.creation.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.creation.rst b/doc/source/user/basics.creation.rst index b6f46a783..b0b54d4ed 100644 --- a/doc/source/user/basics.creation.rst +++ b/doc/source/user/basics.creation.rst @@ -59,8 +59,8 @@ in overflow. This feature can often be misunderstood. If you perform calculations with mismatching ``dtypes``, you can get unwanted results, for example:: - >>> a = array([2, 3, 4], dtype = np.uint32) - >>> b = array([5, 6, 7], dtype = np.uint32) + >>> a = np.array([2, 3, 4], dtype=np.uint32) + >>> b = np.array([5, 6, 7], dtype=np.uint32) >>> c_unsigned32 = a - b >>> print('unsigned c:', c_unsigned32, c_unsigned32.dtype) unsigned c: [4294967293 4294967293 4294967293] uint32 -- cgit v1.2.1 From e68b8043467056880a2b93933ce16d7b66946adf Mon Sep 17 00:00:00 2001 From: Mukulika Date: Sat, 28 Aug 2021 18:09:29 +0530 Subject: DOC: Added examples and other minor changes after review --- doc/source/user/basics.indexing.rst | 201 +++++++++++++++++++++++++----------- 1 file changed, 142 insertions(+), 59 deletions(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index 79108bb20..906408088 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -16,8 +16,8 @@ Indexing on :class:`ndarrays <.ndarray>` :class:`ndarrays ` can be indexed using the standard Python ``x[obj]`` syntax, where *x* is the array and *obj* the selection. -There are four kinds of indexing available depending on *obj*: -single element indexing, basic slicing, advanced indexing and field access. +There are different kinds of indexing available depending on *obj*: +basic slicing, advanced indexing and field access. Most of the following examples show the use of indexing when referencing data in an array. The examples work just as well @@ -28,10 +28,16 @@ Note that in Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to ``x[exp1, exp2, ..., expN]``; the latter is just syntactic sugar for the former. + +.. _basic-slicing-and-indexing: + +Basic slicing and indexing +-------------------------- + .. _single-element-indexing: Single element indexing ------------------------ +^^^^^^^^^^^^^^^^^^^^^^^ Single element indexing works exactly like that for other standard Python sequences. It is 0-based, @@ -72,7 +78,7 @@ element being returned. That is: :: >>> x[0][2] 2 -So note that ``x[0, 2] = x[0][2]`` though the second case is more +So note that ``x[0, 2] == x[0][2]`` though the second case is more inefficient as a new temporary array is created after the first index that is subsequently indexed by 2. @@ -84,10 +90,9 @@ that is subsequently indexed by 2. rapidly changing location in memory. This difference represents a great potential for confusion. -.. _basic-slicing-and-indexing: -Basic slicing and indexing --------------------------- +Slicing and striding +^^^^^^^^^^^^^^^^^^^^ Basic slicing extends Python's basic concept of slicing to N dimensions. Basic slicing occurs when *obj* is a :class:`slice` object @@ -223,8 +228,10 @@ concepts to remember include: .. index:: pair: ndarray; view -Structural indexing tools -^^^^^^^^^^^^^^^^^^^^^^^^^ + +Dimensional indexing tools +^^^^^^^^^^^^^^^^^^^^^^^^^^ + There are some tools to facilitate the easy matching of array shapes with expressions and in assignments. @@ -248,7 +255,7 @@ Each :const:`newaxis` object in the selection tuple serves to expand the dimensions of the resulting selection by one unit-length dimension. The added dimension is the position of the :const:`newaxis` object in the selection tuple. :const:`newaxis` is an alias for -'None', and 'None' can be used in place of this with the same result. +``None``, and ``None`` can be used in place of this with the same result. From the above example:: >>> x[:, np.newaxis, :, :].shape @@ -257,7 +264,7 @@ From the above example:: (2, 1, 3, 1) This can be handy to combine two -arrays in a way that otherwise would require explicitly reshaping +arrays in a way that otherwise would require explicit reshaping operations. For example:: >>> x = np.arange(5) @@ -274,8 +281,6 @@ operations. For example:: Advanced indexing ----------------- -.. seealso:: :ref:`basics.broadcasting` - Advanced indexing is triggered when the selection object, *obj*, is a non-tuple sequence object, an :class:`ndarray` (of data type integer or bool), or a tuple with at least one sequence object or ndarray (of data type @@ -305,8 +310,17 @@ based on their *N*-dimensional index. Each integer array represents a number of indices into that dimension. Negative values are permitted in the index arrays and work as they do with -single indices or slices. If the index values are out of bounds then an -``IndexError`` is thrown:: +single indices or slices:: + + >>> x = np.arange(10, 1, -1) + >>> x + array([10, 9, 8, 7, 6, 5, 4, 3, 2]) + >>> x[np.array([3, 3, 1, 8])] + array([7, 7, 9, 2]) + >>> x[np.array([3, 3, -3, 8])] + array([7, 7, 4, 2]) + +If the index values are out of bounds then an ``IndexError`` is thrown:: >>> x = np.array([[1, 2], [3, 4], [5, 6]]) >>> x[np.array([1, -1])] @@ -315,11 +329,10 @@ single indices or slices. If the index values are out of bounds then an >>> x[np.array([3, 4])] IndexError: index 3 is out of bounds for axis 0 with size 3 - When the index consists of as many integer arrays as dimensions of the array being indexed, the indexing is straightforward, but different from slicing. -Advanced indices always are :ref:`broadcast` and +Advanced indices always are :ref:`broadcast` and iterated as *one*:: result[i_1, ..., i_M] == x[ind_1[i_1, ..., i_M], ind_2[i_1, ..., i_M], @@ -328,7 +341,63 @@ iterated as *one*:: Note that the resulting shape is identical to the (broadcast) indexing array shapes ``ind_1, ..., ind_N``. If the indices cannot be broadcast to the same shape, an exception ``IndexError: shape mismatch: indexing arrays could -not be broadcast together with shapes...`` is raised. +not be broadcast together with shapes...`` is raised. + +Indexing with multidimensional index arrays tend +to be more unusual uses, but they are permitted, and they are useful for some +problems. We’ll start with the simplest multidimensional case:: + + >>> y = np.arange(35).reshape(5, 7) + >>> y + array([[ 0, 1, 2, 3, 4, 5, 6], + [ 7, 8, 9, 10, 11, 12, 13], + [14, 15, 16, 17, 18, 19, 20], + [21, 22, 23, 24, 25, 26, 27], + [28, 29, 30, 31, 32, 33, 34]]) + >>> y[np.array([0, 2, 4]), np.array([0, 1, 2])] + array([ 0, 15, 30]) + +In this case, if the index arrays have a matching shape, and there is an +index array for each dimension of the array being indexed, the resultant +array has the same shape as the index arrays, and the values correspond +to the index set for each position in the index arrays. In this example, +the first index value is 0 for both index arrays, and thus the first value +of the resultant array is ``y[0, 0]``. The next value is ``y[2, 1]``, and +the last is ``y[4, 2]``. + +If the index arrays do not have the same shape, there is an attempt to +broadcast them to the same shape. If they cannot be broadcast to the same +shape, an exception is raised:: + + >>> y[np.array([0, 2, 4]), np.array([0, 1])] + IndexError: shape mismatch: indexing arrays could not be broadcast + together with shapes (3,) (2,) + +The broadcasting mechanism permits index arrays to be combined with +scalars for other indices. The effect is that the scalar value is used +for all the corresponding values of the index arrays:: + + >>> y[np.array([0, 2, 4]), 1] + array([ 1, 15, 29]) + +Jumping to the next level of complexity, it is possible to only partially +index an array with index arrays. It takes a bit of thought to understand +what happens in such cases. For example if we just use one index array +with y:: + + >>> y[np.array([0, 2, 4])] + array([[ 0, 1, 2, 3, 4, 5, 6], + [14, 15, 16, 17, 18, 19, 20], + [28, 29, 30, 31, 32, 33, 34]]) + +It results in the construction of a new array where each value of the +index array selects one row from the array being indexed and the resultant +array has the resulting shape (number of index elements, size of row). + +In general, the shape of the resultant array will be the concatenation of +the shape of the index array (or the shape that all the index arrays were +broadcast to) with the shape of any unused dimensions (those not indexed) +in the array being indexed. .. rubric:: Example @@ -385,19 +454,13 @@ This broadcasting can also be achieved using the function :func:`ix_`: [ 9, 11]]) Note that without the ``np.ix_`` call, only the diagonal elements would -be selected, as was used in the previous example. This difference is the -most important thing to remember about indexing with multiple advanced -indices. +be selected:: -.. rubric:: Example + >>> x[rows, columns] + array([ 0, 11]) -The broadcasting mechanism permits index arrays to be combined with -scalars for other indices. The effect is that the scalar value is used -for all the corresponding values of the index arrays:: - - >>> x = np.arange(35).reshape(5, 7) - >>> x[np.array([0, 2, 4]), 1] - array([ 1, 15, 29]) +This difference is the most important thing to remember about +indexing with multiple advanced indices. .. rubric:: Example @@ -430,7 +493,7 @@ smaller than *x* it is identical to filling it with :py:data:`False`. A common use case for this is filtering for desired element values. For example, one may wish to select all entries from an array which -are not NaN:: +are not :const:`NaN`:: >>> x = np.array([[1., 2.], [np.nan, 3.], [np.nan, np.nan]]) >>> x[~np.isnan(x)] @@ -450,10 +513,28 @@ and using the integer array indexing mechanism described above. ``x[(ind_1,) + boolean_array.nonzero() + (ind_2,)]``. If there is only one Boolean array and no integer indexing array present, -this is straight forward. Care must only be taken to make sure that the +this is straightforward. Care must only be taken to make sure that the boolean index has *exactly* as many dimensions as it is supposed to work with. +In general, when the boolean array has fewer dimensions than the array being +indexed, this is equivalent to ``x[b, …]``, which means x is indexed by b +followed by as many ``:`` as are needed to fill out the rank of x. Thus the +shape of the result is one dimension containing the number of True elements of +the boolean array, followed by the remaining dimensions of the array being +indexed:: + + >>> x = np.arange(35).reshape(5, 7) + >>> b = x > 20 + >>> b[:, 5] + array([False, False, False, True, True]) + >>> x[b[:, 5]] + array([[21, 22, 23, 24, 25, 26, 27], + [28, 29, 30, 31, 32, 33, 34]]) + +Here the 4th and 5th rows are selected from the indexed array and +combined to make a 2-D array. + .. rubric:: Example From an array, select all rows which sum up to less or equal two:: @@ -501,22 +582,7 @@ Or without ``np.ix_`` (compare the integer array examples):: .. rubric:: Example -If x has more dimensions than b then the result will be multi-dimensional:: - - >>> x = np.arange(35).reshape(5, 7) - >>> b = x > 20 - >>> b[:, 5] - array([False, False, False, True, True]) - >>> x[b[:, 5]] - array([[21, 22, 23, 24, 25, 26, 27], - [28, 29, 30, 31, 32, 33, 34]]) - -Here the 4th and 5th rows are selected from the indexed array and -combined to make a 2-D array. - -.. rubric:: Example - -Using a 2-D boolean array of shape (2, 3) +Use a 2-D boolean array of shape (2, 3) with four True elements to select rows from a 3-D array of shape (2, 3, 5) results in a 2-D result of shape (4, 5):: @@ -546,9 +612,27 @@ in the index (or the array has more dimensions than there are advanced indices), then the behaviour can be more complicated. It is like concatenating the indexing result for each advanced index element. -In the simplest case, there is only a *single* advanced index. A single -advanced index can for example replace a slice and the result array will be -the same, however, it is a copy and may have a different memory layout. +In the simplest case, there is only a *single* advanced index combined with +a slice. For example: + + >>> y = np.arange(35).reshape(5,7) + >>> y[np.array([0, 2, 4]), 1:3] + array([[ 1, 2], + [15, 16], + [29, 30]]) + +In effect, the slice and index array operation are independent. The slice +operation extracts columns with index 1 and 2, (i.e. the 2nd and 3rd columns), +followed by the index array operation which extracts rows with index 0, 2 and 4 +(i.e the first, third and fifth rows). This is equivalent to:: + + >>> y[:, 1:3][np.array([0, 2, 4]), :] + array([[ 1, 2], + [15, 16], + [29, 30]]) + +A single advanced index can, for example, replace a slice and the result array +will be the same. However, it is a copy and may have a different memory layout. A slice is preferable when it is possible. For example:: @@ -561,10 +645,10 @@ For example:: >>> x[1:2, [1, 2]] array([[4, 5]]) -The easiest way to understand the situation may be to think in -terms of the resulting shape. There are two parts to the indexing operation, -the subspace defined by the basic indexing (excluding integers) and the -subspace from the advanced indexing part. Two cases of index combination +The easiest way to understand a combination of *multiple* advanced indices may +be to think in terms of the resulting shape. There are two parts to the indexing +operation, the subspace defined by the basic indexing (excluding integers) and +the subspace from the advanced indexing part. Two cases of index combination need to be distinguished: * The advanced indices are separated by a slice, :py:data:`Ellipsis` or @@ -627,8 +711,7 @@ Slicing can be combined with broadcasted boolean indices:: Field access ------------- -.. seealso:: :ref:`arrays.dtypes`, :ref:`arrays.scalars`, - :ref:`structured_arrays` +.. seealso:: :ref:`structured_arrays` If the :class:`ndarray` object is a structured array the :term:`fields ` of the array can be accessed by indexing the array with strings, @@ -726,7 +809,7 @@ In fact, it will only be incremented by 1. The reason is that a new array is extracted from the original (as a temporary) containing the values at 1, 1, 3, 1, then the value 1 is added to the temporary, and then the temporary is assigned back to the original array. Thus -the value of the array at x[1] + 1 is assigned to x[1] three times, +the value of the array at ``x[1] + 1`` is assigned to ``x[1]`` three times, rather than being incremented 3 times. .. _dealing-with-variable-indices: @@ -740,9 +823,9 @@ a function that can handle arguments with various numbers of dimensions without having to write special case code for each number of possible dimensions, how can that be done? If one supplies to the index a tuple, the tuple will be interpreted -as a list of indices. For example (using the previous definition -for the array z): :: +as a list of indices. For example:: + >>> z = np.arange(81).reshape(3, 3, 3, 3) >>> indices = (1, 1, 1, 1) >>> z[indices] 40 -- cgit v1.2.1 From 4d898a2f554c692ebed1fad4010ce570bf7d9294 Mon Sep 17 00:00:00 2001 From: Mukulika <60316606+Mukulikaa@users.noreply.github.com> Date: Sat, 28 Aug 2021 21:48:53 +0530 Subject: DOC: Renamed section to Basic indexing Co-authored-by: Ross Barnowski --- doc/source/user/basics.indexing.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index 906408088..e35c9ea94 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -17,7 +17,7 @@ Indexing on :class:`ndarrays <.ndarray>` :class:`ndarrays ` can be indexed using the standard Python ``x[obj]`` syntax, where *x* is the array and *obj* the selection. There are different kinds of indexing available depending on *obj*: -basic slicing, advanced indexing and field access. +basic indexing, advanced indexing and field access. Most of the following examples show the use of indexing when referencing data in an array. The examples work just as well @@ -29,9 +29,9 @@ Note that in Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to for the former. -.. _basic-slicing-and-indexing: +.. _basic-indexing: -Basic slicing and indexing +Basic indexing -------------------------- .. _single-element-indexing: @@ -518,7 +518,7 @@ boolean index has *exactly* as many dimensions as it is supposed to work with. In general, when the boolean array has fewer dimensions than the array being -indexed, this is equivalent to ``x[b, …]``, which means x is indexed by b +indexed, this is equivalent to ``x[b, ...]``, which means x is indexed by b followed by as many ``:`` as are needed to fill out the rank of x. Thus the shape of the result is one dimension containing the number of True elements of the boolean array, followed by the remaining dimensions of the array being @@ -613,7 +613,7 @@ then the behaviour can be more complicated. It is like concatenating the indexing result for each advanced index element. In the simplest case, there is only a *single* advanced index combined with -a slice. For example: +a slice. For example:: >>> y = np.arange(35).reshape(5,7) >>> y[np.array([0, 2, 4]), 1:3] -- cgit v1.2.1 From 4c852578190d9fe649bd8a1354e0ccac2423fde6 Mon Sep 17 00:00:00 2001 From: Mukulika <60316606+Mukulikaa@users.noreply.github.com> Date: Mon, 30 Aug 2021 19:02:56 +0530 Subject: DOC: Removed unnecessary section link --- doc/source/user/basics.indexing.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.indexing.rst b/doc/source/user/basics.indexing.rst index e35c9ea94..264c3d721 100644 --- a/doc/source/user/basics.indexing.rst +++ b/doc/source/user/basics.indexing.rst @@ -29,10 +29,8 @@ Note that in Python, ``x[(exp1, exp2, ..., expN)]`` is equivalent to for the former. -.. _basic-indexing: - Basic indexing --------------------------- +-------------- .. _single-element-indexing: -- cgit v1.2.1 From de5d3454820cf1918b7241c2c60c26347f4f06e0 Mon Sep 17 00:00:00 2001 From: Robert Kern Date: Wed, 1 Sep 2021 23:03:23 -0400 Subject: DOC: Fix import of default_rng Fixes #19812 19812 --- doc/source/user/basics.creation.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/user/basics.creation.rst b/doc/source/user/basics.creation.rst index b0b54d4ed..a68def887 100644 --- a/doc/source/user/basics.creation.rst +++ b/doc/source/user/basics.creation.rst @@ -235,7 +235,7 @@ library. Below, two arrays are created with shapes (2,3) and (2,3,2), respectively. The seed is set to 42 so you can reproduce these pseudorandom numbers:: - >>> import numpy.random.default_rng + >>> from numpy.random import default_rng >>> default_rng(42).random((2,3)) array([[0.77395605, 0.43887844, 0.85859792], [0.69736803, 0.09417735, 0.97562235]]) -- cgit v1.2.1