summaryrefslogtreecommitdiff
path: root/docs
Commit message (Collapse)AuthorAgeFilesLines
* Memory leak test: take fluctuations into account (#1757)Giampaolo Rodola2020-05-131-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Preamble ======= We have a [memory leak test suite](https://github.com/giampaolo/psutil/blob/e1ea2bccf8aea404dca0f79398f36f37217c45f6/psutil/tests/__init__.py#L897), which calls a function many times and fails if the process memory increased. We do this in order to detect missing `free()` or `Py_DECREF` calls in the C modules. When we do, then we have a memory leak. The problem ========== A problem we've been having for probably over 10 years, is the false positives. That's because the memory fluctuates. Sometimes it may increase (or even decrease!) due to how the OS handles memory, the Python's garbage collector, the fact that RSS is an approximation and who knows what else. So thus far we tried to compensate that by using the following logic: - warmup (call fun 10 times) - call the function many times (1000) - if memory increased before/after calling function 1000 times, then keep calling it for another 3 secs - if it still increased at all (> 0) then fail This logic didn't really solve the problem, as we still had occasional false positives, especially lately on FreeBSD. The solution ========= This PR changes the internal algorithm so that in case of failure (mem > 0 after calling fun() N times) we retry the test for up to 5 times, increasing N (repetitions) each time, so we consider it a failure only if the memory **keeps increasing** between runs. So for instance, here's a legitimate failure: ``` psutil.tests.test_memory_leaks.TestModuleFunctionsLeaks.test_disk_partitions ... Run #1: extra-mem=696.0K, per-call=3.5K, calls=200 Run #2: extra-mem=1.4M, per-call=3.5K, calls=400 Run #3: extra-mem=2.1M, per-call=3.5K, calls=600 Run #4: extra-mem=2.7M, per-call=3.5K, calls=800 Run #5: extra-mem=3.4M, per-call=3.5K, calls=1000 FAIL ``` If, on the other hand, the memory increased on one run (say 200 calls) but decreased on the next run (say 400 calls), then it clearly means it's a false positive, because memory consumption may be > 0 on second run, but if it's lower than the previous run with less repetitions, then it cannot possibly represent a leak (just a fluctuation): ``` psutil.tests.test_memory_leaks.TestModuleFunctionsLeaks.test_net_connections ... Run #1: extra-mem=568.0K, per-call=2.8K, calls=200 Run #2: extra-mem=24.0K, per-call=61.4B, calls=400 OK ``` Note about mallinfo() ================ Aka #1275. `mallinfo()` on Linux is supposed to provide memory metrics about how many bytes gets allocated on the heap by `malloc()`, so it's supposed to be way more precise than RSS and also [USS](http://grodola.blogspot.com/2016/02/psutil-4-real-process-memory-and-environ.html). In another branch were I exposed it, I verified that fluctuations still occur even when using `mallinfo()` though, despite less often. So that means even `mallinfo()` would not grant 100% stability.
* Process wait() improvements (#1747)Giampaolo Rodola2020-05-032-19/+35
| | | | | | | | | | | | | | | | | | | | | | * `Process.wait()` on POSIX now returns an `enum` showing the negative which was used to terminate the process: ```python >>> import psutil >>> p = psutil.Process(9891) >>> p.terminate() >>> p.wait() <Negsignal.SIGTERM: -15> ``` * the return value is cached so that the exit code can be retrieved on then next call, mimicking `subprocess.Popen.wait()` * `Process` object provides more `status` and `exitcode` additional info on `str()` and `repr()`: ``` >>> proc psutil.Process(pid=12739, name='python3', status='terminated', exitcode=<Negsigs.SIGTERM: -15>, started='15:08:20') ``` Extra: * improved `wait()` doc * reverted #1736: `psutil.Popen` uses original `subprocess.Popen.wait` method (safer)
* Revert #1736: Popen inheriting from subprocess (#1744)Giampaolo Rodola2020-05-011-34/+13
|
* psutil.Popen: inherit from subprocess + support wait(timeout=...) parameter ↵Giampaolo Rodola2020-04-281-12/+15
| | | | (#1736)
* Parallel tests (UNIX) (#1709)Giampaolo Rodola2020-04-261-6/+7
|
* sensors_fans is not available on MacOS (#1710)crusaderky2020-03-041-1/+1
|
* Git hook for renamed/added/deleted files + flake8 print() + tidelift (#1704)Giampaolo Rodola2020-02-211-2/+1
|
* Add C linter script (#1698)Giampaolo Rodola2020-02-182-1/+5
|
* revert process_iter() exactly how it was pre #1667Giampaolo Rodola2020-02-181-2/+3
|
* revert #1667 process_iter() new_only paramGiampaolo Rodola2020-02-181-12/+1
| | | | | | | | | | On a second thought I realized that process_iter() uses a global variable, so it's not thread safe. That means that if the are 2 threads using it, the first thread one calling the function (+ consume the iterator), will "steal" the processes of the second thread. psutil.cpu_percent() has the same problem. That means we have a problem can't solve with the current API and requires a lot of thinking on how to solve it as it's not obvious.
* refactor print colors utilsGiampaolo Rodola2020-02-151-0/+7
|
* update docGiampaolo Rodola2020-02-151-0/+3
|
* update docGiampaolo Rodola2020-02-151-3/+3
|
* update docGiampaolo Rodola2020-02-151-10/+10
|
* update docGiampaolo Rodola2020-02-151-27/+18
|
* #1681, revert 00a3398Giampaolo Rodola2020-02-131-2/+0
|
* refactor docGiampaolo Rodola2020-02-124-166/+47
|
* small refactGiampaolo Rodola2020-02-111-1/+1
|
* fix #1681 / linux / disk_partitions: show swapGiampaolo Rodola2020-02-111-0/+2
|
* skip memleak tests on PyPy: they are unreliable probably because of the JITGiampaolo Rodola2020-02-101-2/+2
|
* update docGiampaolo Rodola2020-02-071-1/+1
|
* #1610: clarify in the doc what cpu_count meansGiampaolo Rodola2020-02-041-2/+4
|
* [Windows] use NtQuerySystemInformation to determine process exe() (#1677)Giampaolo Rodola2020-02-011-2/+2
|
* Add CI testing for FreeBSD (#1671)Giampaolo Rodola2020-01-241-0/+4
|
* Add *new_only* parameter for process_iter() (#1667)Giampaolo Rodola2020-01-181-20/+15
|
* #1020: remove doc part mentionin open_files() on win is only able to list ↵Giampaolo Rodola2020-01-091-4/+3
| | | | files living on the C drive
* #1652: also drop support for Windows Server 2003Giampaolo Rodola2020-01-021-3/+2
|
* Merge branch 'master' of github.com:giampaolo/psutilGiampaolo Rodola2020-01-021-2/+2
|\
| * Drop windows XP support (#1652)Giampaolo Rodola2020-01-011-2/+2
| | | | | | | | minimum supported now is Windows Vista
* | update/fix wait_procs() docGiampaolo Rodola2019-12-291-3/+3
|/
* Readme tidelift update (#1636)Giampaolo Rodola2019-12-051-2/+2
|
* total physical memory (exclusive swap). (#1634)Thomas Güttler2019-12-051-1/+1
| | | fix #1633
* update docrelease-5.6.7Giampaolo Rodola2019-11-261-8/+40
|
* pre releaserelease-5.6.6Giampaolo Rodola2019-11-251-0/+4
|
* #1615, 1614: remove pyproject.tomlGiampaolo Rodola2019-11-061-0/+4
|
* pre-releaserelease-5.6.4Giampaolo Rodola2019-11-041-0/+4
|
* docs: fix TypeError in example (#1580)Jérome Perrin2019-11-041-2/+2
| | | | | | Terminate my children example had some error: log("process {} survived SIGTERM; trying SIGKILL" % p) TypeError: not all arguments converted during string formatting
* update docGiampaolo Rodola2019-10-171-7/+9
|
* Removed a typo/copy'n'paste mistake Linux->Windows (#1588)Bastian Ebeling2019-09-181-1/+1
|
* Merge branch 'master' of github.com:giampaolo/psutilGiampaolo Rodola2019-09-151-4/+4
|\
| * broken links fix docs/index.rst (#1555)SteveMoto2019-08-271-3/+3
| |
| * typo fix docs/index.rst (#1553)SteveMoto2019-08-271-1/+1
| |
* | update getloadavg docGiampaolo Rodola2019-09-151-7/+15
| |
* | fix #1527: Linux process CPU iowait timeGiampaolo Rodola2019-09-071-6/+24
|/
* Connection family/type are not converted to enums (#1535)Giampaolo Rodola2019-06-141-3/+10
|
* update docs / HISTORY / CREDITS / @krytarowski for #1530Giampaolo Rodola2019-06-121-0/+2
|
* #1515: document how to get a list of valid attrs names for as_dict()Giampaolo Rodola2019-05-291-0/+4
|
* Update index.rst (#1516)Michael Yoo2019-05-281-1/+1
|
* update CREDITS + small style fixGiampaolo Rodola2019-05-081-0/+1
|
* Fix Process.ionice example using wrong keyword arg (#1504)Tongzhou Wang2019-05-061-2/+2
|
View Lorry Controller Status