delta/postgresql.git/src/test/modules/test_misc, branch master git.postgresql.org: git/postgresql.git Rename io_direct to debug_io_direct. 2023-05-14T22:31:14+00:00 Thomas Munro tmunro@postgresql.org 2023-05-14T22:31:14+00:00 319bae9a8da66678d80d862440e54cebc7b1cc24 Give the new GUC introduced by d4e71df6 a name that is clearly not intended for mainstream use quite yet. Future proposals would drop the prefix only after adding infrastructure to make it efficient. Having the switch in the tree sooner is good because it might lead to new discoveries about the hazards awaiting us on a wide range of systems, but that name was too enticing and could lead to cross-version confusion in future, per complaints from Noah and Justin. Suggested-by: Noah Misch <noah@leadboat.com> Reviewed-by: Noah Misch <noah@leadboat.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> (the idea, not the patch) Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (ditto) Discussion: https://postgr.es/m/20230430041106.GA2268796%40rfd.leadboat.com 
Give the new GUC introduced by d4e71df6 a name that is clearly not
intended for mainstream use quite yet.

Future proposals would drop the prefix only after adding infrastructure
to make it efficient.  Having the switch in the tree sooner is good
because it might lead to new discoveries about the hazards awaiting us
on a wide range of systems, but that name was too enticing and could
lead to cross-version confusion in future, per complaints from Noah and
Justin.

Suggested-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> (the idea, not the patch)
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> (ditto)
Discussion: https://postgr.es/m/20230430041106.GA2268796%40rfd.leadboat.com
Skip the 004_io_direct.pl test if a pre-flight check fails. 2023-04-13T01:57:46+00:00 Thomas Munro tmunro@postgresql.org 2023-04-13T01:47:39+00:00 6ca8df2d61473327b0e90b9becfe109299f47b8a The test previously had a list of OSes that direct I/O was expected to work on. That worked well enough for the systems in our build farm, but didn't survive contact with the Debian build bots running on tmpfs via overlayfs. tmpfs does not support O_DIRECT, but we don't want to exclude Linux generally. The new approach is to try to create an empty file with O_DIRECT from Perl first. If that fails, we'll skip the test and report what the error was. Reported-by: Christoph Berg <myon@debian.org> Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Andrew Dunstan <andrew@dunslane.net> Discussion: https://postgr.es/m/ZDYd4A78cT2ULxZZ%40msg.df7cb.de 
The test previously had a list of OSes that direct I/O was expected to
work on.  That worked well enough for the systems in our build farm, but
didn't survive contact with the Debian build bots running on tmpfs via
overlayfs.  tmpfs does not support O_DIRECT, but we don't want to
exclude Linux generally.

The new approach is to try to create an empty file with O_DIRECT from
Perl first.  If that fails, we'll skip the test and report what the
error was.

Reported-by: Christoph Berg <myon@debian.org>
Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Reviewed-by: Andrew Dunstan <andrew@dunslane.net>
Discussion: https://postgr.es/m/ZDYd4A78cT2ULxZZ%40msg.df7cb.de
Use higher wal_level for 004_io_direct.pl. 2023-04-08T20:27:36+00:00 Thomas Munro tmunro@postgresql.org 2023-04-08T20:27:36+00:00 980e8879f54a7a00ca6a5bae2fe9486c87ef3e8e The new direct I/O test deliberately uses a very small shared_buffers to force some disk transfers without making the data set large and slow, but ran into a problem with wal_level = minimal: log_newpage_range() pins many buffers, leading to a few intermittent "no unpinned buffers available" errors. We could presumably fix that by adjusting shared_buffers, but crake seems to be trying to tell us something interesting with these settings, so let's just avoid wal_level = minimal in this test for now. Reported-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/20230408060408.n7xdwk3mxj5oykt6%40awork3.anarazel.de 
The new direct I/O test deliberately uses a very small shared_buffers to
force some disk transfers without making the data set large and slow,
but ran into a problem with wal_level = minimal: log_newpage_range()
pins many buffers, leading to a few intermittent "no unpinned buffers
available" errors.

We could presumably fix that by adjusting shared_buffers, but crake
seems to be trying to tell us something interesting with these settings,
so let's just avoid wal_level = minimal in this test for now.

Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20230408060408.n7xdwk3mxj5oykt6%40awork3.anarazel.de
Add io_direct setting (developer-only). 2023-04-08T04:35:07+00:00 Thomas Munro tmunro@postgresql.org 2023-04-07T23:04:49+00:00 d4e71df6d757fd21c363164a3a4d3b5681462662 Provide a way to ask the kernel to use O_DIRECT (or local equivalent) where available for data and WAL files, to avoid or minimize kernel caching. This hurts performance currently and is not intended for end users yet. Later proposed work would introduce our own I/O clustering, read-ahead, etc to replace the facilities the kernel disables with this option. The only user-visible change, if the developer-only GUC is not used, is that this commit also removes the obscure logic that would activate O_DIRECT for the WAL when wal_sync_method=open_[data]sync and wal_level=minimal (which also requires max_wal_senders=0). Those are non-default and unlikely settings, and this behavior wasn't (correctly) documented. The same effect can be achieved with io_direct=wal. Author: Thomas Munro <thomas.munro@gmail.com> Author: Andres Freund <andres@anarazel.de> Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Discussion: https://postgr.es/m/CA%2BhUKGK1X532hYqJ_MzFWt0n1zt8trz980D79WbjwnT-yYLZpg%40mail.gmail.com 
Provide a way to ask the kernel to use O_DIRECT (or local equivalent)
where available for data and WAL files, to avoid or minimize kernel
caching.  This hurts performance currently and is not intended for end
users yet.  Later proposed work would introduce our own I/O clustering,
read-ahead, etc to replace the facilities the kernel disables with this
option.

The only user-visible change, if the developer-only GUC is not used, is
that this commit also removes the obscure logic that would activate
O_DIRECT for the WAL when wal_sync_method=open_[data]sync and
wal_level=minimal (which also requires max_wal_senders=0).  Those are
non-default and unlikely settings, and this behavior wasn't (correctly)
documented.  The same effect can be achieved with io_direct=wal.

Author: Thomas Munro <thomas.munro@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGK1X532hYqJ_MzFWt0n1zt8trz980D79WbjwnT-yYLZpg%40mail.gmail.com
Fix outdated references to guc.c 2023-03-02T12:49:39+00:00 Daniel Gustafsson dgustafsson@postgresql.org 2023-03-02T12:49:39+00:00 7ab1bc2939f32aa995bdb81eaac3bb28f908e980 Commit 0a20ff54f split out the GUC variables from guc.c into a new file guc_tables.c. This updates comments referencing guc.c regarding variables which are now in guc_tables.c. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/6B50C70C-8C1F-4F9A-A7C0-EEAFCC032406@yesql.se 
Commit 0a20ff54f split out the GUC variables from guc.c into a new file
guc_tables.c. This updates comments referencing guc.c regarding variables
which are now in guc_tables.c.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/6B50C70C-8C1F-4F9A-A7C0-EEAFCC032406@yesql.se
Update copyright for 2023 2023-01-02T20:00:37+00:00 Bruce Momjian bruce@momjian.us 2023-01-02T20:00:37+00:00 c8e1ba736b2b9e8c98d37a5b77c4ed31baf94147 Backpatch-through: 11 
Backpatch-through: 11
Add copyright notices to meson files 2022-12-20T12:54:39+00:00 Andrew Dunstan andrew@dunslane.net 2022-12-20T12:54:39+00:00 8284cf5f746f84303eda34d213e89c8439a83a42 Discussion: https://postgr.es/m/222b43a5-2fb3-2c1b-9cd0-375d376c8246@dunslane.net 
Discussion: https://postgr.es/m/222b43a5-2fb3-2c1b-9cd0-375d376c8246@dunslane.net
Use installed postgresql.conf.sample for GUC sanity TAP test 2022-11-13T14:07:53+00:00 Andrew Dunstan andrew@dunslane.net 2022-11-13T14:07:53+00:00 9c7eb9d85a5687d4f9624c40ee3a508d8c7de35a The current code looks for the sample file in the source directory, but it seems better to test against the installed sample file. Backpatch to release 15 where the test was introduced. Discussion: https://postgr.es/m/73eea68e-3b6f-5f63-6024-25ed26b52016@dunslane.net Reviewed by Tom Lane, Alvaro Herrera, Michael Paquier. 
The current code looks for the sample file in the source directory, but
it seems better to test against the installed sample file.

Backpatch to release 15 where the test was introduced.

Discussion: https://postgr.es/m/73eea68e-3b6f-5f63-6024-25ed26b52016@dunslane.net

Reviewed by Tom Lane, Alvaro Herrera, Michael Paquier.
meson: Add initial version of meson based build system 2022-09-22T05:37:17+00:00 Andres Freund andres@anarazel.de 2022-09-22T04:53:12+00:00 e6927270cd18d535b77cbe79c55c6584351524be Autoconf is showing its age, fewer and fewer contributors know how to wrangle it. Recursive make has a lot of hard to resolve dependency issues and slow incremental rebuilds. Our home-grown MSVC build system is hard to maintain for developers not using Windows and runs tests serially. While these and other issues could individually be addressed with incremental improvements, together they seem best addressed by moving to a more modern build system. After evaluating different build system choices, we chose to use meson, to a good degree based on the adoption by other open source projects. We decided that it's more realistic to commit a relatively early version of the new build system and mature it in tree. This commit adds an initial version of a meson based build system. It supports building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD, Solaris and Windows (however only gcc is supported on aix, solaris). For Windows/MSVC postgres can now be built with ninja (faster, particularly for incremental builds) and msbuild (supporting the visual studio GUI, but building slower). Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM bitcode generation, documentation adjustments) are done in subsequent commits requiring further review. Other aspects (e.g. not installing test-only extensions) are not yet addressed. When building on Windows with msbuild, builds are slower when using a visual studio version older than 2019, because those versions do not support MultiToolTask, required by meson for intra-target parallelism. The plan is to remove the MSVC specific build system in src/tools/msvc soon after reaching feature parity. However, we're not planning to remove the autoconf/make build system in the near future. Likely we're going to keep at least the parts required for PGXS to keep working around until all supported versions build with meson. Some initial help for postgres developers is at https://wiki.postgresql.org/wiki/Meson With contributions from Thomas Munro, John Naylor, Stone Tickle and others. Author: Andres Freund <andres@anarazel.de> Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Author: Peter Eisentraut <peter@eisentraut.org> Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de 
Autoconf is showing its age, fewer and fewer contributors know how to wrangle
it. Recursive make has a lot of hard to resolve dependency issues and slow
incremental rebuilds. Our home-grown MSVC build system is hard to maintain for
developers not using Windows and runs tests serially. While these and other
issues could individually be addressed with incremental improvements, together
they seem best addressed by moving to a more modern build system.

After evaluating different build system choices, we chose to use meson, to a
good degree based on the adoption by other open source projects.

We decided that it's more realistic to commit a relatively early version of
the new build system and mature it in tree.

This commit adds an initial version of a meson based build system. It supports
building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD,
Solaris and Windows (however only gcc is supported on aix, solaris). For
Windows/MSVC postgres can now be built with ninja (faster, particularly for
incremental builds) and msbuild (supporting the visual studio GUI, but
building slower).

Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM
bitcode generation, documentation adjustments) are done in subsequent commits
requiring further review. Other aspects (e.g. not installing test-only
extensions) are not yet addressed.

When building on Windows with msbuild, builds are slower when using a visual
studio version older than 2019, because those versions do not support
MultiToolTask, required by meson for intra-target parallelism.

The plan is to remove the MSVC specific build system in src/tools/msvc soon
after reaching feature parity. However, we're not planning to remove the
autoconf/make build system in the near future. Likely we're going to keep at
least the parts required for PGXS to keep working around until all supported
versions build with meson.

Some initial help for postgres developers is at
https://wiki.postgresql.org/wiki/Meson

With contributions from Thomas Munro, John Naylor, Stone Tickle and others.

Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Author: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de
Pre-beta mechanical code beautification. 2022-05-12T19:17:30+00:00 Tom Lane tgl@sss.pgh.pa.us 2022-05-12T19:17:30+00:00 23e7b38bfe396f919fdb66057174d29e17086418 Run pgindent, pgperltidy, and reformat-dat-files. I manually fixed a couple of comments that pgindent uglified. 
Run pgindent, pgperltidy, and reformat-dat-files.
I manually fixed a couple of comments that pgindent uglified.