summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* Fix typo in CI READMEDaniel Gustafsson2023-04-031-1/+1
| | | | s/fron/from/
* Only make buffer strategy for vacuum when it's likely neededDavid Rowley2023-04-031-1/+3
| | | | | | | | | | | | | | | | VACUUM FULL and VACUUM ONLY_DATABASE_STATS will not use the vacuum strategy ring created in vacuum(), so don't waste effort making it in those cases. There are other conceivable cases where the buffer strategy also won't be used, but those are probably less common and not worth troubling over too much. For example VACUUM (PROCESS_MAIN false, PROCESS_TOAST false). There are other cases too, but many of these are only discovered once inside vacuum_rel(). Author: Melanie Plageman Reviewed-by: David Rowley Discussion: https://postgr.es/m/CAAKRu_ZLRuzkM3zKogiZAz2hUony37yLY4aaLb8fPf8fgqs5Og@mail.gmail.com
* pg_basebackup: Correct type of WalSegSzPeter Eisentraut2023-04-032-2/+2
| | | | | | | | | The pg_basebackup code had WalSegSz as uint32, whereas the rest of the code has it as int. This seems confusing, and using the extra range wouldn't actually work. Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Discussion: https://www.postgresql.org/message-id/flat/1bf15c7a-0acd-1864-081e-7a28814310fe%40enterprisedb.com
* Remove some global variables from vacuum.cDavid Rowley2023-04-031-23/+20
| | | | | | | | | | | | | | Using global variables because we don't want to pass these values around as parameters does not really seem like a great idea, so let's remove these two global variables and adjust a few functions to accept these values as parameters instead. This is part of a wider patch which intends to allow the size of the buffer access strategy that vacuum uses to be adjusted. Author: Melanie Plageman Reviewed-by: Bharath Rupireddy Discussion: https://postgr.es/m/CAAKRu_b1q_07uquUtAvLqTM%3DW9nzee7QbtzHwA4XdUo7KX_Cnw%40mail.gmail.com
* Doc: update pgindent/README.Tom Lane2023-04-021-3/+3
| | | | I missed updating this when we pulled pg_bsd_indent into the tree.
* Add info in WAL records in preparation for logical slot conflict handlingAndres Freund2023-04-0215-29/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit only implements one prerequisite part for allowing logical decoding. The commit message contains an explanation of the overall design, which later commits will refer back to. Overall design: 1. We want to enable logical decoding on standbys, but replay of WAL from the primary might remove data that is needed by logical decoding, causing error(s) on the standby. To prevent those errors, a new replication conflict scenario needs to be addressed (as much as hot standby does). 2. Our chosen strategy for dealing with this type of replication slot is to invalidate logical slots for which needed data has been removed. 3. To do this we need the latestRemovedXid for each change, just as we do for physical replication conflicts, but we also need to know whether any particular change was to data that logical replication might access. That way, during WAL replay, we know when there is a risk of conflict and, if so, if there is a conflict. 4. We can't rely on the standby's relcache entries for this purpose in any way, because the startup process can't access catalog contents. 5. Therefore every WAL record that potentially removes data from the index or heap must carry a flag indicating whether or not it is one that might be accessed during logical decoding. Why do we need this for logical decoding on standby? First, let's forget about logical decoding on standby and recall that on a primary database, any catalog rows that may be needed by a logical decoding replication slot are not removed. This is done thanks to the catalog_xmin associated with the logical replication slot. But, with logical decoding on standby, in the following cases: - hot_standby_feedback is off - hot_standby_feedback is on but there is no a physical slot between the primary and the standby. Then, hot_standby_feedback will work, but only while the connection is alive (for example a node restart would break it) Then, the primary may delete system catalog rows that could be needed by the logical decoding on the standby (as it does not know about the catalog_xmin on the standby). So, it’s mandatory to identify those rows and invalidate the slots that may need them if any. Identifying those rows is the purpose of this commit. Implementation: When a WAL replay on standby indicates that a catalog table tuple is to be deleted by an xid that is greater than a logical slot's catalog_xmin, then that means the slot's catalog_xmin conflicts with the xid, and we need to handle the conflict. While subsequent commits will do the actual conflict handling, this commit adds a new field isCatalogRel in such WAL records (and a new bit set in the xl_heap_visible flags field), that is true for catalog tables, so as to arrange for conflict handling. The affected WAL records are the ones that already contain the snapshotConflictHorizon field, namely: - gistxlogDelete - gistxlogPageReuse - xl_hash_vacuum_one_page - xl_heap_prune - xl_heap_freeze_page - xl_heap_visible - xl_btree_reuse_page - xl_btree_delete - spgxlogVacuumRedirect Due to this new field being added, xl_hash_vacuum_one_page and gistxlogDelete do now contain the offsets to be deleted as a FLEXIBLE_ARRAY_MEMBER. This is needed to ensure correct alignment. It's not needed on the others struct where isCatalogRel has been added. This commit just introduces the WAL format changes mentioned above. Handling the actual conflicts will follow in future commits. Bumps XLOG_PAGE_MAGIC as the several WAL records are changed. Author: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com> Author: Andres Freund <andres@anarazel.de> (in an older version) Author: Amit Khandekar <amitdkhan.pg@gmail.com> (in an older version) Reviewed-by: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Robert Haas <robertmhaas@gmail.com> Reviewed-by: Fabrízio de Royes Mello <fabriziomello@gmail.com> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
* Fix copy-pasto in contrib/auth_delay/meson.build variable name.Noah Misch2023-04-021-2/+2
|
* Use PG_TEST_TIMEOUT_DEFAULT in 019_replslot_limit.pl.Noah Misch2023-04-021-5/+4
| | | | | | | | | Oversight in f2698ea02ca8a56f38935d2b300ac54936712558, which introduced the variable. This lowers some 1000s timeouts to the configurable default of 180s, due to a lack of evidence for needing the longer timeout. The alternative was 6*PG_TEST_TIMEOUT_DEFAULT, which we can adopt if the need arises. Given the lack of observed trouble with these timeouts, no back-patch.
* Pass down table relation into more index relation functionsAndres Freund2023-04-0126-170/+214
| | | | | | | | | | | | This is done in preparation for logical decoding on standby, which needs to include whether visibility affecting WAL records are about a (user) catalog table. Which is only known for the table, not the indexes. It's also nice to be able to pass the heap relation to GlobalVisTestFor() in vacuumRedirectAndPlaceholder(). Author: "Drouvot, Bertrand" <bertranddrouvot.pg@gmail.com> Discussion: https://postgr.es/m/21b700c3-eecf-2e05-a699-f8c78dd31ec7@gmail.com
* Assert only valid flag bits are passed to visibilitymap_set()Andres Freund2023-04-011-1/+1
| | | | | | | | | If visibilitymap_set() is called with flags containing a higher bit than VISIBILITYMAP_ALL_FROZEN, the state of neighboring pages is affected. While there was an assertion that *some* valid bits were set, it did not check that *only* valid bits were. Change that. Discussion: https://postgr.es/m/20230331043300.gux3s5wzrapqi4oe@awork3.anarazel.de
* hio: Release extension lock before initializing page / pinning VMAndres Freund2023-04-011-7/+7
| | | | | | | | | | | | | | | | PageInit() while holding the extension lock is unnecessary after 0d1fe9f74e3 started to use RBM_ZERO_AND_LOCK - nobody can look at the new page before we release the page lock. PageInit() zeroes the page, which isn't that cheap, so deferring it until after the extension lock is released seems like a good idea. Doing visibilitymap_pin() while holding the extension lock, introduced in 7db0cd2145f2, looks like an accident. Due to the restrictions on HEAP_INSERT_FROZEN it's unlikely to be a performance issue, but it still seems better to move it out. We also are doing the visibilitymap_pin() while holding the buffer lock, which will be fixed in a separate commit. Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi> Discussion: http://postgr.es/m/419312fd-9255-078c-c3e3-f0525f911d7f@iki.fi
* pg_dump: Use only LZ4 frame format for compressionTomas Vondra2023-04-011-211/+343
| | | | | | | | | | | | | | | | | | | | | | | | After 0da243fed0 got committed, it was reported that in some cases the compression ratio is rather poor - particularly for custom format with narrow tables - due to writing the LZ4 header/footer for each row. This commit switches to LZ4F (LZ4 frame format), eliminating most of the overhead and greatly improving the compression ratio. This makes the compressed size about the same for plain and custom formats (just like for gzip, for example). LZ4F is now used by both compression APIs, which allowed refactoring and reusing more of the code. For consistency this also renames the LZ4File struct to LZ4State, and a number of functions are now prefixed with LZ4Stream_ (instead of LZ4File_). Patch by Georgios Kokolatos, based on report and initial patch by Justin Pryzby. Review and minor cleanups by me. Author: Georgios Kokolatos, Justin Pryzby Reported-by: Justin Pryzby Reviewed-by: Tomas Vondra Discussion: https://postgr.es/m/20230227044910.GO1653%40telsasoft.com
* Doc: add Buffer Access Strategy to the glossaryDavid Rowley2023-04-011-0/+28
| | | | | | | | | | It seems useful to add this to the glossary as there's discussion around adding an option to VACUUM to disable and adjust the size of the buffer access strategy that VACUUM uses. Author: Melanie Plageman Reviewed-by: Justin Pryzby, David Rowley Discussion: https://postgr.es/m/ZBYDTrD1kyGg%2BHkS%40telsasoft.com
* Add show_data option to pg_get_wal_block_info.Peter Geoghegan2023-03-316-24/+39
| | | | | | | | | | | | | | | | | | | | | | | Allow users to opt out of returning FPI data and block data from pg_get_wal_block_info as an optimization. Testing has shown that this can make function execution over twice as fast in some cases. When pg_get_wal_block_info is called with "show_data := false", it always returns NULL values for its block_data and block_fpi_data bytea output parameters. Nothing else changes. In particular, the function will still return the usual per-block summary of block data/FPI space overhead. Use of "show_data := false" is therefore feasible with all queries that don't specifically require these raw binary strings. Follow-up to recent work in commit 122376f0. There still hasn't been a stable release with the pg_get_wal_block_info function, so no bump in the pg_walinspect extension version. Per suggestion from Melanie Plageman. Author: Peter Geoghegan <pg@bowt.ie> Discussion: https://postgr.es/m/CAAKRu_bJvbcYBRj2cN6G2xV7B7-Ja+pjTO1nEnEhRR8OXYiABA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-Wzm9shOkEDM10_+qOZkRSQhKVxwBFiehH6EHWQQRd_rDPw@mail.gmail.com
* SQL/JSON: support the IS JSON predicateAlvaro Herrera2023-03-3125-67/+1030
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch introduces the SQL standard IS JSON predicate. It operates on text and bytea values representing JSON, as well as on the json and jsonb types. Each test has IS and IS NOT variants and supports a WITH UNIQUE KEYS flag. The tests are: IS JSON [VALUE] IS JSON ARRAY IS JSON OBJECT IS JSON SCALAR These should be self-explanatory. The WITH UNIQUE KEYS flag makes these return false when duplicate keys exist in any object within the value, not necessarily directly contained in the outermost object. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Amit Langote <amitlangote09@gmail.com> Author: Andrew Dunstan <andrew@dunslane.net> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/CAF4Au4w2x-5LTnN_bxky-mq4=WOqsGsxSpENCzHRAzSnEd8+WQ@mail.gmail.com Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org
* Further tweaking of width_bucket() edge cases.Tom Lane2023-03-314-28/+88
| | | | | | | | | | | | | | | | | | I realized that the third overflow case I posited in commit b0e9e4d76 actually should be handled in a different way: rather than tolerating the idea that the quotient could round to 1, we should clamp so that the output cannot be more than "count" when we know that the operand is less than bound2. That being the case, we don't need an overflow-aware increment in that code path, which leads me to revert the movement of the pg_add_s32_overflow() call. (The diff in width_bucket_float8 might be easier to read by comparing against b0e9e4d76^.) What's more, width_bucket_numeric also has this problem of the quotient potentially rounding to 1, so add a clamp there too. As before, I'm not quite convinced that a back-patch is warranted. Discussion: https://postgr.es/m/391415.1680268470@sss.pgh.pa.us
* Reject system columns as elements of foreign keys.Tom Lane2023-03-313-10/+23
| | | | | | | | | | | | | | | | | | | | | | Up through v11 it was sensible to use the "oid" system column as a foreign key column, but since that was removed there's no visible usefulness in making any of the remaining system columns a foreign key. Moreover, since the TupleTableSlot rewrites in v12, such cases actively fail because of implicit assumptions that only user columns appear in foreign keys. The lack of complaints about that seems like good evidence that no one is trying to do it. Hence, rather than trying to repair those assumptions (of which there are at least two, maybe more), let's just forbid the case up front. Per this patch, a system column in either the referenced or referencing side of a foreign key will draw this error; however, putting one in the referenced side would have failed later anyway, since we don't allow unique indexes to be made on system columns. Per bug #17877 from Alexander Lakhin. Back-patch to v12; the case still appears to work in v11, so we shouldn't break it there. Discussion: https://postgr.es/m/17877-4bcc658e33df6de1@postgresql.org
* Ensure acquire_inherited_sample_rows sets its output parameters.Tom Lane2023-03-311-2/+4
| | | | | | | | | | | | The totalrows/totaldeadrows outputs were left uninitialized in cases where we find no analyzable child tables of a partitioned table. This could lead to setting the partitioned table's pg_class.reltuples value to garbage. It's not clear that that would have any very bad effects in practice, but fix it anyway because it's making valgrind unhappy. Reported and diagnosed by Alexander Lakhin (bug #17880). Discussion: https://postgr.es/m/17880-9282037c923d856e@postgresql.org
* pg_regress: Emit TAP compliant outputDaniel Gustafsson2023-03-314-262/+325
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This converts pg_regress output format to emit TAP compliant output while keeping it as human readable as possible for use without TAP test harnesses. As verbose harness related information isn't really supported by TAP this also reduces the verbosity of pg_regress runs which makes scrolling through log output in buildfarm/CI runs a bit easier as well. As the meson TAP parser conumes whitespace, the leading indentation for differentiating parallel tests from sequential tests has been changed to a single character prefix. This patch has been around for an extended period of time, reviewers listed below may have been involved in reviewing a version quite different from the version in this commit. The original idea for this patch was a hacking session with Jinbao Chen. TAP format testing is also enabled in meson as of this. Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Nikolay Shaplov <dhyan@nataraj.su> Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/BD4B107D-7E53-4794-ACBA-275BEB4327C9@yesql.se Discussion: https://postgr.es/m/20220221164736.rq3ornzjdkmwk2wo@alap3.anarazel.de
* Move ExecEvalJsonConstructor new function to a more natural placeAlvaro Herrera2023-03-312-39/+39
| | | | | Commit 7081ac46ace8 put it at the end of the file, but that doesn't look very nice.
* No need to add FORMAT to the keyword precedence listAlvaro Herrera2023-03-311-1/+0
| | | | Commit 7081ac46ace8 put it there. Remove it.
* Add XML ID attributes to create_publication.sgml.Amit Kapila2023-03-315-51/+68
| | | | | | | | | | This commit adds XML ID attributes to all varlistentries in create_publication.sgml. This allows us to include links to refer to publication options, making documents more readable. Author: Kuroda Hayato Reviewed-by: Peter Smith, Amit Kapila Discussion: https://postgr.es/m/TYAPR01MB58668219FEA4EC231486A433F58E9@TYAPR01MB5866.jpnprd01.prod.outlook.com
* Bump PGSTAT_FILE_FORMAT_ID, omitted in 8aaa04b32d7Andres Freund2023-03-301-1/+1
| | | | | | I forgot to do so in the referenced commit. While the consequences of omitting the version change are likely to be harmless (besides discarding stats, as a PGSTAT_FILE_FORMAT_ID bump also does), it still seems worth doing.
* Track shared buffer hits in pg_stat_ioAndres Freund2023-03-3012-47/+109
| | | | | | | | | | | | Among other things, this should make it easier to calculate a useful cache hit ratio by excluding buffer reads via buffer access strategies. As buffer access strategies reuse buffers (and thus evict the prior buffer contents), it is normal to see reads on repeated scans of the same data. Author: Melanie Plageman <melanieplageman@gmail.com> Reviewed-by: Bertrand Drouvot <bertranddrouvot.pg@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Discussion: https://postgr.es/m/CAAKRu_beMa9Hzih40%3DXPYqhDVz6tsgUGTrhZXRo%3Dunp%2Bszb%3DUA%40mail.gmail.com
* Fix List memory issue in transformColumnDefinitionDavid Rowley2023-03-311-0/+3
| | | | | | | | | | | | | | | | | | | | | When calling generateSerialExtraStmts(), we would pass in the constraint->options. In some cases, generateSerialExtraStmts() would modify the referenced List to remove elements from it, but doing so is invalid without assigning the list back to all variables that point to it. In the particular reported problem case, the List became empty, in which cases it became NIL, but the passed in constraint->options didn't get to find out about that and was left pointing to free'd memory. To fix this, just perform a list_copy() inside generateSerialExtraStmts(). We could just do a list_copy() just before we perform the delete from the list, however, that seems less robust. Let's make sure the generated CreateSeqStmt gets a completely different copy of the list to be safe. Bug: #17879 Reported-by: Fei Changhong Diagnosed-by: Fei Changhong Discussion: https://postgr.es/m/17879-b7dfb5debee58ff5@postgresql.org Backpatch-through: 11, all supported versions
* Parallel Hash Full Join.Thomas Munro2023-03-317-48/+323
| | | | | | | | | | | | | | | | | | | | | | | | | | Full and right outer joins were not supported in the initial implementation of Parallel Hash Join because of deadlock hazards (see discussion). Therefore FULL JOIN inhibited parallelism, as the other join strategies can't do that in parallel either. Add a new PHJ phase PHJ_BATCH_SCAN that scans for unmatched tuples on the inner side of one batch's hash table. For now, sidestep the deadlock problem by terminating parallelism there. The last process to arrive at that phase emits the unmatched tuples, while others detach and are free to go and work on other batches, if there are any, but otherwise they finish the join early. That unfairness is considered acceptable for now, because it's better than no parallelism at all. The build and probe phases are run in parallel, and the new scan-for-unmatched phase, while serial, is usually applied to the smaller of the two relations and is either limited by some multiple of work_mem, or it's too big and is partitioned into batches and then the situation is improved by batch-level parallelism. Author: Melanie Plageman <melanieplageman@gmail.com> Author: Thomas Munro <thomas.munro@gmail.com> Reviewed-by: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/CA%2BhUKG%2BA6ftXPz4oe92%2Bx8Er%2BxpGZqto70-Q_ERwRaSyA%3DafNg%40mail.gmail.com
* pg_stat_wal: Accumulate time as instr_time instead of microsecondsAndres Freund2023-03-305-24/+37
| | | | | | | | | | | | | | | | | | | In instr_time.h it is stated that: * When summing multiple measurements, it's recommended to leave the * running sum in instr_time form (ie, use INSTR_TIME_ADD or * INSTR_TIME_ACCUM_DIFF) and convert to a result format only at the end. The reason for that is that converting to microseconds is not cheap, and can loose precision. Therefore this commit changes 'PendingWalStats' to use 'instr_time' instead of 'PgStat_Counter' while accumulating 'wal_write_time' and 'wal_sync_time'. Author: Nazir Bilal Yavuz <byavuz81@gmail.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Melanie Plageman <melanieplageman@gmail.com> Discussion: https://postgr.es/m/1feedb83-7aa9-cb4b-5086-598349d3f555@gmail.com
* Show record information in pg_get_wal_block_info.Peter Geoghegan2023-03-305-160/+273
| | | | | | | | | | | | | | | | | | | | | | | | | | | Expand the output parameters in pg_walinspect's pg_get_wal_block_info function to return additional information that was previously only available from pg_walinspect's pg_get_wal_records_info function. Some of the details are attributed to individual block references, rather than aggregated into whole-record values, since the function returns one row per block reference per WAL record (unlike pg_get_wal_records_info, which always returns one row per WAL record). This structure is much easier to work with when writing queries that track how individual blocks changed over time, or when attributing costs to individual blocks (not WAL records) is useful. This is the second time that pg_get_wal_block_info has been enhanced in recent weeks. Commit 9ecb134a expanded on the original version of the function added in commit c31cf1c0 (where it first appeared under the name pg_get_wal_fpi_info). There still hasn't been a stable release since commit c31cf1c0, so no bump in the pg_walinspect extension version. Author: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Author: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Peter Geoghegan <pg@bowt.ie> Reviewed-By: Kyotaro HORIGUCHI <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/CALj2ACVRK5=Z+2ZVsjgTTSkfEnQzCuwny7iigpG7g1btk4Ws2A@mail.gmail.com
* Simplify transformJsonAggConstructor() APIAlvaro Herrera2023-03-301-21/+15
| | | | | | There's no need for callers to pass aggregate names so that the function can resolve them to OIDs, when callers can just pass aggregate OIDs directly to begin with.
* Fix inconsistencies and style issues in new SQL/JSON codeAlvaro Herrera2023-03-305-17/+21
| | | | | | Reported by Alexander Lakhin. Discussion: https://postgr.es/m/60483139-5c34-851d-baee-6c0d014e1710@gmail.com
* Fix setrefs.c code for adjusting partPruneInfosAlvaro Herrera2023-03-301-31/+52
| | | | | | | | | | | | | | | | | | | | | We were transferring partPruneInfos from PlannerInfo into PlannerGlobal wrong, essentially relying on all of them being transferred, and adjusting their list indexes based on that. But apparently it's possible that some of them are skipped, so that strategy leads to a corrupted execution tree. Instead, adjust each Append/MergeAppend's partpruneinfo index as we copy from one list to the other, which seems safer anyway. This requires adjusting the RT offset of the RTE referenced in each partPruneInfo ahead of actually adjusting the RTE itself, which seems a bit too ad-hoc. This problem was introduced by commit ec386948948c. However, it may be that we no longer require the change introduced there, so perhaps we should revert both the present commit and that one. Problem noticed by sqlsmith. Author: Amit Langote <amitlangote09@gmail.com Discussion: https://postgr.es/m/CA+HiwqG6tbc2oadsbyyy24b2AL295XHQgyLRWghmA7u_SL1K8A@mail.gmail.com
* Fix format code in fd.c debugging infrastructureAndres Freund2023-03-301-3/+3
| | | | These were not sufficiently adjusted in 2d4f1ba6cfc.
* bufmgr: Fix undefined behaviour with, unrealistically, large temp_buffersAndres Freund2023-03-301-1/+1
| | | | | | | | | | | | | | | | | | Quoting Melanie: > Since if buffer is INT_MAX, then the -(buffer + 1) version invokes > undefined behavior while the -buffer - 1 version doesn't. All other places were already using the correct version. I (Andres), copied the code into more places in a patch. Melanie caught it in review, but to prevent more people from copying the bad code, fix it. Even if it is a theoretical issue. We really ought to wrap these accesses in a helper function... As this is a theoretical issue, don't backpatch. Reported-by: Melanie Plageman <melanieplageman@gmail.com> Discussion: https://postgr.es/m/CAAKRu_aW2SX_LWtwHgfnqYpBrunMLfE9PD6-ioPpkh92XH0qpg@mail.gmail.com
* Clean up role created in new subscription test.Tom Lane2023-03-302-0/+2
| | | | This oversight broke repeated runs of "make installcheck".
* Fix documentation build for c3afe8cf5a1e465bd71e48e4bc717f5bfdc7a7d6.Robert Haas2023-03-301-13/+13
| | | | | This documentation hunk was intended to be part of that commit, but I goofed.
* Add new predefined role pg_create_subscription.Robert Haas2023-03-3021-60/+384
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This role can be granted to non-superusers to allow them to issue CREATE SUBSCRIPTION. The non-superuser must additionally have CREATE permissions on the database in which the subscription is to be created. Most forms of ALTER SUBSCRIPTION, including ALTER SUBSCRIPTION .. SKIP, now require only that the role performing the operation own the subscription, or inherit the privileges of the owner. However, to use ALTER SUBSCRIPTION ... RENAME or ALTER SUBSCRIPTION ... OWNER TO, you also need CREATE permission on the database. This is similar to what we do for schemas. To change the owner of a schema, you must also have permission to SET ROLE to the new owner, similar to what we do for other object types. Non-superusers are required to specify a password for authentication and the remote side must use the password, similar to what is required for postgres_fdw and dblink. A superuser who wants a non-superuser to own a subscription that does not rely on password authentication may set the new password_required=false property on that subscription. A non-superuser may not set password_required=false and may not modify a subscription that already has password_required=false. This new password_required subscription property works much like the eponymous postgres_fdw property. In both cases, the actual semantics are that a password is not required if either (1) the property is set to false or (2) the relevant user is the superuser. Patch by me, reviewed by Andres Freund, Jeff Davis, Mark Dilger, and Stephen Frost (but some of those people did not fully endorse all of the decisions that the patch makes). Discussion: http://postgr.es/m/CA+TgmoaDH=0Xj7OBiQnsHTKcF2c4L+=gzPBUKSJLh8zed2_+Dg@mail.gmail.com
* Avoid overflow in width_bucket_float8().Tom Lane2023-03-303-15/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The original coding of this function paid little attention to the possibility of overflow. There were actually three different hazards: 1. The range from bound1 to bound2 could exceed DBL_MAX, which on IEEE-compliant machines produces +Infinity in the subtraction. At best we'd lose all precision in the result, and at worst produce NaN due to dividing Inf/Inf. The range can't exceed twice DBL_MAX though, so we can fix this case by scaling all the inputs by 0.5. 2. We computed count * (operand - bound1), which is also at risk of float overflow, before dividing. Safer is to do the division first, producing a quotient that should be in [0,1), and even after allowing for roundoff error can't be outside [0,1]; then multiplying by count can't produce a result overflowing an int. (width_bucket_numeric does the multiplication first on the grounds that that improves accuracy of its result, but I don't think that a similar argument can be made in float arithmetic.) 3. If the division result does round to 1, and count is INT_MAX, the final addition of 1 would overflow an int. We took care of that in the operand >= bound2 case but did not consider that it could be possible in the main path. Fix that by moving the overflow-aware addition of 1 so it is done that way in all cases. The fix for point 2 creates a possibility that values very close to a bucket boundary will be rounded differently than they were before. I'm not troubled by that for HEAD, but it is an argument against putting this into the stable branches. Given that the cases being fixed here are fairly extreme and unlikely to be hit in normal use, it seems best not to back-patch. Mats Kindahl and Tom Lane Discussion: https://postgr.es/m/17876-61f280d1601f978d@postgresql.org
* Fix pointer cast for seed calculation on 32-bit systemsDaniel Gustafsson2023-03-301-1/+1
| | | | | | | | | | | | | | | | | | The fallback seed for when pg_strong_random cannot generate a high quality seed mixes in the address of the conn object, but the cast failed to take the word size into consideration. Fix by casting to a uintptr_t instead. The seed calculation was added in 7f5b19817e. The code as it stood generated the following warning on mamba and lapwing in the buildfarm: fe-connect.c: In function 'libpq_prng_init': fe-connect.c:1048:11: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast] 1048 | rseed = ((uint64) conn) ^ | ^ Author: Hayato Kuroda <kuroda.hayato@fujitsu.com> Discussion: https://postgr.es/m/TYAPR01MB58665250EDCD551CCA9AD117F58E9@TYAPR01MB5866.jpnprd01.prod.outlook.com
* Fix incorrect format placeholdersPeter Eisentraut2023-03-301-2/+2
|
* Refactor pgoutput_change().Amit Kapila2023-03-301-156/+79
| | | | | | | | | | Instead of mostly-duplicate code for different operation (insert/update/delete) types, write a common code to compute old/new tuples, and check the row filter. Author: Hou Zhijie Reviewed-by: Peter Smith, Amit Kapila Discussion: https://postgr.es/m/OS0PR01MB5716194A47FFA8D91133687D94BF9@OS0PR01MB5716.jpnprd01.prod.outlook.com
* Fix outdated comments regarding TupleTableSlotsDavid Rowley2023-03-302-9/+10
| | | | | | | | | | | | The tts_flag is named TTS_FLAG_SHOULDFREE, so use that instead of TTS_SHOULDFREE, which is the name of the macro that checks for that flag. Additionally, 4da597edf got rid of the TupleTableSlot.tts_tuple field but forgot to update a comment which referenced that field. Fix that. Reported-by: Zhen Mingyang <zhenmingyang@yeah.net> Reported-by: Richard Guo <guofenglinux@gmail.com> Discussion: https://postgr.es/m/1a96696c.9d3.187193989c3.Coremail.zhenmingyang@yeah.net
* Support connection load balancing in libpqDaniel Gustafsson2023-03-2910-3/+431
| | | | | | | | | | | | | | | | | | | | | This adds support for load balancing connections with libpq using a connection parameter: load_balance_hosts=<string>. When setting the param to random, hosts and addresses will be connected to in random order. This then results in load balancing across these addresses and hosts when multiple clients or frequent connection setups are used. The randomization employed performs two levels of shuffling: 1. The given hosts are randomly shuffled, before resolving them one-by-one. 2. Once a host its addresses get resolved, the returned addresses are shuffled, before trying to connect to them one-by-one. Author: Jelte Fennema <postgres@jeltef.nl> Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Reviewed-by: Michael Banck <mbanck@gmx.net> Reviewed-by: Andrey Borodin <amborodin86@gmail.com> Discussion: https://postgr.es/m/PR3PR83MB04768E2FF04818EEB2179949F7A69@PR3PR83MB0476.EURPRD83.prod.outlook.
* Copy and store addrinfo in libpq-owned private memoryDaniel Gustafsson2023-03-294-34/+92
| | | | | | | | | | | | | | | | This refactors libpq to copy addrinfos returned by getaddrinfo to memory owned by libpq such that future improvements can alter for example the order of entries. As a nice side effect of this refactor the mechanism for iteration over addresses in PQconnectPoll is now identical to its iteration over hosts. Author: Jelte Fennema <postgres@jeltef.nl> Reviewed-by: Aleksander Alekseev <aleksander@timescale.com> Reviewed-by: Michael Banck <mbanck@gmx.net> Reviewed-by: Andrey Borodin <amborodin86@gmail.com> Discussion: https://postgr.es/m/PR3PR83MB04768E2FF04818EEB2179949F7A69@PR3PR83MB0476.EURPRD83.prod.outlook.com
* Fix dereference of dangling pointer in GiST index buffering build.Tom Lane2023-03-291-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | gistBuildCallback tried to fetch the size of an index tuple that might have already been freed by gistProcessEmptyingQueue. While this seems to usually be harmless in production builds, in principle it could result in a SIGSEGV, or more likely a bogus value for indtuplesSize leading to poor page-split decisions later in the build. The memory management here is confusing and could stand to be refactored, but for the moment it seems to be enough to fetch the tuple size sooner. AFAICT the indtuples[Size] totals aren't used in between these places; even if they were, the updated values shouldn't be any worse to use. So just move the incrementing of the totals up. It's not very clear why our valgrind-using buildfarm animals haven't noticed this problem, because the relevant code path does seem to be exercised according to the code coverage report. I think the reason that we didn't fix this bug after the first report is that I'd wanted to try to understand that better. However, now that it's been re-discovered let's just be pragmatic and fix it already. Original report by Alexander Lakhin (bug #16329), later rediscovered by Egor Chindyaskin (bug #17874). Patch by Alexander Lakhin (commentary by Pavel Borisov and me). Back-patch to all supported branches. Discussion: https://postgr.es/m/16329-7a6aa9b6fa1118a1@postgresql.org Discussion: https://postgr.es/m/17874-63ca6c7ce42d2103@postgresql.org
* Add missing .gitignore entries.Tom Lane2023-03-291-0/+2
| | | | Oversight in commit 7081ac46ace8c459966174400b53418683c9fe5c.
* Remove empty function BufmgrCommit().Tom Lane2023-03-293-20/+3
| | | | | | | | | | | | | | This function has been a no-op for over a decade. Even if bufmgr regains a need to be called during commit, it seems unlikely that the most appropriate call points would be precisely here, so it's not doing us much good as a placeholder either. Now, removing it probably doesn't save any noticeable number of cycles --- but the main call is inside the commit critical section, and the less work done there the better. Matthias van de Meent Discussion: https://postgr.es/m/CAEze2Wi1=tLKbxZnXzcD+8fYKyKqBtivVakLQC_mYBsP4Y8qVA@mail.gmail.com
* SQL/JSON: add standard JSON constructor functionsAlvaro Herrera2023-03-2942-141/+4470
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit introduces the SQL/JSON standard-conforming constructors for JSON types: JSON_ARRAY() JSON_ARRAYAGG() JSON_OBJECT() JSON_OBJECTAGG() Most of the functionality was already present in PostgreSQL-specific functions, but these include some new functionality such as the ability to skip or include NULL values, and to allow duplicate keys or throw error when they are found, as well as the standard specified syntax to specify output type and format. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> Author: Teodor Sigaev <teodor@sigaev.ru> Author: Oleg Bartunov <obartunov@gmail.com> Author: Alexander Korotkov <aekorotkov@gmail.com> Author: Amit Langote <amitlangote09@gmail.com> Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/CAF4Au4w2x-5LTnN_bxky-mq4=WOqsGsxSpENCzHRAzSnEd8+WQ@mail.gmail.com Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org
* Fix some section numbers in information_schema.sqlPeter Eisentraut2023-03-291-4/+4
| | | | | Some of the section numbers that appeared multiple times were not updated completely by previous changes d61d9aa750 and eb3a1376c9.
* meson: Change default buildtype to debugoptimizedPeter Eisentraut2023-03-292-2/+2
| | | | | | | | | This matches the Autoconf default (-O2 + debug) better. The previous default setting "release" used -O3, which resulted in different compiler warnings. At least for now, we want to avoid such divergence. Discussion: https://www.postgresql.org/message-id/flat/CAFj8pRBJD_Y-XcqwXSbWS24z%2B84FFX7ajhCan9ixc_m4bD63sA%40mail.gmail.com
* Move definition of standard collations from initdb to pg_collation.datPeter Eisentraut2023-03-293-15/+9
| | | | | | | | | | The standard collations "ucs_basic" and "unicode" were defined in initdb, even though pg_collation.dat seems like the correct place for them. It seems this was just forgotten during various reorganizations of initdb and pg_collation.dat/.h over time. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/08b58ecd-0d50-9395-ed51-dc8294e3fd2b%40enterprisedb.com
View Lorry Controller Status