summaryrefslogtreecommitdiff
path: root/storage/connect/libdoc.cpp
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch '10.4' into 10.5Sergei Golubchik2022-10-021-38/+0
|\
| * Merge branch '10.3' into 10.4Sergei Golubchik2022-10-011-38/+0
| |\
| | * MDEV-29579 Fix CONNECT ASAN hits (#2277)Andrew Hutchings2022-09-261-38/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are currently two things causing ASAN hits on CONNECT engine when the plugin is used as a dynamic module. These are libxml2 and libodbc. libxml2 has some quirks when not the first and last thing called in the main thread of an application, some of the global memory isn't cleaned up correctly. The same is assumed of libodbc but this does not have explicit API for this. This is being fixed in two ways. First we are removing the libxml2 cleanup call. This is because the current one is messy and whatever it fixed has gone away. But also because if this is called and libxml2 is used again this can cause issues. For example if two different plugins to MariaDB both happen to use libxml2. The second fix is a hack that exploits `STB_GNU_UNIQUE` so that when compiled with ASAN the plugin will remain in memory after dlclose(). This allows libodbc to cleanup and has the added advatage that we will get clean stacks from ASAN for CONNECT when the leak is detected at the end of execution. Details of the `STB_GNU_UNIQUE` method can be found here: https://web.archive.org/web/20100730094324/http://www.redhat.com/archives/posix-c++-wg/2009-August/msg00002.html
* | | Merge 10.4 into 10.5Marko Mäkelä2022-07-271-4/+4
|\ \ \ | |/ /
| * | Merge branch '10.3' into 10.4Oleksandr Byelkin2022-07-271-4/+4
| |\ \ | | |/
| | * Use memory safe snprintf() in Connect EngineMikhail Chalov2022-07-261-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit replaces sprintf(buf, ...) with snprintf(buf, sizeof(buf), ...), specifically in the "easy" cases where buf is allocated with a size known at compile time. The changes make sure we are not write outside array/string bounds which will lead to undefined behaviour. In case the code is trying to write outside bounds - safe version of functions simply cut the string messages so we process this gracefully. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc. bsonudf.cpp warnings cleanup by Daniel Black Reviewer: Daniel Black
| * | Merge branch 'bb-10.3-release' into bb-10.4-releaseSergei Golubchik2021-02-121-1/+1
| |\ \ | | |/ | | | | | | | | | Note, the fix for "MDEV-23328 Server hang due to Galera lock conflict resolution" was null-merged. 10.4 version of the fix is coming up separately
| | * CONNECT: compiler warningsSergei Golubchik2021-02-011-1/+1
| | |
* | | Merge 10.4 into 10.5Marko Mäkelä2020-02-071-33/+33
|\ \ \ | |/ /
| * | Fixed compiler warnings from gcc 7.4.1Monty2020-01-291-34/+34
| |/ | | | | | | - Fixed possible error in rocksdb/rdb_datadic.cc
* | Merge 10.4 into 10.5Marko Mäkelä2019-09-061-1/+1
|\ \ | |/
| * In CONNECT version 1.6.10 NOSQL facility is enhanced by a new way to ↵Olivier Bertrand2019-07-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | retrieve NOSQL data. In addition to files and Mongo collections, JSON as well as XML and CSV data can be retrieved from the net as answers from REST queries. Because it uses and external package (cpprestsdk) this is currently available only to MariaDB servers compiled from source. -- Add the REST support when Microsoft Casablanca package (cpprestsdk) is installed. -- Also include some changes specific to MariaDB 10.3. modified: storage/connect/CMakeLists.txt -- Add conditional REST support -- Added string options HTTP and URI. -- Added added internal table type TAB_REST. modified: storage/connect/ha_connect.cc modified: storage/connect/mycat.cc modified: storage/connect/mycat.h modified: storage/connect/plgdbsem.h -- Fix MDEV-19648 Variable connect_conv_size doesn't change -- Change Variable wrong block parameter from 8169 to 1. -- Also change connect_conv_size default value to 1024. modified: storage/connect/ha_connect.cc -- Avoid possible buffer overflow -- In particular by the function ShowValue. modified: storage/connect/tabdos.cpp modified: storage/connect/tabfmt.cpp modified: storage/connect/value.cpp modified: storage/connect/value.h -- Add some cast to avoid some compiler warnings modified: storage/connect/filamdbf.cpp -- Fix some C++ error modified: storage/connect/javaconn.cpp modified: storage/connect/jmgoconn.cpp modified: storage/connect/plugutil.cpp -- Miscellaneous Typo and warning suppressing changes modified: storage/connect/connect.cpp modified: storage/connect/connect.h modified: storage/connect/filamvct.cpp modified: storage/connect/inihandl.cpp modified: storage/connect/jsonudf.cpp modified: storage/connect/libdoc.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/tabtbl.cpp modified: storage/connect/tabxml.cpp modified: storage/connect/user_connect.cc modified: storage/connect/user_connect.h -- Update failing test results and disbling modified: storage/connect/mysql-test/connect/disabled.def modified: storage/connect/mysql-test/connect/r/dir.result modified: storage/connect/mysql-test/connect/r/grant.result modified: storage/connect/mysql-test/connect/r/jdbc.result modified: storage/connect/mysql-test/connect/r/jdbc_postgresql.result modified: storage/connect/mysql-test/connect/r/xml.result modified: storage/connect/mysql-test/connect/r/xml2.result modified: storage/connect/mysql-test/connect/r/xml2_mult.result modified: storage/connect/mysql-test/connect/r/xml_mult.result -- Add an option modified: storage/connect/mysql-test/connect/t/grant.test
* | Fix compiler warnings GCC8Noel Kuntze2019-07-301-1/+1
| |
* | MDEV-19897 Rename source code variable names from utf8 to utf8mb3Alexander Barkov2019-06-281-4/+4
|/
* - Change the connect_xtrace variable to from int to setOlivier Bertrand2018-01-311-49/+49
| | | | | modified: storage/connect/ha_connect.cc modified: storage/connect/libdoc.cpp
* Fix gcc compiler warnings reported by SergeiOlivier Bertrand2017-05-231-14/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | modified: storage/connect/array.cpp modified: storage/connect/array.h modified: storage/connect/blkfil.cpp modified: storage/connect/blkfil.h modified: storage/connect/block.h modified: storage/connect/colblk.cpp modified: storage/connect/colblk.h modified: storage/connect/csort.h modified: storage/connect/filamvct.cpp modified: storage/connect/filter.cpp modified: storage/connect/filter.h modified: storage/connect/global.h modified: storage/connect/json.h modified: storage/connect/plgdbsem.h modified: storage/connect/plgdbutl.cpp modified: storage/connect/tabcol.cpp modified: storage/connect/tabcol.h modified: storage/connect/tabdos.cpp modified: storage/connect/tabdos.h modified: storage/connect/tabjson.cpp modified: storage/connect/table.cpp modified: storage/connect/tabodbc.cpp modified: storage/connect/tabodbc.h modified: storage/connect/tabsys.h modified: storage/connect/tabxml.h modified: storage/connect/value.cpp modified: storage/connect/value.h modified: storage/connect/xindex.cpp modified: storage/connect/xindex.h modified: storage/connect/xobject.cpp modified: storage/connect/xobject.h modified: storage/connect/xtable.h Set values as nullable when retrieving catalog info modified: storage/connect/jdbconn.cpp modified: storage/connect/mysql-test/connect/r/odbc_oracle.result modified: storage/connect/odbconn.cpp Change format of Jpath modified: storage/connect/json.cpp modified: storage/connect/jsonudf.cpp modified: storage/connect/mysql-test/connect/r/json.result modified: storage/connect/mysql-test/connect/r/json_udf.result modified: storage/connect/mysql-test/connect/r/json_udf_bin.result modified: storage/connect/mysql-test/connect/r/zip.result modified: storage/connect/mysql-test/connect/t/json.test modified: storage/connect/mysql-test/connect/t/json_udf.test modified: storage/connect/mysql-test/connect/t/json_udf_bin.test modified: storage/connect/mysql-test/connect/t/zip.test modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h modified: storage/connect/tabmgo.cpp Change null representation from ??? to <null> modified: storage/connect/json.cpp Change the name of UDF that are equal to a native JSON function name modified: storage/connect/jsonudf.cpp modified: storage/connect/jsonudf.h modified: storage/connect/mysql-test/connect/t/json_udf.inc modified: storage/connect/mysql-test/connect/t/json_udf2.inc Fix bug in making JSON project info modified: storage/connect/mongofam.cpp Fix COMPUTE when one argument is null modified: storage/connect/value.cpp Value is null only when nullable modified: storage/connect/value.h
* Fix some XML table type bugs:Olivier Bertrand2016-12-231-9/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - in DOMNODELIST::DropItem if (Listp == NULL || Listp->length <= n) return true; is wrong, should be: if (Listp == NULL || Listp->length < n) return true; - Crash in discovery with libxml2 in XMLColumns because: if (!tdp->Usedom) // nl was destroyed vp->nl = vp->pn->GetChildElements(g); is executed with vp->pn uninitialized. Fixed by adding: vp->pn = node; line 264. -In discovery with libxml2 some columns are not found. Because list was not recovered properly, nodes being modified and not reallocated. Fixed lines 214 and 277. modified: storage/connect/domdoc.cpp modified: storage/connect/tabxml.cpp Add support for zipped table files modified: storage/connect/domdoc.cpp modified: storage/connect/domdoc.h modified: storage/connect/filamap.cpp modified: storage/connect/filamap.h modified: storage/connect/filamzip.cpp modified: storage/connect/filamzip.h modified: storage/connect/ha_connect.cc modified: storage/connect/libdoc.cpp modified: storage/connect/plgdbutl.cpp modified: storage/connect/plgxml.cpp modified: storage/connect/plgxml.h modified: storage/connect/tabdos.cpp modified: storage/connect/tabdos.h modified: storage/connect/tabfmt.cpp modified: storage/connect/tabjson.cpp modified: storage/connect/tabxml.cpp
* JSONColumns and XMLColumns revisited. They can retrieve their parameters ↵Olivier Bertrand2015-05-261-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | directly from the PTOS argument. For this to work, finding the table options is now split in HA_CONNECT functions and exported functions available from out of ha_connect. modified: storage/connect/ha_connect.cc modified: storage/connect/libdoc.cpp modified: storage/connect/mycat.h modified: storage/connect/plgdbsem.h modified: storage/connect/tabjson.cpp modified: storage/connect/tabjson.h modified: storage/connect/tabxml.cpp modified: storage/connect/tabxml.h The BIN table formats have been changed to handle the case of floating point values when used with Big Endian or Little Endian machines. modified: storage/connect/ha_connect.cc modified: storage/connect/mysql-test/connect/r/bin.result modified: storage/connect/mysql-test/connect/t/bin.test modified: storage/connect/reldef.cpp modified: storage/connect/tabdos.cpp modified: storage/connect/tabdos.h modified: storage/connect/tabfix.cpp modified: storage/connect/tabfix. h
* - Fix Catalog JSON table crash when no JpathOlivier Bertrand2015-04-171-6/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Added JSON OBJECT specification for pretty != 2. - Fix NULL values not recognized for nullable JSON columns - Issue an error message when a JSON table is created without specifying LRECL if PRETTY != 2. - Make JSONColumns use a TDBJSON class. - Make JSON table using MAPFAM modified: filamap.h filamtxt.h ha_connect.cc json.result tabjson.cpp tabjson.h table.cpp - Implementing Discovery for the XML table type. modified: domdoc.cpp domdoc.h ha_connect.cc libdoc.cpp plgxml.cpp plgxml.h reldef.cpp reldef.h tabxml.cpp tabxml.h - Providing an error message when creating an ODBC table via discovery returns columns of more than one table. modified: ha_connect.cc - TableOptionStruct declaration moved from ha_connect.h to mycat.h To make it easier to use by other classes. modified: ha_connect.cc ha_connect.h mycat.cc mycat.h reldef.cpp tabmysql.cpp taboccur.cpp tabpivot.cpp tabtbl.cpp tabutil.cpp tabxcl.cpp
* This commit includes changes done in a previous (deleted) branch plus new ones.Olivier Bertrand2015-03-181-3/+2
| | | | | | | | | | | | | | | | | | | | From the previous branch: commit eda4928ff122a0845baf5ade83b4aa29244a3a89 Author: Olivier Bertrand <bertrandop@gmail.com> Date: Mon Mar 9 22:34:56 2015 +0100 - Add discovery to JSON tables When columns are not defined, CONNECT analyses the json file to find column definitions. This wors only on table that are an array of objects. Pair keys are used to generate the column names and pair values are used for its definition. When the LEVEL option is defined as a not null integer, the eventual JPATH is scanned up to the LEVEL value. From the current one: - Fix MDEV-7521 when column names are utf8 encoded (not a general multi-charset fix) - Adds more to JSON discovery processing and UDF's - Use PlugDup everywhere it replaces PlugSubAlloc + strcpy.
* 1) Handling string memory allocation with a new STRING class. This is onlyOlivier Bertrand2014-10-211-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the beginning. Defining the STRING class and begining to use it (MYSQL) 2) Change the xtrace, use_tempfile and exact_info connect variables from GLOBAL to SESSION. Remaining GLOBAL variables have been made readonly. 3) Take care of LEX_STRING variables. The .str should not be regarded as allways being 0 terminated. This is handled by the Strz functions that make sure to return 0 terminated strings. Bug fix: - When inserting in MYSQL table with special column(s) a query such as: insert into t2 values(0,4,'new04'),(0,5,'new05'); failed saying: column id (the special column) not found in t2. It is now accepted but must be counted in values (these 0 are ignored) - ROWID was returning row numbers based 0. Now it is from base 1. modified: storage/connect/array.cpp storage/connect/blkfil.cpp storage/connect/colblk.cpp storage/connect/connect.cc storage/connect/filamap.cpp storage/connect/filamdbf.cpp storage/connect/filamfix.cpp storage/connect/filamtxt.cpp storage/connect/filamvct.cpp storage/connect/filamzip.cpp storage/connect/filamzip.h storage/connect/filter.cpp storage/connect/global.h storage/connect/ha_connect.cc storage/connect/ha_connect.h storage/connect/libdoc.cpp storage/connect/mycat.cc storage/connect/myconn.cpp storage/connect/odbconn.cpp storage/connect/plgdbutl.cpp storage/connect/plugutil.c storage/connect/reldef.cpp storage/connect/tabcol.cpp storage/connect/tabdos.cpp storage/connect/tabfix.cpp storage/connect/tabfmt.cpp storage/connect/table.cpp storage/connect/tabmul.cpp storage/connect/tabmysql.cpp storage/connect/tabmysql.h storage/connect/taboccur.cpp storage/connect/tabodbc.cpp storage/connect/tabpivot.cpp storage/connect/tabsys.cpp storage/connect/tabtbl.cpp storage/connect/tabutil.cpp storage/connect/tabvct.cpp storage/connect/tabwmi.cpp storage/connect/tabwmi.h storage/connect/tabxcl.cpp storage/connect/tabxml.cpp storage/connect/user_connect.cc storage/connect/valblk.cpp storage/connect/value.cpp storage/connect/value.h storage/connect/xindex.cpp storage/connect/xobject.cpp storage/connect/xobject.h storage/connect/xtable.h
* 5.5.40+ mergeSergei Golubchik2014-10-091-5/+1
|
* ConnectSE: fixing memory leaks reported by valgrind Alexander Barkov2013-10-291-2/+41
|
* - Reset some variables when closing just in case the XML tre is re-used.Olivier Bertrand2013-09-251-0/+4
| | | | | | modified: storage/connect/libdoc.cpp
* - Fix several bugs causing memory leak or invalid access detectedOlivier Bertrand2013-09-221-27/+142
| | | | | | | | | | | | | | | by Valgrind. This concerns the XML libxml2 support. modified: storage/connect/domdoc.cpp storage/connect/domdoc.h storage/connect/ha_connect.cc storage/connect/libdoc.cpp storage/connect/plgdbsem.h storage/connect/plgxml.h storage/connect/tabxml.cpp storage/connect/tabxml.h
* Fixing warnings:Alexander Barkov2013-07-081-0/+149
| | | | | | | | | | | | | - no previous declaration for ‘void XmlCleanupParserLib()’ - no previous declaration for ‘void CloseXML2File’ - no previous declaration for ‘void XmlInitParserLib()’ modified: storage/connect/ha_connect.cc storage/connect/libdoc.cpp storage/connect/libdoc.h storage/connect/plgdbutl.cpp
* - Try to fix a uninitialised valgrind warningOlivier Bertrand2013-07-051-2/+12
| | | | | | | | | | | | | | | | | modified: storage/connect/ha_connect.cc storage/connect/ha_connect.h storage/connect/mycat.cc - Fix a regression error on XML libdoc wrong Nlist freing modified: storage/connect/domdoc.h storage/connect/libdoc.cpp storage/connect/libdoc.h storage/connect/plgxml.h storage/connect/tabxml.cpp
* - Makes memory check conditionallyOlivier Bertrand2013-07-031-13/+13
| | | | | | modified: storage/connect/libdoc.cpp
* - Fix memory leak in libdoc.cpp in LIBXMLDOC::GetNodeList replacingOlivier Bertrand2013-07-021-15/+138
| | | | | | | | | xmlXPathFreeNodeSetList(Xop); // Caused memory leak, by xmlXPathFreeObject(Xop); modified: storage/connect/libdoc.cpp
* - Fix conversion bug for MS-DOM XML tables. The node content was writtenOlivier Bertrand2013-03-021-12/+20
| | | | | | | | | | | | | | | | | | | | and read as if the table DATA_CHARSET was ANSI instead of UTF-8. Warning are now provided when the read content of a node is truncated. modified: storage/connect/domdoc.cpp storage/connect/domdoc.h storage/connect/libdoc.cpp storage/connect/libdoc.h storage/connect/plgxml.h storage/connect/tabxml.cpp - Conditional compilation of pre_create depending on the MARIADB setting. modified: storage/connect/ha_connect.cc storage/connect/ha_connect.h
* - Re-install blank trimming to have the xml test pass.Olivier Bertrand2013-02-201-2/+3
| | | | | | | | | Note that the problem if far more complex. To be revisited. modified: storage/connect/libdoc.cpp storage/connect/tabxml.h
* - Fix the elimination of control characters from node contentOlivier Bertrand2013-02-201-16/+16
| | | | | | | | | | | | | | | - Take care of XML special chars (<>& etc.) - Remove Encode, Decode modified: storage/connect/libdoc.cpp storage/connect/libdoc.h - Neither libmysql.lib nor mysqlclient.lib needed for MYSQL table type modified: storage/connect/CMakeLists.txt
* - Fix crash on making an XML table with encoding=XXXOlivier Bertrand2013-02-201-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | - Set parameters so libxml2 does not anymore add extra characters when retrieving several subnodes of a node. - Make a CONNECT file header (was PlugDB) modified: storage/connect/domdoc.cpp storage/connect/libdoc.cpp storage/connect/tabxml.cpp - Change the version number modified: storage/connect/ha_connect.cc - Begin eliminate use of libmysql functions in MYSQL table type Not finished yet modified: storage/connect/myconn.cpp storage/connect/myconn.h
* Adding DATA_CHARSET table option.Alexander Barkov2013-02-181-2/+2
|
* Suppress multiple blanks and control characters from theOlivier Bertrand2013-02-151-1/+21
| | | | | | | | | node text retrieved by libxml2. Modified: libdoc.cpp
* - Fixing TAB to 2 spacesAlexander Barkov2013-02-071-751/+751
| | | | | - Fixing line endings from "\r\n" to "\n"
* Introducing functions global_open() and global_fopen() for these purposes:Alexander Barkov2013-02-061-1/+1
| | | | | | | | | | | | | | | | | | | | | - Removing duplicate code to generate error message text - In the future they will most likely check secure_file_priv directory. modified: storage/connect/filamdbf.cpp storage/connect/filamfix.cpp storage/connect/filamtxt.cpp storage/connect/filamvct.cpp storage/connect/libdoc.cpp storage/connect/maputil.cpp storage/connect/plgdbsem.h storage/connect/plgdbutl.cpp storage/connect/tabfmt.cpp storage/connect/tabmul.cpp storage/connect/tabxml.cpp storage/connect/xindex.cpp
* Fixing my_charset_utf8_bin to my_charset_utf8_general_ci.Alexander Barkov2013-02-041-2/+2
| | | | | | | | | | The formed required strings.lib in Windows. The latter is an exported symbol from mysqld and does not need strings.lib. modified: storage/connect/ha_connect.cc storage/connect/libdoc.cpp
* 2>libdoc.cppOlivier Bertrand2013-02-021-6/+7
| | | | | | | | | | | | | 2>D:\CommonSource\mariadb-10.0\include\my_pthread.h(120) : warning C4005: '_REENTRANT' : redéfinition de macro 2> D:\Libxml\include\libxml/xmlexports.h(77) : voir la définition précédente de '_REENTRANT' 2>.\libdoc.cpp(378) : error C2664: 'strlen' : impossible de convertir le paramètre 1 de 'xmlChar *' en 'const char *' 2> Les types pointés n'ont aucun rapport entre eux ; conversion nécessitant reinterpret_cast, cast de style C ou cast de style fonction 2>.\libdoc.cpp(379) : error C2664: 'copy_and_convert' : impossible de convertir le paramètre 4 de 'xmlChar *' en 'const char *' 2> Les types pointés n'ont aucun rapport entre eux ; conversion nécessitant reinterpret_cast, cast de style C ou cast de style fonction 2> Fix for that.
* Replacing iconv converstion routines in libdoc.cc to MariaDB routines.Alexander Barkov2013-02-011-36/+20
| | | | | | | | | TODO: Access to Field->charset() instead of hardcoded latin1 conversion. modified: storage/connect/libdoc.cpp storage/connect/libdoc.h
* Adding the CONNECT storage engine sources.Alexander Barkov2013-01-181-0/+766
View Lorry Controller Status