diff options
782 files changed, 25190 insertions, 12684 deletions
diff --git a/HOWTO/INSTALL-WIN32-OLD.md b/HOWTO/INSTALL-WIN32-OLD.md new file mode 100644 index 0000000000..00dc502a94 --- /dev/null +++ b/HOWTO/INSTALL-WIN32-OLD.md @@ -0,0 +1,898 @@ +How to Build Erlang/OTP on Windows +================================== + +Introduction +------------ + +This HOWTO describes how to build from Cygwin, MSYS or MSYS2 environments, +we currently use WSL to build and test Erlang/OTP so the instructions +herein is no longer tested/used by the OTP team. + +Using Cygwin, MSYS or MSYS2 as a build environment will be removed in +the future. + +This section describes how to build the Erlang emulator and the OTP +libraries on Windows. Note that the Windows binary releases are still +a preferred alternative if one does not have Microsoft’s development +tools and/or don’t want to install Cygwin, MSYS or MSYS2. + +The instructions apply to versions of Windows supporting the Cygwin +emulated gnuish environment or the MSYS or MSYS2 ditto. We’ve built on +the following platforms: Windows 2012, Windows 7, Windows 8 and Windows 10. +It’s probably possible to build on older platforms too, but you might +not be able to install the appropriate Microsoft SDK, Visual Studio or +OpenSSL, in which case you will need to go back to earlier compilers etc. + +The procedure described uses either Cygwin, MSYS or MSYS2 as a build +environment. You run the bash shell in Cygwin/MSYS/MSYS2 and use the gnu +make/configure/autoconf etc to do the build. The emulator C-source code +is, however, mostly compiled with Microsoft Visual C++™, producing a +native Windows binary. This is the same procedure as we use to build the +pre-built binaries. Why we use VC++ and not gcc is explained further in +the FAQ section. + +If you are not familiar with Cygwin, MSYS, MSYS2 or a Unix environment, +you’ll probably need to read up a bit on how that works. There are plenty of +documentation about this online. + +These instructions apply for both 32-bit and 64-bit Windows. Note that even +if you build a 64-bit version of Erlang, most of the directories and files +involved are still named win32. Some occurances of the name win64 are +however present. The installation file for a 64-bit Windows version of +Erlang, for example, is `otp_win64_%OTP-REL%.exe`. + +If you feel comfortable with the environment and build +system, and have all the necessary tools, you have a great opportunity +to make the Erlang/OTP distribution for Windows better. Please submit +any suggestions to our [JIRA] [2] and patches to our [git project] [3] to let +them find their way into the next version of Erlang. If making changes +to the build system (like makefiles etc) please bear in mind that the +same makefiles are used on Unix/VxWorks, so that your changes +don't break other platforms. That of course goes for C-code too; system +specific code resides in the `$ERL_TOP/erts/emulator/sys/win32` and +`$ERL_TOP/erts/etc/win32` directories mostly. The +`$ERL_TOP/erts/emulator/beam` directory is for common code. + +We've used this build procedure for a couple of +releases, and it has worked fine for us. Still, there might be all +sorts of troubles on different machines and with different +setups. We'll try to give hints wherever we've encountered difficulties, +but please share your experiences by using the [erlang-questions] [1] +mailing list. We cannot, of course, help everyone with all +their issues, so please try to solve such issues and submit +solutions/workarounds. + +Lets go then! We’ll start with a short version of the setup procedure, +followed by some FAQ, and then we’ll go into more details of the setup. + + +Short Version +------------- + +In the following sections, we've described as much as we could about the +installation of the tools needed. Once the tools are installed, building +is quite easy. We have also tried to make these instructions understandable +for people with limited Unix experience. Cygwin/MSYS/MSYS2 is a whole new +environment to some Windows users, why careful explanation of environment +variables etc seemed to be in place. + +This is the short story though, for the experienced and impatient: + + * Get and install complete Cygwin (latest), complete MinGW with MSYS or + complete MSYS2 + + * Install Visual Studio 12.0 (2013) + + * Install Microsofts Windows SDK 8.1 + + * Get and install Sun's JDK 1.6.0 or later + + * Get and install NSIS 2.01 or later (up to 2.46 tried and working) + + * Get, build and install OpenSSL 0.9.8r or later (up to 1.0.2d + tried & working) with static libs. + + * Get the Erlang source distribution (from + <http://www.erlang.org/download.html>) and unpack with + Cygwin's/MSYS's/MSYS2's `tar`. + + * Set `ERL_TOP` to where you unpacked the source distribution + + * `$ cd $ERL_TOP` + + * Modify PATH and other environment variables so that all these tools + are runnable from a bash shell. Still standing in `$ERL_TOP`, issue + the following commands (for 32-bit Windows, remove the x64 from the + first row and change `otp_win64_%OTP-REL%` to `otp_win32_%OTP-REL%` on + the last row): + + $ eval `./otp_build env_win32 x64` + $ ./otp_build autoconf + $ ./otp_build configure + $ ./otp_build boot -a + $ ./otp_build release -a + $ ./otp_build installer_win32 + $ release/win32/otp_win64_%OTP-REL% /S + + Voila! `Start->Programs->Erlang OTP %OTP-REL%->Erlang` starts the Erlang + Windows shell. + + +Frequently Asked Questions +-------------------------- + +* Q: So, now I can build Erlang using GCC on Windows? + + A: No, unfortunately not. You'll need Microsoft's Visual C++ + still. A Bourne-shell script (cc.sh) wraps the Visual C++ compiler + and runs it from within the Cygwin environment. All other tools + needed to build Erlang are free-ware/open source, but not the C + compiler. The Windows SDK is however enough to build Erlang, you + do not need to buy Visual C++, just download the SDK (SDK version + 8.1 == Visual studio 2013). + +* Q: Why haven't you got rid of VC++ then, you \*\*\*\*\*\*? + + A: Well, partly because it's a good compiler - really! Actually it's + been possible in late R11-releases to build using mingw instead of + visual C++ (you might see the remnants of that in some scripts and + directories). Unfortunately the development of the SMP version for + Windows broke the mingw build and we chose to focus on the VC++ build + as the performance has been much better in the VC++ versions. The + mingw build will possibly be back, but as long as VC++ gives better + performance, the commercial build will be a VC++ one. + +* Q: OK, you need VC++, but now you've started to demand a quite recent + (and expensive) version of Visual Studio. Why? + + A: Well, it's not expensive, it's free (as in free beer). Just + download and install the latest Windows SDK from Microsoft and all + the tools you need are there. The included debugger (WinDbg) is + also quite usable. That's what I used when porting Erlang to 64bit + Windows. Another reason to use later Microsoft compilers is + DLL compatibility. DLL's using a new version of the standard + library might not load if the VM is compiled with an old VC++ + version. So we should aim to use the latest freely available SDK + and compiler. + +* Q: Can/will I build a Cygwin binary with the procedure you describe? + + A: No, the result will be a pure Windows binary, and as far as I know, + it's not possible to make a Cygwin binary yet. That is of course + something desirable, but there are still some problems with the + dynamic linking (dynamic Erlang driver loading) as well as the TCP/IP + emulation in Cygwin, which, I'm sure of, will improve, but still has + some problems. Fixing those problems might be easy or might be hard. + I suggest you try yourself and share your experience. No one would be + happier if a simple `./configure && make` would produce a fully fledged + Cygwin binary. + +* Q: Hah, I saw you, you used GCC even though you said you didn't! + + A: OK, I admit, one of the files is compiled using Cygwin's or + MinGW's GCC and the resulting object code is then converted to MS + VC++ compatible coff using a small C hack. It's because that + particular file, `beam_emu.c` benefits immensely from being able + to use the GCC labels-as-values extension, which boosts emulator + performance by up to 50%. That does unfortunately not (yet) mean + that all of OTP could be compiled using GCC. That particular + source code does not do anything system specific and actually is + adopted to the fact that GCC is used to compile it on Windows. + +* Q: So now there's a MS VC++ project file somewhere and I can build OTP + using the nifty VC++ GUI? + + A: No, never. The hassle of keeping the project files up to date and + do all the steps that constitute an OTP build from within the VC++ GUI + is simply not worth it, maybe even impossible. A VC++ project + file for Erlang/OTP will never happen. + +* Q: So how does it all work then? + + A: Cygwin, MSYS or MSYS2 is the environment, which closely resembles the + environment found on any Unix machine. It's almost like you had a + virtual Unix machine inside Windows. Configure, given certain + parameters, then creates makefiles that are used by the + environment's gnu-make to built the system. Most of the actual + compilers etc are not, however, Cygwin/MSYS/MSYS2 tools, so we've written + a couple of wrappers (Bourne-shell scripts), which reside in + `$ERL_TOP/etc/win32/cygwin_tools` and + `$ERL_TOP/etc/win32/msys_tools`. They all do conversion of + parameters and switches common in the Unix environment to fit the + native Windows tools. Most notable is of course the paths, which + in Cygwin/MSYS/MSYS2 are Unix-like paths with "forward slashes" (/) and + no drive letters. The Cygwin specific command `cygpath` is used + for most of the path conversions in a Cygwin environment. Other + tools are used (when needed) in the corresponding MSYS and MSYS2 + environment. Luckily most compilers accept forward slashes instead + of backslashes as path separators, but one still have to get the drive + letters etc right, though. The wrapper scripts are not general in + the sense that, for example, cc.sh would understand and translate + every possible gcc option and pass correct options to + cl.exe. The principle is that the scripts are powerful enough to + allow building of Erlang/OTP, no more, no less. They might need + extensions to cope with changes during the development of Erlang, and + that's one of the reasons we made them into shell-scripts and not + Perl-scripts. We believe they are easier to understand and change + that way. + + In `$ERL_TOP`, there is a script called `otp_build`. That script handles + the hassle of giving all the right parameters to `configure`/`make` and + also helps you set up the correct environment variables to work with + the Erlang source under Cygwin/MSYS/MSYS2. + +* Q: You use and need Cygwin, but then you haven't taken the time to + port Erlang to the Cygwin environment but instead focus on your + commercial release, is that really ethical? + + A: No, not really, but see this as a step in the right direction. + +* Q: Can I build something that looks exactly as the commercial release? + + A: Yes, we use the exact same build procedure. + +* Q: Which version of Cygwin/MSYS/MSYS2 and other tools do you use then? + + A: For Cygwin, MSYS and MSYS2 alike, we try to use the latest releases + available when building. What versions you use shouldn't really + matter. We try to include workarounds for the bugs we've found in + different Cygwin/MSYS/MSYS2 releases. Please help us add workarounds + for new Cygwin/MSYS/MSYS2-related bugs as soon as you encounter + them. Also please do submit bug reports to the appropriate Cygwin, MSYS + and/or MSYS2 developers. The GCC we used for %OTP-REL% was version + 4.8.1 (MinGW 32bit) and 4.8.5 (MSYS2 64bit). We used VC++ 12.0 + (i.e. Visual studio 2013), Sun's JDK 1.6.0\_45 (32bit) and Sun's + JDK 1.7.0\_1 (64bit), NSIS 2.46, and Win32 OpenSSL 1.0.2d. Please + read the next section for details on what you need. + +* Q: Can you help me setup X in Cygwin/MSYS/MSYS2? + + A: No, unfortunately we haven't got time to help with Cygwin/MSYS/MSYS2 + related user problems, please read related websites, newsgroups and + mailing lists. + + +Tools you Need and Their Environment +------------------------------------ + +You need some tools to be able to build Erlang/OTP on Windows. Most +notably you'll need Cygwin, MSYS or MSYS2, Visual Studio and Microsofts +Windows SDK, but you might also want a Java compiler, the NSIS install +system and OpenSSL. Well, here's some information about the different +tools: + +* Cygwin, the very latest is usually best. Get all the development + tools and of course all the basic ditto. Make sure to get jar and + also make sure *not* to install a Cygwin'ish Java, since the Cygwin + jar command is used but Sun's Java compiler and virtual machine. + + If you are going to build a 64bit Windows version, you should make + sure to get MinGW's 64bit gcc installed with Cygwin. It's in one of + the development packages. + + URL: <http://www.cygwin.com> + + Get the installer from the website and use it to install + Cygwin. Be sure to have fair privileges. If you're on an NT domain you + should consider running `mkpasswd -d` and `mkgroup -d` after the + installation to get the user databases correct. See their respective + manual pages. + + When you start your first bash shell, you will get an awful prompt. You + might also have a `PATH` environment variable that contains backslashes + and such. Edit `$HOME/.profile` and `$HOME/.bashrc` to set fair prompts + and a correct PATH. Also do an `export SHELL` in `.profile`. For some + non-obvious reason the environment variable `$SHELL` is not exported in + bash. Also note that `.profile` is run at login time and `.bashrc` when + sub shells are created. You'll need to explicitly source `.bashrc` from + `.profile` if you want the commands there to be run at login time (like + setting up aliases, shell functions and the like). You can for example + do like this at the end of `.profile`: + + ENV=$HOME/.bashrc + export ENV + . $ENV + + You might also want to setup X-windows (XFree86). That might be as easy + as running startx from the command prompt and it might be much harder. + Use Google to find help. + + If you don't use X-windows, you might want to setup the Windows + console window by selecting properties in the console system menu + (upper left corner of the window, the Cygwin icon in the title + bar). Especially setting a larger screen buffer size (lines) is useful + as it gets you a scrollbar so you can see whatever error messages + that might appear. + + There are a few other shells available, but in all examples below we assume + that you use bash. + +* Alternatively you download MinGW and MSYS. You'll find the latest + installer at: + + URL: <http://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/> + + Make sure to install the basic dev tools, but avoid the MinGW autoconf and + install the msys one instead. + + To be able to build the 64bit VM, you will also need the 64bit + MinGW compiler from: + + URL: <http://sourceforge.net/projects/mingw-w64/files/latest/download?source=files> + + We've tried up to 1.0, but the latest version should do. Make sure you + download the `mingw-w64-bin_i686-mingw_<something>.zip`, not a linux + version. You unzip the package on top of your MinGW installation + (`c:\MinGW`) and that's it. + +* A third alternative is to download and install MSYS2 from: + + URL: <https://msys2.github.io/> + + When you've followed the instructions there, you also need to install + these packages: autoconf, make, perl, and tar. You do so by running + the following in the msys console: + + pacman -S msys/autoconf msys/make msys/perl msys/tar + + You also need a gcc. If you installed the 64 bit MSYS2 you run: + + mingw64/mingw-w64-x86_64-gcc + + And for 32 bit MSYS2: + + pacman -S mingw32/mingw-w64-i686-gcc + pacman -S mingw-w64-i686-editrights + +* Visual Studio 2013 (Visual Studio 12.0). Download and run the web + installer from: + + https://www.visualstudio.com/ + +* Microsofts Windows SDK version 8.1 (corresponding to VC++ 12.0 and + Visual Studio 2013). You'll find it here: + + URL: <https://msdn.microsoft.com/en-us/windows/desktop/bg162891.aspx> + +* To help setup the environment, there is a bat file, + `%PROGRAMFILES%\Mirosoft Visual Studio 12.0\VC\vcvarsall.bat`, + that set's the appropriate + environment for a Windows command prompt. This is not appropriate + for bash, so you'll need to convert it to bash-style environments + by editing your `.bash_profile`. In my case, where the SDK is + installed in the default directory and `%PROGRAMFILES%` is + `C:\Program Files`, the commands for setting up a 32bit build + environment (on a 64bit or 32bit machine) look like this (in Cygwin): + + # Some common paths + C_DRV=/cygdrive/c + PRG_FLS=$C_DRV/Program\ Files + + # nsis + NSIS_BIN=$PRG_FLS/NSIS + # java + JAVA_BIN=$PROGRAMFILES/Java/jdk1.7.0_02/bin + + ## + ## MS SDK + ## + + CYGWIN=nowinsymlinks + + VISUAL_STUDIO_ROOT=$PRG_FLS/Microsoft\ Visual\ Studio\ 12.0 + WIN_VISUAL_STUDIO_ROOT="C:\\Program Files\\Microsoft Visual Studio 12.0" + SDK=$PRG_FLS/Windows\ Kits/8.1 + WIN_SDK="C:\\Program Files\\Windows Kits\\8.1" + + PATH="$NSIS_BIN:\ + $VISUAL_STUDIO_ROOT/VC/bin:\ + $VISUAL_STUDIO_ROOT/VC/vcpackages:\ + $VISUAL_STUDIO_ROOT/Common7/IDE:\ + $VISUAL_STUDIO_ROOT/Common7/Tools:\ + $SDK/bin/x86 + /usr/local/bin:/usr/bin:/bin:\ + /cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:\ + /cygdrive/c/WINDOWS/system32/Wbem:\ + $JAVA_BIN" + + LIBPATH="$WIN_VISUAL_STUDIO_ROOT\\VC\\lib" + + LIB="$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\;$WIN_SDK\\lib\\winv6.3\\um\\x86" + + INCLUDE="$WIN_VISUAL_STUDIO_ROOT\\VC\\include\\;$WIN_SDK\\include\\shared\\;\ + $WIN_SDK\\include\\um;$WIN_SDK\\include\\winrt\\;$WIN_SDK\\include\\um\\gl" + + export CYGWIN PATH LIBPATH LIB INCLUDE + + If you're using MinGW's MSYS instead, you need to change the `C_DRV` setting, + which would read: + + C_DRV=/c + + and you also need to change the PATH environment variable to: + + MINGW_BIN=/c/MinGW/bin + + + PATH="$NSIS_BIN:\ + $VISUAL_STUDIO_ROOT/VC/bin:\ + $VISUAL_STUDIO_ROOT/VC/vcpackages:\ + $VISUAL_STUDIO_ROOT/Common7/IDE:\ + $VISUAL_STUDIO_ROOT/Common7/Tools:\ + $SDK/bin/x86:/usr/local/bin:\ + $MINGW_BIN:\ + /bin:/c/Windows/system32:/c/Windows:\ + /c/Windows/System32/Wbem:\ + $JAVA_BIN" + + For MSYS2 you use the same `C_DRV` and PATH as for MSYS, only update the `MINGW_BIN`: + + MINGW_BIN=/mingw32/bin + + + If you are building a 64 bit version of Erlang, you should set up + PATHs etc a little differently. We have two templates to make things + work in both Cygwin and MSYS but needs editing to work with MSYS2 (see the + comments in the script). + The following one is for 32 bits: + + make_winpath() + { + P=$1 + if [ "$IN_CYGWIN" = "true" ]; then + cygpath -d "$P" + else + (cd "$P" && /bin/cmd //C "for %i in (".") do @echo %~fsi") + fi + } + + make_upath() + { + P=$1 + if [ "$IN_CYGWIN" = "true" ]; then + cygpath "$P" + else + echo "$P" | /bin/sed 's,^\([a-zA-Z]\):\\,/\L\1/,;s,\\,/,g' + fi + } + + # Some common paths + if [ -x /usr/bin/msys-?.0.dll ]; then + # Without this the path conversion won't work + COMSPEC='C:\Windows\System32\cmd.exe' + MSYSTEM=MINGW32 # Comment out this line if in MSYS2 + export MSYSTEM COMSPEC + # For MSYS2: Change /mingw/bin to the msys bin dir on the line below + PATH=/usr/local/bin:/mingw/bin:/bin:/c/Windows/system32:\ + /c/Windows:/c/Windows/System32/Wbem + C_DRV=/c + IN_CYGWIN=false + else + PATH=/ldisk/overrides:/usr/local/bin:/usr/bin:/bin:\ + /usr/X11R6/bin:/cygdrive/c/windows/system32:\ + /cygdrive/c/windows:/cygdrive/c/windows/system32/Wbem + C_DRV=/cygdrive/c + IN_CYGWIN=true + fi + + obe_otp_gcc_vsn_map=" + .*=>default + " + obe_otp_64_gcc_vsn_map=" + .*=>default + " + # Program Files + PRG_FLS=$C_DRV/Program\ Files + + # Visual Studio + VISUAL_STUDIO_ROOT=$PRG_FLS/Microsoft\ Visual\ Studio\ 12.0 + WIN_VISUAL_STUDIO_ROOT="C:\\Program Files\\Microsoft Visual Studio 12.0" + + # SDK + SDK=$PRG_FLS/Windows\ Kits/8.1 + WIN_SDK="C:\\Program Files\\Windows Kits\\8.1" + + # NSIS + NSIS_BIN=$PROGRAMFILES/NSIS + + # Java + JAVA_BIN=$PROGRAMFILES/Java/jdk1.7.0_02/bin + + ## The PATH variable should be Cygwin'ish + VCPATH= + $VISUAL_STUDIO_ROOT/VC/bin:\ + $VISUAL_STUDIO_ROOT/VC/vcpackages:\ + $VISUAL_STUDIO_ROOT/Common7/IDE:\ + $VISUAL_STUDIO_ROOT/Common7/Tools:\ + $SDK/bin/x86 + + ## Microsoft SDK libs + LIBPATH=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib + + LIB=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\;$WIN_KITS\\lib\\winv6.3\\um\\x86 + + INCLUDE=$WIN_VISUAL_STUDIO_ROOT\\VC\\include\\;\ + $WIN_KITS\\include\\shared\\;$WIN_KITS\\include\\um;\ + $WIN_KITS\\include\\winrt\\;$WIN_KITS\\include\\um\\gl + + # Put nsis, c compiler and java in path + export PATH=$VCPATH:$PATH:$JAVA_BIN:$NSIS_BIN + + # Make sure LIB and INCLUDE is available for others + export LIBPATH LIB INCLUDE + + + + The first part of the 64 bit template is identical to the 32 bit one, + but there are some environment variable differences: + + # Program Files + PRG_FLS64=$C_DRV/Program\ Files + PRG_FLS32=$C_DRV/Program\ Files\ \(x86\) + + # Visual Studio + VISUAL_STUDIO_ROOT=$PRG_FLS32/Microsoft\ Visual\ Studio\ 12.0 + WIN_VISUAL_STUDIO_ROOT="C:\\Program Files (x86)\\Microsoft Visual Studio 12.0" + + # SDK + SDK=$PRG_FLS32/Windows\ Kits/8.1 + WIN_SDK="C:\\Program Files (x86)\\Windows Kits\\8.1" + + # NSIS + NSIS_BIN=$PROGRAMFILES/NSIS + # Java + JAVA_BIN=$PROGRAMFILES/Java/jdk1.7.0_02/bin + + ## The PATH variable should be Cygwin'ish + VCPATH= + $VISUAL_STUDIO_ROOT/VC/bin/amd64:\ + $VISUAL_STUDIO_ROOT/VC/vcpackages:\ + $VISUAL_STUDIO_ROOT/Common7/IDE:\ + $VISUAL_STUDIO_ROOT/Common7/Tools:\ + $SDK/bin/x86 + + ## Microsoft SDK libs + LIBPATH=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\amd64 + + LIB=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\amd64\\;\ + $WIN_KITS\\lib\\winv6.3\\um\\x64 + + INCLUDE=$WIN_VISUAL_STUDIO_ROOT\\VC\\include\\;\ + $WIN_KITS\\include\\shared\\;$WIN_KITS\\include\\um;\ + $WIN_KITS\\include\\winrt\\;$WIN_KITS\\include\\um\\gl + + # Put nsis, c compiler and java in path + export PATH=$VCPATH:$PATH:$JAVA_BIN:$NSIS_BIN + + # Make sure LIB and INCLUDE is available for others + export LIBPATH LIB INCLUDE + + + Make sure to set the PATH so that NSIS and Microsoft SDK is found + before the MSYS/Cygwin tools and that Java is last in the PATH. + + Make a simple hello world and try to compile it with the `cl` + command from within bash. If that does not work, your environment + needs fixing. Remember, there should be + no backslashes in your path environment variable in Cygwin bash, + but LIB and INCLUDE should contain Windows style paths with + semicolon, drive letters and backslashes. + +* Sun's Java JDK 1.6.0 or later. Our Java code (jinterface, ic) is + written for JDK 1.6.0. Get it for Windows and install it, the JRE is + not enough. If you don't care about Java, you can skip this step. The + result will be that jinterface is not built. + + URL: <http://java.sun.com> + + Add javac *LAST* to your path environment in bash, in my case this means: + + `PATH="$PATH:/cygdrive/c/Program Files/Java/jdk1.7.0_02/bin"` + + No `CLASSPATH` or anything is needed. Type `javac` in the bash prompt + and you should get a list of available Java options. Make sure, e.g by + typing `type java`, that you use the Java you installed. Note however that + Cygwin's/MinGW's/MSYS2's `jar.exe` is used. That's why the JDK bin-directory should be + added last in the `PATH`. + +* Nullsoft NSIS installer system. You need this to build the self + installing package. It's a free open source installer that's much + nicer to use than the commercial Wise and Install shield + installers. This is the installer we use for commercial releases as + well. + + URL: <http://nsis.sourceforge.net/download> + + Install the lot, especially the modern user interface components, as + it's definitely needed. Put `makensis` in your path, in my case: + + PATH=/cygdrive/c/Program\ Files/NSIS:$PATH + + Type makensis at the bash prompt and you should get a list of options + if everything is OK. + +* OpenSSL. This is if you want the SSL and crypto applications to + compile (and run). There are prebuilt binaries, which you can just + download and install, available here: + + URL: <http://openssl.org/community/binaries.html> + + We would recommend using 1.0.2d. + +* Building with wxWidgets. Download wxWidgets-3.0.3 or higher. + + Install or unpack it to the pgm folder: + Cygwin: + `DRIVE:/PATH/cygwin/opt/local/pgm` + MSYS: + `DRIVE:/PATH/MinGW/msys/1.0/opt/local/pgm` + MSYS2: + `DRIVE:/PATH/msys<32/64>/opt/local/pgm` + + If the `wxUSE_POSTSCRIPT` isn't enabled in `<path\to\pgm>\wxMSW-3.0.3\include\wx\msw\setup.h`, + enable it. + + build: From a command prompt with the VC tools available (See the + instructions for OpenSSL build above for help on starting the + proper command prompt in RELEASE mode): + + C:\...\> cd <path\to\pgm>\wxMSW-3.0.3\build\msw + C:\...\> nmake BUILD=release SHARED=0 DIR_SUFFIX_CPU= -f makefile.vc + + Or - if building a 64bit version: + + C:\...\> cd <path\to\pgm>\wxMSW-3.0.3\build\msw + C:\...\> nmake TARGET_CPU=amd64 BUILD=release SHARED=0 DIR_SUFFIX_CPU= -f makefile.vc + +* Get the Erlang source distribution (from <http://www.erlang.org/download.html>). + The same as for Unix platforms. Preferably use tar from within Cygwin, MSYS or MSYS2 to + unpack the source tar.gz (`tar zxf otp_src_%OTP-REL%.tar.gz`). + + Set the environment `ERL_TOP` to point to the root directory of the + source distribution. Let's say I stood in `$HOME/src` and unpacked + `otp_src_%OTP-REL%.tar.gz`, I then add the following to `.profile`: + + ERL_TOP=$HOME/src/otp_src_%OTP-REL% + export $ERL_TOP + + +The Shell Environment +--------------------- + +So, if you have followed the instructions above, when you start a bash +shell, you should have an INCLUDE environment with a Windows style +path, a LIB environment variable also in Windows style, and finally a +PATH that let's you reach cl, makensis, javac etc from the +command prompt (use `which cl` etc to verify from bash). + +You should also have an `ERL_TOP` environment variable that is *Cygwin +style*, and points to a directory containing, among other files, the +script `otp_build`. + +A final massage of the environment is needed, and that is done by +the script `$ERL_TOP/otp_build`. Start bash and do the following, note +the "back-ticks" (\`), can be quite hard to get on some keyboards, but +pressing the back-tick key followed by the space bar might do it... + + $ cd $ERL_TOP + $ eval `./otp_build env_win32` + +If you're unable to produce back-ticks on your keyboard, you can use +the ksh variant: + + $ cd $ERL_TOP + $ eval $(./otp_build env_win32) + +If you are building a 64 bit version, you supply `otp_build` with an architecture parameter: + + $ cd $ERL_TOP + $ eval `./otp_build env_win32 x64` + + +This should do the final touch to the environment and building should +be easy after this. You could run `./otp_build env_win32` without +`eval` just to see what it does, and to see that the environment it +sets seems OK. The path is cleaned of spaces if possible (using DOS +style short names instead), the variables `OVERRIDE_TARGET`, `CC`, `CXX`, +`AR` and `RANLIB` are set to their respective wrappers and the directories +`$ERL_TOP/erts/etc/win32/<cygwin/msys>_tools/vc` and +`$ERL_TOP/erts/etc/win32/<cygwin/msys>_tool` are added first in the PATH. + +Now you can check which erlc you have by writing `type erlc` in your shell. +It should reside in `$ERL_TOP/erts/etc/win32/cygwin_tools` +or `$ERL_TOP/erts/etc/win32/msys_tools`. + + +Building and Installing +----------------------- + +Building is easiest using the `otp_build` script: + + $ ./otp_build autoconf # Ignore the warning blob about versions of autoconf + $ ./otp_build configure <optional configure options> + $ ./otp_build boot -a + $ ./otp_build release -a <installation directory> + $ ./otp_build installer_win32 <installation directory> # optional + +Now you will have a file called `otp_win32_%OTP-REL%.exe` or `otp_win64_%OTP-REL%.exe` +in the `<installation directory>`, i.e. `$ERL_TOP/release/win32`. + +Lets get into more detail: + +1. `$ ./otp_build autoconf` - This step rebuilds the configure scripts + to work correctly in your environment. In an ideal world, this + would not be needed, but alas, we have encountered several + incompatibilities between our distributed configure scripts (generated + on a Linux platform) and the Cygwin/MSYS/MSYS2 environment over the + years. Running autoconf in Cygwin/MSYS/MSYS2 ensures that the configure + scripts are generated in a compatible way and that they will work well + in the next step. + +2. `$ ./otp_build configure` - This runs the newly generated configure + scripts with options making configure behave nicely. The target machine + type is plainly `win32`, so a lot of the configure-scripts recognize + this awkward target name and behave accordingly. The CC variable also + makes the compiler be `cc.sh`, which wraps MSVC++, so all configure + tests regarding the C compiler gets to run the right compiler. A lot of + the tests are not needed on Windows, but we thought it best to run the + whole configure anyway. + +3. `$ ./otp_build boot -a` - This uses the bootstrap directory (shipped + with the source, `$ERL_TOP/bootstrap`) to build a complete OTP + system. When this is done you can run erl from within the source tree; + just type `$ERL_TOP/bin/erl` and you whould have the prompt. + +4. `$ ./otp_build release -a` - Builds a commercial release tree from the + source tree. The default is to put it in `$ERL_TOP/release/win32`. You can + give any directory as parameter (Cygwin style), but it doesn't really + matter if you're going to build a self extracting installer too. + +5. `$ ./otp_build installer_win32` - Creates the self extracting installer executable. + The executable `otp_win32_%OTP-REL%.exe` or `otp_win64_%OTP-REL%.exe` will be placed + in the top directory of the release created in the previous step. If + no release directory is specified, the release is expected to have + been built to `$ERL_TOP/release/win32`, which also will be the place + where the installer executable will be placed. If you specified some + other directory for the release (i.e. `./otp_build release -a + /tmp/erl_release`), you're expected to give the same parameter here, + (i.e. `./otp_build installer_win32 /tmp/erl_release`). You need to have + a full NSIS installation and `makensis.exe` in your path for this to + work. Once you have created the installer, you can run it to + install Erlang/OTP in the regular way, just run the executable and + follow the steps in the installation wizard. To get all default settings + in the installation without any questions asked, you run the executable + with the parameter `/S` (capital S) like in: + + $ cd $ERL_TOP + $ release/win32/otp_win32_%OTP-REL% /S + ... + + or + + $ cd $ERL_TOP + $ release/win32/otp_win64_%OTP-REL% /S + ... + + + and after a while Erlang/OTP-%OTP-REL% will have been installed in + `C:\Program Files\erl%ERTS-VSN%\`, with shortcuts in the menu etc. + + +Development +----------- + +Once the system is built, you might want to change it. Having a test +release in some nice directory might be useful, but you can also run +Erlang from within the source tree. The target `local_setup`, makes +the program `$ERL_TOP/bin/erl.exe` usable and it also uses all the OTP +libraries in the source tree. + +If you hack the emulator, you can build the emulator executable +by standing in `$ERL_TOP/erts/emulator` and do a simple + + $ make opt + +Note that you need to have run ``(cd $ERL_TOP && eval `./otp_build env_win32`)`` +in the particular shell before building anything on Windows. After +doing a make opt you can test your result by running `$ERL_TOP/bin/erl`. +If you want to copy the result to a release directory (say +`/tmp/erl_release`), you do this (still in `$ERL_TOP/erts/emulator`) + + $ make TESTROOT=/tmp/erl_release release + +That will copy the emulator executables. + +To make a debug build of the emulator, you need to recompile both +`beam.dll` (the actual runtime system) and `erlexec.dll`. Do like this + + $ cd $ERL_TOP + $ rm bin/win32/erlexec.dll + $ cd erts/emulator + $ make debug + $ cd ../etc + $ make debug + +and sometimes + + $ cd $ERL_TOP + $ make local_setup + +So now when you run `$ERL_TOP/erl.exe`, you should have a debug compiled +emulator, which you will see if you do a: + + 1> erlang:system_info(system_version). + +in the erlang shell. If the returned string contains `[debug]`, you +got a debug compiled emulator. + +To hack the erlang libraries, you simply do a `make opt` in the +specific "applications" directory, like: + + $ cd $ERL_TOP/lib/stdlib + $ make opt + +or even in the source directory... + + $ cd $ERL_TOP/lib/stdlib/src + $ make opt + +Note that you're expected to have a fresh Erlang in your path when +doing this, preferably the plain %OTP-REL% you have built in the previous +steps. You could also add `$ERL_TOP/bootstrap/bin` to your `PATH` before +rebuilding specific libraries. That would give you a good enough +Erlang system to compile any OTP erlang code. Setting up the path +correctly is a little bit tricky. You still need to have +`$ERL_TOP/erts/etc/win32/cygwin_tools/vc` and +`$ERL_TOP/erts/etc/win32/cygwin_tools` *before* the actual emulator +in the path. A typical setting of the path for using the bootstrap +compiler would be: + + $ export PATH=$ERL_TOP/erts/etc/win32/cygwin_tools/vc\ + :$ERL_TOP/erts/etc/win32/cygwin_tools:$ERL_TOP/bootstrap/bin:$PATH + +That should make it possible to rebuild any library without hassle... + +If you want to copy a library (an application) newly built, to a +release area, you do like with the emulator: + + $ cd $ERL_TOP/lib/stdlib + $ make TESTROOT=/tmp/erlang_release release + +Remember that: + +* Windows specific C-code goes in the `$ERL_TOP/erts/emulator/sys/win32`, + `$ERL_TOP/erts/emulator/drivers/win32` or `$ERL_TOP/erts/etc/win32`. + +* Windows specific erlang code should be used conditionally and the + host OS tested in *runtime*, the exactly same beam files should be + distributed for every platform! So write code like: + + case os:type() of + {win32,_} -> + do_windows_specific(); + Other -> + do_fallback_or_exit() + end, + +That's basically all you need to get going. + + +Using GIT +--------- + +You might want to check out versions of the source code from GitHUB. That is possible directly in Cygwin, but not in MSYS. There is a project MsysGIT: + +URL:<http://code.google.com/p/msysgit/> + +that makes a nice Git port. The msys prompt you get from MsysGIT is +however not compatible with the full version from MinGW, so you will +need to check out files using MsysGIT's command prompt and then switch +to a common MSYS command prompt for building. Also all test suites +cannot be built as MsysGIT/MSYS does not handle symbolic links. + + + [1]: http://www.erlang.org/static/doc/mailinglist.html + [2]: http://bugs.erlang.org + [3]: https://github.com/erlang/otp + + [?TOC]: true diff --git a/HOWTO/INSTALL-WIN32.md b/HOWTO/INSTALL-WIN32.md index 98c608060d..aeb5cfa1cb 100644 --- a/HOWTO/INSTALL-WIN32.md +++ b/HOWTO/INSTALL-WIN32.md @@ -4,40 +4,31 @@ How to Build Erlang/OTP on Windows Introduction ------------ -This section describes how to build the Erlang emulator and the OTP -libraries on Windows. Note that the Windows binary releases are still -a preferred alternative if one does not have Microsoft’s development -tools and/or don’t want to install Cygwin, MSYS or MSYS2. - -The instructions apply to versions of Windows supporting the Cygwin -emulated gnuish environment or the MSYS or MSYS2 ditto. We’ve built on -the following platforms: Windows 2012, Windows 7, Windows 8 and Windows 10. -It’s probably possible to build on older platforms too, but you might -not be able to install the appropriate Microsoft SDK, Visual Studio or -OpenSSL, in which case you will need to go back to earlier compilers etc. - -The procedure described uses either Cygwin, MSYS or MSYS2 as a build -environment. You run the bash shell in Cygwin/MSYS/MSYS2 and use the gnu -make/configure/autoconf etc to do the build. The emulator C-source code -is, however, mostly compiled with Microsoft Visual C++™, producing a -native Windows binary. This is the same procedure as we use to build the -pre-built binaries. Why we use VC++ and not gcc is explained further in -the FAQ section. - -If you are not familiar with Cygwin, MSYS, MSYS2 or a Unix environment, -you’ll probably need to read up a bit on how that works. There are plenty of -documentation about this online. - -These instructions apply for both 32-bit and 64-bit Windows. Note that even -if you build a 64-bit version of Erlang, most of the directories and files -involved are still named win32. Some occurances of the name win64 are -however present. The installation file for a 64-bit Windows version of -Erlang, for example, is `otp_win64_%OTP-REL%.exe`. +This section describes how to build the Erlang emulator and the OTP +libraries on Windows. Note that the Windows binary releases are still +a preferred alternative if one does not have Microsoft’s development +tools and/or don’t want to install WSL. + +The instructions apply to Windows 10 (v.1809 and later) supporting the +WSL.1 (Windows Subsystem for Linux v.1) and using Ubuntu 18.04 release. + +The procedure described uses WSL as a build environment. You run the +bash shell in WSL and use the gnu make/configure/autoconf etc to do +the build. The emulator C-source code is, however, mostly compiled +with Microsoft Visual C++™, producing a native Windows binary. This is +the same procedure as we use to build the pre-built binaries. Why we +use VC++ and not gcc is explained further in the FAQ section. + +These instructions apply for both 32-bit and 64-bit Windows. Note that +even if you build a 64-bit version of Erlang, most of the directories +and files involved are still named win32. Some occurrences of the name +win64 are however present. The installation file for a 64-bit Windows +version of Erlang, for example, is `otp_win64_%OTP-REL%.exe`. If you feel comfortable with the environment and build system, and have all the necessary tools, you have a great opportunity to make the Erlang/OTP distribution for Windows better. Please submit -any suggestions to our [JIRA] [2] and patches to our [git project] [3] to let +any suggestions to our [JIRA] [1] and patches to our [git project] [2] to let them find their way into the next version of Erlang. If making changes to the build system (like makefiles etc) please bear in mind that the same makefiles are used on Unix/VxWorks, so that your changes @@ -46,56 +37,44 @@ specific code resides in the `$ERL_TOP/erts/emulator/sys/win32` and `$ERL_TOP/erts/etc/win32` directories mostly. The `$ERL_TOP/erts/emulator/beam` directory is for common code. -We've used this build procedure for a couple of -releases, and it has worked fine for us. Still, there might be all -sorts of troubles on different machines and with different -setups. We'll try to give hints wherever we've encountered difficulties, -but please share your experiences by using the [erlang-questions] [1] -mailing list. We cannot, of course, help everyone with all -their issues, so please try to solve such issues and submit -solutions/workarounds. - -Lets go then! We’ll start with a short version of the setup procedure, -followed by some FAQ, and then we’ll go into more details of the setup. - Short Version ------------- -In the following sections, we've described as much as we could about the -installation of the tools needed. Once the tools are installed, building -is quite easy. We have also tried to make these instructions understandable -for people with limited Unix experience. Cygwin/MSYS/MSYS2 is a whole new -environment to some Windows users, why careful explanation of environment -variables etc seemed to be in place. +In the following sections, we've described as much as we could about +the installation of the tools needed. Once the tools are installed, +building is quite easy. We have also tried to make these instructions +understandable for people with limited Unix experience. WSL is a whole +new environment to some Windows users, why careful explanation of +environment variables etc seemed to be in place. This is the short story though, for the experienced and impatient: - * Get and install complete Cygwin (latest), complete MinGW with MSYS or - complete MSYS2 + * Get and install complete WSL environment - * Install Visual Studio 12.0 (2013) + * Install Visual Studio 2019 - * Install Microsofts Windows SDK 8.1 + * Get and install windows JDK-8 - * Get and install Sun's JDK 1.6.0 or later + * Get and install windows NSIS 3.05 or later (3.05 tried and working) - * Get and install NSIS 2.01 or later (up to 2.46 tried and working) + * Get, build and install OpenSSL v1.1.1d or later (up to 1.1.1d + tried & working) with static libs. - * Get, build and install OpenSSL 0.9.8r or later (up to 1.0.2d + * Get, build and install wxWidgets-3.1.3 or later (up to 3.1.3 tried & working) with static libs. * Get the Erlang source distribution (from - <http://www.erlang.org/download.html>) and unpack with - Cygwin's/MSYS's/MSYS2's `tar`. + <http://www.erlang.org/download.html>) and unpack with `tar` + to the windows disk for example to: /mnt/c/src/ - * Set `ERL_TOP` to where you unpacked the source distribution + * Install mingw-gcc: `sudo apt install gcc-mingw-w64` - * `$ cd $ERL_TOP` + * `$ cd UNPACK_DIR` * Modify PATH and other environment variables so that all these tools are runnable from a bash shell. Still standing in `$ERL_TOP`, issue - the following commands (for 32-bit Windows, remove the x64 from the + the following commands (for 32-bit Windows, remove the x64 from the first row and change `otp_win64_%OTP-REL%` to `otp_win32_%OTP-REL%` on the last row): @@ -111,576 +90,110 @@ This is the short story though, for the experienced and impatient: Windows shell. -Frequently Asked Questions --------------------------- - -* Q: So, now I can build Erlang using GCC on Windows? - - A: No, unfortunately not. You'll need Microsoft's Visual C++ - still. A Bourne-shell script (cc.sh) wraps the Visual C++ compiler - and runs it from within the Cygwin environment. All other tools - needed to build Erlang are free-ware/open source, but not the C - compiler. The Windows SDK is however enough to build Erlang, you - do not need to buy Visual C++, just download the SDK (SDK version - 8.1 == Visual studio 2013). - -* Q: Why haven't you got rid of VC++ then, you \*\*\*\*\*\*? - - A: Well, partly because it's a good compiler - really! Actually it's - been possible in late R11-releases to build using mingw instead of - visual C++ (you might see the remnants of that in some scripts and - directories). Unfortunately the development of the SMP version for - Windows broke the mingw build and we chose to focus on the VC++ build - as the performance has been much better in the VC++ versions. The - mingw build will possibly be back, but as long as VC++ gives better - performance, the commercial build will be a VC++ one. - -* Q: OK, you need VC++, but now you've started to demand a quite recent - (and expensive) version of Visual Studio. Why? - - A: Well, it's not expensive, it's free (as in free beer). Just - download and install the latest Windows SDK from Microsoft and all - the tools you need are there. The included debugger (WinDbg) is - also quite usable. That's what I used when porting Erlang to 64bit - Windows. Another reason to use later Microsoft compilers is - DLL compatibility. DLL's using a new version of the standard - library might not load if the VM is compiled with an old VC++ - version. So we should aim to use the latest freely available SDK - and compiler. - -* Q: Can/will I build a Cygwin binary with the procedure you describe? - - A: No, the result will be a pure Windows binary, and as far as I know, - it's not possible to make a Cygwin binary yet. That is of course - something desirable, but there are still some problems with the - dynamic linking (dynamic Erlang driver loading) as well as the TCP/IP - emulation in Cygwin, which, I'm sure of, will improve, but still has - some problems. Fixing those problems might be easy or might be hard. - I suggest you try yourself and share your experience. No one would be - happier if a simple `./configure && make` would produce a fully fledged - Cygwin binary. - -* Q: Hah, I saw you, you used GCC even though you said you didn't! - - A: OK, I admit, one of the files is compiled using Cygwin's or - MinGW's GCC and the resulting object code is then converted to MS - VC++ compatible coff using a small C hack. It's because that - particular file, `beam_emu.c` benefits immensely from being able - to use the GCC labels-as-values extension, which boosts emulator - performance by up to 50%. That does unfortunately not (yet) mean - that all of OTP could be compiled using GCC. That particular - source code does not do anything system specific and actually is - adopted to the fact that GCC is used to compile it on Windows. - -* Q: So now there's a MS VC++ project file somewhere and I can build OTP - using the nifty VC++ GUI? - - A: No, never. The hassle of keeping the project files up to date and - do all the steps that constitute an OTP build from within the VC++ GUI - is simply not worth it, maybe even impossible. A VC++ project - file for Erlang/OTP will never happen. - -* Q: So how does it all work then? - - A: Cygwin, MSYS or MSYS2 is the environment, which closely resembles the - environment found on any Unix machine. It's almost like you had a - virtual Unix machine inside Windows. Configure, given certain - parameters, then creates makefiles that are used by the - environment's gnu-make to built the system. Most of the actual - compilers etc are not, however, Cygwin/MSYS/MSYS2 tools, so we've written - a couple of wrappers (Bourne-shell scripts), which reside in - `$ERL_TOP/etc/win32/cygwin_tools` and - `$ERL_TOP/etc/win32/msys_tools`. They all do conversion of - parameters and switches common in the Unix environment to fit the - native Windows tools. Most notable is of course the paths, which - in Cygwin/MSYS/MSYS2 are Unix-like paths with "forward slashes" (/) and - no drive letters. The Cygwin specific command `cygpath` is used - for most of the path conversions in a Cygwin environment. Other - tools are used (when needed) in the corresponding MSYS and MSYS2 - environment. Luckily most compilers accept forward slashes instead - of backslashes as path separators, but one still have to get the drive - letters etc right, though. The wrapper scripts are not general in - the sense that, for example, cc.sh would understand and translate - every possible gcc option and pass correct options to - cl.exe. The principle is that the scripts are powerful enough to - allow building of Erlang/OTP, no more, no less. They might need - extensions to cope with changes during the development of Erlang, and - that's one of the reasons we made them into shell-scripts and not - Perl-scripts. We believe they are easier to understand and change - that way. - - In `$ERL_TOP`, there is a script called `otp_build`. That script handles - the hassle of giving all the right parameters to `configure`/`make` and - also helps you set up the correct environment variables to work with - the Erlang source under Cygwin/MSYS/MSYS2. - -* Q: You use and need Cygwin, but then you haven't taken the time to - port Erlang to the Cygwin environment but instead focus on your - commercial release, is that really ethical? - - A: No, not really, but see this as a step in the right direction. - -* Q: Can I build something that looks exactly as the commercial release? - - A: Yes, we use the exact same build procedure. - -* Q: Which version of Cygwin/MSYS/MSYS2 and other tools do you use then? - - A: For Cygwin, MSYS and MSYS2 alike, we try to use the latest releases - available when building. What versions you use shouldn't really - matter. We try to include workarounds for the bugs we've found in - different Cygwin/MSYS/MSYS2 releases. Please help us add workarounds - for new Cygwin/MSYS/MSYS2-related bugs as soon as you encounter - them. Also please do submit bug reports to the appropriate Cygwin, MSYS - and/or MSYS2 developers. The GCC we used for %OTP-REL% was version - 4.8.1 (MinGW 32bit) and 4.8.5 (MSYS2 64bit). We used VC++ 12.0 - (i.e. Visual studio 2013), Sun's JDK 1.6.0\_45 (32bit) and Sun's - JDK 1.7.0\_1 (64bit), NSIS 2.46, and Win32 OpenSSL 1.0.2d. Please - read the next section for details on what you need. - -* Q: Can you help me setup X in Cygwin/MSYS/MSYS2? - - A: No, unfortunately we haven't got time to help with Cygwin/MSYS/MSYS2 - related user problems, please read related websites, newsgroups and - mailing lists. - - Tools you Need and Their Environment ------------------------------------ You need some tools to be able to build Erlang/OTP on Windows. Most -notably you'll need Cygwin, MSYS or MSYS2, Visual Studio and Microsofts -Windows SDK, but you might also want a Java compiler, the NSIS install -system and OpenSSL. Well, here's some information about the different -tools: - -* Cygwin, the very latest is usually best. Get all the development - tools and of course all the basic ditto. Make sure to get jar and - also make sure *not* to install a Cygwin'ish Java, since the Cygwin - jar command is used but Sun's Java compiler and virtual machine. - - If you are going to build a 64bit Windows version, you should make - sure to get MinGW's 64bit gcc installed with Cygwin. It's in one of - the development packages. - - URL: <http://www.cygwin.com> - - Get the installer from the website and use it to install - Cygwin. Be sure to have fair privileges. If you're on an NT domain you - should consider running `mkpasswd -d` and `mkgroup -d` after the - installation to get the user databases correct. See their respective - manual pages. - - When you start your first bash shell, you will get an awful prompt. You - might also have a `PATH` environment variable that contains backslashes - and such. Edit `$HOME/.profile` and `$HOME/.bashrc` to set fair prompts - and a correct PATH. Also do an `export SHELL` in `.profile`. For some - non-obvious reason the environment variable `$SHELL` is not exported in - bash. Also note that `.profile` is run at login time and `.bashrc` when - sub shells are created. You'll need to explicitly source `.bashrc` from - `.profile` if you want the commands there to be run at login time (like - setting up aliases, shell functions and the like). You can for example - do like this at the end of `.profile`: - - ENV=$HOME/.bashrc - export ENV - . $ENV - - You might also want to setup X-windows (XFree86). That might be as easy - as running startx from the command prompt and it might be much harder. - Use Google to find help. - - If you don't use X-windows, you might want to setup the Windows - console window by selecting properties in the console system menu - (upper left corner of the window, the Cygwin icon in the title - bar). Especially setting a larger screen buffer size (lines) is useful - as it gets you a scrollbar so you can see whatever error messages - that might appear. - - There are a few other shells available, but in all examples below we assume - that you use bash. - -* Alternatively you download MinGW and MSYS. You'll find the latest - installer at: - - URL: <http://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/> - - Make sure to install the basic dev tools, but avoid the MinGW autoconf and - install the msys one instead. - - To be able to build the 64bit VM, you will also need the 64bit - MinGW compiler from: - - URL: <http://sourceforge.net/projects/mingw-w64/files/latest/download?source=files> - - We've tried up to 1.0, but the latest version should do. Make sure you - download the `mingw-w64-bin_i686-mingw_<something>.zip`, not a linux - version. You unzip the package on top of your MinGW installation - (`c:\MinGW`) and that's it. - -* A third alternative is to download and install MSYS2 from: +notably you'll need WSL (with ubuntu), Visual Studio and +Microsofts Windows SDK, but you might also want a Java compiler, the +NSIS install system, OpenSSL and wxWidgets. Well, here's some information about +the different tools: - URL: <https://msys2.github.io/> +* WSL: Install WSL and Ubuntu in Windows 10 + <https://docs.microsoft.com/en-us/windows/wsl/install-win10> - When you've followed the instructions there, you also need to install - these packages: autoconf, make, perl, and tar. You do so by running - the following in the msys console: + We have used and tested with WSL-1, WSL-2 was not available and may + not be prefered when building Erlang/OTP since access to the windows + disk is (currently) slower WSL-2. - pacman -S msys/autoconf msys/make msys/perl msys/tar +* Visual Studio 2019 + Download and run the installer from: + <http://visualstudio.microsoft.com/downloads> + Install C++ and SDK packages to the default installation directory. - You also need a gcc. If you installed the 64 bit MSYS2 you run: - - mingw64/mingw-w64-x86_64-gcc - - And for 32 bit MSYS2: - - pacman -S mingw32/mingw-w64-i686-gcc - pacman -S mingw-w64-i686-editrights - -* Visual Studio 2013 (Visual Studio 12.0). Download and run the web - installer from: - - https://www.visualstudio.com/ - -* Microsofts Windows SDK version 8.1 (corresponding to VC++ 12.0 and - Visual Studio 2013). You'll find it here: - - URL: <https://msdn.microsoft.com/en-us/windows/desktop/bg162891.aspx> - -* To help setup the environment, there is a bat file, - `%PROGRAMFILES%\Mirosoft Visual Studio 12.0\VC\vcvarsall.bat`, - that set's the appropriate - environment for a Windows command prompt. This is not appropriate - for bash, so you'll need to convert it to bash-style environments - by editing your `.bash_profile`. In my case, where the SDK is - installed in the default directory and `%PROGRAMFILES%` is - `C:\Program Files`, the commands for setting up a 32bit build - environment (on a 64bit or 32bit machine) look like this (in Cygwin): - - # Some common paths - C_DRV=/cygdrive/c - PRG_FLS=$C_DRV/Program\ Files - - # nsis - NSIS_BIN=$PRG_FLS/NSIS - # java - JAVA_BIN=$PROGRAMFILES/Java/jdk1.7.0_02/bin - - ## - ## MS SDK - ## - - CYGWIN=nowinsymlinks - - VISUAL_STUDIO_ROOT=$PRG_FLS/Microsoft\ Visual\ Studio\ 12.0 - WIN_VISUAL_STUDIO_ROOT="C:\\Program Files\\Microsoft Visual Studio 12.0" - SDK=$PRG_FLS/Windows\ Kits/8.1 - WIN_SDK="C:\\Program Files\\Windows Kits\\8.1" - - PATH="$NSIS_BIN:\ - $VISUAL_STUDIO_ROOT/VC/bin:\ - $VISUAL_STUDIO_ROOT/VC/vcpackages:\ - $VISUAL_STUDIO_ROOT/Common7/IDE:\ - $VISUAL_STUDIO_ROOT/Common7/Tools:\ - $SDK/bin/x86 - /usr/local/bin:/usr/bin:/bin:\ - /cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:\ - /cygdrive/c/WINDOWS/system32/Wbem:\ - $JAVA_BIN" - - LIBPATH="$WIN_VISUAL_STUDIO_ROOT\\VC\\lib" - - LIB="$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\;$WIN_SDK\\lib\\winv6.3\\um\\x86" - - INCLUDE="$WIN_VISUAL_STUDIO_ROOT\\VC\\include\\;$WIN_SDK\\include\\shared\\;\ - $WIN_SDK\\include\\um;$WIN_SDK\\include\\winrt\\;$WIN_SDK\\include\\um\\gl" - - export CYGWIN PATH LIBPATH LIB INCLUDE - - If you're using MinGW's MSYS instead, you need to change the `C_DRV` setting, - which would read: - - C_DRV=/c - - and you also need to change the PATH environment variable to: - - MINGW_BIN=/c/MinGW/bin - - - PATH="$NSIS_BIN:\ - $VISUAL_STUDIO_ROOT/VC/bin:\ - $VISUAL_STUDIO_ROOT/VC/vcpackages:\ - $VISUAL_STUDIO_ROOT/Common7/IDE:\ - $VISUAL_STUDIO_ROOT/Common7/Tools:\ - $SDK/bin/x86:/usr/local/bin:\ - $MINGW_BIN:\ - /bin:/c/Windows/system32:/c/Windows:\ - /c/Windows/System32/Wbem:\ - $JAVA_BIN" - - For MSYS2 you use the same `C_DRV` and PATH as for MSYS, only update the `MINGW_BIN`: - - MINGW_BIN=/mingw32/bin - - - If you are building a 64 bit version of Erlang, you should set up - PATHs etc a little differently. We have two templates to make things - work in both Cygwin and MSYS but needs editing to work with MSYS2 (see the - comments in the script). - The following one is for 32 bits: - - make_winpath() - { - P=$1 - if [ "$IN_CYGWIN" = "true" ]; then - cygpath -d "$P" - else - (cd "$P" && /bin/cmd //C "for %i in (".") do @echo %~fsi") - fi - } - - make_upath() - { - P=$1 - if [ "$IN_CYGWIN" = "true" ]; then - cygpath "$P" - else - echo "$P" | /bin/sed 's,^\([a-zA-Z]\):\\,/\L\1/,;s,\\,/,g' - fi - } - - # Some common paths - if [ -x /usr/bin/msys-?.0.dll ]; then - # Without this the path conversion won't work - COMSPEC='C:\Windows\System32\cmd.exe' - MSYSTEM=MINGW32 # Comment out this line if in MSYS2 - export MSYSTEM COMSPEC - # For MSYS2: Change /mingw/bin to the msys bin dir on the line below - PATH=/usr/local/bin:/mingw/bin:/bin:/c/Windows/system32:\ - /c/Windows:/c/Windows/System32/Wbem - C_DRV=/c - IN_CYGWIN=false - else - PATH=/ldisk/overrides:/usr/local/bin:/usr/bin:/bin:\ - /usr/X11R6/bin:/cygdrive/c/windows/system32:\ - /cygdrive/c/windows:/cygdrive/c/windows/system32/Wbem - C_DRV=/cygdrive/c - IN_CYGWIN=true - fi - - obe_otp_gcc_vsn_map=" - .*=>default - " - obe_otp_64_gcc_vsn_map=" - .*=>default - " - # Program Files - PRG_FLS=$C_DRV/Program\ Files - - # Visual Studio - VISUAL_STUDIO_ROOT=$PRG_FLS/Microsoft\ Visual\ Studio\ 12.0 - WIN_VISUAL_STUDIO_ROOT="C:\\Program Files\\Microsoft Visual Studio 12.0" - - # SDK - SDK=$PRG_FLS/Windows\ Kits/8.1 - WIN_SDK="C:\\Program Files\\Windows Kits\\8.1" - - # NSIS - NSIS_BIN=$PROGRAMFILES/NSIS - - # Java - JAVA_BIN=$PROGRAMFILES/Java/jdk1.7.0_02/bin - - ## The PATH variable should be Cygwin'ish - VCPATH= - $VISUAL_STUDIO_ROOT/VC/bin:\ - $VISUAL_STUDIO_ROOT/VC/vcpackages:\ - $VISUAL_STUDIO_ROOT/Common7/IDE:\ - $VISUAL_STUDIO_ROOT/Common7/Tools:\ - $SDK/bin/x86 - - ## Microsoft SDK libs - LIBPATH=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib - - LIB=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\;$WIN_KITS\\lib\\winv6.3\\um\\x86 - - INCLUDE=$WIN_VISUAL_STUDIO_ROOT\\VC\\include\\;\ - $WIN_KITS\\include\\shared\\;$WIN_KITS\\include\\um;\ - $WIN_KITS\\include\\winrt\\;$WIN_KITS\\include\\um\\gl - - # Put nsis, c compiler and java in path - export PATH=$VCPATH:$PATH:$JAVA_BIN:$NSIS_BIN - - # Make sure LIB and INCLUDE is available for others - export LIBPATH LIB INCLUDE - - - - The first part of the 64 bit template is identical to the 32 bit one, - but there are some environment variable differences: - - # Program Files - PRG_FLS64=$C_DRV/Program\ Files - PRG_FLS32=$C_DRV/Program\ Files\ \(x86\) - - # Visual Studio - VISUAL_STUDIO_ROOT=$PRG_FLS32/Microsoft\ Visual\ Studio\ 12.0 - WIN_VISUAL_STUDIO_ROOT="C:\\Program Files (x86)\\Microsoft Visual Studio 12.0" - - # SDK - SDK=$PRG_FLS32/Windows\ Kits/8.1 - WIN_SDK="C:\\Program Files (x86)\\Windows Kits\\8.1" - - # NSIS - NSIS_BIN=$PROGRAMFILES/NSIS - # Java - JAVA_BIN=$PROGRAMFILES/Java/jdk1.7.0_02/bin - - ## The PATH variable should be Cygwin'ish - VCPATH= - $VISUAL_STUDIO_ROOT/VC/bin/amd64:\ - $VISUAL_STUDIO_ROOT/VC/vcpackages:\ - $VISUAL_STUDIO_ROOT/Common7/IDE:\ - $VISUAL_STUDIO_ROOT/Common7/Tools:\ - $SDK/bin/x86 - - ## Microsoft SDK libs - LIBPATH=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\amd64 - - LIB=$WIN_VISUAL_STUDIO_ROOT\\VC\\lib\\amd64\\;\ - $WIN_KITS\\lib\\winv6.3\\um\\x64 - - INCLUDE=$WIN_VISUAL_STUDIO_ROOT\\VC\\include\\;\ - $WIN_KITS\\include\\shared\\;$WIN_KITS\\include\\um;\ - $WIN_KITS\\include\\winrt\\;$WIN_KITS\\include\\um\\gl - - # Put nsis, c compiler and java in path - export PATH=$VCPATH:$PATH:$JAVA_BIN:$NSIS_BIN - - # Make sure LIB and INCLUDE is available for others - export LIBPATH LIB INCLUDE - - - Make sure to set the PATH so that NSIS and Microsoft SDK is found - before the MSYS/Cygwin tools and that Java is last in the PATH. - - Make a simple hello world and try to compile it with the `cl` - command from within bash. If that does not work, your environment - needs fixing. Remember, there should be - no backslashes in your path environment variable in Cygwin bash, - but LIB and INCLUDE should contain Windows style paths with - semicolon, drive letters and backslashes. - -* Sun's Java JDK 1.6.0 or later. Our Java code (jinterface, ic) is - written for JDK 1.6.0. Get it for Windows and install it, the JRE is - not enough. If you don't care about Java, you can skip this step. The +* Java JDK 8 or later (optional) + If you don't care about Java, you can skip this step. The result will be that jinterface is not built. - URL: <http://java.sun.com> + Our Java code (jinterface, ic) is tested on windows with JDK 8. + Get it for Windows and install it, the JRE is not enough. + + URL: <http://www.oracle.com/java/technologies/javase-downloads.html> - Add javac *LAST* to your path environment in bash, in my case this means: + Add javac to your path environment, in my case this means: - `PATH="$PATH:/cygdrive/c/Program Files/Java/jdk1.7.0_02/bin"` + `PATH="/mnt/c/Program\ Files/Java/jdk1.8.0_241/bin:$PATH` - No `CLASSPATH` or anything is needed. Type `javac` in the bash prompt - and you should get a list of available Java options. Make sure, e.g by - typing `type java`, that you use the Java you installed. Note however that - Cygwin's/MinGW's/MSYS2's `jar.exe` is used. That's why the JDK bin-directory should be - added last in the `PATH`. + No `CLASSPATH` or anything is needed. Type `javac.exe` in the bash prompt + and you should get a list of available Java options. -* Nullsoft NSIS installer system. You need this to build the self - installing package. It's a free open source installer that's much - nicer to use than the commercial Wise and Install shield - installers. This is the installer we use for commercial releases as - well. +* Nullsoft NSIS installer system (optional) + You need this to build the self installing package. + Download and run the installer from: URL: <http://nsis.sourceforge.net/download> - Install the lot, especially the modern user interface components, as - it's definitely needed. Put `makensis` in your path, in my case: + Add 'makensis.exe' to your path environment: - PATH=/cygdrive/c/Program\ Files/NSIS:$PATH + `PATH="/mnt/c/Program\ Files/NSIS/Bin:$PATH` - Type makensis at the bash prompt and you should get a list of options - if everything is OK. + Type `which makensis.exe` in the bash prompt and you should get the + path to the program. -* OpenSSL. This is if you want the SSL and crypto applications to - compile (and run). There are prebuilt binaries, which you can just - download and install, available here: +* OpenSSL (optional) + You need this to build crypto, ssh and ssl libs. - URL: <http://openssl.org/community/binaries.html> + We recommend v1.1.1d or later. + There are prebuilt avaiable binaries, which you can just + download and install, available here: + URL: <http://wiki.openssl.org/index.php/Binaries> - We would recommend using 1.0.2d. + Install into `C:/OpenSSL-Win64` (or `C:/OpenSSL-Win32`) -* Building with wxWidgets. Download wxWidgets-3.0.3 or higher. +* wxWidgets (optional) + You need this to build wx and use gui's in debugger and observer. - Install or unpack it to the pgm folder: - Cygwin: - `DRIVE:/PATH/cygwin/opt/local/pgm` - MSYS: - `DRIVE:/PATH/MinGW/msys/1.0/opt/local/pgm` - MSYS2: - `DRIVE:/PATH/msys<32/64>/opt/local/pgm` + We recommend v3.1.3 or later. + Unpack into `c:/opt/local64/pgm/wxWidgets-3.1.3` - If the `wxUSE_POSTSCRIPT` isn't enabled in `<path\to\pgm>\wxMSW-3.0.3\include\wx\msw\setup.h`, + If the `wxUSE_POSTSCRIPT` isn't enabled in `c:/opt/local64/pgm/wxWidgets-3.1.3/include/wx/msw/setup.h`, enable it. - build: From a command prompt with the VC tools available (See the - instructions for OpenSSL build above for help on starting the - proper command prompt in RELEASE mode): - - C:\...\> cd <path\to\pgm>\wxMSW-3.0.3\build\msw - C:\...\> nmake BUILD=release SHARED=0 DIR_SUFFIX_CPU= -f makefile.vc - - Or - if building a 64bit version: + Build with: - C:\...\> cd <path\to\pgm>\wxMSW-3.0.3\build\msw + C:\...\> cd c:\opt\local64\pgm\wxWidgets-3.1.3\build\msw C:\...\> nmake TARGET_CPU=amd64 BUILD=release SHARED=0 DIR_SUFFIX_CPU= -f makefile.vc - + + Remove the `TARGET_CPU=amd64` for 32bit build. + * Get the Erlang source distribution (from <http://www.erlang.org/download.html>). - The same as for Unix platforms. Preferably use tar from within Cygwin, MSYS or MSYS2 to - unpack the source tar.gz (`tar zxf otp_src_%OTP-REL%.tar.gz`). + The same as for Unix platforms. Preferably use tar to + unpack the source tar.gz (`tar zxf otp_src_%OTP-REL%.tar.gz`) to somewhere + on the windows disk, `/mnt/c/path/to/otp_src` + + NOTE: It is important that source on the windows disk. Set the environment `ERL_TOP` to point to the root directory of the - source distribution. Let's say I stood in `$HOME/src` and unpacked + source distribution. Let's say I stood in `/mnt/c/src` and unpacked `otp_src_%OTP-REL%.tar.gz`, I then add the following to `.profile`: - ERL_TOP=$HOME/src/otp_src_%OTP-REL% - export $ERL_TOP + ERL_TOP=/mnt/c/src/otp_src_%OTP-REL% + export ERL_TOP + The Shell Environment --------------------- -So, if you have followed the instructions above, when you start a bash -shell, you should have an INCLUDE environment with a Windows style -path, a LIB environment variable also in Windows style, and finally a -PATH that let's you reach cl, makensis, javac etc from the -command prompt (use `which cl` etc to verify from bash). +The path variable should now contain the windows paths to javac.exe and makensis.exe. -You should also have an `ERL_TOP` environment variable that is *Cygwin -style*, and points to a directory containing, among other files, the -script `otp_build`. +Setup the environment with: -A final massage of the environment is needed, and that is done by -the script `$ERL_TOP/otp_build`. Start bash and do the following, note -the "back-ticks" (\`), can be quite hard to get on some keyboards, but -pressing the back-tick key followed by the space bar might do it... - - $ cd $ERL_TOP - $ eval `./otp_build env_win32` - -If you're unable to produce back-ticks on your keyboard, you can use -the ksh variant: - - $ cd $ERL_TOP - $ eval $(./otp_build env_win32) - -If you are building a 64 bit version, you supply `otp_build` with an architecture parameter: - - $ cd $ERL_TOP + $ export PATH + $ cd /mnt/c/path/to/otp_src/ $ eval `./otp_build env_win32 x64` - + +This should setup the additional environment variables. This should do the final touch to the environment and building should be easy after this. You could run `./otp_build env_win32` without @@ -688,18 +201,24 @@ be easy after this. You could run `./otp_build env_win32` without sets seems OK. The path is cleaned of spaces if possible (using DOS style short names instead), the variables `OVERRIDE_TARGET`, `CC`, `CXX`, `AR` and `RANLIB` are set to their respective wrappers and the directories -`$ERL_TOP/erts/etc/win32/<cygwin/msys>_tools/vc` and -`$ERL_TOP/erts/etc/win32/<cygwin/msys>_tool` are added first in the PATH. +`$ERL_TOP/erts/etc/win32/wsl_tools/vc` and +`$ERL_TOP/erts/etc/win32/wsl_tools` are added first in the PATH. + +Now you can check which erlc you have by writing `type erlc` in your shell. +It should reside in `$ERL_TOP/erts/etc/win32/wsl_tools`. + +And running `cl.exe` should print the Microsoft compiler usage message. -Now you can check which erlc you have by writing `type erlc` in your shell. -It should reside in `$ERL_TOP/erts/etc/win32/cygwin_tools` -or `$ERL_TOP/erts/etc/win32/msys_tools`. +The needed compiler environment variables are setup inside `otp_build` +via `erts/etc/win32/wsl_tools/SetupWSLcross.bat`. It contains some +hardcoded paths, if your installation path is different it can be added +to that file. Building and Installing ----------------------- -Building is easiest using the `otp_build` script: +Building is easiest using the `otp_build` script: $ ./otp_build autoconf # Ignore the warning blob about versions of autoconf $ ./otp_build configure <optional configure options> @@ -707,7 +226,7 @@ Building is easiest using the `otp_build` script: $ ./otp_build release -a <installation directory> $ ./otp_build installer_win32 <installation directory> # optional -Now you will have a file called `otp_win32_%OTP-REL%.exe` or `otp_win64_%OTP-REL%.exe` +Now you will have a file called `otp_win32_%OTP-REL%.exe` or `otp_win64_%OTP-REL%.exe` in the `<installation directory>`, i.e. `$ERL_TOP/release/win32`. Lets get into more detail: @@ -716,8 +235,8 @@ Lets get into more detail: to work correctly in your environment. In an ideal world, this would not be needed, but alas, we have encountered several incompatibilities between our distributed configure scripts (generated - on a Linux platform) and the Cygwin/MSYS/MSYS2 environment over the - years. Running autoconf in Cygwin/MSYS/MSYS2 ensures that the configure + on a Linux platform) and the Cygwin/MSYS/MSYS2/WSL environment over the + years. Running autoconf in WSL ensures that the configure scripts are generated in a compatible way and that they will work well in the next step. @@ -732,15 +251,15 @@ Lets get into more detail: 3. `$ ./otp_build boot -a` - This uses the bootstrap directory (shipped with the source, `$ERL_TOP/bootstrap`) to build a complete OTP - system. When this is done you can run erl from within the source tree; - just type `$ERL_TOP/bin/erl` and you whould have the prompt. + system. When this is done you can run erl from within the source tree; + just type `$ERL_TOP/bin/erl` and you should have the prompt. 4. `$ ./otp_build release -a` - Builds a commercial release tree from the source tree. The default is to put it in `$ERL_TOP/release/win32`. You can - give any directory as parameter (Cygwin style), but it doesn't really - matter if you're going to build a self extracting installer too. + give any directory as parameter, but it doesn't really + matter if you're going to build a self extracting installer too. -5. `$ ./otp_build installer_win32` - Creates the self extracting installer executable. +5. `$ ./otp_build installer_win32` - Creates the self extracting installer executable. The executable `otp_win32_%OTP-REL%.exe` or `otp_win64_%OTP-REL%.exe` will be placed in the top directory of the release created in the previous step. If no release directory is specified, the release is expected to have @@ -765,7 +284,7 @@ Lets get into more detail: $ cd $ERL_TOP $ release/win32/otp_win64_%OTP-REL% /S ... - + and after a while Erlang/OTP-%OTP-REL% will have been installed in `C:\Program Files\erl%ERTS-VSN%\`, with shortcuts in the menu etc. @@ -835,13 +354,13 @@ steps. You could also add `$ERL_TOP/bootstrap/bin` to your `PATH` before rebuilding specific libraries. That would give you a good enough Erlang system to compile any OTP erlang code. Setting up the path correctly is a little bit tricky. You still need to have -`$ERL_TOP/erts/etc/win32/cygwin_tools/vc` and -`$ERL_TOP/erts/etc/win32/cygwin_tools` *before* the actual emulator +`$ERL_TOP/erts/etc/win32/wsl_tools/vc` and +`$ERL_TOP/erts/etc/win32/wsl_tools` *before* the actual emulator in the path. A typical setting of the path for using the bootstrap compiler would be: - $ export PATH=$ERL_TOP/erts/etc/win32/cygwin_tools/vc\ - :$ERL_TOP/erts/etc/win32/cygwin_tools:$ERL_TOP/bootstrap/bin:$PATH + $ export PATH=$ERL_TOP/erts/etc/win32/wsl_tools/vc\ + :$ERL_TOP/erts/etc/win32/wsl_tools:$ERL_TOP/bootstrap/bin:$PATH That should make it possible to rebuild any library without hassle... @@ -870,22 +389,106 @@ Remember that: That's basically all you need to get going. -Using GIT ---------- -You might want to check out versions of the source code from GitHUB. That is possible directly in Cygwin, but not in MSYS. There is a project MsysGIT: +Frequently Asked Questions +-------------------------- + +* Q: So, now I can build Erlang using GCC on Windows? + + A: No, unfortunately not. You'll need Microsoft's Visual C++ + still. A Bourne-shell script (cc.sh) wraps the Visual C++ compiler + and runs it from within the WSL environment. All other tools + needed to build Erlang are free-ware/open source, but not the C + compiler. + +* Q: Why haven't you got rid of VC++ then, you \*\*\*\*\*\*? + + A: Well, partly because it's a good compiler - really! Actually it's + been possible in late R11-releases to build using mingw instead of + visual C++ (you might see the remnants of that in some scripts and + directories). Unfortunately the development of the SMP version for + Windows broke the mingw build and we chose to focus on the VC++ build + as the performance has been much better in the VC++ versions. The + mingw build will possibly be back, but as long as VC++ gives better + performance, the commercial build will be a VC++ one. + +* Q: OK, you need VC++, but now you've started to demand a quite recent + (and expensive) version of Visual Studio. Why? + + A: Well, it's not expensive, it's free (as in free beer). Just + download and install the latest Windows SDK from Microsoft and all + the tools you need are there. The included debugger (WinDbg) is + also quite usable. That's what I used when porting Erlang to 64bit + Windows. Another reason to use later Microsoft compilers is + DLL compatibility. DLL's using a new version of the standard + library might not load if the VM is compiled with an old VC++ + version. So we should aim to use the latest freely available SDK + and compiler. + +* Q: Hah, I saw you, you used GCC even though you said you didn't! + + A: OK, I admit, one of the files is compiled using + MinGW's GCC and the resulting object code is then converted to MS + VC++ compatible coff using a small C hack. It's because that + particular file, `beam_emu.c` benefits immensely from being able + to use the GCC labels-as-values extension, which boosts emulator + performance by up to 50%. That does unfortunately not (yet) mean + that all of OTP could be compiled using GCC. That particular + source code does not do anything system specific and actually is + adopted to the fact that GCC is used to compile it on Windows. + +* Q: So now there's a MS VC++ project file somewhere and I can build OTP + using the nifty VC++ GUI? + + A: No, never. The hassle of keeping the project files up to date and + do all the steps that constitute an OTP build from within the VC++ GUI + is simply not worth it, maybe even impossible. A VC++ project + file for Erlang/OTP will never happen. + +* Q: So how does it all work then? + + A: WSL/Ubuntu is the environment, it's almost like you had a + virtual Unix machine inside Windows. Configure, given certain + parameters, then creates makefiles that are used by the + environment's gnu-make to built the system. Most of the actual + compilers etc are not, however, WSL tools, so we've written + a couple of wrappers (Bourne-shell scripts), which reside in + `$ERL_TOP/etc/win32/wsl_tools`. They all do conversion of + parameters and switches common in the Unix environment to fit the + native Windows tools. Most notable is of course the paths, which + in WSL are Unix-like paths with "forward slashes" (/) and + no drive letters. The WSL specific command `wslpath` is used + for most of the path conversions in a WSL environment. + Luckily most compilers accept forward slashes instead + of backslashes as path separators, but one still have to get the drive + letters etc right, though. The wrapper scripts are not general in + the sense that, for example, cc.sh would understand and translate + every possible gcc option and pass correct options to + cl.exe. The principle is that the scripts are powerful enough to + allow building of Erlang/OTP, no more, no less. They might need + extensions to cope with changes during the development of Erlang, and + that's one of the reasons we made them into shell-scripts and not + Perl-scripts. We believe they are easier to understand and change + that way. + + In `$ERL_TOP`, there is a script called `otp_build`. That script handles + the hassle of giving all the right parameters to `configure`/`make` and + also helps you set up the correct environment variables to work with + the Erlang source under WSL. + +* Q: Can I build something that looks exactly as the commercial release? + + A: Yes, we use the exact same build procedure. -URL:<http://code.google.com/p/msysgit/> +* Q: Which version of WSL and other tools do you use then? -that makes a nice Git port. The msys prompt you get from MsysGIT is -however not compatible with the full version from MinGW, so you will -need to check out files using MsysGIT's command prompt and then switch -to a common MSYS command prompt for building. Also all test suites -cannot be built as MsysGIT/MSYS does not handle symbolic links. + A: We use WSL 1 with Ubuntu 18.04. + The GCC we used for %OTP-REL% was version 7.3-win32. + We used Visual studio 2019, Sun's JDK 1.8.0\_241, + NSIS 3.05, Win32 OpenSSL 1.1.1d and wxWidgets-3.1.3. - [1]: http://www.erlang.org/static/doc/mailinglist.html - [2]: http://bugs.erlang.org - [3]: https://github.com/erlang/otp + [1]: http://bugs.erlang.org + [2]: https://github.com/erlang/otp [?TOC]: true diff --git a/Makefile.in b/Makefile.in index 938563bd2e..f9bfe123b0 100644 --- a/Makefile.in +++ b/Makefile.in @@ -338,7 +338,7 @@ endif else # Cross compiling -all: cross_check_erl depend build_erl_interface \ +all: cross_check_erl depend erl_interface \ emulator libs start_scripts check_dev_rt_dep endif @@ -378,29 +378,22 @@ endif # the system is delivered in open source, the primary # bootstrap is not included, it requires a pre built emulator... ifeq ($(OTP_TINY_BUILD),true) -all_bootstraps: build_erl_interface emulator bootstrap_setup \ +all_bootstraps: erl_interface emulator bootstrap_setup \ tiny_secondary_bootstrap_build tiny_secondary_bootstrap_copy else -all_bootstraps: build_erl_interface emulator \ +all_bootstraps: erl_interface emulator \ bootstrap_setup \ secondary_bootstrap_build secondary_bootstrap_copy \ tertiary_bootstrap_build tertiary_bootstrap_copy endif -.PHONY: build_erl_interface - -build_erl_interface: - $(make_verbose)cd lib/erl_interface && \ - ERL_TOP=$(ERL_TOP) PATH=$(BOOT_PREFIX)"$${PATH}" \ - $(MAKE) $(TYPE) - # # Use these targets when you want to use the erl and erlc # binaries in your PATH instead of those created from the # pre-compiled Erlang modules under bootstrap/. # noboot: - $(MAKE) USE_PGO=false BOOT_PREFIX= build_erl_interface emulator libs local_setup + $(MAKE) USE_PGO=false BOOT_PREFIX= erl_interface emulator libs local_setup noboot_install: $(MAKE) USE_PGO=false BOOT_PREFIX= install @@ -480,7 +473,7 @@ BOOTSTRAP_COMPILER = $(BOOTSTRAP_TOP)/primary_compiler # otp.mk is only used to figure out if we are doing PGO or not include $(ERL_TOP)/make/$(TARGET)/otp.mk -.PHONY: emulator libs kernel stdlib compiler hipe syntax_tools preloaded +.PHONY: emulator libs kernel preloaded ifeq ($(USE_PGO), true) PROFILE=use @@ -517,6 +510,8 @@ endif APPS=$(patsubst $(ERL_TOP)/lib/%/doc,%,$(wildcard $(ERL_TOP)/lib/*/doc)) +.PHONY: $(APPS) + $(APPS): $(make_verbose)cd lib/$@ && \ ERL_TOP=$(ERL_TOP) PATH=$(BOOT_PREFIX)"$${PATH}" \ diff --git a/OTP_VERSION b/OTP_VERSION index 0af13d0821..c471f62163 100644 --- a/OTP_VERSION +++ b/OTP_VERSION @@ -1 +1 @@ -23.0-rc1 +23.0.1 diff --git a/bootstrap/bin/no_dot_erlang.boot b/bootstrap/bin/no_dot_erlang.boot Binary files differindex 5c92c8fbdf..e7186a3055 100644 --- a/bootstrap/bin/no_dot_erlang.boot +++ b/bootstrap/bin/no_dot_erlang.boot diff --git a/bootstrap/bin/start.boot b/bootstrap/bin/start.boot Binary files differindex 5c92c8fbdf..e7186a3055 100644 --- a/bootstrap/bin/start.boot +++ b/bootstrap/bin/start.boot diff --git a/bootstrap/bin/start_clean.boot b/bootstrap/bin/start_clean.boot Binary files differindex 5c92c8fbdf..e7186a3055 100644 --- a/bootstrap/bin/start_clean.boot +++ b/bootstrap/bin/start_clean.boot diff --git a/bootstrap/lib/compiler/ebin/beam_a.beam b/bootstrap/lib/compiler/ebin/beam_a.beam Binary files differindex f9bcd13b57..37c074fe11 100644 --- a/bootstrap/lib/compiler/ebin/beam_a.beam +++ b/bootstrap/lib/compiler/ebin/beam_a.beam diff --git a/bootstrap/lib/compiler/ebin/beam_asm.beam b/bootstrap/lib/compiler/ebin/beam_asm.beam Binary files differindex 9550cbf34a..1ef964e83f 100644 --- a/bootstrap/lib/compiler/ebin/beam_asm.beam +++ b/bootstrap/lib/compiler/ebin/beam_asm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_block.beam b/bootstrap/lib/compiler/ebin/beam_block.beam Binary files differindex cdac62f730..1443b24523 100644 --- a/bootstrap/lib/compiler/ebin/beam_block.beam +++ b/bootstrap/lib/compiler/ebin/beam_block.beam diff --git a/bootstrap/lib/compiler/ebin/beam_call_types.beam b/bootstrap/lib/compiler/ebin/beam_call_types.beam Binary files differindex ebfd8f61ce..d0ba1a1494 100644 --- a/bootstrap/lib/compiler/ebin/beam_call_types.beam +++ b/bootstrap/lib/compiler/ebin/beam_call_types.beam diff --git a/bootstrap/lib/compiler/ebin/beam_clean.beam b/bootstrap/lib/compiler/ebin/beam_clean.beam Binary files differindex 3494688fbc..e91269bcc2 100644 --- a/bootstrap/lib/compiler/ebin/beam_clean.beam +++ b/bootstrap/lib/compiler/ebin/beam_clean.beam diff --git a/bootstrap/lib/compiler/ebin/beam_dict.beam b/bootstrap/lib/compiler/ebin/beam_dict.beam Binary files differindex dd564efdb0..96a973cf91 100644 --- a/bootstrap/lib/compiler/ebin/beam_dict.beam +++ b/bootstrap/lib/compiler/ebin/beam_dict.beam diff --git a/bootstrap/lib/compiler/ebin/beam_digraph.beam b/bootstrap/lib/compiler/ebin/beam_digraph.beam Binary files differindex eab1e25cdb..f09e0b3f05 100644 --- a/bootstrap/lib/compiler/ebin/beam_digraph.beam +++ b/bootstrap/lib/compiler/ebin/beam_digraph.beam diff --git a/bootstrap/lib/compiler/ebin/beam_disasm.beam b/bootstrap/lib/compiler/ebin/beam_disasm.beam Binary files differindex fd7fa77060..505014fbf5 100644 --- a/bootstrap/lib/compiler/ebin/beam_disasm.beam +++ b/bootstrap/lib/compiler/ebin/beam_disasm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_flatten.beam b/bootstrap/lib/compiler/ebin/beam_flatten.beam Binary files differindex 93a34eda47..9c16ffa8e7 100644 --- a/bootstrap/lib/compiler/ebin/beam_flatten.beam +++ b/bootstrap/lib/compiler/ebin/beam_flatten.beam diff --git a/bootstrap/lib/compiler/ebin/beam_jump.beam b/bootstrap/lib/compiler/ebin/beam_jump.beam Binary files differindex 370bfd3309..e9674e364d 100644 --- a/bootstrap/lib/compiler/ebin/beam_jump.beam +++ b/bootstrap/lib/compiler/ebin/beam_jump.beam diff --git a/bootstrap/lib/compiler/ebin/beam_kernel_to_ssa.beam b/bootstrap/lib/compiler/ebin/beam_kernel_to_ssa.beam Binary files differindex 2ad54c19d9..44868beab7 100644 --- a/bootstrap/lib/compiler/ebin/beam_kernel_to_ssa.beam +++ b/bootstrap/lib/compiler/ebin/beam_kernel_to_ssa.beam diff --git a/bootstrap/lib/compiler/ebin/beam_listing.beam b/bootstrap/lib/compiler/ebin/beam_listing.beam Binary files differindex d67ccd1854..cb196fb100 100644 --- a/bootstrap/lib/compiler/ebin/beam_listing.beam +++ b/bootstrap/lib/compiler/ebin/beam_listing.beam diff --git a/bootstrap/lib/compiler/ebin/beam_peep.beam b/bootstrap/lib/compiler/ebin/beam_peep.beam Binary files differindex 66e11fbf58..a1ea4c56d5 100644 --- a/bootstrap/lib/compiler/ebin/beam_peep.beam +++ b/bootstrap/lib/compiler/ebin/beam_peep.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa.beam b/bootstrap/lib/compiler/ebin/beam_ssa.beam Binary files differindex 84643e15e3..bfd5d0152e 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_bool.beam b/bootstrap/lib/compiler/ebin/beam_ssa_bool.beam Binary files differindex 9e8bb9dfe8..ad1f517670 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_bool.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_bool.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_bsm.beam b/bootstrap/lib/compiler/ebin/beam_ssa_bsm.beam Binary files differindex 5cd40730cd..507f510009 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_bsm.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_bsm.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_codegen.beam b/bootstrap/lib/compiler/ebin/beam_ssa_codegen.beam Binary files differindex 9e24bd2132..fec36ab2fa 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_codegen.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_codegen.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_dead.beam b/bootstrap/lib/compiler/ebin/beam_ssa_dead.beam Binary files differindex 8e3cd65412..0b0ea05b03 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_dead.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_dead.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_funs.beam b/bootstrap/lib/compiler/ebin/beam_ssa_funs.beam Binary files differindex 1c4ee15df4..ab4cb275a5 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_funs.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_funs.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_lint.beam b/bootstrap/lib/compiler/ebin/beam_ssa_lint.beam Binary files differindex ecd80f90eb..08344d0398 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_lint.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_lint.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_opt.beam b/bootstrap/lib/compiler/ebin/beam_ssa_opt.beam Binary files differindex 27147c97f8..3712ff72a1 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_opt.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_opt.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_pp.beam b/bootstrap/lib/compiler/ebin/beam_ssa_pp.beam Binary files differindex 9d73d015e0..724c155021 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_pp.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_pp.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_pre_codegen.beam b/bootstrap/lib/compiler/ebin/beam_ssa_pre_codegen.beam Binary files differindex e7b1622522..4ae9121480 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_pre_codegen.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_pre_codegen.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_recv.beam b/bootstrap/lib/compiler/ebin/beam_ssa_recv.beam Binary files differindex 8dfde56ce4..9c4080d5e5 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_recv.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_recv.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_share.beam b/bootstrap/lib/compiler/ebin/beam_ssa_share.beam Binary files differindex edfd070ba4..96840309aa 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_share.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_share.beam diff --git a/bootstrap/lib/compiler/ebin/beam_ssa_type.beam b/bootstrap/lib/compiler/ebin/beam_ssa_type.beam Binary files differindex 18c7359178..b04b0499ae 100644 --- a/bootstrap/lib/compiler/ebin/beam_ssa_type.beam +++ b/bootstrap/lib/compiler/ebin/beam_ssa_type.beam diff --git a/bootstrap/lib/compiler/ebin/beam_trim.beam b/bootstrap/lib/compiler/ebin/beam_trim.beam Binary files differindex 980f8e2d61..08c1cb1db5 100644 --- a/bootstrap/lib/compiler/ebin/beam_trim.beam +++ b/bootstrap/lib/compiler/ebin/beam_trim.beam diff --git a/bootstrap/lib/compiler/ebin/beam_types.beam b/bootstrap/lib/compiler/ebin/beam_types.beam Binary files differindex 08e684f1d3..9f409ae6e0 100644 --- a/bootstrap/lib/compiler/ebin/beam_types.beam +++ b/bootstrap/lib/compiler/ebin/beam_types.beam diff --git a/bootstrap/lib/compiler/ebin/beam_validator.beam b/bootstrap/lib/compiler/ebin/beam_validator.beam Binary files differindex ac3625a297..a705beca10 100644 --- a/bootstrap/lib/compiler/ebin/beam_validator.beam +++ b/bootstrap/lib/compiler/ebin/beam_validator.beam diff --git a/bootstrap/lib/compiler/ebin/beam_z.beam b/bootstrap/lib/compiler/ebin/beam_z.beam Binary files differindex a62598b443..3dea70729c 100644 --- a/bootstrap/lib/compiler/ebin/beam_z.beam +++ b/bootstrap/lib/compiler/ebin/beam_z.beam diff --git a/bootstrap/lib/compiler/ebin/cerl.beam b/bootstrap/lib/compiler/ebin/cerl.beam Binary files differindex c8cee8870e..480da51872 100644 --- a/bootstrap/lib/compiler/ebin/cerl.beam +++ b/bootstrap/lib/compiler/ebin/cerl.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_clauses.beam b/bootstrap/lib/compiler/ebin/cerl_clauses.beam Binary files differindex 34e810e7d0..977fb2eee6 100644 --- a/bootstrap/lib/compiler/ebin/cerl_clauses.beam +++ b/bootstrap/lib/compiler/ebin/cerl_clauses.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_inline.beam b/bootstrap/lib/compiler/ebin/cerl_inline.beam Binary files differindex 9438c6c038..f764bf0270 100644 --- a/bootstrap/lib/compiler/ebin/cerl_inline.beam +++ b/bootstrap/lib/compiler/ebin/cerl_inline.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_sets.beam b/bootstrap/lib/compiler/ebin/cerl_sets.beam Binary files differindex cfb8b7289f..c12a16f10d 100644 --- a/bootstrap/lib/compiler/ebin/cerl_sets.beam +++ b/bootstrap/lib/compiler/ebin/cerl_sets.beam diff --git a/bootstrap/lib/compiler/ebin/cerl_trees.beam b/bootstrap/lib/compiler/ebin/cerl_trees.beam Binary files differindex c8e672ec7a..ef29f6fdfc 100644 --- a/bootstrap/lib/compiler/ebin/cerl_trees.beam +++ b/bootstrap/lib/compiler/ebin/cerl_trees.beam diff --git a/bootstrap/lib/compiler/ebin/compile.beam b/bootstrap/lib/compiler/ebin/compile.beam Binary files differindex 1ed262a5c7..27a8a497b6 100644 --- a/bootstrap/lib/compiler/ebin/compile.beam +++ b/bootstrap/lib/compiler/ebin/compile.beam diff --git a/bootstrap/lib/compiler/ebin/compiler.app b/bootstrap/lib/compiler/ebin/compiler.app index ba5179fc60..dd5738c847 100644 --- a/bootstrap/lib/compiler/ebin/compiler.app +++ b/bootstrap/lib/compiler/ebin/compiler.app @@ -19,7 +19,7 @@ {application, compiler, [{description, "ERTS CXC 138 10"}, - {vsn, "7.5.3"}, + {vsn, "7.5.4"}, {modules, [ beam_a, beam_asm, diff --git a/bootstrap/lib/compiler/ebin/core_lib.beam b/bootstrap/lib/compiler/ebin/core_lib.beam Binary files differindex aebf95b9ce..c73352bda3 100644 --- a/bootstrap/lib/compiler/ebin/core_lib.beam +++ b/bootstrap/lib/compiler/ebin/core_lib.beam diff --git a/bootstrap/lib/compiler/ebin/core_lint.beam b/bootstrap/lib/compiler/ebin/core_lint.beam Binary files differindex 07c1501c22..d4a5ad17d6 100644 --- a/bootstrap/lib/compiler/ebin/core_lint.beam +++ b/bootstrap/lib/compiler/ebin/core_lint.beam diff --git a/bootstrap/lib/compiler/ebin/core_parse.beam b/bootstrap/lib/compiler/ebin/core_parse.beam Binary files differindex 4d6fb708ec..a595b4b7c1 100644 --- a/bootstrap/lib/compiler/ebin/core_parse.beam +++ b/bootstrap/lib/compiler/ebin/core_parse.beam diff --git a/bootstrap/lib/compiler/ebin/core_pp.beam b/bootstrap/lib/compiler/ebin/core_pp.beam Binary files differindex a1cb68641a..db0e52f459 100644 --- a/bootstrap/lib/compiler/ebin/core_pp.beam +++ b/bootstrap/lib/compiler/ebin/core_pp.beam diff --git a/bootstrap/lib/compiler/ebin/core_scan.beam b/bootstrap/lib/compiler/ebin/core_scan.beam Binary files differindex e537870f8e..f964b39cf7 100644 --- a/bootstrap/lib/compiler/ebin/core_scan.beam +++ b/bootstrap/lib/compiler/ebin/core_scan.beam diff --git a/bootstrap/lib/compiler/ebin/rec_env.beam b/bootstrap/lib/compiler/ebin/rec_env.beam Binary files differindex aeafd5e121..74f85b4f52 100644 --- a/bootstrap/lib/compiler/ebin/rec_env.beam +++ b/bootstrap/lib/compiler/ebin/rec_env.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_alias.beam b/bootstrap/lib/compiler/ebin/sys_core_alias.beam Binary files differindex bbb40e4b4d..a7e2c47364 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_alias.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_alias.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_bsm.beam b/bootstrap/lib/compiler/ebin/sys_core_bsm.beam Binary files differindex 50903e68f3..ddd3849ef2 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_bsm.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_bsm.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_fold.beam b/bootstrap/lib/compiler/ebin/sys_core_fold.beam Binary files differindex 81e21f1d11..51d2940fd4 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_fold.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_fold.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_inline.beam b/bootstrap/lib/compiler/ebin/sys_core_inline.beam Binary files differindex 404b4ce9bf..0c0e121687 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_inline.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_inline.beam diff --git a/bootstrap/lib/compiler/ebin/sys_core_prepare.beam b/bootstrap/lib/compiler/ebin/sys_core_prepare.beam Binary files differindex d973d575b7..4f69160a38 100644 --- a/bootstrap/lib/compiler/ebin/sys_core_prepare.beam +++ b/bootstrap/lib/compiler/ebin/sys_core_prepare.beam diff --git a/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam b/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam Binary files differindex 03b9744ca5..dafcb2cf3b 100644 --- a/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam +++ b/bootstrap/lib/compiler/ebin/sys_pre_attributes.beam diff --git a/bootstrap/lib/compiler/ebin/v3_core.beam b/bootstrap/lib/compiler/ebin/v3_core.beam Binary files differindex 08ea4ceb70..a256327666 100644 --- a/bootstrap/lib/compiler/ebin/v3_core.beam +++ b/bootstrap/lib/compiler/ebin/v3_core.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel.beam b/bootstrap/lib/compiler/ebin/v3_kernel.beam Binary files differindex d6ae439e74..1e161305c9 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel.beam diff --git a/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam b/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam Binary files differindex e1b5d87971..3eda809d20 100644 --- a/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam +++ b/bootstrap/lib/compiler/ebin/v3_kernel_pp.beam diff --git a/bootstrap/lib/kernel/ebin/application.beam b/bootstrap/lib/kernel/ebin/application.beam Binary files differindex 4d04b88ec5..5e90a7734c 100644 --- a/bootstrap/lib/kernel/ebin/application.beam +++ b/bootstrap/lib/kernel/ebin/application.beam diff --git a/bootstrap/lib/kernel/ebin/application_controller.beam b/bootstrap/lib/kernel/ebin/application_controller.beam Binary files differindex ad289c3f35..618dad4942 100644 --- a/bootstrap/lib/kernel/ebin/application_controller.beam +++ b/bootstrap/lib/kernel/ebin/application_controller.beam diff --git a/bootstrap/lib/kernel/ebin/application_master.beam b/bootstrap/lib/kernel/ebin/application_master.beam Binary files differindex 4fae3d515b..c090f903b0 100644 --- a/bootstrap/lib/kernel/ebin/application_master.beam +++ b/bootstrap/lib/kernel/ebin/application_master.beam diff --git a/bootstrap/lib/kernel/ebin/auth.beam b/bootstrap/lib/kernel/ebin/auth.beam Binary files differindex e6c084e5d2..c4e00d2f20 100644 --- a/bootstrap/lib/kernel/ebin/auth.beam +++ b/bootstrap/lib/kernel/ebin/auth.beam diff --git a/bootstrap/lib/kernel/ebin/code.beam b/bootstrap/lib/kernel/ebin/code.beam Binary files differindex fedfbf2164..eafa1d5aa3 100644 --- a/bootstrap/lib/kernel/ebin/code.beam +++ b/bootstrap/lib/kernel/ebin/code.beam diff --git a/bootstrap/lib/kernel/ebin/code_server.beam b/bootstrap/lib/kernel/ebin/code_server.beam Binary files differindex 9b9b091a3c..98cdcde9bf 100644 --- a/bootstrap/lib/kernel/ebin/code_server.beam +++ b/bootstrap/lib/kernel/ebin/code_server.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log.beam b/bootstrap/lib/kernel/ebin/disk_log.beam Binary files differindex a2c576fff5..6f9e212438 100644 --- a/bootstrap/lib/kernel/ebin/disk_log.beam +++ b/bootstrap/lib/kernel/ebin/disk_log.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_1.beam b/bootstrap/lib/kernel/ebin/disk_log_1.beam Binary files differindex e3581d2b18..543620a65f 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_1.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_1.beam diff --git a/bootstrap/lib/kernel/ebin/disk_log_server.beam b/bootstrap/lib/kernel/ebin/disk_log_server.beam Binary files differindex 67582bf7b6..18b975fbaf 100644 --- a/bootstrap/lib/kernel/ebin/disk_log_server.beam +++ b/bootstrap/lib/kernel/ebin/disk_log_server.beam diff --git a/bootstrap/lib/kernel/ebin/dist_ac.beam b/bootstrap/lib/kernel/ebin/dist_ac.beam Binary files differindex e4606dbf30..d1f54037e5 100644 --- a/bootstrap/lib/kernel/ebin/dist_ac.beam +++ b/bootstrap/lib/kernel/ebin/dist_ac.beam diff --git a/bootstrap/lib/kernel/ebin/dist_util.beam b/bootstrap/lib/kernel/ebin/dist_util.beam Binary files differindex 99e50b6053..2a9c561e94 100644 --- a/bootstrap/lib/kernel/ebin/dist_util.beam +++ b/bootstrap/lib/kernel/ebin/dist_util.beam diff --git a/bootstrap/lib/kernel/ebin/erl_boot_server.beam b/bootstrap/lib/kernel/ebin/erl_boot_server.beam Binary files differindex 0f6e93795a..60382dd4ec 100644 --- a/bootstrap/lib/kernel/ebin/erl_boot_server.beam +++ b/bootstrap/lib/kernel/ebin/erl_boot_server.beam diff --git a/bootstrap/lib/kernel/ebin/erl_compile_server.beam b/bootstrap/lib/kernel/ebin/erl_compile_server.beam Binary files differindex 97c108b30d..d11096d521 100644 --- a/bootstrap/lib/kernel/ebin/erl_compile_server.beam +++ b/bootstrap/lib/kernel/ebin/erl_compile_server.beam diff --git a/bootstrap/lib/kernel/ebin/erl_ddll.beam b/bootstrap/lib/kernel/ebin/erl_ddll.beam Binary files differindex a4555a39ab..d1d3bc0d3b 100644 --- a/bootstrap/lib/kernel/ebin/erl_ddll.beam +++ b/bootstrap/lib/kernel/ebin/erl_ddll.beam diff --git a/bootstrap/lib/kernel/ebin/erl_distribution.beam b/bootstrap/lib/kernel/ebin/erl_distribution.beam Binary files differindex d87c17789e..4c499ee49b 100644 --- a/bootstrap/lib/kernel/ebin/erl_distribution.beam +++ b/bootstrap/lib/kernel/ebin/erl_distribution.beam diff --git a/bootstrap/lib/kernel/ebin/erl_epmd.beam b/bootstrap/lib/kernel/ebin/erl_epmd.beam Binary files differindex 7f6ae48cce..28c1a2428f 100644 --- a/bootstrap/lib/kernel/ebin/erl_epmd.beam +++ b/bootstrap/lib/kernel/ebin/erl_epmd.beam diff --git a/bootstrap/lib/kernel/ebin/erl_reply.beam b/bootstrap/lib/kernel/ebin/erl_reply.beam Binary files differindex e74194895d..823ba8cf2c 100644 --- a/bootstrap/lib/kernel/ebin/erl_reply.beam +++ b/bootstrap/lib/kernel/ebin/erl_reply.beam diff --git a/bootstrap/lib/kernel/ebin/erpc.beam b/bootstrap/lib/kernel/ebin/erpc.beam Binary files differindex 820909853b..0f87cabc51 100644 --- a/bootstrap/lib/kernel/ebin/erpc.beam +++ b/bootstrap/lib/kernel/ebin/erpc.beam diff --git a/bootstrap/lib/kernel/ebin/error_logger.beam b/bootstrap/lib/kernel/ebin/error_logger.beam Binary files differindex 928b7787ff..e7e5d58a44 100644 --- a/bootstrap/lib/kernel/ebin/error_logger.beam +++ b/bootstrap/lib/kernel/ebin/error_logger.beam diff --git a/bootstrap/lib/kernel/ebin/erts_debug.beam b/bootstrap/lib/kernel/ebin/erts_debug.beam Binary files differindex f96754ec49..6d4eaa9be5 100644 --- a/bootstrap/lib/kernel/ebin/erts_debug.beam +++ b/bootstrap/lib/kernel/ebin/erts_debug.beam diff --git a/bootstrap/lib/kernel/ebin/file.beam b/bootstrap/lib/kernel/ebin/file.beam Binary files differindex 993967fb09..1e12563771 100644 --- a/bootstrap/lib/kernel/ebin/file.beam +++ b/bootstrap/lib/kernel/ebin/file.beam diff --git a/bootstrap/lib/kernel/ebin/file_io_server.beam b/bootstrap/lib/kernel/ebin/file_io_server.beam Binary files differindex f582978f0c..dc6175e504 100644 --- a/bootstrap/lib/kernel/ebin/file_io_server.beam +++ b/bootstrap/lib/kernel/ebin/file_io_server.beam diff --git a/bootstrap/lib/kernel/ebin/file_server.beam b/bootstrap/lib/kernel/ebin/file_server.beam Binary files differindex 76852ac2c4..8701a0ef65 100644 --- a/bootstrap/lib/kernel/ebin/file_server.beam +++ b/bootstrap/lib/kernel/ebin/file_server.beam diff --git a/bootstrap/lib/kernel/ebin/gen_sctp.beam b/bootstrap/lib/kernel/ebin/gen_sctp.beam Binary files differindex 31d1cc9185..4ee051d780 100644 --- a/bootstrap/lib/kernel/ebin/gen_sctp.beam +++ b/bootstrap/lib/kernel/ebin/gen_sctp.beam diff --git a/bootstrap/lib/kernel/ebin/gen_tcp.beam b/bootstrap/lib/kernel/ebin/gen_tcp.beam Binary files differindex c6d3d52151..6d4d0a2fa6 100644 --- a/bootstrap/lib/kernel/ebin/gen_tcp.beam +++ b/bootstrap/lib/kernel/ebin/gen_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/gen_tcp_socket.beam b/bootstrap/lib/kernel/ebin/gen_tcp_socket.beam Binary files differindex 05a5cfe5ca..7793e1eca9 100644 --- a/bootstrap/lib/kernel/ebin/gen_tcp_socket.beam +++ b/bootstrap/lib/kernel/ebin/gen_tcp_socket.beam diff --git a/bootstrap/lib/kernel/ebin/gen_udp.beam b/bootstrap/lib/kernel/ebin/gen_udp.beam Binary files differindex de97446ecc..d0555b4dc3 100644 --- a/bootstrap/lib/kernel/ebin/gen_udp.beam +++ b/bootstrap/lib/kernel/ebin/gen_udp.beam diff --git a/bootstrap/lib/kernel/ebin/global.beam b/bootstrap/lib/kernel/ebin/global.beam Binary files differindex 616709b5a5..57bb34e2a5 100644 --- a/bootstrap/lib/kernel/ebin/global.beam +++ b/bootstrap/lib/kernel/ebin/global.beam diff --git a/bootstrap/lib/kernel/ebin/global_group.beam b/bootstrap/lib/kernel/ebin/global_group.beam Binary files differindex 5602dca0ac..e0c1cd7f6f 100644 --- a/bootstrap/lib/kernel/ebin/global_group.beam +++ b/bootstrap/lib/kernel/ebin/global_group.beam diff --git a/bootstrap/lib/kernel/ebin/global_search.beam b/bootstrap/lib/kernel/ebin/global_search.beam Binary files differindex 6adc30b7bb..6c5a9d207c 100644 --- a/bootstrap/lib/kernel/ebin/global_search.beam +++ b/bootstrap/lib/kernel/ebin/global_search.beam diff --git a/bootstrap/lib/kernel/ebin/group.beam b/bootstrap/lib/kernel/ebin/group.beam Binary files differindex 88db53afb2..3cb9f92447 100644 --- a/bootstrap/lib/kernel/ebin/group.beam +++ b/bootstrap/lib/kernel/ebin/group.beam diff --git a/bootstrap/lib/kernel/ebin/heart.beam b/bootstrap/lib/kernel/ebin/heart.beam Binary files differindex 55ba5a65cc..39798eca70 100644 --- a/bootstrap/lib/kernel/ebin/heart.beam +++ b/bootstrap/lib/kernel/ebin/heart.beam diff --git a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam Binary files differindex 922e80231d..45c54199e1 100644 --- a/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam +++ b/bootstrap/lib/kernel/ebin/hipe_unified_loader.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex 7c54dc67ee..37130e7a9a 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/inet6_tcp.beam b/bootstrap/lib/kernel/ebin/inet6_tcp.beam Binary files differindex 8e9bfc1fc8..a48597421f 100644 --- a/bootstrap/lib/kernel/ebin/inet6_tcp.beam +++ b/bootstrap/lib/kernel/ebin/inet6_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/inet_config.beam b/bootstrap/lib/kernel/ebin/inet_config.beam Binary files differindex 25b2df43de..a75aabf04f 100644 --- a/bootstrap/lib/kernel/ebin/inet_config.beam +++ b/bootstrap/lib/kernel/ebin/inet_config.beam diff --git a/bootstrap/lib/kernel/ebin/inet_db.beam b/bootstrap/lib/kernel/ebin/inet_db.beam Binary files differindex 783e48517d..d5bac62aed 100644 --- a/bootstrap/lib/kernel/ebin/inet_db.beam +++ b/bootstrap/lib/kernel/ebin/inet_db.beam diff --git a/bootstrap/lib/kernel/ebin/inet_dns.beam b/bootstrap/lib/kernel/ebin/inet_dns.beam Binary files differindex 7baf58a789..66003e4474 100644 --- a/bootstrap/lib/kernel/ebin/inet_dns.beam +++ b/bootstrap/lib/kernel/ebin/inet_dns.beam diff --git a/bootstrap/lib/kernel/ebin/inet_gethost_native.beam b/bootstrap/lib/kernel/ebin/inet_gethost_native.beam Binary files differindex 5b9460572b..ecfb340eae 100644 --- a/bootstrap/lib/kernel/ebin/inet_gethost_native.beam +++ b/bootstrap/lib/kernel/ebin/inet_gethost_native.beam diff --git a/bootstrap/lib/kernel/ebin/inet_parse.beam b/bootstrap/lib/kernel/ebin/inet_parse.beam Binary files differindex 880e065055..9557186443 100644 --- a/bootstrap/lib/kernel/ebin/inet_parse.beam +++ b/bootstrap/lib/kernel/ebin/inet_parse.beam diff --git a/bootstrap/lib/kernel/ebin/inet_res.beam b/bootstrap/lib/kernel/ebin/inet_res.beam Binary files differindex cd9d0502cb..d7dc10c484 100644 --- a/bootstrap/lib/kernel/ebin/inet_res.beam +++ b/bootstrap/lib/kernel/ebin/inet_res.beam diff --git a/bootstrap/lib/kernel/ebin/inet_tcp.beam b/bootstrap/lib/kernel/ebin/inet_tcp.beam Binary files differindex 4658fa2f4e..1d61c1e5ed 100644 --- a/bootstrap/lib/kernel/ebin/inet_tcp.beam +++ b/bootstrap/lib/kernel/ebin/inet_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam b/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam Binary files differindex cd0b9f2543..b6211f3538 100644 --- a/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam +++ b/bootstrap/lib/kernel/ebin/inet_tcp_dist.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.app b/bootstrap/lib/kernel/ebin/kernel.app index 7a881c6eb2..7288d0e0ba 100644 --- a/bootstrap/lib/kernel/ebin/kernel.app +++ b/bootstrap/lib/kernel/ebin/kernel.app @@ -96,6 +96,7 @@ gen_tcp, gen_udp, gen_sctp, + gen_tcp_socket, inet, inet_db, inet_dns, @@ -114,6 +115,7 @@ raw_file_io_list, raw_file_io_raw, seq_trace, + socket, standard_error, wrap_log_reader]}, {registered, [application_controller, diff --git a/bootstrap/lib/kernel/ebin/kernel.beam b/bootstrap/lib/kernel/ebin/kernel.beam Binary files differindex 3d97857282..65c566fe59 100644 --- a/bootstrap/lib/kernel/ebin/kernel.beam +++ b/bootstrap/lib/kernel/ebin/kernel.beam diff --git a/bootstrap/lib/kernel/ebin/kernel_config.beam b/bootstrap/lib/kernel/ebin/kernel_config.beam Binary files differindex f0793bff3b..e310d47355 100644 --- a/bootstrap/lib/kernel/ebin/kernel_config.beam +++ b/bootstrap/lib/kernel/ebin/kernel_config.beam diff --git a/bootstrap/lib/kernel/ebin/kernel_refc.beam b/bootstrap/lib/kernel/ebin/kernel_refc.beam Binary files differindex 96da03d795..fa2e54d616 100644 --- a/bootstrap/lib/kernel/ebin/kernel_refc.beam +++ b/bootstrap/lib/kernel/ebin/kernel_refc.beam diff --git a/bootstrap/lib/kernel/ebin/local_tcp.beam b/bootstrap/lib/kernel/ebin/local_tcp.beam Binary files differindex 49ff2386b3..e25175b07b 100644 --- a/bootstrap/lib/kernel/ebin/local_tcp.beam +++ b/bootstrap/lib/kernel/ebin/local_tcp.beam diff --git a/bootstrap/lib/kernel/ebin/logger.beam b/bootstrap/lib/kernel/ebin/logger.beam Binary files differindex 163d589709..8844841373 100644 --- a/bootstrap/lib/kernel/ebin/logger.beam +++ b/bootstrap/lib/kernel/ebin/logger.beam diff --git a/bootstrap/lib/kernel/ebin/logger_backend.beam b/bootstrap/lib/kernel/ebin/logger_backend.beam Binary files differindex c9e518bff2..f735d40960 100644 --- a/bootstrap/lib/kernel/ebin/logger_backend.beam +++ b/bootstrap/lib/kernel/ebin/logger_backend.beam diff --git a/bootstrap/lib/kernel/ebin/logger_config.beam b/bootstrap/lib/kernel/ebin/logger_config.beam Binary files differindex 8dba727792..e8f35ea43e 100644 --- a/bootstrap/lib/kernel/ebin/logger_config.beam +++ b/bootstrap/lib/kernel/ebin/logger_config.beam diff --git a/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam b/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam Binary files differindex 89eab84dbc..97161d656f 100644 --- a/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam +++ b/bootstrap/lib/kernel/ebin/logger_disk_log_h.beam diff --git a/bootstrap/lib/kernel/ebin/logger_formatter.beam b/bootstrap/lib/kernel/ebin/logger_formatter.beam Binary files differindex c6fd2abc6c..b9b82075ca 100644 --- a/bootstrap/lib/kernel/ebin/logger_formatter.beam +++ b/bootstrap/lib/kernel/ebin/logger_formatter.beam diff --git a/bootstrap/lib/kernel/ebin/logger_h_common.beam b/bootstrap/lib/kernel/ebin/logger_h_common.beam Binary files differindex 423667299b..d565915fed 100644 --- a/bootstrap/lib/kernel/ebin/logger_h_common.beam +++ b/bootstrap/lib/kernel/ebin/logger_h_common.beam diff --git a/bootstrap/lib/kernel/ebin/logger_handler_watcher.beam b/bootstrap/lib/kernel/ebin/logger_handler_watcher.beam Binary files differindex e64ad85427..4fa97161a3 100644 --- a/bootstrap/lib/kernel/ebin/logger_handler_watcher.beam +++ b/bootstrap/lib/kernel/ebin/logger_handler_watcher.beam diff --git a/bootstrap/lib/kernel/ebin/logger_olp.beam b/bootstrap/lib/kernel/ebin/logger_olp.beam Binary files differindex 34f3e402ce..ee1baed8c2 100644 --- a/bootstrap/lib/kernel/ebin/logger_olp.beam +++ b/bootstrap/lib/kernel/ebin/logger_olp.beam diff --git a/bootstrap/lib/kernel/ebin/logger_proxy.beam b/bootstrap/lib/kernel/ebin/logger_proxy.beam Binary files differindex c63397ae61..fbfb960705 100644 --- a/bootstrap/lib/kernel/ebin/logger_proxy.beam +++ b/bootstrap/lib/kernel/ebin/logger_proxy.beam diff --git a/bootstrap/lib/kernel/ebin/logger_server.beam b/bootstrap/lib/kernel/ebin/logger_server.beam Binary files differindex 0fb100687d..f85bbe4234 100644 --- a/bootstrap/lib/kernel/ebin/logger_server.beam +++ b/bootstrap/lib/kernel/ebin/logger_server.beam diff --git a/bootstrap/lib/kernel/ebin/logger_simple_h.beam b/bootstrap/lib/kernel/ebin/logger_simple_h.beam Binary files differindex 0530b0991a..ee0a4ffdf2 100644 --- a/bootstrap/lib/kernel/ebin/logger_simple_h.beam +++ b/bootstrap/lib/kernel/ebin/logger_simple_h.beam diff --git a/bootstrap/lib/kernel/ebin/logger_std_h.beam b/bootstrap/lib/kernel/ebin/logger_std_h.beam Binary files differindex c45585bff9..8f0adc65a3 100644 --- a/bootstrap/lib/kernel/ebin/logger_std_h.beam +++ b/bootstrap/lib/kernel/ebin/logger_std_h.beam diff --git a/bootstrap/lib/kernel/ebin/net.beam b/bootstrap/lib/kernel/ebin/net.beam Binary files differindex 21b60335c6..e14ae8eb5c 100644 --- a/bootstrap/lib/kernel/ebin/net.beam +++ b/bootstrap/lib/kernel/ebin/net.beam diff --git a/bootstrap/lib/kernel/ebin/net_adm.beam b/bootstrap/lib/kernel/ebin/net_adm.beam Binary files differindex 9206927979..ef65337bda 100644 --- a/bootstrap/lib/kernel/ebin/net_adm.beam +++ b/bootstrap/lib/kernel/ebin/net_adm.beam diff --git a/bootstrap/lib/kernel/ebin/net_kernel.beam b/bootstrap/lib/kernel/ebin/net_kernel.beam Binary files differindex 303ee8442a..95deb12ff3 100644 --- a/bootstrap/lib/kernel/ebin/net_kernel.beam +++ b/bootstrap/lib/kernel/ebin/net_kernel.beam diff --git a/bootstrap/lib/kernel/ebin/os.beam b/bootstrap/lib/kernel/ebin/os.beam Binary files differindex 06c4c97c37..261456162d 100644 --- a/bootstrap/lib/kernel/ebin/os.beam +++ b/bootstrap/lib/kernel/ebin/os.beam diff --git a/bootstrap/lib/kernel/ebin/pg.beam b/bootstrap/lib/kernel/ebin/pg.beam Binary files differindex b8235e4ea0..9a0018ee63 100644 --- a/bootstrap/lib/kernel/ebin/pg.beam +++ b/bootstrap/lib/kernel/ebin/pg.beam diff --git a/bootstrap/lib/kernel/ebin/pg2.beam b/bootstrap/lib/kernel/ebin/pg2.beam Binary files differindex b3596675b5..e2e587a71d 100644 --- a/bootstrap/lib/kernel/ebin/pg2.beam +++ b/bootstrap/lib/kernel/ebin/pg2.beam diff --git a/bootstrap/lib/kernel/ebin/ram_file.beam b/bootstrap/lib/kernel/ebin/ram_file.beam Binary files differindex b75394e007..cd32cce3c1 100644 --- a/bootstrap/lib/kernel/ebin/ram_file.beam +++ b/bootstrap/lib/kernel/ebin/ram_file.beam diff --git a/bootstrap/lib/kernel/ebin/raw_file_io.beam b/bootstrap/lib/kernel/ebin/raw_file_io.beam Binary files differindex 3802b11701..2617ebe6af 100644 --- a/bootstrap/lib/kernel/ebin/raw_file_io.beam +++ b/bootstrap/lib/kernel/ebin/raw_file_io.beam diff --git a/bootstrap/lib/kernel/ebin/raw_file_io_compressed.beam b/bootstrap/lib/kernel/ebin/raw_file_io_compressed.beam Binary files differindex 4562900b86..2861389eff 100644 --- a/bootstrap/lib/kernel/ebin/raw_file_io_compressed.beam +++ b/bootstrap/lib/kernel/ebin/raw_file_io_compressed.beam diff --git a/bootstrap/lib/kernel/ebin/raw_file_io_deflate.beam b/bootstrap/lib/kernel/ebin/raw_file_io_deflate.beam Binary files differindex 2af151bdb5..0bab0a081d 100644 --- a/bootstrap/lib/kernel/ebin/raw_file_io_deflate.beam +++ b/bootstrap/lib/kernel/ebin/raw_file_io_deflate.beam diff --git a/bootstrap/lib/kernel/ebin/raw_file_io_delayed.beam b/bootstrap/lib/kernel/ebin/raw_file_io_delayed.beam Binary files differindex 643f928d7d..82c49b4d8e 100644 --- a/bootstrap/lib/kernel/ebin/raw_file_io_delayed.beam +++ b/bootstrap/lib/kernel/ebin/raw_file_io_delayed.beam diff --git a/bootstrap/lib/kernel/ebin/raw_file_io_inflate.beam b/bootstrap/lib/kernel/ebin/raw_file_io_inflate.beam Binary files differindex 2b3dc87ddc..46f8033560 100644 --- a/bootstrap/lib/kernel/ebin/raw_file_io_inflate.beam +++ b/bootstrap/lib/kernel/ebin/raw_file_io_inflate.beam diff --git a/bootstrap/lib/kernel/ebin/raw_file_io_list.beam b/bootstrap/lib/kernel/ebin/raw_file_io_list.beam Binary files differindex 1185a1798f..5522898428 100644 --- a/bootstrap/lib/kernel/ebin/raw_file_io_list.beam +++ b/bootstrap/lib/kernel/ebin/raw_file_io_list.beam diff --git a/bootstrap/lib/kernel/ebin/rpc.beam b/bootstrap/lib/kernel/ebin/rpc.beam Binary files differindex 4751b9f0a6..00740253d0 100644 --- a/bootstrap/lib/kernel/ebin/rpc.beam +++ b/bootstrap/lib/kernel/ebin/rpc.beam diff --git a/bootstrap/lib/kernel/ebin/socket.beam b/bootstrap/lib/kernel/ebin/socket.beam Binary files differnew file mode 100644 index 0000000000..e7af0b892b --- /dev/null +++ b/bootstrap/lib/kernel/ebin/socket.beam diff --git a/bootstrap/lib/kernel/ebin/standard_error.beam b/bootstrap/lib/kernel/ebin/standard_error.beam Binary files differindex bb945e48ee..c6f4a0232d 100644 --- a/bootstrap/lib/kernel/ebin/standard_error.beam +++ b/bootstrap/lib/kernel/ebin/standard_error.beam diff --git a/bootstrap/lib/kernel/ebin/user.beam b/bootstrap/lib/kernel/ebin/user.beam Binary files differindex 2342a29a54..24c2045e56 100644 --- a/bootstrap/lib/kernel/ebin/user.beam +++ b/bootstrap/lib/kernel/ebin/user.beam diff --git a/bootstrap/lib/kernel/ebin/user_drv.beam b/bootstrap/lib/kernel/ebin/user_drv.beam Binary files differindex 77cec15050..cff059790e 100644 --- a/bootstrap/lib/kernel/ebin/user_drv.beam +++ b/bootstrap/lib/kernel/ebin/user_drv.beam diff --git a/bootstrap/lib/kernel/ebin/user_sup.beam b/bootstrap/lib/kernel/ebin/user_sup.beam Binary files differindex b0de6cf7d1..05c7f80e5d 100644 --- a/bootstrap/lib/kernel/ebin/user_sup.beam +++ b/bootstrap/lib/kernel/ebin/user_sup.beam diff --git a/bootstrap/lib/kernel/ebin/wrap_log_reader.beam b/bootstrap/lib/kernel/ebin/wrap_log_reader.beam Binary files differindex 55060bfc79..9ac6df5c7a 100644 --- a/bootstrap/lib/kernel/ebin/wrap_log_reader.beam +++ b/bootstrap/lib/kernel/ebin/wrap_log_reader.beam diff --git a/bootstrap/lib/stdlib/ebin/array.beam b/bootstrap/lib/stdlib/ebin/array.beam Binary files differindex 88db6d5957..d8819f13e6 100644 --- a/bootstrap/lib/stdlib/ebin/array.beam +++ b/bootstrap/lib/stdlib/ebin/array.beam diff --git a/bootstrap/lib/stdlib/ebin/beam_lib.beam b/bootstrap/lib/stdlib/ebin/beam_lib.beam Binary files differindex 42b21db164..cb3b35e6ba 100644 --- a/bootstrap/lib/stdlib/ebin/beam_lib.beam +++ b/bootstrap/lib/stdlib/ebin/beam_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/binary.beam b/bootstrap/lib/stdlib/ebin/binary.beam Binary files differindex a5022bed68..451d95746b 100644 --- a/bootstrap/lib/stdlib/ebin/binary.beam +++ b/bootstrap/lib/stdlib/ebin/binary.beam diff --git a/bootstrap/lib/stdlib/ebin/c.beam b/bootstrap/lib/stdlib/ebin/c.beam Binary files differindex f30a98d5e7..6407a36fda 100644 --- a/bootstrap/lib/stdlib/ebin/c.beam +++ b/bootstrap/lib/stdlib/ebin/c.beam diff --git a/bootstrap/lib/stdlib/ebin/calendar.beam b/bootstrap/lib/stdlib/ebin/calendar.beam Binary files differindex 164a380330..3515420af0 100644 --- a/bootstrap/lib/stdlib/ebin/calendar.beam +++ b/bootstrap/lib/stdlib/ebin/calendar.beam diff --git a/bootstrap/lib/stdlib/ebin/dets.beam b/bootstrap/lib/stdlib/ebin/dets.beam Binary files differindex acfeed3c3c..20db2e7d82 100644 --- a/bootstrap/lib/stdlib/ebin/dets.beam +++ b/bootstrap/lib/stdlib/ebin/dets.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_server.beam b/bootstrap/lib/stdlib/ebin/dets_server.beam Binary files differindex 967267b9c2..f704dfb58e 100644 --- a/bootstrap/lib/stdlib/ebin/dets_server.beam +++ b/bootstrap/lib/stdlib/ebin/dets_server.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_utils.beam b/bootstrap/lib/stdlib/ebin/dets_utils.beam Binary files differindex 47a2f53956..1d3481448e 100644 --- a/bootstrap/lib/stdlib/ebin/dets_utils.beam +++ b/bootstrap/lib/stdlib/ebin/dets_utils.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v9.beam b/bootstrap/lib/stdlib/ebin/dets_v9.beam Binary files differindex 31351a678d..69822567ef 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v9.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v9.beam diff --git a/bootstrap/lib/stdlib/ebin/dict.beam b/bootstrap/lib/stdlib/ebin/dict.beam Binary files differindex 8ac1457af5..6b0b4f7327 100644 --- a/bootstrap/lib/stdlib/ebin/dict.beam +++ b/bootstrap/lib/stdlib/ebin/dict.beam diff --git a/bootstrap/lib/stdlib/ebin/digraph.beam b/bootstrap/lib/stdlib/ebin/digraph.beam Binary files differindex aeca4eaf86..b28fe8aae1 100644 --- a/bootstrap/lib/stdlib/ebin/digraph.beam +++ b/bootstrap/lib/stdlib/ebin/digraph.beam diff --git a/bootstrap/lib/stdlib/ebin/digraph_utils.beam b/bootstrap/lib/stdlib/ebin/digraph_utils.beam Binary files differindex bdd382fec7..0c0cc94b09 100644 --- a/bootstrap/lib/stdlib/ebin/digraph_utils.beam +++ b/bootstrap/lib/stdlib/ebin/digraph_utils.beam diff --git a/bootstrap/lib/stdlib/ebin/edlin.beam b/bootstrap/lib/stdlib/ebin/edlin.beam Binary files differindex e1d0f2ce4e..cbc8a9d3d0 100644 --- a/bootstrap/lib/stdlib/ebin/edlin.beam +++ b/bootstrap/lib/stdlib/ebin/edlin.beam diff --git a/bootstrap/lib/stdlib/ebin/edlin_expand.beam b/bootstrap/lib/stdlib/ebin/edlin_expand.beam Binary files differindex 59237a9413..32833b51da 100644 --- a/bootstrap/lib/stdlib/ebin/edlin_expand.beam +++ b/bootstrap/lib/stdlib/ebin/edlin_expand.beam diff --git a/bootstrap/lib/stdlib/ebin/epp.beam b/bootstrap/lib/stdlib/ebin/epp.beam Binary files differindex edbd5ec0ed..fe630ad03d 100644 --- a/bootstrap/lib/stdlib/ebin/epp.beam +++ b/bootstrap/lib/stdlib/ebin/epp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam b/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam Binary files differindex 8046b67672..0759d1a464 100644 --- a/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam +++ b/bootstrap/lib/stdlib/ebin/erl_abstract_code.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_bits.beam b/bootstrap/lib/stdlib/ebin/erl_bits.beam Binary files differindex 80daf0155a..8714ea5f00 100644 --- a/bootstrap/lib/stdlib/ebin/erl_bits.beam +++ b/bootstrap/lib/stdlib/ebin/erl_bits.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_compile.beam b/bootstrap/lib/stdlib/ebin/erl_compile.beam Binary files differindex 9c93ed3076..7c6f7f924d 100644 --- a/bootstrap/lib/stdlib/ebin/erl_compile.beam +++ b/bootstrap/lib/stdlib/ebin/erl_compile.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_error.beam b/bootstrap/lib/stdlib/ebin/erl_error.beam Binary files differindex 6de6d60c45..fadaecea89 100644 --- a/bootstrap/lib/stdlib/ebin/erl_error.beam +++ b/bootstrap/lib/stdlib/ebin/erl_error.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_eval.beam b/bootstrap/lib/stdlib/ebin/erl_eval.beam Binary files differindex 6df4543d5f..6e346923fa 100644 --- a/bootstrap/lib/stdlib/ebin/erl_eval.beam +++ b/bootstrap/lib/stdlib/ebin/erl_eval.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam Binary files differindex c942d68d59..c196ae05cf 100644 --- a/bootstrap/lib/stdlib/ebin/erl_expand_records.beam +++ b/bootstrap/lib/stdlib/ebin/erl_expand_records.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_internal.beam b/bootstrap/lib/stdlib/ebin/erl_internal.beam Binary files differindex 814f9d7faf..e48f3a1b3e 100644 --- a/bootstrap/lib/stdlib/ebin/erl_internal.beam +++ b/bootstrap/lib/stdlib/ebin/erl_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 3be79b9828..11c5f1c1cf 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_parse.beam b/bootstrap/lib/stdlib/ebin/erl_parse.beam Binary files differindex 4f5c45af49..1f7680b6b2 100644 --- a/bootstrap/lib/stdlib/ebin/erl_parse.beam +++ b/bootstrap/lib/stdlib/ebin/erl_parse.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_pp.beam b/bootstrap/lib/stdlib/ebin/erl_pp.beam Binary files differindex fd3db98635..6cfad9980b 100644 --- a/bootstrap/lib/stdlib/ebin/erl_pp.beam +++ b/bootstrap/lib/stdlib/ebin/erl_pp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_scan.beam b/bootstrap/lib/stdlib/ebin/erl_scan.beam Binary files differindex 2628c38a23..1e3b49e695 100644 --- a/bootstrap/lib/stdlib/ebin/erl_scan.beam +++ b/bootstrap/lib/stdlib/ebin/erl_scan.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_tar.beam b/bootstrap/lib/stdlib/ebin/erl_tar.beam Binary files differindex 4c840b99f4..8fecbca62a 100644 --- a/bootstrap/lib/stdlib/ebin/erl_tar.beam +++ b/bootstrap/lib/stdlib/ebin/erl_tar.beam diff --git a/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam b/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam Binary files differindex 0d6208ce45..ef98fc4293 100644 --- a/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam +++ b/bootstrap/lib/stdlib/ebin/error_logger_file_h.beam diff --git a/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam b/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam Binary files differindex 4094f0d739..5326c31d44 100644 --- a/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam +++ b/bootstrap/lib/stdlib/ebin/error_logger_tty_h.beam diff --git a/bootstrap/lib/stdlib/ebin/escript.beam b/bootstrap/lib/stdlib/ebin/escript.beam Binary files differindex 069ee49e16..3e535cdd7d 100644 --- a/bootstrap/lib/stdlib/ebin/escript.beam +++ b/bootstrap/lib/stdlib/ebin/escript.beam diff --git a/bootstrap/lib/stdlib/ebin/ets.beam b/bootstrap/lib/stdlib/ebin/ets.beam Binary files differindex 842f1440f1..f80809f26e 100644 --- a/bootstrap/lib/stdlib/ebin/ets.beam +++ b/bootstrap/lib/stdlib/ebin/ets.beam diff --git a/bootstrap/lib/stdlib/ebin/eval_bits.beam b/bootstrap/lib/stdlib/ebin/eval_bits.beam Binary files differindex 71bffd4249..92a7e8d1de 100644 --- a/bootstrap/lib/stdlib/ebin/eval_bits.beam +++ b/bootstrap/lib/stdlib/ebin/eval_bits.beam diff --git a/bootstrap/lib/stdlib/ebin/file_sorter.beam b/bootstrap/lib/stdlib/ebin/file_sorter.beam Binary files differindex 7c49a54f07..672a43d820 100644 --- a/bootstrap/lib/stdlib/ebin/file_sorter.beam +++ b/bootstrap/lib/stdlib/ebin/file_sorter.beam diff --git a/bootstrap/lib/stdlib/ebin/filelib.beam b/bootstrap/lib/stdlib/ebin/filelib.beam Binary files differindex 5e591db2cc..389dbd012a 100644 --- a/bootstrap/lib/stdlib/ebin/filelib.beam +++ b/bootstrap/lib/stdlib/ebin/filelib.beam diff --git a/bootstrap/lib/stdlib/ebin/filename.beam b/bootstrap/lib/stdlib/ebin/filename.beam Binary files differindex 1215a0fc62..68fa20fdb9 100644 --- a/bootstrap/lib/stdlib/ebin/filename.beam +++ b/bootstrap/lib/stdlib/ebin/filename.beam diff --git a/bootstrap/lib/stdlib/ebin/gb_sets.beam b/bootstrap/lib/stdlib/ebin/gb_sets.beam Binary files differindex 7f5ff986f7..298f49f6bc 100644 --- a/bootstrap/lib/stdlib/ebin/gb_sets.beam +++ b/bootstrap/lib/stdlib/ebin/gb_sets.beam diff --git a/bootstrap/lib/stdlib/ebin/gb_trees.beam b/bootstrap/lib/stdlib/ebin/gb_trees.beam Binary files differindex b5f394689c..d5b69b24e1 100644 --- a/bootstrap/lib/stdlib/ebin/gb_trees.beam +++ b/bootstrap/lib/stdlib/ebin/gb_trees.beam diff --git a/bootstrap/lib/stdlib/ebin/gen.beam b/bootstrap/lib/stdlib/ebin/gen.beam Binary files differindex f6ee64e845..574d24bdbf 100644 --- a/bootstrap/lib/stdlib/ebin/gen.beam +++ b/bootstrap/lib/stdlib/ebin/gen.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_event.beam b/bootstrap/lib/stdlib/ebin/gen_event.beam Binary files differindex 8afdacb45e..f3c5564736 100644 --- a/bootstrap/lib/stdlib/ebin/gen_event.beam +++ b/bootstrap/lib/stdlib/ebin/gen_event.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_fsm.beam b/bootstrap/lib/stdlib/ebin/gen_fsm.beam Binary files differindex a58e1256f4..5d7a91dc91 100644 --- a/bootstrap/lib/stdlib/ebin/gen_fsm.beam +++ b/bootstrap/lib/stdlib/ebin/gen_fsm.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_server.beam b/bootstrap/lib/stdlib/ebin/gen_server.beam Binary files differindex 13e71bb2cb..b697a060e5 100644 --- a/bootstrap/lib/stdlib/ebin/gen_server.beam +++ b/bootstrap/lib/stdlib/ebin/gen_server.beam diff --git a/bootstrap/lib/stdlib/ebin/gen_statem.beam b/bootstrap/lib/stdlib/ebin/gen_statem.beam Binary files differindex 4ed5efb2d1..e226e9cf3a 100644 --- a/bootstrap/lib/stdlib/ebin/gen_statem.beam +++ b/bootstrap/lib/stdlib/ebin/gen_statem.beam diff --git a/bootstrap/lib/stdlib/ebin/io.beam b/bootstrap/lib/stdlib/ebin/io.beam Binary files differindex 3eff6444fc..60fbdadc21 100644 --- a/bootstrap/lib/stdlib/ebin/io.beam +++ b/bootstrap/lib/stdlib/ebin/io.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib.beam b/bootstrap/lib/stdlib/ebin/io_lib.beam Binary files differindex 0bde1f95fe..94380578cc 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_format.beam b/bootstrap/lib/stdlib/ebin/io_lib_format.beam Binary files differindex f420f6ab6a..daed34b0d3 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_format.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_format.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_fread.beam b/bootstrap/lib/stdlib/ebin/io_lib_fread.beam Binary files differindex 9d120987b5..e180b1fcb3 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_fread.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_fread.beam diff --git a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam Binary files differindex 193c0984ae..7c821ff6c1 100644 --- a/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam +++ b/bootstrap/lib/stdlib/ebin/io_lib_pretty.beam diff --git a/bootstrap/lib/stdlib/ebin/lists.beam b/bootstrap/lib/stdlib/ebin/lists.beam Binary files differindex 9b25bbd5d8..233b63982e 100644 --- a/bootstrap/lib/stdlib/ebin/lists.beam +++ b/bootstrap/lib/stdlib/ebin/lists.beam diff --git a/bootstrap/lib/stdlib/ebin/log_mf_h.beam b/bootstrap/lib/stdlib/ebin/log_mf_h.beam Binary files differindex 743487b523..0304375c3e 100644 --- a/bootstrap/lib/stdlib/ebin/log_mf_h.beam +++ b/bootstrap/lib/stdlib/ebin/log_mf_h.beam diff --git a/bootstrap/lib/stdlib/ebin/maps.beam b/bootstrap/lib/stdlib/ebin/maps.beam Binary files differindex 3aec184bf5..a5c10ae33a 100644 --- a/bootstrap/lib/stdlib/ebin/maps.beam +++ b/bootstrap/lib/stdlib/ebin/maps.beam diff --git a/bootstrap/lib/stdlib/ebin/ms_transform.beam b/bootstrap/lib/stdlib/ebin/ms_transform.beam Binary files differindex 99492a51ce..615deb46cf 100644 --- a/bootstrap/lib/stdlib/ebin/ms_transform.beam +++ b/bootstrap/lib/stdlib/ebin/ms_transform.beam diff --git a/bootstrap/lib/stdlib/ebin/orddict.beam b/bootstrap/lib/stdlib/ebin/orddict.beam Binary files differindex 5a3ada2bc5..1e7a73ec2a 100644 --- a/bootstrap/lib/stdlib/ebin/orddict.beam +++ b/bootstrap/lib/stdlib/ebin/orddict.beam diff --git a/bootstrap/lib/stdlib/ebin/otp_internal.beam b/bootstrap/lib/stdlib/ebin/otp_internal.beam Binary files differindex 1dbdd7f102..547f105df8 100644 --- a/bootstrap/lib/stdlib/ebin/otp_internal.beam +++ b/bootstrap/lib/stdlib/ebin/otp_internal.beam diff --git a/bootstrap/lib/stdlib/ebin/pool.beam b/bootstrap/lib/stdlib/ebin/pool.beam Binary files differindex 698a8d3bd1..1891f85ecb 100644 --- a/bootstrap/lib/stdlib/ebin/pool.beam +++ b/bootstrap/lib/stdlib/ebin/pool.beam diff --git a/bootstrap/lib/stdlib/ebin/proc_lib.beam b/bootstrap/lib/stdlib/ebin/proc_lib.beam Binary files differindex 54b3f00dd7..ab699fba83 100644 --- a/bootstrap/lib/stdlib/ebin/proc_lib.beam +++ b/bootstrap/lib/stdlib/ebin/proc_lib.beam diff --git a/bootstrap/lib/stdlib/ebin/proplists.beam b/bootstrap/lib/stdlib/ebin/proplists.beam Binary files differindex 247c7ca209..85f8f0af36 100644 --- a/bootstrap/lib/stdlib/ebin/proplists.beam +++ b/bootstrap/lib/stdlib/ebin/proplists.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc.beam b/bootstrap/lib/stdlib/ebin/qlc.beam Binary files differindex 3bade1a5b1..8e608d07a3 100644 --- a/bootstrap/lib/stdlib/ebin/qlc.beam +++ b/bootstrap/lib/stdlib/ebin/qlc.beam diff --git a/bootstrap/lib/stdlib/ebin/qlc_pt.beam b/bootstrap/lib/stdlib/ebin/qlc_pt.beam Binary files differindex c8dc7c68a3..0b69f638fb 100644 --- a/bootstrap/lib/stdlib/ebin/qlc_pt.beam +++ b/bootstrap/lib/stdlib/ebin/qlc_pt.beam diff --git a/bootstrap/lib/stdlib/ebin/queue.beam b/bootstrap/lib/stdlib/ebin/queue.beam Binary files differindex 05423bc36a..a29c6917fe 100644 --- a/bootstrap/lib/stdlib/ebin/queue.beam +++ b/bootstrap/lib/stdlib/ebin/queue.beam diff --git a/bootstrap/lib/stdlib/ebin/rand.beam b/bootstrap/lib/stdlib/ebin/rand.beam Binary files differindex 04326f14a1..392e99b528 100644 --- a/bootstrap/lib/stdlib/ebin/rand.beam +++ b/bootstrap/lib/stdlib/ebin/rand.beam diff --git a/bootstrap/lib/stdlib/ebin/re.beam b/bootstrap/lib/stdlib/ebin/re.beam Binary files differindex 124c1fc53c..0de43ec600 100644 --- a/bootstrap/lib/stdlib/ebin/re.beam +++ b/bootstrap/lib/stdlib/ebin/re.beam diff --git a/bootstrap/lib/stdlib/ebin/sets.beam b/bootstrap/lib/stdlib/ebin/sets.beam Binary files differindex 41fc00ad23..3dcb065519 100644 --- a/bootstrap/lib/stdlib/ebin/sets.beam +++ b/bootstrap/lib/stdlib/ebin/sets.beam diff --git a/bootstrap/lib/stdlib/ebin/shell.beam b/bootstrap/lib/stdlib/ebin/shell.beam Binary files differindex a370e30981..82dcca6255 100644 --- a/bootstrap/lib/stdlib/ebin/shell.beam +++ b/bootstrap/lib/stdlib/ebin/shell.beam diff --git a/bootstrap/lib/stdlib/ebin/shell_default.beam b/bootstrap/lib/stdlib/ebin/shell_default.beam Binary files differindex f6deb33d6b..d4f079dc45 100644 --- a/bootstrap/lib/stdlib/ebin/shell_default.beam +++ b/bootstrap/lib/stdlib/ebin/shell_default.beam diff --git a/bootstrap/lib/stdlib/ebin/shell_docs.beam b/bootstrap/lib/stdlib/ebin/shell_docs.beam Binary files differindex 844f0b1667..658e97a4e7 100644 --- a/bootstrap/lib/stdlib/ebin/shell_docs.beam +++ b/bootstrap/lib/stdlib/ebin/shell_docs.beam diff --git a/bootstrap/lib/stdlib/ebin/slave.beam b/bootstrap/lib/stdlib/ebin/slave.beam Binary files differindex 97bea03b29..af6a3a4a5b 100644 --- a/bootstrap/lib/stdlib/ebin/slave.beam +++ b/bootstrap/lib/stdlib/ebin/slave.beam diff --git a/bootstrap/lib/stdlib/ebin/sofs.beam b/bootstrap/lib/stdlib/ebin/sofs.beam Binary files differindex c63539796e..b593b134fc 100644 --- a/bootstrap/lib/stdlib/ebin/sofs.beam +++ b/bootstrap/lib/stdlib/ebin/sofs.beam diff --git a/bootstrap/lib/stdlib/ebin/stdlib.app b/bootstrap/lib/stdlib/ebin/stdlib.app index 9b72762923..26d27e9986 100644 --- a/bootstrap/lib/stdlib/ebin/stdlib.app +++ b/bootstrap/lib/stdlib/ebin/stdlib.app @@ -20,7 +20,7 @@ %% {application, stdlib, [{description, "ERTS CXC 138 10"}, - {vsn, "3.12"}, + {vsn, "3.12.1"}, {modules, [array, base64, beam_lib, @@ -109,6 +109,6 @@ dets]}, {applications, [kernel]}, {env, []}, - {runtime_dependencies, ["sasl-3.0","kernel-@OTP-15251@","erts-@OTP-15251:OTP-16431:OTP-16553@","crypto-3.3", + {runtime_dependencies, ["sasl-3.0","kernel-@OTP-15251@","erts-@OTP-15251:OTP-16431@","crypto-3.3", "compiler-5.0"]} ]}. diff --git a/bootstrap/lib/stdlib/ebin/string.beam b/bootstrap/lib/stdlib/ebin/string.beam Binary files differindex d99b287e0e..63a26170d1 100644 --- a/bootstrap/lib/stdlib/ebin/string.beam +++ b/bootstrap/lib/stdlib/ebin/string.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex 4e138ed238..42d19e46f3 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam Binary files differindex d11613b068..2cdc7c5142 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor_bridge.beam diff --git a/bootstrap/lib/stdlib/ebin/sys.beam b/bootstrap/lib/stdlib/ebin/sys.beam Binary files differindex 1a0a612b09..9d86998ce1 100644 --- a/bootstrap/lib/stdlib/ebin/sys.beam +++ b/bootstrap/lib/stdlib/ebin/sys.beam diff --git a/bootstrap/lib/stdlib/ebin/timer.beam b/bootstrap/lib/stdlib/ebin/timer.beam Binary files differindex b6b67e8f44..bfd4057f29 100644 --- a/bootstrap/lib/stdlib/ebin/timer.beam +++ b/bootstrap/lib/stdlib/ebin/timer.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode.beam b/bootstrap/lib/stdlib/ebin/unicode.beam Binary files differindex c65febb5ca..024dfce787 100644 --- a/bootstrap/lib/stdlib/ebin/unicode.beam +++ b/bootstrap/lib/stdlib/ebin/unicode.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode_util.beam b/bootstrap/lib/stdlib/ebin/unicode_util.beam Binary files differindex 6867a779e8..5d6e4c977e 100644 --- a/bootstrap/lib/stdlib/ebin/unicode_util.beam +++ b/bootstrap/lib/stdlib/ebin/unicode_util.beam diff --git a/bootstrap/lib/stdlib/ebin/uri_string.beam b/bootstrap/lib/stdlib/ebin/uri_string.beam Binary files differindex 70b6213783..1557c34f51 100644 --- a/bootstrap/lib/stdlib/ebin/uri_string.beam +++ b/bootstrap/lib/stdlib/ebin/uri_string.beam diff --git a/bootstrap/lib/stdlib/ebin/win32reg.beam b/bootstrap/lib/stdlib/ebin/win32reg.beam Binary files differindex c9c29747a0..e12757251b 100644 --- a/bootstrap/lib/stdlib/ebin/win32reg.beam +++ b/bootstrap/lib/stdlib/ebin/win32reg.beam diff --git a/bootstrap/lib/stdlib/ebin/zip.beam b/bootstrap/lib/stdlib/ebin/zip.beam Binary files differindex dbc259f8f3..b68b8136ad 100644 --- a/bootstrap/lib/stdlib/ebin/zip.beam +++ b/bootstrap/lib/stdlib/ebin/zip.beam diff --git a/erts/Makefile b/erts/Makefile index 5ca0167fb3..7808dccecc 100644 --- a/erts/Makefile +++ b/erts/Makefile @@ -79,6 +79,7 @@ local_setup: cp $(ERL_TOP)/bin/$(TARGET)/ct_run.exe $(ERL_TOP)/bin/ct_run.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/erlc.exe $(ERL_TOP)/bin/erlc.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/erl.exe $(ERL_TOP)/bin/erl.exe; \ + cp $(ERL_TOP)/bin/$(TARGET)/erl_call.exe $(ERL_TOP)/bin/erl_call.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/werl.exe $(ERL_TOP)/bin/werl.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/escript.exe $(ERL_TOP)/bin/escript.exe; \ chmod 755 $(ERL_TOP)/bin/erl.exe $(ERL_TOP)/bin/erlc.exe \ @@ -94,6 +95,7 @@ local_setup: -e "s;%TARGET%;$(TARGET);" \ -e "s;%VSN%;$(VSN);" \ $(ERL_TOP)/erts/etc/unix/cerl.src > $(ERL_TOP)/bin/cerl; \ + cp $(ERL_TOP)/bin/$(TARGET)/erl_call $(ERL_TOP)/bin/erl_call; \ cp $(ERL_TOP)/bin/$(TARGET)/dialyzer $(ERL_TOP)/bin/dialyzer; \ cp $(ERL_TOP)/bin/$(TARGET)/typer $(ERL_TOP)/bin/typer; \ cp $(ERL_TOP)/bin/$(TARGET)/ct_run $(ERL_TOP)/bin/ct_run; \ diff --git a/erts/configure.in b/erts/configure.in index 368ca5b6cf..9f169d9cc5 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -1389,6 +1389,19 @@ fi AC_SUBST(USE_ESOCK) +dnl *** ESOCK_USE_RCVSNDTIMEO *** + +AC_ARG_ENABLE(esock_use_rcvsndtimeo, +AS_HELP_STRING([--enable-esock-rcvsndtimeo], [enable use of the option(s) rcvtimeo and sndtimeo]) +AS_HELP_STRING([--disable-esock-rcvsndtimeo], [disable use of the option(s) rcvtimeo and sndtimeo (default)])) + +if test "x$enable_esock_rcvsndtimeo" = "xyes"; then + AC_DEFINE(ESOCK_USE_RCVSNDTIMEO, [1], [Use SO_[RCV|SND]TMIEO]) +fi + + +dnl *** ESOCK_COUNTER_SIZE *** + AC_ARG_WITH(esock-counter-size, AS_HELP_STRING([--with-esock-counter-size=SZ], [Size of the esock counters, in number of bits: 16 | 24 | 32 | 48 | 64; default is 64]), diff --git a/erts/doc/src/Makefile b/erts/doc/src/Makefile index 3e2eb80b50..41ade2bba7 100644 --- a/erts/doc/src/Makefile +++ b/erts/doc/src/Makefile @@ -47,20 +47,6 @@ XML_REF1_FILES = epmd_cmd.xml \ run_erl_cmd.xml \ start_cmd.xml -ifeq ($(USE_ESOCK), yes) -XML_REF3_ESOCK_EFILES = socket.xml -XML_CHAPTER_ESOCK_EFILES = socket_usage.xml -ESOCK_USE_SOCKET_XML=<xi:include href="socket.xml"\/> -ESOCK_USE_SOCKET_SPECS_XML=<xi:include href="../specs/specs_socket.xml"/> -ESOCK_USE_SOCKET_USAGE_XML=<xi:include href="socket_usage.xml"/> -else -XML_REF3_ESOCK_EFILES = -XML_CHAPTER_ESOCK_EFILES = -ESOCK_USE_SOCKET_XML = -ESOCK_USE_SOCKET_SPECS_XML = -ESOCK_USE_SOCKET_USAGE_XML = -endif - XML_REF3_EFILES = \ erl_prim_loader.xml \ erlang.xml \ @@ -69,8 +55,7 @@ XML_REF3_EFILES = \ persistent_term.xml \ atomics.xml \ counters.xml \ - zlib.xml \ - $(XML_REF3_ESOCK_EFILES) + zlib.xml XML_REF3_CREF = \ driver_entry.xml \ @@ -109,7 +94,6 @@ XML_CHAPTER_FILES = \ driver.xml \ absform.xml \ inet_cfg.xml \ - $(XML_CHAPTER_ESOCK_EFILES) \ erl_ext_dist.xml \ erl_dist_protocol.xml \ communication.xml \ @@ -171,18 +155,6 @@ $(XML_REF3_CREF:%.xml=$(SPECDIR)/specs_%.xml): $(@:%.xml=%.xml) $(XMLDIR)/%.xml: ../../emulator/internal_doc/%.md $(ERL_TOP)/make/emd2exml $(ERL_TOP)/make/emd2exml $< $@ -ref_man.xml: ref_man.xml.src - ($(PERL) -p -e 's?%ESOCK_USE_SOCKET_XML%?$(ESOCK_USE_SOCKET_XML)?' \ - $<) > $@ - -part.xml: part.xml.src - ($(PERL) -p -e 's?%ESOCK_USE_SOCKET_USAGE_XML%?$(ESOCK_USE_SOCKET_USAGE_XML)?' \ - $<) > $@ - -specs.xml: specs.xml.src - ($(PERL) -p -e 's?%ESOCK_USE_SOCKET_SPECS_XML%?$(ESOCK_USE_SOCKET_SPECS_XML)?' \ - $<) > $@ - # ---------------------------------------------------- # Release Target diff --git a/erts/doc/src/erl_cmd.xml b/erts/doc/src/erl_cmd.xml index 2f37006822..1ca81a4798 100644 --- a/erts/doc/src/erl_cmd.xml +++ b/erts/doc/src/erl_cmd.xml @@ -286,7 +286,7 @@ see <seeerl marker="kernel:heart"> <c>heart(3)</c></seeerl>.</p> </item> - <tag><c><![CDATA[-hidden]]></c></tag> + <tag><marker id="hidden"/><c><![CDATA[-hidden]]></c></tag> <item> <p>Starts the Erlang runtime system as a hidden node, if it is run as a distributed node. Hidden nodes always establish @@ -373,7 +373,7 @@ modules, as conventionally happens in OTP releases. See <seeerl marker="kernel:code"><c>code(3)</c></seeerl></p>. </item> - <tag><c><![CDATA[-name Name]]></c></tag> + <tag><marker id="name"/><c><![CDATA[-name Name]]></c></tag> <item> <p>Makes the Erlang runtime system into a distributed node. This flag invokes all network servers necessary for a node to @@ -386,6 +386,31 @@ <c><![CDATA[Host]]></c> is the fully qualified host name of the current host. For short names, use flag <c><![CDATA[-sname]]></c> instead.</p> + <p>If <c>Name</c> is set to <em><c>undefined</c></em> the node will + be started in a special mode optimized to be the temporary client + of another node. When enabled the node will request a + <seeguide marker="erl_dist_protocol#DFLAG_NAME_ME">dynamic + node name</seeguide> from the first node it connects to. In addition + these distribution settings will be set: + </p> + <pre><seeapp marker="kernel:kernel_app#dist_listen">-dist_listen false</seeapp> <seecom marker="#hidden">-hidden</seecom> <seeapp marker="kernel:kernel_app#dist_auto_connect">-dist_auto_connect never</seeapp></pre> + <p>Because <c>-dist_auto_connect</c> is set to <c>never</c>, the system will have + to manually call <seemfa marker="kernel:net_kernel#connect_node/1"> + <c>net_kernel:connect_node/1</c></seemfa> in order to start the distribution. + If the distribution channel is closed, when a node uses a dynamic + node name, the node will stop the distribution and a new call to + <seemfa marker="kernel:net_kernel#connect_node/1"> + <c>net_kernel:connect_node/1</c></seemfa> has to be made. Note that the node + name may change if the distribution is dropped and then set up again. + </p> + <note> + <p> + The <em>dynamic node name</em> feature is supported from + OTP 23. Both the temporary client node and the first + connected peer node (supplying the dynamic node name) must + be at least OTP 23 for it to work. + </p> + </note> <warning> <p> Starting a distributed node without also specifying @@ -453,7 +478,7 @@ <tag><c>inet_tls</c></tag> <item>Distribution over TLS/SSL, See the <seeguide marker="ssl:ssl_distribution"> - Using SSL for Erlang Distribution</seeguide> User's Guide + Using SSL for Erlang Distribution</seeguide> User's Guide for details on how to setup a secure distributed node. </item> <tag><c>inet6_tcp</c></tag> @@ -463,13 +488,18 @@ <pre> % <input>erl -name test@ipv6node.example.com -proto_dist inet6_tcp</input></pre> </item> - <tag><c><![CDATA[-remsh Node]]></c></tag> + <tag><marker id="remsh"/><c><![CDATA[-remsh Node]]></c></tag> <item> - <p>Starts Erlang with a remote shell connected to - <c><![CDATA[Node]]></c>. Requires either <c><![CDATA[-name]]></c> - or <c><![CDATA[-sname]]></c> to be given. If <c><![CDATA[Node]]></c> - does not contain a hostname, one is automatically taken from - <c><![CDATA[-name]]></c> or <c><![CDATA[-sname]]></c></p> + <p>Starts Erlang with a remote shell connected to <c><![CDATA[Node]]></c>.</p> + <p>If no <c><![CDATA[-name]]></c> or <c><![CDATA[-sname]]></c> is given + the node will be started using <c>-sname undefined</c>. If <c>Node</c> + is using long names then you should give <c>-name undefined</c>. + If <c><![CDATA[Node]]></c> does not contain a hostname, one is automatically + taken from <c><![CDATA[-name]]></c> or <c><![CDATA[-sname]]></c> + </p> + <note><p>Before OTP-23 the user <em>needed</em> to supply a valid <c>-sname</c> or + <c>-name</c> for <c>-remsh</c> to work. This is still the case if the target + node is not running OTP-23 or later.</p></note> </item> <tag><c><![CDATA[-rsh Program]]></c></tag> <item> @@ -514,9 +544,9 @@ <tag><c><![CDATA[-sname Name]]></c></tag> <item> <p>Makes the Erlang runtime system into a distributed node, similar to - <c><![CDATA[-name]]></c>, but the host name portion of the node - name <c><![CDATA[Name@Host]]></c> will be the short name, not fully - qualified.</p> + <seecom marker="#name"><c><![CDATA[-name]]></c></seecom>, but the host + name portion of the node name <c><![CDATA[Name@Host]]></c> will be the + short name, not fully qualified.</p> <p>This is sometimes the only way to run distributed Erlang if the Domain Name System (DNS) is not running. No communication can exist between nodes running with flag <c><![CDATA[-sname]]></c> @@ -691,12 +721,12 @@ code points > 255.</p> <p>For more information about Unicode filenames, see section <seeguide marker="stdlib:unicode_usage#unicode_file_names">Unicode - Filenames</seeguide> in the STDLIB User's Guide. Notice that + Filenames</seeguide> in the STDLIB User's Guide. Notice that this value also applies to command-line parameters and environment variables (see section <seeguide marker="stdlib:unicode_usage#unicode_in_environment_and_parameters"> Unicode in Environment and Parameters</seeguide> in the STDLIB - User's Guide).</p> + User's Guide).</p> </item> <tag><c><![CDATA[+fnu[{w|i|e}]]]></c></tag> <item> @@ -727,12 +757,12 @@ points to an invalid filename.</p> <p>For more information about Unicode filenames, see section <seeguide marker="stdlib:unicode_usage#unicode_file_names">Unicode - Filenames</seeguide> in the STDLIB User's Guide. Notice that + Filenames</seeguide> in the STDLIB User's Guide. Notice that this value also applies to command-line parameters and environment variables (see section <seeguide marker="stdlib:unicode_usage#unicode_in_environment_and_parameters"> Unicode in Environment and Parameters</seeguide> in the STDLIB - User's Guide).</p> + User's Guide).</p> </item> <tag><c><![CDATA[+fna[{w|i|e}]]]></c></tag> <item> @@ -748,12 +778,12 @@ selected, then <c>w</c>, <c>i</c>, or <c>e</c> have no effect.</p> <p>For more information about Unicode filenames, see section <seeguide marker="stdlib:unicode_usage#unicode_file_names">Unicode - Filenames</seeguide> in the STDLIB User's Guide. Notice that + Filenames</seeguide> in the STDLIB User's Guide. Notice that this value also applies to command-line parameters and environment variables (see section <seeguide marker="stdlib:unicode_usage#unicode_in_environment_and_parameters"> Unicode in Environment and Parameters</seeguide> in the STDLIB - User's Guide).</p> + User's Guide).</p> </item> <tag><c><![CDATA[+hms Size]]></c></tag> <item> @@ -1115,7 +1145,7 @@ </item> <tag><c>ts</c></tag> <item><c>thread_spread</c> - Thread refers to hardware threads - (such as Intel's hyper-threads). Schedulers with low scheduler + (such as Intel's hyper-threads). Schedulers with low scheduler identifiers, are bound to the first hardware thread of each core, then schedulers with higher scheduler identifiers are bound to the second hardware thread of each core,and so on. @@ -1578,8 +1608,7 @@ <tag><marker id="+zdbbl"/><c>+zdbbl size</c></tag> <item> <p>Sets the distribution buffer busy limit - (<seeerl marker="erlang#system_info_dist_buf_busy_limit"> - <c>dist_buf_busy_limit</c></seeerl>) + (<seeerl marker="erlang#system_info_dist_buf_busy_limit"><c>dist_buf_busy_limit</c></seeerl>) in kilobytes. Valid range is 1-2097151. Defaults to 1024.</p> <p>A larger buffer limit allows processes to buffer more outgoing messages over the distribution. When the @@ -1592,8 +1621,7 @@ <tag><marker id="+zdntgc"/><c>+zdntgc time</c></tag> <item> <p>Sets the delayed node table garbage collection time - (<seeerl marker="erlang#system_info_delayed_node_table_gc"> - <c>delayed_node_table_gc</c></seeerl>) + (<seeerl marker="erlang#system_info_delayed_node_table_gc"><c>delayed_node_table_gc</c></seeerl>) in seconds. Valid values are either <c>infinity</c> or an integer in the range 0-100000000. Defaults to 60.</p> <p>Node table entries that are not referred linger @@ -1733,7 +1761,7 @@ <tag>The <c>.erlang</c> startup file</tag> <item> <p>When Erlang/OTP is started, the system searches for a file named - <c>.erlang</c> in the user's home directory.</p> + <c>.erlang</c> in the user's home directory.</p> <p>If an <c>.erlang</c> file is found, it is assumed to contain valid Erlang expressions. These expressions are evaluated as if they were input to the shell.</p> diff --git a/erts/doc/src/erl_dist_protocol.xml b/erts/doc/src/erl_dist_protocol.xml index 743175592e..8cb8e09615 100644 --- a/erts/doc/src/erl_dist_protocol.xml +++ b/erts/doc/src/erl_dist_protocol.xml @@ -1038,14 +1038,12 @@ DiB == gen_digest(ChA, ICA)? <item> <p>The node implements atom cache in distribution header.</p> </item> - <marker id="DFLAG_SMALL_ATOM_TAGS"/> - <tag><c>-define(DFLAG_SMALL_ATOM_TAGS, 16#4000).</c></tag> + <tag><marker id="DFLAG_SMALL_ATOM_TAGS"/><c>-define(DFLAG_SMALL_ATOM_TAGS, 16#4000).</c></tag> <item> <p>The node understands the <seeguide marker="erl_ext_dist#SMALL_ATOM_EXT"> <c>SMALL_ATOM_EXT</c></seeguide> tag.</p> </item> - <marker id="DFLAG_UTF8_ATOMS"/> - <tag><c>-define(DFLAG_UTF8_ATOMS, 16#10000).</c></tag> + <tag><marker id="DFLAG_UTF8_ATOMS"/><c>-define(DFLAG_UTF8_ATOMS, 16#10000).</c></tag> <item> <p>The node understands UTF-8 atoms encoded with <seeguide marker="erl_ext_dist#ATOM_UTF8_EXT"> @@ -1058,8 +1056,7 @@ DiB == gen_digest(ChA, ICA)? <p>The node understands the map tag <seeguide marker="erl_ext_dist#MAP_EXT"><c>MAP_EXT</c></seeguide>.</p> </item> - <marker id="DFLAG_BIG_CREATION"/> - <tag><c>-define(DFLAG_BIG_CREATION, 16#40000).</c></tag> + <tag><marker id="DFLAG_BIG_CREATION"/><c>-define(DFLAG_BIG_CREATION, 16#40000).</c></tag> <item> <p>The node understands big node creation tags <seeguide marker="erl_ext_dist#NEW_PID_EXT"><c>NEW_PID_EXT</c></seeguide>, @@ -1089,14 +1086,12 @@ DiB == gen_digest(ChA, ICA)? <seeguide marker="#control_message">control message</seeguide>s instead of the non-PAYLOAD variants.</p> </item> - <marker id="DFLAG_FRAGMENTS"/> - <tag><c>-define(DFLAG_FRAGMENTS, 16#800000).</c></tag> + <tag><marker id="DFLAG_FRAGMENTS"/><c>-define(DFLAG_FRAGMENTS, 16#800000).</c></tag> <item> <p>Use <seeguide marker="erl_ext_dist#fragments">fragmented</seeguide> distribution messages to send large messages.</p> </item> - <marker id="DFLAG_HANDSHAKE_23"/> - <tag><c>-define(DFLAG_HANDSHAKE_23, 16#1000000).</c></tag> + <tag><marker id="DFLAG_HANDSHAKE_23"/><c>-define(DFLAG_HANDSHAKE_23, 16#1000000).</c></tag> <item> <p>The node supports the new connection setup handshake (version 6) introduced in OTP 23.</p> diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 14f9b5a79f..67d7a07bae 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -375,7 +375,7 @@ Z = erlang:adler32_combine(X,Y,iolist_size(Data2)).</code> </func> <func> - <name name="atom_to_binary" arity="1" since="OTP @OTP-15995@"/> + <name name="atom_to_binary" arity="1" since="OTP 23.0"/> <fsummary>Return the binary representation of an atom.</fsummary> <desc> <p> @@ -457,7 +457,7 @@ Z = erlang:adler32_combine(X,Y,iolist_size(Data2)).</code> </func> <func> - <name name="binary_to_atom" arity="1" since="OTP @OTP-15995@"/> + <name name="binary_to_atom" arity="1" since="OTP 23.0"/> <fsummary>Convert from text representation to an atom.</fsummary> <desc> <p> @@ -493,7 +493,7 @@ Z = erlang:adler32_combine(X,Y,iolist_size(Data2)).</code> </func> <func> - <name name="binary_to_existing_atom" arity="1" since="OTP @OTP-15995@"/> + <name name="binary_to_existing_atom" arity="1" since="OTP 23.0"/> <fsummary>Convert from text representation to an atom.</fsummary> <desc> <p> @@ -6315,7 +6315,7 @@ true</pre> </func> <func> - <name name="spawn_monitor" arity="2" since="OTP @OTP-15251@"/> + <name name="spawn_monitor" arity="2" since="OTP 23.0"/> <fsummary>Create and monitor a new process with a fun as entry point. </fsummary> <desc> @@ -6346,7 +6346,7 @@ true</pre> </func> <func> - <name name="spawn_monitor" arity="4" since="OTP @OTP-15251@"/> + <name name="spawn_monitor" arity="4" since="OTP 23.0"/> <fsummary>Create and monitor a new process with a function as entry point. </fsummary> <desc> @@ -6564,7 +6564,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="1" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="1" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6576,7 +6576,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="2" clause_i="1" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="2" clause_i="1" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6588,7 +6588,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="2" clause_i="2" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="2" clause_i="2" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6600,7 +6600,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="3" clause_i="1" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="3" clause_i="1" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6620,7 +6620,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="3" clause_i="2" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="3" clause_i="2" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6632,7 +6632,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="4" clause_i="1" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="4" clause_i="1" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6644,7 +6644,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="4" clause_i="2" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="4" clause_i="2" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6656,7 +6656,7 @@ true</pre> </func> <func> - <name name="spawn_request" arity="5" since="OTP @OTP-15251@"/> + <name name="spawn_request" arity="5" since="OTP 23.0"/> <fsummary>Asynchronously send a request to spawn a new process.</fsummary> <desc> <p> @@ -6893,7 +6893,7 @@ true</pre> </func> <func> - <name name="spawn_request_abandon" arity="1" since="OTP @OTP-15251@"/> + <name name="spawn_request_abandon" arity="1" since="OTP 23.0"/> <fsummary>Abandon a previously issued spawn request.</fsummary> <desc> <p> @@ -10217,7 +10217,7 @@ hello </func> <func> - <name name="term_to_iovec" arity="1" since="OTP @OTP-15618@"/> + <name name="term_to_iovec" arity="1" since="OTP 23.0"/> <fsummary>Encode a term to an Erlang external term format as iovec(). </fsummary> <desc> @@ -10242,7 +10242,7 @@ hello </func> <func> - <name name="term_to_iovec" arity="2" since="OTP @OTP-15618@"/> + <name name="term_to_iovec" arity="2" since="OTP 23.0"/> <fsummary>Encode a term to en Erlang external term format as iovec(). </fsummary> <desc> diff --git a/erts/doc/src/erlc_cmd.xml b/erts/doc/src/erlc_cmd.xml index f9ea8e90f2..25c6ebf23d 100644 --- a/erts/doc/src/erlc_cmd.xml +++ b/erts/doc/src/erlc_cmd.xml @@ -106,7 +106,7 @@ must be quoted. Terms containing spaces must be quoted on all platforms.</p> </item> - <tag><c>-W<Error></c></tag> + <tag><c>-WError</c></tag> <item> <p>Makes all warnings into errors.</p> </item> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index 9c8890fc67..cdee14a460 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -31,6 +31,716 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 11.0.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The functionality utilized by BIFs for temporary + disabling of garbage collection while yielding could + cause system task queues to become inconsistent on a + process executing such a BIF. Process system tasks are + for example utilized when purging code, garbage + collecting literal data, and when issuing an ordinary + garbage collection from another process.</p> + <p> + The bug does not trigger frequently. Multiple code purges + in direct sequence makes it more likely that this bug is + triggered. In the cases observed, this has resulted in a + hanging code purge operation.</p> + <p> + Own Id: OTP-16639 Aux Id: ERL-1236 </p> + </item> + <item> + <p> + SCTP and UDP recv/2,3 hangs indefinitely if socket is + closed while recv is called (socket in passive mode).</p> + <p> + Own Id: OTP-16654 Aux Id: ERL-1242 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 11.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>BIFs now behave like ordinary functions with regard to + tracing, allowing <c>call_count</c> tracing and fixing a + few bugs where return trace messages were lost when BIFs + tail-called themselves or other functions + ("trapping").</p> + <p> + Own Id: OTP-14734 Aux Id: ERL-496 </p> + </item> + <item> + <p> + Fix various compiler warnings on 64-bit Windows.</p> + <p> + Own Id: OTP-15800</p> + </item> + <item> + <p><c>erlang:fun_info(fun foo/1, name/1)</c> used to + return a function name based on the name of the function + that <c>fun foo/1</c> was used in. The name returned is + now <c>-fun.foo/1-</c>.</p> + <p> + Own Id: OTP-15837</p> + </item> + <item> + <p><c>file:allocate/3</c> will now update the file size + on all platforms.</p> + <p> + Own Id: OTP-16155 Aux Id: PR-2408 </p> + </item> + <item> + <p> + <c>erlang:decode_packet</c> with type set to <c>httph</c> + no longer accepts http headers that have whitespaces in + between the header name and the colon. That is:</p> + <p> + <c> Content-Type : text/html </c></p> + <p> + is no longer allowed. This has been changed to conform + with RFC 7230 and thus protect against http desync + attacks.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16168 Aux Id: ERL-1053 </p> + </item> + <item> + <p> + Fix the quoting rules in <c>erl -args_file</c>, + <c>ERL_FLAGS</c>, <c>ERL_AFLAGS</c> and <c>ERL_ZFLAGS</c> + to work as unix sh quoting.</p> + <p> + This bug fix can make previous configuration options to + <c>erl</c> passed through <c>ERL_FLAGS</c>, + <c>ERL_AFLAGS</c>, <c>ERL_ZFLAGS</c> or <c>-args_file</c> + not be interpreted in the same way as before the fix.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16244 Aux Id: ERL-1051 </p> + </item> + <item> + <p> + Fix the Erlang distribution to handle the scenario when a + node connects that can handle message fragmentation but + can not handle the atom cache. This bug only affects + users that have implemented a custom distribution + carrier. It has been present since OTP-21.</p> + <p> + The <c>DFLAG_FRAGMENT</c> distribution flag was added to + the set of flags that can be rejected by a distribution + implementation.</p> + <p> + Own Id: OTP-16284</p> + </item> + <item> + <p> + Compiling a match specification with excessive nesting + caused the runtime system to crash due to scheduler stack + exhaustion. Instead of crashing the runtime system, + effected functions will now raise a <c>system_limit</c> + error exception in this situation.</p> + <p> + Own Id: OTP-16431 Aux Id: ERL-592 </p> + </item> + <item> + <p>Fixed a bug that prevented Erlang from being started + on Windows if it were installed on certain paths.</p> + <p> + Own Id: OTP-16478 Aux Id: ERL-1115 </p> + </item> + <item> + <p>Fix bug on Windows causing bad performance when + standard input is closed, especially if the VM is only + assigned one CPU core. Could be provoked for example by + starting erl or escript via function <c>os:cmd/1</c>. + Could be avoided with command line option + <c>-noinput</c>.</p> <p>The bad performance was caused by + an io thread spinning indefinitely.</p> + <p> + Own Id: OTP-16521 Aux Id: ERL-716 </p> + </item> + <item> + <p>Fixed a bug on Unix platforms that would cause + <c>file:read_file_info/1</c> to return incorrect results + if the emulator's effective user or group id differed + from its actual ones.</p> + <p> + Own Id: OTP-16571</p> + </item> + <item> + <p> + socket: Compile problems on Android when PACKET_FASTROUTE + and PACKET_USER are both defined and has the same value. + Use of PACKET_FASTROUTE has been removed as it may be + unused and also only for none user-land.</p> + <p> + Own Id: OTP-16576 Aux Id: ERL-1208 </p> + </item> + <item> + <p> + Fixed bug in erl_crash.dump generation that could cause a + SEGV core dump if a recently cancelled timer was found.</p> + <p> + Own Id: OTP-16596 Aux Id: ERL-1105, PR-2606 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved concurrency of <c>erlang:load_nif/2</c> as it + does no longer block other schedulers from executing + during initial load of a NIF library.</p> + <p> + Own Id: OTP-10278</p> + </item> + <item> + <p>EEP-52 has been implemented.</p> + <p>In binary matching, the size of the segment to be + matched is now allowed to be a guard expression, and + similarly in map matching the keys can now be guard + expressions. See the Erlang Reference Manual and + Programming Examples for more details.</p> + <p>Language compilers or code generators that generate + Core Erlang code may need to be updated to be compatible + with the compiler in OTP 23. For more details, see the + section Backwards Compatibility in <url + href="http://erlang.org/eeps/eep-0052.html">EEP + 52</url>.</p> + <p> + Own Id: OTP-14708</p> + </item> + <item> + <p>Internally in BEAM, handling of continuation pointers + has been simplified. This change is not user-visible, + except when examing a process stack in the crashdump + viewer. The continuation pointer for a function will now + be stored below the y(0) for that function.</p> + <p> + Own Id: OTP-15077</p> + </item> + <item> + <p><c>seq_trace</c> tokens are now propagated to spawned + processes.</p> + <p> + Own Id: OTP-15232 Aux Id: ERL-700 </p> + </item> + <item> + <p>Improvements of distributed spawn operations. These + include both scalability and performance improvements as + well as new functionality.</p> <p>New functionality:</p> + <list> <item><p>A distributed <seemfa + marker="erts:erlang#spawn_monitor/4"><c>spawn_monitor()</c></seemfa> + BIF.</p></item> <item><p>Support for <c>monitor</c> + option in the distributed <seemfa + marker="erts:erlang#spawn_opt/5"><c>spawn_opt()</c></seemfa> + BIF.</p></item> <item><p>New <seemfa + marker="erts:erlang#spawn_request/5"><c>spawn_request()</c></seemfa> + BIFs for asynchronous spawn of processes. + <c>spawn_request()</c> supports all options that + <c>spawn_opt()</c> support plus a few more.</p></item> + </list> + <p> + Own Id: OTP-15251</p> + </item> + <item> + <p> + Make <c>ets:insert/2</c> and <c>ets:insert_new/2</c> + yield scheduler execution on long lists of records to + insert.</p> + <p> + Own Id: OTP-15517 Aux Id: ERL-560 </p> + </item> + <item> + <p> + Increased size of node incarnation numbers (aka + "creation"), from 2 bits to 32 bits. This will reduce the + risk of pids/ports/refs, from different node incarnation + with the same name, being mixed up.</p> + <p> + Own Id: OTP-15603</p> + </item> + <item> + <p>The runtime system can now encode Erlang terms to the + Erlang external term format as I/O vectors. The main + benefit of this is that reference counted binaries can be + referred to directly instead of copied into a new + binary.</p> <p>The default Erlang distribution over TCP + will always utilize this. Alternate distribution + implementations utilizing a port as distribution + controller will utilize this if the driver implements the + <seecref + marker="erts:driver_entry#outputv"><c>outputv</c></seecref> + callback. Alternate Erlang distribution implementations + utilizing a process as distribution controller will + utilize this if I/O vectors are utilized by the + functionality that processes the data returned from + <seemfa + marker="erts:erlang#dist_ctrl_get_data/1"><c>erlang:dist_ctrl_get_data()</c></seemfa>.</p> + <p>The return type for data returned by <seemfa + marker="erts:erlang#dist_ctrl_get_data/1"><c>erlang:dist_ctrl_get_data()</c></seemfa> + has been changed from <c>iodata()</c> to <c>iovec()</c>. + Note that <c>iovec()</c> data is valid <c>iodata()</c> so + old implementations using + <c>erlang:dist_ctrl_get_data()</c> do not need to be + changed, but may benefit from being changed depending on + usage scenario.</p> <p>The new BIFs <seemfa + marker="erts:erlang#term_to_iovec/1"><c>term_to_iovec/1</c></seemfa> + and <seemfa + marker="erts:erlang#term_to_iovec/2"><c>term_to_iovec/2</c></seemfa> + have been introduced. These work exactly as + <c>term_to_binary()</c> with the corresponding arity + except the return type.</p> + <p> + Own Id: OTP-15618</p> + </item> + <item> + <p>Improved ETS scalability of concurrent calls that + change the size of a table, like <c>ets:insert/2</c> and + <c>ets:delete/2</c>.</p> <p>This performance feature was + implemented for <c>ordered_set</c> in OTP 22.0 and does + now apply for all ETS table types.</p> <p>The improved + scalability may come at the cost of longer latency of + <c>ets:info(T,size)</c> and <c>ets:info(T,memory)</c>. A + new table option <c>decentralized_counters</c> has + therefore been added. It is default <c>true</c> for + <c>ordered_set</c> with <c>write_concurrency</c> enabled + and default <c>false</c> for all other table types.</p> + <p> + Own Id: OTP-15744 Aux Id: OTP-15623, PR-2229 </p> + </item> + <item> + <p>Directories can now be opened by <c>file:open/2</c> + when passing the <c>directory</c> option.</p> + <p> + Own Id: OTP-15835 Aux Id: PR-2212 </p> + </item> + <item> + <p> + Add Hygon Dhyana as known processor to enable support for + atomic operations.</p> + <p> + Own Id: OTP-15840</p> + </item> + <item> + <p> + Make <c>erlang:phash2</c> functions consume reductions + proportional to the size of the input term and yield + scheduler when reductions are depleted.</p> + <p> + Own Id: OTP-15842 Aux Id: PR-2182 </p> + </item> + <item> + <p> + Fix various build issues when compiling Erlang/OTP to the + IBM AIX platform.</p> + <p> + Own Id: OTP-15866 Aux Id: PR-2110 </p> + </item> + <item> + <p> + Add configure options <c>--enable-pie</c> and + <c>--disable-pie</c> to control the build of position + independent executables.</p> + <p> + Own Id: OTP-15868</p> + </item> + <item> + <p><c>file:read_file_info/2</c> can now be used on opened + files and directories.</p> + <p> + Own Id: OTP-15956 Aux Id: PR-2231 </p> + </item> + <item> + <p> + Add arity-1 versions of <c>atom_to_binary</c>, + <c>binary_to_atom</c> and <c>binary_to_existing_atom</c>, + all with <c>utf8</c> as default encoding.</p> + <p> + Own Id: OTP-15995 Aux Id: PR-2358 </p> + </item> + <item> + <p> + Optimized the erts internal hash table implementation for + faster lookups. The internal hash is used for things + like; the process registry, executing erlang:apply/2, + executing M:func(test), and more.</p> + <p> + Own Id: OTP-16014 Aux Id: PR-2345 </p> + </item> + <item> + <p>CPU quotas are now taken into account when deciding + the default number of online schedulers, improving + performance in container environments where quotas are + applied, such as <c>docker</c> with the <c>--cpus</c> + flag.</p> + <p> + Own Id: OTP-16105 Aux Id: ERL-927 </p> + </item> + <item> + <p> + The <c>-config</c> option to <c>erl</c> now can take + multiple config files without repeating the + <c>-config</c> option. Example:</p> + <p> + erl -config sys local</p> + <p> + Own Id: OTP-16148 Aux Id: PR-2373 </p> + </item> + <item> + <p> + Removed the <c>scheduler_poll</c> and <c>async I/O</c> + dtrace and LTTng trace probes.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16215</p> + </item> + <item> + <p> + Optimized <c>persistent_term:put/2</c> and <c>erase/1</c> + to consume less CPU in many cases.</p> + <p> + Own Id: OTP-16237 Aux Id: PR-2389 </p> + </item> + <item> + <p>The possibility to run Erlang distribution without + relying on EPMD has been extended. To achieve this a + couple of new options to the inet distribution has been + added.</p> <taglist> <tag>-dist_listen false</tag> + <item>Setup the distribution channel, but do not listen + for incoming connection. This is useful when you want to + use the current node to interact with another node on the + same machine without it joining the entire + cluster.</item> <tag>-erl_epmd_port Port</tag> + <item>Configure a default port that the built-in EPMD + client should return. This allows the local node to know + the port to connect to for any other node in the + cluster.</item> </taglist> <p>The <c>erl_epmd</c> + callback API has also been extended to allow returning + <c>-1</c> as the creation which means that a random + creation will be created by the node.</p> + <p>In addition a new callback function called + <c>listen_port_please</c> has been added that allows the + callback to return which listen port the distribution + should use. This can be used instead of + <c>inet_dist_listen_min/max</c> if the listen port is to + be fetched from an external service.</p> + <p> + Own Id: OTP-16250</p> + </item> + <item> + <p> + On systems without <c>closefrom()</c>, such as Linux, + iterating over all possible file descriptors and calling + <c>close()</c> for each is inefficient. This is markedly + so when the maximum number of file descriptors has been + tuned to a large number.</p> + <p> + Instead, in erl_child_setup, walk the open descriptors + under <c>/dev/fd</c> and close only those which are open.</p> + <p> + This optimization affects the CPU usage of starting a new + Erlang instance.</p> + <p> + Own Id: OTP-16270</p> + </item> + <item> + <p> + Optimized <c>maps:merge/2</c> for trivial cases of an + empty map(s) or same map.</p> + <p> + Own Id: OTP-16283 Aux Id: PR-2441 </p> + </item> + <item> + <p> + The new experimental <c>socket</c> module has been moved + to the Kernel application.</p> + <p> + Own Id: OTP-16312</p> + </item> + <item> + <p>Improved the presentation of allocations and carriers + in the <c>instrument</c> module.</p> + <p> + Own Id: OTP-16327</p> + </item> + <item> + <p> + As announced in OTP 22.0, the previously existing limited + support for VxWorks has now been removed.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16329 Aux Id: OTP-15621 </p> + </item> + <item> + <p> The return value when using the <c>httph</c> and + <c>httph_bin</c> option to <c>erlang:decode_packet/3</c> + and <c>inet:setopts/2</c> has been changed to also + include the original header unmodified. See <seemfa + marker="erlang#decode_packet/3"><c>erlang:decode_packet/3</c></seemfa>. + Example: </p> <pre> > + erlang:decode_packet(httph_bin,<<"HELLO: + hi\r\n\r\n">>,[]). + {ok,{http_header,0,<<"Hello">>,<<"HELLO">>,<<"hi">>},<<"\r\n">>} + </pre> + <p> + Own Id: OTP-16347 Aux Id: PR-2466 </p> + </item> + <item> + <p> + Ensure <c>net_kernel:monitor_nodes/1</c> sends + <c>nodedown</c> messages of a failed connection before + <c>nodeup</c> messages of a reestablished connection + toward the same node.</p> + <p> + Own Id: OTP-16362</p> + </item> + <item> + <p> + Update of <seeerl + marker="kernel:seq_trace#whatis">sequential + tracing</seeerl> to also support other information + transfers than message passing.</p> + <p> + Own Id: OTP-16370 Aux Id: OTP-15251, OTP-15232 </p> + </item> + <item> + <p> + socket: It is now possible to create a socket from an + already existing file descriptor.</p> + <p> + Own Id: OTP-16398 Aux Id: ERL-1154 </p> + </item> + <item> + <p> + socket: The socket:supports/1 function now also report if + netns is supported or not.</p> + <p> + Own Id: OTP-16432</p> + </item> + <item> + <p><c>=:=</c> has been optimized to return <c>false</c> + immediately when comparing two maps of different + sizes.</p> + <p> + Own Id: OTP-16454</p> + </item> + <item> + <p> + Changed the behaviour of passing the <c>erl</c> command + line argument <seecom + marker="erts:erl#async_thread_pool_size"><c>+A + 0</c></seecom> to silently imply <c>+A 1</c>. That is, it + will no longer be possible to completely disable the + async thread pool. Disabling of the async thread pool has + since OTP 21 had no benefits; only lots of drawbacks.</p> + <p> + Own Id: OTP-16482</p> + </item> + <item> + <p>The deprecated <c>erlang:get_stacktrace/0</c> BIF now + returns an empty list instead of a stacktrace. To + retrieve the stacktrace, use the extended try/catch + syntax that was introduced in OTP 21. + <c>erlang:get_stacktrace/0</c> is scheduled for removal + in OTP 24.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16484</p> + </item> + <item> + <p> + <seemfa + marker="erts:init#restart/1"><c>init:restart/1</c></seemfa> + has been introduced. <c>init:restart/1</c> can be + utilized for changing the code loading mode during a + restart.</p> + <p> + Own Id: OTP-16492 Aux Id: PR-2461 </p> + </item> + <item> + <p> + Improve configure for the net nif, which should increase + portability.</p> + <p> + Own Id: OTP-16530 Aux Id: OTP-16464 </p> + </item> + <item> + <p> + socket: Socket counters and socket global counters are + now represented as maps (instead of property lists).</p> + <p> + Own Id: OTP-16535</p> + </item> + <item> + <p>Reduced the resource usage of <c>erlc</c> in parallel + builds (e.g. <c>make -j128</c>).</p> + <p> + Own Id: OTP-16543 Aux Id: ERL-1186 </p> + </item> + <item> + <p> + The experimental socket module has gotten restrictions + removed so now the 'seqpacket' socket type should work + for any communication domain (protocol family) where the + OS supports it, typically the Unix Domain.</p> + <p> + Own Id: OTP-16550 Aux Id: ERIERL-476 </p> + </item> + <item> + <p>Updated the internal <c>pcre</c> library to + <c>8.44</c>.</p> + <p> + Own Id: OTP-16557</p> + </item> + <item> + <p>There is now cost in terms of reductions when copying + binary data using the binary syntax.</p> + <p> + Own Id: OTP-16601 Aux Id: OTP-16577 </p> + </item> + <item> + <p> + The executable <c>erl_call</c> is now part of the + <c>erts</c> distribution in addition to + <c>erl_interface</c>.</p> + <p> + Own Id: OTP-16602</p> + </item> + <item> + <p> + Fix a buffer overflow bug that caused EPMD to consume + 100% CPU after many nodes had been connected on the same + time on NetBSD.</p> + <p> + Own Id: OTP-16615</p> + </item> + <item> + <p> + <c>erl -remsh</c> now uses the dynamic node names feature + by default. See the <seecom + marker="erts:erl">erl</seecom> documentation for details.</p> + <p> + Own Id: OTP-16616</p> + </item> + <item> + <p> + socket: By default the socket options rcvtimeo and + sndtimeo are now disabled. To enable these, OTP now has + to be built with the configure option + --enable-esock-rcvsndtimeo</p> + <p> + Own Id: OTP-16620</p> + </item> + <item> + <p> + The environment variable $HOME does no longer have to be + set before Erlang can be started.</p> + <p> + Own Id: OTP-16635 Aux Id: ERL-476 PR-2390 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.7.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fixed bug in erl_crash.dump generation that could cause a + SEGV core dump if a recently cancelled timer was found.</p> + <p> + Own Id: OTP-16596 Aux Id: ERL-1105, PR-2606 </p> + </item> + <item> + <p> + The functionality utilized by BIFs for temporary + disabling of garbage collection while yielding could + cause system task queues to become inconsistent on a + process executing such a BIF. Process system tasks are + for example utilized when purging code, garbage + collecting literal data, and when issuing an ordinary + garbage collection from another process.</p> + <p> + The bug does not trigger frequently. Multiple code purges + in direct sequence makes it more likely that this bug is + triggered. In the cases observed, this has resulted in a + hanging code purge operation.</p> + <p> + Own Id: OTP-16639 Aux Id: ERL-1236 </p> + </item> + <item> + <p> + SCTP and UDP recv/2,3 hangs indefinitely if socket is + closed while recv is called (socket in passive mode).</p> + <p> + Own Id: OTP-16654 Aux Id: ERL-1242 </p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A literal area could prematurely be released before all + uses of it had been removed. This occurred either when a + terminating process had a complex exit reason referring + to a literal that concurrently was removed, or when a + terminating process continued executing a dirty NIF + accessing a literal (via the heap) that concurrently was + removed.</p> + <p> + Own Id: OTP-16640 Aux Id: OTP-16193 </p> + </item> + <item> + <p> + The VM could potentially crash when checking process code + of a process that terminated while executing a dirty NIF. + The checking of process code is part of a code purge + operation.</p> + <p> + Own Id: OTP-16641</p> + </item> + <item> + <p> + System tasks of <c>low</c> priority were not interleaved + with <c>normal</c> priority system tasks as they should. + This could potentially delay garbage collection of + another process longer than intended if the garbage + collection was requested from a <c>low</c> priority + process.</p> + <p> + Own Id: OTP-16642</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.7.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -167,7 +877,7 @@ <list> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, not all + For <seeerl marker="kernel:socket#">socket</seeerl>, not all send and receive flags are supported on all platforms. In order to (at least) simplify testing, the socket:supports/0,1,2,3 functions has been extended with @@ -386,7 +1096,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, invalid + For <seeerl marker="kernel:socket#">socket</seeerl>, invalid encoding of send and receive flags caused badarg and send failure.</p> <p> @@ -401,7 +1111,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, unable to + For <seeerl marker="kernel:socket#">socket</seeerl>, unable to properly decode the timestamp control message header on FreeBSD. We incorrectly used the SO_TIMESTAMP flag for the timestamp control message header type. It should have @@ -412,7 +1122,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, when + For <seeerl marker="kernel:socket#">socket</seeerl>, when setting the ip option 'recvtos' to true, thereby indicating that we want to receive the TOS control message header, we don't actually get TOS but RECVTOS on @@ -429,7 +1139,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, wrong + For <seeerl marker="kernel:socket#">socket</seeerl>, wrong type for protocol caused segmentation fault if protocol was provided as {raw, integer()}.</p> <p> @@ -437,7 +1147,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, when + For <seeerl marker="kernel:socket#">socket</seeerl>, when setting the ip option 'recvttl' to true, thereby indicating that we want to receive the TTL control message header, we don't actually get TTL but RECVTTL on @@ -448,7 +1158,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, IPv6 + For <seeerl marker="kernel:socket#">socket</seeerl>, IPv6 control message headers was incorrectly decoded with level ip instead of ipv6.</p> <p> @@ -494,7 +1204,7 @@ </item> <item> <p> - The <seeerl marker="socket#">socket</seeerl> socket + The <seeerl marker="kernel:socket#">socket</seeerl> socket option 'peek_off' has been disabled. If peek_off was set and then socket:recv/3 was called with the peek flag, the call could hang.</p> @@ -544,7 +1254,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, the + For <seeerl marker="kernel:socket#">socket</seeerl>, the timestamp creation used when printing warning messages and debug printouts did not work. The used buffer was too small.</p> @@ -590,7 +1300,7 @@ <list> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, not all + For <seeerl marker="kernel:socket#">socket</seeerl>, not all send and receive flags are supported on all platforms. In order to (at least) simplify testing, the socket:supports/0,1,2,3 functions has been extended with @@ -601,7 +1311,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, add + For <seeerl marker="kernel:socket#">socket</seeerl>, add support for IPv6 socket options tclass and recvtclass. Both has been added, but the use of them are platform dependent. Call socket:supports(options, ipv6, Opt) to be @@ -612,7 +1322,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, the TCP + For <seeerl marker="kernel:socket#">socket</seeerl>, the TCP socket option cork was not supported even though the supports function reported it as such.</p> <p> @@ -645,7 +1355,7 @@ </item> <item> <p> - For <seeerl marker="socket#">socket</seeerl>, added + For <seeerl marker="kernel:socket#">socket</seeerl>, added support for the socket option extended_err. Andreas Schultz.</p> <p> @@ -1751,6 +2461,91 @@ </section> +<section><title>Erts 10.3.5.12</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The functionality utilized by BIFs for temporary + disabling of garbage collection while yielding could + cause system task queues to become inconsistent on a + process executing such a BIF. Process system tasks are + for example utilized when purging code, garbage + collecting literal data, and when issuing an ordinary + garbage collection from another process.</p> + <p> + The bug does not trigger frequently. Multiple code purges + in direct sequence makes it more likely that this bug is + triggered. In the cases observed, this has resulted in a + hanging code purge operation.</p> + <p> + Own Id: OTP-16639 Aux Id: ERL-1236 </p> + </item> + <item> + <p> + A literal area could prematurely be released before all + uses of it had been removed. This occurred either when a + terminating process had a complex exit reason referring + to a literal that concurrently was removed, or when a + terminating process continued executing a dirty NIF + accessing a literal (via the heap) that concurrently was + removed.</p> + <p> + Own Id: OTP-16640 Aux Id: OTP-16193 </p> + </item> + <item> + <p> + The VM could potentially crash when checking process code + of a process that terminated while executing a dirty NIF. + The checking of process code is part of a code purge + operation.</p> + <p> + Own Id: OTP-16641</p> + </item> + <item> + <p> + System tasks of <c>low</c> priority were not interleaved + with <c>normal</c> priority system tasks as they should. + This could potentially delay garbage collection of + another process longer than intended if the garbage + collection was requested from a <c>low</c> priority + process.</p> + <p> + Own Id: OTP-16642</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 10.3.5.11</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + <seemfa marker="stdlib:re#run/3">re:run(Subject, RE, + [unicode])</seemfa> returned <c>nomatch</c> instead of + failing with a <c>badarg</c> error exception when + <c>Subject</c> contained illegal utf8 and <c>RE</c> was + passed as a binary. This has been corrected along with + corrections of reduction counting in <c>re:run()</c> + error cases.</p> + <p> + Own Id: OTP-16553</p> + </item> + <item> + <p>Fixed a bug that could cause the emulator to crash + when purging modules or persistent terms.</p> + <p> + Own Id: OTP-16555 Aux Id: ERL-1188 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 10.3.5.10</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/doc/src/part.xml.src b/erts/doc/src/part.xml index 9b20beffad..5c5e1bedb4 100644 --- a/erts/doc/src/part.xml.src +++ b/erts/doc/src/part.xml @@ -42,7 +42,6 @@ <xi:include href="tty.xml"/> <xi:include href="driver.xml"/> <xi:include href="inet_cfg.xml"/> - %ESOCK_USE_SOCKET_USAGE_XML% <xi:include href="erl_ext_dist.xml"/> <xi:include href="erl_dist_protocol.xml"/> </part> diff --git a/erts/doc/src/ref_man.xml.src b/erts/doc/src/ref_man.xml index 5b327165d4..f78d6549a8 100644 --- a/erts/doc/src/ref_man.xml.src +++ b/erts/doc/src/ref_man.xml @@ -48,7 +48,6 @@ <xi:include href="init.xml"/> <xi:include href="persistent_term.xml"/> <xi:include href="run_erl_cmd.xml"/> - %ESOCK_USE_SOCKET_XML% <xi:include href="start_cmd.xml"/> <xi:include href="start_erl_cmd.xml"/> <xi:include href="werl_cmd.xml"/> diff --git a/erts/doc/src/specs.xml.src b/erts/doc/src/specs.xml index 54224c15f5..0b943e6295 100644 --- a/erts/doc/src/specs.xml.src +++ b/erts/doc/src/specs.xml @@ -5,7 +5,6 @@ <xi:include href="../specs/specs_erl_tracer.xml"/> <xi:include href="../specs/specs_init.xml"/> <xi:include href="../specs/specs_persistent_term.xml"/> - %ESOCK_USE_SOCKET_SPECS_XML% <xi:include href="../specs/specs_zlib.xml"/> <xi:include href="../specs/specs_atomics.xml"/> <xi:include href="../specs/specs_counters.xml"/> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 275b489bb0..6760207fa8 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -662,7 +662,7 @@ GENERATE += $(TARGET)/erl_db_insert_list.ycf.h ifeq ($(USE_ESOCK), yes) ESOCK_PRELOAD_BEAM = \ $(ERL_TOP)/erts/preloaded/ebin/socket_registry.beam \ - $(ERL_TOP)/erts/preloaded/ebin/socket.beam \ + $(ERL_TOP)/erts/preloaded/ebin/prim_socket.beam \ $(ERL_TOP)/erts/preloaded/ebin/prim_net.beam else ESOCK_PRELOAD_BEAM = @@ -875,7 +875,7 @@ ifeq ($(USE_ESOCK), yes) # WE ARE USING ESOCK ESOCK_NIF_OBJS = \ - $(OBJDIR)/socket_nif.o \ + $(OBJDIR)/prim_socket_nif.o \ $(OBJDIR)/prim_net_nif.o ifneq ($(TARGET), win32) @@ -1222,8 +1222,7 @@ ifeq ($(TARGET), win32) # These are *currently* only needed for non-win32, # since the nif-functions for socket and net are basically # stubbed with badarg in the win32 case. -NIF_SOCKET_UTILS_SRC=$(filter-out nifs/common/socket_nif.c, $(wildcard nifs/common/socket_*.c)) -NIF_COMMON_SRC=$(filter-out $(NIF_SOCKET_UTILS_SRC), $(wildcard nifs/common/*.c)) +NIF_COMMON_SRC=$(filter-out $(wildcard nifs/common/socket_*.c), $(wildcard nifs/common/*.c)) else NIF_COMMON_SRC=$(wildcard nifs/common/*.c) endif diff --git a/erts/emulator/beam/beam_bif_load.c b/erts/emulator/beam/beam_bif_load.c index 6f65734e9c..d551863978 100644 --- a/erts/emulator/beam/beam_bif_load.c +++ b/erts/emulator/beam/beam_bif_load.c @@ -600,7 +600,7 @@ BIF_RETTYPE erts_internal_check_dirty_process_code_2(BIF_ALIST_2) if (BIF_ARG_1 == BIF_P->common.id) BIF_RET(am_normal); - rp = erts_proc_lookup_raw(BIF_ARG_1); + rp = erts_proc_lookup(BIF_ARG_1); if (!rp) BIF_RET(am_false); @@ -619,7 +619,9 @@ BIF_RETTYPE erts_internal_check_dirty_process_code_2(BIF_ALIST_2) if (busy) BIF_RET(am_busy); - res = erts_check_process_code(rp, BIF_ARG_2, &reds, BIF_P->fcalls); + res = (ERTS_PROC_IS_EXITING(rp) + ? am_false + : erts_check_process_code(rp, BIF_ARG_2, &reds, BIF_P->fcalls)); erts_proc_unlock(rp, ERTS_PROC_LOCK_MAIN); diff --git a/erts/emulator/beam/bs_instrs.tab b/erts/emulator/beam/bs_instrs.tab index fc3da7d016..df45dd54c6 100644 --- a/erts/emulator/beam/bs_instrs.tab +++ b/erts/emulator/beam/bs_instrs.tab @@ -294,10 +294,13 @@ i_new_bs_put_binary(Fail, Sz, Flags, Src) { Eterm sz = $Sz; Sint _size; $BS_GET_UNCHECKED_FIELD_SIZE(sz, (($Flags) >> 3), $BADARG($Fail), _size); - if (!erts_new_bs_put_binary(ERL_BITS_ARGS_2(($Src), _size))) { + c_p->fcalls = FCALLS; + if (!erts_new_bs_put_binary(c_p, $Src, _size)) { $BADARG($Fail); } + FCALLS = c_p->fcalls; } + i_new_bs_put_binary_all := i_new_bs_put_binary_all.fetch.execute; i_new_bs_put_binary_all.head() { @@ -309,15 +312,19 @@ i_new_bs_put_binary_all.fetch(Src) { } i_new_bs_put_binary_all.execute(Fail, Unit) { - if (!erts_new_bs_put_binary_all(ERL_BITS_ARGS_2(src, ($Unit)))) { + c_p->fcalls = FCALLS; + if (!erts_new_bs_put_binary_all(c_p, src, ($Unit))) { $BADARG($Fail); } + FCALLS = c_p->fcalls; } i_new_bs_put_binary_imm(Fail, Sz, Src) { - if (!erts_new_bs_put_binary(ERL_BITS_ARGS_2(($Src), ($Sz)))) { + c_p->fcalls = FCALLS; + if (!erts_new_bs_put_binary(c_p, ($Src), ($Sz))) { $BADARG($Fail); } + FCALLS = c_p->fcalls; } i_new_bs_put_float(Fail, Sz, Flags, Src) { @@ -707,11 +714,13 @@ i_bs_append(Fail, ExtraHeap, Live, Unit, Size, Dst) { i_bs_private_append(Fail, Unit, Size, Src, Dst) { Eterm res; + c_p->fcalls = FCALLS; res = erts_bs_private_append(c_p, $Src, $Size, $Unit); if (is_non_value(res)) { /* c_p->freason is already set (to BADARG or SYSTEM_LIMIT). */ $FAIL_HEAD_OR_BODY($Fail); } + FCALLS = c_p->fcalls; $Dst = res; } diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index f61aad241e..de7309b248 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -1091,6 +1091,9 @@ void init_dist(void) szp = NULL; } } + ERTS_CT_ASSERT(sizeof(ErtsDistOutputBuf) % sizeof(void*) == 0); + ERTS_CT_ASSERT(sizeof(SysIOVec) % sizeof(void*) == 0); + ERTS_CT_ASSERT(sizeof(ErlIOVec) % sizeof(void*) == 0); } static ERTS_INLINE ErtsDistOutputBuf * @@ -1102,12 +1105,9 @@ alloc_dist_obufs(byte **extp, TTBEncodeContext *ctx, char *ptr; Uint iov_sz, obsz; Binary *bin; - ErlIOVec **feiov; Uint fragment_size; obsz = sizeof(ErtsDistOutputBuf)*fragments; - if (obsz % sizeof(void *) != 0) - obsz += sizeof(void *) - (obsz % sizeof(void *)); iov_sz = erts_ttb_iov_size(0, vlen, fragments); @@ -1123,15 +1123,14 @@ alloc_dist_obufs(byte **extp, TTBEncodeContext *ctx, else fragment_size = ~((Uint) 0); - feiov = erts_ttb_iov_init(ctx, 0, ptr, vlen, - fragments, fragment_size); + erts_ttb_iov_init(ctx, 0, ptr, vlen, fragments, fragment_size); ptr += iov_sz; erts_refc_add(&bin->intern.refc, fragments - 1, 1); for (ix = 0; ix < fragments; ix++) { obuf[ix].bin = bin; - obuf[ix].eiov = feiov[ix]; + obuf[ix].eiov = &ctx->fragment_eiovs[ix]; #ifdef DEBUG obuf[ix].dbg_pattern = ERTS_DIST_OUTPUT_BUF_DBG_PATTERN; #endif @@ -5041,10 +5040,17 @@ BIF_RETTYPE erts_internal_get_creation_0(BIF_ALIST_0) { Eterm *hp; Uint hsz = 0; + Uint32 cr = erts_this_dist_entry->creation; + Eterm ret; - erts_bld_uint(NULL, &hsz, erts_this_dist_entry->creation); - hp = HAlloc(BIF_P, hsz); - return erts_bld_uint(&hp, NULL, erts_this_dist_entry->creation); + if (cr == 0) + ret = am_undefined; + else { + erts_bld_uint(NULL, &hsz, cr); + hp = HAlloc(BIF_P, hsz); + ret = erts_bld_uint(&hp, NULL, cr); + } + return ret; } BIF_RETTYPE erts_internal_new_connection_1(BIF_ALIST_1) diff --git a/erts/emulator/beam/dist.h b/erts/emulator/beam/dist.h index d14b94613a..170bb569f5 100644 --- a/erts/emulator/beam/dist.h +++ b/erts/emulator/beam/dist.h @@ -261,7 +261,7 @@ typedef struct TTBEncodeContext_ { int iovec; Uint fragment_size; Sint frag_ix; - ErlIOVec **fragment_eiovs; + ErlIOVec *fragment_eiovs; #ifdef DEBUG int debug_fragments; int debug_vlen; @@ -414,6 +414,6 @@ void erts_dist_print_procs_suspended_on_de(fmtfn_t to, void *to_arg); int erts_auto_connect(DistEntry* dep, Process *proc, ErtsProcLocks proc_locks); Uint erts_ttb_iov_size(int use_termv, Sint vlen, Uint fragments); -ErlIOVec **erts_ttb_iov_init(TTBEncodeContext *ctx, int use_termv, char *ptr, - Sint vlen, Uint fragments, Uint fragments_size); +void erts_ttb_iov_init(TTBEncodeContext *ctx, int use_termv, char *ptr, + Sint vlen, Uint fragments, Uint fragments_size); #endif diff --git a/erts/emulator/beam/erl_bif_lists.c b/erts/emulator/beam/erl_bif_lists.c index 2302f4f88f..7cb9b4900b 100644 --- a/erts/emulator/beam/erl_bif_lists.c +++ b/erts/emulator/beam/erl_bif_lists.c @@ -953,10 +953,8 @@ static int subtract_continue(Process *p, ErtsSubtractContext *context) { case SUBTRACT_STAGE_SET_FINISH: { return subtract_set_finish(p, context); } - - default: - ERTS_ASSERT(!"unreachable"); } + ERTS_INTERNAL_ERROR("unreachable"); } static int subtract_start(Process *p, Eterm lhs, Eterm rhs, diff --git a/erts/emulator/beam/erl_bif_persistent.c b/erts/emulator/beam/erl_bif_persistent.c index efdffb39fb..91cc03fe57 100644 --- a/erts/emulator/beam/erl_bif_persistent.c +++ b/erts/emulator/beam/erl_bif_persistent.c @@ -1101,7 +1101,7 @@ static OldLiteral* alloc_old_literal(void) static void free_old_literal(OldLiteral* olp) { - return erts_free(ERTS_ALC_T_RELEASE_LAREA, olp); + erts_free(ERTS_ALC_T_RELEASE_LAREA, olp); } static void diff --git a/erts/emulator/beam/erl_bits.c b/erts/emulator/beam/erl_bits.c index 7084642f7c..79c5781929 100644 --- a/erts/emulator/beam/erl_bits.c +++ b/erts/emulator/beam/erl_bits.c @@ -41,6 +41,18 @@ #define BIT_IS_MACHINE_ENDIAN(x) (((x)&BSF_LITTLE) == BIT_ENDIAN_MACHINE) /* + * Here is how many bits we can copy in each reduction. + * + * At the time of writing of this comment, CONTEXT_REDS was 4000 and + * BITS_PER_REDUCTION was 1 KiB (8192 bits). The time for copying an + * unaligned 4000 KiB binary on my computer (which has a 4,2 GHz Intel + * i7 CPU) was about 5 ms. The time was approximately 4 times lower if + * the source and destinations binaries were aligned. + */ + +#define BITS_PER_REDUCTION (8*1024) + +/* * MAKE_MASK(n) constructs a mask with n bits. * Example: MAKE_MASK(3) returns the binary number 00000111. */ @@ -977,14 +989,14 @@ erts_bs_put_utf16(ERL_BITS_PROTO_2(Eterm arg, Uint flags)) erts_bin_offset += num_bits; return 1; } - int -erts_new_bs_put_binary(ERL_BITS_PROTO_2(Eterm arg, Uint num_bits)) +erts_new_bs_put_binary(Process *c_p, Eterm arg, Uint num_bits) { byte *bptr; Uint bitoffs; Uint bitsize; + ERL_BITS_DEFINE_STATEP(c_p); if (!is_binary(arg)) { return 0; @@ -995,16 +1007,18 @@ erts_new_bs_put_binary(ERL_BITS_PROTO_2(Eterm arg, Uint num_bits)) } copy_binary_to_buffer(erts_current_bin, erts_bin_offset, bptr, bitoffs, num_bits); erts_bin_offset += num_bits; + BUMP_REDS(c_p, num_bits / BITS_PER_REDUCTION); return 1; } int -erts_new_bs_put_binary_all(ERL_BITS_PROTO_2(Eterm arg, Uint unit)) +erts_new_bs_put_binary_all(Process *c_p, Eterm arg, Uint unit) { byte *bptr; Uint bitoffs; Uint bitsize; Uint num_bits; + ERL_BITS_DEFINE_STATEP(c_p); /* * This type test is not needed if the code was compiled with @@ -1029,6 +1043,7 @@ erts_new_bs_put_binary_all(ERL_BITS_PROTO_2(Eterm arg, Uint unit)) } copy_binary_to_buffer(erts_current_bin, erts_bin_offset, bptr, bitoffs, num_bits); erts_bin_offset += num_bits; + BUMP_REDS(c_p, num_bits / BITS_PER_REDUCTION); return 1; } @@ -1352,6 +1367,7 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term, binp = erts_bin_realloc(binp, new_size); pb->val = binp; pb->bytes = (byte *) binp->orig_bytes; + BUMP_REDS(c_p, pb->size / BITS_PER_REDUCTION); } erts_current_bin = pb->bytes; @@ -1473,6 +1489,7 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term, * Now copy the data into the binary. */ copy_binary_to_buffer(erts_current_bin, 0, src_bytes, bitoffs, erts_bin_offset); + BUMP_REDS(c_p, erts_bin_offset / BITS_PER_REDUCTION); return make_binary(sb); } @@ -1537,6 +1554,7 @@ erts_bs_private_append(Process* p, Eterm bin, Eterm build_size_term, Uint unit) if (binp->orig_size < pb->size) { Uint new_size = 2*pb->size; + BUMP_REDS(p, pb->size / BITS_PER_REDUCTION); if (pb->flags & PB_IS_WRITABLE) { /* * This is the normal case - the binary is writable. diff --git a/erts/emulator/beam/erl_bits.h b/erts/emulator/beam/erl_bits.h index eced966a7f..d9262bea00 100644 --- a/erts/emulator/beam/erl_bits.h +++ b/erts/emulator/beam/erl_bits.h @@ -164,8 +164,8 @@ Eterm erts_bs_get_binary_all_2(Process *p, ErlBinMatchBuffer* mb); int erts_new_bs_put_integer(ERL_BITS_PROTO_3(Eterm Integer, Uint num_bits, unsigned flags)); int erts_bs_put_utf8(ERL_BITS_PROTO_1(Eterm Integer)); int erts_bs_put_utf16(ERL_BITS_PROTO_2(Eterm Integer, Uint flags)); -int erts_new_bs_put_binary(ERL_BITS_PROTO_2(Eterm Bin, Uint num_bits)); -int erts_new_bs_put_binary_all(ERL_BITS_PROTO_2(Eterm Bin, Uint unit)); +int erts_new_bs_put_binary(Process *c_p, Eterm Bin, Uint num_bits); +int erts_new_bs_put_binary_all(Process *c_p, Eterm Bin, Uint unit); int erts_new_bs_put_float(Process *c_p, Eterm Float, Uint num_bits, int flags); void erts_new_bs_put_string(ERL_BITS_PROTO_2(byte* iptr, Uint num_bytes)); diff --git a/erts/emulator/beam/erl_hl_timer.c b/erts/emulator/beam/erl_hl_timer.c index b0eb0e85c0..aec31080d2 100644 --- a/erts/emulator/beam/erl_hl_timer.c +++ b/erts/emulator/beam/erl_hl_timer.c @@ -2819,6 +2819,10 @@ btm_tree_print(ErtsBifTimer *tmr, void *vbtmp, Sint reds) { int is_hlt = !!(tmr->type.head.roflgs & ERTS_TMR_ROFLG_HLT); ErtsMonotonicTime tpos; + + if (erts_atomic32_read_nob(&tmr->btm.state) != ERTS_TMR_STATE_ACTIVE) + return 1; + if (is_hlt) tpos = 0; else diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index da9fb454e8..9b12c459fe 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -1774,16 +1774,20 @@ ERL_NIF_TERM enif_make_uint(ErlNifEnv* env, unsigned i) ERL_NIF_TERM enif_make_long(ErlNifEnv* env, long i) { +#if SIZEOF_LONG < ERTS_SIZEOF_ETERM + return make_small(i); +#else if (IS_SSMALL(i)) { return make_small(i); } -#if SIZEOF_LONG == ERTS_SIZEOF_ETERM +# if SIZEOF_LONG == ERTS_SIZEOF_ETERM return small_to_big(i, alloc_heap(env,2)); -#elif SIZEOF_LONG_LONG == ERTS_SIZEOF_ETERM +# elif SIZEOF_LONG_LONG == ERTS_SIZEOF_ETERM return make_small(i); -#elif SIZEOF_LONG == 8 +# elif SIZEOF_LONG == 8 ensure_heap(env,3); return erts_sint64_to_big(i, &env->hp); +# endif #endif } diff --git a/erts/emulator/beam/erl_node_tables.c b/erts/emulator/beam/erl_node_tables.c index 252bbbb229..929795fcb0 100644 --- a/erts/emulator/beam/erl_node_tables.c +++ b/erts/emulator/beam/erl_node_tables.c @@ -172,6 +172,7 @@ dist_table_alloc(void *dep_tmpl) erts_rwmtx_init_opt(&dep->rwmtx, &rwmtx_opt, "dist_entry", sysname, ERTS_LOCK_FLAGS_CATEGORY_DISTRIBUTION); dep->sysname = sysname; + dep->creation = 0; /* undefined */ dep->cid = NIL; erts_atomic_init_nob(&dep->input_handler, (erts_aint_t) NIL); dep->connection_id = 0; diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index 5139ede1f1..6c83693df8 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -6933,13 +6933,25 @@ schedule_process_sys_task(Process *p, erts_aint32_t prio, ErtsProcSysTask *st, { erts_aint32_t fail_state, state; + fail_state = *fail_state_p; + /* Elevate priority if needed. */ - state = erts_atomic32_read_nob(&p->state); - if (ERTS_PSFLGS_GET_ACT_PRIO(state) > prio) { + state = erts_atomic32_read_acqb(&p->state); + if (ERTS_PSFLGS_GET_ACT_PRIO(state) <= prio) { + if (state & fail_state) { + *fail_state_p = state & fail_state; + return 0; + } + } + else { erts_aint32_t n, a, e; a = state; do { + if (a & fail_state) { + *fail_state_p = a & fail_state; + return 0; + } if (ERTS_PSFLGS_GET_ACT_PRIO(a) <= prio) { n = a; break; @@ -6953,8 +6965,6 @@ schedule_process_sys_task(Process *p, erts_aint32_t prio, ErtsProcSysTask *st, state = n; } - fail_state = *fail_state_p; - return !(active_sys_enqueue(p, st, prio, ERTS_PSFLG_SYS_TASKS, state, fail_state_p) & fail_state); } @@ -10191,13 +10201,14 @@ fetch_sys_task(Process *c_p, erts_aint32_t state, int *qmaskp, int *priop) *priop = PRIORITY_HIGH; break; case NORMAL_BIT: - if (!(qmask & PRIORITY_LOW) + if (!(qmask & LOW_BIT) || ++c_p->sys_task_qs->ncount <= RESCHEDULE_LOW) { qp = &c_p->sys_task_qs->q[PRIORITY_NORMAL]; *priop = PRIORITY_NORMAL; break; } c_p->sys_task_qs->ncount = 0; + qbit = LOW_BIT; /* Fall through */ case LOW_BIT: qp = &c_p->sys_task_qs->q[PRIORITY_LOW]; @@ -10727,7 +10738,7 @@ request_system_task(Process *c_p, Eterm requester, Eterm target, Eterm priority, Eterm operation) { BIF_RETTYPE ret; - Process *rp = erts_proc_lookup(target); + Process *rp = erts_proc_lookup_raw(target); ErtsProcSysTask *st = NULL; erts_aint32_t prio, fail_state = ERTS_PSFLG_EXITING; Eterm noproc_res, req_type; @@ -11177,7 +11188,7 @@ erts_set_gc_state(Process *c_p, int enable) first1 = dgc_tsk_qs->q[prio]; last1 = first1->prev; first2 = stsk_qs->q[prio]; - last2 = first1->prev; + last2 = first2->prev; last1->next = first2; first2->prev = last1; @@ -12563,7 +12574,7 @@ delete_process(Process* p) * The mso list should not be used anymore, but if it is, make sure that * we'll notice. */ - p->off_heap.first = (void *) 0x8DEFFACD; + p->off_heap.first = (void*)(UWord)0x8DEFFACD; if (p->arg_reg != p->def_arg_reg) { erts_free(ERTS_ALC_T_ARG_REG, p->arg_reg); diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 1febf95b33..8dfa5b1bbb 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -2706,6 +2706,7 @@ erts_get_atom_cache_map(Process *c_p) * erl_process.c on windows. */ # define ERTS_TIME2REDS_IMPL__ erts_time2reds__ +Sint64 erts_time2reds(ErtsMonotonicTime start, ErtsMonotonicTime end); #else # define ERTS_TIME2REDS_IMPL__ erts_time2reds #endif diff --git a/erts/emulator/beam/erl_sys_driver.h b/erts/emulator/beam/erl_sys_driver.h index a58f11036c..85c2d6c4ca 100644 --- a/erts/emulator/beam/erl_sys_driver.h +++ b/erts/emulator/beam/erl_sys_driver.h @@ -31,7 +31,7 @@ #define ERL_SYS_DRV -typedef long ErlDrvEvent; /* An event to be selected on. */ +typedef SWord ErlDrvEvent; /* An event to be selected on. */ typedef struct _SysDriverOpts SysDriverOpts; diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index dd08bc2d02..6981e14fa8 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -67,6 +67,19 @@ #define ERTS_MAX_TINY_CREATION (3) #define is_tiny_creation(Cre) ((unsigned)(Cre) <= ERTS_MAX_TINY_CREATION) +/* + * When 0 is used as creation, the real creation + * is unknown. Creation 0 on data will be changed to current + * creation of the node which it belongs to when it enters + * that node. + * This typically happens when a remote pid is created with + * list_to_pid/1 and then sent to the remote node. This behavior + * has the undesirable effect that a pid can be passed between nodes, + * and as a result of that not being equal to itself (the pid that + * comes back isn't equal to the original pid). + * + */ + #undef ERTS_DEBUG_USE_DIST_SEP #ifdef DEBUG # if 0 @@ -85,20 +98,6 @@ */ #define IS_SSMALL32(x) (((Uint) (((x) >> (32-1)) + 1)) < 2) -/* - * Valid creations for nodes are 1, 2, or 3. 0 can also be sent - * as creation, though. When 0 is used as creation, the real creation - * is unknown. Creation 0 on data will be changed to current - * creation of the node which it belongs to when it enters - * that node. - * This typically happens when a remote pid is created with - * list_to_pid/1 and then sent to the remote node. This behavior - * has the undesirable effect that a pid can be passed between nodes, - * and as a result of that not being equal to itself (the pid that - * comes back isn't equal to the original pid). - * - */ - static Export term_to_binary_trap_export; static byte* enc_term(ErtsAtomCacheMap *, Eterm, byte*, Uint64, struct erl_off_heap_header** off_heap); @@ -2334,31 +2333,24 @@ erts_ttb_iov_size(int use_termv, Sint vlen, Uint fragments) ASSERT(vlen > 0); ASSERT(fragments > 0); sz = sizeof(SysIOVec)*vlen; - if (sz % sizeof(void *) != 0) - sz += sizeof(void *) - (sz % sizeof(void *)); sz += sizeof(ErlDrvBinary *)*vlen; if (use_termv) sz += sizeof(Eterm)*vlen; sz += sizeof(ErlIOVec *)*fragments; sz += sizeof(ErlIOVec)*fragments; - if (sz % sizeof(void *) != 0) - sz += sizeof(void *) - (sz % sizeof(void *)); + ASSERT(sz % sizeof(void*) == 0); return sz; } -ErlIOVec ** +void erts_ttb_iov_init(TTBEncodeContext *ctx, int use_termv, char *ptr, Sint vlen, Uint fragments, Uint fragment_size) { - int i; - ErlIOVec *feiovp; ctx->vlen = 0; ctx->size = 0; ctx->iov = (SysIOVec *) ptr; ptr += sizeof(SysIOVec)*vlen; - if (((UWord) ptr) % sizeof(void *) != 0) - ptr += sizeof(void *) - (((UWord) ptr) % sizeof(void *)); ASSERT(((UWord) ptr) % sizeof(void *) == 0); ctx->binv = (ErlDrvBinary **) ptr; @@ -2371,19 +2363,10 @@ erts_ttb_iov_init(TTBEncodeContext *ctx, int use_termv, char *ptr, ptr += sizeof(Eterm)*vlen; } - ctx->fragment_eiovs = (ErlIOVec **) ptr; - ptr += sizeof(ErlIOVec *)*fragments; - - feiovp = (ErlIOVec *) ptr; + ctx->fragment_eiovs = (ErlIOVec *) ptr; ptr += sizeof(ErlIOVec)*fragments; - - if (((UWord) ptr) % sizeof(void *) != 0) - ptr += sizeof(void *) - (((UWord) ptr) % sizeof(void *)); ASSERT(((UWord) ptr) % sizeof(void *) == 0); - for (i = 0; i < fragments; i++) - ctx->fragment_eiovs[i] = &feiovp[i]; - ctx->frag_ix = -1; ctx->fragment_size = fragment_size; @@ -2392,7 +2375,6 @@ erts_ttb_iov_init(TTBEncodeContext *ctx, int use_termv, char *ptr, ctx->debug_fragments = fragments; ctx->debug_vlen = vlen; #endif - return ctx->fragment_eiovs; } static Eterm erts_term_to_binary_int(Process* p, Sint bif_ix, Eterm Term, Eterm opts, @@ -3659,36 +3641,27 @@ enc_term_int(TTBEncodeContext* ctx, ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, } static ERTS_INLINE void -store_in_vec_aux(Uint *szp, - Sint *vlenp, - SysIOVec *iov, - ErlDrvBinary **binv, - Eterm *termv, +store_in_vec_aux(TTBEncodeContext *ctx, Binary *bin, Eterm term, byte *ptr, - Uint len, - Sint *fixp, - ErlIOVec **frag_eiovpp, - Uint frag_size) + Uint len) { ErlDrvBinary *dbin = Binary2ErlDrvBinary(bin); - Uint size = 0; - int vlen = *vlenp; + int vlen = ctx->vlen; Uint iov_len; ErlIOVec *feiovp; - Sint fix = *fixp; ASSERT(((byte *) &bin->orig_bytes[0]) <= ptr); ASSERT(ptr + len <= ((byte *) &bin->orig_bytes[0]) + bin->orig_size); - if (fix >= 0) { - feiovp = frag_eiovpp[fix]; + if (ctx->frag_ix >= 0) { + feiovp = &ctx->fragment_eiovs[ctx->frag_ix]; ASSERT(0 < feiovp->size); - ASSERT(feiovp->size <= frag_size); - if (feiovp->size != frag_size) { + ASSERT(feiovp->size <= ctx->fragment_size); + if (feiovp->size != ctx->fragment_size) { /* current fragment not full yet... */ - iov_len = frag_size - feiovp->size; + iov_len = ctx->fragment_size - feiovp->size; if (len < iov_len) iov_len = len; goto store_iov_data; @@ -3697,33 +3670,33 @@ store_in_vec_aux(Uint *szp, while (len) { /* Start new fragment... */ + ctx->frag_ix++; + feiovp = &ctx->fragment_eiovs[ctx->frag_ix]; + ASSERT(ctx->frag_ix >= 0); - feiovp = frag_eiovpp[++fix]; - ASSERT(fix >= 0); - - if (termv) { - termv[vlen] = THE_NON_VALUE; - termv[vlen+1] = THE_NON_VALUE; + if (ctx->termv) { + ctx->termv[vlen] = THE_NON_VALUE; + ctx->termv[vlen+1] = THE_NON_VALUE; } feiovp->vsize = 2; feiovp->size = 0; - feiovp->iov = &iov[vlen]; - feiovp->binv = &binv[vlen]; + feiovp->iov = &ctx->iov[vlen]; + feiovp->binv = &ctx->binv[vlen]; /* entry for driver header */ - iov[vlen].iov_base = NULL; - iov[vlen].iov_len = 0; - binv[vlen] = NULL; + ctx->iov[vlen].iov_base = NULL; + ctx->iov[vlen].iov_len = 0; + ctx->binv[vlen] = NULL; vlen++; /* entry for dist header */ - iov[vlen].iov_base = NULL; - iov[vlen].iov_len = 0; - binv[vlen] = NULL; + ctx->iov[vlen].iov_base = NULL; + ctx->iov[vlen].iov_len = 0; + ctx->binv[vlen] = NULL; vlen++; - iov_len = len < frag_size ? len : frag_size; + iov_len = len < ctx->fragment_size ? len : ctx->fragment_size; store_iov_data: @@ -3739,14 +3712,14 @@ store_in_vec_aux(Uint *szp, iov_len = MAX_SYSIOVEC_IOVLEN; } - iov[vlen].iov_base = ptr; - iov[vlen].iov_len = iov_len; - binv[vlen] = dbin; - if (termv) - termv[vlen] = term; + ctx->iov[vlen].iov_base = ptr; + ctx->iov[vlen].iov_len = iov_len; + ctx->binv[vlen] = dbin; + if (ctx->termv) + ctx->termv[vlen] = term; else erts_refc_inc(&bin->intern.refc, 2); - size += iov_len; + ctx->size += iov_len; len -= iov_len; ptr += iov_len; vlen++; @@ -3757,9 +3730,7 @@ store_in_vec_aux(Uint *szp, } while (iov_len); } - *fixp = fix; - *vlenp = vlen; - *szp += size; + ctx->vlen = vlen; } static void @@ -3773,32 +3744,22 @@ store_in_vec(TTBEncodeContext *ctx, byte *cp = ctx->cptr; if (cp != ep) { /* save data in common binary... */ - store_in_vec_aux(&ctx->size, - &ctx->vlen, - ctx->iov, - ctx->binv, - ctx->termv, + store_in_vec_aux(ctx, ctx->result_bin, THE_NON_VALUE, - cp, ep - cp, - &ctx->frag_ix, - ctx->fragment_eiovs, - ctx->fragment_size); + cp, + ep - cp); ASSERT(ctx->vlen <= ctx->debug_vlen); ASSERT(ctx->frag_ix <= ctx->debug_fragments); ctx->cptr = ep; } if (ohbin) { /* save off-heap binary... */ - store_in_vec_aux(&ctx->size, - &ctx->vlen, - ctx->iov, - ctx->binv, - ctx->termv, - ohbin, ohpb, ohp, ohsz, - &ctx->frag_ix, - ctx->fragment_eiovs, - ctx->fragment_size); + store_in_vec_aux(ctx, + ohbin, + ohpb, + ohp, + ohsz); ASSERT(ctx->vlen <= ctx->debug_vlen); ASSERT(ctx->frag_ix <= ctx->debug_fragments); } @@ -5808,6 +5769,11 @@ Sint transcode_dist_obuf(ErtsDistOutputBuf* ob, * Suppress monitor control msg (see erts_dsig_send_monitor) * by converting it to an empty (tick) packet. */ + int i; + for (i = 1; i < ob->eiov->vsize; i++) { + if (ob->eiov->binv[i]) + driver_free_binary(ob->eiov->binv[i]); + } ob->eiov->vsize = 1; ob->eiov->size = 0; return reds; @@ -5838,6 +5804,7 @@ Sint transcode_dist_obuf(ErtsDistOutputBuf* ob, byte *buf_start, *buf_end; byte *ptr; Uint hsz; + int i; hdr += 4; payload_ix = get_int32(hdr); @@ -5899,6 +5866,10 @@ Sint transcode_dist_obuf(ErtsDistOutputBuf* ob, if (buf_start != (byte *) iov[2].iov_base) erts_free(ERTS_ALC_T_TMP, buf_start); + for (i = 1; i < ob->eiov->vsize; i++) { + if (ob->eiov->binv[i]) + driver_free_binary(ob->eiov->binv[i]); + } ob->eiov->vsize = 1; ob->eiov->size = 0; diff --git a/erts/emulator/beam/ops.tab b/erts/emulator/beam/ops.tab index 686299b366..f3ab8bf48d 100644 --- a/erts/emulator/beam/ops.tab +++ b/erts/emulator/beam/ops.tab @@ -390,6 +390,12 @@ move S1=x D1=x | move Y1=y X1=x | independent_moves(Y1, X1, S1, D1) => \ move2_par Y1 X1 S1 D1 move2_par y x x x +# move2_par y y y y + +move Y1=y Y2=y | move Y3=y Y4=y | independent_moves(Y1, Y2, Y3, Y4) => \ + move2_par Y1 Y2 Y3 Y4 +move2_par y y y y + # move3 move2_par Y1=y X1=x Y2=y X2=x | move Y3=y X3=x => move3 Y1 X1 Y2 X2 Y3 X3 diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 868a001a36..6797b5d32f 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -591,7 +591,10 @@ static int my_strncasecmp(const char *s1, const char *s2, size_t n) #ifdef HAVE_SOCKLEN_T # define SOCKLEN_T socklen_t +#elif defined(__WIN32__) +# define SOCKLEN_T int #else +# warning "Non-Windows OS without type 'socklen_t'" # define SOCKLEN_T size_t #endif @@ -1113,8 +1116,8 @@ typedef struct { inet_address* peer_ptr; /* fake peername or NULL */ inet_address* name_ptr; /* fake sockname or NULL */ - SOCKLEN_T peer_addr_len; /* fake peername size */ - SOCKLEN_T name_addr_len; /* fake sockname size */ + unsigned int peer_addr_len; /* fake peername size */ + unsigned int name_addr_len; /* fake sockname size */ int bufsz; /* minimum buffer constraint */ unsigned int hsz; /* the list header size, -1 is large !!! */ @@ -4628,10 +4631,10 @@ static void desc_close(inet_descriptor* desc) * be selecting on it. */ if (!INET_IGNORED(desc)) - driver_select(desc->port,(ErlDrvEvent)(long)desc->event, + driver_select(desc->port,(ErlDrvEvent)(SWord)desc->event, ERL_DRV_USE, 0); else - inet_stop_select((ErlDrvEvent)(long)desc->event,NULL); + inet_stop_select((ErlDrvEvent)(SWord)desc->event,NULL); desc->event = INVALID_EVENT; /* closed by stop_select callback */ desc->s = INVALID_SOCKET; desc->event_mask = 0; @@ -4780,7 +4783,7 @@ static ErlDrvSSizeT inet_ctl_fdopen(inet_descriptor* desc, int domain, int type, char** rbuf, ErlDrvSizeT rsize) { inet_address name; - unsigned int sz; + SOCKLEN_T sz; if (bound) { /* check that it is a socket and that the socket is bound */ @@ -5854,9 +5857,10 @@ static ErlDrvSSizeT inet_ctl_getifaddrs(inet_descriptor* desc_p, *buf_p++ = INET_REP_OK; /* Iterate over MIB_IPADDRTABLE or IP_ADAPTER_ADDRESSES */ - for (ia_p = NULL, ip_addrs_p ? ((void *)(i = 0)) : (ia_p = ip_adaddrs_p); + ia_p = NULL; + for (ip_addrs_p ? (void)(i = 0) : (void)(ia_p = ip_adaddrs_p); ip_addrs_p ? (i < ip_addrs_p->dwNumEntries) : (ia_p != NULL); - ip_addrs_p ? ((void *)(i++)) : (ia_p = ia_p->Next)) { + ip_addrs_p ? (void)(i++) : (void)(ia_p = ia_p->Next)) { MIB_IPADDRROW *ipaddrrow_p = NULL; DWORD flags = INET_IFF_MULTICAST; DWORD index = 0; @@ -9458,7 +9462,7 @@ static ErlDrvSSizeT inet_ctl(inet_descriptor* desc, int cmd, char* buf, return ctl_xerror(xerror, rbuf, rsize); else { desc->peer_ptr = &desc->peer_addr; - desc->peer_addr_len = (SOCKLEN_T) len; + desc->peer_addr_len = (unsigned int) len; return ctl_reply(INET_REP_OK, NULL, 0, rbuf, rsize); } } @@ -9533,7 +9537,7 @@ static ErlDrvSSizeT inet_ctl(inet_descriptor* desc, int cmd, char* buf, return ctl_xerror(xerror, rbuf, rsize); else { desc->name_ptr = &desc->name_addr; - desc->name_addr_len = (SOCKLEN_T) len; + desc->name_addr_len = (unsigned int) len; return ctl_reply(INET_REP_OK, NULL, 0, rbuf, rsize); } } @@ -9922,7 +9926,7 @@ static tcp_descriptor* tcp_inet_copy(tcp_descriptor* desc,SOCKET s, /* The new port will be linked and connected to the original caller */ port = driver_create_port(port, owner, "tcp_inet", (ErlDrvData) copy_desc); - if ((long)port == -1) { + if ((SWord)port == -1) { *err = INET_ERRNO_SYSTEM_LIMIT; FREE(copy_desc); return NULL; @@ -12250,14 +12254,15 @@ static void packet_inet_stop(ErlDrvData e) into "udp_descriptor*" or "inet_descriptor*": */ udp_descriptor * udesc = (udp_descriptor*) e; - inet_descriptor* descr = INETP(udesc); + inet_descriptor* desc = INETP(udesc); if (udesc->i_buf != NULL) { release_buffer(udesc->i_buf); udesc->i_buf = NULL; } - ASSERT(NO_SUBSCRIBERS(&(descr->empty_out_q_subs))); - inet_stop(descr); + ASSERT(NO_SUBSCRIBERS(&(desc->empty_out_q_subs))); + async_error_am_all(desc, am_closed); + inet_stop(desc); } static int packet_error(udp_descriptor* udesc, int err) diff --git a/erts/emulator/drivers/win32/registry_drv.c b/erts/emulator/drivers/win32/registry_drv.c index a03a030df8..7d71ef7f53 100644 --- a/erts/emulator/drivers/win32/registry_drv.c +++ b/erts/emulator/drivers/win32/registry_drv.c @@ -195,7 +195,7 @@ reg_from_erlang(ErlDrvData clientData, char* buf, ErlDrvSizeT count) * [HKEY(DWORD), KeyString(string)] */ - hkey = (HKEY) get_int32(buf+0); + hkey = (HKEY)(SWord) get_int32(buf+0); rp->hkey_root = hkey; key = buf+4; result = RegOpenKeyEx(hkey, key, 0, rp->sam, &newKey); @@ -217,7 +217,7 @@ reg_from_erlang(ErlDrvData clientData, char* buf, ErlDrvSizeT count) HKEY newKey; DWORD disposition; - hkey = (HKEY) get_int32(buf+0); + hkey = (HKEY)(SWord) get_int32(buf+0); rp->hkey_root = hkey; key = buf+4; result = RegCreateKeyEx(hkey, key, 0, "", 0, rp->sam, NULL, @@ -512,7 +512,7 @@ state_reply(RegPort* rp, /* Pointer to port structure. */ s[0] = 's'; i = 1; - put_int32((DWORD) root, s+i); + put_int32((DWORD)(SWord) root, s+i); i += 4; memcpy(s+i, name, nameSize); ASSERT(i+nameSize == needed); diff --git a/erts/emulator/drivers/win32/ttsl_drv.c b/erts/emulator/drivers/win32/ttsl_drv.c index 9b72d1fea9..8d59682766 100644 --- a/erts/emulator/drivers/win32/ttsl_drv.c +++ b/erts/emulator/drivers/win32/ttsl_drv.c @@ -149,7 +149,7 @@ static int ttysl_init() static ErlDrvData ttysl_start(ErlDrvPort port, char* buf) { - if ((int)ttysl_port != -1 || console_thread == NULL) { + if ((SWord)ttysl_port != -1 || console_thread == NULL) { return ERL_DRV_ERROR_GENERAL; } start_lbuf(); @@ -215,7 +215,7 @@ static ErlDrvSSizeT ttysl_control(ErlDrvData drv_data, static void ttysl_stop(ErlDrvData ttysl_data) { - if ((int)ttysl_port != -1) { + if ((SWord)ttysl_port != -1) { driver_select(ttysl_port, console_input_event, ERL_DRV_READ, 0); } diff --git a/erts/emulator/nifs/common/socket_nif.c b/erts/emulator/nifs/common/prim_socket_nif.c index c38a3f5511..3ecc2a6b54 100644 --- a/erts/emulator/nifs/common/socket_nif.c +++ b/erts/emulator/nifs/common/prim_socket_nif.c @@ -368,7 +368,7 @@ static void (*esock_sctp_freepaddrs)(struct sockaddr *addrs) = NULL; #define ESOCK_GLOBAL_DEBUG_DEFAULT FALSE #define ESOCK_DEBUG_DEFAULT FALSE -/* Counters and stuff (Don't know where to sen2 this stuff anyway) */ +/* Counters and stuff (Don't know where to send this stuff anyway) */ #define ESOCK_NIF_IOW_DEFAULT FALSE @@ -445,23 +445,6 @@ static void (*esock_sctp_freepaddrs)(struct sockaddr *addrs) = NULL; #define ESOCK_GET_RESOURCE(ENV, REF, RES) \ enif_get_resource((ENV), (REF), esocks, (RES)) -#define ESOCK_SEND_FLAG_CONFIRM 0 -#define ESOCK_SEND_FLAG_DONTROUTE 1 -#define ESOCK_SEND_FLAG_EOR 2 -#define ESOCK_SEND_FLAG_MORE 3 -#define ESOCK_SEND_FLAG_NOSIGNAL 4 -#define ESOCK_SEND_FLAG_OOB 5 -#define ESOCK_SEND_FLAG_LOW ESOCK_SEND_FLAG_CONFIRM -#define ESOCK_SEND_FLAG_HIGH ESOCK_SEND_FLAG_OOB - -#define ESOCK_RECV_FLAG_CMSG_CLOEXEC 0 -#define ESOCK_RECV_FLAG_ERRQUEUE 1 -#define ESOCK_RECV_FLAG_OOB 2 -#define ESOCK_RECV_FLAG_PEEK 3 -#define ESOCK_RECV_FLAG_TRUNC 4 -#define ESOCK_RECV_FLAG_LOW ESOCK_RECV_FLAG_CMSG_CLOEXEC -#define ESOCK_RECV_FLAG_HIGH ESOCK_RECV_FLAG_TRUNC - #define ESOCK_RECV_BUFFER_COUNT_DEFAULT 0 #define ESOCK_RECV_BUFFER_SIZE_DEFAULT 8192 #define ESOCK_RECV_CTRL_BUFFER_SIZE_DEFAULT 1024 @@ -504,26 +487,23 @@ typedef union { * This section must be "identical" to the corresponding socket.erl */ -/* domain */ -#define ESOCK_DOMAIN_LOCAL 1 -#define ESOCK_DOMAIN_INET 2 -#define ESOCK_DOMAIN_INET6 3 +#define ESOCK_SEND_FLAG_CONFIRM (1 << 0) +#define ESOCK_SEND_FLAG_DONTROUTE (1 << 1) +#define ESOCK_SEND_FLAG_EOR (1 << 2) +#define ESOCK_SEND_FLAG_MORE (1 << 3) +#define ESOCK_SEND_FLAG_NOSIGNAL (1 << 4) +#define ESOCK_SEND_FLAG_OOB (1 << 5) +/**/ +#define ESOCK_SEND_FLAG_MASK ((1 << 6) - 1) -/* type */ -#define ESOCK_TYPE_STREAM 1 -#define ESOCK_TYPE_DGRAM 2 -#define ESOCK_TYPE_RAW 3 -// #define ESOCK_TYPE_RDM 4 -#define ESOCK_TYPE_SEQPACKET 5 +#define ESOCK_RECV_FLAG_CMSG_CLOEXEC (1 << 0) +#define ESOCK_RECV_FLAG_ERRQUEUE (1 << 1) +#define ESOCK_RECV_FLAG_OOB (1 << 2) +#define ESOCK_RECV_FLAG_PEEK (1 << 3) +#define ESOCK_RECV_FLAG_TRUNC (1 << 4) +/**/ +#define ESOCK_RECV_FLAG_MASK ((1 << 5) - 1) -/* protocol */ -#define ESOCK_PROTOCOL_DEFAULT 0 -#define ESOCK_PROTOCOL_IP 1 -#define ESOCK_PROTOCOL_TCP 2 -#define ESOCK_PROTOCOL_UDP 3 -#define ESOCK_PROTOCOL_SCTP 4 -#define ESOCK_PROTOCOL_ICMP 5 -#define ESOCK_PROTOCOL_IGMP 6 /* shutdown how */ #define ESOCK_SHUTDOWN_HOW_RD 0 @@ -531,121 +511,212 @@ typedef union { #define ESOCK_SHUTDOWN_HOW_RDWR 2 -#define ESOCK_OPT_LEVEL_OTP 0 -#define ESOCK_OPT_LEVEL_SOCKET 1 -#define ESOCK_OPT_LEVEL_IP 2 -#define ESOCK_OPT_LEVEL_IPV6 3 -#define ESOCK_OPT_LEVEL_TCP 4 -#define ESOCK_OPT_LEVEL_UDP 5 -#define ESOCK_OPT_LEVEL_SCTP 6 - -#define ESOCK_OPT_OTP_DEBUG 1 -#define ESOCK_OPT_OTP_IOW 2 -#define ESOCK_OPT_OTP_CTRL_PROC 3 -#define ESOCK_OPT_OTP_RCVBUF 4 -#define ESOCK_OPT_OTP_RCVCTRLBUF 6 -#define ESOCK_OPT_OTP_SNDCTRLBUF 7 -#define ESOCK_OPT_OTP_FD 8 -#define ESOCK_OPT_OTP_META 9 -#define ESOCK_OPT_OTP_DOMAIN 0xFF01 // INTERNAL AND ONLY GET -#define ESOCK_OPT_OTP_TYPE 0xFF02 // INTERNAL AND ONLY GET -#define ESOCK_OPT_OTP_PROTOCOL 0xFF03 // INTERNAL AND ONLY GET -#define ESOCK_OPT_OTP_DTP 0xFF04 // INTERNAL AND ONLY GET - -#define ESOCK_OPT_SOCK_ACCEPTCONN 1 -#define ESOCK_OPT_SOCK_BINDTODEVICE 3 -#define ESOCK_OPT_SOCK_BROADCAST 4 -#define ESOCK_OPT_SOCK_DEBUG 6 -#define ESOCK_OPT_SOCK_DOMAIN 7 -#define ESOCK_OPT_SOCK_DONTROUTE 8 -#define ESOCK_OPT_SOCK_KEEPALIVE 10 -#define ESOCK_OPT_SOCK_LINGER 11 -#define ESOCK_OPT_SOCK_OOBINLINE 13 -#define ESOCK_OPT_SOCK_PASSCRED 14 -#define ESOCK_OPT_SOCK_PEEK_OFF 15 -#define ESOCK_OPT_SOCK_PRIORITY 17 -#define ESOCK_OPT_SOCK_PROTOCOL 18 -#define ESOCK_OPT_SOCK_RCVBUF 19 -#define ESOCK_OPT_SOCK_RCVLOWAT 21 -#define ESOCK_OPT_SOCK_RCVTIMEO 22 -#define ESOCK_OPT_SOCK_REUSEADDR 23 -#define ESOCK_OPT_SOCK_REUSEPORT 24 -#define ESOCK_OPT_SOCK_SNDBUF 27 -#define ESOCK_OPT_SOCK_SNDLOWAT 29 -#define ESOCK_OPT_SOCK_SNDTIMEO 30 -#define ESOCK_OPT_SOCK_TIMESTAMP 31 -#define ESOCK_OPT_SOCK_TYPE 32 - -#define ESOCK_OPT_IP_ADD_MEMBERSHIP 1 -#define ESOCK_OPT_IP_ADD_SOURCE_MEMBERSHIP 2 -#define ESOCK_OPT_IP_BLOCK_SOURCE 3 -#define ESOCK_OPT_IP_DROP_MEMBERSHIP 5 -#define ESOCK_OPT_IP_DROP_SOURCE_MEMBERSHIP 6 -#define ESOCK_OPT_IP_FREEBIND 7 -#define ESOCK_OPT_IP_HDRINCL 8 -#define ESOCK_OPT_IP_MINTTL 9 -#define ESOCK_OPT_IP_MSFILTER 10 -#define ESOCK_OPT_IP_MTU 11 -#define ESOCK_OPT_IP_MTU_DISCOVER 12 -#define ESOCK_OPT_IP_MULTICAST_ALL 13 -#define ESOCK_OPT_IP_MULTICAST_IF 14 -#define ESOCK_OPT_IP_MULTICAST_LOOP 15 -#define ESOCK_OPT_IP_MULTICAST_TTL 16 -#define ESOCK_OPT_IP_NODEFRAG 17 -#define ESOCK_OPT_IP_PKTINFO 19 -#define ESOCK_OPT_IP_RECVDSTADDR 20 -#define ESOCK_OPT_IP_RECVERR 21 -#define ESOCK_OPT_IP_RECVIF 22 -#define ESOCK_OPT_IP_RECVOPTS 23 -#define ESOCK_OPT_IP_RECVORIGDSTADDR 24 -#define ESOCK_OPT_IP_RECVTOS 25 -#define ESOCK_OPT_IP_RECVTTL 26 -#define ESOCK_OPT_IP_RETOPTS 27 -#define ESOCK_OPT_IP_ROUTER_ALERT 28 -#define ESOCK_OPT_IP_SENDSRCADDR 29 // Same as IP_RECVDSTADDR? -#define ESOCK_OPT_IP_TOS 30 -#define ESOCK_OPT_IP_TRANSPARENT 31 -#define ESOCK_OPT_IP_TTL 32 -#define ESOCK_OPT_IP_UNBLOCK_SOURCE 33 - -#define ESOCK_OPT_IPV6_ADDRFORM 1 -#define ESOCK_OPT_IPV6_ADD_MEMBERSHIP 2 -#define ESOCK_OPT_IPV6_AUTHHDR 3 -#define ESOCK_OPT_IPV6_DROP_MEMBERSHIP 6 -#define ESOCK_OPT_IPV6_DSTOPTS 7 -#define ESOCK_OPT_IPV6_FLOWINFO 11 -#define ESOCK_OPT_IPV6_HOPLIMIT 12 -#define ESOCK_OPT_IPV6_HOPOPTS 13 -#define ESOCK_OPT_IPV6_MTU 17 -#define ESOCK_OPT_IPV6_MTU_DISCOVER 18 -#define ESOCK_OPT_IPV6_MULTICAST_HOPS 19 -#define ESOCK_OPT_IPV6_MULTICAST_IF 20 -#define ESOCK_OPT_IPV6_MULTICAST_LOOP 21 -#define ESOCK_OPT_IPV6_RECVERR 24 -#define ESOCK_OPT_IPV6_RECVHOPLIMIT 25 -#define ESOCK_OPT_IPV6_RECVPKTINFO 26 // PKTINFO on FreeBSD -#define ESOCK_OPT_IPV6_RECVTCLASS 27 // Linux and ? -#define ESOCK_OPT_IPV6_ROUTER_ALERT 28 -#define ESOCK_OPT_IPV6_RTHDR 29 -#define ESOCK_OPT_IPV6_TCLASS 30 -#define ESOCK_OPT_IPV6_UNICAST_HOPS 31 -#define ESOCK_OPT_IPV6_V6ONLY 33 - -#define ESOCK_OPT_TCP_CONGESTION 1 -#define ESOCK_OPT_TCP_CORK 2 -#define ESOCK_OPT_TCP_MAXSEG 7 -#define ESOCK_OPT_TCP_NODELAY 9 - -#define ESOCK_OPT_UDP_CORK 1 - -#define ESOCK_OPT_SCTP_ASSOCINFO 2 -#define ESOCK_OPT_SCTP_AUTOCLOSE 8 -#define ESOCK_OPT_SCTP_DISABLE_FRAGMENTS 12 -#define ESOCK_OPT_SCTP_EVENTS 14 -#define ESOCK_OPT_SCTP_INITMSG 18 -#define ESOCK_OPT_SCTP_MAXSEG 21 -#define ESOCK_OPT_SCTP_NODELAY 23 -#define ESOCK_OPT_SCTP_RTOINFO 29 + +/* domain */ +#define ESOCK_DOMAIN_LOCAL 1 +#define ESOCK_DOMAIN_INET 2 +#define ESOCK_DOMAIN_INET6 3 + +/* type */ +#define ESOCK_TYPE_STREAM 101 +#define ESOCK_TYPE_DGRAM 102 +#define ESOCK_TYPE_RAW 103 +// #define ESOCK_TYPE_RDM 104 +#define ESOCK_TYPE_SEQPACKET 105 + +/* protocol */ +#define ESOCK_PROTOCOL_DEFAULT 200 +#define ESOCK_PROTOCOL_IP 201 +#define ESOCK_PROTOCOL_TCP 202 +#define ESOCK_PROTOCOL_UDP 203 +#define ESOCK_PROTOCOL_SCTP 204 +#define ESOCK_PROTOCOL_ICMP 205 +#define ESOCK_PROTOCOL_IGMP 206 + +/* option level */ +#define ESOCK_OPT_LEVEL_OTP 301 +#define ESOCK_OPT_LEVEL_SOCKET 302 +#define ESOCK_OPT_LEVEL_IP 303 +#define ESOCK_OPT_LEVEL_IPV6 304 +#define ESOCK_OPT_LEVEL_TCP 305 +#define ESOCK_OPT_LEVEL_UDP 306 +#define ESOCK_OPT_LEVEL_SCTP 307 + +/* level 'otp' options */ +#define ESOCK_OPT_OTP_DEBUG 1001 +#define ESOCK_OPT_OTP_IOW 1002 +#define ESOCK_OPT_OTP_CTRL_PROC 1003 +#define ESOCK_OPT_OTP_RCVBUF 1004 +//#define ESOCK_OPT_OTP_SNDBUF 1005 +#define ESOCK_OPT_OTP_RCVCTRLBUF 1006 +#define ESOCK_OPT_OTP_SNDCTRLBUF 1007 +#define ESOCK_OPT_OTP_FD 1008 +#define ESOCK_OPT_OTP_META 1009 +/**/ +#define ESOCK_OPT_OTP_DOMAIN 1999 // INTERNAL AND ONLY GET +#define ESOCK_OPT_OTP_TYPE 1998 // INTERNAL AND ONLY GET +#define ESOCK_OPT_OTP_PROTOCOL 1997 // INTERNAL AND ONLY GET +#define ESOCK_OPT_OTP_DTP 1996 // INTERNAL AND ONLY GET + +/* level 'socket' options */ +#define ESOCK_OPT_SOCK_ACCEPTCONN 2001 +//#define ESOCK_OPT_SOCK_ACCEPTFILTER 2002 +#define ESOCK_OPT_SOCK_BINDTODEVICE 2003 +#define ESOCK_OPT_SOCK_BROADCAST 2004 +//#define ESOCK_OPT_SOCK_BUSY_POLL 2005 +#define ESOCK_OPT_SOCK_DEBUG 2006 +#define ESOCK_OPT_SOCK_DOMAIN 2007 +#define ESOCK_OPT_SOCK_DONTROUTE 2008 +//#define ESOCK_OPT_SOCK_ERROR 2009 +#define ESOCK_OPT_SOCK_KEEPALIVE 2010 +#define ESOCK_OPT_SOCK_LINGER 2011 +//#define ESOCK_OPT_SOCK_MARK 2012 +#define ESOCK_OPT_SOCK_OOBINLINE 2013 +#define ESOCK_OPT_SOCK_PASSCRED 2014 +#define ESOCK_OPT_SOCK_PEEK_OFF 2015 +//#define ESOCK_OPT_SOCK_PEERCRED 2016 +#define ESOCK_OPT_SOCK_PRIORITY 2017 +#define ESOCK_OPT_SOCK_PROTOCOL 2018 +#define ESOCK_OPT_SOCK_RCVBUF 2019 +//#define ESOCK_OPT_SOCK_RCVBUFFORCE 2020 +#define ESOCK_OPT_SOCK_RCVLOWAT 2021 +#define ESOCK_OPT_SOCK_RCVTIMEO 2022 +#define ESOCK_OPT_SOCK_REUSEADDR 2023 +#define ESOCK_OPT_SOCK_REUSEPORT 2024 +//#define ESOCK_OPT_SOCK_RXQ_OVFL 2025 +//#define ESOCK_OPT_SOCK_SETFIB 2026 +#define ESOCK_OPT_SOCK_SNDBUF 2027 +//#define ESOCK_OPT_SOCK_SNDBUFFORCE 2028 +#define ESOCK_OPT_SOCK_SNDLOWAT 2029 +#define ESOCK_OPT_SOCK_SNDTIMEO 2030 +#define ESOCK_OPT_SOCK_TIMESTAMP 2031 +#define ESOCK_OPT_SOCK_TYPE 2032 + +/* level 'ip' options */ +#define ESOCK_OPT_IP_ADD_MEMBERSHIP 3001 +#define ESOCK_OPT_IP_ADD_SOURCE_MEMBERSHIP 3002 +#define ESOCK_OPT_IP_BLOCK_SOURCE 3003 +//#define ESOCK_OPT_IP_DONTFRAG 3004 +#define ESOCK_OPT_IP_DROP_MEMBERSHIP 3005 +#define ESOCK_OPT_IP_DROP_SOURCE_MEMBERSHIP 3006 +#define ESOCK_OPT_IP_FREEBIND 3007 +#define ESOCK_OPT_IP_HDRINCL 3008 +#define ESOCK_OPT_IP_MINTTL 3009 +#define ESOCK_OPT_IP_MSFILTER 3010 +#define ESOCK_OPT_IP_MTU 3011 +#define ESOCK_OPT_IP_MTU_DISCOVER 3012 +#define ESOCK_OPT_IP_MULTICAST_ALL 3013 +#define ESOCK_OPT_IP_MULTICAST_IF 3014 +#define ESOCK_OPT_IP_MULTICAST_LOOP 3015 +#define ESOCK_OPT_IP_MULTICAST_TTL 3016 +#define ESOCK_OPT_IP_NODEFRAG 3017 +//#define ESOCK_OPT_IP_OPTIONS 3018 +#define ESOCK_OPT_IP_PKTINFO 3019 +#define ESOCK_OPT_IP_RECVDSTADDR 3020 +#define ESOCK_OPT_IP_RECVERR 3021 +#define ESOCK_OPT_IP_RECVIF 3022 +#define ESOCK_OPT_IP_RECVOPTS 3023 +#define ESOCK_OPT_IP_RECVORIGDSTADDR 3024 +#define ESOCK_OPT_IP_RECVTOS 3025 +#define ESOCK_OPT_IP_RECVTTL 3026 +#define ESOCK_OPT_IP_RETOPTS 3027 +#define ESOCK_OPT_IP_ROUTER_ALERT 3028 +#define ESOCK_OPT_IP_SENDSRCADDR 3029 // Same as IP_RECVDSTADDR? +#define ESOCK_OPT_IP_TOS 3030 +#define ESOCK_OPT_IP_TRANSPARENT 3031 +#define ESOCK_OPT_IP_TTL 3032 +#define ESOCK_OPT_IP_UNBLOCK_SOURCE 3033 + +/* level 'ipv6' options */ +#define ESOCK_OPT_IPV6_ADDRFORM 4001 +#define ESOCK_OPT_IPV6_ADD_MEMBERSHIP 4002 +#define ESOCK_OPT_IPV6_AUTHHDR 4003 +//#define ESOCK_OPT_IPV6_AUTH_LEVEL 4004 +//#define ESOCK_OPT_IPV6_CHECKSUM 4005 +#define ESOCK_OPT_IPV6_DROP_MEMBERSHIP 4006 +#define ESOCK_OPT_IPV6_DSTOPTS 4007 +//#define ESOCK_OPT_IPV6_ESP_NETWORK_LEVEL 4008 +//#define ESOCK_OPT_IPV6_ESP_TRANS_LEVEL 4009 +//#define ESOCK_OPT_IPV6_FAITH 4010 +#define ESOCK_OPT_IPV6_FLOWINFO 4011 +#define ESOCK_OPT_IPV6_HOPLIMIT 4012 +#define ESOCK_OPT_IPV6_HOPOPTS 4013 +//#define ESOCK_OPT_IPV6_IPCOMP_LEVEL 4014 +//#define ESOCK_OPT_IPV6_JOIN_GROUP 4015 +//#define ESOCK_OPT_IPV6_LEAVE_GROUP 4016 +#define ESOCK_OPT_IPV6_MTU 4017 +#define ESOCK_OPT_IPV6_MTU_DISCOVER 4018 +#define ESOCK_OPT_IPV6_MULTICAST_HOPS 4019 +#define ESOCK_OPT_IPV6_MULTICAST_IF 4020 +#define ESOCK_OPT_IPV6_MULTICAST_LOOP 4021 +//#define ESOCK_OPT_IPV6_PORTRANGE 4022 +//#define ESOCK_OPT_IPV6_PKTOPTIONS 4023 +#define ESOCK_OPT_IPV6_RECVERR 4024 +#define ESOCK_OPT_IPV6_RECVHOPLIMIT 4025 +#define ESOCK_OPT_IPV6_RECVPKTINFO 4026 // PKTINFO on FreeBSD +#define ESOCK_OPT_IPV6_RECVTCLASS 4027 // Linux and ? +#define ESOCK_OPT_IPV6_ROUTER_ALERT 4028 +#define ESOCK_OPT_IPV6_RTHDR 4029 +#define ESOCK_OPT_IPV6_TCLASS 4030 +#define ESOCK_OPT_IPV6_UNICAST_HOPS 4031 +//#define ESOCK_OPT_IPV6_USE_MIN_MTU 4032 +#define ESOCK_OPT_IPV6_V6ONLY 4033 + +/* level 'tcp' options */ +#define ESOCK_OPT_TCP_CONGESTION 5001 +#define ESOCK_OPT_TCP_CORK 5002 +//#define ESOCK_OPT_TCP_INFO 5003 +//#define ESOCK_OPT_TCP_KEEPCNT 5004 +//#define ESOCK_OPT_TCP_KEEPIDLE 5005 +//#define ESOCK_OPT_TCP_KEEPINTVL 5006 +#define ESOCK_OPT_TCP_MAXSEG 5007 +//#define ESOCK_OPT_TCP_MD5SIG 5008 +#define ESOCK_OPT_TCP_NODELAY 5009 +//#define ESOCK_OPT_TCP_NOOPT 5010 +//#define ESOCK_OPT_TCP_NOPUSH 5011 +//#define ESOCK_OPT_TCP_SYNCNT 5012 +//#define ESOCK_OPT_TCP_USER_TIMEOUT 5013 + +/* level 'udp' options */ +#define ESOCK_OPT_UDP_CORK 6001 + +/* level 'sctp' options */ +//#define ESOCK_OPT_SCTP_ADAPTION_LAYER 7001 +#define ESOCK_OPT_SCTP_ASSOCINFO 7002 +//#define ESOCK_OPT_SCTP_AUTH_ACTIVE_KEY 7003 +//#define ESOCK_OPT_SCTP_AUTH_ASCONF 7004 +//#define ESOCK_OPT_SCTP_AUTH_CHUNK 7005 +//#define ESOCK_OPT_SCTP_AUTH_KEY 7006 +//#define ESOCK_OPT_SCTP_AUTH_DELETE_KEY 7007 +#define ESOCK_OPT_SCTP_AUTOCLOSE 7008 +//#define ESOCK_OPT_SCTP_CONTEXT 7009 +//#define ESOCK_OPT_SCTP_DEFAULT_SEND_PARAMS 7010 +//#define ESOCK_OPT_SCTP_DELAYED_ACK_TIME 7011 +#define ESOCK_OPT_SCTP_DISABLE_FRAGMENTS 7012 +//#define ESOCK_OPT_SCTP_HMAC_IDENT 7013 +#define ESOCK_OPT_SCTP_EVENTS 7014 +//#define ESOCK_OPT_SCTP_EXPLICIT_EOR 7015 +//#define ESOCK_OPT_SCTP_FRAGMENT_INTERLEAVE 7016 +//#define ESOCK_OPT_SCTP_GET_PEER_ADDR_INFO 7017 +#define ESOCK_OPT_SCTP_INITMSG 7018 +//#define ESOCK_OPT_SCTP_I_WANT_MAPPED_V4_ADDR 7019 +//#define ESOCK_OPT_SCTP_LOCAL_AUTH_CHUNKS 7020 +#define ESOCK_OPT_SCTP_MAXSEG 7021 +//#define ESOCK_OPT_SCTP_MAXBURST 7022 +#define ESOCK_OPT_SCTP_NODELAY 7023 +//#define ESOCK_OPT_SCTP_PARTIAL_DELIVERY_POINT 7024 +//#define ESOCK_OPT_SCTP_PEER_ADDR_PARAMS 7025 +//#define ESOCK_OPT_SCTP_PEER_AUTH_CHUNKS 7026 +//#define ESOCK_OPT_SCTP_PRIMARY_ADDR 7027 +//#define ESOCK_OPT_SCTP_RESET_STREAMS 7028 +#define ESOCK_OPT_SCTP_RTOINFO 7029 +//#define ESOCK_OPT_SCTP_SET_PEER_PRIMARY_ADDR 7030 +//#define ESOCK_OPT_SCTP_STATUS 7031 +//#define ESOCK_OPT_SCTP_USE_EXT_RECVINFO 7032 + + +/*--------------------------------------------------------------------------*/ + /* We should *eventually* use this instead of hard-coding the size (to 1) */ #define ESOCK_RECVMSG_IOVEC_SZ 1 @@ -653,14 +724,6 @@ typedef union { #define ESOCK_CMD_DEBUG 0x0001 #define ESOCK_CMD_SOCKET_DEBUG 0x0002 -#define ESOCK_SUPPORTS_OPTIONS 0x0001 -#define ESOCK_SUPPORTS_SCTP 0x0002 -#define ESOCK_SUPPORTS_IPV6 0x0003 -#define ESOCK_SUPPORTS_LOCAL 0x0004 -#define ESOCK_SUPPORTS_SEND_FLAGS 0x0005 -#define ESOCK_SUPPORTS_RECV_FLAGS 0x0006 -#define ESOCK_SUPPORTS_NETNS 0x0007 - #define ESOCK_WHICH_DOMAIN_ERROR -1 #define ESOCK_WHICH_DOMAIN_UNSUP -2 #define ESOCK_WHICH_TYPE_ERROR -1 @@ -677,9 +740,9 @@ typedef union { /* Global socket debug */ #define SGDBG( proto ) ESOCK_DBG_PRINTF( data.dbg , proto ) -#define SGDBG2( __DBG__ , proto ) ESOCK_DBG_PRINTF( __DBG__ || data.dbg , proto ) /* Socket specific debug */ #define SSDBG( __D__ , proto ) ESOCK_DBG_PRINTF( (__D__)->dbg , proto ) +#define SSDBG2( __DBG__ , proto ) ESOCK_DBG_PRINTF( (__DBG__) , proto ) #define ESOCK_CNT_INC( __E__, __D__, SF, ACNT, CNT, INC) \ { \ @@ -902,9 +965,6 @@ typedef struct { ESockCounter writeTries; ESockCounter writeWaits; ESockCounter writeFails; - /* Connect */ - ESockAddress remote; - socklen_t addrLen; /* +++ Connector +++ */ ESockRequestor connector; ESockRequestor* connectorP; @@ -1058,7 +1118,6 @@ extern char* erl_errno_id(int error); /* THIS IS JUST TEMPORARY??? */ * nif_getopt * nif_sockname * nif_peername - * nif_finalize_connection * nif_finalize_close * nif_cancel */ @@ -1084,7 +1143,6 @@ extern char* erl_errno_id(int error); /* THIS IS JUST TEMPORARY??? */ ESOCK_NIF_FUNC_DEF(getopt); \ ESOCK_NIF_FUNC_DEF(sockname); \ ESOCK_NIF_FUNC_DEF(peername); \ - ESOCK_NIF_FUNC_DEF(finalize_connection); \ ESOCK_NIF_FUNC_DEF(finalize_close); \ ESOCK_NIF_FUNC_DEF(cancel); @@ -1139,26 +1197,27 @@ static ERL_NIF_TERM socket_info_reqs(ErlNifEnv* env, ESockRequestor* crp, ESockRequestQueue* q); -static ERL_NIF_TERM esock_supports(ErlNifEnv* env, int key); -static ERL_NIF_TERM esock_supports_options(ErlNifEnv* env); +static ERL_NIF_TERM esock_supports_0(ErlNifEnv* env); +static ERL_NIF_TERM esock_supports_1(ErlNifEnv* env, ERL_NIF_TERM key); +static ERL_NIF_TERM esock_supports_2(ErlNifEnv* env, + ERL_NIF_TERM key1, int key2); +//static ERL_NIF_TERM esock_supports_options(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options_socket(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options_ip(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options_ipv6(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options_tcp(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options_udp(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_options_sctp(ErlNifEnv* env); -static ERL_NIF_TERM esock_supports_sctp(ErlNifEnv* env); -static ERL_NIF_TERM esock_supports_ipv6(ErlNifEnv* env); -static ERL_NIF_TERM esock_supports_local(ErlNifEnv* env); -static ERL_NIF_TERM esock_supports_netns(ErlNifEnv* env); +//static ERL_NIF_TERM esock_supports_sctp(ErlNifEnv* env); +//static ERL_NIF_TERM esock_supports_ipv6(ErlNifEnv* env); +//static ERL_NIF_TERM esock_supports_local(ErlNifEnv* env); +//static ERL_NIF_TERM esock_supports_netns(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_send_flags(ErlNifEnv* env); static ERL_NIF_TERM esock_supports_recv_flags(ErlNifEnv* env); static ERL_NIF_TERM esock_open2(ErlNifEnv* env, int fd, ERL_NIF_TERM eextra); -static BOOLEAN_T esock_open2_is_debug(ErlNifEnv* env, - ERL_NIF_TERM eextra); static BOOLEAN_T esock_open2_todup(ErlNifEnv* env, ERL_NIF_TERM eextra); static BOOLEAN_T esock_open2_get_domain(ErlNifEnv* env, @@ -1175,6 +1234,9 @@ static ERL_NIF_TERM esock_open4(ErlNifEnv* env, int type, int protocol, ERL_NIF_TERM eopts); +static BOOLEAN_T esock_open_is_debug(ErlNifEnv* env, + ERL_NIF_TERM eextra, + BOOLEAN_T dflt); static BOOLEAN_T esock_open_which_domain(SOCKET sock, int* domain); static BOOLEAN_T esock_open_which_type(SOCKET sock, int* type); static BOOLEAN_T esock_open_which_protocol(SOCKET sock, int* proto); @@ -1205,19 +1267,16 @@ static ERL_NIF_TERM esock_accept_listening_accept(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM sockRef, SOCKET accSock, - ErlNifPid caller, - ESockAddress* remote, - unsigned int len); + ErlNifPid caller); static ERL_NIF_TERM esock_accept_accepting_current(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM sockRef, ERL_NIF_TERM ref); -static ERL_NIF_TERM esock_accept_accepting_current_accept(ErlNifEnv* env, - ESockDescriptor* descP, - ERL_NIF_TERM sockRef, - SOCKET accSock, - ESockAddress* remote, - unsigned int len); +static ERL_NIF_TERM +esock_accept_accepting_current_accept(ErlNifEnv* env, + ESockDescriptor* descP, + ERL_NIF_TERM sockRef, + SOCKET accSock); static ERL_NIF_TERM esock_accept_accepting_current_error(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM sockRef, @@ -1237,8 +1296,6 @@ static BOOLEAN_T esock_accept_accepted(ErlNifEnv* env, ERL_NIF_TERM sockRef, SOCKET accSock, ErlNifPid pid, - ESockAddress* remote, - unsigned int len, ERL_NIF_TERM* result); static ERL_NIF_TERM esock_send(ErlNifEnv* env, ESockDescriptor* descP, @@ -1401,7 +1458,7 @@ static ERL_NIF_TERM esock_setopt_lvl_sock_rcvlowat(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM eVal); #endif -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_setopt_lvl_sock_rcvtimeo(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM eVal); @@ -1426,7 +1483,7 @@ static ERL_NIF_TERM esock_setopt_lvl_sock_sndlowat(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM eVal); #endif -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_setopt_lvl_sock_sndtimeo(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM eVal); @@ -1939,7 +1996,7 @@ static ERL_NIF_TERM esock_getopt_lvl_sock_rcvbuf(ErlNifEnv* env, static ERL_NIF_TERM esock_getopt_lvl_sock_rcvlowat(ErlNifEnv* env, ESockDescriptor* descP); #endif -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_getopt_lvl_sock_rcvtimeo(ErlNifEnv* env, ESockDescriptor* descP); #endif @@ -1959,7 +2016,7 @@ static ERL_NIF_TERM esock_getopt_lvl_sock_sndbuf(ErlNifEnv* env, static ERL_NIF_TERM esock_getopt_lvl_sock_sndlowat(ErlNifEnv* env, ESockDescriptor* descP); #endif -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_getopt_lvl_sock_sndtimeo(ErlNifEnv* env, ESockDescriptor* descP); #endif @@ -2284,11 +2341,13 @@ static ERL_NIF_TERM esock_setopt_int_opt(ErlNifEnv* env, int level, int opt, ERL_NIF_TERM eVal); +#if (defined(SO_RCVTIMEO) || defined(SO_SNDTIMEO)) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_setopt_timeval_opt(ErlNifEnv* env, ESockDescriptor* descP, int level, int opt, ERL_NIF_TERM eVal); +#endif #if defined(USE_GETOPT_STR_OPT) static ERL_NIF_TERM esock_getopt_str_opt(ErlNifEnv* env, @@ -2305,10 +2364,12 @@ static ERL_NIF_TERM esock_getopt_int_opt(ErlNifEnv* env, ESockDescriptor* descP, int level, int opt); +#if (defined(SO_RCVTIMEO) || defined(SO_SNDTIMEO)) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_getopt_timeval_opt(ErlNifEnv* env, ESockDescriptor* descP, int level, int opt); +#endif static BOOLEAN_T send_check_writer(ErlNifEnv* env, ESockDescriptor* descP, @@ -2443,8 +2504,6 @@ static ERL_NIF_TERM recvmsg_check_msg(ErlNifEnv* env, ErlNifBinary* ctrlBufP, ERL_NIF_TERM sockRef); -static ERL_NIF_TERM esock_finalize_connection(ErlNifEnv* env, - ESockDescriptor* descP); static ERL_NIF_TERM esock_finalize_close(ErlNifEnv* env, ESockDescriptor* descP); static int esock_close_socket(ErlNifEnv* env, @@ -2734,7 +2793,7 @@ static size_t my_strnlen(const char *s, size_t maxlen); static void esock_dtor(ErlNifEnv* env, void* obj); static void esock_stop(ErlNifEnv* env, void* obj, - int fd, + ErlNifEvent fd, int is_direct_call); static void esock_down(ErlNifEnv* env, void* obj, @@ -2862,7 +2921,7 @@ static const struct in6_addr in6addr_loopback = /* (special) error string constants */ -static char str_exmon[] = "exmonitor"; // failed monitor +static char str_exmonitor[] = "exmonitor"; // failed monitor static char str_exself[] = "exself"; // failed self static char str_exsend[] = "exsend"; // failed send @@ -3134,6 +3193,7 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(dest_unreach); \ LOCAL_ATOM_DECL(do); \ LOCAL_ATOM_DECL(dont); \ + LOCAL_ATOM_DECL(dup); \ LOCAL_ATOM_DECL(exclude); \ LOCAL_ATOM_DECL(false); \ LOCAL_ATOM_DECL(frag_needed); \ @@ -3181,6 +3241,7 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(num_tstreams); \ LOCAL_ATOM_DECL(num_writers); \ LOCAL_ATOM_DECL(offender); \ + LOCAL_ATOM_DECL(options); \ LOCAL_ATOM_DECL(origin); \ LOCAL_ATOM_DECL(partial_delivery); \ LOCAL_ATOM_DECL(peer_error); \ @@ -3195,12 +3256,14 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') LOCAL_ATOM_DECL(read_pkg_max); \ LOCAL_ATOM_DECL(read_tries); \ LOCAL_ATOM_DECL(read_waits); \ + LOCAL_ATOM_DECL(recv_flags); \ LOCAL_ATOM_DECL(registry); \ LOCAL_ATOM_DECL(reject_route); \ LOCAL_ATOM_DECL(remote); \ LOCAL_ATOM_DECL(select); \ LOCAL_ATOM_DECL(sender_dry); \ LOCAL_ATOM_DECL(send_failure); \ + LOCAL_ATOM_DECL(send_flags); \ LOCAL_ATOM_DECL(shutdown); \ LOCAL_ATOM_DECL(slist); \ LOCAL_ATOM_DECL(socket_debug); \ @@ -3223,12 +3286,12 @@ ERL_NIF_TERM esock_atom_socket_tag; // This has a "special" name ('$socket') #define LOCAL_ERROR_REASON_ATOMS \ LOCAL_ATOM_DECL(econnreset); \ LOCAL_ATOM_DECL(eisconn); \ - LOCAL_ATOM_DECL(enotclosing); \ LOCAL_ATOM_DECL(enotconn); \ + \ LOCAL_ATOM_DECL(exalloc); \ LOCAL_ATOM_DECL(exbadstate); \ - LOCAL_ATOM_DECL(exbusy); \ - LOCAL_ATOM_DECL(exmon); \ + LOCAL_ATOM_DECL(exmonitor); \ + LOCAL_ATOM_DECL(exselect); \ LOCAL_ATOM_DECL(exself); \ LOCAL_ATOM_DECL(exsend) @@ -3393,44 +3456,64 @@ ERL_NIF_TERM esock_global_info(ErlNifEnv* env) numProtoIP, numProtoTCP, numProtoUDP, numProtoSCTP; MLOCK(data.cntMtx); - numBits = MKCT(env, atom_num_cnt_bits, ESOCK_COUNTER_SIZE); - numSockets = MKCT(env, atom_num_sockets, data.numSockets); - numTypeDGrams = MKCT(env, atom_num_tdgrams, data.numTypeDGrams); - numTypeStreams = MKCT(env, atom_num_tstreams, data.numTypeStreams); - numTypeSeqPkgs = MKCT(env, atom_num_tseqpkgs, data.numTypeSeqPkgs); - numDomLocal = MKCT(env, atom_num_dlocal, data.numDomainLocal); - numDomInet = MKCT(env, atom_num_dinet, data.numDomainInet); - numDomInet6 = MKCT(env, atom_num_dinet6, data.numDomainInet6); - numProtoIP = MKCT(env, atom_num_pip, data.numProtoIP); - numProtoTCP = MKCT(env, atom_num_ptcp, data.numProtoTCP); - numProtoUDP = MKCT(env, atom_num_pudp, data.numProtoUDP); - numProtoSCTP = MKCT(env, atom_num_psctp, data.numProtoSCTP); + numBits = MKUI(env, ESOCK_COUNTER_SIZE); + numSockets = MKUI(env, data.numSockets); + numTypeDGrams = MKUI(env, data.numTypeDGrams); + numTypeStreams = MKUI(env, data.numTypeStreams); + numTypeSeqPkgs = MKUI(env, data.numTypeSeqPkgs); + numDomLocal = MKUI(env, data.numDomainLocal); + numDomInet = MKUI(env, data.numDomainInet); + numDomInet6 = MKUI(env, data.numDomainInet6); + numProtoIP = MKUI(env, data.numProtoIP); + numProtoTCP = MKUI(env, data.numProtoTCP); + numProtoUDP = MKUI(env, data.numProtoUDP); + numProtoSCTP = MKUI(env, data.numProtoSCTP); MUNLOCK(data.cntMtx); { - ERL_NIF_TERM gcnt[] = + ERL_NIF_TERM gcntVals[] = {numBits, numSockets, numTypeDGrams, numTypeStreams, numTypeSeqPkgs, numDomLocal, numDomInet, numDomInet6, numProtoIP, numProtoTCP, numProtoUDP, numProtoSCTP}; - unsigned int lenGCnt = - sizeof(gcnt) / sizeof(ERL_NIF_TERM); - ERL_NIF_TERM - lgcnt = MKLA(env, gcnt, lenGCnt), - keys[] = {esock_atom_debug, atom_iow, atom_counters}, - vals[] = {BOOL2ATOM(data.dbg), BOOL2ATOM(data.iow), lgcnt}, - info; - unsigned int - numKeys = sizeof(keys) / sizeof(ERL_NIF_TERM), - numVals = sizeof(vals) / sizeof(ERL_NIF_TERM); - - ESOCK_ASSERT( (numKeys == numVals) ); - - if (!MKMA(env, keys, vals, numKeys, &info)) + ERL_NIF_TERM gcntKeys[] = + {atom_num_cnt_bits, + atom_num_sockets, + atom_num_tdgrams, atom_num_tstreams, atom_num_tseqpkgs, + atom_num_dlocal, atom_num_dinet, atom_num_dinet6, + atom_num_pip, atom_num_ptcp, atom_num_pudp, atom_num_psctp}; + unsigned int numGCntVals = sizeof(gcntVals) / sizeof(ERL_NIF_TERM); + unsigned int numGCntKeys = sizeof(gcntKeys) / sizeof(ERL_NIF_TERM); + ERL_NIF_TERM gcnt; + + ESOCK_ASSERT( (numGCntKeys == numGCntVals) ); + + if (!MKMA(env, gcntKeys, gcntVals, numGCntKeys, &gcnt)) return enif_make_badarg(env); - return info; + { + ERL_NIF_TERM + keys[] = {esock_atom_debug, + atom_socket_debug, + atom_iow, + atom_counters}, + vals[] = {BOOL2ATOM(data.dbg), + BOOL2ATOM(data.sockDbg), + BOOL2ATOM(data.iow), + gcnt}, + info; + unsigned int + numKeys = sizeof(keys) / sizeof(ERL_NIF_TERM), + numVals = sizeof(vals) / sizeof(ERL_NIF_TERM); + + ESOCK_ASSERT( (numKeys == numVals) ); + + if (!MKMA(env, keys, vals, numKeys, &info)) + return enif_make_badarg(env); + + return info; + } } } @@ -3446,7 +3529,6 @@ ERL_NIF_TERM esock_global_info(ErlNifEnv* env) * readers: The number of current and waiting readers * writers: The number of current and waiting writers * acceptors: The number of current and waiting acceptors - * (remote: (socket) Address of the peer (if connected)) */ static ERL_NIF_TERM esock_socket_info(ErlNifEnv* env, @@ -3461,12 +3543,6 @@ ERL_NIF_TERM esock_socket_info(ErlNifEnv* env, ERL_NIF_TERM readers = esock_socket_info_readers(env, descP); ERL_NIF_TERM writers = esock_socket_info_writers(env, descP); ERL_NIF_TERM acceptors = esock_socket_info_acceptors(env, descP); - ERL_NIF_TERM remote; - BOOLEAN_T isConnected = (descP->writeState & ESOCK_STATE_CONNECTED) != 0; - if ((! isConnected) || - esock_encode_sockaddr(env, &descP->remote, - descP->addrLen, &remote) != NULL) - remote = esock_atom_undefined; // not used { ERL_NIF_TERM keys[] @@ -3478,8 +3554,7 @@ ERL_NIF_TERM esock_socket_info(ErlNifEnv* env, atom_counters, atom_num_readers, atom_num_writers, - atom_num_acceptors, - atom_remote}; + atom_num_acceptors}; ERL_NIF_TERM vals[] = {domain, type, @@ -3489,8 +3564,7 @@ ERL_NIF_TERM esock_socket_info(ErlNifEnv* env, counters, readers, writers, - acceptors, - remote}; + acceptors}; ERL_NIF_TERM info; unsigned int numKeys = sizeof(keys) / sizeof(ERL_NIF_TERM); unsigned int numVals = sizeof(vals) / sizeof(ERL_NIF_TERM); @@ -3502,8 +3576,7 @@ ERL_NIF_TERM esock_socket_info(ErlNifEnv* env, ESOCK_ASSERT( (numKeys == numVals) ); - if (!MKMA(env, keys, vals, - isConnected ? numKeys : numKeys - 1, &info)) + if (!MKMA(env, keys, vals, numKeys, &info)) return enif_make_badarg(env); return info; @@ -3612,38 +3685,54 @@ static ERL_NIF_TERM esock_socket_info_counters(ErlNifEnv* env, ESockDescriptor* descP) { + ERL_NIF_TERM keys[] = {atom_read_byte, + atom_read_fails, + atom_read_pkg, + atom_read_pkg_max, + atom_read_tries, + atom_read_waits, + atom_write_byte, + atom_write_fails, + atom_write_pkg, + atom_write_pkg_max, + atom_write_tries, + atom_write_waits, + atom_acc_success, + atom_acc_fails, + atom_acc_tries, + atom_acc_waits}; + unsigned int numKeys = sizeof(keys) / sizeof(ERL_NIF_TERM); + ERL_NIF_TERM vals[] = {MKUI(env, descP->readByteCnt), + MKUI(env, descP->readFails), + MKUI(env, descP->readPkgCnt), + MKUI(env, descP->readPkgMax), + MKUI(env, descP->readTries), + MKUI(env, descP->readWaits), + MKUI(env, descP->writeByteCnt), + MKUI(env, descP->writeFails), + MKUI(env, descP->writePkgCnt), + MKUI(env, descP->writePkgMax), + MKUI(env, descP->writeTries), + MKUI(env, descP->writeWaits), + MKUI(env, descP->accSuccess), + MKUI(env, descP->accFails), + MKUI(env, descP->accTries), + MKUI(env, descP->accWaits)}; + unsigned int numVals = sizeof(vals) / sizeof(ERL_NIF_TERM); ERL_NIF_TERM info; - ERL_NIF_TERM readByteCnt = MKCT(env, atom_read_byte, descP->readByteCnt); - ERL_NIF_TERM readFails = MKCT(env, atom_read_fails, descP->readFails); - ERL_NIF_TERM readPkgCnt = MKCT(env, atom_read_pkg, descP->readPkgCnt); - ERL_NIF_TERM readPkgMax = MKCT(env, atom_read_pkg_max, descP->readPkgMax); - ERL_NIF_TERM readTries = MKCT(env, atom_read_tries, descP->readTries); - ERL_NIF_TERM readWaits = MKCT(env, atom_read_waits, descP->readWaits); - ERL_NIF_TERM writeByteCnt = MKCT(env, atom_write_byte, descP->writeByteCnt); - ERL_NIF_TERM writeFails = MKCT(env, atom_write_fails, descP->writeFails); - ERL_NIF_TERM writePkgCnt = MKCT(env, atom_write_pkg, descP->writePkgCnt); - ERL_NIF_TERM writePkgMax = MKCT(env, atom_write_pkg_max, descP->writePkgMax); - ERL_NIF_TERM writeTries = MKCT(env, atom_write_tries, descP->writeTries); - ERL_NIF_TERM writeWaits = MKCT(env, atom_write_waits, descP->writeWaits); - - ERL_NIF_TERM accSuccess = MKCT(env, atom_acc_success, descP->accSuccess); - ERL_NIF_TERM accFails = MKCT(env, atom_acc_fails, descP->accFails); - ERL_NIF_TERM accTries = MKCT(env, atom_acc_tries, descP->accTries); - ERL_NIF_TERM accWaits = MKCT(env, atom_acc_waits, descP->accWaits); - ERL_NIF_TERM acnt[] = - {readByteCnt, readFails, readPkgCnt, readPkgMax, - readTries, readWaits, - writeByteCnt, writeFails, writePkgCnt, writePkgMax, - writeTries, writeWaits, - accSuccess, accFails, accTries, accWaits}; - unsigned int lenACnt = sizeof(acnt) / sizeof(ERL_NIF_TERM); - - info = MKLA(env, acnt, lenACnt); SSDBG( descP, ("SOCKET", "esock_socket_info_counters -> " - "\r\n lenACnt: %d" - "\r\n info: %T" - "\r\n", lenACnt, info) ); + "\r\n numKeys: %d" + "\r\n numVals: %d" + "\r\n", numKeys, numVals) ); + + ESOCK_ASSERT( (numKeys == numVals) ); + + if (!MKMA(env, keys, vals, numKeys, &info)) { + SSDBG( descP, ("SOCKET", "esock_socket_info_counters -> " + "failed creating counters map\r\n") ); + return enif_make_badarg(env); + } SSDBG( descP, ("SOCKET", "esock_socket_info_counters -> done with" "\r\n info: %T" @@ -3882,101 +3971,141 @@ ERL_NIF_TERM nif_supports(ErlNifEnv* env, #if defined(__WIN32__) return enif_raise_exception(env, MKA(env, "notsup")); #else - int key; + int key2; SGDBG( ("SOCKET", "nif_supports -> entry with %d args\r\n", argc) ); /* Extract arguments and perform preliminary validation */ - if ((argc != 1) || - !GET_INT(env, argv[0], &key)) { + if (argc == 0) + return esock_supports_0(env); + + if (argc == 1) + return esock_supports_1(env, argv[0]); + + if (! GET_INT(env, argv[1], &key2)) return enif_make_badarg(env); - } - return esock_supports(env, key); + if (argc == 2) + return esock_supports_2(env, argv[0], key2); + + return enif_make_badarg(env); #endif } - - -/* esock_supports - what features do we support +/* esock_supports - what features do we support? * - * This is to prove information about what features actually + * This gives information about what features actually * work on the current platform. */ #if !defined(__WIN32__) + static -ERL_NIF_TERM esock_supports(ErlNifEnv* env, int key) +ERL_NIF_TERM esock_supports_0(ErlNifEnv* env) { - ERL_NIF_TERM result; + SocketTArray opts = TARRAY_CREATE(8); + ERL_NIF_TERM is_supported, opts_list; - SGDBG( ("SOCKET", "esock_supports -> entry with 0x%lX\r\n", key) ); + SGDBG( ("SOCKET", "esock_supports_0 -> entry\r\n") ); - switch (key) { - case ESOCK_SUPPORTS_OPTIONS: - result = esock_supports_options(env); - break; +#if defined(HAVE_SCTP) + is_supported = esock_atom_true; +#else + is_supported = esock_atom_false; +#endif + TARRAY_ADD(opts, MKT2(env, esock_atom_sctp, is_supported)); + + /* Is this (test) really sufficient for testing if we support IPv6? */ +#if defined(HAVE_IPV6) + is_supported = esock_atom_true; +#else + is_supported = esock_atom_false; +#endif + TARRAY_ADD(opts, MKT2(env, esock_atom_ipv6, is_supported)); - case ESOCK_SUPPORTS_SCTP: - result = esock_supports_sctp(env); - break; +#if defined(AF_LOCAL) + is_supported = esock_atom_true; +#else + is_supported = esock_atom_false; +#endif + TARRAY_ADD(opts, MKT2(env, esock_atom_local, is_supported)); - case ESOCK_SUPPORTS_IPV6: - result = esock_supports_ipv6(env); - break; +#if defined(HAVE_SETNS) + is_supported = esock_atom_true; +#else + is_supported = esock_atom_false; +#endif + TARRAY_ADD(opts, MKT2(env, atom_netns, is_supported)); - case ESOCK_SUPPORTS_LOCAL: - result = esock_supports_local(env); - break; + TARRAY_TOLIST(opts, env, &opts_list); + return opts_list; +} - case ESOCK_SUPPORTS_NETNS: - result = esock_supports_netns(env); - break; +static +ERL_NIF_TERM esock_supports_1(ErlNifEnv* env, ERL_NIF_TERM key) +{ + ERL_NIF_TERM result; - case ESOCK_SUPPORTS_SEND_FLAGS: - result = esock_supports_send_flags(env); - break; + SGDBG( ("SOCKET", + "esock_supports_2 -> entry" + "\r\n key: %T" + "\r\n", key) ); - case ESOCK_SUPPORTS_RECV_FLAGS: + if (COMPARE(key, atom_send_flags) == 0) + result = esock_supports_send_flags(env); + else if (COMPARE(key, atom_recv_flags) == 0) result = esock_supports_recv_flags(env); - break; - - default: - result = esock_atom_false; - break; - } + else + result = MKEL(env); return result; } -#endif - -#if !defined(__WIN32__) static -ERL_NIF_TERM esock_supports_options(ErlNifEnv* env) -{ - ERL_NIF_TERM sockOpts = esock_supports_options_socket(env); - ERL_NIF_TERM sockOptsT = MKT2(env, esock_atom_socket, sockOpts); - ERL_NIF_TERM ipOpts = esock_supports_options_ip(env); - ERL_NIF_TERM ipOptsT = MKT2(env, esock_atom_ip, ipOpts); - ERL_NIF_TERM ipv6Opts = esock_supports_options_ipv6(env); - ERL_NIF_TERM ipv6OptsT = MKT2(env, esock_atom_ipv6, ipv6Opts); - ERL_NIF_TERM tcpOpts = esock_supports_options_tcp(env); - ERL_NIF_TERM tcpOptsT = MKT2(env, esock_atom_tcp, tcpOpts); - ERL_NIF_TERM udpOpts = esock_supports_options_udp(env); - ERL_NIF_TERM udpOptsT = MKT2(env, esock_atom_udp, udpOpts); - ERL_NIF_TERM sctpOpts = esock_supports_options_sctp(env); - ERL_NIF_TERM sctpOptsT = MKT2(env, esock_atom_sctp, sctpOpts); - ERL_NIF_TERM optsA[] = {sockOptsT, - ipOptsT, ipv6OptsT, - tcpOptsT, udpOptsT, sctpOptsT}; - unsigned int lenOptsA = sizeof(optsA) / sizeof(ERL_NIF_TERM); - ERL_NIF_TERM optsL = MKLA(env, optsA, lenOptsA); +ERL_NIF_TERM esock_supports_2(ErlNifEnv* env, ERL_NIF_TERM key1, int key2) +{ + ERL_NIF_TERM result; - return optsL; + SGDBG( ("SOCKET", + "esock_supports_2 -> entry" + "\r\n key1: %%T" + "\r\n key2: %d" + "\r\n", key1, key2) ); + + if (COMPARE(key1, atom_options) == 0) { + + switch (key2) { + case ESOCK_OPT_LEVEL_SOCKET: + result = esock_supports_options_socket(env); + break; + case ESOCK_OPT_LEVEL_IP: + result = esock_supports_options_ip(env); + break; + case ESOCK_OPT_LEVEL_IPV6: + result = esock_supports_options_ipv6(env); + break; + case ESOCK_OPT_LEVEL_TCP: + result = esock_supports_options_tcp(env); + break; + case ESOCK_OPT_LEVEL_UDP: + result = esock_supports_options_udp(env); + break; + case ESOCK_OPT_LEVEL_SCTP: + result = esock_supports_options_sctp(env); + break; + default: + result = MKEL(env); + break; + } + + } else { + result = MKEL(env); + } + + return result; } -#endif +#endif #if !defined(__WIN32__) @@ -4163,7 +4292,7 @@ ERL_NIF_TERM esock_supports_options_socket(ErlNifEnv* env) /* *** ESOCK_OPT_SOCK_RCVTIMEO => SO_RCVTIMEO *** */ -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) tmp = MKT2(env, esock_atom_rcvtimeo, esock_atom_true); #else tmp = MKT2(env, esock_atom_rcvtimeo, esock_atom_false); @@ -4223,7 +4352,7 @@ ERL_NIF_TERM esock_supports_options_socket(ErlNifEnv* env) /* *** ESOCK_OPT_SOCK_SNDTIMEO => SO_SNDTIMEO *** */ -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) tmp = MKT2(env, esock_atom_sndtimeo, esock_atom_true); #else tmp = MKT2(env, esock_atom_sndtimeo, esock_atom_false); @@ -5156,6 +5285,7 @@ ERL_NIF_TERM esock_supports_options_sctp(ErlNifEnv* env) #endif +#if 0 #if !defined(__WIN32__) static @@ -5228,6 +5358,7 @@ ERL_NIF_TERM esock_supports_netns(ErlNifEnv* env) } #endif +#endif // #if 0 #if !defined(__WIN32__) @@ -5432,7 +5563,9 @@ ERL_NIF_TERM nif_open(ErlNifEnv* env, "\r\n eopts: %T" "\r\n", fd, eopts) ); + MLOCK(data.cntMtx); result = esock_open2(env, fd, eopts); + MUNLOCK(data.cntMtx); } break; @@ -5476,8 +5609,9 @@ ERL_NIF_TERM nif_open(ErlNifEnv* env, return esock_make_error(env, esock_atom_einval); } + MLOCK(data.cntMtx); result = esock_open4(env, domain, type, proto, eopts); - + MUNLOCK(data.cntMtx); } break; @@ -5512,7 +5646,7 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, int fd, ERL_NIF_TERM eopts) { - BOOLEAN_T dbg = esock_open2_is_debug(env, eopts); + BOOLEAN_T dbg = esock_open_is_debug(env, eopts, data.sockDbg); ESockDescriptor* descP; ERL_NIF_TERM res, reason; int domain, type, protocol; @@ -5521,7 +5655,7 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, SOCKET sock; HANDLE event; - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open2 -> entry with" "\r\n fd: %d" "\r\n eopts: %T" @@ -5529,46 +5663,46 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, /* * Before we do anything else, we try to retrieve domain, type and protocol - * This information is either present in the eextra map or if not we need + * This information is either present in the eopts map or if not we need * to "get" it from the system (getsockopt). * Note that its not possible to get all of these on all platoforms, - * and in those cases the user *must* provide us with them (eextra). + * and in those cases the user *must* provide us with them (eopts). * * We try the system first (since its more reliable) and if that fails - * we check the eextra map. If neither one works, we *give up*! + * we check the eopts map. If neither one works, we *give up*! */ if (!esock_open_which_domain(fd, &domain)) { - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open2 -> failed get domain from system\r\n") ); if (!esock_open2_get_domain(env, eopts, &domain)) { - reason = MKT2(env, atom_missing, esock_atom_domain); + reason = MKA(env, "epfnosupport"); return esock_make_error(env, reason); } } if (!esock_open_which_type(fd, &type)) { - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open2 -> failed get type from system\r\n") ); if (!esock_open2_get_type(env, eopts, &type)) { - reason = MKT2(env, atom_missing, esock_atom_type); + reason = MKA(env, "esocktnosupport"); return esock_make_error(env, reason); } } if (!esock_open_which_protocol(fd, &protocol)) { - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open2 -> failed get protocol from system\r\n") ); if (!esock_open2_get_protocol(env, eopts, &protocol)) { - reason = MKT2(env, atom_missing, esock_atom_protocol); + reason = MKA(env, "eprotonosupport"); return esock_make_error(env, reason); } } - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open2 -> " "\r\n domain: %d" "\r\n type: %d" @@ -5581,11 +5715,12 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, if (IS_SOCKET_ERROR(sock = dup(fd))) { save_errno = sock_errno(); - SGDBG2( dbg, - ("SOCKET", "esock_open2 -> dup failed: %d\r\n", save_errno) ); + SSDBG2( dbg, + ("SOCKET", + "esock_open2 -> dup failed: %d\r\n", + save_errno) ); - /* reason = {dup, 'errno atom'} */ - reason = MKT2(env, MKA(env, "dup"), MKA(env, erl_errno_id(save_errno))); + reason = MKA(env, erl_errno_id(save_errno)); return esock_make_error(env, reason); } closeOnClose = TRUE; @@ -5611,16 +5746,18 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, descP->closeOnClose = closeOnClose; descP->origFD = fd; + /* Check if we are already connected, if so change state */ { - /* Check if we are already connected, if so change state */ - unsigned int sz = sizeof(descP->remote); - int code = sock_peer(descP->sock, - (struct sockaddr*) &descP->remote, &sz); - if (! IS_SOCKET_ERROR(code)) { - SGDBG2( dbg, ("SOCKET", "esock_open2 -> connected\r\n") ); - descP->writeState = ESOCK_STATE_CONNECTED; + ESockAddress remote; + socklen_t addrLen = sizeof(remote); + sys_memzero((char *) &remote, addrLen); + if (sock_peer(descP->sock, + (struct sockaddr*) &remote, + &addrLen) == 0) { + SSDBG2( dbg, ("SOCKET", "esock_open2 -> connected\r\n") ); + descP->writeState |= ESOCK_STATE_CONNECTED; } else { - SGDBG2( dbg, ("SOCKET", "esock_open2 -> not connected\r\n") ); + SSDBG2( dbg, ("SOCKET", "esock_open2 -> not connected\r\n") ); } } @@ -5640,9 +5777,10 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, env, descP, &descP->ctrlPid, &descP->ctrlMon) != 0) - return esock_make_error(env, atom_exmon); + return esock_make_error(env, atom_exmonitor); + descP->dbg = dbg; inc_socket(domain, type, protocol); /* And finally update the registry. @@ -5650,29 +5788,19 @@ ERL_NIF_TERM esock_open2(ErlNifEnv* env, */ esock_send_reg_add_msg(env, res); - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open2 -> done: %T\r\n", res) ); return esock_make_ok2(env, res); } -/* The eextra map "may" contain a boolean 'debug' key, if not we assume - * the default of FALSE. - */ -static -BOOLEAN_T esock_open2_is_debug(ErlNifEnv* env, ERL_NIF_TERM eextra) -{ - return esock_get_bool_from_map(env, eextra, MKA(env, "debug"), FALSE); -} - - /* The eextra contains a boolean 'dup' key. Defaults to TRUE. */ static BOOLEAN_T esock_open2_todup(ErlNifEnv* env, ERL_NIF_TERM eextra) { - return esock_get_bool_from_map(env, eextra, MKA(env, "dup"), TRUE); + return esock_get_bool_from_map(env, eextra, atom_dup, TRUE); } /* The eextra contains an integer 'domain' key. @@ -5769,7 +5897,7 @@ ERL_NIF_TERM esock_open4(ErlNifEnv* env, int protocol, ERL_NIF_TERM eopts) { - BOOLEAN_T dbg = esock_open2_is_debug(env, eopts); + BOOLEAN_T dbg = esock_open_is_debug(env, eopts, data.sockDbg); ESockDescriptor* descP; ERL_NIF_TERM res; int proto = protocol, save_errno; @@ -5780,7 +5908,7 @@ ERL_NIF_TERM esock_open4(ErlNifEnv* env, int current_ns = 0; #endif - SGDBG2( dbg, + SSDBG2( dbg, ("SOCKET", "esock_open4 -> entry with" "\r\n domain: %d" "\r\n type: %d" @@ -5811,7 +5939,7 @@ ERL_NIF_TERM esock_open4(ErlNifEnv* env, return esock_make_error_errno(env, sock_errno()); } - SGDBG2( dbg, ("SOCKET", "esock_open -> open success: %d\r\n", sock) ); + SSDBG2( dbg, ("SOCKET", "esock_open -> open success: %d\r\n", sock) ); /* NOTE that if the protocol = 0 (default) and the domain is not @@ -5857,7 +5985,7 @@ ERL_NIF_TERM esock_open4(ErlNifEnv* env, return esock_make_error_errno(env, save_errno); } - SGDBG2( dbg, ("SOCKET", "esock_open4 -> event success: %d\r\n", event) ); + SSDBG2( dbg, ("SOCKET", "esock_open4 -> event success: %d\r\n", event) ); SET_NONBLOCKING(sock); @@ -5896,14 +6024,12 @@ ERL_NIF_TERM esock_open4(ErlNifEnv* env, &descP->ctrlPid, &descP->ctrlMon) != 0) { sock_close(sock); - return esock_make_error(env, atom_exmon); + return esock_make_error(env, atom_exmonitor); } - MLOCK(data.cntMtx); descP->dbg = data.sockDbg; inc_socket(domain, type, protocol); - MUNLOCK(data.cntMtx); /* And finally update the registry */ esock_send_reg_add_msg(env, res); @@ -5912,6 +6038,16 @@ ERL_NIF_TERM esock_open4(ErlNifEnv* env, } +/* The eextra map "may" contain a boolean 'debug' key. + */ +static +BOOLEAN_T esock_open_is_debug(ErlNifEnv* env, ERL_NIF_TERM eextra, + BOOLEAN_T dflt) +{ + return esock_get_bool_from_map(env, eextra, MKA(env, "debug"), dflt); +} + + static BOOLEAN_T esock_open_which_domain(SOCKET sock, int* domain) { @@ -6213,37 +6349,54 @@ ERL_NIF_TERM nif_connect(ErlNifEnv* env, return enif_raise_exception(env, MKA(env, "notsup")); #else ESockDescriptor* descP; - ERL_NIF_TERM res, eSockAddr, sockRef; + ERL_NIF_TERM res, sockRef; char* xres; - ESockAddress addr; + ESockAddress addr, *addrP; socklen_t addrLen; SGDBG( ("SOCKET", "nif_connect -> entry with argc: %d\r\n", argc) ); /* Extract arguments and perform preliminary validation */ - sockRef = argv[0]; - if ((argc != 2) || - !ESOCK_GET_RESOURCE(env, sockRef, (void**) &descP)) { + if (argc >= 1) { + sockRef = argv[0]; + if (! ESOCK_GET_RESOURCE(env, sockRef, (void**) &descP)) + return enif_make_badarg(env); + } else { return enif_make_badarg(env); } - eSockAddr = argv[1]; - if ((xres = esock_decode_sockaddr(env, eSockAddr, &addr, &addrLen)) - != NULL) { - return esock_make_error_str(env, xres); - } + if (argc >= 2) { + ERL_NIF_TERM eSockAddr = argv[1]; - MLOCK(descP->writeMtx); + if ((xres = esock_decode_sockaddr(env, eSockAddr, &addr, &addrLen)) + != NULL) { + return esock_make_error_str(env, xres); + } + addrP = &addr; - SSDBG( descP, - ("SOCKET", "nif_connect(%T), {%d,%s,0x%X} ->" - "\r\n SockAddr: %T" - "\r\n", - sockRef, descP->sock, B2S(descP->closing), descP->writeState, - eSockAddr) ); + MLOCK(descP->writeMtx); + + SSDBG( descP, + ("SOCKET", "nif_connect(%T), {%d,%s,0x%X} ->" + "\r\n SockAddr: %T" + "\r\n", + sockRef, descP->sock, B2S(descP->closing), descP->writeState, + eSockAddr) ); + } else { + addrP = NULL; + addrLen = 0; + + MLOCK(descP->writeMtx); - res = esock_connect(env, descP, sockRef, &addr, addrLen); + SSDBG( descP, + ("SOCKET", "nif_connect(%T), {%d,%s,0x%X} ->" + "\r\n", + sockRef, descP->sock, B2S(descP->closing), descP->writeState + ) ); + } + + res = esock_connect(env, descP, sockRef, addrP, addrLen); SSDBG( descP, ("SOCKET", "nif_connect(%T) -> done with" "\r\n res: %T" @@ -6265,7 +6418,7 @@ ERL_NIF_TERM esock_connect(ErlNifEnv* env, ESockAddress* addrP, socklen_t addrLen) { - int code, sres, save_errno; + int sres, save_errno; ErlNifPid self; /* @@ -6284,53 +6437,52 @@ ERL_NIF_TERM esock_connect(ErlNifEnv* env, if (descP->currentWriterP != NULL) return esock_make_error(env, esock_atom_einval); - /* Either this should be the first connect attempt, - * or if it is a subsequent it should be from the same process - */ - if (descP->connectorP != NULL && - COMPARE_PIDS(&self, &descP->connector.pid) != 0) { - return esock_make_error_errno(env, EALREADY); - } + if (descP->connectorP != NULL) { + /* Connect in progress */ - /* connectorP is NULL or self - connect */ + if (COMPARE_PIDS(&self, &descP->connector.pid) != 0) { + /* Other process has connect in progress */ + return esock_make_error_errno(env, EALREADY); + } - code = sock_connect(descP->sock, (struct sockaddr*) addrP, addrLen); - if (IS_SOCKET_ERROR(code)) - save_errno = sock_errno(); - else - save_errno = 0; + /* Finalize after received select message */ - SSDBG( descP, - ("SOCKET", "esock_connect {%d} -> connect result: %d, %d\r\n", - descP->sock, code, save_errno) ); + requestor_release("esock_connect finalize -> connected", + env, descP, descP->connectorP); + descP->connectorP = NULL; - switch (save_errno) { - case 0: /* No error */ - SSDBG( descP, ("SOCKET", "esock_connect {%d} -> connected\r\n", - descP->sock) ); - { - int err; + descP->writeState &= ~ESOCK_STATE_CONNECTING; - sys_memcpy(&descP->remote, addrP, addrLen); - descP->addrLen = addrLen; + if (! verify_is_connected(descP, &save_errno)) { + return esock_make_error_errno(env, save_errno); + } - if (descP->connectorP != NULL) { - requestor_release("esock_connect -> connected", - env, descP, &descP->connector); - descP->connectorP = NULL; - } + descP->writeState |= ESOCK_STATE_CONNECTED; - descP->writeState &= ~ESOCK_STATE_CONNECTING; + return esock_atom_ok; + } - if (! verify_is_connected(descP, &err)) { - return esock_make_error_errno(env, err); - } + /* No connect in progress */ - descP->writeState |= ESOCK_STATE_CONNECTED; + if (addrP == NULL) + return esock_make_error(env, esock_atom_einval); + + /* Initial connect call, with address */ - return esock_atom_ok; - } - break; + if (sock_connect(descP->sock, (struct sockaddr*) addrP, addrLen) == 0) { + /* Success already! */ + SSDBG( descP, ("SOCKET", "esock_connect {%d} -> connected\r\n", + descP->sock) ); + + descP->writeState |= ESOCK_STATE_CONNECTED; + + return esock_atom_ok; + } + + /* Connect returned error */ + save_errno = sock_errno(); + + switch (save_errno) { case ERRNO_BLOCK: /* Winsock2 */ case EINPROGRESS: /* Unix & OSE!! */ @@ -6338,23 +6490,17 @@ ERL_NIF_TERM esock_connect(ErlNifEnv* env, ("SOCKET", "esock_connect {%d} -> would block => select\r\n", descP->sock) ); { - if (descP->connectorP == NULL) { - - /* First time here - initiate connector */ + /* Initiate connector */ - descP->connector.pid = self; - if (MONP("esock_connect -> conn", - env, descP, &self, &descP->connector.mon) != 0) { - - MON_INIT(&descP->connector.mon); - return esock_make_error(env, atom_exmon); - } - descP->connector.env = esock_alloc_env("connector"); - descP->connectorP = &descP->connector; - } else - enif_clear_env(descP->connector.env); + descP->connector.pid = self; + if (MONP("esock_connect -> conn", + env, descP, &self, &descP->connector.mon) != 0) { - /* A retry overwrites with a new ref */ + MON_INIT(&descP->connector.mon); + return esock_make_error(env, atom_exmonitor); + } + descP->connector.env = esock_alloc_env("connector"); + descP->connectorP = &descP->connector; descP->connector.ref = MKREF(descP->connector.env); if ((sres = @@ -6366,157 +6512,28 @@ ERL_NIF_TERM esock_connect(ErlNifEnv* env, requestor_release("esock_connect -> select failed", env, descP, descP->connectorP); descP->connectorP = NULL; - return esock_make_error(env, - MKT2(env, - esock_atom_select_failed, - MKI(env, sres))); + return esock_make_error(env, atom_exselect); } else { - - sys_memcpy(&descP->remote, addrP, addrLen); - descP->addrLen = addrLen; - descP->writeState |= ESOCK_STATE_CONNECTING; - return - esock_make_ok2(env, - CP_TERM(env, descP->connector.ref)); - } - } - break; - - case EISCONN: - SSDBG( descP, - ("SOCKET", "esock_connect {%d} -> *already* connected\r\n", - descP->sock) ); - { - int err; - - if (descP->connectorP != NULL) { - requestor_release("esock_connect -> already connected", - env, descP, descP->connectorP); - descP->connectorP = NULL; + MKT2(env, atom_select, + CP_TERM(env, descP->connector.ref)); } - - descP->writeState &= ~ESOCK_STATE_CONNECTING; - - if (! verify_is_connected(descP, &err)) - return esock_make_error_errno(env, err); - - descP->writeState |= ESOCK_STATE_CONNECTED; - - return esock_atom_ok; } break; default: SSDBG( descP, - ("SOCKET", "esock_connect {%d} -> other error: %d\r\n", + ("SOCKET", "esock_connect {%d} -> error: %d\r\n", descP->sock, save_errno) ); - { - if (descP->connectorP != NULL) { - requestor_release("esock_connect -> already connected", - env, descP, descP->connectorP); - descP->connectorP = NULL; - } - return esock_make_error_errno(env, save_errno); - } - break; - } -} -#endif // if !defined(__WIN32__) - - -/* ---------------------------------------------------------------------- - * nif_finalize_connection - * - * Description: - * Make socket ready for input and output. - * This function is called if we where made to wait when we called the - * nif_connect function (we made a select, and the select message has - * now been received). - * - * Arguments: - * Socket (ref) - Points to the socket descriptor. - */ -static -ERL_NIF_TERM nif_finalize_connection(ErlNifEnv* env, - int argc, - const ERL_NIF_TERM argv[]) -{ - ERL_NIF_TERM ret; - ESockDescriptor* descP; -#if defined(__WIN32__) - return enif_raise_exception(env, MKA(env, "notsup")); -#else - - /* Extract arguments and perform preliminary validation */ - - if ((argc != 1) || - !ESOCK_GET_RESOURCE(env, argv[0], (void**) &descP)) { - return enif_make_badarg(env); - } - - MLOCK(descP->writeMtx); - - SSDBG( descP, - ("SOCKET", "nif_finalize_connection(%T), {%d,%s,0x%X}" - "\r\n", argv[0], - descP->sock, B2S(descP->closing), descP->writeState) ); - - ret = esock_finalize_connection(env, descP); - - SSDBG( descP, ("SOCKET", "nif_finalize_connection(%T) -> done with" - "\r\n ret: %T" - "\r\n", argv[0], ret) ); - - MUNLOCK(descP->writeMtx); + return esock_make_error_errno(env, save_errno); - return ret; -#endif + } // switch(save_errno) } +#endif // if !defined(__WIN32__) -/* *** esock_finalize_connection *** - * Perform the final check to verify a connection. - */ -#if !defined(__WIN32__) -static -ERL_NIF_TERM esock_finalize_connection(ErlNifEnv* env, - ESockDescriptor* descP) -{ - int error; - ErlNifPid self; - - /* Verify that we are in the proper state */ - - if (! IS_OPEN(descP)) - return esock_make_error(env, atom_closed); - - if (descP->connectorP == NULL) - return esock_make_error(env, esock_atom_einval); - - if (enif_self(env, &self) == NULL) - return esock_make_error(env, atom_exself); - - if (COMPARE_PIDS(&self, &descP->connector.pid) != 0) - return esock_make_error(env, esock_atom_einval); - - requestor_release("esock_finalize_connection -> connected", - env, descP, descP->connectorP); - descP->connectorP = NULL; - - descP->writeState &= ~ESOCK_STATE_CONNECTING; - - if (! verify_is_connected(descP, &error)) - return esock_make_error_errno(env, error); - - descP->writeState |= ESOCK_STATE_CONNECTED; - - return esock_atom_ok; -} -#endif - /* *** verify_is_connected *** * Check if a connection has been established. @@ -6531,41 +6548,44 @@ BOOLEAN_T verify_is_connected(ESockDescriptor* descP, int* err) * This *should* work on Windows NT too, but doesn't. * An bug in Winsock 2.0 for Windows NT? * - * See "Unix Netwok Programming", W.R.Stevens, p 412 for a - * discussion about Unix portability and non blocking connect. + * See "Unix Netwok Programming", "The Sockets Networking API", + * W.R.Stevens, Volume 1, third edition, 16.4 Nonblocking 'connect', + * before Interrupted 'connect' (p 412) for a discussion about + * Unix portability and non blocking connect. */ -#ifndef SO_ERROR - - int sz, code; + int error = 0; - sz = sizeof(descP->remote); - sys_memzero((char *) &descP->remote, sz); - code = sock_peer(descP->sock, - (struct sockaddr*) &descP->remote, &sz); +#ifdef SO_ERROR + SOCKLEN_T sz = sizeof(error); - if (IS_SOCKET_ERROR(code)) { - *err = sock_errno(); - return FALSE; + if (IS_SOCKET_ERROR(sock_getopt(descP->sock, SOL_SOCKET, SO_ERROR, + (void *)&error, &sz))) { + // Solaris does it this way according to W.R.Stevens + error = sock_errno(); + } +#elif 1 + char buf[0]; + if (IS_SOCKET_ERROR(read(descP->sock, buf, 0))) { + error = sock_errno(); } - #else + /* This variant probably returns wrong error value + * ENOTCONN instead of the actual connect error + */ + ESockAddress remote; + socklen_t addrLen = sizeof(remote); + sys_memzero((char *) &remote, addrLen); + if (IS_SOCKET_ERROR(sock_peer(descP->sock, + (struct sockaddr*) &remote, &addrLen))) { + error = sock_errno(); + } +#endif - int error = 0; /* Has to be initiated, we check it */ - unsigned int sz = sizeof(error); /* even if we get -1 */ - int code = sock_getopt(descP->sock, - SOL_SOCKET, SO_ERROR, - (void *)&error, &sz); - - if ((code < 0) || error) { + if (error != 0) { *err = error; return FALSE; } - -#endif /* SO_ERROR */ - - *err = 0; - return TRUE; } #endif @@ -6748,20 +6768,15 @@ ERL_NIF_TERM esock_accept(ErlNifEnv* env, return esock_make_error(env, atom_exself); if (descP->currentAcceptorP == NULL) { - ESockAddress remote; - unsigned int len; SOCKET accSock; /* We have no active acceptor (and therefor no acceptors in queue) */ - len = sizeof(remote); - sys_memzero((char *) &remote, len); - SSDBG( descP, ("SOCKET", "esock_accept {%d} -> try accept\r\n", descP->sock) ); - accSock = sock_accept(descP->sock, (struct sockaddr*) &remote, &len); + accSock = sock_accept(descP->sock, NULL, NULL); if (IS_SOCKET_ERROR(accSock)) { int save_errno; @@ -6774,7 +6789,7 @@ ERL_NIF_TERM esock_accept(ErlNifEnv* env, /* We got an incoming connection */ return esock_accept_listening_accept(env, descP, sockRef, - accSock, caller, &remote, len); + accSock, caller); } } else { @@ -6847,7 +6862,7 @@ ERL_NIF_TERM esock_accept_listening_error(ErlNifEnv* env, &descP->currentAcceptor.pid, &descP->currentAcceptor.mon) != 0) { enif_set_pid_undefined(&descP->currentAcceptor.pid); - res = esock_make_error(env, atom_exmon); + res = esock_make_error(env, atom_exmonitor); } else { ESOCK_ASSERT(!descP->currentAcceptor.env); descP->currentAcceptor.env = esock_alloc_env("current acceptor"); @@ -6880,13 +6895,11 @@ ERL_NIF_TERM esock_accept_listening_accept(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM sockRef, SOCKET accSock, - ErlNifPid caller, - ESockAddress* remote, - unsigned int len) + ErlNifPid caller) { ERL_NIF_TERM res; - esock_accept_accepted(env, descP, sockRef, accSock, caller, remote, len, &res); + esock_accept_accepted(env, descP, sockRef, accSock, caller, &res); return res; } @@ -6901,8 +6914,6 @@ ERL_NIF_TERM esock_accept_accepting_current(ErlNifEnv* env, ERL_NIF_TERM sockRef, ERL_NIF_TERM accRef) { - ESockAddress remote; - unsigned int len; SOCKET accSock; int save_errno; ERL_NIF_TERM res; @@ -6912,9 +6923,7 @@ ERL_NIF_TERM esock_accept_accepting_current(ErlNifEnv* env, "esock_accept_accepting_current {%d} -> try accept\r\n", descP->sock) ); - len = sizeof(descP->remote); - sys_memzero((char *) &remote, len); - accSock = sock_accept(descP->sock, (struct sockaddr*) &remote, &len); + accSock = sock_accept(descP->sock, NULL, NULL); if (IS_SOCKET_ERROR(accSock)) { @@ -6925,7 +6934,7 @@ ERL_NIF_TERM esock_accept_accepting_current(ErlNifEnv* env, } else { res = esock_accept_accepting_current_accept(env, descP, sockRef, - accSock, &remote, len); + accSock); } return res; @@ -6940,9 +6949,7 @@ static ERL_NIF_TERM esock_accept_accepting_current_accept(ErlNifEnv* env, ESockDescriptor* descP, ERL_NIF_TERM sockRef, - SOCKET accSock, - ESockAddress* remote, - unsigned int len) + SOCKET accSock) { ERL_NIF_TERM res; @@ -6952,7 +6959,7 @@ ERL_NIF_TERM esock_accept_accepting_current_accept(ErlNifEnv* env, "\r\n", descP->sock) ); if (esock_accept_accepted(env, descP, sockRef, accSock, - descP->currentAcceptor.pid, remote, len, &res)) { + descP->currentAcceptor.pid, &res)) { if (!activate_next_acceptor(env, descP, sockRef)) { @@ -7074,7 +7081,7 @@ ERL_NIF_TERM esock_accept_busy_retry(ErlNifEnv* env, ErlNifPid* pid) { int sres; - ERL_NIF_TERM res, reason; + ERL_NIF_TERM res; if ((sres = esock_select_read(env, descP->sock, descP, pid, sockRef, accRef)) < 0) { @@ -7095,11 +7102,10 @@ ERL_NIF_TERM esock_accept_busy_retry(ErlNifEnv* env, descP->currentAcceptorP = NULL; } - reason = MKT2(env, esock_atom_select_failed, MKI(env, sres)); - res = esock_make_error(env, reason); + res = esock_make_error(env, atom_exselect); } else { descP->readState |= ESOCK_STATE_ACCEPTING; - res = esock_make_error(env, esock_atom_eagain); // OK!! + res = esock_make_error(env, esock_atom_eagain); // OK!! } return res; @@ -7117,8 +7123,6 @@ BOOLEAN_T esock_accept_accepted(ErlNifEnv* env, ERL_NIF_TERM sockRef, SOCKET accSock, ErlNifPid pid, - ESockAddress* remote, - unsigned int len, ERL_NIF_TERM* result) { ESockDescriptor* accDescP; @@ -7171,15 +7175,12 @@ BOOLEAN_T esock_accept_accepted(ErlNifEnv* env, sock_close(accSock); enif_set_pid_undefined(&descP->ctrlPid); MUNLOCK(descP->writeMtx); - *result = esock_make_error(env, atom_exmon); + *result = esock_make_error(env, atom_exmonitor); return FALSE; } - accDescP->remote = *remote; - accDescP->addrLen = len; SET_NONBLOCKING(accDescP->sock); - descP->readState &= ~ESOCK_STATE_ACCEPTING; descP->writeState |= ESOCK_STATE_CONNECTED; MUNLOCK(descP->writeMtx); @@ -8420,7 +8421,7 @@ ERL_NIF_TERM esock_close(ErlNifEnv* env, &descP->closerMon) != 0) { enif_set_pid_undefined(&descP->closerPid); - return esock_make_error(env, atom_exmon); + return esock_make_error(env, atom_exmonitor); } } @@ -8434,8 +8435,6 @@ ERL_NIF_TERM esock_close(ErlNifEnv* env, sres = esock_do_stop(env, descP); if (sres < 0) { // Error - ERL_NIF_TERM reason; - /* Calling esock_select_stop failed in some mysterious way, * we are kind of toasted - we'll leave the socket leaked * with descP->closing == TRUE @@ -8467,8 +8466,7 @@ ERL_NIF_TERM esock_close(ErlNifEnv* env, esock_free_env("esock_close_do - close-env", descP->closeEnv); descP->closeEnv = NULL; descP->closeRef = esock_atom_undefined; - reason = MKT2(env, esock_atom_select_failed, MKI(env, sres)); - return esock_make_error(env, reason); + return esock_make_error(env, atom_exselect); } @@ -8704,18 +8702,21 @@ ERL_NIF_TERM esock_finalize_close(ErlNifEnv* env, int err; ErlNifPid self; - if (IS_CLOSED(descP) || (! IS_CLOSING(descP))) - return esock_make_error(env, atom_enotclosing); + if (IS_CLOSED(descP)) + return esock_make_error(env, atom_closed); + + if (! IS_CLOSING(descP)) + return esock_make_error_errno(env, EALREADY); if (enif_self(env, &self) == NULL) return esock_make_error(env, atom_exself); if (COMPARE_PIDS(&descP->closerPid, &self) != 0) - return esock_make_error(env, atom_enotclosing); + return esock_make_error_errno(env, EALREADY); /* closeEnv should be NULL or else esock_stop() has not been called */ if (descP->closeEnv != NULL) - return esock_make_error(env, atom_enotclosing); + return esock_make_error_errno(env, EALREADY); /* This process is the closer - go ahead and close the socket */ @@ -8740,7 +8741,7 @@ ERL_NIF_TERM esock_finalize_close(ErlNifEnv* env, err = esock_close_socket(env, descP); if (err != 0) { - if (err != ERRNO_BLOCK) { + if (err == ERRNO_BLOCK) { /* Not all data in the buffers where sent, * make sure the caller gets this. */ @@ -8807,9 +8808,6 @@ ERL_NIF_TERM nif_shutdown(ErlNifEnv* env, return enif_make_badarg(env); } - if (! IS_OPEN(descP)) - return esock_make_error(env, atom_closed); - if (!ehow2how(ehow, &how)) return enif_make_badarg(env); @@ -8845,6 +8843,9 @@ ERL_NIF_TERM esock_shutdown(ErlNifEnv* env, ESockDescriptor* descP, int how) { + if (! IS_OPEN(descP)) + return esock_make_error(env, atom_closed); + if (sock_shutdown(descP->sock, how) == 0) return esock_atom_ok; else @@ -8882,10 +8883,8 @@ ERL_NIF_TERM nif_setopt(ErlNifEnv* env, return enif_raise_exception(env, MKA(env, "notsup")); #else ESockDescriptor* descP = NULL; - int eLevel, level = -1; - int eOpt; - ERL_NIF_TERM eIsEncoded; - ERL_NIF_TERM eVal; + int eLevel, level = -1, eOpt; + ERL_NIF_TERM eIsEncoded, eVal; BOOLEAN_T isEncoded, isOTP; SGDBG( ("SOCKET", "nif_setopt -> entry with argc: %d\r\n", argc) ); @@ -8913,11 +8912,10 @@ ERL_NIF_TERM nif_setopt(ErlNifEnv* env, return esock_make_error(env, esock_atom_einval); } - if (! IS_OPEN(descP)) - return esock_make_error(env, atom_closed); - + MLOCK(descP->readMtx); return esock_setopt(env, descP, isEncoded, isOTP, level, eOpt, eVal); - + /* Surprise! MUNLOCK in called function */ + #endif // if defined(__WIN32__) } @@ -8932,20 +8930,21 @@ ERL_NIF_TERM esock_setopt(ErlNifEnv* env, int eOpt, ERL_NIF_TERM eVal) { - ERL_NIF_TERM result; + if (! IS_OPEN(descP)) { + MUNLOCK(descP->readMtx); + return esock_make_error(env, atom_closed); + } if (isOTP) { /* These are not actual socket options, * but options for our implementation. */ - result = esock_setopt_otp(env, descP, eOpt, eVal); - } else if (!isEncoded) { - result = esock_setopt_native(env, descP, level, eOpt, eVal); + return esock_setopt_otp(env, descP, eOpt, eVal); + } else if (! isEncoded) { + return esock_setopt_native(env, descP, level, eOpt, eVal); } else { - result = esock_setopt_level(env, descP, level, eOpt, eVal); + return esock_setopt_level(env, descP, level, eOpt, eVal); } - - return result; } @@ -8962,7 +8961,6 @@ ERL_NIF_TERM esock_setopt_otp(ErlNifEnv* env, switch (eOpt) { case ESOCK_OPT_OTP_DEBUG: - MLOCK(descP->readMtx); MLOCK(descP->writeMtx); result = esock_setopt_otp_debug(env, descP, eVal); MUNLOCK(descP->writeMtx); @@ -8970,7 +8968,6 @@ ERL_NIF_TERM esock_setopt_otp(ErlNifEnv* env, break; case ESOCK_OPT_OTP_IOW: - MLOCK(descP->readMtx); MLOCK(descP->writeMtx); result = esock_setopt_otp_iow(env, descP, eVal); MUNLOCK(descP->writeMtx); @@ -8978,7 +8975,6 @@ ERL_NIF_TERM esock_setopt_otp(ErlNifEnv* env, break; case ESOCK_OPT_OTP_CTRL_PROC: - MLOCK(descP->readMtx); MLOCK(descP->writeMtx); result = esock_setopt_otp_ctrl_proc(env, descP, eVal); MUNLOCK(descP->writeMtx); @@ -8986,31 +8982,32 @@ ERL_NIF_TERM esock_setopt_otp(ErlNifEnv* env, break; case ESOCK_OPT_OTP_RCVBUF: - MLOCK(descP->readMtx); result = esock_setopt_otp_rcvbuf(env, descP, eVal); MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_RCVCTRLBUF: - MLOCK(descP->readMtx); result = esock_setopt_otp_rcvctrlbuf(env, descP, eVal); MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_SNDCTRLBUF: MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); result = esock_setopt_otp_sndctrlbuf(env, descP, eVal); MUNLOCK(descP->writeMtx); break; case ESOCK_OPT_OTP_META: MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); result = esock_setopt_otp_meta(env, descP, eVal); MUNLOCK(descP->writeMtx); break; default: MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); SSDBG( descP, ("SOCKET", "esock_setopt_otp {%d} -> einval with" "\r\n eOpt: %d" @@ -9304,6 +9301,7 @@ ERL_NIF_TERM esock_setopt_native(ErlNifEnv* env, ERL_NIF_TERM result; MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); SSDBG( descP, ("SOCKET", "esock_setopt_native {%d} -> entry" @@ -9348,6 +9346,7 @@ ERL_NIF_TERM esock_setopt_level(ErlNifEnv* env, ERL_NIF_TERM result; MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); SSDBG( descP, ("SOCKET", "esock_setopt_level {%d} -> entry with" @@ -9500,7 +9499,7 @@ ERL_NIF_TERM esock_setopt_lvl_socket(ErlNifEnv* env, break; #endif -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) case ESOCK_OPT_SOCK_RCVTIMEO: result = esock_setopt_lvl_sock_rcvtimeo(env, descP, eVal); break; @@ -9530,7 +9529,7 @@ ERL_NIF_TERM esock_setopt_lvl_socket(ErlNifEnv* env, break; #endif -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) case ESOCK_OPT_SOCK_SNDTIMEO: result = esock_setopt_lvl_sock_sndtimeo(env, descP, eVal); break; @@ -9708,7 +9707,7 @@ ERL_NIF_TERM esock_setopt_lvl_sock_rcvlowat(ErlNifEnv* env, #endif -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_setopt_lvl_sock_rcvtimeo(ErlNifEnv* env, ESockDescriptor* descP, @@ -9763,7 +9762,7 @@ ERL_NIF_TERM esock_setopt_lvl_sock_sndlowat(ErlNifEnv* env, #endif -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_setopt_lvl_sock_sndtimeo(ErlNifEnv* env, ESockDescriptor* descP, @@ -12393,7 +12392,7 @@ ERL_NIF_TERM esock_setopt_str_opt(ErlNifEnv* env, if (GET_STR(env, eVal, val, max) > 0) { int optLen = strlen(val); - int res = socket_setopt(descP->sock, level, opt, &val, optLen); + int res = socket_setopt(descP->sock, level, opt, val, optLen); if (res != 0) result = esock_make_error_errno(env, sock_errno()); @@ -12413,6 +12412,7 @@ ERL_NIF_TERM esock_setopt_str_opt(ErlNifEnv* env, /* esock_setopt_timeval_opt - set an option that has an (timeval) bool value */ +#if (defined(SO_RCVTIMEO) || defined(SO_SNDTIMEO)) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_setopt_timeval_opt(ErlNifEnv* env, ESockDescriptor* descP, @@ -12451,6 +12451,7 @@ ERL_NIF_TERM esock_setopt_timeval_opt(ErlNifEnv* env, return result; } +#endif @@ -12668,10 +12669,9 @@ ERL_NIF_TERM nif_getopt(ErlNifEnv* env, if (!elevel2level(isEncoded, eLevel, &isOTP, &level)) return esock_make_error(env, esock_atom_einval); - if (! IS_OPEN(descP)) - return esock_make_error(env, atom_closed); - + MLOCK(descP->readMtx); return esock_getopt(env, descP, isEncoded, isOTP, level, eOpt); + /* Surprise! MUNLOCK in called function */ #endif // if defined(__WIN32__) } @@ -12698,28 +12698,30 @@ ERL_NIF_TERM esock_getopt(ErlNifEnv* env, "\r\n eOpt: %T" "\r\n", B2S(isEncoded), B2S(isOTP), level, eOpt) ); - if (isOTP) { + if (! IS_OPEN(descP)) { + result = esock_make_error(env, atom_closed); + } else if (isOTP) { /* These are not actual socket options, * but options for our implementation. */ if (GET_INT(env, eOpt, &opt)) - result = esock_getopt_otp(env, descP, opt); + return esock_getopt_otp(env, descP, opt); else result = esock_make_error(env, esock_atom_einval); } else if (!isEncoded) { - result = esock_getopt_native(env, descP, level, eOpt); + return esock_getopt_native(env, descP, level, eOpt); } else { if (GET_INT(env, eOpt, &opt)) - result = esock_getopt_level(env, descP, level, opt); + return esock_getopt_level(env, descP, level, opt); else - result = esock_make_error(env, esock_atom_einval); + return esock_getopt_native(env, descP, level, eOpt); } SSDBG( descP, ("SOCKET", "esock_getopt -> done when" "\r\n result: %T" "\r\n", result) ); - + MUNLOCK(descP->readMtx); return result; } @@ -12741,49 +12743,45 @@ ERL_NIF_TERM esock_getopt_otp(ErlNifEnv* env, switch (eOpt) { case ESOCK_OPT_OTP_DEBUG: - MLOCK(descP->writeMtx); result = esock_getopt_otp_debug(env, descP); - MUNLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_IOW: - MLOCK(descP->writeMtx); result = esock_getopt_otp_iow(env, descP); - MUNLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_CTRL_PROC: - MLOCK(descP->writeMtx); result = esock_getopt_otp_ctrl_proc(env, descP); - MUNLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_RCVBUF: - MLOCK(descP->readMtx); result = esock_getopt_otp_rcvbuf(env, descP); MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_RCVCTRLBUF: - MLOCK(descP->readMtx); result = esock_getopt_otp_rcvctrlbuf(env, descP); MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_SNDCTRLBUF: MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); result = esock_getopt_otp_sndctrlbuf(env, descP); MUNLOCK(descP->writeMtx); break; case ESOCK_OPT_OTP_FD: - MLOCK(descP->readMtx); result = esock_getopt_otp_fd(env, descP); MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_META: MLOCK(descP->writeMtx); + MUNLOCK(descP->readMtx); result = esock_getopt_otp_meta(env, descP); MUNLOCK(descP->writeMtx); break; @@ -12791,22 +12789,25 @@ ERL_NIF_TERM esock_getopt_otp(ErlNifEnv* env, /* *** INTERNAL *** */ case ESOCK_OPT_OTP_DOMAIN: result = esock_getopt_otp_domain(env, descP); + MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_TYPE: result = esock_getopt_otp_type(env, descP); + MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_PROTOCOL: result = esock_getopt_otp_protocol(env, descP); + MUNLOCK(descP->readMtx); break; case ESOCK_OPT_OTP_DTP: result = esock_getopt_otp_dtp(env, descP); + MUNLOCK(descP->readMtx); break; default: - MLOCK(descP->readMtx); SSDBG( descP, ("SOCKET", "esock_getopt_otp {%d} -> einval with" "\r\n eOpt: %d" @@ -13023,7 +13024,7 @@ ERL_NIF_TERM getopt_otp_type(ErlNifEnv* env, int type) result = esock_atom_dgram; break; -#ifdef HAVE_SCTP +#ifdef SOCK_SEQPACKET case SOCK_SEQPACKET: result = esock_atom_seqpacket; break; @@ -13165,8 +13166,6 @@ ERL_NIF_TERM esock_getopt_native(ErlNifEnv* env, Uint16 valueType; SOCKOPTLEN_T valueSz; - MLOCK(descP->readMtx); - SSDBG( descP, ("SOCKET", "esock_getopt_native {%d} -> entry" "\r\n level: %d" @@ -13282,8 +13281,6 @@ ERL_NIF_TERM esock_getopt_level(ErlNifEnv* env, { ERL_NIF_TERM result; - MLOCK(descP->readMtx); - SSDBG( descP, ("SOCKET", "esock_getopt_level {%d} -> entry with" "\r\n level: %d" @@ -13443,7 +13440,7 @@ ERL_NIF_TERM esock_getopt_lvl_socket(ErlNifEnv* env, break; #endif -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) case ESOCK_OPT_SOCK_RCVTIMEO: result = esock_getopt_lvl_sock_rcvtimeo(env, descP); break; @@ -13473,7 +13470,7 @@ ERL_NIF_TERM esock_getopt_lvl_socket(ErlNifEnv* env, break; #endif -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) case ESOCK_OPT_SOCK_SNDTIMEO: result = esock_getopt_lvl_sock_sndtimeo(env, descP); break; @@ -13770,7 +13767,7 @@ ERL_NIF_TERM esock_getopt_lvl_sock_rcvlowat(ErlNifEnv* env, #endif -#if defined(SO_RCVTIMEO) +#if defined(SO_RCVTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_getopt_lvl_sock_rcvtimeo(ErlNifEnv* env, ESockDescriptor* descP) @@ -13820,7 +13817,7 @@ ERL_NIF_TERM esock_getopt_lvl_sock_sndlowat(ErlNifEnv* env, #endif -#if defined(SO_SNDTIMEO) +#if defined(SO_SNDTIMEO) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_getopt_lvl_sock_sndtimeo(ErlNifEnv* env, ESockDescriptor* descP) @@ -13862,7 +13859,7 @@ ERL_NIF_TERM esock_getopt_lvl_sock_type(ErlNifEnv* env, case SOCK_DGRAM: result = esock_make_ok2(env, esock_atom_dgram); break; -#ifdef HAVE_SCTP +#ifdef SOCK_SEQPACKET case SOCK_SEQPACKET: result = esock_make_ok2(env, esock_atom_seqpacket); break; @@ -15191,7 +15188,8 @@ ERL_NIF_TERM esock_getopt_lvl_sctp(ErlNifEnv* env, * association (and not an endpoint) then it will have an * assoc id. But since the sctp support at present is "limited", * we leave it for now. - * What do we do if this is an endpoint? Invalid op? + * What do we do if this is an endpoint? Invalid op? Or just leave + * it for the OS? * * </KOLLA> */ @@ -15437,6 +15435,7 @@ ERL_NIF_TERM esock_getopt_int_opt(ErlNifEnv* env, /* esock_getopt_timeval_opt - get an timeval option */ +#if (defined(SO_RCVTIMEO) || defined(SO_SNDTIMEO)) && defined(ESOCK_USE_RCVSNDTIMEO) static ERL_NIF_TERM esock_getopt_timeval_opt(ErlNifEnv* env, ESockDescriptor* descP, @@ -15465,6 +15464,7 @@ ERL_NIF_TERM esock_getopt_timeval_opt(ErlNifEnv* env, return result; } +#endif @@ -16245,7 +16245,7 @@ BOOLEAN_T send_check_writer(ErlNifEnv* env, if (!writer_search4pid(env, descP, &caller)) *checkResult = writer_push(env, descP, caller, ref); else - *checkResult = esock_make_error(env, atom_exbusy); + *checkResult = esock_make_error_errno(env, EALREADY); SSDBG( descP, ("SOCKET", @@ -16499,7 +16499,7 @@ ERL_NIF_TERM send_check_retry(ErlNifEnv* env, &descP->currentWriter.pid, &descP->currentWriter.mon) != 0) { enif_set_pid_undefined(&descP->currentWriter.pid); - return esock_make_error(env, atom_exmon); + return esock_make_error(env, atom_exmonitor); } else { ESOCK_ASSERT(descP->currentWriter.env == NULL); descP->currentWriter.env = esock_alloc_env("current-writer"); @@ -16596,7 +16596,7 @@ BOOLEAN_T recv_check_reader(ErlNifEnv* env, if (!reader_search4pid(env, descP, &caller)) *checkResult = reader_push(env, descP, caller, ref); else - *checkResult = esock_make_error(env, atom_exbusy); + *checkResult = esock_make_error_errno(env, EALREADY); SSDBG( descP, ("SOCKET", @@ -16637,7 +16637,7 @@ char* recv_init_current_reader(ErlNifEnv* env, &descP->currentReader.pid, &descP->currentReader.mon) != 0) { enif_set_pid_undefined(&descP->currentReader.pid); - return str_exmon; + return str_exmonitor; } else { ESOCK_ASSERT(!descP->currentReader.env); descP->currentReader.env = esock_alloc_env("current-reader"); @@ -17144,7 +17144,7 @@ ERL_NIF_TERM recv_check_retry(ErlNifEnv* env, /* Ouch * Now what? We have copied ref into *its own* environment! */ - reason = MKT2(env, esock_atom_select_failed, MKI(env, sres)); + reason = atom_exselect; } else { reason = esock_atom_eagain; } @@ -19866,7 +19866,7 @@ void dec_socket(int domain, int type, int protocol) cnt_dec(&data.numTypeStreams, 1); else if (type == SOCK_DGRAM) cnt_dec(&data.numTypeDGrams, 1); -#ifdef HAVE_SCTP +#ifdef SOCK_SEQPACKET else if (type == SOCK_SEQPACKET) cnt_dec(&data.numTypeSeqPkgs, 1); #endif @@ -19911,7 +19911,7 @@ void inc_socket(int domain, int type, int protocol) cnt_inc(&data.numTypeStreams, 1); else if (type == SOCK_DGRAM) cnt_inc(&data.numTypeDGrams, 1); -#ifdef HAVE_SCTP +#ifdef SOCK_SEQPACKET else if (type == SOCK_SEQPACKET) cnt_inc(&data.numTypeSeqPkgs, 1); #endif @@ -19992,7 +19992,7 @@ BOOLEAN_T etype2type(int etype, int* type) *type = SOCK_RAW; break; -#ifdef HAVE_SCTP +#ifdef SOCK_SEQPACKET case ESOCK_TYPE_SEQPACKET: *type = SOCK_SEQPACKET; break; @@ -20132,93 +20132,73 @@ BOOLEAN_T esock_open4_get_netns(ErlNifEnv* env, ERL_NIF_TERM opts, char** netns) static BOOLEAN_T esendflags2sendflags(unsigned int eflags, int* flags) { - unsigned int ef; - int tmp = 0; + int tmp = 0; - /* First, check if we have any flags at all */ + /* Optimize for no flags */ if (eflags == 0) { *flags = 0; return TRUE; } - - for (ef = ESOCK_SEND_FLAG_LOW; ef <= ESOCK_SEND_FLAG_HIGH; ef++) { - switch (ef) { - case ESOCK_SEND_FLAG_CONFIRM: - if ((1 << ESOCK_SEND_FLAG_CONFIRM) & eflags) { + /* Check for flags out of range */ + if ((eflags & ~ESOCK_SEND_FLAG_MASK) != 0) { + esock_warning_msg("Use of unknown send flag (0x%lX)\r\n", + eflags); + } + + if ((eflags & ESOCK_SEND_FLAG_CONFIRM) != 0) { #if defined(MSG_CONFIRM) - tmp |= MSG_CONFIRM; + tmp |= MSG_CONFIRM; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_SEND_FLAG_DONTROUTE: - if ((1 << ESOCK_SEND_FLAG_DONTROUTE) & eflags) { + if ((eflags & ESOCK_SEND_FLAG_DONTROUTE) != 0) { #if defined(MSG_DONTROUTE) - tmp |= MSG_DONTROUTE; + tmp |= MSG_DONTROUTE; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_SEND_FLAG_EOR: - if ((1 << ESOCK_SEND_FLAG_EOR) & eflags) { + if ((eflags & ESOCK_SEND_FLAG_EOR) != 0) { #if defined(MSG_EOR) - tmp |= MSG_EOR; + tmp |= MSG_EOR; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_SEND_FLAG_MORE: - if ((1 << ESOCK_SEND_FLAG_MORE) & eflags) { + if ((eflags & ESOCK_SEND_FLAG_MORE) != 0) { #if defined(MSG_MORE) - tmp |= MSG_MORE; + tmp |= MSG_MORE; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_SEND_FLAG_NOSIGNAL: - if ((1 << ESOCK_SEND_FLAG_NOSIGNAL) & eflags) { + if ((eflags & ESOCK_SEND_FLAG_NOSIGNAL) != 0) { #if defined(MSG_NOSIGNAL) - tmp |= MSG_NOSIGNAL; + tmp |= MSG_NOSIGNAL; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_SEND_FLAG_OOB: - if ((1 << ESOCK_SEND_FLAG_OOB) & eflags) { + if ((eflags & ESOCK_SEND_FLAG_OOB) != 0) { #if defined(MSG_OOB) - tmp |= MSG_OOB; + tmp |= MSG_OOB; #else - return FALSE; + return FALSE; #endif - } - break; - - default: - esock_warning_msg("Use of unknown send flag %d (0x%lX)\r\n", - ef, eflags); - return FALSE; - } - } *flags = tmp; - return TRUE; } - /* erecvflags2recvflags - convert internal (erlang) send flags to (proper) * send flags. * @@ -20228,92 +20208,75 @@ BOOLEAN_T esendflags2sendflags(unsigned int eflags, int* flags) static BOOLEAN_T erecvflags2recvflags(unsigned int eflags, int* flags) { - unsigned int ef; - int tmp = 0; + int tmp = 0; + + /* Optimize for no flags */ if (eflags == 0) { *flags = 0; return TRUE; } - for (ef = ESOCK_RECV_FLAG_LOW; ef <= ESOCK_RECV_FLAG_HIGH; ef++) { + if ((eflags & ~ESOCK_RECV_FLAG_MASK) != 0) { + esock_warning_msg("Use of unknown recv flag (0x%lX)\r\n", + eflags); + } - switch (ef) { - case ESOCK_RECV_FLAG_CMSG_CLOEXEC: - if ((1 << ESOCK_RECV_FLAG_CMSG_CLOEXEC) & eflags) { + if ((eflags & ESOCK_RECV_FLAG_CMSG_CLOEXEC) != 0) { #if defined(MSG_CMSG_CLOEXEC) - tmp |= MSG_CMSG_CLOEXEC; + tmp |= MSG_CMSG_CLOEXEC; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_RECV_FLAG_ERRQUEUE: - if ((1 << ESOCK_RECV_FLAG_ERRQUEUE) & eflags) { + if ((eflags & ESOCK_RECV_FLAG_ERRQUEUE) != 0) { #if defined(MSG_ERRQUEUE) - tmp |= MSG_ERRQUEUE; + tmp |= MSG_ERRQUEUE; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_RECV_FLAG_OOB: - if ((1 << ESOCK_RECV_FLAG_OOB) & eflags) { + if ((eflags & ESOCK_RECV_FLAG_OOB) != 0) { #if defined(MSG_OOB) - tmp |= MSG_OOB; + tmp |= MSG_OOB; #else - return FALSE; + return FALSE; #endif - } - break; + } - /* - * <KOLLA> - * - * We need to handle this, because it may effect the read algorithm - * - * </KOLLA> - */ - case ESOCK_RECV_FLAG_PEEK: - if ((1 << ESOCK_RECV_FLAG_PEEK) & eflags) { + /* + * <KOLLA> + * + * We need to handle this, because it may effect the read algorithm + * + * </KOLLA> + */ + if ((eflags & ESOCK_RECV_FLAG_PEEK) != 0) { #if defined(MSG_PEEK) - tmp |= MSG_PEEK; + tmp |= MSG_PEEK; #else - return FALSE; + return FALSE; #endif - } - break; + } - case ESOCK_RECV_FLAG_TRUNC: - if ((1 << ESOCK_RECV_FLAG_TRUNC) & eflags) { + if ((eflags & ESOCK_RECV_FLAG_TRUNC) != 0) { #if defined(MSG_TRUNC) - tmp |= MSG_TRUNC; + tmp |= MSG_TRUNC; #else - return FALSE; + return FALSE; #endif - } - break; - - default: - esock_warning_msg("Use of unknown recv flag %d (0x%lX)\r\n", - ef, eflags); - return FALSE; - } - } *flags = tmp; - return TRUE; } -/* eproto2proto - convert internal (erlang) protocol to (proper) protocol - * - * Note that only a subset is supported. +/* ehow2how - convert internal (erlang) "shutdown how" to + * (proper) "shutdown how" */ static BOOLEAN_T ehow2how(unsigned int ehow, int* how) @@ -20669,7 +20632,7 @@ ERL_NIF_TERM mk_select_msg(ErlNifEnv* env, * * {'$socket', Socket, Tag, Info} * - * Socket :: socket() (#socket{}) + * Socket :: socket:socket() * Tag :: atom() * Info :: term() * @@ -20688,15 +20651,15 @@ ERL_NIF_TERM mk_socket_msg(ErlNifEnv* env, /* *** mk_socket *** * - * Simple utility function that construct the socket resord: + * Simple utility function that construct the socket tuple: * - * #socket{ref = SockRef} => {socket, SockRef :: reference()} + * socket:socket() :: {'$socket', SockRef :: reference()} */ static ERL_NIF_TERM mk_socket(ErlNifEnv* env, ERL_NIF_TERM sockRef) { - return MKT2(env, esock_atom_socket, sockRef); + return MKT2(env, esock_atom_socket_tag, sockRef); } #endif // #if defined(__WIN32__) @@ -20955,7 +20918,7 @@ REQ_SEARCH4PID_FUNCS if (MONP("reader_push -> " #F " request", \ env, descP, &pid, &reqP->mon) != 0) { \ FREE(e); \ - return esock_make_error(env, atom_exmon); \ + return esock_make_error(env, atom_exmonitor); \ } \ reqP->env = esock_alloc_env(#F "_push"); \ reqP->ref = CP_TERM(reqP->env, ref); \ @@ -21399,7 +21362,7 @@ void esock_dtor(ErlNifEnv* env, void* obj) * */ static -void esock_stop(ErlNifEnv* env, void* obj, int fd, int is_direct_call) +void esock_stop(ErlNifEnv* env, void* obj, ErlNifEvent fd, int is_direct_call) { #if !defined(__WIN32__) ESockDescriptor* descP = (ESockDescriptor*) obj; @@ -21952,13 +21915,16 @@ ErlNifFunc esock_funcs[] = // Some utility and support functions {"nif_info", 0, nif_info, 0}, {"nif_info", 1, nif_info, 0}, + {"nif_supports", 0, nif_supports, 0}, {"nif_supports", 1, nif_supports, 0}, + {"nif_supports", 2, nif_supports, 0}, {"nif_command", 1, nif_command, 0}, // The proper "socket" interface {"nif_open", 2, nif_open, 0}, {"nif_open", 4, nif_open, 0}, {"nif_bind", 2, nif_bind, 0}, + {"nif_connect", 1, nif_connect, 0}, {"nif_connect", 2, nif_connect, 0}, {"nif_listen", 2, nif_listen, 0}, {"nif_accept", 2, nif_accept, 0}, @@ -21978,10 +21944,9 @@ ErlNifFunc esock_funcs[] = /* Misc utility functions */ /* "Extra" functions to "complete" the socket interface. - * For instance, the function nif_finalize_connection - * is called after the connect *select* has "completed". + * For instance, the function nif_finalize_close + * is called after the close *select* has "completed". */ - {"nif_finalize_connection", 1, nif_finalize_connection, 0}, {"nif_cancel", 3, nif_cancel, 0}, {"nif_finalize_close", 1, nif_finalize_close, ERL_NIF_DIRTY_JOB_IO_BOUND} }; @@ -22105,4 +22070,4 @@ int on_load(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) * NULL: THIS IS NOT USED * unload: NULL (not used) */ -ERL_NIF_INIT(socket, esock_funcs, on_load, NULL, NULL, NULL) +ERL_NIF_INIT(prim_socket, esock_funcs, on_load, NULL, NULL, NULL) diff --git a/erts/emulator/nifs/common/socket_int.h b/erts/emulator/nifs/common/socket_int.h index 0095ddbe08..c224f82c47 100644 --- a/erts/emulator/nifs/common/socket_int.h +++ b/erts/emulator/nifs/common/socket_int.h @@ -91,6 +91,9 @@ typedef union { struct sockaddr_ll ll; #endif + /* Max size sockaddr on system */ + struct sockaddr_storage ss; + } ESockAddress; @@ -113,7 +116,7 @@ typedef unsigned int BOOLEAN_T; * "Global" atoms (esock_atom_...) * * Note that when an (global) atom is added here, it must also be added - * in the socket_nif.c file! + * in the prim_socket_nif.c file! */ #define GLOBAL_ATOM_DEFS \ diff --git a/erts/emulator/nifs/common/socket_util.c b/erts/emulator/nifs/common/socket_util.c index 4741f2eb69..7c9b38ddec 100644 --- a/erts/emulator/nifs/common/socket_util.c +++ b/erts/emulator/nifs/common/socket_util.c @@ -440,9 +440,9 @@ char* esock_encode_sockaddr(ErlNifEnv* env, UDBG( ("SUTIL", "esock_encode_sockaddr -> entry with" "\r\n family: %d" "\r\n addrLen: %d" - "\r\n", sockAddrP->sa.sa_family, addrLen) ); + "\r\n", sockAddrP->ss.ss_family, addrLen) ); - switch (sockAddrP->sa.sa_family) { + switch (sockAddrP->ss.ss_family) { case AF_INET: xres = esock_encode_sockaddr_in4(env, &sockAddrP->in4, addrLen, eSockAddr); break; @@ -1353,7 +1353,7 @@ char* esock_decode_type(ErlNifEnv* env, } else if (COMPARE(esock_atom_raw, eType) == 0) { *type = SOCK_RAW; -#if defined(HAVE_SCTP) +#ifdef SOCK_SEQPACKET } else if (COMPARE(esock_atom_seqpacket, eType) == 0) { *type = SOCK_SEQPACKET; #endif @@ -1398,7 +1398,7 @@ char* esock_encode_type(ErlNifEnv* env, *eType = esock_atom_raw; break; -#if defined(HAVE_SCTP) +#ifdef SOCK_SEQPACKET case SOCK_SEQPACKET: *eType = esock_atom_seqpacket; break; diff --git a/erts/emulator/pcre/LICENCE b/erts/emulator/pcre/LICENCE index 760a6666b6..57a544814c 100644 --- a/erts/emulator/pcre/LICENCE +++ b/erts/emulator/pcre/LICENCE @@ -25,7 +25,7 @@ Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. -Copyright (c) 1997-2019 University of Cambridge +Copyright (c) 1997-2020 University of Cambridge All rights reserved. @@ -36,7 +36,7 @@ Written by: Zoltan Herczeg Email local part: hzmester Email domain: freemail.hu -Copyright(c) 2010-2019 Zoltan Herczeg +Copyright(c) 2010-2020 Zoltan Herczeg All rights reserved. @@ -47,7 +47,7 @@ Written by: Zoltan Herczeg Email local part: hzmester Email domain: freemail.hu -Copyright(c) 2009-2019 Zoltan Herczeg +Copyright(c) 2009-2020 Zoltan Herczeg All rights reserved. diff --git a/erts/emulator/pcre/README.pcre_update.md b/erts/emulator/pcre/README.pcre_update.md index 5df1e15bde..7703db614d 100644 --- a/erts/emulator/pcre/README.pcre_update.md +++ b/erts/emulator/pcre/README.pcre_update.md @@ -244,7 +244,10 @@ To begin with you will need a default table for Latin-1 characters, so: Compare it to the pcre\_latin\_1\_table.c in the old version, they should not differ in any significant way. If they do, it might be -that you do not have the sv_SE locale installed on your machine. +that you do not have the `sv_SE` locale installed on your machine. + +You can test whether it's installed with `locale -a | grep sv_SE$`, and +install with `sudo locale-gen sv_SE && sudo update-locale` if needed. A good starting point is then to try to find all files in the new version of the library that have (probably) the same names as the diff --git a/erts/emulator/pcre/local_config.h b/erts/emulator/pcre/local_config.h index c3b4dab586..178c4d4281 100644 --- a/erts/emulator/pcre/local_config.h +++ b/erts/emulator/pcre/local_config.h @@ -86,4 +86,4 @@ #define SUPPORT_UTF /* Version number of package */ -#define VERSION "8.42" +#define VERSION "8.44" diff --git a/erts/emulator/pcre/pcre-8.43.tar.bz2 b/erts/emulator/pcre/pcre-8.43.tar.bz2 Binary files differdeleted file mode 100644 index e20c601f71..0000000000 --- a/erts/emulator/pcre/pcre-8.43.tar.bz2 +++ /dev/null diff --git a/erts/emulator/pcre/pcre-8.44.tar.bz2 b/erts/emulator/pcre/pcre-8.44.tar.bz2 Binary files differnew file mode 100644 index 0000000000..dc978b77a7 --- /dev/null +++ b/erts/emulator/pcre/pcre-8.44.tar.bz2 diff --git a/erts/emulator/pcre/pcre.h b/erts/emulator/pcre/pcre.h index 49c9fc6dc8..c33c93b720 100644 --- a/erts/emulator/pcre/pcre.h +++ b/erts/emulator/pcre/pcre.h @@ -43,9 +43,9 @@ POSSIBILITY OF SUCH DAMAGE. /* The current PCRE version information. */ #define PCRE_MAJOR 8 -#define PCRE_MINOR 43 +#define PCRE_MINOR 44 #define PCRE_PRERELEASE -#define PCRE_DATE 2019-02-23 +#define PCRE_DATE 2020-02-12 /* When an application links to a PCRE DLL in Windows, the symbols that are imported have to be identified as such. When building PCRE, the appropriate diff --git a/erts/emulator/pcre/pcre_compile.c b/erts/emulator/pcre/pcre_compile.c index 6ac222b27e..80b966869a 100644 --- a/erts/emulator/pcre/pcre_compile.c +++ b/erts/emulator/pcre/pcre_compile.c @@ -6,7 +6,7 @@ and semantics are as close as possible to those of the Perl 5 language. Written by Philip Hazel - Copyright (c) 1997-2018 University of Cambridge + Copyright (c) 1997-2020 University of Cambridge ----------------------------------------------------------------------------- Redistribution and use in source and binary forms, with or without @@ -69,7 +69,7 @@ COMPILE_PCREx macro will already be appropriately set. */ /* Macro for setting individual bits in class bitmaps. */ -#define SETBIT(a,b) a[(b)/8] |= (1 << ((b)&7)) +#define SETBIT(a,b) a[(b)/8] |= (1U << ((b)&7)) /* Maximum length value to check against when making sure that the integer that holds the compiled pattern length does not overflow. We make it a bit less than @@ -130,8 +130,8 @@ overrun before it actually does run off the end of the data block. */ /* Private flags added to firstchar and reqchar. */ -#define REQ_CASELESS (1 << 0) /* Indicates caselessness */ -#define REQ_VARY (1 << 1) /* Reqchar followed non-literal item */ +#define REQ_CASELESS (1U << 0) /* Indicates caselessness */ +#define REQ_VARY (1U << 1) /* Reqchar followed non-literal item */ /* Negative values for the firstchar and reqchar flags */ #define REQ_UNSET (-2) #define REQ_NONE (-1) @@ -3612,7 +3612,7 @@ for(;;) if (chr > 255) break; class_bitset = (pcre_uint8 *) ((list_ptr == list ? code : base_end) - list_ptr[2]); - if ((class_bitset[chr >> 3] & (1 << (chr & 7))) != 0) return FALSE; + if ((class_bitset[chr >> 3] & (1U << (chr & 7))) != 0) return FALSE; break; #if defined SUPPORT_UTF || !defined COMPILE_PCRE8 @@ -7131,17 +7131,19 @@ for (;; ptr++) int n = 0; ptr++; while(IS_DIGIT(*ptr)) + { n = n * 10 + *ptr++ - CHAR_0; + if (n > 255) + { + *errorcodeptr = ERR38; + goto FAILED; + } + } if (*ptr != CHAR_RIGHT_PARENTHESIS) { *errorcodeptr = ERR39; goto FAILED; } - if (n > 255) - { - *errorcodeptr = ERR38; - goto FAILED; - } *code++ = n; PUT(code, 0, (int)(ptr - cd->start_pattern + 1)); /* Pattern offset */ PUT(code, LINK_SIZE, 0); /* Default length */ @@ -7457,7 +7459,7 @@ for (;; ptr++) { open_capitem *oc; recno = GET2(slot, 0); - cd->backref_map |= (recno < 32)? (1 << recno) : 1; + cd->backref_map |= (recno < 32)? (1U << recno) : 1; if (recno > cd->top_backref) cd->top_backref = recno; /* Check to see if this back reference is recursive, that it, it @@ -8068,7 +8070,7 @@ for (;; ptr++) item_hwm_offset = cd->hwm - cd->start_workspace; *code++ = ((options & PCRE_CASELESS) != 0)? OP_REFI : OP_REF; PUT2INC(code, 0, recno); - cd->backref_map |= (recno < 32)? (1 << recno) : 1; + cd->backref_map |= (recno < 32)? (1U << recno) : 1; if (recno > cd->top_backref) cd->top_backref = recno; /* Check to see if this back reference is recursive, that it, it @@ -8681,7 +8683,7 @@ do { op == OP_SCBRA || op == OP_SCBRAPOS) { int n = GET2(scode, 1+LINK_SIZE); - int new_map = bracket_map | ((n < 32)? (1 << n) : 1); + int new_map = bracket_map | ((n < 32)? (1U << n) : 1); if (!is_anchored(scode, new_map, cd, atomcount)) return FALSE; } @@ -8809,7 +8811,7 @@ do { op == OP_SCBRA || op == OP_SCBRAPOS) { int n = GET2(scode, 1+LINK_SIZE); - int new_map = bracket_map | ((n < 32)? (1 << n) : 1); + int new_map = bracket_map | ((n < 32)? (1U << n) : 1); if (!is_startline(scode, new_map, cd, atomcount, inassert)) return FALSE; } @@ -9033,7 +9035,7 @@ Returns: pointer to compiled data block, or NULL on error, #if defined(ERLANG_INTEGRATION) PCRE_EXP_DEFN pcre * PCRE_CALL_CONVENTION erts_pcre_compile(const char *pattern, int options, const char **errorptr, - int *erroroffset, const unsigned char *tables) + int *erroroffset, const unsigned char *tables) #else PCRE_EXP_DEFN pcre * PCRE_CALL_CONVENTION pcre_compile(const char *pattern, int options, const char **errorptr, @@ -9051,8 +9053,8 @@ pcre32_compile(PCRE_SPTR32 pattern, int options, const char **errorptr, { #if defined COMPILE_PCRE8 #if defined(ERLANG_INTEGRATION) -return erts_pcre_compile2(pattern, options, NULL, errorptr, - erroroffset, tables); +return erts_pcre_compile2(pattern, options, NULL, errorptr, + erroroffset, tables); #else return pcre_compile2(pattern, options, NULL, errorptr, erroroffset, tables); #endif diff --git a/erts/emulator/pcre/pcre_jit_compile.c b/erts/emulator/pcre/pcre_jit_compile.c index 2d2288f81e..44288b7ec2 100644 --- a/erts/emulator/pcre/pcre_jit_compile.c +++ b/erts/emulator/pcre/pcre_jit_compile.c @@ -3938,10 +3938,10 @@ static sljit_s32 character_to_int32(pcre_uchar chr) sljit_s32 value = (sljit_s32)chr; #if defined COMPILE_PCRE8 #define SSE2_COMPARE_TYPE_INDEX 0 -return (value << 24) | (value << 16) | (value << 8) | value; +return ((unsigned int)value << 24) | ((unsigned int)value << 16) | ((unsigned int)value << 8) | (unsigned int)value; #elif defined COMPILE_PCRE16 #define SSE2_COMPARE_TYPE_INDEX 1 -return (value << 16) | value; +return ((unsigned int)value << 16) | value; #elif defined COMPILE_PCRE32 #define SSE2_COMPARE_TYPE_INDEX 2 return value; @@ -8507,7 +8507,7 @@ if (opcode == OP_ONCE) /* We temporarily encode the needs_control_head in the lowest bit. Note: on the target architectures of SLJIT the ((x << 1) >> 1) returns the same value for small signed numbers (including negative numbers). */ - BACKTRACK_AS(bracket_backtrack)->u.framesize = (BACKTRACK_AS(bracket_backtrack)->u.framesize << 1) | (needs_control_head ? 1 : 0); + BACKTRACK_AS(bracket_backtrack)->u.framesize = ((unsigned int)BACKTRACK_AS(bracket_backtrack)->u.framesize << 1) | (needs_control_head ? 1 : 0); } return cc + repeat_length; } diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c index bd9f6d33de..6305590690 100644 --- a/erts/emulator/sys/common/erl_check_io.c +++ b/erts/emulator/sys/common/erl_check_io.c @@ -184,11 +184,14 @@ int ERTS_WRITE_UNLIKELY(erts_no_pollsets) = 1; int ERTS_WRITE_UNLIKELY(erts_no_poll_threads) = 1; struct drv_ev_state_shared drv_ev_state; -static ERTS_INLINE int fd_hash(ErtsSysFdType fd) { +static ERTS_INLINE int fd_hash(ErtsSysFdType fd) +{ +#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS int hash = (int)fd; -# ifndef ERTS_SYS_CONTINOUS_FD_NUMBERS +#else + int hash = (int)(SWord)fd; hash ^= (hash >> 9); -# endif +#endif return hash; } @@ -1339,7 +1342,7 @@ print_driver_name(erts_dsprintf_buf_t *dsbufp, Eterm id) static void steal(erts_dsprintf_buf_t *dsbufp, ErtsDrvEventState *state, int mode) { - erts_dsprintf(dsbufp, "stealing control of fd=%d from ", (int) state->fd); + erts_dsprintf(dsbufp, "stealing control of fd=%bpd from ", (SWord) state->fd); switch (state->type) { case ERTS_EV_TYPE_DRV_SEL: { int deselect_mode = 0; @@ -1363,7 +1366,7 @@ steal(erts_dsprintf_buf_t *dsbufp, ErtsDrvEventState *state, int mode) if (deselect_mode) deselect(state, deselect_mode); else { - erts_dsprintf(dsbufp, "no one", (int) state->fd); + erts_dsprintf(dsbufp, "no one"); ASSERT(0); } erts_dsprintf(dsbufp, "\n"); @@ -1394,7 +1397,7 @@ steal(erts_dsprintf_buf_t *dsbufp, ErtsDrvEventState *state, int mode) break; } default: - erts_dsprintf(dsbufp, "no one\n", (int) state->fd); + erts_dsprintf(dsbufp, "no one\n"); ASSERT(0); } } @@ -1405,10 +1408,10 @@ print_drv_select_op(erts_dsprintf_buf_t *dsbufp, { Port *pp = erts_drvport2port(ix); erts_dsprintf(dsbufp, - "driver_select(%p, %d,%s%s%s%s, %d) " + "driver_select(%p, %bpd,%s%s%s%s, %d) " "by ", ix, - (int) fd, + (SWord) fd, mode & ERL_DRV_READ ? " ERL_DRV_READ" : "", mode & ERL_DRV_WRITE ? " ERL_DRV_WRITE" : "", mode & ERL_DRV_USE ? " ERL_DRV_USE" : "", @@ -1424,8 +1427,8 @@ print_nif_select_op(erts_dsprintf_buf_t *dsbufp, ErtsResource* resource, Eterm ref) { erts_dsprintf(dsbufp, - "enif_select(_, %d,%s%s%s, %T:%T, %T) ", - (int) fd, + "enif_select(_, %bpd,%s%s%s, %T:%T, %T) ", + (SWord) fd, mode & ERL_NIF_SELECT_READ ? " READ" : "", mode & ERL_NIF_SELECT_WRITE ? " WRITE" : "", (mode & ERL_NIF_SELECT_STOP ? " STOP" @@ -1715,10 +1718,10 @@ erts_check_io(ErtsPollThread *psi, ErtsMonotonicTime timeout_time) ErtsDrvEventState *state; ErtsPollEvents revents = ERTS_POLL_RES_GET_EVTS(&psi->pollres[i]); - /* The fd will be set to -1 if a pollset internal fd was triggered + /* The fd will be set to INVALID if a pollset internal fd was triggered that was determined to be too expensive to remove from the result. */ - if (fd == -1) continue; + if (fd == ERTS_SYS_FD_INVALID) continue; erts_mtx_lock(fd_mtx(fd)); @@ -1866,7 +1869,7 @@ erts_check_io(ErtsPollThread *psi, ErtsMonotonicTime timeout_time) dsbufp = erts_create_logger_dsbuf(); erts_dsprintf(dsbufp, "Invalid event request type for fd in erts_poll()! " - "fd=%d, event request type=%d\n", (int) state->fd, + "fd=%bpd, event request type=%d\n", (SWord) state->fd, (int) state->type); ASSERT(0); deselect(state, 0); @@ -1944,8 +1947,8 @@ bad_fd_in_pollset(ErtsDrvEventState *state, Eterm inport, Eterm outport) } } erts_dsprintf(dsbufp, - "Bad %s fd in erts_poll()! fd=%d, ", - io_str, (int) state->fd); + "Bad %s fd in erts_poll()! fd=%bpd, ", + io_str, (SWord) state->fd); if (state->type == ERTS_EV_TYPE_DRV_SEL) { if (is_nil(port)) { ErtsPortNames *ipnp = erts_get_port_names(inport, ERTS_INVALID_ERL_DRV_PORT); @@ -1978,7 +1981,8 @@ bad_fd_in_pollset(ErtsDrvEventState *state, Eterm inport, Eterm outport) } } else { - erts_dsprintf(dsbufp, "Bad fd in erts_poll()! fd=%d\n", (int) state->fd); + erts_dsprintf(dsbufp, "Bad fd in erts_poll()! fd=%bpd\n", + (SWord) state->fd); } erts_send_error_to_logger_nogl(dsbufp); @@ -1997,7 +2001,7 @@ stale_drv_select(Eterm id, ErtsDrvEventState *state, int mode) static SafeHashValue drv_ev_state_hash(void *des) { - SafeHashValue val = (SafeHashValue) ((ErtsDrvEventState *) des)->fd; + SafeHashValue val = (SafeHashValue)(SWord) ((ErtsDrvEventState *) des)->fd; return val ^ (val >> 8); /* Good enough for aligned pointer values? */ } @@ -2548,8 +2552,9 @@ static int erts_debug_print_checkio_state(erts_dsprintf_buf_t *dsbufp, #ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS ErtsPollEvents aio_events = state->active_events; #endif - erts_dsprintf(dsbufp, "pollset=%d fd=%d ", - state->flags & ERTS_EV_FLAG_FALLBACK ? -1 : get_pollset_id(fd), (int) fd); + erts_dsprintf(dsbufp, "pollset=%d fd=%bpd ", + state->flags & ERTS_EV_FLAG_FALLBACK ? -1 : get_pollset_id(fd), + (SWord) fd); #if defined(HAVE_FSTAT) && !defined(NO_FSTAT_ON_SYS_FD_TYPE) if (fstat((int) fd, &stat_buf) < 0) diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index e669572499..6eb5a99b06 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -1459,13 +1459,13 @@ ERTS_POLL_EXPORT(save_result)(ErtsPollSet *ps, ErtsPollResFd pr[], int max_res, if (ERTS_POLL_USE_WAKEUP(ps) && fd == wake_fd) { cleanup_wakeup_pipe(ps); - ERTS_POLL_RES_SET_FD(&pr[i], -1); + ERTS_POLL_RES_SET_FD(&pr[i], ERTS_SYS_FD_INVALID); ERTS_POLL_RES_SET_EVTS(&pr[i], ERTS_POLL_EV_NONE); res--; } #if ERTS_POLL_USE_TIMERFD else if (fd == ps->timer_fd) { - ERTS_POLL_RES_SET_FD(&pr[i], -1); + ERTS_POLL_RES_SET_FD(&pr[i], ERTS_SYS_FD_INVALID); ERTS_POLL_RES_SET_EVTS(&pr[i], ERTS_POLL_EV_NONE); res--; } diff --git a/erts/emulator/sys/win32/sys.c b/erts/emulator/sys/win32/sys.c index b4099776bb..244fed720d 100644 --- a/erts/emulator/sys/win32/sys.c +++ b/erts/emulator/sys/win32/sys.c @@ -2112,15 +2112,15 @@ translate_fd(int fd) handle = GetStdHandle(STD_ERROR_HANDLE); break; default: - return (HANDLE) fd; + return (HANDLE)(SWord) fd; } - DEBUGF(("translate_fd(%d) -> std(%d)\n", fd, handle)); + DEBUGF(("translate_fd(%d) -> std(%p)\n", fd, (void*)handle)); if (handle == INVALID_HANDLE_VALUE || handle == 0) { handle = CreateFile("nul", access, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL); } - DEBUGF(("translate_fd(%d) -> %d\n", fd, handle)); + DEBUGF(("translate_fd(%d) -> %p\n", fd, (void*)handle)); return handle; } @@ -2925,9 +2925,9 @@ unsigned char* sys_preload_begin(Preload* pp) ASSERT(beam_module != NULL); resource = res_name[pp-preloaded]; - DEBUGF(("Loading name: %s; size: %d; resource: %p\n", + DEBUGF(("Loading name: %s; size: %d; resource: %u\n", pp->name, pp->size, resource)); - hRes = FindResource(beam_module, (char *) resource, "ERLANG_CODE"); + hRes = FindResource(beam_module, (char*)(UWord) resource, "ERLANG_CODE"); return pp->code = LoadResource(beam_module, hRes); } diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index e0e38e34f9..3825796322 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -28,21 +28,6 @@ EBIN = . # Target Specs # ---------------------------------------------------- -SOCKET_MODULES = \ - socket_test_lib \ - socket_test_logger \ - socket_test_evaluator \ - socket_test_ttest_lib \ - socket_test_ttest_tcp_gen \ - socket_test_ttest_tcp_socket \ - socket_test_ttest_tcp_client \ - socket_test_ttest_tcp_client_gen \ - socket_test_ttest_tcp_client_socket \ - socket_test_ttest_tcp_server \ - socket_test_ttest_tcp_server_gen \ - socket_test_ttest_tcp_server_socket \ - socket_SUITE - MODULES= \ a_SUITE \ after_SUITE \ @@ -123,7 +108,6 @@ MODULES= \ signal_SUITE \ small_SUITE \ smoke_test_SUITE \ - $(SOCKET_MODULES) \ statistics_SUITE \ system_info_SUITE \ system_profile_SUITE \ @@ -173,13 +157,8 @@ NATIVE_MODULES= $(NATIVE:%=%_native_SUITE) NATIVE_ERL_FILES= $(NATIVE_MODULES:%=%.erl) ERL_FILES= $(MODULES:%=%.erl) -HRL_FILES= \ - socket_test_evaluator.hrl \ - socket_test_ttest.hrl \ - socket_test_ttest_client.hrl TARGET_FILES = $(MODULES:%=$(EBIN)/%.$(EMULATOR)) -SOCKET_TARGETS = $(SOCKET_MODULES:%=$(EBIN)/%.$(EMULATOR)) EMAKEFILE=Emakefile @@ -223,7 +202,6 @@ clean: docs: targets: $(TARGET_FILES) -socket_targets: $(SOCKET_TARGETS) # ---------------------------------------------------- # Special targets @@ -245,7 +223,7 @@ release_spec: release_tests_spec: make_emakefile $(INSTALL_DIR) "$(RELSYSDIR)" $(INSTALL_DATA) $(EMAKEFILE) $(TEST_SPEC_FILES) \ - $(ERL_FILES) $(HRL_FILES) "$(RELSYSDIR)" + $(ERL_FILES) "$(RELSYSDIR)" $(INSTALL_DATA) $(NO_OPT_ERL_FILES) "$(RELSYSDIR)" $(INSTALL_DATA) $(NATIVE_ERL_FILES) "$(RELSYSDIR)" chmod -R u+w "$(RELSYSDIR)" diff --git a/erts/emulator/test/alloc_SUITE.erl b/erts/emulator/test/alloc_SUITE.erl index 24677247a7..8288da9f61 100644 --- a/erts/emulator/test/alloc_SUITE.erl +++ b/erts/emulator/test/alloc_SUITE.erl @@ -370,7 +370,7 @@ many_shot(CaseName, I, Mem) -> Result1. concurrent(CaseName) -> - NSched = erlang:system_info(schedulers), + NSched = erlang:system_info(schedulers_online), Mem = (free_memory() * 3) div 4, PRs = lists:map(fun(I) -> spawn_opt(fun() -> many_shot(CaseName, I, diff --git a/erts/emulator/test/bs_construct_SUITE.erl b/erts/emulator/test/bs_construct_SUITE.erl index 29d9c5a74e..25549a4b73 100644 --- a/erts/emulator/test/bs_construct_SUITE.erl +++ b/erts/emulator/test/bs_construct_SUITE.erl @@ -27,7 +27,8 @@ mem_leak/1, coerce_to_float/1, bjorn/1, append_empty_is_same/1, huge_float_field/1, system_limit/1, badarg/1, copy_writable_binary/1, kostis/1, dynamic/1, bs_add/1, - otp_7422/1, zero_width/1, bad_append/1, bs_append_overflow/1]). + otp_7422/1, zero_width/1, bad_append/1, bs_append_overflow/1, + reductions/1]). -include_lib("common_test/include/ct.hrl"). @@ -40,7 +41,7 @@ all() -> in_guard, mem_leak, coerce_to_float, bjorn, append_empty_is_same, huge_float_field, system_limit, badarg, copy_writable_binary, kostis, dynamic, bs_add, otp_7422, zero_width, - bad_append, bs_append_overflow]. + bad_append, bs_append_overflow, reductions]. init_per_suite(Config) -> Config. @@ -889,6 +890,33 @@ bs_append_overflow_unsigned() -> C = <<A/binary,1,B/binary>>, true = byte_size(B) < byte_size(C). +reductions(_Config) -> + TwoMeg = <<0:(2_000*1024)/unit:8>>, + reds_at_least(2000, fun() -> <<0:8,TwoMeg/binary>> end), + reds_at_least(4000, fun() -> <<0:8,TwoMeg/binary,TwoMeg/binary>> end), + reds_at_least(1000, fun() -> <<0:8,TwoMeg:(1000*1024)/binary>> end), + + %% Here we expect about 500 reductions in the bs_append + %% instruction for setting up a writable binary and about 2000 + %% reductions in the bs_put_binary instruction for copying the + %% binary data. + reds_at_least(2500, fun() -> <<TwoMeg/binary,TwoMeg:(2000*1024)/binary>> end), + ok. + +reds_at_least(N, Fun) -> + receive after 1 -> ok end, + {reductions,Red0} = process_info(self(), reductions), + _ = Fun(), + {reductions,Red1} = process_info(self(), reductions), + Diff = Red1 - Red0, + io:format("Expected at least ~p; got ~p\n", [N,Diff]), + if + Diff >= N -> + ok; + Diff -> + ct:fail({expected,N,got,Diff}) + end. + id(I) -> I. memsize() -> diff --git a/erts/emulator/test/counters_SUITE.erl b/erts/emulator/test/counters_SUITE.erl index b3f0358c1e..7ddb574b36 100644 --- a/erts/emulator/test/counters_SUITE.erl +++ b/erts/emulator/test/counters_SUITE.erl @@ -66,7 +66,7 @@ check_memory(atomics, Memory, Size) -> {_,true} = {Memory, Memory > Size*8}, {_,true} = {Memory, Memory < Size*max_atomic_sz() + 100}; check_memory(write_concurrency, Memory, Size) -> - NWords = erlang:system_info(schedulers) + 1, + NWords = erlang:system_info(schedulers_online) + 1, {_,true} = {Memory, Memory > NWords*Size*8}, {_,true} = {Memory, Memory < NWords*(Size+7)*max_atomic_sz() + 100}. @@ -130,7 +130,7 @@ limits_do(Ref) -> %% Verify that independent workers, using different counters %% within the same array, do not interfere with each other. indep(Config) when is_list(Config) -> - NScheds = erlang:system_info(schedulers), + NScheds = erlang:system_info(schedulers_online), Ref = counters:new(NScheds,[write_concurrency]), Rounds = 100, Papa = self(), @@ -182,7 +182,7 @@ indep_subber(_Ref, _I, Val) -> write_concurrency(Config) when is_list(Config) -> rand:seed(exs1024s), io:format("*** SEED: ~p ***\n", [rand:export_seed()]), - NScheds = erlang:system_info(schedulers), + NScheds = erlang:system_info(schedulers_online), Size = 100, Ref = counters:new(Size,[write_concurrency]), Rounds = 1000, diff --git a/erts/emulator/test/distribution_SUITE.erl b/erts/emulator/test/distribution_SUITE.erl index 17e3ff0127..65b580f12c 100644 --- a/erts/emulator/test/distribution_SUITE.erl +++ b/erts/emulator/test/distribution_SUITE.erl @@ -68,6 +68,10 @@ message_latency_large_link_exit/1, message_latency_large_monitor_exit/1, message_latency_large_exit2/1, + message_latency_large_message/0, + message_latency_large_link_exit/0, + message_latency_large_monitor_exit/0, + message_latency_large_exit2/0, system_limit/1, hopefull_data_encoding/1, mk_hopefull_data/0]). @@ -1409,7 +1413,6 @@ get_conflicting_unicode_atoms(CIX, N) -> %% kept as they continue to find bugs in the distribution implementation. message_latency_large_message(Config) when is_list(Config) -> measure_latency_large_message(?FUNCTION_NAME, fun(Dropper, Payload) -> Dropper ! Payload end). - message_latency_large_exit2(Config) when is_list(Config) -> measure_latency_large_message(?FUNCTION_NAME, fun erlang:exit/2). @@ -1417,10 +1420,20 @@ message_latency_large_link_exit(Config) when is_list(Config) -> message_latency_large_exit(?FUNCTION_NAME, fun erlang:link/1). message_latency_large_monitor_exit(Config) when is_list(Config) -> - message_latency_large_exit(?FUNCTION_NAME, fun(Dropper) -> - Dropper ! {monitor, self()}, - receive ok -> ok end - end). + message_latency_large_exit(?FUNCTION_NAME, + fun(Dropper) -> + Dropper ! {monitor, self()}, + receive ok -> ok end + end). + +message_latency_large_message() -> + [{timetrap, {minutes, 6}}]. +message_latency_large_exit2() -> + message_latency_large_message(). +message_latency_large_link_exit() -> + message_latency_large_message(). +message_latency_large_monitor_exit() -> + message_latency_large_message(). message_latency_large_exit(Nodename, ReasonFun) -> measure_latency_large_message( diff --git a/erts/emulator/test/hash_SUITE.erl b/erts/emulator/test/hash_SUITE.erl index dd71c3da58..c4a700d1a7 100644 --- a/erts/emulator/test/hash_SUITE.erl +++ b/erts/emulator/test/hash_SUITE.erl @@ -560,17 +560,22 @@ last_byte(Bin) -> test_phash2_4GB_plus_bin(Config) when is_list(Config) -> run_when_enough_resources( fun() -> - erts_debug:set_internal_state(available_internal_state, true), - %% Created Bin4GB here so it only needs to be created once - erts_debug:set_internal_state(force_gc, self()), - Bin4GB = get_4GB_bin(), - test_phash2_plus_bin_helper1(Bin4GB, <<>>, <<>>, 13708901), - erts_debug:set_internal_state(force_gc, self()), - test_phash2_plus_bin_helper1(Bin4GB, <<>>, <<3:5>>, 66617678), - erts_debug:set_internal_state(force_gc, self()), - test_phash2_plus_bin_helper1(Bin4GB, <<13>>, <<>>, 31308392), - erts_debug:set_internal_state(force_gc, self()), - erts_debug:set_internal_state(available_internal_state, false) + {ok, N} = start_node(?FUNCTION_NAME), + erpc:call(N, + fun() -> + erts_debug:set_internal_state(available_internal_state, true), + %% Created Bin4GB here so it only needs to be created once + erts_debug:set_internal_state(force_gc, self()), + Bin4GB = get_4GB_bin(), + test_phash2_plus_bin_helper1(Bin4GB, <<>>, <<>>, 13708901), + erts_debug:set_internal_state(force_gc, self()), + test_phash2_plus_bin_helper1(Bin4GB, <<>>, <<3:5>>, 66617678), + erts_debug:set_internal_state(force_gc, self()), + test_phash2_plus_bin_helper1(Bin4GB, <<13>>, <<>>, 31308392), + erts_debug:set_internal_state(force_gc, self()), + erts_debug:set_internal_state(available_internal_state, false) + end), + stop_node(N) end). @@ -661,6 +666,36 @@ total_memory() -> undefined end. +start_node(X) -> + start_node(X, [], []). + +start_node(X, Y) -> + start_node(X, Y, []). + +start_node(Name, Args, Rel) when is_atom(Name), is_list(Rel) -> + Pa = filename:dirname(code:which(?MODULE)), + Cookie = atom_to_list(erlang:get_cookie()), + RelArg = case Rel of + [] -> []; + _ -> [{erl,[{release,Rel}]}] + end, + test_server:start_node(Name, slave, + [{args, + Args++" -setcookie "++Cookie++" -pa \""++Pa++"\""} + | RelArg]); +start_node(Config, Args, Rel) when is_list(Config), is_list(Rel) -> + Name = list_to_atom((atom_to_list(?MODULE) + ++ "-" + ++ atom_to_list(proplists:get_value(testcase, Config)) + ++ "-" + ++ integer_to_list(erlang:system_time(second)) + ++ "-" + ++ integer_to_list(erlang:unique_integer([positive])))), + start_node(Name, Args, Rel). + +stop_node(Node) -> + test_server:stop_node(Node). + -ifdef(FALSE). f1() -> abc. diff --git a/erts/emulator/test/lcnt_SUITE.erl b/erts/emulator/test/lcnt_SUITE.erl index d09bfd5776..3f56c820f4 100644 --- a/erts/emulator/test/lcnt_SUITE.erl +++ b/erts/emulator/test/lcnt_SUITE.erl @@ -192,7 +192,9 @@ remove_untoggleable_locks([]) -> []; remove_untoggleable_locks([{resource_monitors, _, _, _} | T]) -> remove_untoggleable_locks(T); -remove_untoggleable_locks([{'esock[gcnt]', _, _, _} | T]) -> +remove_untoggleable_locks([{nif_load, _, _, _} | T]) -> + remove_untoggleable_locks(T); +remove_untoggleable_locks([{'esock.gcnt', _, _, _} | T]) -> %% Global lock used by socket NIF remove_untoggleable_locks(T); remove_untoggleable_locks([H | T]) -> diff --git a/erts/emulator/test/port_bif_SUITE.erl b/erts/emulator/test/port_bif_SUITE.erl index b71f8f2715..c238d0bb40 100644 --- a/erts/emulator/test/port_bif_SUITE.erl +++ b/erts/emulator/test/port_bif_SUITE.erl @@ -487,11 +487,16 @@ busy_options(Config) when is_list(Config) -> process_flag(trap_exit, true), Tester = self(), - HejLoop = fun (Prt, _F, 1000) -> + + %% We want this loop to write enough data to tirgger the busy limits, + %% this means that it first has to fill the pipe buffer on Linux which + %% can be anything from 4k to 1 MB, so we make sure to fill it with + %% at least 2 MB of data here. + HejLoop = fun (Prt, _F, N) when N > (2 bsl 20) -> Prt; (Prt, F, N) -> Prt ! {Tester, {command, Data}}, - F(Prt, F, N+1) + F(Prt, F, N + iolist_size(Data)) end, io:format("Test1...~n", []), @@ -640,6 +645,3 @@ wait_until_aux(Fun, End) -> end end end. - - - diff --git a/erts/emulator/test/process_SUITE.erl b/erts/emulator/test/process_SUITE.erl index f36e00d3ce..e8d5845df6 100644 --- a/erts/emulator/test/process_SUITE.erl +++ b/erts/emulator/test/process_SUITE.erl @@ -64,6 +64,7 @@ gc_request_when_gc_disabled/1, gc_request_blast_when_gc_disabled/1, otp_16436/1, + otp_16642/1, spawn_huge_arglist/1, spawn_request_bif/1, spawn_request_monitor_demonitor/1, @@ -135,7 +136,7 @@ groups() -> [no_priority_inversion, no_priority_inversion2, system_task_blast, system_task_on_suspended, system_task_failed_enqueue, gc_request_when_gc_disabled, gc_request_blast_when_gc_disabled, - otp_16436]}]. + otp_16436, otp_16642]}]. init_per_suite(Config) -> A0 = case application:start(sasl) of @@ -1127,7 +1128,7 @@ process_info_status_handled_signal(Config) when is_list(Config) -> %% And a bug where process_info(reductions) on a process which was releasing its %% main lock during execution could result in negative reduction diffs. process_info_reductions(Config) when is_list(Config) -> - {S1, S2} = case erlang:system_info(schedulers) of + {S1, S2} = case erlang:system_info(schedulers_online) of 1 -> {1,1}; _ -> {1,2} end, @@ -3863,8 +3864,121 @@ otp_16436(Config) when is_list(Config) -> exit(P, kill), ok. +otp_16642(Config) when is_list(Config) -> + %% + %% Whitebox testing... + %% + %% Ensure that low prio system tasks are interleaved with + %% normal prio system tasks as they should. + %% + process_flag(priority, high), + process_flag(scheduler, 1), + Pid = spawn_opt(fun () -> receive after infinity -> ok end end, + [link, {scheduler, 1}]), + ReqSTasks = fun (Prio, Start, Stop) -> + lists:foreach( + fun (N) -> + erts_internal:request_system_task( + Pid, + Prio, + {check_process_code, + {Prio, N}, + '__non_existing_module__'}) + end, + lists:seq(Start, Stop)) + end, + MkResList = fun (Prio, Start, Stop) -> + lists:map(fun (N) -> + {check_process_code, + {Prio, N}, + false} + end, + lists:seq(Start, Stop)) + end, + + %%% Test when normal queue clears first... + + ReqSTasks(low, 0, 1), + ReqSTasks(normal, 0, 10), + ReqSTasks(low, 2, 4), + ReqSTasks(normal, 11, 26), + + Msgs1 = recv_msgs(32), + io:format("Got test 1 messages: ~p~n", [Msgs1]), + + ExpMsgs1 = + MkResList(normal, 0, 7) + ++ MkResList(low, 0, 0) + ++ MkResList(normal, 8, 15) + ++ MkResList(low, 1, 1) + ++ MkResList(normal, 16, 23) + ++ MkResList(low, 2, 2) + ++ MkResList(normal, 24, 26) + ++ MkResList(low, 3, 4), + + case Msgs1 =:= ExpMsgs1 of + true -> + ok; + false -> + io:format("Expected test 1 messages ~p~n", + [ExpMsgs1]), + ct:fail(unexpected_messages) + end, + + receive Unexp1 -> ct:fail({unexpected_message, Unexp1}) + after 500 -> ok + end, + + io:format("Test 1 as expected~n", []), + + %%% Test when low queue clears first... + + ReqSTasks(low, 0, 1), + ReqSTasks(normal, 0, 20), + + Msgs2 = recv_msgs(23), + io:format("Got test 2 messages: ~p~n", [Msgs2]), + + ExpMsgs2 = + MkResList(normal, 0, 7) + ++ MkResList(low, 0, 0) + ++ MkResList(normal, 8, 15) + ++ MkResList(low, 1, 1) + ++ MkResList(normal, 16, 20), + + case Msgs2 =:= ExpMsgs2 of + true -> + ok; + false -> + io:format("Expected test 2 messages ~p~n", + [ExpMsgs2]), + ct:fail(unexpected_messages) + end, + + receive Unexp2 -> ct:fail({unexpected_message, Unexp2}) + after 500 -> ok + end, + + io:format("Test 2 as expected~n", []), + + unlink(Pid), + exit(Pid, kill), + false = is_process_alive(Pid), + ok. + %% Internal functions +recv_msgs(N) -> + recv_msgs(N, []). + +recv_msgs(0, Msgs) -> + lists:reverse(Msgs); +recv_msgs(N, Msgs) -> + receive + Msg -> + recv_msgs(N-1, [Msg|Msgs]) + end. + wait_until(Fun) -> case Fun() of true -> true; diff --git a/erts/emulator/test/receive_SUITE.erl b/erts/emulator/test/receive_SUITE.erl index 339507c9d8..333ca6a02a 100644 --- a/erts/emulator/test/receive_SUITE.erl +++ b/erts/emulator/test/receive_SUITE.erl @@ -43,7 +43,7 @@ all() -> erl_1199]. init_per_testcase(receive_opt_deferred_save, Config) -> - case erlang:system_info(schedulers) of + case erlang:system_info(schedulers_online) of 1 -> {skip, "Needs more schedulers to run"}; _ -> diff --git a/erts/emulator/test/timer_bif_SUITE.erl b/erts/emulator/test/timer_bif_SUITE.erl index 7d682897f8..2382dc992e 100644 --- a/erts/emulator/test/timer_bif_SUITE.erl +++ b/erts/emulator/test/timer_bif_SUITE.erl @@ -492,7 +492,7 @@ same_time_yielding(Config) when is_list(Config) -> Mem = mem(), Ref = make_ref(), SchdlrsOnln = erlang:system_info(schedulers_online), - Tmo = erlang:monotonic_time(millisecond) + 3000, + Tmo = erlang:monotonic_time(millisecond) + 6000, Tmrs = lists:map(fun (I) -> process_flag(scheduler, (I rem SchdlrsOnln) + 1), erlang:start_timer(Tmo, self(), Ref, [{abs, true}]) @@ -536,7 +536,7 @@ same_time_yielding_with_cancel_other(Config) when is_list(Config) -> do_cancel_tmrs(Tmo, Tmrs, Tester) -> BeginCancel = erlang:convert_time_unit(Tmo, millisecond, - microsecond) - 100, + microsecond) - 500, busy_wait_until(fun () -> erlang:monotonic_time(microsecond) >= BeginCancel end), @@ -553,7 +553,7 @@ do_cancel_tmrs(Tmo, Tmrs, Tester) -> same_time_yielding_with_cancel_test(Other, Accessor) -> Mem = mem(), SchdlrsOnln = erlang:system_info(schedulers_online), - Tmo = erlang:monotonic_time(millisecond) + 3000, + Tmo = erlang:monotonic_time(millisecond) + 6000, Tester = self(), Cancelor = case Other of false -> diff --git a/erts/emulator/valgrind/suppress.standard b/erts/emulator/valgrind/suppress.standard index 99a3ee4048..3d6b4b50d3 100644 --- a/erts/emulator/valgrind/suppress.standard +++ b/erts/emulator/valgrind/suppress.standard @@ -252,7 +252,19 @@ obj:*/ssleay.* fun:start_thread fun:clone } - +{ + Crypto engine internal... + Memcheck:Leak + fun:malloc + fun:CRYPTO_malloc + fun:ENGINE_new + fun:ENGINE_by_id + fun:engine_by_id_nif + fun:process_main + fun:sched_thread_func + fun:thr_wrapper + fun:start_thread +} { Harmless assembler bug in openssl Memcheck:Addr8 diff --git a/erts/epmd/src/epmd_srv.c b/erts/epmd/src/epmd_srv.c index 2f4f189bb4..ad264585c6 100644 --- a/erts/epmd/src/epmd_srv.c +++ b/erts/epmd/src/epmd_srv.c @@ -198,10 +198,46 @@ static EPMD_INLINE void select_fd_set(EpmdVars* g, int fd) } } +#ifdef HAVE_SOCKLEN_T +static const char *epmd_ntop(struct sockaddr_storage *sa, char *buff, socklen_t len) { +#else +static const char *epmd_ntop(struct sockaddr_storage *sa, char *buff, size_t len) { +#endif + /* Save errno so that it is not changed by inet_ntop */ + int myerrno = errno; + const char *res; +#if !defined(EPMD6) + if (sa->ss_family == AF_INET6) { + struct sockaddr_in6 *addr = (struct sockaddr_in6 *)sa; + res = inet_ntop( + addr->sin6_family, + (void*)&addr->sin6_addr, + buff, len); + } else { + struct sockaddr_in *addr = (struct sockaddr_in *)sa; + res = inet_ntop( + addr->sin_family, &addr->sin_addr.s_addr, + buff, len); + } +#else + struct sockaddr_in *addr = (struct sockaddr_in *)sa; + res = inet_ntoa(addr->sin_addr); + erts_snprintf(buff,len,"%s", res); + res = buff; +#endif + errno = myerrno; + return res; +} + void run(EpmdVars *g) { struct EPMD_SOCKADDR_IN iserv_addr[MAX_LISTEN_SOCKETS]; int listensock[MAX_LISTEN_SOCKETS]; +#if defined(EPMD6) + char socknamebuf[INET6_ADDRSTRLEN]; +#else + char socknamebuf[INET_ADDRSTRLEN]; +#endif int num_sockets = 0; int i; int opt; @@ -415,26 +451,32 @@ void run(EpmdVars *g) opt = fcntl(listensock[i], F_GETFL, 0); if (fcntl(listensock[i], F_SETFL, opt | O_NONBLOCK) == -1) #endif /* __WIN32__ */ - dbg_perror(g,"failed to set non-blocking mode of listening socket %d", - listensock[i]); + dbg_perror(g,"failed to set non-blocking mode of listening socket %d on ipaddr %s", + listensock[i], epmd_ntop(&iserv_addr[num_sockets], + socknamebuf, sizeof(socknamebuf))); - if (bind(listensock[i], (struct sockaddr*) &iserv_addr[i], salen) < 0) + if (bind(listensock[i], sa, salen) < 0) { if (errno == EADDRINUSE) { - dbg_tty_printf(g,1,"there is already a epmd running at port %d", - g->port); + dbg_tty_printf(g,1,"there is already a epmd running at port %d on ipaddr %s", + g->port, epmd_ntop(&iserv_addr[num_sockets], + socknamebuf, sizeof(socknamebuf))); epmd_cleanup_exit(g,0); } else { - dbg_perror(g,"failed to bind socket"); - epmd_cleanup_exit(g,1); + dbg_perror(g,"failed to bind on ipaddr %s", + epmd_ntop(&iserv_addr[num_sockets], + socknamebuf, sizeof(socknamebuf))); + epmd_cleanup_exit(g,1); } } if(listen(listensock[i], SOMAXCONN) < 0) { - dbg_perror(g,"failed to listen on socket"); + dbg_perror(g,"failed to listen on ipaddr %s", + epmd_ntop(&iserv_addr[num_sockets], + socknamebuf, sizeof(socknamebuf))); epmd_cleanup_exit(g,1); } select_fd_set(g, listensock[i]); @@ -1054,6 +1096,14 @@ static int conn_open(EpmdVars *g,int fd) int i; Connection *s; +#if !defined(__WIN32__) + if (fd >= FD_SETSIZE) { + dbg_tty_printf(g,0,"fd does not fit in fd_set fd=%d, FD_SETSIZE=%d",fd, FD_SETSIZE); + close(fd); + return EPMD_FALSE; + } +#endif + for (i = 0; i < g->max_conn; i++) { if (g->conn[i].open == EPMD_FALSE) { g->active_conn++; diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index ef40341c87..650ab1f6fe 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -154,7 +154,14 @@ MC_OUTPUTS=$(OBJDIR)/erlsrv_logmess.h $(OBJDIR)/erlsrv_logmess.res MT_FLAG="-MD" endif INET_GETHOST = $(BINDIR)/inet_gethost.exe -INSTALL_EMBEDDED_PROGS += $(BINDIR)/typer.exe $(BINDIR)/dialyzer.exe $(BINDIR)/erlc.exe $(BINDIR)/start_erl.exe $(BINDIR)/escript.exe $(BINDIR)/ct_run.exe +INSTALL_EMBEDDED_PROGS += \ + $(BINDIR)/typer.exe \ + $(BINDIR)/dialyzer.exe \ + $(BINDIR)/erlc.exe \ + $(BINDIR)/start_erl.exe \ + $(BINDIR)/escript.exe \ + $(BINDIR)/ct_run.exe \ + $(BINDIR)/erl_call.exe INSTALL_SRC = $(WINETC)/start_erl.c $(WINETC)/Nmakefile.start_erl ERLEXECDIR=. INSTALL_LIBS = @@ -165,9 +172,11 @@ INSTALL_TOP = Install.ini INSTALL_TOP_BIN = $(BINDIR)/Install.exe INSTALL_PROGS = \ $(INET_GETHOST) \ - $(BINDIR)/heart.exe $(BINDIR)/erlsrv.exe \ - $(BINDIR)/erl.exe $(BINDIR)/erl_log.exe \ - $(BINDIR)/werl.exe \ + $(BINDIR)/heart.exe \ + $(BINDIR)/erlsrv.exe \ + $(BINDIR)/erl.exe \ + $(BINDIR)/erl_log.exe\ + $(BINDIR)/werl.exe \ $(BINDIR)/$(ERLEXEC) \ $(INSTALL_EMBEDDED_PROGS) @@ -187,9 +196,16 @@ ENTRY_OBJ= ERLSRV_OBJECTS= MC_OUTPUTS= INET_GETHOST = $(BINDIR)/inet_gethost@EXEEXT@ -INSTALL_EMBEDDED_PROGS += $(BINDIR)/typer@EXEEXT@ $(BINDIR)/dialyzer@EXEEXT@ \ - $(BINDIR)/erlc@EXEEXT@ $(BINDIR)/escript@EXEEXT@ $(BINDIR)/ct_run@EXEEXT@ \ - $(BINDIR)/run_erl $(BINDIR)/to_erl $(BINDIR)/dyn_erl +INSTALL_EMBEDDED_PROGS += \ + $(BINDIR)/typer@EXEEXT@ \ + $(BINDIR)/dialyzer@EXEEXT@ \ + $(BINDIR)/erlc@EXEEXT@ \ + $(BINDIR)/escript@EXEEXT@ \ + $(BINDIR)/ct_run@EXEEXT@ \ + $(BINDIR)/run_erl@EXEEXT@ \ + $(BINDIR)/to_erl@EXEEXT@ \ + $(BINDIR)/dyn_erl@EXEEXT@ \ + $(BINDIR)/erl_call@EXEEXT@ INSTALL_EMBEDDED_DATA = $(UXETC)/start.src $(UXETC)/start_erl.src INSTALL_TOP = Install INSTALL_TOP_BIN = @@ -424,6 +440,10 @@ $(OBJDIR)/dyn_erl.o: $(UXETC)/dyn_erl.c $(RC_GENERATED) $(OBJDIR)/safe_string.o: $(ETC)/safe_string.c $(RC_GENERATED) $(V_CC) $(CFLAGS) -o $@ -c $(ETC)/safe_string.c +# erl_call +$(BINDIR)/erl_call@EXEEXT@: $(ERL_TOP)/lib/erl_interface/bin/$(TARGET)/erl_call@EXEEXT@ + $(ld_verbose)cp $< $@ + ifneq ($(TARGET),win32) $(BINDIR)/$(ERLEXEC): $(OBJDIR)/$(ERLEXEC).o $(ERTS_LIB) $(ld_verbose)$(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/$(ERLEXEC).o $(ERTS_INTERNAL_LIBS) diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c index 890a5b381a..9a2488c93c 100644 --- a/erts/etc/common/erlexec.c +++ b/erts/etc/common/erlexec.c @@ -219,7 +219,7 @@ static void get_start_erl_data(char *); static char* get_value(HKEY key, char* value_name, BOOL mustExit); static char* possibly_quote(char* arg); -/* +/* * Functions from win_erlexec.c */ int start_win_emulator(char* emu, char *startprog,char** argv, int start_detached); @@ -249,7 +249,7 @@ static int start_smp_emu = 1; /* Start the smp emulator. */ static const char* emu_type = 0; /* Type of emulator (lcnt, valgrind, etc) */ #ifdef __WIN32__ -static char *start_emulator_program = NULL; /* For detachec mode - +static char *start_emulator_program = NULL; /* For detached mode - erl.exe/werl.exe */ static char* key_val_name = ERLANG_VERSION; /* Used by the registry * access functions. @@ -267,7 +267,7 @@ static WCHAR *latin1_to_utf16(char *str); #endif /* - * Needed parameters to be fetched from the environment (Unix) + * Parameters to be fetched from the environment (Unix) * or the ini file (Win32). */ @@ -275,7 +275,7 @@ static char* bindir; /* Location of executables. */ static char* rootdir; /* Root location of Erlang installation. */ static char* emu; /* Emulator to run. */ static char* progname; /* Name of this program. */ -static char* home; /* Path of user's home directory. */ +static char* home; /* Path of user's home directory, if any. */ static void set_env(char *key, char *value) @@ -415,7 +415,7 @@ __declspec(dllexport) int win_erlexec(int argc, char **argv, HANDLE module, int int main(int argc, char **argv) #endif { - int haltAfterwards = 0; /* If true, put 's erlang halt' at the end + int haltAfterwards = 0; /* If true, put '-s erlang halt' at the end * of the arguments. */ int isdistributed = 0; int no_epmd = 0; @@ -431,7 +431,7 @@ int main(int argc, char **argv) #ifdef __WIN32__ this_module_handle = module; run_werl = windowed; - /* if we started this erl just to get a detached emulator, + /* if we started this erl just to get a detached emulator, * the arguments are already prepared for beam, so we skip * directly to start_emulator */ s = get_env("ERL_CONSOLE_MODE"); @@ -449,7 +449,7 @@ int main(int argc, char **argv) emu = argv[0]; start_emulator_program = strsave(argv[0]); goto skip_arg_massage; - } + } free_env_val(s); #else int reset_cerl_detached = 0; @@ -480,7 +480,7 @@ int main(int argc, char **argv) #endif get_parameters(argc, argv); - + /* * Construct the path of the executable. */ @@ -596,11 +596,16 @@ int main(int argc, char **argv) free_env_val(s); set_env("PATH", tmpStr); - + i = 1; get_home(); - add_args("-home", home, NULL); + /* Add the home parameter when available. This is optional to support + systems that don't have the notion of a home directory and setups + that don't have the HOME environment variable set (ERL-476). */ + if (home != NULL) { + add_args("-home", home, NULL); + } add_epmd_port(); @@ -634,7 +639,7 @@ int main(int argc, char **argv) case 'c': if (strcmp(argv[i], "-compile") == 0) { /* - * Note that the shell script erl.exec does an recursive call + * Note that the shell script erl.exec does a recursive call * on itself here. We'll avoid doing that. */ add_args("-noshell", "-noinput", "-s", "c", "lc_batch", @@ -690,7 +695,7 @@ int main(int argc, char **argv) usage("-env"); set_env(argv[i+1], argv[i+2]); i += 2; - } else if (strcmp(argv[i], "-epmd") == 0) { + } else if (strcmp(argv[i], "-epmd") == 0) { if (i+1 >= argc) usage("-epmd"); epmd_prog = argv[i+1]; @@ -708,7 +713,7 @@ int main(int argc, char **argv) case 'm': /* - * Note that the shell script erl.exec does an recursive call + * Note that the shell script erl.exec does a recursive call * on itself here. We'll avoid doing that. */ if (strcmp(argv[i], "-make") == 0) { @@ -734,7 +739,7 @@ int main(int argc, char **argv) if (strcmp(argv[i], "-name") == 0) { /* -name NAME */ if (i+1 >= argc) usage("-name"); - + /* * Note: Cannot use add_args() here, due to non-defined * evaluation order. @@ -771,7 +776,7 @@ int main(int argc, char **argv) add_arg(argv[i]); add_arg(argv[i+1]); i++; - } + } else if (strcmp(argv[i], "-start_erl") == 0) { if (i+1 < argc && argv[i+1][0] != '-') { get_start_erl_data(argv[i+1]); @@ -893,7 +898,7 @@ int main(int argc, char **argv) if (argv[i][2] != '\0') { if ((argv[i][2] != 'i') && (argv[i][2] != 'c') && - (argv[i][2] != 'd')) { + (argv[i][2] != 'd')) { usage(argv[i]); } else { add_Eargs(argv[i]); @@ -1025,7 +1030,7 @@ int main(int argc, char **argv) add_Eargs(argv[i]); } break; - + default: add_arg(argv[i]); } /* switch(argv[i][0] */ @@ -1043,7 +1048,7 @@ int main(int argc, char **argv) if (haltAfterwards) { add_args("-s", "erlang", "halt", NULL); } - + if (isdistributed && !no_epmd) start_epmd(epmd_prog); @@ -1052,8 +1057,8 @@ int main(int argc, char **argv) /* Start the emulator within an xterm. * Move up all arguments and insert * "xterm -e " first. - * The path must be searched for this - * to work, i.e execvp() must be used. + * The path must be searched for this + * to work, i.e execvp() must be used. */ ensure_EargsSz(EargsCnt+2); for (i = EargsCnt; i > 0; i--) @@ -1061,9 +1066,9 @@ int main(int argc, char **argv) EargsCnt += 2; /* Two args to insert */ Eargsp[0] = emu = "xterm"; Eargsp[1] = "-e"; - } + } #endif - + add_Eargs("--"); add_Eargs("-root"); add_Eargs(rootdir); @@ -1074,7 +1079,7 @@ int main(int argc, char **argv) for (i = 0; i < argsCnt; i++) Eargsp[EargsCnt++] = argsp[i]; Eargsp[EargsCnt] = NULL; - + if (print_qouted_cmd_exit) { printf("\"%s\" ", emu); for (i = 1; i < EargsCnt; i++) @@ -1099,7 +1104,7 @@ int main(int argc, char **argv) #ifdef __WIN32__ if (EargsSz != EargsCnt + 1) - Eargsp = (char **) erealloc((void *) Eargsp, (EargsCnt + 1) * + Eargsp = (char **) erealloc((void *) Eargsp, (EargsCnt + 1) * sizeof(char *)); efree((void *) argsp); @@ -1165,7 +1170,7 @@ int main(int argc, char **argv) #ifdef DEBUG execvp(emu, Eargsp); /* "xterm ..." needs to search the path */ #endif - } + } #ifdef DEBUG else #endif @@ -1245,7 +1250,7 @@ start_epmd(char *epmd) #else erts_snprintf(epmd_cmd, sizeof(epmd_cmd), "\"%s" DIRSEP "epmd\" -daemon", bindir); #endif - } + } #ifdef __WIN32__ if (arg1 != NULL) { strcat(epmd, " "); @@ -1259,7 +1264,7 @@ start_epmd(char *epmd) start.cb = sizeof (start); MultiByteToWideChar(CP_UTF8, 0, epmd, -1, wcepmd, MAXPATHLEN+100); - if (!CreateProcessW(NULL, wcepmd, NULL, NULL, FALSE, + if (!CreateProcessW(NULL, wcepmd, NULL, NULL, FALSE, CREATE_DEFAULT_ERROR_MODE | DETACHED_PROCESS, NULL, NULL, &start, &pi)) result = -1; @@ -1289,7 +1294,7 @@ add_args(char *first_arg, ...) { va_list ap; char* arg; - + add_arg(first_arg); va_start(ap, first_arg); while ((arg = va_arg(ap, char *)) != NULL) { @@ -1348,7 +1353,7 @@ erealloc(void *p, size_t size) } static void -efree(void *p) +efree(void *p) { free(p); } @@ -1395,7 +1400,7 @@ static void get_start_erl_data(char *file) char* reldir; char* otpstring; char* tprogname; - if (boot_script) + if (boot_script) error("Conflicting -start_erl and -boot options"); if (config_scripts) error("Conflicting -start_erl and -config options"); @@ -1452,12 +1457,12 @@ static void get_start_erl_data(char *file) erts_snprintf(a_config_script, 512, "%s/%s/sys", reldir, otpstring); config_scripts = &a_config_script; config_script_cnt = 1; - + got_start_erl = 1; } -static wchar_t *replace_filename(wchar_t *path, wchar_t *new_base) +static wchar_t *replace_filename(wchar_t *path, wchar_t *new_base) { int plen = wcslen(path); wchar_t *res = (wchar_t *) emalloc((plen+wcslen(new_base)+1)*sizeof(wchar_t)); @@ -1480,8 +1485,8 @@ static char *path_massage(wchar_t *long_path) WideCharToMultiByte(CP_UTF8, 0, long_path, -1, p, len, NULL, NULL); return p; } - -static char *do_lookup_in_section(InitSection *inis, char *name, + +static char *do_lookup_in_section(InitSection *inis, char *name, char *section, wchar_t *filename, int is_path) { char *p = lookup_init_entry(inis, name); @@ -1500,8 +1505,8 @@ static void get_parameters(int argc, char** argv) wchar_t *p; wchar_t buffer[MAX_PATH]; wchar_t *ini_filename; - HANDLE module = GetModuleHandle(NULL); /* This might look strange, but we want the erl.ini - that resides in the same dir as erl.exe, not + HANDLE module = GetModuleHandle(NULL); /* This might look strange, but we want the erl.ini + that resides in the same dir as erl.exe, not an erl.ini in our directory */ InitFile *inif; InitSection *inis; @@ -1550,9 +1555,9 @@ static void get_parameters(int argc, char** argv) } bindir = do_lookup_in_section(inis, "Bindir", INI_SECTION, ini_filename,1); - rootdir = do_lookup_in_section(inis, "Rootdir", INI_SECTION, + rootdir = do_lookup_in_section(inis, "Rootdir", INI_SECTION, ini_filename,1); - progname = do_lookup_in_section(inis, "Progname", INI_SECTION, + progname = do_lookup_in_section(inis, "Progname", INI_SECTION, ini_filename,0); free_init_file(inif); } @@ -1577,7 +1582,7 @@ get_home(void) home = utf16_to_utf8(profile); /* CoTaskMemFree(profile); */ } else - error("HOMEDRIVE or HOMEPATH is not set and GetWindowsDir failed"); + error("HOMEDRIVE or HOMEPATH not set and getting USERPROFILE failed"); } else { home = emalloc(strlen(homedrive)+strlen(homepath)+1); strcpy(home, homedrive); @@ -1642,8 +1647,6 @@ static void get_home(void) { home = get_env("HOME"); - if (home == NULL) - error("HOME must be set"); } #endif @@ -1989,7 +1992,7 @@ get_file_args(char *filename, argv_buf *abp, argv_buf *xabp) i = 0; argv = read_args_file(filename); - + while (argv) { while (argv[i]) { @@ -2092,7 +2095,7 @@ initial_argv_massage(int *argc, char ***argv) build_new_argv: save_arg(&ab, (*argv)[0]); - + vix = 0; while (avv[vix].argv) { ac = avv[vix].argc; @@ -2184,12 +2187,12 @@ possibly_quote(char* arg) * Unicode helpers to handle environment and command line parameters on * Windows. We internally handle all environment variables in UTF8, * but put and get the environment using the WCHAR (limited UTF16) interface - * - * These are simplified to only handle Unicode characters that can fit in + * + * These are simplified to only handle Unicode characters that can fit in * Windows simplified UTF16, i.e. characters that fit in 16 bits. */ -static int utf8_len(unsigned char first) +static int utf8_len(unsigned char first) { if ((first & ((unsigned char) 0x80)) == 0) { return 1; @@ -2199,7 +2202,7 @@ static int utf8_len(unsigned char first) return 3; } else if ((first & ((unsigned char) 0xF8)) == 0xF0) { return 4; - } + } return 1; /* will be a '?' */ } @@ -2209,7 +2212,7 @@ static WCHAR *utf8_to_utf16(unsigned char *bytes) unsigned char *tmp = bytes; WCHAR *target, *res; int num = 0; - + while (*tmp) { num++; tmp += utf8_len(*tmp); @@ -2220,12 +2223,12 @@ static WCHAR *utf8_to_utf16(unsigned char *bytes) unipoint = (unsigned int) *bytes; ++bytes; } else if (((*bytes) & ((unsigned char) 0xE0)) == 0xC0) { - unipoint = + unipoint = (((unsigned int) ((*bytes) & ((unsigned char) 0x1F))) << 6) | ((unsigned int) (bytes[1] & ((unsigned char) 0x3F))); bytes += 2; } else if (((*bytes) & ((unsigned char) 0xF0)) == 0xE0) { - unipoint = + unipoint = (((unsigned int) ((*bytes) & ((unsigned char) 0xF))) << 12) | (((unsigned int) (bytes[1] & ((unsigned char) 0x3F))) << 6) | ((unsigned int) (bytes[2] & ((unsigned char) 0x3F))); @@ -2258,9 +2261,9 @@ static int put_utf8(WCHAR ch, unsigned char *target, int sz, int *pos) if (((*pos) + 1) >= sz) { return -1; } - target[(*pos)++] = (((unsigned char) (x >> 6)) | + target[(*pos)++] = (((unsigned char) (x >> 6)) | ((unsigned char) 0xC0)); - target[(*pos)++] = (((unsigned char) (x & 0x3F)) | + target[(*pos)++] = (((unsigned char) (x & 0x3F)) | ((unsigned char) 0x80)); } else { if ((x >= 0xD800 && x <= 0xDFFF) || @@ -2272,11 +2275,11 @@ static int put_utf8(WCHAR ch, unsigned char *target, int sz, int *pos) return -1; } - target[(*pos)++] = (((unsigned char) (x >> 12)) | + target[(*pos)++] = (((unsigned char) (x >> 12)) | ((unsigned char) 0xE0)); - target[(*pos)++] = ((((unsigned char) (x >> 6)) & 0x3F) | + target[(*pos)++] = ((((unsigned char) (x >> 6)) & 0x3F) | ((unsigned char) 0x80)); - target[(*pos)++] = (((unsigned char) (x & 0x3F)) | + target[(*pos)++] = (((unsigned char) (x & 0x3F)) | ((unsigned char) 0x80)); } return 0; @@ -2288,7 +2291,7 @@ static int need_bytes_for_utf8(WCHAR x) return 1; else if (x < 0x800) return 2; - else + else return 3; } @@ -2303,7 +2306,7 @@ static WCHAR *latin1_to_utf16(char *str) return wstr; } -static char *utf16_to_utf8(WCHAR *wstr) +static char *utf16_to_utf8(WCHAR *wstr) { int len = wcslen(wstr); char *result; @@ -2322,5 +2325,5 @@ static char *utf16_to_utf8(WCHAR *wstr) result[pos] = '\0'; return result; } - + #endif diff --git a/erts/etc/common/heart.c b/erts/etc/common/heart.c index a37bb4cfef..215c152008 100644 --- a/erts/etc/common/heart.c +++ b/erts/etc/common/heart.c @@ -977,7 +977,7 @@ debugf(const char *format,...) #ifdef __WIN32__ void print_last_error() { - LPVOID lpMsgBuf; + LPTSTR lpMsgBuf; FormatMessage( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM, NULL, diff --git a/erts/etc/common/inet_gethost.c b/erts/etc/common/inet_gethost.c index 4fc7a93348..12012a056e 100644 --- a/erts/etc/common/inet_gethost.c +++ b/erts/etc/common/inet_gethost.c @@ -1578,8 +1578,7 @@ static int create_worker(Worker *pworker, int save_que) pworker->que_first = pworker->que_last = NULL; pworker->que_size = 0; } - DEBUGF(3,("Created worker[%ld] with fd %d", - (long) pworker->pid, (int) pworker->readfrom)); + DEBUGF(3,("Created worker[%ld]", (long) pworker->pid)); return 0; } diff --git a/erts/etc/unix/Install.src b/erts/etc/unix/Install.src index e4b842877c..2dbf628972 100644 --- a/erts/etc/unix/Install.src +++ b/erts/etc/unix/Install.src @@ -88,6 +88,7 @@ cd "$ERL_ROOT/bin" cp -p "$ERL_ROOT/erts-%I_VSN%/bin/erl" . cp -p "$ERL_ROOT/erts-%I_VSN%/bin/erlc" . +cp -p "$ERL_ROOT/erts-%I_VSN%/bin/erl_call" . cp -p "$ERL_ROOT/erts-%I_VSN%/bin/dialyzer" . cp -p "$ERL_ROOT/erts-%I_VSN%/bin/typer" . cp -p "$ERL_ROOT/erts-%I_VSN%/bin/ct_run" . diff --git a/erts/etc/win32/Install.c b/erts/etc/win32/Install.c index 06594a107f..c7bf96c124 100644 --- a/erts/etc/win32/Install.c +++ b/erts/etc/win32/Install.c @@ -47,7 +47,7 @@ int wmain(int argc, wchar_t **argv) InitFile *ini_file; InitSection *ini_section; HANDLE module = GetModuleHandle(NULL); - wchar_t *binaries[] = { L"erl.exe", L"werl.exe", L"erlc.exe", + wchar_t *binaries[] = { L"erl.exe", L"werl.exe", L"erlc.exe", L"erl_call.exe", L"dialyzer.exe", L"typer.exe", L"escript.exe", L"ct_run.exe", NULL }; diff --git a/erts/etc/win32/erl_log.c b/erts/etc/win32/erl_log.c index de0d8e39f0..ed2fb294de 100644 --- a/erts/etc/win32/erl_log.c +++ b/erts/etc/win32/erl_log.c @@ -57,7 +57,7 @@ main() static void print_last_error(char* message) { - LPTSTR* lpBufPtr; + LPTSTR lpBufPtr; FormatMessage( FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM, diff --git a/erts/etc/win32/erlsrv/erlsrv_interactive.c b/erts/etc/win32/erlsrv/erlsrv_interactive.c index c616ef86e3..9f1312b5e9 100644 --- a/erts/etc/win32/erlsrv/erlsrv_interactive.c +++ b/erts/etc/win32/erlsrv/erlsrv_interactive.c @@ -1273,7 +1273,7 @@ int interactive_main(int argc, wchar_t **argv){ _setmode(_fileno(stderr), _O_U8TEXT); /* set stderr to UTF8 */ if (take_lock() != 0) { - fwprintf(stderr,L"%s: unable to acquire global lock (%s).\n",argv[0], + fwprintf(stderr,L"%s: unable to acquire global lock (%S).\n",argv[0], ERLSRV_INTERACTIVE_GLOBAL_SEMAPHORE); return 1; } diff --git a/erts/etc/win32/nsis/erlang20.nsi b/erts/etc/win32/nsis/erlang20.nsi index 63e63a06cb..950f3ec0a3 100644 --- a/erts/etc/win32/nsis/erlang20.nsi +++ b/erts/etc/win32/nsis/erlang20.nsi @@ -52,7 +52,7 @@ Var STARTMENU_FOLDER !define MY_STARTMENUPAGE_REGISTRY_VALUENAME "Start Menu Folder"
;General
- OutFile "${OUTFILEDIR}\otp_${WINTYPE}_${OTP_VERSION}.exe"
+ OutFile "${OUTFILEDIR}\otp_${WINTYPE}_${OTP_RELEASE_VERSION}.exe"
;Folder selection page
!if ${WINTYPE} == "win64"
@@ -68,7 +68,7 @@ Var STARTMENU_FOLDER !if ${WINTYPE} == "win64"
!define MUI_STARTMENUPAGE_DEFAULTFOLDER "${OTP_PRODUCT} ${OTP_VERSION} (x64)"
!else
- !define MUI_STARTMENUPAGE_DEFAULTFOLDER "${OTP_PRODUCT} ${OTP_VERSION}"
+ !define MUI_STARTMENUPAGE_DEFAULTFOLDER "${OTP_PRODUCT} ${OTP_VERSION} (i386)"
!endif
;--------------------------------
@@ -201,7 +201,7 @@ done_startmenu: WriteRegStr HKLM \
"SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\Erlang OTP ${OTP_VERSION} (${ERTS_VERSION})" \
- "DisplayName" "Erlang OTP ${OTP_VERSION} (${ERTS_VERSION})"
+ "DisplayName" "Erlang OTP ${OTP_RELEASE_VERSION} (${ERTS_VERSION})"
WriteRegStr HKLM \
"SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\Erlang OTP ${OTP_VERSION} (${ERTS_VERSION})" \
"UninstallString" "$INSTDIR\Uninstall.exe"
@@ -225,7 +225,7 @@ done_startmenu: WriteRegStr HKCU \
"Software\Microsoft\Windows\CurrentVersion\Uninstall\Erlang OTP ${OTP_VERSION} (${ERTS_VERSION})" \
- "DisplayName" "Erlang OTP ${OTP_VERSION} (${ERTS_VERSION})"
+ "DisplayName" "Erlang OTP ${OTP_RELEASE_VERSION} (${ERTS_VERSION})"
WriteRegStr HKCU \
"Software\Microsoft\Windows\CurrentVersion\Uninstall\Erlang OTP ${OTP_VERSION} (${ERTS_VERSION})" \
"UninstallString" "$INSTDIR\Uninstall.exe"
diff --git a/erts/etc/win32/win_erlexec.c b/erts/etc/win32/win_erlexec.c index 39dac358e9..c0bb92793e 100644 --- a/erts/etc/win32/win_erlexec.c +++ b/erts/etc/win32/win_erlexec.c @@ -272,7 +272,7 @@ start_emulator(char* utf8emu, char *utf8start_prog, char** utf8argv, int start_d */ if (start_detached) { - int result; + HANDLE result; int i; wchar_t *start_prog=NULL; wchar_t **argv; @@ -311,17 +311,17 @@ start_emulator(char* utf8emu, char *utf8start_prog, char** utf8argv, int start_d MessageBoxW(NULL, buffer, L"Start detached",MB_OK); } #endif - result = _wspawnv(_P_DETACH, start_prog, argv); + result = (HANDLE) _wspawnv(_P_DETACH, start_prog, argv); free_fnuttified(utf8argv); free(start_prog); - if (result == -1) { + if (result == (HANDLE)-1) { #ifdef ARGS_HARDDEBUG MessageBox(NULL, "_wspawnv failed","Start detached",MB_OK); #endif return 1; } - SetPriorityClass((HANDLE) result, GetPriorityClass(GetCurrentProcess())); + SetPriorityClass(result, GetPriorityClass(GetCurrentProcess())); } else { wchar_t *emu=NULL; #ifdef LOAD_BEAM_DYNAMICALLY diff --git a/erts/etc/win32/wsl_tools/SetupWSLcross.bat b/erts/etc/win32/wsl_tools/SetupWSLcross.bat index 860a7d5e60..5de4cd8bb1 100644 --- a/erts/etc/win32/wsl_tools/SetupWSLcross.bat +++ b/erts/etc/win32/wsl_tools/SetupWSLcross.bat @@ -8,11 +8,6 @@ IF "%~1"=="x64" GOTO search GOTO badarg :search -IF EXIST "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat". ( - call "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" %~1 > nul - goto continue -) - IF EXIST "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat". ( call "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" %~1 > nul goto continue @@ -33,6 +28,11 @@ IF EXIST "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat". goto continue ) +IF EXIST "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat". ( + call "C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" %~1 > nul + goto continue +) + GOTO no_vcvars :continue diff --git a/erts/lib_src/yielding_c_fun/lib/simple_c_gc/simple_c_gc.c b/erts/lib_src/yielding_c_fun/lib/simple_c_gc/simple_c_gc.c index a6932ab99b..0eeb050282 100644 --- a/erts/lib_src/yielding_c_fun/lib/simple_c_gc/simple_c_gc.c +++ b/erts/lib_src/yielding_c_fun/lib/simple_c_gc/simple_c_gc.c @@ -90,7 +90,7 @@ static void *scgc_extract_key(void *v, int keyPos) { static unsigned int scgc_hash_key(void *keyPtr) { /* From * https://lemire.me/blog/2018/08/15/fast-strongly-universal-64-bit-hashing-everywhere*/ - int64_t x = (long)*((void **)keyPtr); + int64_t x = (intptr_t)*((void **)keyPtr); int64_t a = 2348923; int64_t b = 3292; int64_t c = 9893487421; @@ -117,7 +117,7 @@ static void scgc_initialize_global_state() { scgc_objects = ch_set_create(0, scgc_extract_key, scgc_hash_key, scgc_are_equal, scgc_to_string, scgc_malloc, scgc_free); - srand((int)(long)scgc_objects * 2654435761); + srand((int)(intptr_t)scgc_objects * 2654435761); } static void scgc_do_gc(bool no_stack); diff --git a/erts/lib_src/yielding_c_fun/ycf_yield_fun.c b/erts/lib_src/yielding_c_fun/ycf_yield_fun.c index 6260e5402e..a597df87c0 100644 --- a/erts/lib_src/yielding_c_fun/ycf_yield_fun.c +++ b/erts/lib_src/yielding_c_fun/ycf_yield_fun.c @@ -135,8 +135,8 @@ ycf_node* mk_yield_code(ycf_node* f_node, ycf_symbol_list ret_type = ycf_node_get_return_type(f_node); ret_code = ycf_string_new(" {\n" - " %s ycf_ret_value;\n" - " return ycf_ret_value;\n" + " static %s const ycf_unused_ret_value;\n" + " return ycf_unused_ret_value;\n" " }\n", ycf_symbol_list_to_str(&ret_type)); } diff --git a/erts/preloaded/ebin/erl_init.beam b/erts/preloaded/ebin/erl_init.beam Binary files differindex bcfc4d2e22..4f22f77a27 100644 --- a/erts/preloaded/ebin/erl_init.beam +++ b/erts/preloaded/ebin/erl_init.beam diff --git a/erts/preloaded/ebin/prim_net.beam b/erts/preloaded/ebin/prim_net.beam Binary files differindex 6e8073ae5a..510856e5c0 100644 --- a/erts/preloaded/ebin/prim_net.beam +++ b/erts/preloaded/ebin/prim_net.beam diff --git a/erts/preloaded/ebin/prim_socket.beam b/erts/preloaded/ebin/prim_socket.beam Binary files differnew file mode 100644 index 0000000000..7a76f69ae4 --- /dev/null +++ b/erts/preloaded/ebin/prim_socket.beam diff --git a/erts/preloaded/ebin/socket.beam b/erts/preloaded/ebin/socket.beam Binary files differdeleted file mode 100644 index e303eaa71d..0000000000 --- a/erts/preloaded/ebin/socket.beam +++ /dev/null diff --git a/erts/preloaded/src/Makefile b/erts/preloaded/src/Makefile index d2153f23e1..b0c205cec8 100644 --- a/erts/preloaded/src/Makefile +++ b/erts/preloaded/src/Makefile @@ -36,7 +36,7 @@ include $(ERL_TOP)/lib/kernel/vsn.mk ifeq ($(USE_ESOCK), yes) PRE_LOADED_ERL_ESOCK_MODULES = \ socket_registry \ - socket \ + prim_socket \ prim_net else PRE_LOADED_ERL_ESOCK_MODULES = @@ -82,7 +82,7 @@ APP_FILE= erts.app APP_SRC= $(APP_FILE).src APP_TARGET= $(STATIC_EBIN)/$(APP_FILE) ifeq ($(USE_ESOCK), yes) -APP_ESOCK_MODS= prim_net, socket, socket_registry +APP_ESOCK_MODS= prim_net, prim_socket, socket_registry, else APP_ESOCK_MODS= endif diff --git a/erts/preloaded/src/erl_init.erl b/erts/preloaded/src/erl_init.erl index dadf7dda6f..c19bebdd20 100644 --- a/erts/preloaded/src/erl_init.erl +++ b/erts/preloaded/src/erl_init.erl @@ -35,8 +35,8 @@ start(Mod, BootArgs) -> erl_tracer:on_load(), prim_buffer:on_load(), prim_file:on_load(), - %% socket:on_load(), prim_net:on_load(), - conditional_load(socket, [socket, prim_net]), + %% prim_socket:on_load(), prim_net:on_load(), + conditional_load(prim_socket, [prim_socket, prim_net]), %% Proceed to the specified boot module run(Mod, boot, BootArgs). diff --git a/erts/preloaded/src/erts.app.src b/erts/preloaded/src/erts.app.src index f8b2338c44..3a98f6f0f1 100644 --- a/erts/preloaded/src/erts.app.src +++ b/erts/preloaded/src/erts.app.src @@ -36,13 +36,13 @@ atomics, counters, persistent_term, - zlib, %ESOCK_MODS% + zlib ]}, {registered, []}, {applications, []}, {env, []}, - {runtime_dependencies, ["stdlib-@OTP-15251@", "kernel-@OTP-15251@", "sasl-3.3"]} + {runtime_dependencies, ["stdlib-3.13", "kernel-7.0", "sasl-3.3"]} ]}. %% vim: ft=erlang diff --git a/erts/preloaded/src/prim_net.erl b/erts/preloaded/src/prim_net.erl index fcd528bdaa..e82ae33b8a 100644 --- a/erts/preloaded/src/prim_net.erl +++ b/erts/preloaded/src/prim_net.erl @@ -157,15 +157,14 @@ gethostname() -> Info :: name_info(), Reason :: term(). -getnameinfo(SockAddr, [] = _Flags) -> - getnameinfo(SockAddr, undefined); -getnameinfo(#{family := Fam, addr := _Addr} = SockAddr, Flags) - when ((Fam =:= inet) orelse (Fam =:= inet6)) andalso - (is_list(Flags) orelse (Flags =:= undefined)) -> - nif_getnameinfo(socket:ensure_sockaddr(SockAddr), Flags); -getnameinfo(#{family := Fam, path := _Path} = SockAddr, Flags) - when (Fam =:= local) andalso (is_list(Flags) orelse (Flags =:= undefined)) -> - nif_getnameinfo(SockAddr, Flags). +getnameinfo(SockAddr, Flags) -> + try + ESockAddr = prim_socket:encode_sockaddr(SockAddr), + nif_getnameinfo(ESockAddr, Flags) + catch + throw : ERROR -> + ERROR + end. %% =========================================================================== diff --git a/erts/preloaded/src/prim_socket.erl b/erts/preloaded/src/prim_socket.erl new file mode 100644 index 0000000000..9639b606d0 --- /dev/null +++ b/erts/preloaded/src/prim_socket.erl @@ -0,0 +1,1344 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2018-2020. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(prim_socket). + +-compile(no_native). + +-export([on_load/0, on_load/1]). + +-export( + [encode_path/1, encode_sockaddr/1, + info/0, info/1, + debug/1, socket_debug/1, + supports/0, supports/1, supports/2, + open/2, open/4, + bind/2, bind/3, + connect/1, connect/2, + listen/2, + accept/2, + send/4, sendto/5, sendmsg/4, + recv/4, recvfrom/4, recvmsg/5, + close/1, finalize_close/1, + shutdown/2, setopt/4, getopt/3, + sockname/1, peername/1, + cancel/3 + ]). + +%% Also in socket +-define(REGISTRY, socket_registry). + + +%% =========================================================================== +%% +%% Defaults +%% + +-define(ESOCK_SEND_FLAGS_DEFAULT, []). +-define(ESOCK_SEND_TIMEOUT_DEFAULT, infinity). +-define(ESOCK_SENDTO_FLAGS_DEFAULT, []). +-define(ESOCK_SENDTO_TIMEOUT_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). +-define(ESOCK_SENDMSG_FLAGS_DEFAULT, []). +-define(ESOCK_SENDMSG_TIMEOUT_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). + +-define(ESOCK_RECV_FLAGS_DEFAULT, []). +-define(ESOCK_RECV_TIMEOUT_DEFAULT, infinity). + +-define(ESOCK_SOCKADDR_IN4_DEFAULTS, + (#{port => 0, addr => any})). +-define(ESOCK_SOCKADDR_IN6_DEFAULTS, + (#{port => 0, addr => any, + flowinfo => 0, scope_id => 0})). + +%% =========================================================================== +%% +%% Constants common to prim_socket_nif.c - has to be "identical" +%% + +-define(ESOCK_SEND_FLAG_CONFIRM, (1 bsl 0)). +-define(ESOCK_SEND_FLAG_DONTROUTE, (1 bsl 1)). +-define(ESOCK_SEND_FLAG_EOR, (1 bsl 2)). +-define(ESOCK_SEND_FLAG_MORE, (1 bsl 3)). +-define(ESOCK_SEND_FLAG_NOSIGNAL, (1 bsl 4)). +-define(ESOCK_SEND_FLAG_OOB, (1 bsl 5)). + +-define(ESOCK_RECV_FLAG_CMSG_CLOEXEC, (1 bsl 0)). +-define(ESOCK_RECV_FLAG_ERRQUEUE, (1 bsl 1)). +-define(ESOCK_RECV_FLAG_OOB, (1 bsl 2)). +-define(ESOCK_RECV_FLAG_PEEK, (1 bsl 3)). +-define(ESOCK_RECV_FLAG_TRUNC, (1 bsl 4)). + + +%% shutdown/2 +-define(ESOCK_SHUTDOWN_HOW_READ, 0). +-define(ESOCK_SHUTDOWN_HOW_WRITE, 1). +-define(ESOCK_SHUTDOWN_HOW_READ_WRITE, 2). + + + +%% ---------------------------------- +%% Address domain / protcol family + +-define(ESOCK_DOMAIN_LOCAL, 1). +-define(ESOCK_DOMAIN_UNIX, ?ESOCK_DOMAIN_LOCAL). +-define(ESOCK_DOMAIN_INET, 2). +-define(ESOCK_DOMAIN_INET6, 3). + +%% ---------------------------------- +%% Protocol type + +-define(ESOCK_TYPE_STREAM, 101). +-define(ESOCK_TYPE_DGRAM, 102). +-define(ESOCK_TYPE_RAW, 103). +%% -define(ESOCK_TYPE_RDM, 104). +-define(ESOCK_TYPE_SEQPACKET, 105). + +%% ---------------------------------- +%% Protocol + +-define(ESOCK_PROTOCOL_DEFAULT, 200). +-define(ESOCK_PROTOCOL_IP, 201). +-define(ESOCK_PROTOCOL_TCP, 202). +-define(ESOCK_PROTOCOL_UDP, 203). +-define(ESOCK_PROTOCOL_SCTP, 204). +-define(ESOCK_PROTOCOL_ICMP, 205). +-define(ESOCK_PROTOCOL_IGMP, 206). + +%% ---------------------------------- +%% Option level + +-define(ESOCK_OPT_LEVEL_OTP, 301). +-define(ESOCK_OPT_LEVEL_SOCKET, 302). +-define(ESOCK_OPT_LEVEL_IP, 303). +-define(ESOCK_OPT_LEVEL_IPV6, 304). +-define(ESOCK_OPT_LEVEL_TCP, 305). +-define(ESOCK_OPT_LEVEL_UDP, 306). +-define(ESOCK_OPT_LEVEL_SCTP, 307). + +%% ---------------------------------- +%% *** OTP (socket) options + +-define(ESOCK_OPT_OTP_DEBUG, 1001). +-define(ESOCK_OPT_OTP_IOW, 1002). +-define(ESOCK_OPT_OTP_CTRL_PROC, 1003). +-define(ESOCK_OPT_OTP_RCVBUF, 1004). +%%-define(ESOCK_OPT_OTP_SNDBUF, 1005). +-define(ESOCK_OPT_OTP_RCVCTRLBUF, 1006). +-define(ESOCK_OPT_OTP_SNDCTRLBUF, 1007). +-define(ESOCK_OPT_OTP_FD, 1008). +-define(ESOCK_OPT_OTP_META, 1009). +%% +-define(ESOCK_OPT_OTP_DOMAIN, 1999). % INTERNAL +%%-define(ESOCK_OPT_OTP_TYPE, 1998). % INTERNAL +%%-define(ESOCK_OPT_OTP_PROTOCOL, 1997). % INTERNAL +%%-define(ESOCK_OPT_OTP_DTP, 1996). % INTERNAL + +%% ---------------------------------- +%% *** SOCKET (socket) options + +-define(ESOCK_OPT_SOCK_ACCEPTCONN, 2001). +-define(ESOCK_OPT_SOCK_ACCEPTFILTER, 2002). % FreeBSD +-define(ESOCK_OPT_SOCK_BINDTODEVICE, 2003). +-define(ESOCK_OPT_SOCK_BROADCAST, 2004). +-define(ESOCK_OPT_SOCK_BUSY_POLL, 2005). +-define(ESOCK_OPT_SOCK_DEBUG, 2006). +-define(ESOCK_OPT_SOCK_DOMAIN, 2007). +-define(ESOCK_OPT_SOCK_DONTROUTE, 2008). +-define(ESOCK_OPT_SOCK_ERROR, 2009). +-define(ESOCK_OPT_SOCK_KEEPALIVE, 2010). +-define(ESOCK_OPT_SOCK_LINGER, 2011). +-define(ESOCK_OPT_SOCK_MARK, 2012). +-define(ESOCK_OPT_SOCK_OOBINLINE, 2013). +-define(ESOCK_OPT_SOCK_PASSCRED, 2014). +-define(ESOCK_OPT_SOCK_PEEK_OFF, 2015). +-define(ESOCK_OPT_SOCK_PEERCRED, 2016). +-define(ESOCK_OPT_SOCK_PRIORITY, 2017). +-define(ESOCK_OPT_SOCK_PROTOCOL, 2018). +-define(ESOCK_OPT_SOCK_RCVBUF, 2019). +-define(ESOCK_OPT_SOCK_RCVBUFFORCE, 2020). +-define(ESOCK_OPT_SOCK_RCVLOWAT, 2021). +-define(ESOCK_OPT_SOCK_RCVTIMEO, 2022). +-define(ESOCK_OPT_SOCK_REUSEADDR, 2023). +-define(ESOCK_OPT_SOCK_REUSEPORT, 2024). +-define(ESOCK_OPT_SOCK_RXQ_OVFL, 2025). +-define(ESOCK_OPT_SOCK_SETFIB, 2026). % FreeBSD +-define(ESOCK_OPT_SOCK_SNDBUF, 2027). +-define(ESOCK_OPT_SOCK_SNDBUFFORCE, 2028). +-define(ESOCK_OPT_SOCK_SNDLOWAT, 2029). +-define(ESOCK_OPT_SOCK_SNDTIMEO, 2030). +-define(ESOCK_OPT_SOCK_TIMESTAMP, 2031). +-define(ESOCK_OPT_SOCK_TYPE, 2032). + +%% ---------------------------------- +%% *** IP (socket) options + +-define(ESOCK_OPT_IP_ADD_MEMBERSHIP, 3001). +-define(ESOCK_OPT_IP_ADD_SOURCE_MEMBERSHIP, 3002). +-define(ESOCK_OPT_IP_BLOCK_SOURCE, 3003). +-define(ESOCK_OPT_IP_DONTFRAG, 3004). % FreeBSD +-define(ESOCK_OPT_IP_DROP_MEMBERSHIP, 3005). +-define(ESOCK_OPT_IP_DROP_SOURCE_MEMBERSHIP,3006). +-define(ESOCK_OPT_IP_FREEBIND, 3007). +-define(ESOCK_OPT_IP_HDRINCL, 3008). +-define(ESOCK_OPT_IP_MINTTL, 3009). +-define(ESOCK_OPT_IP_MSFILTER, 3010). +-define(ESOCK_OPT_IP_MTU, 3011). +-define(ESOCK_OPT_IP_MTU_DISCOVER, 3012). +-define(ESOCK_OPT_IP_MULTICAST_ALL, 3013). +-define(ESOCK_OPT_IP_MULTICAST_IF, 3014). +-define(ESOCK_OPT_IP_MULTICAST_LOOP, 3015). +-define(ESOCK_OPT_IP_MULTICAST_TTL, 3016). +-define(ESOCK_OPT_IP_NODEFRAG, 3017). +-define(ESOCK_OPT_IP_OPTIONS, 3018). % FreeBSD +-define(ESOCK_OPT_IP_PKTINFO, 3019). +-define(ESOCK_OPT_IP_RECVDSTADDR, 3020). % FreeBSD +-define(ESOCK_OPT_IP_RECVERR, 3021). +-define(ESOCK_OPT_IP_RECVIF, 3022). +-define(ESOCK_OPT_IP_RECVOPTS, 3023). +-define(ESOCK_OPT_IP_RECVORIGDSTADDR, 3024). +-define(ESOCK_OPT_IP_RECVTOS, 3025). +-define(ESOCK_OPT_IP_RECVTTL, 3026). +-define(ESOCK_OPT_IP_RETOPTS, 3027). +-define(ESOCK_OPT_IP_ROUTER_ALERT, 3028). +-define(ESOCK_OPT_IP_SENDSRCADDR, 3029). % FreeBSD +-define(ESOCK_OPT_IP_TOS, 3030). +-define(ESOCK_OPT_IP_TRANSPARENT, 3031). +-define(ESOCK_OPT_IP_TTL, 3032). +-define(ESOCK_OPT_IP_UNBLOCK_SOURCE, 3033). + +%% ---------------------------------- +%% *** IPv6 (socket) options + +-define(ESOCK_OPT_IPV6_ADDRFORM, 4001). +-define(ESOCK_OPT_IPV6_ADD_MEMBERSHIP, 4002). +-define(ESOCK_OPT_IPV6_AUTHHDR, 4003). % Obsolete? +-define(ESOCK_OPT_IPV6_AUTH_LEVEL, 4004). % FreeBSD +-define(ESOCK_OPT_IPV6_CHECKSUM, 4005). % FreeBSD +-define(ESOCK_OPT_IPV6_DROP_MEMBERSHIP, 4006). +-define(ESOCK_OPT_IPV6_DSTOPTS, 4007). +-define(ESOCK_OPT_IPV6_ESP_NETWORK_LEVEL, 4008). % FreeBSD +-define(ESOCK_OPT_IPV6_ESP_TRANS_LEVEL, 4009). % FreeBSD +-define(ESOCK_OPT_IPV6_FAITH, 4010). % FreeBSD +-define(ESOCK_OPT_IPV6_FLOWINFO, 4011). +-define(ESOCK_OPT_IPV6_HOPLIMIT, 4012). +-define(ESOCK_OPT_IPV6_HOPOPTS, 4013). +-define(ESOCK_OPT_IPV6_IPCOMP_LEVEL, 4014). % FreeBSD +-define(ESOCK_OPT_IPV6_JOIN_GROUP, 4015). % FreeBSD +-define(ESOCK_OPT_IPV6_LEAVE_GROUP, 4016). % FreeBSD +-define(ESOCK_OPT_IPV6_MTU, 4017). +-define(ESOCK_OPT_IPV6_MTU_DISCOVER, 4018). +-define(ESOCK_OPT_IPV6_MULTICAST_HOPS, 4019). +-define(ESOCK_OPT_IPV6_MULTICAST_IF, 4020). +-define(ESOCK_OPT_IPV6_MULTICAST_LOOP, 4021). +-define(ESOCK_OPT_IPV6_PORTRANGE, 4022). % FreeBSD +-define(ESOCK_OPT_IPV6_PKTOPTIONS, 4023). % FreeBSD +-define(ESOCK_OPT_IPV6_RECVERR, 4024). +-define(ESOCK_OPT_IPV6_RECVHOPLIMIT, 4025). +-define(ESOCK_OPT_IPV6_RECVPKTINFO, 4026). % On FreeBSD: PKTINFO +-define(ESOCK_OPT_IPV6_RECVTCLASS, 4027). +-define(ESOCK_OPT_IPV6_ROUTER_ALERT, 4028). +-define(ESOCK_OPT_IPV6_RTHDR, 4029). +-define(ESOCK_OPT_IPV6_TCLASS, 4030). % FreeBSD +-define(ESOCK_OPT_IPV6_UNICAST_HOPS, 4031). +-define(ESOCK_OPT_IPV6_USE_MIN_MTU, 4032). % FreeBSD +-define(ESOCK_OPT_IPV6_V6ONLY, 4033). + +%% ---------------------------------- +%% *** TCP (socket) options + +-define(ESOCK_OPT_TCP_CONGESTION, 5001). +-define(ESOCK_OPT_TCP_CORK, 5002). +-define(ESOCK_OPT_TCP_INFO, 5003). +-define(ESOCK_OPT_TCP_KEEPCNT, 5004). +-define(ESOCK_OPT_TCP_KEEPIDLE, 5005). +-define(ESOCK_OPT_TCP_KEEPINTVL, 5006). +-define(ESOCK_OPT_TCP_MAXSEG, 5007). +-define(ESOCK_OPT_TCP_MD5SIG, 5008). +-define(ESOCK_OPT_TCP_NODELAY, 5009). +-define(ESOCK_OPT_TCP_NOOPT, 5010). +-define(ESOCK_OPT_TCP_NOPUSH, 5011). +-define(ESOCK_OPT_TCP_SYNCNT, 5012). +-define(ESOCK_OPT_TCP_USER_TIMEOUT, 5013). + +%% ---------------------------------- +%% *** UDP (socket) options + +-define(ESOCK_OPT_UDP_CORK, 6001). + +%% ---------------------------------- +%% *** SCTP (socket) options + +-define(ESOCK_OPT_SCTP_ADAPTION_LAYER, 7001). +-define(ESOCK_OPT_SCTP_ASSOCINFO, 7002). +-define(ESOCK_OPT_SCTP_AUTH_ACTIVE_KEY, 7003). +-define(ESOCK_OPT_SCTP_AUTH_ASCONF, 7004). +-define(ESOCK_OPT_SCTP_AUTH_CHUNK, 7005). +-define(ESOCK_OPT_SCTP_AUTH_KEY, 7006). +-define(ESOCK_OPT_SCTP_AUTH_DELETE_KEY, 7007). +-define(ESOCK_OPT_SCTP_AUTOCLOSE, 7008). +-define(ESOCK_OPT_SCTP_CONTEXT, 7009). +-define(ESOCK_OPT_SCTP_DEFAULT_SEND_PARAMS, 7010). +-define(ESOCK_OPT_SCTP_DELAYED_ACK_TIME, 7011). +-define(ESOCK_OPT_SCTP_DISABLE_FRAGMENTS, 7012). +-define(ESOCK_OPT_SCTP_HMAC_IDENT, 7013). +-define(ESOCK_OPT_SCTP_EVENTS, 7014). +-define(ESOCK_OPT_SCTP_EXPLICIT_EOR, 7015). +-define(ESOCK_OPT_SCTP_FRAGMENT_INTERLEAVE, 7016). +-define(ESOCK_OPT_SCTP_GET_PEER_ADDR_INFO, 7017). +-define(ESOCK_OPT_SCTP_INITMSG, 7018). +-define(ESOCK_OPT_SCTP_I_WANT_MAPPED_V4_ADDR, 7019). +-define(ESOCK_OPT_SCTP_LOCAL_AUTH_CHUNKS, 7020). +-define(ESOCK_OPT_SCTP_MAXSEG, 7021). +-define(ESOCK_OPT_SCTP_MAXBURST, 7022). +-define(ESOCK_OPT_SCTP_NODELAY, 7023). +-define(ESOCK_OPT_SCTP_PARTIAL_DELIVERY_POINT, 7024). +-define(ESOCK_OPT_SCTP_PEER_ADDR_PARAMS, 7025). +-define(ESOCK_OPT_SCTP_PEER_AUTH_CHUNKS, 7026). +-define(ESOCK_OPT_SCTP_PRIMARY_ADDR, 7027). +-define(ESOCK_OPT_SCTP_RESET_STREAMS, 7028). +-define(ESOCK_OPT_SCTP_RTOINFO, 7029). +-define(ESOCK_OPT_SCTP_SET_PEER_PRIMARY_ADDR, 7030). +-define(ESOCK_OPT_SCTP_STATUS, 7031). +-define(ESOCK_OPT_SCTP_USE_EXT_RECVINFO, 7032). + + +%% =========================================================================== +%% +%% Guard macros +%% + +%% Check that there are 1:s just in the lowest 8 bits of all 4 elements +-define( + IS_IPV4_ADDR(A), + (is_tuple(A) andalso tuple_size(A) =:= 4 + andalso + ((element(1, (A)) bor element(2, (A)) + bor element(3, (A)) bor element(4, (A)) + ) band (bnot 16#FF) + ) =:= 0)). + +%% Check that there are 1:s just in the lowest 16 bits of all 8 elements +-define( + IS_IPV6_ADDR(A), + (is_tuple(A) andalso tuple_size(A) =:= 8 + andalso + ((element(1, (A)) bor element(2, (A)) + bor element(3, (A)) bor element(4, (A)) + bor element(5, (A)) bor element(6, (A)) + bor element(7, (A)) bor element(8, (A)) + ) band (bnot 16#FFFF) + ) =:= 0)). + +%% =========================================================================== +%% API for 'erl_init' +%% + +on_load() -> + on_load(#{}). + +on_load(Extra) when is_map(Extra) -> + %% This is spawned as a system process to prevent init:restart/0 from + %% killing it. + Pid = erts_internal:spawn_system_process(?REGISTRY, start, []), + DebugFilename = + case os:get_env_var("ESOCK_DEBUG_FILENAME") of + "*" -> + "/tmp/esock-dbg-??????"; + F -> + F + end, + ok = + erlang:load_nif( + atom_to_list(?MODULE), + case DebugFilename of + false -> + Extra#{registry => Pid}; + _ -> + Extra + #{registry => Pid, + debug => true, + socket_debug => true, + debug_filename => + encode_path(DebugFilename)} + end). + +%% =========================================================================== +%% API for 'socket' +%% + +%% File names has to be encoded according to +%% the native file encoding +%% +encode_path(Path) -> + %% These are all BIFs - will not cause code loading + unicode:characters_to_binary(Path, file:native_name_encoding()). + +encode_sockaddr(SockAddr) -> + enc_sockaddr(SockAddr). + +%% ---------------------------------- + +info() -> + nif_info(). + +info(SockRef) -> + nif_info(SockRef). + +%% ---------------------------------- + +debug(D) -> + nif_command(#{command => ?FUNCTION_NAME, data => D}). + + +socket_debug(D) -> + nif_command(#{command => ?FUNCTION_NAME, data => D}). + +%% ---------------------------------- + +supports() -> + nif_supports(). + +supports(Key) -> + nif_supports(Key). + +supports(options = Key, Level) -> + case Level of + socket -> + nif_supports(Key, ?ESOCK_OPT_LEVEL_SOCKET); + ip -> + nif_supports(Key, ?ESOCK_OPT_LEVEL_IP); + ipv6 -> + nif_supports(Key, ?ESOCK_OPT_LEVEL_IPV6); + tcp -> + nif_supports(Key, ?ESOCK_OPT_LEVEL_TCP); + udp -> + nif_supports(Key, ?ESOCK_OPT_LEVEL_UDP); + sctp -> + nif_supports(Key, ?ESOCK_OPT_LEVEL_SCTP); + _ -> + [] + end; +supports(_Key1, _Key2) -> + []. + +%% ---------------------------------- + +open(FD, Opts) when is_map(Opts) -> + EOpts = + maps:map( + fun (Key, Val) -> + case Key of + domain -> + enc_domain(Val); + type -> + enc_type(Val); + protocol -> + enc_protocol(Val); + _ -> + Val + end + end, Opts), + nif_open(FD, EOpts). + +open(Domain, Type, Protocol, Opts) when is_map(Opts) -> + EDomain = enc_domain(Domain), + EType = enc_type(Type), + EProtocol = enc_protocol(Protocol), + EOpts = + case Opts of + #{netns := Path} when is_list(Path) -> + Opts#{netns := encode_path(Path)}; + _ -> + Opts + end, + nif_open(EDomain, EType, EProtocol, EOpts). + +%% ---------------------------------- + +bind(SockRef, Addr) -> + nif_bind(SockRef, enc_sockaddr(Addr)). + +bind(SockRef, Addrs, Action) when is_list(Addrs) -> + EAddrs = [enc_sockaddr(Addr) || Addr <- Addrs], + nif_bind(SockRef, EAddrs, Action). + +%% ---------------------------------- + +connect(SockRef, SockAddr) -> + nif_connect(SockRef, enc_sockaddr(SockAddr)). + +connect(SockRef) -> + nif_connect(SockRef). + +%% ---------------------------------- + +listen(SockRef, Backlog) -> + nif_listen(SockRef, Backlog). + +%% ---------------------------------- + +accept(ListenSockRef, AccRef) -> + case nif_accept(ListenSockRef, AccRef) of + {ok, _SockRef} = Result -> + Result; + {error, eagain} -> + select; + {error, _} = Result -> + Result + end. + +%% ---------------------------------- + +send(SockRef, SendRef, Data, Flags) -> + EFlags = enc_send_flags(Flags), + send_result( + nif_send(SockRef, SendRef, Data, EFlags)). + +sendto(SockRef, SendRef, Data, To, Flags) -> + ETo = enc_sockaddr(To), + EFlags = enc_send_flags(Flags), + send_result( + nif_sendto(SockRef, SendRef, Data, ETo, EFlags)). + +sendmsg(SockRef, SendRef, MsgHdr, Flags) -> + EMsgHdr = enc_msghdr(MsgHdr), + EFlags = enc_send_flags(Flags), + send_result( + nif_sendmsg(SockRef, SendRef, EMsgHdr, EFlags)). + +send_result(Result) -> + case Result of + ok -> ok; + {ok, _Written} = OK -> OK; + {error, eagain} -> select; + {error, _} = ERROR -> ERROR + end. + +%% ---------------------------------- + +recv(SockRef, RecvRef, Length, Flags) -> + EFlags = enc_recv_flags(Flags), + case nif_recv(SockRef, RecvRef, Length, EFlags) of + {ok, true, Bin} -> + {ok, Bin}; + %% + {ok, false, Bin} -> + %% Depending on the number of bytes we tried to read: + if + Length =:= 0 -> + %% 0 - Read everything available + %% We got something, but there may be more + %% - keep reading. + {more, Bin}; + true -> + %% > 0 - We got a part of the message + %% and we will be notified when there is more to read + %% (a select message) + {select, Bin} + end; + %% + {error, eagain} -> + select; + {error, _} = ERROR -> + ERROR + end. + +recvfrom(SockRef, RecvRef, Length, Flags) -> + EFlags = enc_recv_flags(Flags), + recvfromsg_result( + nif_recvfrom(SockRef, RecvRef, Length, EFlags)). + +recvmsg(SockRef, RecvRef, BufSz, CtrlSz, Flags) -> + EFlags = enc_recv_flags(Flags), + recvfromsg_result( + nif_recvmsg(SockRef, RecvRef, BufSz, CtrlSz, EFlags)). + +recvfromsg_result(Result) -> + case Result of + {ok, _} = OK -> OK; + {error, eagain} -> select; + {error, _} = ERROR -> ERROR + end. + +%% ---------------------------------- + +close(SockRef) -> + nif_close(SockRef). + +finalize_close(SockRef) -> + nif_finalize_close(SockRef). + +%% ---------------------------------- + +shutdown(SockRef, How) -> + nif_shutdown(SockRef, enc_shutdown_how(How)). + +%% ---------------------------------- + +setopt(SockRef, Level, Opt, Val) + when is_integer(Level), is_integer(Opt), is_binary(Val) -> + nif_setopt(SockRef, false, Level, Opt, Val); +setopt(SockRef, Level, Opt, Val) + when is_integer(Opt), is_binary(Val) -> + ELevel = enc_sockopt_type(Level, []), + nif_setopt(SockRef, true, ELevel, Opt, Val); +setopt(SockRef, Level, Opt, Value) when Opt =/= [] -> + case enc_sockopt_type(Level, Opt) of + {undefined, _ELevel, _EOpt} -> + {error, einval}; + {Type, ELevel, EOpt} -> + EValue = enc_setopt_value(Level, Opt, Value, Type), + nif_setopt(SockRef, true, ELevel, EOpt, EValue) + end. + +getopt(SockRef, Level, Opt) + when is_integer(Level) -> + nif_getopt(SockRef, false, Level, Opt); +getopt(SockRef, Level, Opt) when is_atom(Opt) -> + {_Type, ELevel, EOpt} = enc_sockopt_type(Level, Opt), + case nif_getopt(SockRef, true, ELevel, EOpt) of + {ok, Value} -> + {ok, dec_getopt_value(Value, Level, Opt)}; + {error, _} = Error -> + Error + end; +getopt(SockRef, Level, Opt) -> + ELevel = enc_sockopt_type(Level, []), + nif_getopt(SockRef, true, ELevel, Opt). + +%% ---------------------------------- + +sockname(Ref) -> + nif_sockname(Ref). + +peername(Ref) -> + nif_peername(Ref). + +%% ---------------------------------- + +cancel(SRef, Op, Ref) -> + nif_cancel(SRef, Op, Ref). + +%% =========================================================================== +%% Encode / decode +%% + +enc_domain(local) -> ?ESOCK_DOMAIN_LOCAL; +enc_domain(inet) -> ?ESOCK_DOMAIN_INET; +enc_domain(inet6) -> ?ESOCK_DOMAIN_INET6; +enc_domain(Domain) -> invalid(domain, Domain). + +enc_type(stream) -> ?ESOCK_TYPE_STREAM; +enc_type(dgram) -> ?ESOCK_TYPE_DGRAM; +enc_type(raw) -> ?ESOCK_TYPE_RAW; +enc_type(seqpacket) -> ?ESOCK_TYPE_SEQPACKET; +enc_type(Type) -> invalid(type, Type). + +enc_protocol(default) -> ?ESOCK_PROTOCOL_DEFAULT; +enc_protocol(ip) -> ?ESOCK_PROTOCOL_IP; +enc_protocol(tcp) -> ?ESOCK_PROTOCOL_TCP; +enc_protocol(udp) -> ?ESOCK_PROTOCOL_UDP; +enc_protocol(sctp) -> ?ESOCK_PROTOCOL_SCTP; +enc_protocol(icmp) -> ?ESOCK_PROTOCOL_ICMP; +enc_protocol(igmp) -> ?ESOCK_PROTOCOL_IGMP; +enc_protocol({raw, P} = RAW) when is_integer(P) -> RAW; +enc_protocol(Proto) -> + invalid(protocol, Proto). + + +enc_sockaddr(#{family := inet} = SockAddr) -> + maps:merge(?ESOCK_SOCKADDR_IN4_DEFAULTS, SockAddr); +enc_sockaddr(#{family := inet6} = SockAddr) -> + maps:merge(?ESOCK_SOCKADDR_IN6_DEFAULTS, SockAddr); +enc_sockaddr(#{family := local, path := Path} = SockAddr) + when is_list(Path) andalso + (length(Path) > 0) andalso + (length(Path) =< 255) -> + BinPath = encode_path(Path), + enc_sockaddr(SockAddr#{path => BinPath}); +enc_sockaddr(#{family := local, path := Path} = SockAddr) + when is_binary(Path) andalso + (byte_size(Path) > 0) andalso + (byte_size(Path) =< 255) -> + SockAddr; +enc_sockaddr(SockAddr) -> + invalid(address, SockAddr). + + +enc_send_flags([]) -> 0; +enc_send_flags([Flag | Flags]) -> + case Flag of + confirm -> ?ESOCK_SEND_FLAG_CONFIRM; + dontroute -> ?ESOCK_SEND_FLAG_DONTROUTE; + eor -> ?ESOCK_SEND_FLAG_EOR; + more -> ?ESOCK_SEND_FLAG_MORE; + nosignal -> ?ESOCK_SEND_FLAG_NOSIGNAL; + oob -> ?ESOCK_SEND_FLAG_OOB; + _ -> invalid(flag, Flag) + end bor enc_send_flags(Flags). + + +enc_recv_flags([]) -> 0; +enc_recv_flags([Flag | Flags]) -> + case Flag of + cmsg_cloexec -> ?ESOCK_RECV_FLAG_CMSG_CLOEXEC; + errqueue -> ?ESOCK_RECV_FLAG_ERRQUEUE; + oob -> ?ESOCK_RECV_FLAG_OOB; + peek -> ?ESOCK_RECV_FLAG_PEEK; + trunc -> ?ESOCK_RECV_FLAG_TRUNC; + _ -> invalid(flag, Flag) + end bor enc_recv_flags(Flags). + + +enc_msghdr(#{ctrl := []} = M) -> + enc_msghdr(maps:remove(ctrl, M)); +enc_msghdr(#{iov := IOV, addr := Addr} = M) + when is_list(IOV), IOV =/= [] -> + M#{iov => erlang:iolist_to_iovec(IOV), + addr => enc_sockaddr(Addr)}; +enc_msghdr(#{iov := IOV} = M) + when is_list(IOV), IOV =/= [] -> + M#{iov => erlang:iolist_to_iovec(IOV)}; +enc_msghdr(#{} = M) -> + not_supported(M). + + +%% Common to setopt and getopt +%% - the returned first tuple element is a type tag +%% that is used by setopt through enc_setopt_value/4 +%% and ignored by getopt; +%% 'undefined' means that encoding for setopt/4 +%% cannot be done and setopt/4 returns {error, einval}. +%% +%% Opt values not handled here will be passed to not_supported/1,2 +%% which causes a runtime error. +%% +%% Values that fails encoding in enc_setopt_value/4 +%% will be passed to not_supported/3. +%% +%% So, all combinations of Level and Opt enumerated here +%% will either return {error, einval} because they can not +%% be encoded for setopt/4, or they are passed to +%% the setopt or getopt NIF. The NIF will return +%% {error, einval} for unknown Level and Opt. +%% +%% Therefore all Level and Opt enumerated here will produce +%% a return value from setopt/4 and getopt/3 and should +%% be in the type spec for socket:setopt() and socket:getopt(). +%% +%% Opt =:= [] just encodes the Level +%% +enc_sockopt_type(otp = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_OTP, + case Opt of + [] -> L; + %% + debug -> {boolean, L, ?ESOCK_OPT_OTP_DEBUG}; + iow -> {boolean, L, ?ESOCK_OPT_OTP_IOW}; + controlling_process -> {pid, L, ?ESOCK_OPT_OTP_CTRL_PROC}; + rcvbuf -> {Opt, L, ?ESOCK_OPT_OTP_RCVBUF}; + rcvctrlbuf -> {buf, L, ?ESOCK_OPT_OTP_RCVCTRLBUF}; + sndctrlbuf -> {buf, L, ?ESOCK_OPT_OTP_SNDCTRLBUF}; + fd -> {undefined, L, ?ESOCK_OPT_OTP_FD}; + meta -> {term, L, ?ESOCK_OPT_OTP_META}; + domain -> {undefined, L, ?ESOCK_OPT_OTP_DOMAIN}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(socket = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_SOCKET, + case Opt of + [] -> L; + %% + acceptconn -> {undefined, L, ?ESOCK_OPT_SOCK_ACCEPTCONN}; + acceptfilter -> {undefined, L, ?ESOCK_OPT_SOCK_ACCEPTFILTER}; + bindtodevice -> + %% Before linux 3.8, this socket option + %% could be set but not get. + %% Maximum size of buffer for name: IFNAMSIZ + %% So, we let the implementation decide. + {list, L, ?ESOCK_OPT_SOCK_BINDTODEVICE}; + broadcast -> {boolean, L, ?ESOCK_OPT_SOCK_BROADCAST}; + busy_poll -> {undefined, L, ?ESOCK_OPT_SOCK_BUSY_POLL}; + debug -> {integer, L, ?ESOCK_OPT_SOCK_DEBUG}; + domain -> {undefined, L, ?ESOCK_OPT_SOCK_DOMAIN}; + dontroute -> {boolean, L, ?ESOCK_OPT_SOCK_DONTROUTE}; + error -> {undefined, L, ?ESOCK_OPT_SOCK_ERROR}; + keepalive -> {boolean, L, ?ESOCK_OPT_SOCK_KEEPALIVE}; + linger -> {linger, L, ?ESOCK_OPT_SOCK_LINGER}; + mark -> {undefined, L, ?ESOCK_OPT_SOCK_MARK}; + oobinline -> {boolean, L, ?ESOCK_OPT_SOCK_OOBINLINE}; + passcred -> {boolean, L, ?ESOCK_OPT_SOCK_PASSCRED}; + peek_off -> {integer, L, ?ESOCK_OPT_SOCK_PEEK_OFF}; + peercred -> {undefined, L, ?ESOCK_OPT_SOCK_PEERCRED}; + priority -> {integer, L, ?ESOCK_OPT_SOCK_PRIORITY}; + protocol -> {undefined, L, ?ESOCK_OPT_SOCK_PROTOCOL}; + rcvbuf -> {integer, L, ?ESOCK_OPT_SOCK_RCVBUF}; + rcvbufforce -> {undefined, L, ?ESOCK_OPT_SOCK_RCVBUFFORCE}; + rcvlowat -> + %% May not work on Linux + {integer, L, ?ESOCK_OPT_SOCK_RCVLOWAT}; + rcvtimeo -> {timeval, L, ?ESOCK_OPT_SOCK_RCVTIMEO}; + reuseaddr -> {boolean, L, ?ESOCK_OPT_SOCK_REUSEADDR}; + reuseport -> {boolean, L, ?ESOCK_OPT_SOCK_REUSEPORT}; + rxq_ovfl -> {undefined, L, ?ESOCK_OPT_SOCK_RXQ_OVFL}; + setfib -> {undefined, L, ?ESOCK_OPT_SOCK_SETFIB}; + sndbuf -> {integer, L, ?ESOCK_OPT_SOCK_SNDBUF}; + sndbufforce -> {undefined, L, ?ESOCK_OPT_SOCK_SNDBUFFORCE}; + sndlowat -> + %% Not changeable on Linux + {integer, L, ?ESOCK_OPT_SOCK_SNDLOWAT}; + sndtimeo -> {timeval, L, ?ESOCK_OPT_SOCK_SNDTIMEO}; + timestamp -> {boolean, L, ?ESOCK_OPT_SOCK_TIMESTAMP}; + type -> {undefined, L, ?ESOCK_OPT_SOCK_TYPE}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(ip = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_IP, + case Opt of + [] -> L; + %% + add_membership -> + {addr_if, L, ?ESOCK_OPT_IP_ADD_MEMBERSHIP}; + add_source_membership -> + {addr_if_src, L, ?ESOCK_OPT_IP_ADD_SOURCE_MEMBERSHIP}; + block_source -> + {addr_if_src, L, ?ESOCK_OPT_IP_BLOCK_SOURCE}; + dontfrag -> + %% FreeBSD only? + %% Only respected on udp and raw ip + %% (unless the hdrincl option has been set) + {boolean, L, ?ESOCK_OPT_IP_DONTFRAG}; + drop_membership -> + {addr_if, L, ?ESOCK_OPT_IP_DROP_MEMBERSHIP}; + drop_source_membership -> + {addr_if_src, L, ?ESOCK_OPT_IP_DROP_SOURCE_MEMBERSHIP}; + freebind -> + %% Linux only? + {boolean, L, ?ESOCK_OPT_IP_FREEBIND}; + hdrincl -> + {boolean, L, ?ESOCK_OPT_IP_HDRINCL}; + minttl -> + {integer, L, ?ESOCK_OPT_IP_MINTTL}; + msfilter -> {Opt, L, ?ESOCK_OPT_IP_MSFILTER}; + mtu -> {undefined, L, ?ESOCK_OPT_IP_MTU}; + mtu_discover -> {Opt, L, ?ESOCK_OPT_IP_MTU_DISCOVER}; + multicast_all -> + {boolean, L, ?ESOCK_OPT_IP_MULTICAST_ALL}; + multicast_if -> + {boolean, L, ?ESOCK_OPT_IP_MULTICAST_ALL}; + multicast_loop -> + {boolean, L, ?ESOCK_OPT_IP_MULTICAST_LOOP}; + multicast_ttl -> + {byte, L, ?ESOCK_OPT_IP_MULTICAST_TTL}; + nodefrag -> + {boolean, L, ?ESOCK_OPT_IP_NODEFRAG}; + options -> + {undefined, L, ?ESOCK_OPT_IP_OPTIONS}; + pktinfo -> + {boolean, L, ?ESOCK_OPT_IP_PKTINFO}; + recvdstaddr -> + {boolean, L, ?ESOCK_OPT_IP_RECVDSTADDR}; + recverr -> + {boolean, L, ?ESOCK_OPT_IP_RECVERR}; + recvif -> + {boolean, L, ?ESOCK_OPT_IP_RECVIF}; + recvopts -> + {boolean, L, ?ESOCK_OPT_IP_RECVOPTS}; + recvorigdstaddr -> + {boolean, L, ?ESOCK_OPT_IP_RECVORIGDSTADDR}; + recvtos -> + {boolean, L, ?ESOCK_OPT_IP_RECVTOS}; + recvttl -> + {boolean, L, ?ESOCK_OPT_IP_RECVTTL}; + retopts -> + {boolean, L, ?ESOCK_OPT_IP_RETOPTS}; + router_alert -> + {integer, L, ?ESOCK_OPT_IP_ROUTER_ALERT}; + sendsrcaddr -> + {boolean, L, ?ESOCK_OPT_IP_SENDSRCADDR}; + tos -> + %% On FreeBSD it specifies that this option is only valid + %% for stream, dgram and "some" raw sockets... + %% No such condition on linux (in the man page)... + {Opt, L, ?ESOCK_OPT_IP_TOS}; + transparent -> + {boolean, L, ?ESOCK_OPT_IP_TRANSPARENT}; + ttl -> + {integer, L, ?ESOCK_OPT_IP_TTL}; + unblock_source -> + {addr_if_src, L, ?ESOCK_OPT_IP_UNBLOCK_SOURCE}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(ipv6 = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_IPV6, + case Opt of + [] -> L; + %% + addrform -> {Opt, L, ?ESOCK_OPT_IPV6_ADDRFORM}; + add_membership -> + {addr_if, L, ?ESOCK_OPT_IPV6_ADD_MEMBERSHIP}; + authhdr -> + %% Is this obsolete? When get, the result is enoprotoopt + %% and in the header file it says 'obsolete'... + %% But there might be (old?) versions of linux + %% where it still works... + {boolean, L, ?ESOCK_OPT_IPV6_AUTHHDR}; + auth_level -> + {undefined, L, ?ESOCK_OPT_IPV6_AUTH_LEVEL}; + checksum -> + {undefined, L, ?ESOCK_OPT_IPV6_CHECKSUM}; + drop_membership -> + {addr_if, L, ?ESOCK_OPT_IPV6_DROP_MEMBERSHIP}; + dstopts -> + {boolean, L, ?ESOCK_OPT_IPV6_DSTOPTS}; + esp_network_level -> + {undefined, L, ?ESOCK_OPT_IPV6_ESP_NETWORK_LEVEL}; + esp_trans_level -> + {undefined, L, ?ESOCK_OPT_IPV6_ESP_TRANS_LEVEL}; + flowinfo -> + {boolean, L, ?ESOCK_OPT_IPV6_FLOWINFO}; + hoplimit -> + {boolean, L, ?ESOCK_OPT_IPV6_HOPLIMIT}; + hopopts -> + {boolean, L, ?ESOCK_OPT_IPV6_HOPOPTS}; + ipcomp_level -> + {undefined, L, ?ESOCK_OPT_IPV6_IPCOMP_LEVEL}; + join_group -> + {undefined, L, ?ESOCK_OPT_IPV6_JOIN_GROUP}; + leave_group -> + {undefined, L, ?ESOCK_OPT_IPV6_LEAVE_GROUP}; + mtu -> + {integer, L, ?ESOCK_OPT_IPV6_MTU}; + mtu_discover -> {Opt, L, ?ESOCK_OPT_IPV6_MTU_DISCOVER}; + multicast_hops -> + {hops, L, ?ESOCK_OPT_IPV6_MULTICAST_HOPS}; + multicast_if -> + {integer, L, ?ESOCK_OPT_IPV6_MULTICAST_IF}; + multicast_loop -> + {boolean, L, ?ESOCK_OPT_IPV6_MULTICAST_LOOP}; + portrange -> + {undefined, L, ?ESOCK_OPT_IPV6_PORTRANGE}; + pktoptions -> + {undefined, L, ?ESOCK_OPT_IPV6_PKTOPTIONS}; + recverr -> + {boolean, L, ?ESOCK_OPT_IPV6_RECVERR}; + recvhoplimit -> + {boolean, L, ?ESOCK_OPT_IPV6_RECVHOPLIMIT}; + recvpktinfo -> + {boolean, L, ?ESOCK_OPT_IPV6_RECVPKTINFO}; + pktinfo -> % alias on FreeBSD + {boolean, L, ?ESOCK_OPT_IPV6_RECVPKTINFO}; + recvtclass -> + {boolean, L, ?ESOCK_OPT_IPV6_RECVTCLASS}; + router_alert -> + {integer, L, ?ESOCK_OPT_IPV6_ROUTER_ALERT}; + rthdr -> + {boolean, L, ?ESOCK_OPT_IPV6_RTHDR}; + tclass -> + {boolean, L, ?ESOCK_OPT_IPV6_TCLASS}; + unicast_hops -> + {hops, L, ?ESOCK_OPT_IPV6_UNICAST_HOPS}; + use_min_mtu -> + {undefined, L, ?ESOCK_OPT_IPV6_USE_MIN_MTU}; + v6only -> + {boolean, L, ?ESOCK_OPT_IPV6_V6ONLY}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(tcp = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_TCP, + case Opt of + [] -> L; + %% + congestion -> + {list, L, ?ESOCK_OPT_TCP_CONGESTION}; + cork -> + {boolean, L, ?ESOCK_OPT_TCP_CORK}; + info -> + {undefined, L, ?ESOCK_OPT_TCP_INFO}; + keepcnt -> + {undefined, L, ?ESOCK_OPT_TCP_KEEPCNT}; + keepidle -> + {undefined, L, ?ESOCK_OPT_TCP_KEEPIDLE}; + keepintvl -> + {undefined, L, ?ESOCK_OPT_TCP_KEEPINTVL}; + maxseg -> + {integer, L, ?ESOCK_OPT_TCP_MAXSEG}; + md5seg -> + {undefined, L, ?ESOCK_OPT_TCP_MD5SIG}; + nodelay -> + {boolean, L, ?ESOCK_OPT_TCP_NODELAY}; + noopt -> + {undefined, L, ?ESOCK_OPT_TCP_NOOPT}; + nopush -> + {undefined, L, ?ESOCK_OPT_TCP_NOPUSH}; + syncnt -> + %% Only set? 1..255 + {undefined, L, ?ESOCK_OPT_TCP_SYNCNT}; + user_timeout -> + {undefined, L, ?ESOCK_OPT_TCP_USER_TIMEOUT}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(udp = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_UDP, + case Opt of + [] -> L; + %% + cork -> + {boolean, L, ?ESOCK_OPT_UDP_CORK}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(sctp = Level, Opt) -> + L = ?ESOCK_OPT_LEVEL_SCTP, + case Opt of + [] -> L; + %% + adaption_layer -> + {undefined, L, ?ESOCK_OPT_SCTP_ADAPTION_LAYER}; + associnfo -> {Opt, L, ?ESOCK_OPT_SCTP_ASSOCINFO}; + auth_active_key -> + {undefined, L, ?ESOCK_OPT_SCTP_AUTH_ACTIVE_KEY}; + auth_asconf -> + {undefined, L, ?ESOCK_OPT_SCTP_AUTH_ASCONF}; + auth_chunk -> + {undefined, L, ?ESOCK_OPT_SCTP_AUTH_CHUNK}; + auth_key -> + {undefined, L, ?ESOCK_OPT_SCTP_AUTH_KEY}; + auth_delete_key -> + {undefined, L, ?ESOCK_OPT_SCTP_AUTH_DELETE_KEY}; + autoclose -> + {integer, L, ?ESOCK_OPT_SCTP_AUTOCLOSE}; + context -> + {undefined, L, ?ESOCK_OPT_SCTP_CONTEXT}; + default_send_params -> + {undefined, L, ?ESOCK_OPT_SCTP_DEFAULT_SEND_PARAMS}; + delayed_ack_time -> + {undefined, L, ?ESOCK_OPT_SCTP_DELAYED_ACK_TIME}; + disable_fragments -> + {boolean, L, ?ESOCK_OPT_SCTP_DISABLE_FRAGMENTS}; + hmac_ident -> + {undefined, L, ?ESOCK_OPT_SCTP_HMAC_IDENT}; + events -> {Opt, L, ?ESOCK_OPT_SCTP_EVENTS}; + explicit_eor -> + {undefined, L, ?ESOCK_OPT_SCTP_EXPLICIT_EOR}; + fragment_intreleave -> + {undefined, L, ?ESOCK_OPT_SCTP_FRAGMENT_INTERLEAVE}; + get_peer_addr_info -> + {undefined, L, ?ESOCK_OPT_SCTP_GET_PEER_ADDR_INFO}; + initmsg -> {Opt, L, ?ESOCK_OPT_SCTP_INITMSG}; + i_want_mapped_v4_addr -> + {undefined, L, ?ESOCK_OPT_SCTP_I_WANT_MAPPED_V4_ADDR}; + local_auth_chunks -> + {undefined, L, ?ESOCK_OPT_SCTP_LOCAL_AUTH_CHUNKS}; + maxseg -> + {integer, L, ?ESOCK_OPT_SCTP_MAXSEG}; + maxburst -> + {undefined, L, ?ESOCK_OPT_SCTP_MAXBURST}; + nodelay -> + {boolean, L, ?ESOCK_OPT_SCTP_NODELAY}; + partial_delivery_point -> + {undefined, L, ?ESOCK_OPT_SCTP_PARTIAL_DELIVERY_POINT}; + peer_addr_params -> + {undefined, L, ?ESOCK_OPT_SCTP_PEER_ADDR_PARAMS}; + peer_auth_chunks -> + {undefined, L, ?ESOCK_OPT_SCTP_PEER_AUTH_CHUNKS}; + primary_addr -> + {undefined, L, ?ESOCK_OPT_SCTP_PRIMARY_ADDR}; + reset_streams -> + {undefined, L, ?ESOCK_OPT_SCTP_RESET_STREAMS}; + rtoinfo -> {Opt, L, ?ESOCK_OPT_SCTP_RTOINFO}; + set_peer_primary_addr -> + {undefined, L, ?ESOCK_OPT_SCTP_SET_PEER_PRIMARY_ADDR}; + status -> % ?ESOCK_OPT_SCTP_RTOINFO; + {undefined, L, ?ESOCK_OPT_SCTP_STATUS}; + use_ext_recvinfo -> + {undefined, L, ?ESOCK_OPT_SCTP_USE_EXT_RECVINFO}; + _ -> + not_supported(Level, Opt) + end; +enc_sockopt_type(Level, _Opt) -> + not_supported(Level). + +%% Validate and possibly encode the setopt value +%% +enc_setopt_value(Level, Opt, Value, Type) -> + %% + %% When guards are enough + %% + case Type of + boolean when is_boolean(Value) -> + Value; + integer when is_integer(Value) -> + Value; + non_neg_integer when is_integer(Value), 0 =< Value -> + Value; + byte when is_integer(Value), 0 =< Value, Value =< 255 -> + Value; + pid when is_pid(Value) -> + Value; + list when is_list(Value) -> + Value; + term -> + Value; + %% + mtu_discover + when Value =:= want; + Value =:= dont; + Value =:= do; + Value =:= probe; + is_integer(Value) -> + Value; + tos + when Value =:= lowdelay; + Value =:= throughput; + Value =:= reliability; + Value =:= mincost; + is_integer(Value) -> + Value; + addrform when Value =:= inet -> + enc_domain(Value); + _ -> + enc_setopt_value(Level, Opt, {Type, Value}) + end. +%% +enc_setopt_value(Level, Opt, TypeValue) -> + %% + %% When matching is needed + %% + case TypeValue of + {rcvbuf, {N, BufSz} = BufSpec} + when is_integer(N), 0 < N, is_integer(BufSz), 0 < BufSz -> + %% N: Number of reads (when specifying length = 0) + %% BufSz: Size of the "read" buffer + BufSpec; + {rcvbuf, Value} -> + enc_setopt_value(Level, Opt, Value, buf); + %% + {buf, default} -> + 0; % This will cause the nif-code to choose the default value + {buf, Value} -> + enc_setopt_value(Level, Opt, Value, non_neg_integer); + %% + {linger, abort} -> + {true, 0}; + {linger, {OnOff, Secs} = Linger} + when is_boolean(OnOff), is_integer(Secs), 0 =< Secs -> + Linger; + %% + {timeval, + #{sec := Sec, usec := USec} = Timeval} + when is_integer(Sec), is_integer(USec) -> + Timeval; + %% + {addr_if, + #{multiaddr := MA, interface := IF} = AddrIf} + when Level =:= ip + andalso ?IS_IPV4_ADDR(MA) + andalso (IF =:= any orelse ?IS_IPV4_ADDR(IF)) -> + AddrIf; + {addr_if, + #{multiaddr := MA, interface := IF} = AddrIf} + when Level =:= ipv6 + andalso ?IS_IPV6_ADDR(MA) + andalso (is_integer(IF) andalso (0 =< IF)) -> + AddrIf; + %% + {addr_if_src, + #{multiaddr := MA, interface := IF, sourceaddr := SA} = AddrIfSrc} + when ?IS_IPV4_ADDR(MA) + andalso ?IS_IPV4_ADDR(IF) + andalso ?IS_IPV4_ADDR(SA) -> + AddrIfSrc; + %% + {msfilter, null = Value} -> + Value; + {addr_msfilter, + #{multiaddr := MA, interface := IF, fmode := FMode, slist := SL} = + AddrMsfilter} + when ?IS_IPV4_ADDR(MA) + andalso ?IS_IPV4_ADDR(IF) + andalso (FMode =:= include orelse FMode =:= exclude) + andalso is_list(SL) -> + AddrMsfilter; + {hops, default} -> + -1; + {hops, Value} -> + enc_setopt_value(Level, Opt, Value, byte); + %% + {associnfo, + #{assoc_id := AssocId, + asocmaxrxt := MaxRxt, + num_peer_dests := NumPeerDests, + peer_rwnd := PeerRWND, + local_rwnd := LocalRWND, + cookie_life := CLife} = AssocInfo} + when is_integer(AssocId), + is_integer(MaxRxt), (MaxRxt >= 0), + is_integer(NumPeerDests), (NumPeerDests >= 0), + is_integer(PeerRWND), (PeerRWND >= 0), + is_integer(LocalRWND), (LocalRWND >= 0), + is_integer(CLife), (CLife >= 0) -> + AssocInfo; + {events, + #{data_in := DataIn, + association := Assoc, + address := Addr, + send_failure := SndFailure, + peer_error := PeerError, + shutdown := Shutdown, + partial_delivery := PartialDelivery, + adaptation_layer := AdaptLayer, + authentication := Auth, + sender_dry := SndDry} = Events} + when is_boolean(DataIn), + is_boolean(Assoc), + is_boolean(Addr), + is_boolean(SndFailure), + is_boolean(PeerError), + is_boolean(Shutdown), + is_boolean(PartialDelivery), + is_boolean(AdaptLayer), + is_boolean(Auth), + is_boolean(SndDry) -> + Events; + {initmsg, + #{num_outstreams := NumOut, + max_instreams := MaxIn, + max_attempts := MaxAttempts, + max_init_timeo := MaxInitTO} = InitMsg} + when is_integer(NumOut), (NumOut >= 0), + is_integer(MaxIn), (MaxIn >= 0), + is_integer(MaxAttempts), (MaxAttempts >= 0), + is_integer(MaxInitTO), (MaxInitTO >= 0) -> + InitMsg; + {rtoinfo, + #{assoc_id := AssocId, + initial := Init, + max := Max, + min := Min} = RTOInfo} + when is_integer(AssocId), + is_integer(Init), (Init >= 0), + is_integer(Max), (Max >= 0), + is_integer(Min), (Min >= 0) -> + RTOInfo; + {Type, Value} when Type =/= undefined -> + %% No match + not_supported(Level, Opt, Value) + end. + + +enc_shutdown_how(How) -> + case How of + read -> ?ESOCK_SHUTDOWN_HOW_READ; + write -> ?ESOCK_SHUTDOWN_HOW_WRITE; + read_write -> ?ESOCK_SHUTDOWN_HOW_READ_WRITE; + _ -> + invalid(how, How) + end. + + +%% +++ Decode getopt value +++ +%% +%% For the most part, we simply let the value pass through, but for some +%% values we may need to do an actual decode. +%% +dec_getopt_value(Alg, tcp, congestion) -> + %% This string is NULL-terminated, but the general function we use + %% in the nif code does not know that. So, deal with it here. + {Str, _} = lists:splitwith(fun(0) -> false; (_) -> true end, Alg), + Str; +dec_getopt_value(Value, _Level, _Opt) -> + Value. + +%% =========================================================================== +%% Error functions +%% + +-dialyzer({no_return, not_supported/3}). +not_supported(Level, Opt, Value) -> + not_supported({Level, Opt, Value}). +%% +-dialyzer({no_return, not_supported/2}). +not_supported(Level, Opt) -> + not_supported({Level, Opt}). +%% +-dialyzer({no_return, not_supported/1}). +not_supported(What) -> + invalid(not_supported, What). + +-dialyzer({no_return, invalid/2}). +invalid(What, Info) -> + err({invalid, {What, Info}}). + +err(Reason) -> + erlang:error(Reason). + +%% =========================================================================== +%% NIF functions +%% + +nif_info() -> erlang:nif_error(undef). +nif_info(_SRef) -> erlang:nif_error(undef). + +nif_command(_Command) -> erlang:nif_error(undef). + +nif_supports() -> erlang:nif_error(undef). +nif_supports(_Key) -> erlang:nif_error(undef). +nif_supports(_Key1, _Key2) -> erlang:nif_error(undef). + +nif_open(_FD, _Opts) -> erlang:nif_error(undef). +nif_open(_Domain, _Type, _Protocol, _Opts) -> erlang:nif_error(undef). + +nif_bind(_SRef, _SockAddr) -> erlang:nif_error(undef). +nif_bind(_SRef, _SockAddrs, _Action) -> erlang:nif_error(undef). + +nif_connect(_SRef) -> erlang:nif_error(undef). +nif_connect(_SRef, _SockAddr) -> erlang:nif_error(undef). + +nif_listen(_SRef, _Backlog) -> erlang:nif_error(undef). + +nif_accept(_SRef, _Ref) -> erlang:nif_error(undef). + +nif_send(_SockRef, _SendRef, _Data, _Flags) -> erlang:nif_error(undef). +nif_sendto(_SRef, _SendRef, _Data, _Dest, _Flags) -> erlang:nif_error(undef). +nif_sendmsg(_SRef, _SendRef, _MsgHdr, _Flags) -> erlang:nif_error(undef). + +nif_recv(_SRef, _RecvRef, _Length, _Flags) -> erlang:nif_error(undef). +nif_recvfrom(_SRef, _RecvRef, _Length, _Flags) -> erlang:nif_error(undef). +nif_recvmsg(_SRef, _RecvRef, _BufSz, _CtrlSz, _Flags) -> erlang:nif_error(undef). + +nif_close(_SRef) -> erlang:nif_error(undef). +nif_finalize_close(_SRef) -> erlang:nif_error(undef). +nif_shutdown(_SRef, _How) -> erlang:nif_error(undef). + +nif_setopt(_Ref, _IsEnc, _Lev, _Opt, _Val) -> erlang:nif_error(undef). +nif_getopt(_Ref, _IsEnc, _Lev, _Opt) -> erlang:nif_error(undef). + +nif_sockname(_Ref) -> erlang:nif_error(undef). +nif_peername(_Ref) -> erlang:nif_error(undef). + +nif_cancel(_SRef, _Op, _Ref) -> erlang:nif_error(undef). + +%% =========================================================================== diff --git a/erts/preloaded/src/socket.erl b/erts/preloaded/src/socket.erl deleted file mode 100644 index d05b4b818f..0000000000 --- a/erts/preloaded/src/socket.erl +++ /dev/null @@ -1,4192 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2018-2020. All Rights Reserved. -%% -%% Licensed under the Apache License, Version 2.0 (the "License"); -%% you may not use this file except in compliance with the License. -%% You may obtain a copy of the License at -%% -%% http://www.apache.org/licenses/LICENSE-2.0 -%% -%% Unless required by applicable law or agreed to in writing, software -%% distributed under the License is distributed on an "AS IS" BASIS, -%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -%% See the License for the specific language governing permissions and -%% limitations under the License. -%% -%% %CopyrightEnd% -%% - --module(socket). - --compile(nowarn_nif_inline). %% Nobody please enable module inlining! --compile(no_native). --compile({no_auto_import,[error/1]}). - -%% Administrative and "global" utility functions --export([ - on_load/0, on_load/1, - - number_of/0, - which_sockets/0, which_sockets/1, - - ensure_sockaddr/1, - - debug/1, socket_debug/1, - info/0, info/1, - supports/0, supports/1, supports/2, supports/3 - ]). - --export([ - open/1, open/2, open/3, open/4, - bind/2, bind/3, - connect/2, connect/3, - listen/1, listen/2, - accept/1, accept/2, - - send/2, send/3, send/4, - sendto/3, sendto/4, sendto/5, - sendmsg/2, sendmsg/3, sendmsg/4, - - recv/1, recv/2, recv/3, recv/4, - recvfrom/1, recvfrom/2, recvfrom/3, recvfrom/4, - recvmsg/1, recvmsg/2, recvmsg/3, recvmsg/5, - - close/1, - shutdown/2, - - setopt/4, - getopt/3, - - sockname/1, - peername/1, - - cancel/2 - ]). - --export_type([ - select_tag/0, - select_ref/0, - select_info/0, - - socket_counters/0, - socket_counter/0, - socket_info/0, - - domain/0, - type/0, - protocol/0, - socket/0, - - port_number/0, - ip_address/0, - ip4_address/0, - ip6_address/0, - sockaddr/0, - sockaddr_in4/0, - sockaddr_in6/0, - sockaddr_un/0, - sockaddr_ll/0, - - send_flags/0, - send_flag/0, - - recv_flags/0, - recv_flag/0, - - shutdown_how/0, - - sockopt_level/0, - otp_socket_option/0, - socket_option/0, - ip_socket_option/0, - ipv6_socket_option/0, - tcp_socket_option/0, - udp_socket_option/0, - sctp_socket_option/0, - raw_socket_option/0, - - timeval/0, - ip_tos/0, - ip_mreq/0, - ip_mreq_source/0, - ip_pmtudisc/0, - ip_msfilter_mode/0, - ip_msfilter/0, - ip_pktinfo/0, - ipv6_mreq/0, - ipv6_pmtudisc/0, - ipv6_pktinfo/0, - in6_flow_info/0, - in6_scope_id/0, - sctp_assoc_id/0, - sctp_sndrcvinfo/0, - sctp_event_subscribe/0, - sctp_assocparams/0, - sctp_initmsg/0, - sctp_rtoinfo/0, - - - msghdr_flag/0, - msghdr_flags/0, - msghdr/0, - cmsghdr_level/0, - cmsghdr_type/0, - %% cmsghdr_data/0, - cmsghdr_recv/0, cmsghdr_send/0, - - ee_origin/0, - icmp_dest_unreach/0, - icmpv6_dest_unreach/0, - extended_err/0, - - uint8/0, - uint16/0, - uint20/0, - uint32/0, - int32/0 - ]). - - --define(REGISTRY, socket_registry). - - --type socket_counters() :: [{socket_counter(), non_neg_integer()}]. --type socket_counter() :: read_byte | read_fails | read_pkg | read_pkg_max | - read_tries | read_waits | - write_byte | write_fails | write_pkg | write_pkg_max | - write_tries | write_waits | - acc_success | acc_fails | acc_tries | acc_waits. --type socket_info() :: #{domain := domain(), - type := type(), - protocol := protocol(), - ctrl := pid(), - ctype := normal | fromfd | {fromfd, integer()}, - counters := socket_counters(), - num_readers := non_neg_integer(), - num_writers := non_neg_integer(), - num_acceptors := non_neg_integer(), - writable := boolean(), - readable := boolean()}. - --type uint8() :: 0..16#FF. --type uint16() :: 0..16#FFFF. --type uint20() :: 0..16#FFFFF. --type uint32() :: 0..16#FFFFFFFF. --type int32() :: -2147483648..2147483647. - - -%% We support only a subset of all domains. --type domain() :: local | inet | inet6. - -%% We support only a subset of all types. -%% RDM - Reliably Delivered Messages --type type() :: stream | dgram | raw | rdm | seqpacket. - -%% We support only a subset of all protocols: -%% Note that the '{raw, integer()}' construct is intended -%% to be used with type = raw. -%% Note also that only the "superuser" can create a raw socket. --type protocol() :: ip | tcp | udp | sctp | icmp | igmp | {raw, integer()}. - --type port_number() :: 0..65535. - --type ip_address() :: ip4_address() | ip6_address(). - --type ip4_address() :: {0..255, 0..255, 0..255, 0..255}. - --type in6_flow_info() :: uint20(). --type in6_scope_id() :: uint32(). - --type ip6_address() :: - {0..65535, - 0..65535, - 0..65535, - 0..65535, - 0..65535, - 0..65535, - 0..65535, - 0..65535}. - --type timeval() :: #{sec := integer(), - usec := integer()}. - --type ip_pktinfo() :: #{ - ifindex := non_neg_integer(), % Interface Index - spec_dst := ip4_address(), % Local Address - addr := ip4_address() % Header Destination address - }. - -%% If the integer value is used, its up to the caller to ensure its valid! --type ip_tos() :: lowdelay | - throughput | - reliability | - mincost | - integer(). - -%% This type is used when requesting to become member of a multicast -%% group with a call to setopt. Example: -%% -%% socket:setopt(Socket, ip, add_membership, #{multiaddr => Addr, -%% interface => any}). -%% -%% Its also used when removing from a multicast group. Example: -%% -%% socket:setopt(Socket, ip, drop_membership, #{multiaddr => Addr, -%% interface => any}). -%% - --type ip_mreq() :: #{multiaddr := ip4_address(), - interface := any | ip4_address()}. -%% -type ip_mreqn() :: #{multiaddr := ip4_address(), -%% address := any | ip4_address(), -%% ifindex := integer()}. - --type ip_mreq_source() :: #{multiaddr := ip4_address(), - interface := ip4_address(), - sourceaddr := ip4_address()}. - --type ip_pmtudisc() :: want | dont | do | probe. - -%% multiaddr: Multicast group address -%% interface: Address of local interface -%% mode: Filter mode -%% slist: List of source addresses --type ip_msfilter_mode() :: include | exclude. - --type ip_msfilter() :: #{multiaddr := ip4_address(), - interface := ip4_address(), - mode := ip_msfilter_mode(), - slist := [ip4_address()]}. - --type ipv6_mreq() :: #{multiaddr := ip6_address(), - interface := non_neg_integer()}. - --type ipv6_pmtudisc() :: ip_pmtudisc(). - --type ipv6_pktinfo() :: #{ - addr := ip6_address(), - ifindex := integer() - }. - - --type sctp_assoc_id() :: int32(). --type sctp_sndrcvinfo() :: #{ - stream := uint16(), - ssn := uint16(), - flags := uint16(), - ppid := uint16(), - context := uint16(), - timetolive := uint16(), - tsn := uint16(), - cumtsn := uint16(), - assoc_id := sctp_assoc_id() - }. - --type sctp_event_subscribe() :: #{data_in := boolean(), - association := boolean(), - address := boolean(), - send_failure := boolean(), - peer_error := boolean(), - shutdown := boolean(), - partial_delivery := boolean(), - adaptation_layer := boolean(), - authentication := boolean(), - sender_dry := boolean()}. - --type sctp_assocparams() :: #{assoc_id := sctp_assoc_id(), - max_rxt := uint16(), - num_peer_dests := uint16(), - peer_rwnd := uint32(), - local_rwnd := uint32(), - cookie_life := uint32()}. - --type sctp_initmsg() :: #{num_outstreams := uint16(), - max_instreams := uint16(), - max_attempts := uint16(), - max_init_timeo := uint16() - }. - --type sctp_rtoinfo() :: #{assoc_id := sctp_assoc_id(), - initial := uint32(), - max := uint32(), - min := uint32()}. - --type sockaddr_un() :: #{family := local, - path := binary() | string()}. --type sockaddr_in4() :: #{family := inet, - port := port_number(), - %% The 'broadcast' here is the "limited broadcast" - addr := any | broadcast | loopback | ip4_address()}. --type sockaddr_in6() :: #{family := inet6, - port := port_number(), - addr := any | loopback | ip6_address(), - flowinfo := in6_flow_info(), - scope_id := in6_scope_id()}. --type sockaddr_ll() :: #{family := packet, - protocol := non_neg_integer(), - ifindex := integer(), - pkttype := packet_type(), - hatype := non_neg_integer(), - addr := binary()}. --type packet_type() :: host | broadcast | multicast | otherhost | - outgoing | loopback | user | kernel | fastroute | - non_neg_integer(). --type sockaddr() :: sockaddr_in4() | - sockaddr_in6() | - sockaddr_un() | - sockaddr_ll(). - --define(OPEN2_OPTS_DEFAULTS, #{debug => false, dup => true}). --define(OPEN4_OPTS_DEFAULTS, #{debug => false}). - --define(SOCKADDR_IN4_DEFAULTS(A), #{port => 0, - addr => A}). --define(SOCKADDR_IN4_DEFAULTS, ?SOCKADDR_IN4_DEFAULTS(any)). --define(SOCKADDR_IN4_DEFAULT(A), (?SOCKADDR_IN4_DEFAULTS(A))#{family => inet}). --define(SOCKADDR_IN6_DEFAULTS(A), #{port => 0, - addr => A, - flowinfo => 0, - scope_id => 0}). --define(SOCKADDR_IN6_DEFAULTS, ?SOCKADDR_IN6_DEFAULTS(any)). --define(SOCKADDR_IN6_DEFAULT(A), (?SOCKADDR_IN6_DEFAULTS(A))#{family => inet6}). - -%% otp - This option is internal to our (OTP) implementation. -%% socket - The socket layer (SOL_SOCKET). -%% ip - The IP layer (SOL_IP or is it IPPROTO_IP?). -%% ipv6 - The IPv6 layer (SOL_IPV6). -%% tcp - The TCP (Transport Control Protocol) layer (IPPROTO_TCP). -%% udp - The UDP (User Datagram Protocol) layer (IPPROTO_UDP). -%% sctp - The SCTP (Stream Control Transmission Protocol) layer (IPPROTO_SCTP). -%% Int - Raw level, sent down and used "as is". -%% Its up to the caller to make sure this is correct! --type sockopt_level() :: otp | - socket | - ip | ipv6 | tcp | udp | sctp | - non_neg_integer(). - -%% There are some options that are 'read-only'. -%% Should those be included here or in a special list? -%% Should we just document it and leave it to the user? -%% Or catch it in the encode functions? -%% A setopt for a readonly option leads to einval? -%% Do we really need a sndbuf? - --type otp_socket_option() :: debug | - iow | - controlling_process | - rcvbuf | % sndbuf | - rcvctrlbuf | - sndctrlbuf | - meta | - fd. -%% Shall we have special treatment of linger?? -%% read-only options: -%% domain | protocol | type. -%% FreeBSD (only?): acceptfilter --type socket_option() :: acceptconn | - acceptfilter | - bindtodevice | - broadcast | - busy_poll | - debug | - domain | - dontroute | - error | - keepalive | - linger | - mark | - oobinline | - passcred | - peek_off | - peercred | - priority | - protocol | - rcvbuf | - rcvbufforce | - rcvlowat | - rcvtimeo | - reuseaddr | - reuseport | - rxq_ovfl | - setfib | - sndbuf | - sndbufforce | - sndlowat | - sndtimeo | - timestamp | - type. - -%% Read-only options: -%% mtu -%% -%% Options only valid on FreeBSD?: -%% dontfrag -%% Options only valid for RAW sockets: -%% nodefrag (linux only?) --type ip_socket_option() :: add_membership | - add_source_membership | - block_source | - dontfrag | - drop_membership | - drop_source_membership | - freebind | - hdrincl | - minttl | - msfilter | - mtu | - mtu_discover | - multicast_all | - multicast_if | - multicast_loop | - multicast_ttl | - nodefrag | - options | - pktinfo | - recverr | - recvif | - recvdstaddr | - recvopts | - recvorigdstaddr | - recvtos | - recvttl | - retopts | - router_alert | - sndsrcaddr | - tos | - transparent | - ttl | - unblock_source. --type ipv6_socket_option() :: - addrform | - add_membership | - authhdr | - auth_level | - checksum | - drop_membership | - dstopts | - esp_trans_level | - esp_network_level | - faith | - flowinfo | - hopopts | - ipcomp_level | - join_group | - leave_group | - mtu | - mtu_discover | - multicast_hops | - multicast_if | - multicast_loop | - portrange | - pktoptions | - recverr | - recvhoplimit | hoplimit | - recvpktinfo | pktinfo | - recvtclass | - router_alert | - rthdr | - tclass | - unicast_hops | - use_min_mtu | - v6only. - --type tcp_socket_option() :: congestion | - cork | - info | - keepcnt | - keepidle | - keepintvl | - maxseg | - md5sig | - nodelay | - noopt | - nopush | - syncnt | - user_timeout. - --type udp_socket_option() :: cork. - --type sctp_socket_option() :: - adaption_layer | - associnfo | - auth_active_key | - auth_asconf | - auth_chunk | - auth_key | - auth_delete_key | - autoclose | - context | - default_send_params | - delayed_ack_time | - disable_fragments | - hmac_ident | - events | - explicit_eor | - fragment_interleave | - get_peer_addr_info | - initmsg | - i_want_mapped_v4_addr | - local_auth_chunks | - maxseg | - maxburst | - nodelay | - partial_delivery_point | - peer_addr_params | - peer_auth_chunks | - primary_addr | - reset_streams | - rtoinfo | - set_peer_primary_addr | - status | - use_ext_recvinfo. - --type raw_socket_option() :: filter. - -%% -type plain_socket_option() :: integer(). -%% -type sockopt() :: otp_socket_option() | -%% socket_option() | -%% ip_socket_option() | -%% ipv6_socket_option() | -%% tcp_socket_option() | -%% udp_socket_option() | -%% sctp_socket_option() | -%% raw_socket_option() | -%% plain_socket_option(). - --record(socket, {ref :: reference()}). - --opaque socket() :: #socket{}. - --type send_flags() :: [send_flag()]. --type send_flag() :: confirm | - dontroute | - eor | - more | - nosignal | - oob. - -%% Note that not all of these flags are useful for every recv function! -%% --type recv_flags() :: [recv_flag()]. --type recv_flag() :: cmsg_cloexec | - errqueue | - oob | - peek | - trunc. - --type shutdown_how() :: read | write | read_write. - --type msghdr_flag() :: ctrunc | eor | errqueue | oob | trunc. --type msghdr_flags() :: [msghdr_flag()]. --type msghdr() :: #{ - %% *Optional* target address - %% Used on an unconnected socket to specify the - %% target address for a datagram. - addr := sockaddr(), - - iov := [binary()], - - %% The maximum size of the control buffer is platform - %% specific. It is the users responsibility to ensure - %% that its not exceeded. - ctrl := [cmsghdr_recv()] | [cmsghdr_send()], - - %% Only valid with recvmsg - flags := msghdr_flags() - }. -%% We are able to (completely) decode *some* control message headers. -%% Even if we are able to decode both level and type, we may not be -%% able to decode the data, in which case it will be a binary. - --type cmsghdr_level() :: socket | ip | ipv6 | integer(). --type cmsghdr_type() :: credentials | - hoplevel | - origdstaddr | - pktinfo | - recvtos | - rights | - timestamp | - tos | - ttl | - integer(). --type cmsghdr_recv() :: - #{level := socket, type := timestamp, data := timeval()} | - #{level := socket, type := rights, data := binary()} | - #{level := socket, type := credentials, data := binary()} | - #{level := socket, type := integer(), data := binary()} | - #{level := ip, type := tos, data := ip_tos()} | - #{level := ip, type := recvtos, data := ip_tos()} | - #{level := ip, type := ttl, data := integer()} | - #{level := ip, type := recvttl, data := integer()} | - #{level := ip, type := pktinfo, data := ip_pktinfo()} | - #{level := ip, type := origdstaddr, data := sockaddr_in4()} | - #{level := ip, type := recverr, data := extended_err() | binary()} | - #{level := ip, type := integer(), data := binary()} | - #{level := ipv6, type := hoplevel, data := integer()} | - #{level := ipv6, type := pktinfo, data := ipv6_pktinfo()} | - #{level := ipv6, type := recverr, data := extended_err() | binary()} | - #{level := ipv6, type := tclass, data := integer()} | - #{level := ipv6, type := integer(), data := binary()} | - #{level := integer(), type := integer(), data := binary()}. --type cmsghdr_send() :: - #{level := socket, type := timestamp, data := binary()} | - #{level := socket, type := rights, data := binary()} | - #{level := socket, type := credentials, data := binary()} | - #{level := socket, type := integer(), data := binary()} | - #{level := ip, type := tos, data := ip_tos() | binary()} | - #{level := ip, type := ttl, data := integer() | binary()} | - #{level := ip, type := integer(), data := binary()} | - #{level := ipv6, type := tclass, data := integer()} | - #{level := ipv6, type := integer(), data := binary()} | - #{level := udp, type := integer(), data := binary()} | - #{level := integer(), type := integer(), data := binary()}. - --type ee_origin() :: none | local | icmp | icmp6 | uint8(). --type icmp_dest_unreach() :: net_unreach | host_unreach | port_unreach | frag_needed | - net_unknown | host_unknown | uint8(). --type icmpv6_dest_unreach() :: noroute | adm_prohibited | not_neighbour | addr_unreach | - port_unreach | policy_fail | reject_route | uint8(). --type extended_err() :: - #{error := term(), - origin := icmp, - type := dest_unreach, - code := icmp_dest_unreach(), - info := uint32(), - data := uint32(), - offender := undefined | sockaddr()} | - #{error := term(), - origin := icmp, - type := time_exceeded | uint8(), - code := uint8(), - info := uint32(), - data := uint32(), - offender := undefined | sockaddr()} | - #{error := term(), - origin := icmp6, - type := dest_unreach, - code := icmpv6_dest_unreach(), - info := uint32(), - data := uint32(), - offender := undefined | sockaddr()} | - #{error := term(), - origin := icmp6, - type := pkt_toobig | time_exceeded | uint8(), - code := uint8(), - info := uint32(), - data := uint32(), - offender := undefined | sockaddr()} | - #{error := term(), - origin := ee_origin(), - type := uint8(), - code := uint8(), - info := uint32(), - data := uint32(), - offender := undefined | sockaddr()}. - --opaque select_tag() :: atom(). --opaque select_ref() :: reference(). - -%% -record(select_info, {tag :: select_tag(), ref :: select_ref()}). - --type select_info() :: {select_info, select_tag(), select_ref()}. - --define(SELECT_INFO(T, R), {select_info, T, R}). --define(SELECT(T, R), {select, ?SELECT_INFO(T, R)}). - - -%% This is used in messages sent from the nif-code to erlang processes: -%% -%% {?ESOCK_TAG, Socket :: socket(), Tag :: atom(), Info :: term()} -%% --define(ESOCK_TAG, '$socket'). - --define(ESOCK_DOMAIN_LOCAL, 1). --define(ESOCK_DOMAIN_UNIX, ?ESOCK_DOMAIN_LOCAL). --define(ESOCK_DOMAIN_INET, 2). --define(ESOCK_DOMAIN_INET6, 3). - --define(ESOCK_TYPE_STREAM, 1). --define(ESOCK_TYPE_DGRAM, 2). --define(ESOCK_TYPE_RAW, 3). -%% -define(ESOCK_TYPE_RDM, 4). --define(ESOCK_TYPE_SEQPACKET, 5). - --define(ESOCK_PROTOCOL_DEFAULT, 0). --define(ESOCK_PROTOCOL_IP, 1). --define(ESOCK_PROTOCOL_TCP, 2). --define(ESOCK_PROTOCOL_UDP, 3). --define(ESOCK_PROTOCOL_SCTP, 4). --define(ESOCK_PROTOCOL_ICMP, 5). --define(ESOCK_PROTOCOL_IGMP, 6). - --define(ESOCK_LISTEN_BACKLOG_DEFAULT, 5). - --define(ESOCK_ACCEPT_TIMEOUT_DEFAULT, infinity). - --define(ESOCK_SEND_FLAG_CONFIRM, 0). --define(ESOCK_SEND_FLAG_DONTROUTE, 1). --define(ESOCK_SEND_FLAG_EOR, 2). --define(ESOCK_SEND_FLAG_MORE, 3). --define(ESOCK_SEND_FLAG_NOSIGNAL, 4). --define(ESOCK_SEND_FLAG_OOB, 5). - --define(ESOCK_SEND_FLAGS_DEFAULT, []). --define(ESOCK_SEND_TIMEOUT_DEFAULT, infinity). --define(ESOCK_SENDTO_FLAGS_DEFAULT, []). --define(ESOCK_SENDTO_TIMEOUT_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). --define(ESOCK_SENDMSG_FLAGS_DEFAULT, []). --define(ESOCK_SENDMSG_TIMEOUT_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). - --define(ESOCK_RECV_FLAG_CMSG_CLOEXEC, 0). --define(ESOCK_RECV_FLAG_ERRQUEUE, 1). --define(ESOCK_RECV_FLAG_OOB, 2). --define(ESOCK_RECV_FLAG_PEEK, 3). --define(ESOCK_RECV_FLAG_TRUNC, 4). - --define(ESOCK_RECV_FLAGS_DEFAULT, []). --define(ESOCK_RECV_TIMEOUT_DEFAULT, infinity). - --define(ESOCK_OPT_LEVEL_OTP, 0). --define(ESOCK_OPT_LEVEL_SOCKET, 1). --define(ESOCK_OPT_LEVEL_IP, 2). --define(ESOCK_OPT_LEVEL_IPV6, 3). --define(ESOCK_OPT_LEVEL_TCP, 4). --define(ESOCK_OPT_LEVEL_UDP, 5). --define(ESOCK_OPT_LEVEL_SCTP, 6). - -%% *** OTP (socket) options --define(ESOCK_OPT_OTP_DEBUG, 1). --define(ESOCK_OPT_OTP_IOW, 2). --define(ESOCK_OPT_OTP_CTRL_PROC, 3). --define(ESOCK_OPT_OTP_RCVBUF, 4). -%%-define(ESOCK_OPT_OTP_SNDBUF, 5). --define(ESOCK_OPT_OTP_RCVCTRLBUF, 6). --define(ESOCK_OPT_OTP_SNDCTRLBUF, 7). --define(ESOCK_OPT_OTP_FD, 8). --define(ESOCK_OPT_OTP_META, 9). --define(ESOCK_OPT_OTP_DOMAIN, 16#FF01). % INTERNAL --define(ESOCK_OPT_OTP_TYPE, 16#FF02). % INTERNAL --define(ESOCK_OPT_OTP_PROTOCOL, 16#FF03). % INTERNAL --define(ESOCK_OPT_OTP_DTP, 16#FF04). % INTERNAL - -%% *** SOCKET (socket) options --define(ESOCK_OPT_SOCK_ACCEPTCONN, 1). -%% -define(ESOCK_OPT_SOCK_ACCEPTFILTER, 2). % FreeBSD --define(ESOCK_OPT_SOCK_BINDTODEVICE, 3). --define(ESOCK_OPT_SOCK_BROADCAST, 4). -%% -define(ESOCK_OPT_SOCK_BUSY_POLL, 5). --define(ESOCK_OPT_SOCK_DEBUG, 6). --define(ESOCK_OPT_SOCK_DOMAIN, 7). --define(ESOCK_OPT_SOCK_DONTROUTE, 8). -%% -define(ESOCK_OPT_SOCK_ERROR, 9). --define(ESOCK_OPT_SOCK_KEEPALIVE, 10). --define(ESOCK_OPT_SOCK_LINGER, 11). -%% -define(ESOCK_OPT_SOCK_MARK, 12). --define(ESOCK_OPT_SOCK_OOBINLINE, 13). --define(ESOCK_OPT_SOCK_PASSCRED, 14). --define(ESOCK_OPT_SOCK_PEEK_OFF, 15). -%% -define(ESOCK_OPT_SOCK_PEERCRED, 16). --define(ESOCK_OPT_SOCK_PRIORITY, 17). --define(ESOCK_OPT_SOCK_PROTOCOL, 18). --define(ESOCK_OPT_SOCK_RCVBUF, 19). -%% -define(ESOCK_OPT_SOCK_RCVBUFFORCE, 20). --define(ESOCK_OPT_SOCK_RCVLOWAT, 21). --define(ESOCK_OPT_SOCK_RCVTIMEO, 22). --define(ESOCK_OPT_SOCK_REUSEADDR, 23). --define(ESOCK_OPT_SOCK_REUSEPORT, 24). -%% -define(ESOCK_OPT_SOCK_RXQ_OVFL, 25). -%% -define(ESOCK_OPT_SOCK_SETFIB, 26). % FreeBSD --define(ESOCK_OPT_SOCK_SNDBUF, 27). -%% -define(ESOCK_OPT_SOCK_SNDBUFFORCE, 28). --define(ESOCK_OPT_SOCK_SNDLOWAT, 29). --define(ESOCK_OPT_SOCK_SNDTIMEO, 30). --define(ESOCK_OPT_SOCK_TIMESTAMP, 31). --define(ESOCK_OPT_SOCK_TYPE, 32). - -%% *** IP (socket) options --define(ESOCK_OPT_IP_ADD_MEMBERSHIP, 1). --define(ESOCK_OPT_IP_ADD_SOURCE_MEMBERSHIP, 2). --define(ESOCK_OPT_IP_BLOCK_SOURCE, 3). -%% -define(ESOCK_OPT_IP_DONTFRAG, 4). % FreeBSD --define(ESOCK_OPT_IP_DROP_MEMBERSHIP, 5). --define(ESOCK_OPT_IP_DROP_SOURCE_MEMBERSHIP, 6). --define(ESOCK_OPT_IP_FREEBIND, 7). --define(ESOCK_OPT_IP_HDRINCL, 8). --define(ESOCK_OPT_IP_MINTTL, 9). --define(ESOCK_OPT_IP_MSFILTER, 10). --define(ESOCK_OPT_IP_MTU, 11). --define(ESOCK_OPT_IP_MTU_DISCOVER, 12). --define(ESOCK_OPT_IP_MULTICAST_ALL, 13). --define(ESOCK_OPT_IP_MULTICAST_IF, 14). --define(ESOCK_OPT_IP_MULTICAST_LOOP, 15). --define(ESOCK_OPT_IP_MULTICAST_TTL, 16). --define(ESOCK_OPT_IP_NODEFRAG, 17). -%% -define(ESOCK_OPT_IP_OPTIONS, 18). % FreeBSD --define(ESOCK_OPT_IP_PKTINFO, 19). --define(ESOCK_OPT_IP_RECVDSTADDR, 20). % FreeBSD --define(ESOCK_OPT_IP_RECVERR, 21). --define(ESOCK_OPT_IP_RECVIF, 22). --define(ESOCK_OPT_IP_RECVOPTS, 23). --define(ESOCK_OPT_IP_RECVORIGDSTADDR, 24). --define(ESOCK_OPT_IP_RECVTOS, 25). --define(ESOCK_OPT_IP_RECVTTL, 26). --define(ESOCK_OPT_IP_RETOPTS, 27). --define(ESOCK_OPT_IP_ROUTER_ALERT, 28). --define(ESOCK_OPT_IP_SENDSRCADDR, 29). % FreeBSD --define(ESOCK_OPT_IP_TOS, 30). --define(ESOCK_OPT_IP_TRANSPARENT, 31). --define(ESOCK_OPT_IP_TTL, 32). --define(ESOCK_OPT_IP_UNBLOCK_SOURCE, 33). - -%% *** IPv6 (socket) options --define(ESOCK_OPT_IPV6_ADDRFORM, 1). --define(ESOCK_OPT_IPV6_ADD_MEMBERSHIP, 2). --define(ESOCK_OPT_IPV6_AUTHHDR, 3). % Obsolete? -%% -define(ESOCK_OPT_IPV6_AUTH_LEVEL, 4). % FreeBSD -%% -define(ESOCK_OPT_IPV6_CHECKSUM, 5). % FreeBSD --define(ESOCK_OPT_IPV6_DROP_MEMBERSHIP, 6). --define(ESOCK_OPT_IPV6_DSTOPTS, 7). -%% -define(ESOCK_OPT_IPV6_ESP_NETWORK_LEVEL, 8). % FreeBSD -%% -define(ESOCK_OPT_IPV6_ESP_TRANS_LEVEL, 9). % FreeBSD -%% -define(ESOCK_OPT_IPV6_FAITH, 10). % FreeBSD --define(ESOCK_OPT_IPV6_FLOWINFO, 11). --define(ESOCK_OPT_IPV6_HOPLIMIT, 12). --define(ESOCK_OPT_IPV6_HOPOPTS, 13). -%% -define(ESOCK_OPT_IPV6_IPCOMP_LEVEL, 14). % FreeBSD -%% -define(ESOCK_OPT_IPV6_JOIN_GROUP, 15). % FreeBSD -%% -define(ESOCK_OPT_IPV6_LEAVE_GROUP, 16). % FreeBSD --define(ESOCK_OPT_IPV6_MTU, 17). --define(ESOCK_OPT_IPV6_MTU_DISCOVER, 18). --define(ESOCK_OPT_IPV6_MULTICAST_HOPS, 19). --define(ESOCK_OPT_IPV6_MULTICAST_IF, 20). --define(ESOCK_OPT_IPV6_MULTICAST_LOOP, 21). -%% -define(ESOCK_OPT_IPV6_PORTRANGE, 22). % FreeBSD -%% -define(ESOCK_OPT_IPV6_PKTOPTIONS, 23). % FreeBSD --define(ESOCK_OPT_IPV6_RECVERR, 24). --define(ESOCK_OPT_IPV6_RECVHOPLIMIT, 25). --define(ESOCK_OPT_IPV6_RECVPKTINFO, 26). % On FreeBSD: PKTINFO --define(ESOCK_OPT_IPV6_RECVTCLASS, 27). --define(ESOCK_OPT_IPV6_ROUTER_ALERT, 28). --define(ESOCK_OPT_IPV6_RTHDR, 29). --define(ESOCK_OPT_IPV6_TCLASS, 30). % FreeBSD --define(ESOCK_OPT_IPV6_UNICAST_HOPS, 31). -%% -define(ESOCK_OPT_IPV6_USE_MIN_MTU, 32). % FreeBSD --define(ESOCK_OPT_IPV6_V6ONLY, 33). - -%% *** TCP (socket) options --define(ESOCK_OPT_TCP_CONGESTION, 1). --define(ESOCK_OPT_TCP_CORK, 2). -%% -define(ESOCK_OPT_TCP_INFO, 3). -%% -define(ESOCK_OPT_TCP_KEEPCNT, 4). -%% -define(ESOCK_OPT_TCP_KEEPIDLE, 5). -%% -define(ESOCK_OPT_TCP_KEEPINTVL, 6). --define(ESOCK_OPT_TCP_MAXSEG, 7). -%% -define(ESOCK_OPT_TCP_MD5SIG, 8). --define(ESOCK_OPT_TCP_NODELAY, 9). -%% -define(ESOCK_OPT_TCP_NOOPT, 10). -%% -define(ESOCK_OPT_TCP_NOPUSH, 11). -%% -define(ESOCK_OPT_TCP_SYNCNT, 12). -%% -define(ESOCK_OPT_TCP_USER_TIMEOUT, 13). - -%% *** UDP (socket) options --define(ESOCK_OPT_UDP_CORK, 1). - -%% *** SCTP (socket) options -%% -define(ESOCK_OPT_SCTP_ADAPTION_LAYER, 1). --define(ESOCK_OPT_SCTP_ASSOCINFO, 2). -%% -define(ESOCK_OPT_SCTP_AUTH_ACTIVE_KEY, 3). -%% -define(ESOCK_OPT_SCTP_AUTH_ASCONF, 4). -%% -define(ESOCK_OPT_SCTP_AUTH_CHUNK, 5). -%% -define(ESOCK_OPT_SCTP_AUTH_KEY, 6). -%% -define(ESOCK_OPT_SCTP_AUTH_DELETE_KEY, 7). --define(ESOCK_OPT_SCTP_AUTOCLOSE, 8). -%% -define(ESOCK_OPT_SCTP_CONTEXT, 9). -%% -define(ESOCK_OPT_SCTP_DEFAULT_SEND_PARAMS, 10). -%% -define(ESOCK_OPT_SCTP_DELAYED_ACK_TIME, 11). --define(ESOCK_OPT_SCTP_DISABLE_FRAGMENTS, 12). -%% -define(ESOCK_OPT_SCTP_HMAC_IDENT, 13). --define(ESOCK_OPT_SCTP_EVENTS, 14). -%% -define(ESOCK_OPT_SCTP_EXPLICIT_EOR, 15). -%% -define(ESOCK_OPT_SCTP_FRAGMENT_INTERLEAVE, 16). -%% -define(ESOCK_OPT_SCTP_GET_PEER_ADDR_INFO, 17). --define(ESOCK_OPT_SCTP_INITMSG, 18). -%% -define(ESOCK_OPT_SCTP_I_WANT_MAPPED_V4_ADDR, 19). -%% -define(ESOCK_OPT_SCTP_LOCAL_AUTH_CHUNKS, 20). --define(ESOCK_OPT_SCTP_MAXSEG, 21). -%% -define(ESOCK_OPT_SCTP_MAXBURST, 22). --define(ESOCK_OPT_SCTP_NODELAY, 23). -%% -define(ESOCK_OPT_SCTP_PARTIAL_DELIVERY_POINT, 24). -%% -define(ESOCK_OPT_SCTP_PEER_ADDR_PARAMS, 25). -%% -define(ESOCK_OPT_SCTP_PEER_AUTH_CHUNKS, 26). -%% -define(ESOCK_OPT_SCTP_PRIMARY_ADDR, 27). -%% -define(ESOCK_OPT_SCTP_RESET_STREAMS, 28). --define(ESOCK_OPT_SCTP_RTOINFO, 29). -%% -define(ESOCK_OPT_SCTP_SET_PEER_PRIMARY_ADDR, 30). -%% -define(ESOCK_OPT_SCTP_STATUS, 31). -%% -define(ESOCK_OPT_SCTP_USE_EXT_RECVINFO, 32). - --define(ESOCK_SHUTDOWN_HOW_READ, 0). --define(ESOCK_SHUTDOWN_HOW_WRITE, 1). --define(ESOCK_SHUTDOWN_HOW_READ_WRITE, 2). - - --define(ESOCK_SUPPORTS_OPTIONS, 16#0001). --define(ESOCK_SUPPORTS_SCTP, 16#0002). --define(ESOCK_SUPPORTS_IPV6, 16#0003). --define(ESOCK_SUPPORTS_LOCAL, 16#0004). --define(ESOCK_SUPPORTS_SEND_FLAGS, 16#0005). --define(ESOCK_SUPPORTS_RECV_FLAGS, 16#0006). --define(ESOCK_SUPPORTS_NETNS, 16#0007). - - -%% =========================================================================== -%% -%% Administrative and utility API -%% -%% =========================================================================== - --spec on_load() -> ok. - -%% Should we require that the Extra arg is a map? -on_load() -> - on_load(#{}). - --spec on_load(Extra) -> ok when - Extra :: map(). - -on_load(Extra) -> - %% This is spawned as a system process to prevent init:restart/0 from - %% killing it. - Pid = erts_internal:spawn_system_process(?REGISTRY, start, []), - DebugFilename = - case os:get_env_var("ESOCK_DEBUG_FILENAME") of - "*" -> - "/tmp/esock-dbg-??????"; - F -> - F - end, - ok = - erlang:load_nif( - atom_to_list(?MODULE), - case DebugFilename of - false -> - Extra#{registry => Pid}; - _ -> - Extra - #{registry => Pid, - debug => true, - socket_debug => true, - debug_filename => - enc_path(DebugFilename)} - end). - -%% *** number_of *** -%% -%% Interface function to the socket registry -%% returns the number of existing (and "alive") sockets. -%% --spec number_of() -> non_neg_integer(). - -number_of() -> - ?REGISTRY:number_of(). - - -%% *** which_sockets/0,1 *** -%% -%% Interface function to the socket registry -%% Returns a list of all the sockets, accoring to the filter rule. -%% --spec which_sockets() -> [socket()]. - -which_sockets() -> - ?REGISTRY:which_sockets(fun(_) -> true end). - --spec which_sockets(FilterRule) -> [socket()] when - FilterRule :: inet | inet6 | - stream | dgram | seqpacket | - sctp | tcp | udp | - pid() | - fun((socket_info()) -> boolean()). - -which_sockets(Domain) - when ((Domain =:= inet) orelse (Domain =:= inet6)) -> - ?REGISTRY:which_sockets(fun(#{domain := D}) when (D =:= Domain) -> true; - (_) -> false end); -which_sockets(Type) - when ((Type =:= stream) orelse (Type =:= dgram) orelse (Type =:= seqpacket)) -> - ?REGISTRY:which_sockets(fun(#{type := T}) when (T =:= Type) -> true; - (_) -> false end); -which_sockets(Proto) - when ((Proto =:= sctp) orelse (Proto =:= tcp) orelse (Proto =:= udp)) -> - ?REGISTRY:which_sockets(fun(#{protocol := P}) when (P =:= Proto) -> true; - (_) -> false end); -which_sockets(CTRL) - when is_pid(CTRL) -> - ?REGISTRY:which_sockets(fun(#{ctrl := C}) when (C =:= CTRL) -> true; - (_) -> false end); -which_sockets(Filter) when is_function(Filter, 1) -> - ?REGISTRY:which_sockets(Filter). - - - - - --spec info() -> map(). - -info() -> - nif_info(). - - --spec debug(D :: boolean()) -> ok. -debug(D) when is_boolean(D) -> - nif_command(#{command => ?FUNCTION_NAME, - data => D}). - --spec socket_debug(D :: boolean()) -> ok. -socket_debug(D) when is_boolean(D) -> - nif_command(#{command => ?FUNCTION_NAME, - data => D}). - - -%% =========================================================================== -%% -%% info - Get miscellaneous information about a socket. -%% -%% Generates a list of various info about the socket, such as counter values. -%% -%% Do *not* call this function often. -%% -%% =========================================================================== - --spec info(Socket) -> socket_info() when - Socket :: socket(). - -info(#socket{ref = SockRef}) -> - nif_info(SockRef). - - - -%% =========================================================================== -%% -%% supports - get information about what the platform "supports". -%% -%% Generates a list of various info about what the plaform can support. -%% The most obvious case is 'options'. -%% -%% Each item in a 'supports'-list will appear only *one* time. -%% -%% =========================================================================== - --type supports_options_socket() :: [{socket_option(), boolean()}]. --type supports_options_ip() :: [{ip_socket_option(), boolean()}]. --type supports_options_ipv6() :: [{ipv6_socket_option(), boolean()}]. --type supports_options_tcp() :: [{tcp_socket_option(), boolean()}]. --type supports_options_udp() :: [{udp_socket_option(), boolean()}]. --type supports_options_sctp() :: [{sctp_socket_option(), boolean()}]. --type supports_options() :: [{socket, supports_options_socket()} | - {ip, supports_options_ip()} | - {ipv6, supports_options_ipv6()} | - {tcp, supports_options_tcp()} | - {udp, supports_options_udp()} | - {sctp, supports_options_sctp()}]. --type supports_send_flags() :: [{send_flag(), boolean()}]. --type supports_recv_flags() :: [{recv_flag(), boolean()}]. - --spec supports() -> [{options, supports_options()} | - {sctp, boolean()} | - {ipv6, boolean()} | - {local, boolean()} | - {send_flags, supports_send_flags()} | - {recv_flags, supports_recv_flags()} | - {netns, boolean()}]. - -supports() -> - [ - {options, supports(options)}, - {sctp, supports(sctp)}, - {ipv6, supports(ipv6)}, - {local, supports(local)}, - {send_flags, supports(send_flags)}, - {recv_flags, supports(recv_flags)}, - {netns, supports(netns)} - ]. - - --dialyzer({no_contracts, supports/1}). --spec supports(options) -> supports_options(); - (sctp) -> boolean(); - (ipv6) -> boolean(); - (local) -> boolean(); - (send_flags) -> supports_send_flags(); - (recv_flags) -> supports_recv_flags(); - (netns) -> boolean(); - (Key1) -> false when - Key1 :: term(). - -supports(options) -> - nif_supports(?ESOCK_SUPPORTS_OPTIONS); -supports(sctp) -> - nif_supports(?ESOCK_SUPPORTS_SCTP); -supports(ipv6) -> - nif_supports(?ESOCK_SUPPORTS_IPV6); -supports(local) -> - nif_supports(?ESOCK_SUPPORTS_LOCAL); -supports(send_flags) -> - nif_supports(?ESOCK_SUPPORTS_SEND_FLAGS); -supports(recv_flags) -> - nif_supports(?ESOCK_SUPPORTS_RECV_FLAGS); -supports(netns) -> - nif_supports(?ESOCK_SUPPORTS_NETNS); -supports(_Key1) -> - false. - --dialyzer({no_contracts, supports/2}). --spec supports(options, socket) -> supports_options_socket(); - (options, ip) -> supports_options_ip(); - (options, ipv6) -> supports_options_ipv6(); - (options, tcp) -> supports_options_tcp(); - (options, udp) -> supports_options_udp(); - (options, sctp) -> supports_options_sctp(); - (send_flags, SendFlag :: send_flag()) -> boolean(); - (recv_flags, RecvFlag :: recv_flag()) -> boolean(); - (Key1, Key2) -> false when - Key1 :: term(), - Key2 :: term(). - -supports(options, Level) -> - proplists:get_value(Level, supports(options), false); -supports(send_flags, Flag) -> - proplists:get_value(Flag, supports(send_flags), false); -supports(recv_flags, Flag) -> - proplists:get_value(Flag, supports(recv_flags), false); -supports(_Key1, _Level) -> - false. - - --spec supports(options, socket, Opt :: socket_option()) -> boolean(); - (options, ip, Opt :: ip_socket_option()) -> boolean(); - (options, ipv6, Opt :: ipv6_socket_option()) -> boolean(); - (options, tcp, Opt :: tcp_socket_option()) -> boolean(); - (options, udp, Opt :: udp_socket_option()) -> boolean(); - (options, sctp, Opt :: sctp_socket_option()) -> boolean(); - (Key1, Key2, Key3) -> false when - Key1 :: term(), - Key2 :: term(), - Key3 :: term(). - --dialyzer({no_contracts, supports/3}). -supports(options, Level, Opt) -> - case supports(options, Level) of - S when is_list(S) -> - proplists:get_value(Opt, S, false); - _ -> - false - end; -supports(_Key1, _Key2, _Key3) -> - false. - - - -%% =========================================================================== -%% -%% The proper socket API -%% -%% =========================================================================== - -%% =========================================================================== -%% -%% <KOLLA> -%% -%% The nif sets up a monitor to this process, and if it dies the socket -%% is closed. It is also used if someone wants to monitor the socket. -%% -%% We may therefor need monitor function(s): -%% -%% socket:monitor(Socket) -%% socket:demonitor(Socket) -%% -%% </KOLLA> -%% - - - -%% =========================================================================== -%% -%% open - create an endpoint for communication -%% -%% Extra: Currently only used for netns -%% - --spec open(FD) -> {ok, Socket} | {error, Reason} when - FD :: integer(), - Socket :: socket(), - Reason :: term(). - -open(FD) -> - open(FD, ?OPEN2_OPTS_DEFAULTS). - --spec open(FD, Opts) -> {ok, Socket} | {error, Reason} when - FD :: integer(), - Opts :: map(), - Socket :: socket(), - Reason :: term(); - (Domain, Type) -> {ok, Socket} | {error, Reason} when - Domain :: domain(), - Type :: type(), - Socket :: socket(), - Reason :: term(). - -open(FD, Opts) when is_integer(FD) andalso is_map(Opts) -> - case nif_open(FD, ensure_open2_opts(Opts)) of - {ok, SockRef} -> - Socket = #socket{ref = SockRef}, - {ok, Socket}; - {error, _} = ERROR -> - ERROR - end; -open(Domain, Type) -> - open(Domain, Type, default). - --spec open(Domain, Type, Protocol) -> {ok, Socket} | {error, Reason} when - Domain :: domain(), - Type :: type(), - Protocol :: default | protocol(), - Socket :: socket(), - Reason :: term(). - -open(Domain, Type, Protocol) -> - open(Domain, Type, Protocol, ?OPEN4_OPTS_DEFAULTS). - --spec open(Domain, Type, Protocol, Opts) -> {ok, Socket} | {error, Reason} when - Domain :: domain(), - Type :: type(), - Protocol :: default | protocol(), - Opts :: map(), - Socket :: socket(), - Reason :: term(). - -open(Domain, Type, Protocol, Opts) when is_map(Opts) -> - try - begin - EDomain = enc_domain(Domain), - EType = enc_type(Type), - EProtocol = enc_protocol(Protocol), - Opts_1 = - case Opts of - #{netns := Path} when is_list(Path) -> - Opts#{netns := enc_path(Path)}; - _ -> - Opts - end, - nif_open(EDomain, EType, EProtocol, Opts_1) - end - of - {ok, SockRef} -> - Socket = #socket{ref = SockRef}, - {ok, Socket}; - {error, _} = ERROR -> - ERROR - catch - throw:ERROR -> - ERROR - end. - - - -%% =========================================================================== -%% -%% bind - bind a name to a socket -%% -%% Note that Addr can only have the value of broadcast *if* Domain =:= inet! -%% - --spec bind(Socket, Addr) -> {ok, Port} | {error, Reason} when - Socket :: socket(), - Addr :: any | broadcast | loopback | sockaddr(), - Port :: port_number(), - Reason :: term(). - -bind(#socket{ref = SockRef}, Addr) - when ((Addr =:= any) orelse - (Addr =:= broadcast) orelse - (Addr =:= loopback)) -> - try - case which_domain(SockRef) of - inet -> - nif_bind(SockRef, ?SOCKADDR_IN4_DEFAULT(Addr)); - inet6 when (Addr =:= any) orelse (Addr =:= loopback) -> - nif_bind(SockRef, ?SOCKADDR_IN6_DEFAULT(Addr)); - Domain -> - invalid_domain(Domain) - end - catch - throw:ERROR -> - ERROR - end; -bind(#socket{ref = SockRef} = _Socket, Addr) when is_map(Addr) -> - try - nif_bind(SockRef, ensure_sockaddr(Addr)) - catch - throw:ERROR -> - ERROR - end. - - - -%% =========================================================================== -%% -%% bind - Add or remove a bind addresses on a socket -%% -%% Calling this function is only valid if the socket is: -%% type = seqpacket -%% protocol = sctp -%% -%% If the domain is inet, then all addresses *must* be IPv4. -%% If the domain is inet6, the addresses can be aither IPv4 or IPv6. -%% - --spec bind(Socket, Addrs, Action) -> ok | {error, Reason} when - Socket :: socket(), - Addrs :: [sockaddr()], - Action :: add | remove, - Reason :: term(). - -bind(#socket{ref = SockRef}, Addrs, Action) - when is_list(Addrs) andalso ((Action =:= add) orelse (Action =:= remove)) -> - try - begin - {Domain, Type, Proto} = which_dtp(SockRef), - ensure_domain(Domain, [inet, inet6]), - ensure_type(Type, seqpacket), - ensure_protocol(Proto, sctp), - validate_addrs(Domain, Addrs), - nif_bind(SockRef, Addrs, Action) - end - catch - throw:ERROR -> - ERROR - end. - -ensure_domain(Domain, [Domain | _]) -> ok; -ensure_domain(Domain, [_ | Domains]) -> ensure_domain(Domain, Domains); -ensure_domain(Domain, []) -> invalid_domain(Domain). - -ensure_type(Type, Type) -> ok; -ensure_type(Type, _) -> invalid_type(Type). - -ensure_protocol(Proto, Proto) -> ok; -ensure_protocol(Proto, _) -> invalid_protocol(Proto). - - -validate_addrs(inet = _Domain, Addrs) -> - validate_inet_addrs(Addrs); -validate_addrs(inet6 = _Domain, Addrs) -> - validate_inet6_addrs(Addrs). - -validate_inet_addrs(Addrs) -> - Validator = fun(#{family := inet, - addrs := Addr}) when is_tuple(Addr) andalso - (size(Addr) =:= 4) -> - ok; - (X) -> - throw({error, {invalid_address, X}}) - end, - lists:foreach(Validator, Addrs). - -validate_inet6_addrs(Addrs) -> - Validator = fun(#{family := inet, - addrs := Addr}) when is_tuple(Addr) andalso - (size(Addr) =:= 4) -> - ok; - (#{family := inet6, - addrs := Addr}) when is_tuple(Addr) andalso - (size(Addr) =:= 8) -> - ok; - (X) -> - throw({error, {invalid_address, X}}) - end, - lists:foreach(Validator, Addrs). - - -%% =========================================================================== -%% -%% connect - initiate a connection on a socket -%% - --spec connect(Socket, SockAddr) -> ok | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - Reason :: term(). - -connect(Socket, SockAddr) -> - connect(Socket, SockAddr, infinity). - --spec connect(Socket, SockAddr, nowait) -> - ok | {select, SelectInfo} | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, SockAddr, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - Timeout :: timeout(), - Reason :: term(). - -%% <KOLLA> -%% Is it possible to connect with family = local for the (dest) sockaddr? -%% </KOLLA> -connect(#socket{ref = SockRef}, SockAddr, Timeout) -> - try - do_connect(SockRef, ensure_sockaddr(SockAddr), deadline(Timeout)) - catch - throw:ERROR -> - ERROR - end. - - -do_connect(SockRef, SockAddr, Deadline) -> - case nif_connect(SockRef, ensure_sockaddr(SockAddr)) of - - ok -> - %% Connected! - ok; - - - {ok, Ref} when (Deadline =:= nowait) -> - %% Connecting, but the caller does not want to wait... - ?SELECT(connect, Ref); - - {ok, Ref} -> - %% Connecting... - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, _Socket, select, Ref} -> - nif_finalize_connection(SockRef); - - {?ESOCK_TAG, _Socket, abort, {Ref, Reason}} -> - {error, Reason} - - after Timeout -> - cancel(SockRef, connect, Ref), - {error, timeout} - end; - - - {error, _} = ERROR -> - ERROR - end. - - - -%% =========================================================================== -%% -%% listen - listen for connections on a socket -%% - --spec listen(Socket) -> ok | {error, Reason} when - Socket :: socket(), - Reason :: term(). - -listen(Socket) -> - listen(Socket, ?ESOCK_LISTEN_BACKLOG_DEFAULT). - --spec listen(Socket, Backlog) -> ok | {error, Reason} when - Socket :: socket(), - Backlog :: pos_integer(), - Reason :: term(). - -listen(#socket{ref = SockRef}, Backlog) - when (is_integer(Backlog) andalso (Backlog >= 0)) -> - nif_listen(SockRef, Backlog). - - - - -%% =========================================================================== -%% -%% accept, accept4 - accept a connection on a socket -%% - --spec accept(LSocket) -> {ok, Socket} | {error, Reason} when - LSocket :: socket(), - Socket :: socket(), - Reason :: term(). - -accept(Socket) -> - accept(Socket, ?ESOCK_ACCEPT_TIMEOUT_DEFAULT). - --spec accept(LSocket, nowait) -> - {ok, Socket} | - {select, SelectInfo} | - {error, Reason} when - LSocket :: socket(), - Socket :: socket(), - SelectInfo :: select_info(), - Reason :: term() - ; (LSocket, Timeout) -> {ok, Socket} | {error, Reason} when - LSocket :: socket(), - Timeout :: timeout(), - Socket :: socket(), - Reason :: term(). - -accept(#socket{ref = LSockRef}, Timeout) -> - try - do_accept(LSockRef, deadline(Timeout)) - catch - throw:ERROR -> - ERROR - end. - -do_accept(LSockRef, Deadline) -> - AccRef = make_ref(), - case nif_accept(LSockRef, AccRef) of - - {ok, SockRef} -> - Socket = #socket{ref = SockRef}, - {ok, Socket}; - - - {error, eagain} when (Deadline =:= nowait) -> - ?SELECT(accept, AccRef); - - {error, eagain} -> - %% Each call is non-blocking, but even then it takes - %% *some* time, so just to be sure, recalculate before - %% the receive. - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, #socket{ref = LSockRef}, select, AccRef} -> - do_accept(LSockRef, Deadline); - - {?ESOCK_TAG, _Socket, abort, {AccRef, Reason}} -> - {error, Reason} - - after Timeout -> - cancel(LSockRef, accept, AccRef), - {error, timeout} - end; - - - {error, _} = ERROR -> - cancel(LSockRef, accept, AccRef), % Just to be on the safe side... - ERROR - end. - - - -%% =========================================================================== -%% -%% send, sendto, sendmsg - send a message on a socket -%% - --spec send(Socket, Data) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Reason :: term(). - -send(Socket, Data) -> - send(Socket, Data, ?ESOCK_SEND_FLAGS_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). - --spec send(Socket, Data, Flags) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Flags :: send_flags(), - Reason :: term(); - (Socket, Data, Timeout :: nowait) -> - ok | - {ok, {binary(), SelectInfo}} | - {select, SelectInfo} | - {ok, {RestData, SelectInfo}} | - {error, Reason} when - Socket :: socket(), - Data :: iodata(), - RestData :: binary(), - SelectInfo :: select_info(), - Reason :: term(); - (Socket, Data, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Timeout :: timeout(), - Reason :: term(). - -send(Socket, Data, Flags) when is_list(Flags) -> - send(Socket, Data, Flags, ?ESOCK_SEND_TIMEOUT_DEFAULT); -send(Socket, Data, Timeout) -> - send(Socket, Data, ?ESOCK_SEND_FLAGS_DEFAULT, Timeout). - --spec send(Socket, Data, Flags, nowait) -> ok | - {select, SelectInfo} | - {ok, {RestData, SelectInfo}} | - {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Flags :: send_flags(), - RestData :: binary(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Data, Flags, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Flags :: send_flags(), - Timeout :: timeout(), - Reason :: term(). - -send(Socket, Data, Flags, Timeout) when is_list(Data) -> - Bin = erlang:list_to_binary(Data), - send(Socket, Bin, Flags, Timeout); -send(#socket{ref = SockRef}, Data, Flags, Timeout) - when is_binary(Data), is_list(Flags) -> - To = undefined, - try - begin - EFlags = enc_send_flags(Flags), - Deadline = deadline(Timeout), - send_common(SockRef, Data, To, EFlags, Deadline, send) - end - catch - throw:ERROR -> - ERROR - end. - -send_common(SockRef, Data, To, EFlags, Deadline, SendName) -> - SendRef = make_ref(), - - case - case SendName of - send -> - nif_send(SockRef, SendRef, Data, EFlags); - sendto -> - nif_sendto(SockRef, SendRef, Data, To, EFlags) - end - of - - ok -> ok; - - - {ok, Written} when (Deadline =:= nowait) -> - %% We are partially done, but the user don't want to wait (here) - %% for completion - <<_:Written/binary, Rest/binary>> = Data, - {ok, {Rest, ?SELECT_INFO(SendName, SendRef)}}; - - {ok, Written} -> - %% We are partially done, wait for continuation - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, _Socket, select, SendRef} - when (Written > 0) -> - <<_:Written/binary, Rest/binary>> = Data, - send_common( - SockRef, Rest, To, EFlags, Deadline, SendName); - - {?ESOCK_TAG, _Socket, select, SendRef} -> - send_common( - SockRef, Data, To, EFlags, Deadline, SendName); - - {?ESOCK_TAG, _Socket, abort, {SendRef, Reason}} -> - {error, {Reason, size(Data)}} - - after Timeout -> - _ = cancel(SockRef, SendName, SendRef), - {error, {timeout, size(Data)}} - end; - - - {error, exbusy} = Error when Deadline =:= nowait -> Error; - - {error, exbusy = Reason} -> - %% Internal error: - %% we called send, got eagain, and called send again - %% - without waiting for select message - erlang:error(Reason); - - - {error, eagain} when (Deadline =:= nowait) -> - ?SELECT(SendName, SendRef); - - {error, eagain} -> - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, _Socket, select, SendRef} -> - send_common( - SockRef, Data, To, EFlags, Deadline, SendName); - - {?ESOCK_TAG, _Socket, abort, {SendRef, Reason}} -> - {error, {Reason, size(Data)}} - - after Timeout -> - _ = cancel(SockRef, SendName, SendRef), - {error, {timeout, size(Data)}} - end; - - - {error, Reason} -> - {error, {Reason, size(Data)}} - end. - - -%% --------------------------------------------------------------------------- -%% - --spec sendto(Socket, Data, Dest) -> - ok | {error, Reason} when - Socket :: socket(), - Data :: binary(), - Dest :: sockaddr(), - Reason :: term(). - -sendto(Socket, Data, Dest) -> - sendto(Socket, Data, Dest, ?ESOCK_SENDTO_FLAGS_DEFAULT). - --spec sendto(Socket, Data, Dest, Flags) -> ok | {error, Reason} when - Socket :: socket(), - Data :: binary(), - Dest :: sockaddr(), - Flags :: send_flags(), - Reason :: term() - ; (Socket, Data, Dest, Timeout :: nowait) -> ok | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Dest :: sockaddr(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Data, Dest, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: iodata(), - Dest :: sockaddr(), - Timeout :: timeout(), - Reason :: term(). - -sendto(Socket, Data, Dest, Flags) when is_list(Flags) -> - sendto(Socket, Data, Dest, Flags, ?ESOCK_SENDTO_TIMEOUT_DEFAULT); -sendto(Socket, Data, Dest, Timeout) -> - sendto(Socket, Data, Dest, ?ESOCK_SENDTO_FLAGS_DEFAULT, Timeout). - - --spec sendto(Socket, Data, Dest, Flags, nowait) -> - ok | - {ok, {binary(), SelectInfo}} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - Data :: binary(), - Dest :: sockaddr(), - Flags :: send_flags(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Data, Dest, Flags, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - Data :: binary(), - Dest :: sockaddr(), - Flags :: send_flags(), - Timeout :: timeout(), - Reason :: term(). - -sendto(Socket, Data, Dest, Flags, Timeout) when is_list(Data) -> - Bin = erlang:list_to_binary(Data), - sendto(Socket, Bin, Dest, Flags, Timeout); -sendto(#socket{ref = SockRef}, Data, Dest, Flags, Timeout) - when is_binary(Data), is_list(Flags) -> - try - begin - To = ensure_sockaddr(Dest), - EFlags = enc_send_flags(Flags), - Deadline = deadline(Timeout), - send_common(SockRef, Data, To, EFlags, Deadline, sendto) - end - catch - throw:ERROR -> - ERROR - end. - - -%% --------------------------------------------------------------------------- -%% -%% The only part of the msghdr() that *must* exist (a connected -%% socket need not specify the addr field) is the iov. -%% The ctrl field is optional, and the addr and flags are not -%% used when sending. -%% - --spec sendmsg(Socket, MsgHdr) -> - ok | - {ok, Remaining} | - {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Remaining :: erlang:iovec(), - Reason :: term(). - -sendmsg(Socket, MsgHdr) -> - sendmsg(Socket, MsgHdr, - ?ESOCK_SENDMSG_FLAGS_DEFAULT, ?ESOCK_SENDMSG_TIMEOUT_DEFAULT). - - --spec sendmsg(Socket, MsgHdr, Flags) -> ok | {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Flags :: send_flags(), - Reason :: term(); - (Socket, MsgHdr, Timeout :: nowait) -> - ok | - {ok, Remaining} | - {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Remaining :: erlang:iovec(), - Reason :: term(); - (Socket, MsgHdr, Timeout) -> ok | {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Timeout :: timeout(), - Reason :: term(). - -sendmsg(Socket, MsgHdr, Flags) when is_list(Flags) -> - sendmsg(Socket, MsgHdr, Flags, ?ESOCK_SENDMSG_TIMEOUT_DEFAULT); -sendmsg(Socket, MsgHdr, Timeout) -> - sendmsg(Socket, MsgHdr, ?ESOCK_SENDMSG_FLAGS_DEFAULT, Timeout). - - --spec sendmsg(Socket, MsgHdr, Flags, nowait) -> - ok | - {ok, Remaining} | - {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Flags :: send_flags(), - Remaining :: erlang:iovec(), - Reason :: term() - ; (Socket, MsgHdr, Flags, Timeout) -> - ok | - {ok, Remaining} | - {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Flags :: send_flags(), - Timeout :: timeout(), - Remaining :: erlang:iovec(), - Reason :: term(). - -sendmsg(#socket{ref = SockRef}, #{iov := IOV} = MsgHdr, Flags, Timeout) - when is_list(IOV), is_list(Flags) -> - try - begin - M = ensure_msghdr(MsgHdr), - EFlags = enc_send_flags(Flags), - Deadline = deadline(Timeout), - do_sendmsg(SockRef, M, EFlags, Deadline) - end - catch - throw:ERROR -> - ERROR - end. - - -do_sendmsg(SockRef, MsgHdr, EFlags, Deadline) -> - - SendRef = make_ref(), - - case nif_sendmsg(SockRef, SendRef, MsgHdr, EFlags) of - ok -> - %% We are done - ok; - - - {ok, Written} when is_integer(Written) andalso (Written > 0) -> - %% We should not retry here since the protocol may not - %% be able to handle a message being split. Leave it to - %% the caller to figure out (call again with the rest). - %% - %% We need to cancel this partial write. - %% - _ = cancel(SockRef, sendmsg, SendRef), - {ok, do_sendmsg_rest(maps:get(iov, MsgHdr), Written)}; - - - {error, exbusy} = Error when Deadline =:= nowait -> Error; - - {error, exbusy = Reason} -> - %% Internal error: - %% we called send, got eagain, and called send again - %% - without waiting for select message - erlang:error(Reason); - - - {error, eagain} when (Deadline =:= nowait) -> - ?SELECT(sendmsg, SendRef); - - {error, eagain} -> - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, #socket{ref = SockRef}, select, SendRef} -> - do_sendmsg(SockRef, MsgHdr, EFlags, Deadline); - - {?ESOCK_TAG, _Socket, abort, {SendRef, Reason}} -> - {error, Reason} - - after Timeout -> - _ = cancel(SockRef, sendmsg, SendRef), - {error, timeout} - end; - - - {error, _} = ERROR -> - ERROR - end. - -do_sendmsg_rest([B|IOVec], Written) when (Written >= size(B)) -> - do_sendmsg_rest(IOVec, Written - size(B)); -do_sendmsg_rest([B|IOVec], Written) -> - <<_:Written/binary, Rest/binary>> = B, - [Rest|IOVec]. - -ensure_msghdr(#{ctrl := []} = M) -> - ensure_msghdr(maps:remove(ctrl, M)); -ensure_msghdr(#{iov := IOV, addr := Addr} = M) - when is_list(IOV) andalso (IOV =/= []) -> - M#{iov => erlang:iolist_to_iovec(IOV), addr => ensure_sockaddr(Addr)}; -ensure_msghdr(#{iov := IOV} = M) - when is_list(IOV) andalso (IOV =/= []) -> - M#{iov => erlang:iolist_to_iovec(IOV)}; -ensure_msghdr(_) -> - einval(). - - - -%% =========================================================================== -%% -%% recv, recvfrom, recvmsg - receive a message from a socket -%% -%% Description: -%% There is a special case for the argument Length. If its set to zero (0), -%% it means "give me everything you have". -%% -%% Returns: {ok, Binary} | {error, Reason} -%% Binary - The received data as a binary -%% Reason - The error reason: -%% timeout | {timeout, AccData} | -%% posix() | {posix(), AccData} | -%% atom() | {atom(), AccData} -%% AccData - The data (as a binary) that we did manage to receive -%% before the timeout. -%% -%% Arguments: -%% Socket - The socket to read from. -%% Length - The number of bytes to read. -%% Flags - A list of "options" for the read. -%% Timeout - Time-out in milliseconds. - --spec recv(Socket) -> {ok, Data} | {error, Reason} when - Socket :: socket(), - Data :: binary(), - Reason :: term(). - -recv(Socket) -> - recv(Socket, 0). - --spec recv(Socket, Length) -> {ok, Data} | {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Data :: binary(), - Reason :: term(). - -recv(Socket, Length) -> - recv(Socket, Length, - ?ESOCK_RECV_FLAGS_DEFAULT, - ?ESOCK_RECV_TIMEOUT_DEFAULT). - --spec recv(Socket, Length, Flags) -> {ok, Data} | - {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Flags :: recv_flags(), - Data :: binary(), - Reason :: term() - ; (Socket, Length, Timeout :: nowait) -> {ok, Data} | - {select, SelectInfo} | - {ok, {Data, SelectInfo}} | - {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Data :: binary(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Length, Timeout) -> {ok, Data} | - {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Timeout :: timeout(), - Data :: binary(), - Reason :: term(). - -recv(Socket, Length, Flags) when is_list(Flags) -> - recv(Socket, Length, Flags, ?ESOCK_RECV_TIMEOUT_DEFAULT); -recv(Socket, Length, Timeout) -> - recv(Socket, Length, ?ESOCK_RECV_FLAGS_DEFAULT, Timeout). - --spec recv(Socket, Length, Flags, nowait) -> {ok, Data} | - {select, SelectInfo} | - {ok, {Data, SelectInfo}} | - {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Flags :: recv_flags(), - Data :: binary(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Length, Flags, Timeout) -> {ok, Data} | - {error, Reason} when - Socket :: socket(), - Length :: non_neg_integer(), - Flags :: recv_flags(), - Timeout :: timeout(), - Data :: binary(), - Reason :: term(). - -recv(#socket{ref = SockRef}, Length, Flags, Timeout) - when is_integer(Length), Length >= 0, is_list(Flags) -> - try - EFlags = enc_recv_flags(Flags), - Deadline = deadline(Timeout), - do_recv(SockRef, Length, EFlags, Deadline, <<>>) - catch - throw:ERROR -> - ERROR - end. - -%% We will only recurse with Length == 0 if Length is 0, -%% so Length == 0 means to return all available data also when recursing -%% -%% Note that the Deadline value of 'nowait' has a special meaning. It means -%% that we will either return with data or with the with {error, NNNN}. In -%% wich case the caller will receive a select message at some later time. -%% -do_recv(SockRef, Length, EFlags, Deadline, Acc) -> - - RecvRef = make_ref(), - case nif_recv(SockRef, RecvRef, Length, EFlags) of - - {ok, true = _Complete, Bin} -> - {ok, bincat(Acc, Bin)}; - - - %% It depends on the amount of bytes we tried to read: - %% 0 - Read everything available - %% We got something, but there may be more - keep reading. - %% > 0 - We got a part of the message and we will be notified - %% when there is more to read (a select message) - {ok, false = _Complete, Bin} when Length =:= 0 -> - Timeout = timeout(Deadline), - if - 0 < Timeout -> - do_recv( - SockRef, Length, EFlags, Deadline, bincat(Acc, Bin)); - true -> - {ok, bincat(Acc, Bin)} - end; - - %% Did not get all the user asked for, but the user also - %% specified 'nowait', so deliver what we got and the - %% select info. - {ok, false = _Completed, Bin} when Deadline =:= nowait -> - {ok, {bincat(Acc, Bin), ?SELECT_INFO(recv, RecvRef)}}; - - {ok, false = _Completed, Bin} -> - %% We got a chunk of it! - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, #socket{ref = SockRef}, select, RecvRef} -> - if - 0 < Timeout -> - do_recv( - SockRef, Length - byte_size(Bin), EFlags, - Deadline, bincat(Acc, Bin)); - true -> - {error, {timeout, bincat(Acc, Bin)}} - end; - - {?ESOCK_TAG, _Socket, abort, {RecvRef, Reason}} -> - {error, Reason} - - after Timeout -> - cancel(SockRef, recv, RecvRef), - {error, {timeout, bincat(Acc, Bin)}} - end; - - - {error, exbusy} = Error when Deadline =:= nowait -> Error; - - {error, exbusy = Reason} -> - %% Internal error: - %% we called recv, got eagain, and called recv again - %% - without waiting for select message - erlang:error(Reason); - - - %% The user does not want to wait! - %% The user will be informed that there is something to read - %% via the select socket message (see below). - {error, eagain} when Deadline =:= nowait -> - if - byte_size(Acc) =:= 0 -> - ?SELECT(recv, RecvRef); - true -> - {ok, {Acc, ?SELECT_INFO(recv, RecvRef)}} - end; - - - %% We return with the accumulated binary (if its non-empty) - {error, eagain} when Length =:= 0, 0 < byte_size(Acc) -> - cancel(SockRef, recv, RecvRef), - {ok, Acc}; - - {error, eagain} -> - %% There is nothing just now, but we will be notified when there - %% is something to read (a select message). - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, #socket{ref = SockRef}, select, RecvRef} -> - if - 0 < Timeout -> - do_recv( - SockRef, Length, EFlags, Deadline, Acc); - 0 < byte_size(Acc) -> - {error, {timeout, Acc}}; - true -> - {error, timeout} - end; - - {?ESOCK_TAG, _Socket, abort, {RecvRef, Reason}} -> - {error, Reason} - - after Timeout -> - cancel(SockRef, recv, RecvRef), - {error, timeout} - end; - - - {error, _} = ERROR when byte_size(Acc) =:= 0 -> - ERROR; - - {error, Reason} -> - {error, {Reason, Acc}} - - end. - - - -%% --------------------------------------------------------------------------- -%% -%% With recvfrom we get messages, which means that regardless of how -%% much we want to read, we return when we get a message. -%% The MaxSize argument basically defines the size of our receive -%% buffer. By setting the size to zero (0), we use the configured -%% size (see setopt). -%% It may be impossible to know what (buffer) size is appropriate -%% "in advance", and in those cases it may be convenient to use the -%% (recv) 'peek' flag. When this flag is provided the message is *not* -%% "consumed" from the underlying (OS) buffers, so another recvfrom call -%% is needed, possibly with a then adjusted buffer size. -%% - --spec recvfrom(Socket) -> {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term(). - -recvfrom(Socket) -> - recvfrom(Socket, 0). - --spec recvfrom(Socket, BufSz) -> {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term(). - -recvfrom(Socket, BufSz) -> - recvfrom(Socket, BufSz, - ?ESOCK_RECV_FLAGS_DEFAULT, - ?ESOCK_RECV_TIMEOUT_DEFAULT). - --spec recvfrom(Socket, Flags, nowait) -> - {ok, {Source, Data}} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - Source :: sockaddr() | undefined, - Data :: binary(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Flags, Timeout) -> - {ok, {Source, Data}} | - {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - Timeout :: timeout(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term() - ; (Socket, BufSz, Flags) -> - {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Flags :: recv_flags(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term() - ; (Socket, BufSz, nowait) -> - {ok, {Source, Data}} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Source :: sockaddr() | undefined, - Data :: binary(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, BufSz, Timeout) -> - {ok, {Source, Data}} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Timeout :: timeout(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term(). - -recvfrom(Socket, Flags, Timeout) when is_list(Flags) -> - recvfrom(Socket, 0, Flags, Timeout); -recvfrom(Socket, BufSz, Flags) when is_list(Flags) -> - recvfrom(Socket, BufSz, Flags, ?ESOCK_RECV_TIMEOUT_DEFAULT); -recvfrom(Socket, BufSz, Timeout) -> - recvfrom(Socket, BufSz, ?ESOCK_RECV_FLAGS_DEFAULT, Timeout). - --spec recvfrom(Socket, BufSz, Flags, nowait) -> - {ok, {Source, Data}} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Flags :: recv_flags(), - Source :: sockaddr() | undefined, - Data :: binary(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, BufSz, Flags, Timeout) -> - {ok, {Source, Data}} | - {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - Flags :: recv_flags(), - Timeout :: timeout(), - Source :: sockaddr() | undefined, - Data :: binary(), - Reason :: term(). - -recvfrom(#socket{ref = SockRef}, BufSz, Flags, Timeout) - when is_integer(BufSz), 0 =< BufSz, is_list(Flags) -> - try - EFlags = enc_recv_flags(Flags), - Deadline = deadline(Timeout), - do_recvfrom(SockRef, BufSz, EFlags, Deadline) - catch - throw:ERROR -> - ERROR - end. - -do_recvfrom(SockRef, BufSz, EFlags, Deadline) -> - - RecvRef = make_ref(), - case nif_recvfrom(SockRef, RecvRef, BufSz, EFlags) of - - {ok, {_Source, _NewData}} = OK -> - OK; - - - {error, exbusy} = Error when Deadline =:= nowait -> Error; - - {error, exbusy = Reason} -> - %% Internal error: - %% we called recvfrom, got eagain, and called recvfrom again - %% - without waiting for select message - erlang:error(Reason); - - - {error, eagain} when Deadline =:= nowait -> - ?SELECT(recvfrom, RecvRef); - - {error, eagain} -> - %% There is nothing just now, but we will be notified when there - %% is something to read (a select message). - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, #socket{ref = SockRef}, select, RecvRef} -> - do_recvfrom(SockRef, BufSz, EFlags, Deadline); - - {?ESOCK_TAG, _Socket, abort, {RecvRef, Reason}} -> - {error, Reason} - - after Timeout -> - cancel(SockRef, recvfrom, RecvRef), - {error, timeout} - end; - - - {error, _Reason} = ERROR -> - ERROR - - end. - - -%% --------------------------------------------------------------------------- -%% - --spec recvmsg(Socket) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - Reason :: term(). - -recvmsg(Socket) -> - recvmsg(Socket, 0, 0, - ?ESOCK_RECV_FLAGS_DEFAULT, ?ESOCK_RECV_TIMEOUT_DEFAULT). - --spec recvmsg(Socket, Flags) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - MsgHdr :: msghdr(), - Reason :: term() - ; (Socket, Timeout :: nowait) -> {ok, MsgHdr} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - MsgHdr :: msghdr(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Timeout) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - Timeout :: timeout(), - MsgHdr :: msghdr(), - Reason :: term(). - -recvmsg(Socket, Flags) when is_list(Flags) -> - recvmsg(Socket, 0, 0, Flags, ?ESOCK_RECV_TIMEOUT_DEFAULT); -recvmsg(Socket, Timeout) -> - recvmsg(Socket, 0, 0, ?ESOCK_RECV_FLAGS_DEFAULT, Timeout). - --spec recvmsg(Socket, Flags, nowait) -> {ok, MsgHdr} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - MsgHdr :: msghdr(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - Flags :: recv_flags(), - Timeout :: timeout(), - MsgHdr :: msghdr(), - Reason :: term() - ; (Socket, BufSz, CtrlSz) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - CtrlSz :: non_neg_integer(), - MsgHdr :: msghdr(), - Reason :: term(). - -recvmsg(Socket, Flags, Timeout) when is_list(Flags) -> - recvmsg(Socket, 0, 0, Flags, Timeout); -recvmsg(Socket, BufSz, CtrlSz) when is_integer(BufSz), is_integer(CtrlSz) -> - recvmsg(Socket, BufSz, CtrlSz, - ?ESOCK_RECV_FLAGS_DEFAULT, ?ESOCK_RECV_TIMEOUT_DEFAULT). - - --spec recvmsg(Socket, - BufSz, CtrlSz, - Flags, nowait) -> {ok, MsgHdr} | - {select, SelectInfo} | - {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - CtrlSz :: non_neg_integer(), - Flags :: recv_flags(), - MsgHdr :: msghdr(), - SelectInfo :: select_info(), - Reason :: term() - ; (Socket, - BufSz, CtrlSz, - Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when - Socket :: socket(), - BufSz :: non_neg_integer(), - CtrlSz :: non_neg_integer(), - Flags :: recv_flags(), - Timeout :: timeout(), - MsgHdr :: msghdr(), - Reason :: term(). - -recvmsg(#socket{ref = SockRef}, BufSz, CtrlSz, Flags, Timeout) - when is_integer(BufSz), 0 =< BufSz, - is_integer(CtrlSz), 0 =< CtrlSz, - is_list(Flags) -> - try - EFlags = enc_recv_flags(Flags), - Deadline = deadline(Timeout), - do_recvmsg(SockRef, BufSz, CtrlSz, EFlags, Deadline) - catch - throw:ERROR -> - ERROR - end. - -do_recvmsg(SockRef, BufSz, CtrlSz, EFlags, Deadline) -> - - RecvRef = make_ref(), - case nif_recvmsg(SockRef, RecvRef, BufSz, CtrlSz, EFlags) of - - {ok, _MsgHdr} = OK -> - OK; - - - {error, exbusy} = Error when Deadline =:= nowait -> Error; - - {error, exbusy = Reason} -> - %% Internal error: - %% we called recvmsg, got eagain, and called recvmsg again - %% - without waiting for select message - erlang:error(Reason); - - - {error, eagain} when Deadline =:= nowait -> - ?SELECT(recvmsg, RecvRef); - - {error, eagain} -> - %% There is nothing just now, but we will be notified when there - %% is something to read (a select message). - Timeout = timeout(Deadline), - receive - {?ESOCK_TAG, #socket{ref = SockRef}, select, RecvRef} -> - do_recvmsg(SockRef, BufSz, CtrlSz, EFlags, Deadline); - - {?ESOCK_TAG, _Socket, abort, {RecvRef, Reason}} -> - {error, Reason} - - after Timeout -> - cancel(SockRef, recvmsg, RecvRef), - {error, timeout} - end; - - - {error, _Reason} = ERROR -> - ERROR - - end. - - - -%% =========================================================================== -%% -%% close - close a file descriptor -%% -%% Closing a socket is a two stage rocket (because of linger). -%% We need to perform the actual socket close while in BLOCKING mode. -%% But that would hang the entire VM, so what we do is divide the -%% close in two steps: -%% 1) nif_close + the socket_stop (nif) callback function -%% This is for everything that can be done safely NON-BLOCKING. -%% 2) nif_finalize_close which is executed by a *dirty* scheduler -%% Before we call the socket close function, we set the socket -%% BLOCKING. Thereby linger is handled properly. - --spec close(Socket) -> ok | {error, Reason} when - Socket :: socket(), - Reason :: term(). - -close(#socket{ref = SockRef}) -> - case nif_close(SockRef) of - ok -> - nif_finalize_close(SockRef); - {ok, CloseRef} -> - %% We must wait for the socket_stop callback function to - %% complete its work - receive - {?ESOCK_TAG, #socket{ref = SockRef}, close, CloseRef} -> - nif_finalize_close(SockRef) - end; - {error, _} = ERROR -> - ERROR - end. - - - - -%% =========================================================================== -%% -%% shutdown - shut down part of a full-duplex connection -%% - --spec shutdown(Socket, How) -> ok | {error, Reason} when - Socket :: socket(), - How :: shutdown_how(), - Reason :: term(). - -shutdown(#socket{ref = SockRef}, How) -> - try - nif_shutdown(SockRef, enc_shutdown_how(How)) - catch - throw:T -> - T; - %% <WIN32-TEMPORARY> - error:notsup:S -> - erlang:raise(error, notsup, S); - %% </WIN32-TEMPORARY> - error:Reason -> - {error, Reason} - end. - - - - -%% =========================================================================== -%% -%% setopt - manipulate individual properties of a socket -%% -%% What properties are valid depend on what kind of socket it is -%% (domain, type and protocol) -%% If its an "invalid" option (or value), we should not crash but return some -%% useful error... -%% -%% <KOLLA> -%% -%% WE NEED TO MAKE SURE THAT THE USER DOES NOT MAKE US BLOCKING -%% AS MUCH OF THE CODE EXPECTS TO BE NON-BLOCKING!! -%% -%% </KOLLA> - --spec setopt(Socket, otp, otp_socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, socket, socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, ip, ip_socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, ipv6, ipv6_socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, tcp, tcp_socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, udp, udp_socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, sctp, sctp_socket_option(), Value) -> ok | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, Level, Key, Value) -> ok | {error, Reason} when - Socket :: socket(), - Level :: non_neg_integer(), - Key :: non_neg_integer(), - Value :: binary(), - Reason :: term(). - -setopt(#socket{ref = SockRef}, Level, Key, Value) -> - try - begin - {Domain, Type, Proto} = which_dtp(SockRef), - {EIsEncoded, ELevel} = enc_setopt_level(Level), - EKey = enc_setopt_key(Level, Key, Domain, Type, Proto), - EVal = enc_setopt_value(Level, Key, Value, Domain, Type, Proto), - nif_setopt(SockRef, EIsEncoded, ELevel, EKey, EVal) - end - catch - throw:ERROR -> - ERROR - end. - - - - -%% =========================================================================== -%% -%% getopt - retrieve individual properties of a socket -%% -%% What properties are valid depend on what kind of socket it is -%% (domain, type and protocol). -%% If its an "invalid" option, we should not crash but return some -%% useful error... -%% -%% When specifying level as an integer, and therefor using "native mode", -%% we should make it possible to specify common types instead of the -%% value size. Example: int | bool | {string, pos_integer()} | non_neg_integer() -%% - --spec getopt(Socket, otp, otp_socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, socket, socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, ip, ip_socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, ipv6, ipv6_socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, tcp, tcp_socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, udp, udp_socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, sctp, sctp_socket_option()) -> {ok, Value} | {error, Reason} when - Socket :: socket(), - Value :: term(), - Reason :: term() - ; (Socket, Level, Key) -> ok | {ok, Value} | {error, Reason} when - Socket :: socket(), - Level :: integer(), - Key :: {NativeOpt, ValueSize}, - NativeOpt :: integer(), - ValueSize :: int | bool | non_neg_integer(), - Value :: term(), - Reason :: term(). - -getopt(#socket{ref = SockRef}, Level, Key) -> - try - begin - {Domain, Type, Proto} = which_dtp(SockRef), - {EIsEncoded, ELevel} = enc_getopt_level(Level), - EKey = enc_getopt_key(Level, Key, Domain, Type, Proto), - %% We may need to decode the value (for the same reason - %% we (may have) needed to encode the value for setopt). - case nif_getopt(SockRef, EIsEncoded, ELevel, EKey) of - ok -> - ok; - {ok, EVal} -> - Val = - dec_getopt_value( - Level, Key, EVal, Domain, Type, Proto), - {ok, Val}; - {error, _} = E -> - E - end - end - catch - throw:ERROR -> - ERROR - end. - - -%% These are internal "shortcut" functions for the options -%% domain, type and protocol. - --spec which_domain(SockRef) -> Domain when - SockRef :: reference(), - Domain :: domain(). - -which_domain(SockRef) -> - case nif_getopt(SockRef, true, - ?ESOCK_OPT_LEVEL_OTP, ?ESOCK_OPT_OTP_DOMAIN) of - {ok, Domain} -> - if - is_atom(Domain) -> - Domain; - is_integer(Domain) -> - invalid_domain(Domain) - end; - {error, _} = ERROR -> - throw(ERROR) - end. - - -%%%-spec which_type(SockRef) -> Type when -%%% SockRef :: reference(), -%%% Type :: type(). -%%% -%%%which_type(SockRef) -> -%%% case nif_getopt(SockRef, true, -%%% ?ESOCK_OPT_LEVEL_OTP, ?ESOCK_OPT_OTP_TYPE) of -%%% {ok, Type} -> -%%% if -%%% is_atom(Type) -> -%%% Type; -%%% is_integer(Type) -> -%%% invalid_type(Type) -%%% end; -%%% {error, _} = ERROR -> -%%% throw(ERROR) -%%% end. -%%% -%%%-spec which_protocol(SockRef) -> Protocol when -%%% SockRef :: reference(), -%%% Protocol :: protocol(). -%%% -%%%which_protocol(SockRef) -> -%%% case nif_getopt(SockRef, true, -%%% ?ESOCK_OPT_LEVEL_OTP, ?ESOCK_OPT_OTP_PROTOCOL) of -%%% {ok, Proto} -> -%%% if -%%% is_atom(Proto) -> -%%% Proto; -%%% is_integer(Proto) -> -%%% invalid_protocol(Proto) -%%% end; -%%% {error, _} = ERROR -> -%%% throw(ERROR) -%%% end. - -which_dtp(SockRef) -> - case - nif_getopt( - SockRef, true, ?ESOCK_OPT_LEVEL_OTP, ?ESOCK_OPT_OTP_DTP) - of - {ok, {Domain, Type, Proto} = DTP} -> - if - is_integer(Domain) -> - invalid_domain(Domain); - is_integer(Type) -> - invalid_type(Type); - is_integer(Proto) -> - invalid_protocol(Proto); - is_atom(Domain), is_atom(Type), is_atom(Proto) -> - DTP - end; - {error, _} = ERROR -> - throw(ERROR) - end. - - - -%% =========================================================================== -%% -%% sockname - return the current address of the socket. -%% -%% - --spec sockname(Socket) -> {ok, SockAddr} | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - Reason :: term(). - -sockname(#socket{ref = SockRef}) -> - nif_sockname(SockRef). - - - -%% =========================================================================== -%% -%% peername - return the address of the peer *connected* to the socket. -%% -%% - --spec peername(Socket) -> {ok, SockAddr} | {error, Reason} when - Socket :: socket(), - SockAddr :: sockaddr(), - Reason :: term(). - -peername(#socket{ref = SockRef}) -> - nif_peername(SockRef). - - -%% =========================================================================== -%% -%% cancel - cancel an operation resulting in a select -%% -%% A call to accept, recv/recvfrom/recvmsg and send/sendto/sendmsg -%% can result in a select if they are called with the Timeout argument -%% set to nowait. This is indicated by the return of the select-info. -%% Such a operation can be cancelled by calling this function. -%% - --spec cancel(Socket, SelectInfo) -> ok | {error, Reason} when - Socket :: socket(), - SelectInfo :: select_info(), - Reason :: term(). - -cancel(#socket{ref = SockRef}, ?SELECT_INFO(Tag, Ref)) -> - cancel(SockRef, Tag, Ref). - - - -%% =========================================================================== -%% -%% Encode / decode -%% -%% =========================================================================== - -%% File names has to be encoded according to -%% the native file encoding -%% -enc_path(Path) -> - %% These are all BIFs - will not cause code loading - unicode:characters_to_binary(Path, file:native_name_encoding()). - -%% -spec enc_domain(Domain) -> non_neg_integer() when -%% Domain :: domain(). - -enc_domain(local) -> ?ESOCK_DOMAIN_LOCAL; -enc_domain(inet) -> ?ESOCK_DOMAIN_INET; -enc_domain(inet6) -> ?ESOCK_DOMAIN_INET6; -enc_domain(Domain) -> invalid_domain(Domain). - -%% -spec enc_type(Type) -> non_neg_integer() when -%% Type :: type(). - -enc_type(stream) -> ?ESOCK_TYPE_STREAM; -enc_type(dgram) -> ?ESOCK_TYPE_DGRAM; -enc_type(raw) -> ?ESOCK_TYPE_RAW; -enc_type(seqpacket) -> ?ESOCK_TYPE_SEQPACKET; -enc_type(Type) -> invalid_type(Type). - --spec enc_protocol(Protocol) -> non_neg_integer() | - {raw, non_neg_integer()} when - Protocol :: protocol(). - -enc_protocol(default) -> ?ESOCK_PROTOCOL_DEFAULT; -enc_protocol(ip) -> ?ESOCK_PROTOCOL_IP; -enc_protocol(tcp) -> ?ESOCK_PROTOCOL_TCP; -enc_protocol(udp) -> ?ESOCK_PROTOCOL_UDP; -enc_protocol(sctp) -> ?ESOCK_PROTOCOL_SCTP; -enc_protocol(icmp) -> ?ESOCK_PROTOCOL_ICMP; -enc_protocol(igmp) -> ?ESOCK_PROTOCOL_IGMP; -enc_protocol({raw, P} = RAW) when is_integer(P) -> RAW; -enc_protocol(Proto) -> - invalid_protocol(Proto). - - --spec enc_send_flags(Flags) -> non_neg_integer() when - Flags :: send_flags(). - -enc_send_flags(Flags) -> - EFlags = [{confirm, ?ESOCK_SEND_FLAG_CONFIRM}, - {dontroute, ?ESOCK_SEND_FLAG_DONTROUTE}, - {eor, ?ESOCK_SEND_FLAG_EOR}, - {more, ?ESOCK_SEND_FLAG_MORE}, - {nosignal, ?ESOCK_SEND_FLAG_NOSIGNAL}, - {oob, ?ESOCK_SEND_FLAG_OOB}], - enc_flags(Flags, EFlags). - --spec enc_recv_flags(Flags) -> non_neg_integer() when - Flags :: recv_flags(). - -enc_recv_flags(Flags) -> - EFlags = [{cmsg_cloexec, ?ESOCK_RECV_FLAG_CMSG_CLOEXEC}, - {errqueue, ?ESOCK_RECV_FLAG_ERRQUEUE}, - {oob, ?ESOCK_RECV_FLAG_OOB}, - {peek, ?ESOCK_RECV_FLAG_PEEK}, - {trunc, ?ESOCK_RECV_FLAG_TRUNC}], - enc_flags(Flags, EFlags). - - -enc_flags([], _) -> - 0; -enc_flags(Flags, EFlags) -> - F = fun(Flag, Acc) -> - case lists:keysearch(Flag, 1, EFlags) of - {value, {Flag, EFlag}} -> - Acc bor (1 bsl EFlag); - false -> - throw({error, {unknown_flag, Flag}}) - end - end, - lists:foldl(F, 0, Flags). - - -%% +++ Encode setopt level +++ - --spec enc_setopt_level(Level) -> {IsEncoded, EncodedLevel} when - Level :: sockopt_level(), - IsEncoded :: boolean(), - EncodedLevel :: integer(). - -enc_setopt_level(otp) -> - {true, ?ESOCK_OPT_LEVEL_OTP}; -enc_setopt_level(socket) -> - {true, ?ESOCK_OPT_LEVEL_SOCKET}; -enc_setopt_level(ip) -> - {true, ?ESOCK_OPT_LEVEL_IP}; -enc_setopt_level(ipv6) -> - {true, ?ESOCK_OPT_LEVEL_IPV6}; -enc_setopt_level(tcp) -> - {true, ?ESOCK_OPT_LEVEL_TCP}; -enc_setopt_level(udp) -> - {true, ?ESOCK_OPT_LEVEL_UDP}; -enc_setopt_level(sctp) -> - {true, ?ESOCK_OPT_LEVEL_SCTP}; -%% Any option that is of an plain level must be provided as a binary -%% already fully encoded! -enc_setopt_level(L) when is_integer(L) -> - {false, L}. - - -%% +++ Encode setopt key +++ - -%% We should ...really... do something with the domain, type and protocol args... -%% Also, any option (key) which has an integer level (plain) must also be provided -%% in a plain mode, that is, as an integer. -%% Also, not all options are available on all platforms. That is something we -%% don't check here, but in the nif-code. - -enc_setopt_key(Level, Opt, Domain, Type, Protocol) -> - enc_sockopt_key(Level, Opt, set, Domain, Type, Protocol). - - -%% +++ Encode setopt value +++ -%% -%% For the most part this function does *not* do an actual encode, -%% it simply validates the value type. But in some cases it will -%% encode the value into an more "manageable" type. -%% It also handles "aliases" (see linger). - --spec enc_setopt_value(otp, otp_socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (socket, socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (ip, ip_socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (ipv6, ipv6_socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (tcp, tcp_socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (udp, udp_socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (sctp, sctp_socket_option(), - Value, Domain, Type, Protocol) -> term() when - Value :: term(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol() - ; (Level, Opt, - Value, Domain, Type, Protocol) -> term() when - Level :: integer(), - Opt :: integer(), - Value :: binary(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(). - -enc_setopt_value(otp, debug, V, _, _, _) when is_boolean(V) -> - V; -enc_setopt_value(otp, iow, V, _, _, _) when is_boolean(V) -> - V; -enc_setopt_value(otp, controlling_process, V, _, _, _) when is_pid(V) -> - V; -enc_setopt_value(otp, rcvbuf, V, _, _, _) when (V =:= default) -> - 0; % This will cause the nif-code to choose the default value -enc_setopt_value(otp, rcvbuf, V, _, _, _) when is_integer(V) andalso (V > 0) -> - V; -%% N: Number of reads (when specifying length = 0) -%% V: Size of the "read" buffer -enc_setopt_value(otp, rcvbuf, {N, BufSz} = V, _, stream = _T, _P) - when (is_integer(N) andalso (N > 0)) andalso - (is_integer(BufSz) andalso (BufSz > 0)) -> - V; -enc_setopt_value(otp, rcvctrlbuf, V, _, _, _) when (V =:= default) -> - 0; -enc_setopt_value(otp, rcvctrlbuf, V, _, _, _) when is_integer(V) andalso (V > 0) -> - V; -enc_setopt_value(otp, sndctrlbuf, V, _, _, _) when (V =:= default) -> - 0; -enc_setopt_value(otp, sndctrlbuf, V, _, _, _) when is_integer(V) andalso (V > 0) -> - V; -enc_setopt_value(otp, meta, V, _, _, _) -> - V; -enc_setopt_value(otp = L, Opt, V, _D, _T, _P) -> - not_supported({L, Opt, V}); - -enc_setopt_value(socket, bindtodevice, V, _D, _T, _P) when is_list(V) -> - V; -enc_setopt_value(socket, broadcast, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, debug, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(socket, dontroute, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, keepalive, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, linger, abort, D, T, P) -> - enc_setopt_value(socket, linger, {true, 0}, D, T, P); -enc_setopt_value(socket, linger, {OnOff, Secs} = V, _D, _T, _P) - when is_boolean(OnOff) andalso is_integer(Secs) andalso (Secs >= 0) -> - V; -enc_setopt_value(socket, oobinline, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, passcred, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, peek_off = Opt, V, _D, _T, _P) when is_integer(V) -> - %% V; - not_supported(Opt); -enc_setopt_value(socket, priority, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(socket, rcvbuf, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(socket, rcvlowat, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(socket, rcvtimeo, #{sec := Sec, usec := USec} = V, _D, _T, _P) - when is_integer(Sec) andalso is_integer(USec) -> - V; -enc_setopt_value(socket, reuseaddr, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, reuseport, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket, sndbuf, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(socket, sndlowat, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(socket, sndtimeo, #{sec := Sec, usec := USec} = V, _D, _T, _P) - when is_integer(Sec) andalso is_integer(USec) -> - V; -enc_setopt_value(socket, timestamp, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(socket = L, Opt, V, _D, _T, _P) -> - not_supported({L, Opt, V}); - -enc_setopt_value(ip, add_membership, #{multiaddr := MA, - interface := IF} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - ((IF =:= any) orelse (is_tuple(IF) andalso (size(IF) =:= 4))) -> - V; -enc_setopt_value(ip, add_source_membership, #{multiaddr := MA, - interface := IF, - sourceaddr := SA} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - (is_tuple(IF) andalso (size(IF) =:= 4)) andalso - (is_tuple(SA) andalso (size(SA) =:= 4)) -> - V; -enc_setopt_value(ip, block_source, #{multiaddr := MA, - interface := IF, - sourceaddr := SA} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - (is_tuple(IF) andalso (size(IF) =:= 4)) andalso - (is_tuple(SA) andalso (size(SA) =:= 4)) -> - V; -enc_setopt_value(ip, drop_membership, #{multiaddr := MA, - interface := IF} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - ((IF =:= any) orelse (is_tuple(IF) andalso (size(IF) =:= 4))) -> - V; -enc_setopt_value(ip, drop_source_membership, #{multiaddr := MA, - interface := IF, - sourceaddr := SA} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - (is_tuple(IF) andalso (size(IF) =:= 4)) andalso - (is_tuple(SA) andalso (size(SA) =:= 4)) -> - V; -enc_setopt_value(ip, freebind, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(ip, hdrincl, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(ip, minttl, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(ip, msfilter, null = V, _D, _T, _P) -> - V; -enc_setopt_value(ip, msfilter, #{multiaddr := MA, - interface := IF, - fmode := FMode, - slist := SL} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - (is_tuple(IF) andalso (size(IF) =:= 4)) andalso - ((FMode =:= include) orelse (FMode =:= exclude)) andalso - is_list(SL) -> - ensure_ip_msfilter_slist(SL), - V; -enc_setopt_value(ip, mtu_discover, V, _D, _T, _P) - when (V =:= want) orelse - (V =:= dont) orelse - (V =:= do) orelse - (V =:= probe) orelse - is_integer(V) -> - V; -enc_setopt_value(ip, multicast_all, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, multicast_if, V, _D, _T, _P) - when (V =:= any) orelse (is_tuple(V) andalso (size(V) =:= 4)) -> - V; -enc_setopt_value(ip, multicast_loop, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, multicast_ttl, V, _D, _T, _P) - when is_integer(V) andalso (0 =< V) andalso (V =< 255) -> - V; -enc_setopt_value(ip, nodefrag, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, pktinfo, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recvdstaddr, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recverr, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recvif, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recvopts, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recvorigdstaddr, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recvtos, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, recvttl, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, retopts, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, router_alert, V, _D, _T, _P) - when is_integer(V) -> - V; -enc_setopt_value(ip, sendsrcaddr, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, tos, V, _D, _T, _P) - when (V =:= lowdelay) orelse - (V =:= throughput) orelse - (V =:= reliability) orelse - (V =:= mincost) orelse - is_integer(V) -> - V; -enc_setopt_value(ip, transparent, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ip, ttl, V, _D, _T, _P) - when is_integer(V) -> - V; -enc_setopt_value(ip, unblock_source, #{multiaddr := MA, - interface := IF, - sourceaddr := SA} = V, _D, _T, _P) - when (is_tuple(MA) andalso (size(MA) =:= 4)) andalso - (is_tuple(IF) andalso (size(IF) =:= 4)) andalso - (is_tuple(SA) andalso (size(SA) =:= 4)) -> - V; -enc_setopt_value(ip = L, Opt, V, _D, _T, _P) -> - not_supported({L, Opt, V}); - -enc_setopt_value(ipv6, addrform, inet = V, _D, _T, _P) -> - enc_domain(V); -enc_setopt_value(ipv6, add_membership, #{multiaddr := MA, - interface := IF} = V, _D, _T, _P) - when ((is_tuple(MA) andalso (size(MA) =:= 8)) andalso - (is_integer(IF) andalso (IF >= 0))) -> - V; -%% Is this obsolete? When get, the result is enoprotoopt and in the -%% header file it says 'obsolete'... -%% But there might be (old?) versions of linux where it still works... -enc_setopt_value(ipv6, authhdr, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, dstopts, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, drop_membership, #{multiaddr := MA, - interface := IF} = V, _D, _T, _P) - when ((is_tuple(MA) andalso (size(MA) =:= 8)) andalso - (is_integer(IF) andalso (IF >= 0))) -> - V; -enc_setopt_value(ipv6, flowinfo, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, hoplimit, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, hopopts, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, mtu, V, _D, _T, _P) when is_integer(V) -> - V; -enc_setopt_value(ipv6, mtu_discover, V, _D, _T, _P) - when (V =:= want) orelse - (V =:= dont) orelse - (V =:= do) orelse - (V =:= probe) orelse - is_integer(V) -> - V; -enc_setopt_value(ipv6, multicast_hops, V, _D, _T, _P) - when (V =:= default) -> - -1; -enc_setopt_value(ipv6, multicast_hops, V, _D, _T, _P) - when is_integer(V) andalso (V >= 0) andalso (V =< 255) -> - V; -enc_setopt_value(ipv6, multicast_if, V, _D, _T, _P) - when is_integer(V) -> - V; -enc_setopt_value(ipv6, multicast_loop, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ipv6, recverr, V, _D, _T, _P) - when is_boolean(V) -> - V; -enc_setopt_value(ipv6, recvhoplimit, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, Opt, V, _D, _T, _P) - when ((Opt =:= recvpktinfo) orelse (Opt =:= pktinfo)) andalso - is_boolean(V) -> - V; -enc_setopt_value(ipv6, recvtclass, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, router_alert, V, _D, T, _P) - when is_integer(V) andalso (T =:= raw) -> - V; -enc_setopt_value(ipv6, rthdr, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, tclass, V, _D, T, _P) - when is_boolean(V) andalso ((T =:= dgram) orelse (T =:= raw)) -> - V; -enc_setopt_value(ipv6, unicast_hops, V, _D, _T, _P) - when (V =:= default) -> - -1; -enc_setopt_value(ipv6, unicast_hops, V, _D, _T, _P) - when is_integer(V) andalso (V >= 0) andalso (V =< 255) -> - V; -enc_setopt_value(ipv6, v6only, V, _D, _T, _P) when is_boolean(V) -> - V; -enc_setopt_value(ipv6 = L, Opt, V, _D, _T, _P) -> - not_supported({L, Opt, V}); - -enc_setopt_value(tcp, congestion, V, _D, T, P) - when is_list(V) andalso - (T =:= stream) andalso - (P =:= tcp) -> - V; -enc_setopt_value(tcp, cork, V, _D, T, P) - when is_boolean(V) andalso (T =:= stream) andalso (P =:= tcp) -> - V; -enc_setopt_value(tcp, maxseg, V, _D, T, P) - when is_integer(V) andalso - (T =:= stream) andalso - (P =:= tcp) -> - V; -enc_setopt_value(tcp, nodelay, V, _D, T, P) - when is_boolean(V) andalso - (T =:= stream) andalso - (P =:= tcp) -> - V; -enc_setopt_value(tcp = L, Opt, V, _D, _T, _P) -> - not_supported({L, Opt, V}); - -enc_setopt_value(udp, cork, V, _D, T, P) - when is_boolean(V) andalso (T =:= dgram) andalso (P =:= udp) -> - V; -enc_setopt_value(udp = L, Opt, _V, _D, _T, _P) -> - not_supported({L, Opt}); - -enc_setopt_value(sctp, associnfo, #{assoc_id := AssocId, - asocmaxrxt := MaxRxt, - num_peer_dests := NumPeerDests, - peer_rwnd := PeerRWND, - local_rwnd := LocalRWND, - cookie_life := CLife} = V, - _D, _T, P) - when is_integer(AssocId) andalso - is_integer(MaxRxt) andalso (MaxRxt >= 0) andalso - is_integer(NumPeerDests) andalso (NumPeerDests >= 0) andalso - is_integer(PeerRWND) andalso (PeerRWND >= 0) andalso - is_integer(LocalRWND) andalso (LocalRWND >= 0) andalso - is_integer(CLife) andalso (CLife >= 0) andalso - (P =:= sctp) -> - V; -enc_setopt_value(sctp, autoclose, V, _D, _T, P) - when is_integer(V) andalso (V >= 0) andalso (P =:= sctp) -> - V; -enc_setopt_value(sctp, disable_fragments, V, _D, _T, P) - when is_boolean(V) andalso (P =:= sctp) -> - V; -enc_setopt_value(sctp, events, #{data_in := DataIn, - association := Assoc, - address := Addr, - send_failure := SndFailure, - peer_error := PeerError, - shutdown := Shutdown, - partial_delivery := PartialDelivery, - adaptation_layer := AdaptLayer, - authentication := Auth, - sender_dry := SndDry} = V, _D, _T, P) - when is_boolean(DataIn) andalso - is_boolean(Assoc) andalso - is_boolean(Addr) andalso - is_boolean(SndFailure) andalso - is_boolean(PeerError) andalso - is_boolean(Shutdown) andalso - is_boolean(PartialDelivery) andalso - is_boolean(AdaptLayer) andalso - is_boolean(Auth) andalso - is_boolean(SndDry) andalso - (P =:= sctp) -> - V; -enc_setopt_value(sctp, initmsg, #{num_outstreams := NumOut, - max_instreams := MaxIn, - max_attempts := MaxAttempts, - max_init_timeo := MaxInitTO} = V, - _D, _T, P) - when is_integer(NumOut) andalso (NumOut >= 0) andalso - is_integer(MaxIn) andalso (MaxIn >= 0) andalso - is_integer(MaxAttempts) andalso (MaxAttempts >= 0) andalso - is_integer(MaxInitTO) andalso (MaxInitTO >= 0) andalso - (P =:= sctp) -> - V; -enc_setopt_value(sctp, maxseg, V, _D, _T, P) - when is_integer(V) andalso (V >= 0) andalso (P =:= sctp) -> - V; -enc_setopt_value(sctp, nodelay, V, _D, _T, P) - when is_boolean(V) andalso (P =:= sctp) -> - V; -enc_setopt_value(sctp, rtoinfo, #{assoc_id := AssocId, - initial := Init, - max := Max, - min := Min} = V, - _D, _T, P) - when is_integer(AssocId) andalso - is_integer(Init) andalso (Init >= 0) andalso - is_integer(Max) andalso (Max >= 0) andalso - is_integer(Min) andalso (Min >= 0) andalso - (P =:= sctp) -> - V; -enc_setopt_value(sctp = L, Opt, V, _D, _T, _P) -> - not_supported({L, Opt, V}); - -%% enc_setopt_value(raw = L, Opt, _V, _D, _T, _P) -> -%% not_supported({L, Opt}); - -%% Is this correct? What about getopt? -enc_setopt_value(L, Opt, V, _, _, _) - when is_integer(L) andalso is_integer(Opt) andalso is_binary(V) -> - V. - - - - -%% +++ Encode getopt value +++ - -enc_getopt_level(Level) -> - enc_setopt_level(Level). - - -%% +++ Encode getopt key +++ - -enc_getopt_key(Level, Opt, Domain, Type, Protocol) -> - enc_sockopt_key(Level, Opt, get, Domain, Type, Protocol). - - -%% +++ Decode getopt value +++ -%% -%% For the most part, we simply let the value pass through, but for some -%% values we may need to do an actual decode. -%% - -%% This string is NULL-terminated, but the general function we use -%% in the nif code does not know that. So, deal with it here. -dec_getopt_value(tcp = _L, congestion = _Opt, Alg, _D, _T, _P) when is_list(Alg) -> - {Str, _} = lists:splitwith(fun(0) -> false; (_) -> true end, Alg), - Str; -%% Let the user deal with the rest for now... -dec_getopt_value(_L, _Opt, V, _D, _T, _P) -> - V. - - - -%% +++ Encode socket option key +++ - -%% Most options are usable both for set and get, but some are -%% are only available for e.g. get. --spec enc_sockopt_key(Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: otp, - Direction :: set | get, - Opt :: otp_socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: socket, - Direction :: set | get, - Opt :: socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: ip, - Direction :: set | get, - Opt :: ip_socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: ipv6, - Direction :: set | get, - Opt :: ipv6_socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: tcp, - Direction :: set | get, - Opt :: tcp_socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: udp, - Direction :: set | get, - Opt :: udp_socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: sctp, - Direction :: set | get, - Opt :: sctp_socket_option(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> non_neg_integer() when - Level :: integer(), - Direction :: set, - Opt :: integer(), - Domain :: domain(), - Type :: type(), - Protocol :: protocol(); - (Level, Opt, Direction, - Domain, Type, Protocol) -> {NativeOpt, ValueSize} when - Level :: integer(), - Direction :: get, - Opt :: {NativeOpt, ValueSize}, - NativeOpt :: integer(), - ValueSize :: non_neg_integer() | 'int' | 'bool', - Domain :: domain(), - Type :: type(), - Protocol :: protocol(). - - -%% +++ OTP socket options +++ -enc_sockopt_key(otp, debug, _, _, _, _) -> - ?ESOCK_OPT_OTP_DEBUG; -enc_sockopt_key(otp, iow, _, _, _, _) -> - ?ESOCK_OPT_OTP_IOW; -enc_sockopt_key(otp, controlling_process, _, _, _, _) -> - ?ESOCK_OPT_OTP_CTRL_PROC; -enc_sockopt_key(otp, rcvbuf, _, _, _, _) -> - ?ESOCK_OPT_OTP_RCVBUF; -enc_sockopt_key(otp, rcvctrlbuf, _, _, _, _) -> - ?ESOCK_OPT_OTP_RCVCTRLBUF; -enc_sockopt_key(otp, sndctrlbuf, _, _, _, _) -> - ?ESOCK_OPT_OTP_SNDCTRLBUF; -enc_sockopt_key(otp, fd, get = _Dir, _, _, _) -> - ?ESOCK_OPT_OTP_FD; -enc_sockopt_key(otp, meta, _, _, _, _) -> - ?ESOCK_OPT_OTP_META; -enc_sockopt_key(otp = L, Opt, _, _, _, _) -> - not_supported({L, Opt}); - -%% +++ SOCKET socket options +++ -enc_sockopt_key(socket = _L, acceptconn = _Opt, get = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_ACCEPTCONN; -enc_sockopt_key(socket = L, acceptconn = Opt, Dir, _D, _T, _P) -> - not_supported({L, Opt, Dir}); -enc_sockopt_key(socket = L, acceptfilter = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -%% Before linux 3.8, this socket option could be set but not get. -%% Maximum size of buffer for name: IFNAMSIZ -%% So, we let the implementation decide. -enc_sockopt_key(socket = _L, bindtodevice = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_BINDTODEVICE; -enc_sockopt_key(socket, broadcast = _Opt, _Dir, _D, dgram = _T, _P) -> - ?ESOCK_OPT_SOCK_BROADCAST; -enc_sockopt_key(socket = L, busy_poll = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(socket = _L, debug = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_DEBUG; -enc_sockopt_key(socket, domain = _Opt, get = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_DOMAIN; -enc_sockopt_key(socket, dontroute = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_DONTROUTE; -enc_sockopt_key(socket = L, error = Opt, get = _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -%% This is only for connection-oriented sockets, but who are those? -%% Type = stream or Protocol = tcp? -%% For now, we just let is pass and it will fail later if not ok... -enc_sockopt_key(socket, keepalive = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_KEEPALIVE; -enc_sockopt_key(socket, linger = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_LINGER; -enc_sockopt_key(socket = L, mark = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(socket = _L, oobinline = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_OOBINLINE; -enc_sockopt_key(socket, passcred, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_PASSCRED; -enc_sockopt_key(socket = L, peek_off = Opt, _Dir, local = _D, _T, _P) -> - %% ?ESOCK_OPT_SOCK_PEEK_OFF; - not_supported({L, Opt}); -enc_sockopt_key(socket = L, peercred = Opt, get = _Dir, local = _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(socket, priority = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_PRIORITY; -enc_sockopt_key(socket, protocol = _Opt, get = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_PROTOCOL; -enc_sockopt_key(socket, rcvbuf = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_RCVBUF; -enc_sockopt_key(socket = L, rcvbufforce = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -%% May not work on linux. -enc_sockopt_key(socket = _L, rcvlowat = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_RCVLOWAT; -enc_sockopt_key(socket = _L, rcvtimeo = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_RCVTIMEO; -enc_sockopt_key(socket = _L, reuseaddr = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_REUSEADDR; -enc_sockopt_key(socket = _L, reuseport = _Opt, _Dir, D, _T, _P) - when ((D =:= inet) orelse (D =:= inet6)) -> - ?ESOCK_OPT_SOCK_REUSEPORT; -enc_sockopt_key(socket = L, rxq_ovfl = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(socket = L, setfib = Opt, set = _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(socket = _L, sndbuf = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_SNDBUF; -enc_sockopt_key(socket = L, sndbufforce = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -%% Not changeable on linux. -enc_sockopt_key(socket = _L, sndlowat = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_SNDLOWAT; -enc_sockopt_key(socket = _L, sndtimeo = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_SNDTIMEO; -enc_sockopt_key(socket = _L, timestamp = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_TIMESTAMP; -enc_sockopt_key(socket = _L, type = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SOCK_TYPE; -enc_sockopt_key(socket = L, UnknownOpt, _Dir, _D, _T, _P) -> - unknown({L, UnknownOpt}); - -%% +++ IP socket options +++ -enc_sockopt_key(ip = _L, add_membership = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_ADD_MEMBERSHIP; -enc_sockopt_key(ip = _L, add_source_membership = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_ADD_SOURCE_MEMBERSHIP; -enc_sockopt_key(ip = _L, block_source = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_BLOCK_SOURCE; -%% FreeBSD only? -%% Only respected on udp and raw ip (unless the hdrincl option has been set). -enc_sockopt_key(ip = L, dontfrag = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ip = _L, drop_membership = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_DROP_MEMBERSHIP; -enc_sockopt_key(ip = _L, drop_source_membership = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_DROP_SOURCE_MEMBERSHIP; -%% Linux only? -enc_sockopt_key(ip = _L, freebind = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_FREEBIND; -enc_sockopt_key(ip = _L, hdrincl = _Opt, _Dir, _D, raw = _T, _P) -> - ?ESOCK_OPT_IP_HDRINCL; -enc_sockopt_key(ip = _L, minttl = _Opt, _Dir, _D, raw = _T, _P) -> - ?ESOCK_OPT_IP_MINTTL; -enc_sockopt_key(ip = _L, msfilter = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MSFILTER; -enc_sockopt_key(ip = _L, mtu = _Opt, get = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MTU; -enc_sockopt_key(ip = _L, mtu_discover = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MTU_DISCOVER; -enc_sockopt_key(ip = _L, multicast_all = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MULTICAST_ALL; -enc_sockopt_key(ip = _L, multicast_if = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MULTICAST_IF; -enc_sockopt_key(ip = _L, multicast_loop = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MULTICAST_LOOP; -enc_sockopt_key(ip = _L, multicast_ttl = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_MULTICAST_TTL; -enc_sockopt_key(ip = _L, nodefrag = _Opt, _Dir, _D, raw = _T, _P) -> - ?ESOCK_OPT_IP_NODEFRAG; -enc_sockopt_key(ip = L, options = Opt, _Dir, _D, _T, _P) -> - not_supported({Opt, L}); -enc_sockopt_key(ip = _L, pktinfo = _Opt, _Dir, _D, dgram = _T, _P) -> - ?ESOCK_OPT_IP_PKTINFO; -enc_sockopt_key(ip = _L, recvdstaddr = _Opt, _Dir, _D, T, _P) when (T =:= dgram) -> - ?ESOCK_OPT_IP_RECVDSTADDR; -enc_sockopt_key(ip = _L, recverr = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_RECVERR; -enc_sockopt_key(ip = _L, recvif = _Opt, _Dir, _D, T, _P) - when (T =:= dgram) orelse (T =:= raw) -> - ?ESOCK_OPT_IP_RECVIF; -enc_sockopt_key(ip = _L, recvopts = _Opt, _Dir, _D, T, _P) when (T =/= stream) -> - ?ESOCK_OPT_IP_RECVOPTS; -enc_sockopt_key(ip = _L, recvorigdstaddr = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_RECVORIGDSTADDR; -enc_sockopt_key(ip, recvtos = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_RECVTOS; -enc_sockopt_key(ip = _L, recvttl = _Opt, _Dir, _D, T, _P) when (T =/= stream) -> - ?ESOCK_OPT_IP_RECVTTL; -enc_sockopt_key(ip = _L, retopts = _Opt, _Dir, _D, T, _P) when (T =/= stream) -> - ?ESOCK_OPT_IP_RETOPTS; -enc_sockopt_key(ip, router_alert = _Opt, _Dir, _D, raw = _T, _P) -> - ?ESOCK_OPT_IP_ROUTER_ALERT; -enc_sockopt_key(ip, sendsrcaddr = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_SENDSRCADDR; -%% On FreeBSD it specifies that this option is only valid -%% for stream, dgram and "some" raw sockets... -%% No such condition on linux (in the man page)... -enc_sockopt_key(ip, tos = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_TOS; -enc_sockopt_key(ip = _L, transparent = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_TRANSPARENT; -enc_sockopt_key(ip, ttl = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_TTL; -enc_sockopt_key(ip = _L, unblock_source = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IP_UNBLOCK_SOURCE; -enc_sockopt_key(ip = L, UnknownOpt, _Dir, _D, _T, _P) -> - unknown({L, UnknownOpt}); - -%% IPv6 socket options -enc_sockopt_key(ipv6 = _L, addrform = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_ADDRFORM; -enc_sockopt_key(ipv6, add_membership = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_ADD_MEMBERSHIP; -enc_sockopt_key(ipv6 = _L, authhdr = _Opt, _Dir, _D, T, _P) - when ((T =:= dgram) orelse (T =:= raw)) -> - ?ESOCK_OPT_IPV6_AUTHHDR; -enc_sockopt_key(ipv6 = L, auth_level = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = L, checksum = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6, drop_membership = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_DROP_MEMBERSHIP; -enc_sockopt_key(ipv6 = _L, dstopts = _Opt, _Dir, _D, T, _P) - when (T =:= dgram) orelse (T =:= raw) -> - ?ESOCK_OPT_IPV6_DSTOPTS; -enc_sockopt_key(ipv6 = L, esp_trans_level = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = L, esp_network_level = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = _L, flowinfo = _Opt, _Dir, _D, T, _P) - when (T =:= dgram) orelse (T =:= raw) -> - ?ESOCK_OPT_IPV6_FLOWINFO; -enc_sockopt_key(ipv6, hoplimit = _Opt, _Dir, _D, T, _P) - when (T =:= dgram) orelse (T =:= raw) -> - ?ESOCK_OPT_IPV6_HOPLIMIT; -enc_sockopt_key(ipv6 = _L, hopopts = _Opt, _Dir, _D, T, _P) - when ((T =:= dgram) orelse (T =:= raw)) -> - ?ESOCK_OPT_IPV6_HOPOPTS; -enc_sockopt_key(ipv6 = L, ipcomp_level = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = L, join_group = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = L, leave_group = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = _L, mtu = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_MTU; -enc_sockopt_key(ipv6 = _L, mtu_discover = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_MTU_DISCOVER; -enc_sockopt_key(ipv6 = _L, multicast_hops = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_MULTICAST_HOPS; -enc_sockopt_key(ipv6 = _L, multicast_if = _Opt, _Dir, _D, T, _P) - when (T =:= dgram) orelse (T =:= raw) -> - ?ESOCK_OPT_IPV6_MULTICAST_IF; -enc_sockopt_key(ipv6 = _L, multicast_loop = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_MULTICAST_LOOP; -enc_sockopt_key(ipv6 = L, portrange = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = L, pktoptions = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = _L, recverr = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_RECVERR; -enc_sockopt_key(ipv6, recvhoplimit = _Opt, _Dir, _D, T, _P) - when (T =:= dgram) orelse (T =:= raw) -> - ?ESOCK_OPT_IPV6_RECVHOPLIMIT; -enc_sockopt_key(ipv6 = _L, Opt, _Dir, _D, T, _P) - when ((Opt =:= recvpktinfo) orelse (Opt =:= pktinfo)) andalso - ((T =:= dgram) orelse (T =:= raw)) -> - ?ESOCK_OPT_IPV6_RECVPKTINFO; -enc_sockopt_key(ipv6 = _L, recvtclass = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_RECVTCLASS; -enc_sockopt_key(ipv6 = _L, router_alert = _Opt, _Dir, _D, T, _P) when (T =:= raw) -> - ?ESOCK_OPT_IPV6_ROUTER_ALERT; -enc_sockopt_key(ipv6 = _L, rthdr = _Opt, _Dir, _D, T, _P) - when ((T =:= dgram) orelse (T =:= raw)) -> - ?ESOCK_OPT_IPV6_RTHDR; -enc_sockopt_key(ipv6 = _L, tclass = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_TCLASS; -enc_sockopt_key(ipv6 = _L, unicast_hops = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_UNICAST_HOPS; -enc_sockopt_key(ipv6 = L, use_min_mtu = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(ipv6 = _L, v6only = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_IPV6_V6ONLY; -enc_sockopt_key(ipv6 = L, UnknownOpt, _Dir, _D, _T, _P) -> - unknown({L, UnknownOpt}); - -%% TCP socket options -%% There are other options that would be useful; info, -%% but they are difficult to get portable... -enc_sockopt_key(tcp, congestion = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_TCP_CONGESTION; -enc_sockopt_key(tcp = _L, cork = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_TCP_CORK; -enc_sockopt_key(tcp = L, keepidle = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp = L, keepintvl = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp = L, keepcnt = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp, maxseg = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_TCP_MAXSEG; -enc_sockopt_key(tcp = L, md5sig = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp, nodelay = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_TCP_NODELAY; -enc_sockopt_key(tcp = L, noopt = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp = L, nopush = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp = L, syncnt = Opt, _Dir, _D, _T, _P) -> % Only set? 1..255 - not_supported({L, Opt}); -enc_sockopt_key(tcp = L, user_timeout = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(tcp = L, UnknownOpt, _Dir, _D, _T, _P) -> - unknown({L, UnknownOpt}); - -%% UDP socket options -enc_sockopt_key(udp, cork = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_UDP_CORK; -enc_sockopt_key(udp = L, UnknownOpt, _Dir, _D, _T, _P) -> - unknown({L, UnknownOpt}); - -%% SCTP socket options -enc_sockopt_key(sctp = L, adaption_layer = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = _L, associnfo = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_ASSOCINFO; -enc_sockopt_key(sctp = L, auth_active_key = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, auth_asconf = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, auth_chunk = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, auth_key = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, auth_delete_key = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp, autoclose = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_AUTOCLOSE; -enc_sockopt_key(sctp = L, context = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, default_send_params = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, delayed_ack_time = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = _L, disable_fragments = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_DISABLE_FRAGMENTS; -enc_sockopt_key(sctp = L, hmac_ident = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = _L, events = _Opt, set = _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_EVENTS; -enc_sockopt_key(sctp = L, explicit_eor = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, fragment_interleave = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, get_peer_addr_info = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = _L, initmsg = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_INITMSG; -enc_sockopt_key(sctp = L, i_want_mapped_v4_addr = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, local_auth_chunks = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = _L, maxseg = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_MAXSEG; -enc_sockopt_key(sctp = L, maxburst = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp, nodelay = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_NODELAY; -enc_sockopt_key(sctp = L, partial_delivery_point = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, peer_addr_params = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, peer_auth_chunks = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, primary_addr = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, reset_streams = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = _L, rtoinfo = _Opt, _Dir, _D, _T, _P) -> - ?ESOCK_OPT_SCTP_RTOINFO; -enc_sockopt_key(sctp = L, set_peer_primary_addr = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, status = Opt, get = _Dir, _D, _T, _P) -> - not_supported({L, Opt}); % ?ESOCK_OPT_SCTP_RTOINFO; -enc_sockopt_key(sctp = L, use_exp_recvinfo = Opt, _Dir, _D, _T, _P) -> - not_supported({L, Opt}); -enc_sockopt_key(sctp = L, UnknownOpt, _Dir, _D, _T, _P) -> - unknown({L, UnknownOpt}); - -%% +++ "Native" socket options +++ -enc_sockopt_key(Level, Opt, set = _Dir, _D, _T, _P) - when is_integer(Level) andalso is_integer(Opt) -> - Opt; -enc_sockopt_key(Level, {NativeOpt, ValueSize} = Opt, get = _Dir, _D, _T, _P) - when is_integer(Level) andalso - is_integer(NativeOpt) andalso - ((is_integer(ValueSize) andalso (ValueSize >= 0)) orelse - ((ValueSize =:= int) orelse (ValueSize =:= bool))) -> - Opt; - -enc_sockopt_key(Level, Opt, _Dir, _Domain, _Type, _Protocol) -> - unknown({Level, Opt}). - - - -enc_shutdown_how(read) -> - ?ESOCK_SHUTDOWN_HOW_READ; -enc_shutdown_how(write) -> - ?ESOCK_SHUTDOWN_HOW_WRITE; -enc_shutdown_how(read_write) -> - ?ESOCK_SHUTDOWN_HOW_READ_WRITE. - - - - -%% =========================================================================== -%% -%% Misc utility functions -%% -%% =========================================================================== - --dialyzer({nowarn_function, ensure_ip_msfilter_slist/1}). -ensure_ip_msfilter_slist(SL) -> - EnsureSA = fun(SA) when is_tuple(SA) andalso (size(SA) =:= 4) -> ok; - (_) -> einval() - end, - lists:foreach(EnsureSA, SL). - - -ensure_sockaddr(#{family := inet} = SockAddr) -> - maps:merge(?SOCKADDR_IN4_DEFAULTS, SockAddr); -ensure_sockaddr(#{family := inet6} = SockAddr) -> - maps:merge(?SOCKADDR_IN6_DEFAULTS, SockAddr); -ensure_sockaddr(#{family := local, path := Path} = SockAddr) - when is_list(Path) andalso - (length(Path) > 0) andalso - (length(Path) =< 255) -> - BinPath = enc_path(Path), - ensure_sockaddr(SockAddr#{path => BinPath}); -ensure_sockaddr(#{family := local, path := Path} = SockAddr) - when is_binary(Path) andalso - (byte_size(Path) > 0) andalso - (byte_size(Path) =< 255) -> - SockAddr; -ensure_sockaddr(SockAddr) -> - invalid_address(SockAddr). - -ensure_open2_opts(M) when is_map(M) -> - maps:fold( - fun (Key, Val, Acc) -> - case Key of - domain -> - Acc#{domain => enc_domain(Val)}; - type -> - Acc#{type => enc_type(Val)}; - protocol -> - Acc#{protocol => enc_protocol(Val)}; - dup when is_boolean(Val) -> - Acc#{dup => Val}; - debug when is_boolean(Val) -> - Acc#{debug => Val}; - _ -> - Acc - end - end, ?OPEN2_OPTS_DEFAULTS, M). - -cancel(SockRef, Op, OpRef) -> - case nif_cancel(SockRef, Op, OpRef) of - %% The select has already completed - {error, select_sent} -> - flush_select_msg(SockRef, OpRef), - _ = flush_abort_msg(SockRef, OpRef), - ok; - Other -> - _ = flush_abort_msg(SockRef, OpRef), - Other - end. - -flush_select_msg(SockRef, Ref) -> - receive - {?ESOCK_TAG, #socket{ref = SockRef}, select, Ref} -> - ok - after 0 -> - ok - end. - -flush_abort_msg(SockRef, Ref) -> - receive - {?ESOCK_TAG, #socket{ref = SockRef}, abort, {Ref, Reason}} -> - Reason - after 0 -> - ok - end. - -%% formated_timestamp() -> -%% format_timestamp(os:timestamp()). - -%% format_timestamp(Now) -> -%% N2T = fun(N) -> calendar:now_to_local_time(N) end, -%% format_timestamp(Now, N2T, true). - -%% format_timestamp({_N1, _N2, N3} = N, N2T, true) -> -%% FormatExtra = ".~.2.0w", -%% ArgsExtra = [N3 div 10000], -%% format_timestamp(N, N2T, FormatExtra, ArgsExtra); -%% format_timestamp({_N1, _N2, _N3} = N, N2T, false) -> -%% FormatExtra = "", -%% ArgsExtra = [], -%% format_timestamp(N, N2T, FormatExtra, ArgsExtra). - -%% format_timestamp(N, N2T, FormatExtra, ArgsExtra) -> -%% {Date, Time} = N2T(N), -%% {YYYY,MM,DD} = Date, -%% {Hour,Min,Sec} = Time, -%% FormatDate = -%% io_lib:format("~.4w-~.2.0w-~.2.0w ~.2.0w:~.2.0w:~.2.0w" ++ FormatExtra, -%% [YYYY, MM, DD, Hour, Min, Sec] ++ ArgsExtra), -%% lists:flatten(FormatDate). - - -deadline(Timeout) -> - case Timeout of - nowait -> Timeout; - infinity -> Timeout; - _ when is_integer(Timeout), 0 =< Timeout -> - timestamp() + Timeout; - _ -> - invalid_timeout(Timeout) - end. - -timeout(Deadline) -> - case Deadline of - nowait -> 0; - infinity -> infinity; - _ -> - Now = timestamp(), - if - Now < Deadline -> Deadline - Now; - true -> 0 - end - end. - -timestamp() -> - erlang:monotonic_time(milli_seconds). - - --compile({inline, [bincat/2]}). -bincat(<<>>, <<_/binary>> = B) -> B; -bincat(<<_/binary>> = A, <<>>) -> A; -bincat(<<_/binary>> = A, <<_/binary>> = B) -> - <<A/binary, B/binary>>. - - -%% p(F) -> -%% p(F, []). - -%% p(F, A) -> -%% p(get(sname), F, A). - -%% p(undefined, F, A) -> -%% p("***", F, A); -%% p(SName, F, A) -> -%% TS = formated_timestamp(), -%% io:format(user,"[~s][~s,~p] " ++ F ++ "~n", [TS, SName, self()|A]), -%% io:format("[~s][~s,~p] " ++ F ++ "~n", [TS, SName, self()|A]). - - - -%% =========================================================================== -%% -%% Error functions -%% -%% =========================================================================== - --spec invalid_domain(Domain) -> no_return() when - Domain :: term(). -invalid_domain(Domain) -> - error({invalid_domain, Domain}). - --spec invalid_type(Type) -> no_return() when - Type :: term(). -invalid_type(Type) -> - error({invalid_type, Type}). - --spec invalid_protocol(Proto) -> no_return() when - Proto :: term(). -invalid_protocol(Proto) -> - error({invalid_protocol, Proto}). - --spec invalid_address(SockAddr) -> no_return() when - SockAddr :: term(). -invalid_address(SockAddr) -> - error({invalid_address, SockAddr}). - --spec invalid_timeout(Timeout) -> no_return() when - Timeout :: term(). -invalid_timeout(Timeout) -> - error({invalid_timeout, Timeout}). - --spec not_supported(What) -> no_return() when - What :: term(). -not_supported(What) -> - error({not_supported, What}). - --spec unknown(What) -> no_return() when - What :: term(). -unknown(What) -> - error({unknown, What}). - --spec einval() -> no_return(). -einval() -> - error(einval). - --spec error(Reason) -> no_return() when - Reason :: term(). -error(Reason) -> - throw({error, Reason}). - - -%% =========================================================================== -%% -%% Below follows the actual NIF-functions. -%% -%% =========================================================================== - -nif_info() -> - erlang:nif_error(undef). - -nif_info(_SRef) -> - erlang:nif_error(undef). - -nif_command(_Command) -> - erlang:nif_error(undef). - -nif_supports(_Key) -> - erlang:nif_error(undef). - -nif_open(_FD, _Opts) -> - erlang:nif_error(undef). - -nif_open(_Domain, _Type, _Protocol, _Opts) -> - erlang:nif_error(undef). - -nif_bind(_SRef, _SockAddr) -> - erlang:nif_error(undef). - -nif_bind(_SRef, _SockAddrs, _Action) -> - erlang:nif_error(undef). - -nif_connect(_SRef, _SockAddr) -> - erlang:nif_error(undef). - -nif_finalize_connection(_SRef) -> - erlang:nif_error(undef). - -nif_listen(_SRef, _Backlog) -> - erlang:nif_error(undef). - -nif_accept(_SRef, _Ref) -> - erlang:nif_error(undef). - -nif_send(_SockRef, _SendRef, _Data, _Flags) -> - erlang:nif_error(undef). - -nif_sendto(_SRef, _SendRef, _Data, _Dest, _Flags) -> - erlang:nif_error(undef). - -nif_sendmsg(_SRef, _SendRef, _MsgHdr, _Flags) -> - erlang:nif_error(undef). - -nif_recv(_SRef, _RecvRef, _Length, _Flags) -> - erlang:nif_error(undef). - -nif_recvfrom(_SRef, _RecvRef, _Length, _Flags) -> - erlang:nif_error(undef). - -nif_recvmsg(_SRef, _RecvRef, _BufSz, _CtrlSz, _Flags) -> - erlang:nif_error(undef). - -nif_cancel(_SRef, _Op, _Ref) -> - erlang:nif_error(undef). - -nif_close(_SRef) -> - erlang:nif_error(undef). - -nif_shutdown(_SRef, _How) -> - erlang:nif_error(undef). - -nif_finalize_close(_SRef) -> - erlang:nif_error(undef). - -nif_setopt(_Ref, _IsEnc, _Lev, _Key, _Val) -> - erlang:nif_error(undef). - -nif_getopt(_Ref, _IsEnc, _Lev, _Key) -> - erlang:nif_error(undef). - -nif_sockname(_Ref) -> - erlang:nif_error(undef). - -nif_peername(_Ref) -> - erlang:nif_error(undef). - diff --git a/erts/vsn.mk b/erts/vsn.mk index 5b272fcce1..1f9796c5cd 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -18,7 +18,7 @@ # %CopyrightEnd% # -VSN = 10.7.1 +VSN = 11.0.1 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index ddc115c98c..0412d40423 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -32,6 +32,57 @@ <p>This document describes the changes made to the asn1 application.</p> +<section><title>Asn1 5.0.13</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Adhere to the ASN.1 specification for hstring & + bstring lexical items. That is they may include white + space.</p> + <p> + Own Id: OTP-16490</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p> + Improve handling of ellipsis in a CHOICE</p> + <p> + Own Id: OTP-16554 Aux Id: ERL-1189 </p> + </item> + </list> + </section> + +</section> + +<section><title>Asn1 5.0.12</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Dialyzer warnings of type <c>no_match</c> are now + suppressed in the generated files.</p> + <p> + Own Id: OTP-16636 Aux Id: ERIERL-145 </p> + </item> + </list> + </section> + +</section> + <section><title>Asn1 5.0.11</title> <section><title>Improvements and New Features</title> diff --git a/lib/asn1/src/asn1ct_gen.erl b/lib/asn1/src/asn1ct_gen.erl index 4f8b428b54..30396821b7 100644 --- a/lib/asn1/src/asn1ct_gen.erl +++ b/lib/asn1/src/asn1ct_gen.erl @@ -1314,7 +1314,9 @@ gen_head(#gen{options=Options}=Gen, Mod, Hrl) -> Mod,".",nl,nl, "-module('",Mod,"').",nl, "-compile(nowarn_unused_vars).",nl, - "-dialyzer(no_improper_lists).",nl]), + "-dialyzer(no_improper_lists).",nl, + "-dialyzer(no_match).",nl + ]), case Hrl of 0 -> ok; _ -> emit(["-include(\"",Mod,".hrl\").",nl]) diff --git a/lib/asn1/test/asn1_SUITE.erl b/lib/asn1/test/asn1_SUITE.erl index ca7949fb37..f75a385cf5 100644 --- a/lib/asn1/test/asn1_SUITE.erl +++ b/lib/asn1/test/asn1_SUITE.erl @@ -45,10 +45,112 @@ all() -> [xref, xref_export_all, - {group, compile}, - {group, parallel}, + c_string, + constraint_equivalence, + + ber_decode_invalid_length, + ber_choiceinseq, + ber_optional, + tagdefault_automatic, + + cover, + + parse, + test_undecoded_rest, + specialized_decodes, + special_decode_performance, + + testMegaco, + testConstraints, + testCompactBitString, + default, + testPrim, + rtUI, + testPrimStrings, + + per, + ber, + der, + + h323test, + testExtensibilityImplied, + testChoice, + testDefaultOctetString, + testMultipleLevels, + testOpt, + testSeqDefault, + testMaps, + + testTypeValueNotation, + + testExternal, + + testSeqExtension, + testSeqOptional, + testSeqPrim, + testSeqTypeRefCho, + testSeqTypeRefPrim, + testSeqTypeRefSeq, + testSeqTypeRefSet, + + testSeqOf, + testSeqOfIndefinite, + testSeqOfCho, + testSeqOfChoExt, + + testExtensionAdditionGroup, + + testSet, + testSetOf, + + testEnumExt, + value_test, + testSeq2738, + constructed, + ber_decode_error, + otp_14440, + testSeqSetIndefinite, + testChoiceIndefinite, + per_open_type, + testInfObjectClass, + testUniqueObjectSets, + testInfObjExtract, + testParam, + testFragmented, + testMergeCompile, + testobj, + testDeepTConstr, + testImport, + testDER, + testDEFAULT, + testExtensionDefault, + testMvrasn6, + testContextSwitchingTypes, + testOpenTypeImplicitTag, + testROSE, + testINSTANCE_OF, + testTCAP, + test_ParamTypeInfObj, + test_Defed_ObjectIdentifier, + testSelectionType, + testSSLspecs, + testNortel, + test_WS_ParamClass, + test_modified_x420, + + %% Some heavy tests. + testTcapsystem, + testNBAPsystem, + testS1AP, + testRfcs, + + test_compile_options, + testDoubleEllipses, + test_x691, + ticket_6143, + test_OTP_9688, + testValueTest, - % TODO: Investigate parallel running of these: testComment, testName2Number, ticket_7407, @@ -57,129 +159,7 @@ all() -> {group, performance}]. groups() -> - Parallel = asn1_test_lib:parallel(), - [{compile, Parallel, - [c_string, - constraint_equivalence]}, - - {ber, Parallel, - [ber_decode_invalid_length, - ber_choiceinseq, - % Uses 'SOpttest' - ber_optional, - tagdefault_automatic]}, - - {parallel, Parallel, - [cover, - {group, ber}, - % Uses 'P-Record', 'Constraints', 'MEDIA-GATEWAY-CONTROL'... - {group, [], [parse, - test_undecoded_rest, - specialized_decodes, - special_decode_performance, - testMegaco, - testConstraints, - testCompactBitString]}, - default, - % Uses 'Def', 'MULTIMEDIA-SYSTEM-CONTROL', 'H323-MESSAGES', 'Prim', - % 'Real' - {group, [], [testPrim, - rtUI, - testPrimStrings, - per, - ber_other, - der, - h323test]}, - testExtensibilityImplied, - testChoPrim, - testChoExtension, - testChoOptional, - testChoRecursive, - testChoTypeRefCho, - testChoTypeRefPrim, - testChoTypeRefSeq, - testChoTypeRefSet, - testDefaultOctetString, - testMultipleLevels, - testOpt, - testSeqDefault, - testMaps, - % Uses 'External' - {group, [], [testExternal, - testSeqExtension]}, - testSeqOptional, - testSeqPrim, - testSeqTypeRefCho, - % Uses 'SeqTypeRefPrim' - {group, [], [testSeqTypeRefPrim, - testTypeValueNotation]}, - testSeqTypeRefSeq, - testSeqTypeRefSet, - % Uses 'SeqOf' - {group, [], [testSeqOf, - testSeqOfIndefinite]}, % Uses 'Mvrasn*' - testSeqOfCho, - testSeqOfChoExt, - testSetDefault, - testExtensionAdditionGroup, - testSetOptional, - testSetPrim, - testSetTypeRefCho, - testSetTypeRefPrim, - testSetTypeRefSeq, - testSetTypeRefSet, - testSetOf, - testSetOfCho, - testEnumExt, - value_test, - testSeq2738, - % Uses 'Constructed' - {group, [], [constructed, - ber_decode_error, - otp_14440]}, - testSeqSetIndefinite, - testChoiceIndefinite, - per_open_type, - testInfObjectClass, - testUniqueObjectSets, - testInfObjExtract, - testParam, - testFragmented, - testMergeCompile, - testobj, - testDeepTConstr, - testImport, - testDER, - testDEFAULT, - testExtensionDefault, - testMvrasn6, - testContextSwitchingTypes, - testOpenTypeImplicitTag, - testROSE, - testINSTANCE_OF, - testTCAP, - test_ParamTypeInfObj, - test_Defed_ObjectIdentifier, - testSelectionType, - testSSLspecs, - testNortel, - % Uses 'PKCS7', 'InformationFramework' - {group, [], [test_WS_ParamClass, - test_modified_x420]}, - %% Don't run all these at the same time. - {group, [], - [testTcapsystem, - testNBAPsystem, - testS1AP, - testRfcs]}, - test_compile_options, - testDoubleEllipses, - test_x691, - ticket_6143, - test_OTP_9688, - testValueTest]}, - - {performance, [], + [{performance, [], [testTimer_ber, testTimer_ber_maps, testTimer_per, @@ -326,30 +306,26 @@ do_test_prim(Rule, NoOkWrapper) -> testCompactBitString(Config) -> test(Config, fun testCompactBitString/3). testCompactBitString(Config, Rule, Opts) -> - asn1_test_lib:compile("PrimStrings", Config, - [Rule, compact_bit_string|Opts]), + Files = ["PrimStrings", "Constraints"], + asn1_test_lib:compile_all(Files, Config, [Rule, compact_bit_string|Opts]), testCompactBitString:compact_bit_string(Rule), testCompactBitString:bit_string_unnamed(Rule), testCompactBitString:bit_string_unnamed(Rule), testCompactBitString:ticket_7734(Rule), - asn1_test_lib:compile("Constraints", Config, - [Rule, compact_bit_string|Opts]), testCompactBitString:otp_4869(Rule). testPrimStrings(Config) -> test(Config, fun testPrimStrings/3, [ber,{ber,[der]},per,uper]). testPrimStrings(Config, Rule, Opts) -> LegacyOpts = [legacy_erlang_types|Opts], - asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, - [Rule|LegacyOpts]), + Files = ["PrimStrings", "BitStr"], + asn1_test_lib:compile_all(Files, Config, [Rule|LegacyOpts]), testPrimStrings_cases(Rule, LegacyOpts), - asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, [Rule|Opts]), + asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), testPrimStrings_cases(Rule, Opts), - asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, - [legacy_bit_string,Rule|Opts]), + asn1_test_lib:compile_all(Files, Config, [legacy_bit_string,Rule|Opts]), testPrimStrings:bit_string(Rule, Opts), - asn1_test_lib:compile_all(["PrimStrings", "BitStr"], Config, - [compact_bit_string,Rule|Opts]), + asn1_test_lib:compile_all(Files, Config, [compact_bit_string,Rule|Opts]), testPrimStrings:bit_string(Rule, Opts), testPrimStrings:more_strings(Rule). @@ -398,46 +374,26 @@ testExtensibilityImplied(Config, Rule, Opts) -> [Rule,no_ok_wrapper|Opts]), testExtensibilityImplied:main(Rule). -testChoPrim(Config) -> test(Config, fun testChoPrim/3). -testChoPrim(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoPrim", Config, [Rule|Opts]), +testChoice(Config) -> test(Config, fun testChoice/3). +testChoice(Config, Rule, Opts) -> + Files = ["ChoPrim", + "ChoExtension", + "ChoOptional", + "ChoOptionalImplicitTag", + "ChoRecursive", + "ChoTypeRefCho", + "ChoTypeRefPrim", + "ChoTypeRefSeq", + "ChoTypeRefSet"], + asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), testChoPrim:bool(Rule), - testChoPrim:int(Rule). - -testChoExtension(Config) -> test(Config, fun testChoExtension/3). -testChoExtension(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoExtension", Config, [Rule|Opts]), - testChoExtension:extension(Rule). - -testChoOptional(Config) -> test(Config, fun testChoOptional/3). -testChoOptional(Config, Rule, Opts) -> - asn1_test_lib:compile_all(["ChoOptional", - "ChoOptionalImplicitTag"], Config, [Rule|Opts]), - testChoOptional:run(). - -testChoRecursive(Config) -> test(Config, fun testChoRecursive/3). -testChoRecursive(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoRecursive", Config, [Rule|Opts]), - testChoRecursive:recursive(Rule). - -testChoTypeRefCho(Config) -> test(Config, fun testChoTypeRefCho/3). -testChoTypeRefCho(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoTypeRefCho", Config, [Rule|Opts]), - testChoTypeRefCho:choice(Rule). - -testChoTypeRefPrim(Config) -> test(Config, fun testChoTypeRefPrim/3). -testChoTypeRefPrim(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoTypeRefPrim", Config, [Rule|Opts]), - testChoTypeRefPrim:prim(Rule). - -testChoTypeRefSeq(Config) -> test(Config, fun testChoTypeRefSeq/3). -testChoTypeRefSeq(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoTypeRefSeq", Config, [Rule|Opts]), - testChoTypeRefSeq:seq(Rule). - -testChoTypeRefSet(Config) -> test(Config, fun testChoTypeRefSet/3). -testChoTypeRefSet(Config, Rule, Opts) -> - asn1_test_lib:compile("ChoTypeRefSet", Config, [Rule|Opts]), + testChoPrim:int(Rule), + testChoExtension:extension(Rule), + testChoOptional:run(), + testChoRecursive:recursive(Rule), + testChoTypeRefCho:choice(Rule), + testChoTypeRefPrim:prim(Rule), + testChoTypeRefSeq:seq(Rule), testChoTypeRefSet:set(Rule). testDefaultOctetString(Config) -> test(Config, fun testDefaultOctetString/3). @@ -564,50 +520,33 @@ testSeqOfIndefinite(Config, Rule, Opts) -> asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), testSeqOfIndefinite:main(). -testSetDefault(Config) -> test(Config, fun testSetDefault/3). -testSetDefault(Config, Rule, Opts) -> - asn1_test_lib:compile("SetDefault", Config, [Rule|Opts]), - testSetDefault:main(Rule). +testSet(Config) -> test(Config, fun testSet/3). +testSet(Config, Rule, Opts) -> + Files = ["SetDefault", + "SetOptional", + "SetPrim", + "SetTypeRefCho", + "SetTypeRefPrim", + "SetTypeRefSeq", + "SetTypeRefSet"], + asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), -testSetOptional(Config) -> test(Config, fun testSetOptional/3). -testSetOptional(Config, Rule, Opts) -> - asn1_test_lib:compile("SetOptional", Config, [Rule|Opts]), + testSetDefault:main(Rule), testSetOptional:ticket_7533(Rule), - testSetOptional:main(Rule). - -testSetPrim(Config) -> test(Config, fun testSetPrim/3). -testSetPrim(Config, Rule, Opts) -> - asn1_test_lib:compile("SetPrim", Config, [Rule|Opts]), - testSetPrim:main(Rule). - -testSetTypeRefCho(Config) -> test(Config, fun testSetTypeRefCho/3). -testSetTypeRefCho(Config, Rule, Opts) -> - asn1_test_lib:compile("SetTypeRefCho", Config, [Rule|Opts]), - testSetTypeRefCho:main(Rule). - -testSetTypeRefPrim(Config) -> test(Config, fun testSetTypeRefPrim/3). -testSetTypeRefPrim(Config, Rule, Opts) -> - asn1_test_lib:compile("SetTypeRefPrim", Config, [Rule|Opts]), - testSetTypeRefPrim:main(Rule). - -testSetTypeRefSeq(Config) -> test(Config, fun testSetTypeRefSeq/3). -testSetTypeRefSeq(Config, Rule, Opts) -> - asn1_test_lib:compile("SetTypeRefSeq", Config, [Rule|Opts]), - testSetTypeRefSeq:main(Rule). - -testSetTypeRefSet(Config) -> test(Config, fun testSetTypeRefSet/3). -testSetTypeRefSet(Config, Rule, Opts) -> - asn1_test_lib:compile("SetTypeRefSet", Config, [Rule|Opts]), + testSetOptional:main(Rule), + + testSetPrim:main(Rule), + testSetTypeRefCho:main(Rule), + testSetTypeRefPrim:main(Rule), + testSetTypeRefSeq:main(Rule), testSetTypeRefSet:main(Rule). testSetOf(Config) -> test(Config, fun testSetOf/3). testSetOf(Config, Rule, Opts) -> - asn1_test_lib:compile("SetOf", Config, [Rule|Opts]), - testSetOf:main(Rule). - -testSetOfCho(Config) -> test(Config, fun testSetOfCho/3). -testSetOfCho(Config, Rule, Opts) -> - asn1_test_lib:compile("SetOfCho", Config, [Rule|Opts]), + Files = ["SetOf", + "SetOfCho"], + asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), + testSetOf:main(Rule), testSetOfCho:main(Rule). c_string(Config) -> @@ -657,19 +596,23 @@ parse(Config) -> per(Config) -> test(Config, fun per/3, [per,uper,{per,[maps]},{uper,[maps]}]). per(Config, Rule, Opts) -> - [module_test(M, Config, Rule, Opts) || M <- per_modules()]. + module_test(per_modules(), Config, Rule, Opts). -ber_other(Config) -> - test(Config, fun ber_other/3, [ber,{ber,[maps]}]). +ber(Config) -> + test(Config, fun ber/3, [ber,{ber,[maps]}]). -ber_other(Config, Rule, Opts) -> - [module_test(M, Config, Rule, Opts) || M <- ber_modules()]. +ber(Config, Rule, Opts) -> + module_test(ber_modules(), Config, Rule, Opts). der(Config) -> asn1_test_lib:compile_all(ber_modules(), Config, [der]). -module_test(M0, Config, Rule, Opts) -> - asn1_test_lib:compile(M0, Config, [Rule,?NO_MAPS_MODULE|Opts]), +module_test(Modules, Config, Rule, Opts) -> + asn1_test_lib:compile_all(Modules, Config, [Rule,?NO_MAPS_MODULE|Opts]), + _ = [do_module_test(M, Config, Opts) || M <- Modules], + ok. + +do_module_test(M0, Config, Opts) -> case list_to_atom(M0) of 'LDAP' -> %% Because of the recursive definition of 'Filter' in @@ -815,8 +758,8 @@ per_open_type(Config, Rule, Opts) -> testConstraints(Config) -> test(Config, fun testConstraints/3). testConstraints(Config, Rule, Opts) -> - asn1_test_lib:compile("Constraints", Config, [Rule|Opts]), - asn1_test_lib:compile("LargeConstraints", Config, [Rule|Opts]), + Files = ["Constraints", "LargeConstraints"], + asn1_test_lib:compile_all(Files, Config, [Rule|Opts]), testConstraints:int_constraints(Rule), case Rule of ber -> ok; @@ -1163,9 +1106,6 @@ testExtensionAdditionGroup(Config, Rule, Opts) -> [Rule,{record_name_prefix,"RRC-"}|Opts]), extensionAdditionGroup:run(Rule). -% parse_modules() -> -% ["ImportsFrom"]. - per_modules() -> [X || X <- test_modules()]. diff --git a/lib/asn1/test/asn1_test_lib.erl b/lib/asn1/test/asn1_test_lib.erl index af8462f0c9..2c91256d6d 100644 --- a/lib/asn1/test/asn1_test_lib.erl +++ b/lib/asn1/test/asn1_test_lib.erl @@ -24,7 +24,6 @@ rm_dirs/1, hex_to_bin/1, match_value/2, - parallel/0, roundtrip/3,roundtrip/4,roundtrip_enc/3,roundtrip_enc/4, map_roundtrip/3]). @@ -48,12 +47,6 @@ compile_all(Files, Config, Options0) -> dialyze(Files, Options), ok. -parallel() -> - case erlang:system_info(schedulers) > 1 andalso not run_dialyzer() of - true -> [parallel]; - false -> [] - end. - dialyze(Files, Options) -> case not run_dialyzer() orelse lists:member(abs, Options) of true -> ok; diff --git a/lib/asn1/test/testEnumExt.erl b/lib/asn1/test/testEnumExt.erl index b30750f7fc..cb01ad481f 100644 --- a/lib/asn1/test/testEnumExt.erl +++ b/lib/asn1/test/testEnumExt.erl @@ -41,7 +41,7 @@ main(Rule) when Rule =:= per; Rule =:= uper -> B64 = roundtrip('Noext', red), common(Rule); main(Rule) when Rule =:= ber; Rule =:= jer -> - io:format("main(ber)~n",[]), + io:format("main(~p)~n",[Rule]), %% ENUMERATED with extensionmark (value is in root set) roundtrip('Ext', red), diff --git a/lib/asn1/test/testMegaco.erl b/lib/asn1/test/testMegaco.erl index 0be798b962..6a88117896 100644 --- a/lib/asn1/test/testMegaco.erl +++ b/lib/asn1/test/testMegaco.erl @@ -26,8 +26,8 @@ -include_lib("common_test/include/ct.hrl"). compile(Config, Erule, Options) -> - asn1_test_lib:compile("MEDIA-GATEWAY-CONTROL.asn", Config, [Erule|Options]), - asn1_test_lib:compile("OLD-MEDIA-GATEWAY-CONTROL.asn", Config, [Erule|Options]), + Files = ["MEDIA-GATEWAY-CONTROL.asn","OLD-MEDIA-GATEWAY-CONTROL.asn"], + asn1_test_lib:compile_all(Files, Config, [Erule|Options]), {ok,'OLD-MEDIA-GATEWAY-CONTROL','MEDIA-GATEWAY-CONTROL'}. main(no_module,_) -> ok; diff --git a/lib/asn1/vsn.mk b/lib/asn1/vsn.mk index e1d3b65da2..8ebbe907bc 100644 --- a/lib/asn1/vsn.mk +++ b/lib/asn1/vsn.mk @@ -1 +1 @@ -ASN1_VSN = 5.0.11 +ASN1_VSN = 5.0.13 diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 70653c0711..9545d8352e 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -33,6 +33,44 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.19</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The function <c>ct_property_test:init_tool/1</c> is added + for the cases when the user does not want + ct_property_test to compile properties. init_tool/1 can + be used to set the property_test_tool config value.</p> + <p> + Own Id: OTP-16029 Aux Id: PR-2145 </p> + </item> + <item> + <p> + The built-in Common Test Hook, <c>cth_log_redirect</c>, + has been updated to use the system <c>default</c> Logger + handler's configuration instead of its own. See the + section on <seeguide + marker="common_test:ct_hooks_chapter#built-in-cths">Built-in + Hooks</seeguide> in the Common Test User's Guide.</p> + <p> + Own Id: OTP-16273</p> + </item> + <item> + <p> + Calls of deprecated functions in the <seeguide + marker="crypto:new_api#the-old-api">Old Crypto + API</seeguide> are replaced by calls of their <seeguide + marker="crypto:new_api#the-new-api">substitutions</seeguide>.</p> + <p> + Own Id: OTP-16346</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.18.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index 1e6991d73a..b5fa287e1f 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1 +1 @@ -COMMON_TEST_VSN = 1.18.2 +COMMON_TEST_VSN = 1.19 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 079c84b776..a37ac251de 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -32,6 +32,129 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 7.6.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + In rare circumstances, a guard using 'not' could evaluate + to the wrong boolean value.</p> + <p> + Own Id: OTP-16652 Aux Id: ERL-1246 </p> + </item> + <item> + <p>A guard expression that referenced a variable bound to + a boolean expression could evaluate to the wrong + value.</p> + <p> + Own Id: OTP-16657 Aux Id: ERL-1253 </p> + </item> + </list> + </section> + +</section> + +<section><title>Compiler 7.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p><c>erlang:fun_info(fun foo/1, name/1)</c> used to + return a function name based on the name of the function + that <c>fun foo/1</c> was used in. The name returned is + now <c>-fun.foo/1-</c>.</p> + <p> + Own Id: OTP-15837</p> + </item> + <item> + <p> Initialization of record fields using <c>_</c> is no + longer allowed if the number of affected fields is zero. + </p> + <p> + Own Id: OTP-16516</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>EEP-52 has been implemented.</p> + <p>In binary matching, the size of the segment to be + matched is now allowed to be a guard expression, and + similarly in map matching the keys can now be guard + expressions. See the Erlang Reference Manual and + Programming Examples for more details.</p> + <p>Language compilers or code generators that generate + Core Erlang code may need to be updated to be compatible + with the compiler in OTP 23. For more details, see the + section Backwards Compatibility in <url + href="http://erlang.org/eeps/eep-0052.html">EEP + 52</url>.</p> + <p> + Own Id: OTP-14708</p> + </item> + <item> + <p> Allow underscores in numeric literals to improve + readability. Examples: <c>123_456_789</c>, + <c>16#1234_ABCD</c>. </p> + <p> + Own Id: OTP-16007 Aux Id: PR-2324 </p> + </item> + <item> + <p>Improved the type optimization pass' inference of + types that depend on themselves, giving us more accurate + types and letting us track the content types of + lists.</p> + <p> + Own Id: OTP-16214 Aux Id: PR-2460 </p> + </item> + <item> + <p> + Support message queue optimization also for references + returned from the new <seemfa + marker="erts:erlang#spawn_request/5"><c>spawn_request()</c></seemfa> + BIFs.</p> + <p> + Own Id: OTP-16367 Aux Id: OTP-15251 </p> + </item> + <item> + <p>The compiler will now raise a warning when inlining is + used in modules that load NIFs.</p> + <p> + Own Id: OTP-16429 Aux Id: ERL-303 </p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p>Line information was sometimes incorrect for + floating-point math exceptions.</p> + <p> + Own Id: OTP-16505 Aux Id: ERL-1178 </p> + </item> + <item> + <p>The <c>debug_info</c> option can now be specified in + <c>-compile()</c> attributes.</p> + <p> + Own Id: OTP-16523 Aux Id: ERL-1058 </p> + </item> + <item> + <p>Reduced the resource usage of <c>erlc</c> in parallel + builds (e.g. <c>make -j128</c>).</p> + <p> + Own Id: OTP-16543 Aux Id: ERL-1186 </p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 7.5.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_ssa_bool.erl b/lib/compiler/src/beam_ssa_bool.erl index 7ae9070df2..5b81ca2be1 100644 --- a/lib/compiler/src/beam_ssa_bool.erl +++ b/lib/compiler/src/beam_ssa_bool.erl @@ -922,9 +922,11 @@ opt_digraph_instr(#b_set{dst=Dst}=I, G0, St) -> %% Rewriting 'xor' is not practical. Fortunately, %% 'xor' is almost never used in practice. not_possible(); - #b_set{op={bif,'not'},args=[#b_var{}=Bool]} -> - G = convert_to_br_node(I, Fail, G1, St), - redirect_test(Bool, {fail,Succ}, G, St); + #b_set{op={bif,'not'}} -> + %% This is suprisingly rare. The previous attempt to + %% optimize it was broken, which wasn't noticed because + %% very few test cases triggered this code. + not_possible(); #b_set{op=phi,dst=Bool} -> Vtx = get_vertex(Bool, St), G2 = del_out_edges(Vtx, G1), @@ -1070,7 +1072,60 @@ redirect_phi(Phi, Args, SuccFail, G0, St) -> redirect_phi_1(PhiVtx, [{#b_literal{val=false},FalseExit}, {#b_var{}=SuccBool,_BoolExit}], SuccFail, G0, St) -> + %% This was most likely an `andalso` in the source code. BoolVtx = get_vertex(SuccBool, St), + + %% We must be careful when rewriting guards that reference boolean + %% expressions defined before the guard. Here is an example: + %% + %% Bool = Z =:= false, + %% if + %% X =:= Y andalso Bool -> ok; + %% true -> error + %% end. + %% + %% Slightly simplified, the SSA code will look like this: + %% + %% 10: Bool = bif:'=:=' _2, `false` + %% br ^11 + %% + %% 11: B = bif:'=:=' X, Y + %% br B, ^20, ^30 + %% + %% 20: br ^40 + %% 30: br ^40 + %% + %% 40: Phi = phi { `true`, ^20 }, { Bool, ^30 } + %% br Phi, ^100, ^200 + %% + %% 100: ret `ok` + %% 200: ret `error' + %% + %% The usual rewriting of the phi node will result in the following + %% SSA code: + %% + %% 10: Bool = bif:'=:=' _2, `false` + %% br Bool, ^100, ^200 + %% + %% 11: B = bif:'=:=' X, Y + %% br B, ^100, ^200 + %% + %% 20: br ^40 + %% 30: br ^40 + %% + %% 40: Phi = phi { `true`, ^20 }, { Bool, ^30 } + %% br Phi, ^100, ^200 + %% + %% 100: ret `ok` + %% 200: ret `error' + %% + %% Block 11 is no longer reachable; thus, the X =:= Y test has been dropped. + %% To avoid dropping tests, we should check whether if there is a path from + %% 10 to block 20. If there is, the optimization in its current form is not + %% safe. + %% + ensure_disjoint_paths(G0, BoolVtx, FalseExit), + [FalseOut] = beam_digraph:out_edges(G0, FalseExit), G1 = beam_digraph:del_edge(G0, FalseOut), case SuccFail of @@ -1088,6 +1143,11 @@ redirect_phi_1(PhiVtx, [{#b_literal{val=true},TrueExit}, {fail,Fail}, G0, St) -> %% This was probably an `orelse` in the source code. BoolVtx = get_vertex(SuccBool, St), + + %% See the previous clause for an explanation of why we + %% must ensure that paths are disjoint. + ensure_disjoint_paths(G0, BoolVtx, TrueExit), + [TrueOut] = beam_digraph:out_edges(G0, TrueExit), G1 = beam_digraph:del_edge(G0, TrueOut), G2 = beam_digraph:add_edge(G1, TrueExit, PhiVtx, next), @@ -1117,6 +1177,18 @@ digraph_bool_def(G) -> Ds = [{Dst,Vtx} || {Vtx,#b_set{dst=Dst}} <- Vs], maps:from_list(Ds). + +%% ensure_disjoint_paths(G, Vertex1, Vertex2) -> ok. +%% Ensure that there is no path from Vertex1 to Vertex2 in +%% either direction. (It is probably overkill to test both +%% directions, but better safe than sorry.) + +ensure_disjoint_paths(G, V1, V2) -> + case beam_digraph:is_path(G, V1, V2) orelse beam_digraph:is_path(G, V2, V1) of + true -> not_possible(); + false -> ok + end. + %%% %%% Shortcut branches that branch to other branches. %%% diff --git a/lib/compiler/src/beam_ssa_opt.erl b/lib/compiler/src/beam_ssa_opt.erl index 1549b16cbc..d77b7756e8 100644 --- a/lib/compiler/src/beam_ssa_opt.erl +++ b/lib/compiler/src/beam_ssa_opt.erl @@ -40,7 +40,8 @@ -include("beam_ssa_opt.hrl"). -import(lists, [all/2,append/1,duplicate/2,flatten/1,foldl/3, - keyfind/3,last/1,member/2,reverse/1,reverse/2, + keyfind/3,last/1,mapfoldl/3,member/2, + partition/2,reverse/1,reverse/2, splitwith/2,sort/1,takewhile/2,unzip/1]). -define(MAX_REPETITIONS, 16). @@ -299,6 +300,7 @@ epilogue_passes(Opts) -> ?PASS(ssa_opt_blockify), ?PASS(ssa_opt_merge_blocks), ?PASS(ssa_opt_get_tuple_element), + ?PASS(ssa_opt_tail_calls), ?PASS(ssa_opt_trim_unreachable), ?PASS(ssa_opt_unfold_literals)], passes_1(Ps, Opts). @@ -2185,10 +2187,55 @@ opt_ne_single_use(Var, Uses) when is_map(Uses) -> %%% extracting tuple elements on execution paths that never use the %%% extracted values. %%% +%%% However, there is one caveat to be aware of. Sinking tuple elements +%%% will keep the entire tuple alive longer. In rare circumstance, that +%%% can be a problem. Here is an example: +%%% +%%% mapfoldl(F, Acc0, [Hd|Tail]) -> +%%% {R,Acc1} = F(Hd, Acc0), +%%% {Rs,Acc2} = mapfoldl(F, Acc1, Tail), +%%% {[R|Rs],Acc2}; +%%% mapfoldl(F, Acc, []) -> +%%% {[],Acc}. +%%% +%%% Sinking get_tuple_element instructions will generate code similar +%%% to this example: +%%% +%%% slow_mapfoldl(F, Acc0, [Hd|Tail]) -> +%%% Res1 = F(Hd, Acc0), +%%% Res2 = slow_mapfoldl(F, element(2, Res1), Tail), +%%% {[element(1, Res1)|element(1, Res2)],element(2, Res2)}; +%%% slow_mapfoldl(F, Acc, []) -> +%%% {[],Acc}. +%%% +%%% Here the entire tuple bound to `Res1` will be kept alive until +%%% `slow_mapfoldl/3` returns. That is, every intermediate accumulator +%%% will be kept alive. +%%% +%%% In this case, not sinking is clearly superior: +%%% +%%% fast_mapfoldl(F, Acc0, [Hd|Tail]) -> +%%% Res1 = F(Hd, Acc0), +%%% R = element(1, Res1), +%%% Res2 = fast_mapfoldl(F, element(2, Res1), Tail), +%%% {[R|element(1, Res2)],element(2, Res2)}; +%%% fast_mapfoldl(F, Acc, []) -> +%%% {[],Acc}. +%%% +%%% The `slow_mapfoldl/3` and `fast_mapfoldl/3` use the same number of +%%% stack slots. +%%% +%%% To avoid producing code similar to `slow_mapfoldl/3`, use an +%%% heuristic to only sink when sinking would reduce the number of +%%% stack slots, or if there can't possibly be a recursive call +%%% involved. This is a heuristic because it is difficult to exactly +%%% predict the number of stack slots that will be needed for a given +%%% piece of code. ssa_opt_sink({#opt_st{ssa=Linear}=St, FuncDb}) -> %% Create a map with all variables that define get_tuple_element - %% instructions. The variable name map to the block it is defined in. + %% instructions. The variable name maps to the block it is defined + %% in and the source tuple. case def_blocks(Linear) of [] -> %% No get_tuple_element instructions, so there is nothing to do. @@ -2199,32 +2246,36 @@ ssa_opt_sink({#opt_st{ssa=Linear}=St, FuncDb}) -> end. do_ssa_opt_sink(Defs, #opt_st{ssa=Linear}=St) -> - Blocks0 = maps:from_list(Linear), - - %% Now find all the blocks that use variables defined by get_tuple_element - %% instructions. + %% Find all the blocks that use variables defined by + %% get_tuple_element instructions. Used = used_blocks(Linear, Defs, []), %% Calculate dominators. + Blocks0 = maps:from_list(Linear), {Dom,Numbering} = beam_ssa:dominators(Blocks0), %% It is not safe to move get_tuple_element instructions to blocks %% that begin with certain instructions. It is also unsafe to move - %% the instructions into any part of a receive. To avoid such - %% unsafe moves, pretend that the unsuitable blocks are not - %% dominators. + %% the instructions into any part of a receive. Unsuitable = unsuitable(Linear, Blocks0), %% Calculate new positions for get_tuple_element instructions. The new %% position is a block that dominates all uses of the variable. - DefLoc = new_def_locations(Used, Defs, Dom, Numbering, Unsuitable), + DefLocs0 = new_def_locations(Used, Defs, Dom, Numbering, Unsuitable), + + %% Avoid keeping tuples alive if only one element is accessed later and + %% if there is the chance of a recursive call being made. This is an + %% important precaution to avoid that lists:mapfoldl/3 keeps all previous + %% versions of the accumulator alive until the end of the input list. + Ps = partition_deflocs(DefLocs0, Defs, Blocks0), + DefLocs1 = filter_deflocs(Ps, Blocks0), + DefLocs = sort(DefLocs1), %% Now move all suitable get_tuple_element instructions to their %% new blocks. - Blocks = foldl(fun({V,To}, A) -> - From = map_get(V, Defs), + Blocks = foldl(fun({V,{From,To}}, A) -> move_defs(V, From, To, A) - end, Blocks0, DefLoc), + end, Blocks0, DefLocs), St#opt_st{ssa=beam_ssa:linearize(Blocks)}. @@ -2232,8 +2283,8 @@ def_blocks([{L,#b_blk{is=Is}}|Bs]) -> def_blocks_is(Is, L, def_blocks(Bs)); def_blocks([]) -> []. -def_blocks_is([#b_set{op=get_tuple_element,dst=Dst}|Is], L, Acc) -> - def_blocks_is(Is, L, [{Dst,L}|Acc]); +def_blocks_is([#b_set{op=get_tuple_element,args=[Tuple,_],dst=Dst}|Is], L, Acc) -> + def_blocks_is(Is, L, [{Dst,{L,Tuple}}|Acc]); def_blocks_is([_|Is], L, Acc) -> def_blocks_is(Is, L, Acc); def_blocks_is([], _, Acc) -> Acc. @@ -2245,6 +2296,179 @@ used_blocks([{L,Blk}|Bs], Def, Acc0) -> used_blocks([], _Def, Acc) -> rel2fam(Acc). +%% Partition sinks for get_tuple_element instructions in the same +%% clause extracting from the same tuple. Sort each partition in +%% execution order. +partition_deflocs(DefLoc, _Defs, Blocks) -> + {BlkNums0,_} = mapfoldl(fun(L, N) -> {{L,N},N+1} end, 0, beam_ssa:rpo(Blocks)), + BlkNums = maps:from_list(BlkNums0), + S = [{Tuple,{map_get(To, BlkNums),{V,{From,To}}}} || + {V,Tuple,{From,To}} <- DefLoc], + F = rel2fam(S), + partition_deflocs_1(F, Blocks). + +partition_deflocs_1([{Tuple,DefLocs0}|T], Blocks) -> + DefLocs1 = [DL || {_,DL} <- DefLocs0], + DefLocs = partition_dl(DefLocs1, Blocks), + [{Tuple,DL} || DL <- DefLocs] ++ partition_deflocs_1(T, Blocks); +partition_deflocs_1([], _) -> []. + +partition_dl([_]=DefLoc, _Blocks) -> + [DefLoc]; +partition_dl([{_,{_,First}}|_]=DefLoc0, Blocks) -> + RPO = beam_ssa:rpo([First], Blocks), + {P,DefLoc} = partition_dl_1(DefLoc0, RPO, []), + [P|partition_dl(DefLoc, Blocks)]; +partition_dl([], _Blocks) -> []. + +partition_dl_1([{_,{_,L}}=DL|DLs], [L|_]=Ls, Acc) -> + partition_dl_1(DLs, Ls, [DL|Acc]); +partition_dl_1([_|_]=DLs, [_|Ls], Acc) -> + partition_dl_1(DLs, Ls, Acc); +partition_dl_1([], _, Acc) -> + {reverse(Acc),[]}; +partition_dl_1([_|_]=DLs, [], Acc) -> + {reverse(Acc),DLs}. + +filter_deflocs([{Tuple,DefLoc0}|DLs], Blocks) -> + %% Input is a list of sinks of get_tuple_element instructions in + %% execution order from the same tuple in the same clause. + [{_,{_,First}}|_] = DefLoc0, + Paths = find_paths_to_check(DefLoc0, First), + WillGC0 = ordsets:from_list([FromTo || {{_,_}=FromTo,_} <- Paths]), + WillGC1 = [{{From,To},will_gc(From, To, Blocks, true)} || + {From,To} <- WillGC0], + WillGC = maps:from_list(WillGC1), + + %% Separate sinks that will force the reference to the tuple to be + %% saved on the stack from sinks that don't force. + {DefLocGC0,DefLocNoGC} = + partition(fun({{_,_}=FromTo,_}) -> + map_get(FromTo, WillGC) + end, Paths), + + %% Avoid potentially harmful sinks. + DefLocGC = filter_gc_deflocs(DefLocGC0, Tuple, First, Blocks), + + %% Construct the complete list of sink operations. + DefLoc1 = DefLocGC ++ DefLocNoGC, + [DL || {_,{_,{From,To}}=DL} <- DefLoc1, From =/= To] ++ + filter_deflocs(DLs, Blocks); +filter_deflocs([], _) -> []. + +%% Use an heuristic to avoid harmful sinking in lists:mapfold/3 and +%% similar functions. +filter_gc_deflocs(DefLocGC, Tuple, First, Blocks) -> + case DefLocGC of + [] -> + []; + [{_,{_,{From,To}}}] -> + %% There is only one get_tuple_element instruction that + %% can be sunk. That means that we may not gain any slots + %% by sinking it. + case is_on_stack(First, Tuple, Blocks) of + true -> + %% The tuple itself must be stored in a stack slot + %% (because it will be used later) in addition to + %% the tuple element being extracted. It is + %% probably a win to sink this instruction. + DefLocGC; + false -> + case will_gc(From, To, Blocks, false) of + false -> + %% There is no risk for recursive calls, + %% so it should be safe to + %% sink. Theoretically, we shouldn't win + %% any stack slots, but in practice it + %% seems that sinking makes it more likely + %% that the stack slot for a dying value + %% can be immediately reused for another + %% value. + DefLocGC; + true -> + %% This function could be involved in a + %% recursive call. Since there is no + %% obvious reduction in the number of + %% stack slots, we will not sink. + [] + end + end; + [_,_|_] -> + %% More than one get_tuple_element instruction can be + %% sunk. Sinking will almost certainly reduce the number + %% of stack slots. + DefLocGC + end. + +find_paths_to_check([{_,{_,To}}=Move|T], First) -> + [{{First,To},Move}|find_paths_to_check(T, First)]; +find_paths_to_check([], _First) -> []. + +will_gc(From, To, Blocks, All) -> + will_gc(beam_ssa:rpo([From], Blocks), To, Blocks, All, #{From => false}). + +will_gc([To|_], To, _Blocks, _All, WillGC) -> + map_get(To, WillGC); +will_gc([L|Ls], To, Blocks, All, WillGC0) -> + #b_blk{is=Is} = Blk = map_get(L, Blocks), + GC = map_get(L, WillGC0) orelse will_gc_is(Is, All), + WillGC = gc_update_successors(Blk, GC, WillGC0), + will_gc(Ls, To, Blocks, All, WillGC). + +will_gc_is([#b_set{op=call,args=Args}|Is], false) -> + case Args of + [#b_remote{mod=#b_literal{val=erlang}}|_] -> + %% Assume that remote calls to the erlang module can't cause a recursive + %% call. + will_gc_is(Is, false); + [_|_] -> + true + end; +will_gc_is([_|Is], false) -> + %% Instructions that clobber X registers may cause a GC, but will not cause + %% a recursive call. + will_gc_is(Is, false); +will_gc_is([I|Is], All) -> + beam_ssa:clobbers_xregs(I) orelse will_gc_is(Is, All); +will_gc_is([], _All) -> false. + +is_on_stack(From, Var, Blocks) -> + is_on_stack(beam_ssa:rpo([From], Blocks), Var, Blocks, #{From => false}). + +is_on_stack([L|Ls], Var, Blocks, WillGC0) -> + #b_blk{is=Is} = Blk = map_get(L, Blocks), + GC0 = map_get(L, WillGC0), + try is_on_stack_is(Is, Var, GC0) of + GC -> + WillGC = gc_update_successors(Blk, GC, WillGC0), + is_on_stack(Ls, Var, Blocks, WillGC) + catch + throw:{done,GC} -> + GC + end; +is_on_stack([], _Var, _, _) -> false. + +is_on_stack_is([#b_set{op=get_tuple_element}|Is], Var, GC) -> + is_on_stack_is(Is, Var, GC); +is_on_stack_is([I|Is], Var, GC0) -> + case GC0 andalso member(Var, beam_ssa:used(I)) of + true -> + throw({done,GC0}); + false -> + GC = GC0 orelse beam_ssa:clobbers_xregs(I), + is_on_stack_is(Is, Var, GC) + end; +is_on_stack_is([], _, GC) -> GC. + +gc_update_successors(Blk, GC, WillGC) -> + foldl(fun(L, Acc) -> + case Acc of + #{L := true} -> Acc; + #{L := false} when GC =:= false -> Acc; + #{} -> Acc#{L => GC} + end + end, WillGC, beam_ssa:successors(Blk)). + %% unsuitable(Linear, Blocks) -> Unsuitable. %% Return an ordset of block labels for the blocks that are not %% suitable for sinking of get_tuple_element instructions. @@ -2327,19 +2551,22 @@ is_loop_header(L, Blocks) -> %% of the variable and as near to the uses of as possible. new_def_locations([{V,UsedIn}|Vs], Defs, Dom, Numbering, Unsuitable) -> - DefIn = map_get(V, Defs), + {DefIn,Tuple} = map_get(V, Defs), Common = common_dominator(UsedIn, Dom, Numbering, Unsuitable), - case member(Common, map_get(DefIn, Dom)) of - true -> - %% The common dominator is either DefIn or an - %% ancestor of DefIn. - new_def_locations(Vs, Defs, Dom, Numbering, Unsuitable); - false -> - %% We have found a suitable descendant of DefIn, - %% to which the get_tuple_element instruction can - %% be sunk. - [{V,Common}|new_def_locations(Vs, Defs, Dom, Numbering, Unsuitable)] - end; + Sink = case member(Common, map_get(DefIn, Dom)) of + true -> + %% The common dominator is either DefIn or an + %% ancestor of DefIn. We'll need to consider all + %% get_tuple_element instructions so we will add + %% a dummy sink. + {V,Tuple,{DefIn,DefIn}}; + false -> + %% We have found a suitable descendant of DefIn, + %% to which the get_tuple_element instruction can + %% be sunk. + {V,Tuple,{DefIn,Common}} + end, + [Sink|new_def_locations(Vs, Defs, Dom, Numbering, Unsuitable)]; new_def_locations([], _, _, _, _) -> []. common_dominator(Ls0, Dom, Numbering, Unsuitable) -> @@ -2362,7 +2589,6 @@ move_defs(V, From, To, Blocks) -> {Def,FromBlk} = remove_def(V, FromBlk0), try insert_def(V, Def, ToBlk0) of ToBlk -> - %%io:format("~p: ~p => ~p\n", [V,From,To]), Blocks#{From:=FromBlk,To:=ToBlk} catch throw:not_possible -> @@ -2683,6 +2909,170 @@ unfold_arg(#b_literal{val=Val}=Lit, LitMap, X) -> unfold_arg(Expr, _LitMap, _X) -> Expr. %%% +%%% Optimize tail calls created as the result of optimizations. +%%% +%%% Consider the following example of a tail call in Erlang code: +%%% +%%% bar() -> +%%% foo(). +%%% +%%% The SSA code for the call will look like this: +%%% +%%% @ssa_ret = call (`foo`/0) +%%% ret @ssa_ret +%%% +%%% Sometimes optimizations create new tail calls. Consider this +%%% slight variation of the example: +%%% +%%% bar() -> +%%% {_,_} = foo(). +%%% +%%% foo() -> {a,b}. +%%% +%%% If beam_ssa_type can figure out that `foo/0` always returns a tuple +%%% of size two, the test for a tuple is no longer needed and the call +%%% to `foo/0` will become a tail call. However, the resulting SSA +%%% code will look like this: +%%% +%%% @ssa_ret = call (`foo`/0) +%%% @ssa_bool = succeeded:body @ssa_ret +%%% br @ssa_bool, ^999, ^1 +%%% +%%% 999: +%%% ret @ssa_ret +%%% +%%% The beam_ssa_codegen pass will not recognize this code as a tail +%%% call and will generate an unncessary stack frame. It may also +%%% generate unecessary `kill` instructions. +%%% +%%% To avoid those extra instructions, this optimization will +%%% eliminate the `succeeded:body` and `br` instructions and insert +%%% the `ret` in the same block as the call: +%%% +%%% @ssa_ret = call (`foo`/0) +%%% ret @ssa_ret +%%% +%%% Finally, consider this example: +%%% +%%% bar() -> +%%% foo_ok(), +%%% ok. +%%% +%%% foo_ok() -> ok. +%%% +%%% The SSA code for the call to `foo_ok/0` will look like: +%%% +%%% %% Result type: `ok` +%%% @ssa_ignored = call (`foo_ok`/0) +%%% @ssa_bool = succeeded:body @ssa_ignored +%%% br @ssa_bool, ^999, ^1 +%%% +%%% 999: +%%% ret `ok` +%%% +%%% Since the call to `foo_ok/0` has an annotation indicating that the +%%% call will always return the atom `ok`, the code can be simplified +%%% like this: +%%% +%%% @ssa_ignored = call (`foo_ok`/0) +%%% ret @ssa_ignored +%%% +%%% The beam_jump pass does the same optimization, but it does it too +%%% late to avoid creating an uncessary stack frame or unnecessary +%%% `kill` instructions. +%%% + +ssa_opt_tail_calls({St,FuncDb}) -> + #opt_st{ssa=Blocks0} = St, + Blocks = opt_tail_calls(beam_ssa:rpo(Blocks0), Blocks0), + {St#opt_st{ssa=Blocks},FuncDb}. + +opt_tail_calls([L|Ls], Blocks0) -> + #b_blk{is=Is0,last=Last} = Blk0 = map_get(L, Blocks0), + + %% Does this block end with a two-way branch whose success + %% label targets an empty block with a `ret` terminator? + case is_potential_tail_call(Last, Blocks0) of + {yes,Bool,Ret} -> + %% Yes, `Ret` is the value returned from that block + %% (either a variable or literal). Do the instructions + %% in this block end with a `call` instruction that + %% returns the same value as `Ret`, followed by a + %% `succeeded:body` instruction? + case is_tail_call_is(Is0, Bool, Ret, []) of + {yes,Is,Var} -> + %% Yes, this is a tail call. `Is` is the instructions + %% in the block with `succeeded:body` removed, and + %% `Var` is the destination variable for the return + %% value of the call. Rewrite this block to directly + %% return `Var`. + Blk = Blk0#b_blk{is=Is,last=#b_ret{arg=Var}}, + Blocks = Blocks0#{L:=Blk}, + opt_tail_calls(Ls, Blocks); + no -> + %% No, the block does not end with a call, or the + %% the call instruction has not the same value + %% as `Ret`. + opt_tail_calls(Ls, Blocks0) + end; + no -> + opt_tail_calls(Ls, Blocks0) + end; +opt_tail_calls([], Blocks) -> Blocks. + +is_potential_tail_call(#b_br{bool=#b_var{}=Bool,succ=Succ}, Blocks) -> + case Blocks of + #{Succ := #b_blk{is=[],last=#b_ret{arg=Arg}}} -> + %% This could be a tail call. + {yes,Bool,Arg}; + #{} -> + %% The block is not empty or does not have a `ret` terminator. + no + end; +is_potential_tail_call(_, _) -> + %% Not a two-way branch (a `succeeded:body` instruction must be + %% followed by a two-way branch). + no. + +is_tail_call_is([#b_set{op=call,dst=Dst}=Call, + #b_set{op={succeeded,body},dst=Bool}], + Bool, Ret, Acc) -> + IsTailCall = + case Ret of + #b_literal{val=Val} -> + %% The return value for this function is a literal. + %% Now test whether it is the same literal that the + %% `call` instruction returns. + Type = beam_ssa:get_anno(result_type, Call, any), + case beam_types:get_singleton_value(Type) of + {ok,Val} -> + %% Same value. + true; + {ok,_} -> + %% Wrong value. + false; + error -> + %% The type for the return value is not a singleton value. + false + end; + #b_var{} -> + %% It is a tail call if the variable set by the `call` instruction + %% is the same variable as the argument for the `ret` terminator. + Ret =:= Dst + end, + case IsTailCall of + true -> + %% Return the instructions in the block with `succeeded:body` removed. + Is = reverse(Acc, [Call]), + {yes,Is,Dst}; + false -> + no + end; +is_tail_call_is([I|Is], Bool, Ret, Acc) -> + is_tail_call_is(Is, Bool, Ret, [I|Acc]); +is_tail_call_is([], _Bool, _Ret, _Acc) -> no. + +%%% %%% Common utilities. %%% diff --git a/lib/compiler/src/beam_trim.erl b/lib/compiler/src/beam_trim.erl index 09302b6a79..017d2b7d8c 100644 --- a/lib/compiler/src/beam_trim.erl +++ b/lib/compiler/src/beam_trim.erl @@ -63,7 +63,8 @@ trim([{kill,_}|_]=Is0, St, Acc) -> %% Calculate all recipes that are not worse in terms %% of estimated execution time. The recipes are ordered %% in descending order from how much they trim. - Recipes = trim_recipes(Layout), + IsNotRecursive = is_not_recursive(Is1), + Recipes = trim_recipes(Layout, IsNotRecursive), %% Try the recipes in order. A recipe may not work out because %% a register that was previously killed may be @@ -85,6 +86,21 @@ trim([I|Is], St, Acc) -> trim([], _, Acc) -> reverse(Acc). +%% is_not_recursive([Instruction]) -> true|false. +%% Test whether the next call or apply instruction may +%% do a recursive call. Return `true` if the call is +%% definitely not recursive, and `false` otherwise. +is_not_recursive([{call_ext,_,Ext}|_]) -> + case Ext of + {extfunc,M,F,A} -> + erl_bifs:is_pure(M, F, A); + _ -> + false + end; +is_not_recursive([{block,_}|Is]) -> is_not_recursive(Is); +is_not_recursive([{line,_}|Is]) -> is_not_recursive(Is); +is_not_recursive(_) -> false. + %% trim_recipes([{kill,R}|{live,R}|{dead,R}]) -> [Recipe]. %% Recipe = {Kills,NumberToTrim,Moves} %% Kills = [{kill,Y}] @@ -93,34 +109,34 @@ trim([], _, Acc) -> %% Calculate how to best trim the stack and kill the correct %% Y registers. Return a list of possible recipes. The best %% recipe (the one that trims the most) is first in the list. -%% All of the recipes are no worse in estimated execution time -%% than the original sequences of kill instructions. -trim_recipes(Layout) -> - Cost = length([I || {kill,_}=I <- Layout]), - trim_recipes_1(Layout, 0, [], {Cost,[]}). +trim_recipes(Layout, IsNotRecursive) -> + Recipes = construct_recipes(Layout, 0, [], []), + NumOrigKills = length([I || {kill,_}=I <- Layout]), + IsTooExpensive = is_too_expensive_fun(IsNotRecursive), + [R || R <- Recipes, + not is_too_expensive(R, NumOrigKills, IsTooExpensive)]. -trim_recipes_1([{kill,{y,Trim0}}|Ks], Trim0, Moves, Recipes0) -> +construct_recipes([{kill,{y,Trim0}}|Ks], Trim0, Moves, Acc) -> Trim = Trim0 + 1, - Recipes = save_recipe(Ks, Trim, Moves, Recipes0), - trim_recipes_1(Ks, Trim, Moves, Recipes); -trim_recipes_1([{dead,{y,Trim0}}|Ks], Trim0, Moves, Recipes0) -> + Recipe = {Ks,Trim,Moves}, + construct_recipes(Ks, Trim, Moves, [Recipe|Acc]); +construct_recipes([{dead,{y,Trim0}}|Ks], Trim0, Moves, Acc) -> Trim = Trim0 + 1, - Recipes = save_recipe(Ks, Trim, Moves, Recipes0), - trim_recipes_1(Ks, Trim, Moves, Recipes); -trim_recipes_1([{live,{y,Trim0}=Src}|Ks0], Trim0, Moves0, Recipes0) -> + Recipe = {Ks,Trim,Moves}, + construct_recipes(Ks, Trim, Moves, [Recipe|Acc]); +construct_recipes([{live,{y,Trim0}=Src}|Ks0], Trim0, Moves0, Acc) -> case take_last_dead(Ks0) of none -> - {_,RecipesList} = Recipes0, - RecipesList; + %% No more recipes are possible. + Acc; {Dst,Ks} -> Trim = Trim0 + 1, Moves = [{move,Src,Dst}|Moves0], - Recipes = save_recipe(Ks, Trim, Moves, Recipes0), - trim_recipes_1(Ks, Trim, Moves, Recipes) + Recipe = {Ks,Trim,Moves}, + construct_recipes(Ks, Trim, Moves, [Recipe|Acc]) end; -trim_recipes_1([], _, _, {_,RecipesList}) -> - RecipesList. +construct_recipes([], _, _, Acc) -> Acc. take_last_dead(L) -> take_last_dead_1(reverse(L)). @@ -131,33 +147,47 @@ take_last_dead_1([{dead,Reg}|Is]) -> {Reg,reverse(Is)}; take_last_dead_1(_) -> none. -save_recipe(Ks, Trim, Moves, {MaxCost,Acc}=Recipes) -> - case recipe_cost(Ks, Moves) of - Cost when Cost =< MaxCost -> - %% The price is right. - {MaxCost,[{Ks,Trim,Moves}|Acc]}; - _Cost -> - %% Too expensive. - Recipes +%% Is trimming too expensive? +is_too_expensive({Ks,_,Moves}, NumOrigKills, IsTooExpensive) -> + NumKills = num_kills(Ks, 0), + NumMoves = length(Moves), + IsTooExpensive(NumKills, NumMoves, NumOrigKills). + +num_kills([{kill,_}|T], Acc) -> + num_kills(T, Acc+1); +num_kills([_|T], Acc) -> + num_kills(T, Acc); +num_kills([], Acc) -> Acc. + +is_too_expensive_fun(true) -> + %% This call is not recursive (because it is a call to a BIF). + %% Here we should avoid trimming if the trimming sequence is + %% likely to be more expensive than the original sequence. + fun(NumKills, NumMoves, NumOrigKills) -> + Penalty = + if + %% Slightly penalize the use of any `move` + %% instruction to avoid replacing two `kill` + %% instructions with a `move` and a `trim`. + NumMoves =/= 0 -> 1; + true -> 0 + end, + 1 + Penalty + NumKills + NumMoves > NumOrigKills + end; +is_too_expensive_fun(false) -> + %% This call **may** be recursive. In a recursive function that + %% builds up a huge stack, having unused stack slots will be very + %% expensive. Therefore, we want to be biased towards trimming. + %% We will do that by not counting the `trim` instruction in + %% the formula below. + fun(NumKills, NumMoves, NumOrigKills) -> + NumKills + NumMoves > NumOrigKills end. -recipe_cost(Ks, Moves) -> - %% We estimate that a {move,{y,_},{y,_}} instruction is roughly twice as - %% expensive as a {kill,{y,_}} instruction. A {trim,_} instruction is - %% roughly as expensive as a {kill,{y,_}} instruction. - - recipe_cost_1(Ks, 1+2*length(Moves)). - -recipe_cost_1([{kill,_}|Ks], Cost) -> - recipe_cost_1(Ks, Cost+1); -recipe_cost_1([_|Ks], Cost) -> - recipe_cost_1(Ks, Cost); -recipe_cost_1([], Cost) -> Cost. - %% try_remap([Recipe], [Instruction], FrameSize) -> %% {[Instruction],[TrimInstruction]}. %% Try to renumber Y registers in the instruction stream. The -%% first rececipe that works will be used. +%% first recipe that works will be used. %% %% This function will issue a `not_possible` exception if none %% of the recipes were possible to apply. diff --git a/lib/compiler/src/compiler.app.src b/lib/compiler/src/compiler.app.src index 234f0b7780..92e9fa74e5 100644 --- a/lib/compiler/src/compiler.app.src +++ b/lib/compiler/src/compiler.app.src @@ -80,5 +80,5 @@ {registered, []}, {applications, [kernel, stdlib]}, {env, []}, - {runtime_dependencies, ["stdlib-@OTP-15251@","kernel-@OTP-15251@","hipe-3.12","erts-@OTP-15251@", + {runtime_dependencies, ["stdlib-3.13","kernel-7.0","hipe-3.12","erts-11.0", "crypto-3.6"]}]}. diff --git a/lib/compiler/test/beam_ssa_SUITE.erl b/lib/compiler/test/beam_ssa_SUITE.erl index 03cda0c798..28cc0ae268 100644 --- a/lib/compiler/test/beam_ssa_SUITE.erl +++ b/lib/compiler/test/beam_ssa_SUITE.erl @@ -23,13 +23,15 @@ init_per_group/2,end_per_group/2, calls/1,tuple_matching/1,recv/1,maps/1, cover_ssa_dead/1,combine_sw/1,share_opt/1, - beam_ssa_dead_crash/1,stack_init/1,grab_bag/1, - coverage/1]). + beam_ssa_dead_crash/1,stack_init/1, + mapfoldl/0,mapfoldl/1, + grab_bag/1,coverage/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> - [{group,p}]. + [mapfoldl, + {group,p}]. groups() -> [{p,test_lib:parallel(), @@ -702,6 +704,74 @@ stack_init(Key, Map) -> %% (if the second clause was executed). id(Res). +%% Test that compiler "optimizations" don't rewrite mapfold/3 to the +%% equivalent of slow_mapfoldl/3. +mapfoldl() -> + {N,Size} = mapfoldl_limits(), + {Time,_} = timer:tc(fun() -> + mapfoldl(fun(Sz, _) -> + erlang:garbage_collect(), + {Sz,erlang:make_tuple(Sz, a)} + end, [], [Size]) + end), + Seconds = 15 + ceil(10 * Time * N / 1_000_000), + io:format("~p seconds timetrap\n", [Seconds]), + [{timetrap,{seconds,Seconds}}]. + +mapfoldl(_Config) -> + test_mapfoldl_implementations(), + F = fun(Sz, _) -> + erlang:garbage_collect(), + {Sz,erlang:make_tuple(Sz, a)} + end, + {N,Size} = mapfoldl_limits(), + List = lists:duplicate(N, Size), + {List,Tuple} = mapfoldl(F, [], List), + {List,Tuple} = fast_mapfoldl(F, [], List), + Size = tuple_size(Tuple), + ok. + +mapfoldl_limits() -> + {1_000,100_000}. + +test_mapfoldl_implementations() -> + Seq = lists:seq(1, 10), + F = fun(N, Sum) -> {N,Sum+N} end, + {Seq,55} = mapfoldl(F, 0, Seq), + {Seq,55} = fast_mapfoldl(F, 0, Seq), + {Seq,55} = slow_mapfoldl(F, 0, Seq), + ok. + +mapfoldl(F, Acc0, [Hd|Tail]) -> + {R,Acc1} = F(Hd, Acc0), + {Rs,Acc2} = mapfoldl(F, Acc1, Tail), + {[R|Rs],Acc2}; +mapfoldl(F, Acc, []) when is_function(F, 2) -> {[],Acc}. + +%% Here is an illustration of how the compiler used to sink +%% get_tuple_element instructions in a way that would cause all +%% versions of the accumulator to be kept until the end. The compiler +%% now uses a heuristic to only sink get_tuple_element instructions if +%% that would cause fewer values to be saved in the stack frame. +slow_mapfoldl(F, Acc0, [Hd|Tail]) -> + Res1 = F(Hd, Acc0), + %% By saving the Res1 tuple, all intermediate accumulators will be + %% kept to the end. + Res2 = slow_mapfoldl(F, element(2, Res1), Tail), + {[element(1, Res1)|element(1, Res2)],element(2, Res2)}; +slow_mapfoldl(F, Acc, []) when is_function(F, 2) -> {[],Acc}. + +%% Here is an illustration how the compiler should compile mapfoldl/3 +%% to avoid keeping all intermediate accumulators. Note that +%% slow_mapfoldl/3 and fast_mapfoldl/3 use the same amount of stack +%% space. +fast_mapfoldl(F, Acc0, [Hd|Tail]) -> + Res1 = F(Hd, Acc0), + R = element(1, Res1), + Res2 = fast_mapfoldl(F, element(2, Res1), Tail), + {[R|element(1, Res2)],element(2, Res2)}; +fast_mapfoldl(F, Acc, []) when is_function(F, 2) -> {[],Acc}. + grab_bag(_Config) -> {'EXIT',_} = (catch grab_bag_1()), {'EXIT',_} = (catch grab_bag_2()), diff --git a/lib/compiler/test/guard_SUITE.erl b/lib/compiler/test/guard_SUITE.erl index 4a53698e15..0558b8f300 100644 --- a/lib/compiler/test/guard_SUITE.erl +++ b/lib/compiler/test/guard_SUITE.erl @@ -2300,6 +2300,8 @@ beam_bool_SUITE(_Config) -> in_catch(), recv_semi(), andalso_repeated_var(), + erl1246(), + erl1253(), ok. before_and_inside_if() -> @@ -2597,6 +2599,170 @@ andalso_repeated_var() -> andalso_repeated_var(B) when B andalso B -> ok; andalso_repeated_var(_) -> error. +-record(erl1246, {tran_stat = 0}). + +erl1246() -> + false = erl1246(#erl1246{tran_stat = 0}, #{cid => 1131}), + false = erl1246(#erl1246{tran_stat = 12}, #{cid => 1131}), + false = erl1246(#erl1246{tran_stat = 12}, #{cid => 9502}), + true = erl1246(#erl1246{tran_stat = 0}, #{cid => 9502}), + ok. + +erl1246(Rec, #{cid := CollID}) -> + {GiftCollID, _} = erl1246_conf(gift_coll), + IsTranStat = Rec#erl1246.tran_stat =:= erl1246_conf(transform_id), + if + %% Optimization of 'not' in a guard was broken. + CollID =:= GiftCollID andalso not IsTranStat -> + true; + true -> + false + end. + +erl1246_conf(gift_coll) -> {9502, {112, 45}}; +erl1246_conf(transform_id) -> 12; +erl1246_conf(_) -> undefined. + +erl1253() -> + ok = erl1253_orelse_false(a, a, any), + ok = erl1253_orelse_false(a, a, true), + ok = erl1253_orelse_false(a, a, false), + error = erl1253_orelse_false(a, b, any), + error = erl1253_orelse_false(a, b, true), + ok = erl1253_orelse_false(a, b, false), + + ok = erl1253_orelse_true(a, a, any), + ok = erl1253_orelse_true(a, a, true), + ok = erl1253_orelse_true(a, a, false), + error = erl1253_orelse_true(a, b, any), + ok = erl1253_orelse_true(a, b, true), + error = erl1253_orelse_true(a, b, false), + + error = erl1253_andalso_false(a, a, any), + error = erl1253_andalso_false(a, a, true), + ok = erl1253_andalso_false(a, a, false), + error = erl1253_andalso_false(a, b, any), + error = erl1253_andalso_false(a, b, true), + error = erl1253_andalso_false(a, b, false), + + error = erl1253_andalso_true(a, a, any), + ok = erl1253_andalso_true(a, a, true), + error = erl1253_andalso_true(a, a, false), + error = erl1253_andalso_true(a, b, any), + error = erl1253_andalso_true(a, b, true), + error = erl1253_andalso_true(a, b, false), + + ok. + +erl1253_orelse_false(X, Y, Z) -> + Res = erl1253_orelse_false_1(X, Y, Z), + Res = erl1253_orelse_false_2(X, Y, Z), + Res = erl1253_orelse_false_3(X, Y, Z). + +erl1253_orelse_false_1(X, Y, Z) -> + Bool = Z =:= false, + if + X =:= Y orelse Bool -> ok; + true -> error + end. + +erl1253_orelse_false_2(X, Y, Z) -> + Bool = Z =:= false, + if + Bool orelse X =:= Y -> ok; + true -> error + end. + +erl1253_orelse_false_3(X, Y, Z) -> + Bool1 = X =:= Y, + Bool2 = Z =:= false, + if + Bool1 orelse Bool2 -> ok; + true -> error + end. + +erl1253_orelse_true(X, Y, Z) -> + Res = erl1253_orelse_true_1(X, Y, Z), + Res = erl1253_orelse_true_2(X, Y, Z), + Res = erl1253_orelse_true_3(X, Y, Z). + +erl1253_orelse_true_1(X, Y, Z) -> + Bool = Z =:= true, + if + X =:= Y orelse Bool -> ok; + true -> error + end. + +erl1253_orelse_true_2(X, Y, Z) -> + Bool = Z =:= true, + if + Bool orelse X =:= Y -> ok; + true -> error + end. + +erl1253_orelse_true_3(X, Y, Z) -> + Bool1 = X =:= Y, + Bool2 = Z =:= true, + if + Bool1 orelse Bool2 -> ok; + true -> error + end. + +erl1253_andalso_false(X, Y, Z) -> + Res = erl1253_andalso_false_1(X, Y, Z), + Res = erl1253_andalso_false_2(X, Y, Z), + Res = erl1253_andalso_false_3(X, Y, Z). + +erl1253_andalso_false_1(X, Y, Z) -> + Bool = Z =:= false, + if + X =:= Y andalso Bool -> ok; + true -> error + end. + +erl1253_andalso_false_2(X, Y, Z) -> + Bool1 = X =:= Y, + Bool2 = Z =:= false, + if + Bool1 andalso Bool2 -> ok; + true -> error + end. + +erl1253_andalso_false_3(X, Y, Z) -> + Bool1 = X =:= Y, + Bool2 = Z =:= false, + if + Bool1 andalso Bool2 -> ok; + true -> error + end. + +erl1253_andalso_true(X, Y, Z) -> + Res = erl1253_andalso_true_1(X, Y, Z), + Res = erl1253_andalso_true_2(X, Y, Z), + Res = erl1253_andalso_true_3(X, Y, Z). + +erl1253_andalso_true_1(X, Y, Z) -> + Bool = Z =:= true, + if + X =:= Y andalso Bool -> ok; + true -> error + end. + +erl1253_andalso_true_2(X, Y, Z) -> + Bool = Z =:= true, + if + Bool andalso X =:= Y-> ok; + true -> error + end. + +erl1253_andalso_true_3(X, Y, Z) -> + Bool1 = X =:= Y, + Bool2 = Z =:= true, + if + Bool1 andalso Bool2 -> ok; + true -> error + end. + %%% %%% End of beam_bool_SUITE tests. %%% diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index 87d821d7e8..e167c63a7d 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 7.5.4 +COMPILER_VSN = 7.6.1 diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index b22b46d5e5..5903226f6e 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -31,6 +31,93 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 4.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Crypto reported unsupported elliptic curves as supported + on e.g Fedora distros.</p> + <p> + Own Id: OTP-16579 Aux Id: ERL-825 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Support for ed25519 and ed448 added to + <c>crypto:generate_key</c>.</p> + <p> + Own Id: OTP-15967 Aux Id: PR-2329 </p> + </item> + <item> + <p> + The <seeguide marker="crypto:new_api#the-new-api">new + crypto functions api</seeguide> (crypto_init, + crypto_update and crypto_one_time) has been updated.</p> + <p> + There is now a function <seemfa + marker="crypto:crypto#crypto_final/1"><c>crypto_final/1</c></seemfa> + and a possibility to set options in <seemfa + marker="crypto:crypto#crypto_init/3"><c>crypto_init/3</c></seemfa> + and <seemfa + marker="crypto:crypto#crypto_init/4"><c>crypto_init/4</c></seemfa>. + See the manual for details.</p> + <p> + Own Id: OTP-16160</p> + </item> + <item> + <p> + As <seeguide + marker="crypto:notes#crypto-4.5">announced</seeguide> in + OTP 22.0, a New API was introduced in CRYPTO. See the + <seeguide marker="crypto:new_api"><i>New and Old + API</i></seeguide> chapter in the CRYPTO User's Guide for + more information and suggested replacement functions.</p> + <p> + <seeguide marker="crypto:new_api#the-old-api">The Old + API</seeguide> is now deprecated in OTP-23.0 and will be + removed in OTP-24.0.</p> + <p> + This deprecation includes cipher names. See the section + <seeguide + marker="crypto:new_api#retired-cipher-names">Retired + cipher names</seeguide> in the crypto User's Guide, + chapter <seeguide marker="crypto:new_api#the-old-api">The + Old API</seeguide>.</p> + <p> + Own Id: OTP-16232</p> + </item> + <item> + <p> + Fix C-compilation without deprecated OpenSSL cryptolib + APIs</p> + <p> + Own Id: OTP-16369 Aux Id: PR-2474 </p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p> + Added missing 'eddh' to <seemfa + marker="crypto:crypto#supports/1">crypto:supports(public_keys)</seemfa>.</p> + <p> + Own Id: OTP-16583</p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 4.6.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/crypto/test/crypto_SUITE.erl b/lib/crypto/test/crypto_SUITE.erl index ca72601bef..df830b32f6 100644 --- a/lib/crypto/test/crypto_SUITE.erl +++ b/lib/crypto/test/crypto_SUITE.erl @@ -135,6 +135,7 @@ groups() -> {group, dh}, {group, ecdh}, + {group, eddh}, {group, srp}, {group, chacha20_poly1305}, @@ -217,7 +218,7 @@ groups() -> {dss, [], [sign_verify %% Does not work yet: ,public_encrypt, private_encrypt ]}, - {ecdsa, [], [sign_verify + {ecdsa, [], [sign_verify, use_all_ec_sign_verify %% Does not work yet: ,public_encrypt, private_encrypt ]}, {ed25519, [], [sign_verify, @@ -229,7 +230,8 @@ groups() -> generate ]}, {dh, [], [generate_compute, compute_bug]}, - {ecdh, [], [use_all_elliptic_curves, compute, generate]}, + {ecdh, [], [compute, generate, use_all_ecdh_generate_compute]}, + {eddh, [], [compute, generate, use_all_eddh_generate_compute]}, {srp, [], [generate_compute]}, {des_cbc, [], [block, api_ng, api_ng_one_shot, api_ng_tls]}, {des_cfb, [], [block, api_ng, api_ng_one_shot, api_ng_tls]}, @@ -330,7 +332,7 @@ init_per_suite(Config) -> {ok, _} = zip:unzip("cmactestvectors.zip"), {ok, _} = zip:unzip("gcmtestvectors.zip"), - try crypto:start() of + try is_ok(crypto:start()) of ok -> catch ct:comment("~s",[element(3,hd(crypto:info_lib()))]), catch ct:log("crypto:info_lib() -> ~p~n" @@ -353,10 +355,18 @@ init_per_suite(Config) -> crypto:rand_seed(<< <<Bin/binary>> || _ <- lists:seq(1,16) >>), Config end - catch _:_ -> + + catch C:E:S -> + ct:log("~p ~p~n~p", [C,E,S]), {fail, "Crypto did not start"} end. +is_ok(ok) -> ok; +is_ok({error, already_started}) -> ok; +is_ok({error,{already_started,crypto}}) -> ok. + + + end_per_suite(_Config) -> application:stop(crypto). @@ -471,8 +481,7 @@ hmac() -> [{doc, "Test hmac function"}]. hmac(Config) when is_list(Config) -> Tuples = lazy_eval(proplists:get_value(hmac, Config)), - lists:foreach(fun hmac_check/1, Tuples), - lists:foreach(fun hmac_check/1, mac_listify(Tuples)). + do_cipher_tests(fun hmac_check/1, Tuples++mac_listify(Tuples)). %%-------------------------------------------------------------------- no_hmac() -> @@ -500,8 +509,7 @@ cmac() -> [{doc, "Test all different cmac functions"}]. cmac(Config) when is_list(Config) -> Pairs = lazy_eval(proplists:get_value(cmac, Config)), - lists:foreach(fun cmac_check/1, Pairs), - lists:foreach(fun cmac_check/1, mac_listify(Pairs)). + do_cipher_tests(fun cmac_check/1, Pairs ++ mac_listify(Pairs)). %%-------------------------------------------------------------------- poly1305() -> @@ -531,8 +539,7 @@ block() -> [{doc, "Test block ciphers"}]. block(Config) when is_list(Config) -> [_|_] = Blocks = lazy_eval(proplists:get_value(cipher, Config)), - lists:foreach(fun block_cipher/1, Blocks), - lists:foreach(fun block_cipher/1, block_iolistify(Blocks)), + do_cipher_tests(fun block_cipher/1, Blocks++block_iolistify(Blocks)), lists:foreach(fun block_cipher_increment/1, block_iolistify(Blocks)). %%-------------------------------------------------------------------- @@ -604,7 +611,7 @@ api_ng_cipher_increment_loop(Ref, InTexts) -> Bin catch error:Error -> - ct:pal("Txt = ~p",[Txt]), + ct:log("Txt = ~p",[Txt]), ct:fail("~p",[Error]) end end, InTexts). @@ -676,17 +683,16 @@ api_ng_tls(Config) when is_list(Config) -> lists:foreach(fun do_api_ng_tls/1, Ciphers). -do_api_ng_tls({Type, Key, PlainTexts}=_X) -> - ct:log("~p",[_X]), +do_api_ng_tls({Type, Key, PlainTexts}) -> do_api_ng_tls({Type, Key, <<>>, PlainTexts}); -do_api_ng_tls({Type, Key, IV, PlainTexts}=_X) -> - ct:log("~p",[_X]), +do_api_ng_tls({Type, Key, IV, PlainTexts}) -> do_api_ng_tls({Type, Key, IV, PlainTexts, undefined}); -do_api_ng_tls({Type, Key, IV, PlainText0, ExpectedEncText}=_X) -> - ct:log("~p",[_X]), +do_api_ng_tls({Type, Key, IV, PlainText0, ExpectedEncText}) -> PlainText = iolist_to_binary(lazy_eval(PlainText0)), + ct:log("Type = ~p~nKey = ~p~nIV = ~p~nPlainText = ~p~nExpectedEncText = ~p", + [Type, Key, IV, PlainText, ExpectedEncText]), Renc = crypto:crypto_dyn_iv_init(Type, Key, true), Rdec = crypto:crypto_dyn_iv_init(Type, Key, false), EncTxt = crypto:crypto_dyn_iv_update(Renc, PlainText, IV), @@ -832,8 +838,6 @@ no_stream_ivec(Config) when is_list(Config) -> notsup(fun crypto:stream_init/3, [Type, <<"Key">>, <<"Ivec">>]). %%-------------------------------------------------------------------- -aead() -> - [{doc, "Test AEAD ciphers"}]. aead(Config) when is_list(Config) -> [_|_] = AEADs = lazy_eval(proplists:get_value(cipher, Config)), FilteredAEADs = @@ -848,7 +852,7 @@ aead(Config) when is_list(Config) -> IVLen >= 12 end, AEADs) end, - lists:foreach(fun aead_cipher/1, FilteredAEADs). + do_cipher_tests(fun aead_cipher/1, FilteredAEADs). %%-------------------------------------------------------------------- aead_ng(Config) when is_list(Config) -> @@ -865,7 +869,7 @@ aead_ng(Config) when is_list(Config) -> IVLen >= 12 end, AEADs) end, - lists:foreach(fun aead_cipher_ng/1, FilteredAEADs ++ spec_0_bytes(Config)). + do_cipher_tests(fun aead_cipher_ng/1, FilteredAEADs ++ spec_0_bytes(Config)). %%-------------------------------------------------------------------- aead_bad_tag(Config) -> @@ -882,7 +886,7 @@ aead_bad_tag(Config) -> IVLen >= 12 end, AEADs) end, - lists:foreach(fun aead_cipher_bad_tag/1, FilteredAEADs). + do_cipher_tests(fun aead_cipher_bad_tag/1, FilteredAEADs). %%-------------------------------------------------------------------- sign_verify() -> @@ -902,6 +906,7 @@ no_sign_verify(Config) when is_list(Config) -> public_encrypt() -> [{doc, "Test public_encrypt/decrypt "}]. public_encrypt(Config) when is_list(Config) -> + ct:log("public_encrypt", []), Params = proplists:get_value(pub_pub_encrypt, Config, []), lists:foreach(fun do_public_encrypt/1, Params). @@ -961,9 +966,7 @@ compute(Config) when is_list(Config) -> Gen = proplists:get_value(compute, Config), lists:foreach(fun do_compute/1, Gen). %%-------------------------------------------------------------------- -use_all_elliptic_curves() -> - [{doc, " Test that all curves from crypto:ec_curves/0"}]. -use_all_elliptic_curves(_Config) -> +use_all_ec_sign_verify(_Config) -> Msg = <<"hello world!">>, Sups = crypto:supports(), Curves = proplists:get_value(curves, Sups), @@ -975,6 +978,7 @@ use_all_elliptic_curves(_Config) -> Results = [{{Curve,Hash}, try + ct:log("~p ~p",[Curve,Hash]), {Pub,Priv} = crypto:generate_key(ecdh, Curve), true = is_binary(Pub), true = is_binary(Priv), @@ -1000,6 +1004,57 @@ use_all_elliptic_curves(_Config) -> end. %%-------------------------------------------------------------------- +use_all_ecdh_generate_compute(Config) -> + Curves = crypto:supports(curves) -- [ed25519, ed448, x25519, x448], + do_dh_curves(Config, Curves). + +use_all_eddh_generate_compute(Config) -> + AllCurves = crypto:supports(curves), + Curves = [C || C <- [x25519, x448], + lists:member(C, AllCurves)], + do_dh_curves(Config, Curves). + +do_dh_curves(_Config, Curves) -> + ct:log("Lib: ~p~nFIPS: ~p~nCurves:~n~p~n", [crypto:info_lib(), + crypto:info_fips(), + Curves]), + Results = + [{Curve, + try + ct:log("~p",[Curve]), + {APub,APriv} = crypto:generate_key(ecdh, Curve), + {BPub,BPriv} = crypto:generate_key(ecdh, Curve), + true = is_binary(APub), + true = is_binary(APriv), + true = is_binary(BPub), + true = is_binary(BPriv), + + ACommonSecret = crypto:compute_key(ecdh, BPub, APriv, Curve), + BCommonSecret = crypto:compute_key(ecdh, APub, BPriv, Curve), + ACommonSecret == BCommonSecret + catch + C:E -> + {C,E} + end} + || Curve <- Curves + ], + + Fails = + lists:filter(fun({_,true}) -> false; + (_) -> true + end, Results), + + case Fails of + [] -> + ct:comment("All ~p passed",[length(Results)]), + ok; + _ -> + ct:comment("passed: ~p, failed: ~p",[length(Results),length(Fails)]), + ct:log("Fails:~n~p",[Fails]), + ct:fail("Bad curve(s)",[]) + end. + +%%-------------------------------------------------------------------- generate() -> [{doc, " Test crypto:generate_key"}]. generate(Config) when is_list(Config) -> @@ -1065,13 +1120,14 @@ cipher_info(Config) when is_list(Config) -> of _ -> Ok catch Cls:Exc -> - ct:pal("~p:~p ~p",[Cls,Exc,C]), + ct:log("~p:~p ~p",[Cls,Exc,C]), false end end, true, -crypto:supports(ciphers)) of -%% proplists:get_value(ciphers, crypto:supports())) of + crypto:supports(ciphers) + ) + of true -> ok; false -> @@ -1175,76 +1231,49 @@ hmac_increment(State0, [Increment | Rest]) -> hmac_increment(State, Rest). %%%---------------------------------------------------------------- -cmac_check({cmac, Type, Key, Text, CMac}) -> +cmac_check({cmac, Type, Key, Text, CMac}=T) -> ExpCMac = iolist_to_binary(CMac), - case crypto:cmac(Type, Key, Text) of - ExpCMac -> - ok; - Other -> - ct:fail({{crypto, cmac, [Type, Key, Text]}, {expected, ExpCMac}, {got, Other}}) - end; -cmac_check({cmac, Type, Key, Text, Size, CMac}) -> + cipher_test(T, + fun() -> crypto:cmac(Type, Key, Text) end, + ExpCMac); +cmac_check({cmac, Type, Key, Text, Size, CMac}=T) -> ExpCMac = iolist_to_binary(CMac), - case crypto:cmac(Type, Key, Text, Size) of - ExpCMac -> - ok; - Other -> - ct:fail({{crypto, cmac, [Type, Key, Text, Size]}, {expected, ExpCMac}, {got, Other}}) - end. - + cipher_test(T, + fun() -> crypto:cmac(Type, Key, Text, Size) end, + ExpCMac). -mac_check({MacType, SubType, Key, Text, Mac}) -> +mac_check({MacType, SubType, Key, Text, Mac}=T) -> ExpMac = iolist_to_binary(Mac), - case crypto:mac(MacType, SubType, Key, Text) of - ExpMac -> - ok; - Other -> - ct:fail({{crypto, mac, [MacType, SubType, Key, Text]}, {expected, ExpMac}, {got, Other}}) - end; -mac_check({MacType, SubType, Key, Text, Size, Mac}) -> + cipher_test(T, + fun() -> crypto:mac(MacType, SubType, Key, Text) end, + ExpMac); +mac_check({MacType, SubType, Key, Text, Size, Mac}=T) -> ExpMac = iolist_to_binary(Mac), - case crypto:mac(MacType, SubType, Key, Text, Size) of - ExpMac -> - ok; - Other -> - ct:fail({{crypto, mac, [MacType, SubType, Key, Text]}, {expected, ExpMac}, {got, Other}}) - end. + cipher_test(T, + fun() -> crypto:mac(MacType, SubType, Key, Text, Size) end, + ExpMac). - -block_cipher({Type, Key, PlainText}) -> +block_cipher({Type, Key, PlainText}=T) -> Plain = iolist_to_binary(PlainText), CipherText = crypto:block_encrypt(Type, Key, PlainText), - case crypto:block_decrypt(Type, Key, CipherText) of - Plain -> - ok; - Other -> - ct:fail({{crypto, block_decrypt, [Type, Key, CipherText]}, {expected, Plain}, {got, Other}}) - end; + cipher_test(T, + fun() -> crypto:block_decrypt(Type, Key, CipherText) end, + Plain); -block_cipher({Type, Key, IV, PlainText}) -> +block_cipher({Type, Key, IV, PlainText}=T) -> Plain = iolist_to_binary(PlainText), CipherText = crypto:block_encrypt(Type, Key, IV, PlainText), - case crypto:block_decrypt(Type, Key, IV, CipherText) of - Plain -> - ok; - Other -> - ct:fail({{crypto, block_decrypt, [Type, Key, IV, CipherText]}, {expected, Plain}, {got, Other}}) - end; + cipher_test(T, + fun() -> crypto:block_decrypt(Type, Key, IV, CipherText) end, + Plain); -block_cipher({Type, Key, IV, PlainText, CipherText}) -> +block_cipher({Type, Key, IV, PlainText, CipherText}=T) -> Plain = iolist_to_binary(PlainText), - case crypto:block_encrypt(Type, Key, IV, Plain) of - CipherText -> - ok; - Other0 -> - ct:fail({{crypto, block_encrypt, [Type, Key, IV, Plain]}, {expected, CipherText}, {got, Other0}}) - end, - case crypto:block_decrypt(Type, Key, IV, CipherText) of - Plain -> - ok; - Other1 -> - ct:fail({{crypto, block_decrypt, [Type, Key, IV, CipherText]}, {expected, Plain}, {got, Other1}}) - end. + cipher_test(T, + fun() -> crypto:block_encrypt(Type, Key, IV, Plain) end, + CipherText, + fun() -> crypto:block_decrypt(Type, Key, IV, CipherText) end, + Plain). block_cipher_increment({Type, Key, IV, PlainTexts}) when Type == des_cbc ; Type == des3_cbc ; @@ -1365,124 +1394,99 @@ stream_cipher_incment_loop(State0, OrigState, [PlainText | PlainTexts], Acc, Pla {State, CipherText} = crypto:stream_encrypt(State0, PlainText), stream_cipher_incment_loop(State, OrigState, PlainTexts, [CipherText | Acc], Plain). -aead_cipher({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, Info}) -> +aead_cipher({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, _Info}=T) -> Plain = iolist_to_binary(PlainText), - case crypto:block_encrypt(Type, Key, IV, {AAD, Plain}) of - {CipherText, CipherTag} -> - ok; - Other0 -> - ct:fail({{crypto, - block_encrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}]}, - {expected, {CipherText, CipherTag}}, - {got, Other0}}) - end, - case crypto:block_decrypt(Type, Key, IV, {AAD, CipherText, CipherTag}) of - Plain -> - ok; - Other1 -> - ct:fail({{crypto, - block_decrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}]}, - {expected, Plain}, - {got, Other1}}) - end; -aead_cipher({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, Info}) -> + cipher_test(T, + fun() -> crypto:block_encrypt(Type, Key, IV, {AAD, Plain}) end, + {CipherText, CipherTag}, + fun() -> crypto:block_decrypt(Type, Key, IV, {AAD, CipherText, CipherTag}) end, + Plain); +aead_cipher({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, _Info}=T) -> <<TruncatedCipherTag:TagLen/binary, _/binary>> = CipherTag, Plain = iolist_to_binary(PlainText), - try crypto:block_encrypt(Type, Key, IV, {AAD, Plain, TagLen}) of - {CipherText, TruncatedCipherTag} -> - ok; - Other0 -> - ct:fail({{crypto, - block_encrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}, {taglen,TagLen}]}, - {expected, {CipherText, TruncatedCipherTag}}, - {got, Other0}}) - catch - error:E -> - ct:log("~p",[{Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, Info}]), - try crypto:crypto_one_time_aead(Type, Key, IV, PlainText, AAD, TagLen, true) - of - RR -> - ct:log("Works: ~p",[RR]) - catch - CC:EE -> - ct:log("~p:~p", [CC,EE]) - end, - ct:fail("~p",[E]) - end, - case crypto:block_decrypt(Type, Key, IV, {AAD, CipherText, TruncatedCipherTag}) of - Plain -> - ok; - Other1 -> - ct:fail({{crypto, - block_decrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}, - {truncated,TruncatedCipherTag}]}, - {expected, Plain}, - {got, Other1}}) - end. + cipher_test(T, + fun() -> crypto:block_encrypt(Type, Key, IV, {AAD, Plain, TagLen}) end, + {CipherText, TruncatedCipherTag}, + fun() -> crypto:block_decrypt(Type, Key, IV, {AAD, CipherText, TruncatedCipherTag}) end, + Plain). -aead_cipher_ng({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, Info}) -> +aead_cipher_ng({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, _Info}=T) -> Plain = iolist_to_binary(PlainText), - case crypto:crypto_one_time_aead(Type, Key, IV, PlainText, AAD, true) of - {CipherText, CipherTag} -> - ok; - Other0 -> - ct:fail({{crypto, - block_encrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}]}, - {expected, {CipherText, CipherTag}}, - {got, Other0}}) - end, - case crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, CipherTag, false) of - Plain -> - ok; - Other1 -> - ct:fail({{crypto, - block_decrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}]}, - {expected, Plain}, - {got, Other1}}) - end; -aead_cipher_ng({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, Info}) -> + cipher_test(T, + fun() -> crypto:crypto_one_time_aead(Type, Key, IV, PlainText, AAD, true) end, + {CipherText, CipherTag}, + fun() -> crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, CipherTag, false) end, + Plain); +aead_cipher_ng({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, _Info}=T) -> <<TruncatedCipherTag:TagLen/binary, _/binary>> = CipherTag, Plain = iolist_to_binary(PlainText), - try crypto:crypto_one_time_aead(Type, Key, IV, PlainText, AAD, TagLen, true) of - {CipherText, TruncatedCipherTag} -> - ok; - Other0 -> - ct:fail({{crypto, - block_encrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}, {taglen,TagLen}]}, - {expected, {CipherText, TruncatedCipherTag}}, - {got, Other0}}) + cipher_test(T, + fun() -> crypto:crypto_one_time_aead(Type, Key, IV, PlainText, AAD, TagLen, true) end, + {CipherText, TruncatedCipherTag}, + fun() -> crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, TruncatedCipherTag, false) end, + Plain). + +aead_cipher_bad_tag({Type, Key, _PlainText, IV, AAD, CipherText, CipherTag, _Info}=T) -> + BadTag = mk_bad_tag(CipherTag), + cipher_test(T, + fun() -> crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, BadTag, false) end, + error); +aead_cipher_bad_tag({Type, Key, _PlainText, IV, AAD, CipherText, CipherTag, TagLen, _Info}=T) -> + <<TruncatedCipherTag:TagLen/binary, _/binary>> = CipherTag, + BadTruncatedTag = mk_bad_tag(TruncatedCipherTag), + cipher_test(T, + fun() -> crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, BadTruncatedTag, false) end, + error). + + +cipher_test(T, Fe, Ee, Fd, Ed) -> + %% Test encrypt + Re = cipher_test(encrypt, T, Fe, Ee), + %% Test decrypt + Rd = cipher_test(decrypt, T, Fd, Ed), + case {Re, Rd} of + {ok,ok} -> ok; + {ok,_} -> Rd; + {_,ok} -> Re; + _ -> {Re,Rd} + end. + +cipher_test(T, F, E) -> + cipher_test(notag, T, F, E). + +cipher_test(Tag, T, F, E) -> + try F() of + E -> ok; + Other -> {other, {Tag,T,Other}} catch - error:E -> - ct:log("~p",[{Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, Info}]), - try crypto:crypto_one_time_aead(Type, Key, IV, PlainText, AAD, TagLen, true) - of - RR -> - ct:log("Works: ~p",[RR]) - catch - CC:EE -> - ct:log("~p:~p", [CC,EE]) - end, - ct:fail("~p",[E]) - end, - case crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, TruncatedCipherTag, false) of - Plain -> - ok; - Other1 -> - ct:fail({{crypto, - block_decrypt, - [{info,Info}, {key,Key}, {pt,PlainText}, {iv,IV}, {aad,AAD}, {ct,CipherText}, {tag,CipherTag}, - {truncated,TruncatedCipherTag}]}, - {expected, Plain}, - {got, Other1}}) + error:Error -> {error, {Tag,T,Error}} + end. + +do_cipher_tests(F, TestVectors) when is_function(F,1) -> + {Passed,Failed} = + lists:partition( + fun(R) -> R == ok end, + lists:map(F, TestVectors) + ), + BothFailed = lists:filter(fun({ok,_}) -> false; + ({_,ok}) -> false; + (ok) -> false; + (_) -> true + end, + Failed), + ct:log("Passed: ~p, BothFailed: ~p OnlyOneFailed: ~p", + [length(Passed), length(BothFailed), length(Failed)-length(BothFailed)]), + case Failed of + [] -> + ct:comment("All ~p passed", [length(Passed)]); + _ -> + ct:log("~p",[hd(Failed)]), + ct:comment("Passed: ~p, BothFailed: ~p OnlyOneFailed: ~p", + [length(Passed), length(BothFailed), length(Failed)-length(BothFailed)]), + ct:fail("Failed", []) end. + mk_bad_tag(CipherTag) -> case <<0:(size(CipherTag))/unit:8>> of CipherTag -> % The correct tag may happen to be a suite of zeroes @@ -1491,30 +1495,6 @@ mk_bad_tag(CipherTag) -> X end. -aead_cipher_bad_tag({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, Info}) -> - Plain = iolist_to_binary(PlainText), - BadTag = mk_bad_tag(CipherTag), - case crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, BadTag, false) of - error -> - ok; - Plain -> - ct:log("~p:~p~n info: ~p~n key: ~p~n pt: ~p~n iv: ~p~n aad: ~p~n ct: ~p~n tag: ~p~n bad tag: ~p~n", - [?MODULE,?LINE,Info, Key, PlainText, IV, AAD, CipherText, CipherTag, BadTag]), - ct:fail("Didn't fail on bad tag") - end; -aead_cipher_bad_tag({Type, Key, PlainText, IV, AAD, CipherText, CipherTag, TagLen, Info}) -> - Plain = iolist_to_binary(PlainText), - <<TruncatedCipherTag:TagLen/binary, _/binary>> = CipherTag, - BadTruncatedTag = mk_bad_tag(TruncatedCipherTag), - case crypto:crypto_one_time_aead(Type, Key, IV, CipherText, AAD, BadTruncatedTag, false) of - error -> - ok; - Plain -> - ct:log("~p:~p~n info: ~p~n key: ~p~n pt: ~p~n iv: ~p~n aad: ~p~n ct: ~p~n tag: ~p~n bad tag: ~p~n", - [Info, Key, PlainText, IV, AAD, CipherText, TruncatedCipherTag, BadTruncatedTag]), - ct:fail("Didn't fail on bad tag") - end. - do_sign_verify({Type, undefined=Hash, Private, Public, Msg, Signature}) -> case crypto:sign(eddsa, Hash, Msg, [Private,Type]) of Signature -> @@ -1598,45 +1578,65 @@ negative_verify(Type, Hash, Msg, Signature, Public, Options) -> end. do_public_encrypt({Type, Public, Private, Msg, Padding}) -> + ct:log("do_public_encrypt Type=~p, Padding=~p,~nPublic = ~p,~nPrivate = ~p,~nMsg = ~p.", + [Type, Padding, Public, Private, Msg]), + timer:sleep(100), try crypto:public_encrypt(Type, Msg, Public, Padding) of PublicEcn -> + ct:log("private_decrypt~nPublicEcn = ~p.", [PublicEcn]), + timer:sleep(100), try crypto:private_decrypt(Type, PublicEcn, Private, Padding) of Msg -> + ct:log("~p:~p ok", [?MODULE,?LINE]), + timer:sleep(100), ok; Other -> + ct:log("~p:~p Other = ~p", [?MODULE,?LINE,Other]), + timer:sleep(100), ct:fail({{crypto, private_decrypt, [Type, PublicEcn, Private, Padding]}, {expected, Msg}, {got, Other}}) catch CC:EE -> + ct:log("~p:~p EXC. ~p:~p", [?MODULE,?LINE,CC,EE]), + timer:sleep(100), ct:fail({{crypto, private_decrypt, [Type, PublicEcn, Private, Padding]}, {expected, Msg}, {got, {CC,EE}}}) end catch CC:EE -> + ct:log("~p:~p EXC 2. ~p:~p", [?MODULE,?LINE,CC,EE]), + timer:sleep(100), ct:fail({{crypto, public_encrypt, [Type, Msg, Public, Padding]}, {got, {CC,EE}}}) end. do_private_encrypt({Type, Public, Private, Msg, Padding}) -> + ct:log("do_private_encrypt Type=~p, Padding=~p,~nPublic = ~p,~nPrivate = ~p,~nMsg = ~p.", + [Type, Padding, Public, Private, Msg]), try crypto:private_encrypt(Type, Msg, Private, Padding) of PrivEcn -> try + ct:log("public_decrypt~nPrivEcn = ~p.", [PrivEcn]), crypto:public_decrypt(Type, PrivEcn, Public, Padding) of Msg -> + ct:log("~p:~p ok", [?MODULE,?LINE]), ok; Other -> + ct:log("~p:~p Other = ~p", [?MODULE,?LINE,Other]), ct:fail({{crypto, public_decrypt, [Type, PrivEcn, Public, Padding]}, {expected, Msg}, {got, Other}}) catch CC:EE -> + ct:log("~p:~p EXC. ~p:~p", [?MODULE,?LINE,CC,EE]), ct:fail({{crypto, public_decrypt, [Type, PrivEcn, Public, Padding]}, {expected, Msg}, {got, {CC,EE}}}) end catch CC:EE -> + ct:log("~p:~p EXC 2. ~p:~p", [?MODULE,?LINE,CC,EE]), ct:fail({{crypto, private_encrypt, [Type, Msg, Private, Padding]}, {got, {CC,EE}}}) end. @@ -1648,6 +1648,9 @@ do_generate_compute({srp = Type, UserPrivate, UserGenParams, UserComParams, UserComParams), SessionKey = crypto:compute_key(Type, UserPublic, {HostPublic, HostPrivate}, HostComParam); + + + do_generate_compute({dh, P, G}) -> {UserPub, UserPriv} = crypto:generate_key(dh, [P, G]), {HostPub, HostPriv} = crypto:generate_key(dh, [P, G]), @@ -1655,6 +1658,7 @@ do_generate_compute({dh, P, G}) -> SharedSecret = crypto:compute_key(dh, UserPub, HostPriv, [P, G]). do_compute({ecdh = Type, Pub, Priv, Curve, SharedSecret}) -> + ct:log("~p ~p", [Type,Curve]), Secret = crypto:compute_key(Type, Pub, Priv, Curve), case Secret of SharedSecret -> @@ -1664,6 +1668,7 @@ do_compute({ecdh = Type, Pub, Priv, Curve, SharedSecret}) -> end. do_generate({Type, Curve, Priv, Pub}) when Type == ecdh ; Type == eddsa -> + ct:log("~p ~p", [Type,Curve]), case crypto:generate_key(Type, Curve, Priv) of {Pub, _} -> ok; @@ -1671,6 +1676,7 @@ do_generate({Type, Curve, Priv, Pub}) when Type == ecdh ; Type == eddsa -> ct:fail({{crypto, generate_key, [Type, Priv, Curve]}, {expected, Pub}, {got, Other}}) end; do_generate({rsa = Type, Mod, Exp}) -> + ct:log("~p", [Type]), case crypto:info_fips() of enabled when Mod < 3072 -> ct:log("SKIP do_generate ~p FIPS=~p, Mod=~p Exp=~p", [Type, enabled, Mod, Exp]), @@ -2101,6 +2107,8 @@ group_config(ecdh, Config) -> Compute = ecdh(), Generate = ecc(), [{compute, Compute}, {generate, Generate} | Config]; +group_config(eddh, Config) -> + [{compute, []}, {generate, []} | Config]; group_config(dh, Config) -> GenerateCompute = [dh()], [{generate_compute, GenerateCompute} | Config]; @@ -2239,12 +2247,13 @@ gen_rsa_sign_verify_tests(Hashs, Msg, Public, Private, Opts) -> gen_rsa_pub_priv_tests(Public, Private, Msg, OptsToTry) -> - SupOpts = proplists:get_value(rsa_opts, crypto:supports(), []), + SupOpts = proplists:get_value(rsa_opts, crypto:supports(), []) -- + [rsa_x931_padding], lists:foldr(fun(Opt, Acc) -> case rsa_opt_is_supported(Opt, SupOpts) of true -> [{rsa, Public, Private, Msg, Opt} | Acc]; - false -> + false -> Acc end end, [], OptsToTry). @@ -3975,11 +3984,13 @@ eddsa(ed448) -> ecdh() -> %% http://csrc.nist.gov/groups/STM/cavp/ - Curves = crypto:ec_curves() ++ - [X || X <- proplists:get_value(curves, crypto:supports(), []), - lists:member(X, [x25519,x448])], - TestCases = - [{ecdh, hexstr2point("42ea6dd9969dd2a61fea1aac7f8e98edcc896c6e55857cc0", "dfbe5d7c61fac88b11811bde328e8a0d12bf01a9d204b523"), + Curves = crypto:supports(curves), + lists:filter( + fun ({_Type, _Pub, _Priv, Curve, _SharedSecret}) -> + lists:member(Curve, Curves) + end, + + [{ecdh, hexstr2point("42ea6dd9969dd2a61fea1aac7f8e98edcc896c6e55857cc0", "dfbe5d7c61fac88b11811bde328e8a0d12bf01a9d204b523"), hexstr2bin("f17d3fea367b74d340851ca4270dcb24c271f445bed9d527"), secp192r1, hexstr2bin("803d8ab2e5b6e6fca715737c3a82f7ce3c783124f6d51cd0")}, @@ -4085,11 +4096,8 @@ ecdh() -> 16#9a8f4925d1519f5775cf46b04b5800d4ee9ee8bae8bc5565d498c28dd9c9baf574a9419744897391006382a6f127ab1d9ac2d8c0a598726b, x448, hexstr2bin("07fff4181ac6cc95ec1c16a94a0f74d12da232ce40a77552281d282bb60c0b56fd2464c335543936521c24403085d59a449a5037514a879d")} - ], - lists:filter(fun ({_Type, _Pub, _Priv, Curve, _SharedSecret}) -> - lists:member(Curve, Curves) - end, - TestCases). + ] + ). dh() -> {dh, 90970053988169282502023478715631717259407236400413906591937635666709823903223997309250405131675572047545403771567755831138144089197560332757755059848492919215391041119286178688014693040542889497092308638580104031455627238700168892909539193174537248629499995652186913900511641708112112482297874449292467498403, 2}. @@ -4143,8 +4151,11 @@ ecc() -> %% information about the curves see %% http://csrc.nist.gov/encryption/dss/ecdsa/NISTReCur.pdf %% - Curves = crypto:ec_curves(), - TestCases = + Curves = crypto:supports(curves), + lists:filter( + fun ({_Type, Curve, _Priv, _Pub}) -> + lists:member(Curve, Curves) + end, [{ecdh,secp192r1,1, hexstr2point("188DA80EB03090F67CBF20EB43A18800F4FF0AFD82FF1012", "07192B95FFC8DA78631011ED6B24CDD573F977A11E794811")}, @@ -4174,12 +4185,9 @@ ecc() -> hexstr2bin("8520f0098930a754748b7ddcb43ef75a0dbf3a0d26381af4eba4a98eaa9b4e6a")}, {ecdh, x25519, hexstr2bin("5dab087e624a8a4b79e17f8b83800ee66f3bb1292618b6fd1c2f8b27ff88e0eb"), - hexstr2bin("de9edb7d7b7dc1b4d35b61c2ece435373f8343c85b78674dadfc7e146f882b4f")}], - lists:filter(fun ({_Type, Curve, _Priv, _Pub}) -> - lists:member(Curve, Curves) - end, - TestCases). - + hexstr2bin("de9edb7d7b7dc1b4d35b61c2ece435373f8343c85b78674dadfc7e146f882b4f")} + ] + ). int_to_bin(X) when X < 0 -> int_to_bin_neg(X, []); int_to_bin(X) -> int_to_bin_pos(X, []). diff --git a/lib/crypto/test/crypto_property_test_SUITE.erl b/lib/crypto/test/crypto_property_test_SUITE.erl index bf137363e8..9c958007c7 100644 --- a/lib/crypto/test/crypto_property_test_SUITE.erl +++ b/lib/crypto/test/crypto_property_test_SUITE.erl @@ -35,6 +35,7 @@ init_per_suite(Config) -> try crypto:start() of ok -> true; {error, already_started} -> true; + {error,{already_started,crypto}} -> true; _ -> false catch _:_ -> false diff --git a/lib/crypto/vsn.mk b/lib/crypto/vsn.mk index 72f3b9b792..477280d84b 100644 --- a/lib/crypto/vsn.mk +++ b/lib/crypto/vsn.mk @@ -1 +1 @@ -CRYPTO_VSN = 4.6.5 +CRYPTO_VSN = 4.7 diff --git a/lib/debugger/doc/src/notes.xml b/lib/debugger/doc/src/notes.xml index 64af47a4fb..5e3f2ce878 100644 --- a/lib/debugger/doc/src/notes.xml +++ b/lib/debugger/doc/src/notes.xml @@ -33,6 +33,43 @@ <p>This document describes the changes made to the Debugger application.</p> +<section><title>Debugger 5.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>EEP-52 has been implemented.</p> + <p>In binary matching, the size of the segment to be + matched is now allowed to be a guard expression, and + similarly in map matching the keys can now be guard + expressions. See the Erlang Reference Manual and + Programming Examples for more details.</p> + <p>Language compilers or code generators that generate + Core Erlang code may need to be updated to be compatible + with the compiler in OTP 23. For more details, see the + section Backwards Compatibility in <url + href="http://erlang.org/eeps/eep-0052.html">EEP + 52</url>.</p> + <p> + Own Id: OTP-14708</p> + </item> + <item> + <p>The deprecated <c>erlang:get_stacktrace/0</c> BIF now + returns an empty list instead of a stacktrace. To + retrieve the stacktrace, use the extended try/catch + syntax that was introduced in OTP 21. + <c>erlang:get_stacktrace/0</c> is scheduled for removal + in OTP 24.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16484</p> + </item> + </list> + </section> + +</section> + <section><title>Debugger 4.2.8</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/debugger/vsn.mk b/lib/debugger/vsn.mk index 06fc743270..8e334a00f5 100644 --- a/lib/debugger/vsn.mk +++ b/lib/debugger/vsn.mk @@ -1 +1 @@ -DEBUGGER_VSN = 4.2.8 +DEBUGGER_VSN = 5.0 diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index b5e6ffb485..675a2b43ef 100644 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -32,6 +32,20 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 4.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Improve handling of <c>maps:remove/2</c>. </p> + <p> + Own Id: OTP-16055 Aux Id: ERL-1002 </p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 4.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index ee680f3bcf..b5a3bbf2b4 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 4.1.1 +DIALYZER_VSN = 4.2 diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index 48bc5d9c74..2871a09476 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -32,6 +32,32 @@ <p>This document describes the changes made to the EDoc application.</p> +<section><title>Edoc 0.12</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Remove Inets dependency from EDoc. </p> + <p> + Own Id: OTP-15999 Aux Id: PR-2317 </p> + </item> + <item> + <p> + Add support for overloaded Erlang specifications.</p> + <p> + Own Id: OTP-16407 Aux Id: PR-2430 </p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + </list> + </section> + +</section> + <section><title>Edoc 0.11</title> <section><title>Improvements and New Features</title> diff --git a/lib/edoc/src/edoc_data.erl b/lib/edoc/src/edoc_data.erl index ed81c9f4c3..3075e47942 100644 --- a/lib/edoc/src/edoc_data.erl +++ b/lib/edoc/src/edoc_data.erl @@ -230,9 +230,11 @@ callback({N, A}, _Env, _Opts) -> %% <!ELEMENT throws (type, localdef*)> %% <!ELEMENT equiv (expr, see?)> %% <!ELEMENT expr (#PCDATA)> - -function({N, A}, As, Export, Ts, Env, Opts) -> - {Args, Ret, Spec} = signature(Ts, As, Env), +function({N, A}, []=As, Export, Ts, Env, Opts)-> + function({N, A}, [As], Export, Ts, Env, Opts); +function({N, A}, [HAs | _]=As, Export, Ts, Env, Opts) when not is_list(HAs) -> + function({N, A}, [As], Export, Ts, Env, Opts); +function({N, A}, As0, Export, Ts, Env, Opts) -> {function, [{name, atom_to_list(N)}, {arity, integer_to_list(A)}, {exported, case Export of @@ -240,13 +242,8 @@ function({N, A}, As, Export, Ts, Env, Opts) -> false -> "no" end}, {label, edoc_refs:to_label(edoc_refs:function(N, A))}], - [{args, [{arg, [{argName, [atom_to_list(A)]}] ++ description(D)} - || {A, D} <- Args]}] - ++ Spec - ++ case Ret of - [] -> []; - _ -> [{returns, description(Ret)}] - end + lists:append([get_args(lists:nth(Clause, As0), Ts, Clause, Env) + || Clause <- lists:seq(1, length(As0))]) ++ get_throws(Ts, Env) ++ get_equiv(Ts, Env) ++ get_doc(Ts) @@ -256,6 +253,16 @@ function({N, A}, As, Export, Ts, Env, Opts) -> ++ todos(Ts, Opts) }. +get_args(As, Ts, Clause, Env) -> + {Args, Ret, Spec} = signature(Ts, As, Clause, Env), + [{args, [{arg, [{argName, [atom_to_list(A)]}] ++ description(D)} + || {A, D} <- Args]}] + ++ Spec + ++ case Ret of + [] -> []; + _ -> [{returns, description(Ret)}] + end. + get_throws(Ts, Env) -> case get_tags(throws, Ts) of [Throws] -> @@ -406,10 +413,10 @@ todos(Tags, Opts) -> [] end. -signature(Ts, As, Env) -> +signature(Ts, As, Clause, Env) -> case get_tags(spec, Ts) of [T] -> - Spec = T#tag.data, + Spec = maybe_nth(Clause, T#tag.data), R = merge_returns(Spec, Ts), As0 = edoc_types:arg_names(Spec), Ds0 = edoc_types:arg_descs(Spec), @@ -424,6 +431,11 @@ signature(Ts, As, Env) -> {[{A, ""} || A <- fix_argnames(As, S, 1)], [], []} end. +maybe_nth(N, List) when is_list(List) -> + lists:nth(N, List); +maybe_nth(1, Other) -> + Other. + params(Ts) -> [T#tag.data || T <- get_tags(param, Ts)]. diff --git a/lib/edoc/src/edoc_extract.erl b/lib/edoc/src/edoc_extract.erl index 390851e9ef..f7e2c28b6f 100644 --- a/lib/edoc/src/edoc_extract.erl +++ b/lib/edoc/src/edoc_extract.erl @@ -634,7 +634,7 @@ select_spec(Ts, _Where, _Specs) -> selected_specs([], Ts) -> Ts; selected_specs([F], [_ | Ts]) -> - [edoc_specs:spec(F, _Clause=1) | Ts]. + [edoc_specs:spec(F) | Ts]. %% Macros for modules diff --git a/lib/edoc/src/edoc_layout.erl b/lib/edoc/src/edoc_layout.erl index 47ff7b21fc..3643419ba1 100644 --- a/lib/edoc/src/edoc_layout.erl +++ b/lib/edoc/src/edoc_layout.erl @@ -357,10 +357,10 @@ label_href(Content, F) -> functions(Fs, Opts) -> Es = lists:flatmap(fun ({Name, E}) -> function(Name, E, Opts) end, Fs), if Es == [] -> []; - true -> - [?NL, - {h2, [{a, [{name, ?FUNCTIONS_LABEL}], [?FUNCTIONS_TITLE]}]}, - ?NL | Es] + true -> + [?NL, + {h2, [{a, [{name, ?FUNCTIONS_LABEL}], [?FUNCTIONS_TITLE]}]}, + ?NL | Es] end. function(Name, E=#xmlElement{content = Es}, Opts) -> @@ -369,22 +369,24 @@ function(Name, E=#xmlElement{content = Es}, Opts) -> label_anchor(function_header(Name, E, " *"), E)}, ?NL] ++ [{'div', [{class, "spec"}], - [?NL, - {p, - case typespec(get_content(typespec, Es), Opts) of + case [typespec(T, Opts) || T <- get_contents(typespec, Es)] of [] -> - signature(get_content(args, Es), - atom(get_attrval(name, E), Opts)); - Spec -> Spec - end}, - ?NL] - ++ case params(get_content(args, Es)) of + [?NL,{p, + signature(get_content(args, Es), + atom(get_attrval(name, E), Opts)) + },?NL]; + Specs -> + [?NL]++[{p, Spec} || Spec <- Specs]++[?NL] + end + ++ case [params(A) || A <- get_contents(args, Es)] of [] -> []; - Ps -> [{p, Ps}, ?NL] + As -> + lists:append([[{p, Ps}, ?NL] || Ps <- As]) end - ++ case returns(get_content(returns, Es)) of + ++ case [returns(Ret) || Ret <- get_contents(returns, Es)] of [] -> []; - Rs -> [{p, Rs}, ?NL] + Rets -> + lists:append([[{p, Rs}, ?NL] || Rs <- Rets]) end}] ++ throws(Es, Opts) ++ equiv_p(Es) @@ -968,12 +970,8 @@ seq(F, [E | Es], Sep, Tail) -> seq(_F, [], _Sep, Tail) -> Tail. -get_elem(Name, [#xmlElement{name = Name} = E | Es]) -> - [E | get_elem(Name, Es)]; -get_elem(Name, [_ | Es]) -> - get_elem(Name, Es); -get_elem(_, []) -> - []. +get_elem(Name, Es) -> + [E || #xmlElement{name=N}=E <- Es, N=:=Name]. get_attr(Name, [#xmlAttribute{name = Name} = A | As]) -> [A | get_attr(Name, As)]; @@ -989,6 +987,13 @@ get_attrval(Name, #xmlElement{attributes = As}) -> [] -> "" end. +get_contents(Name, Es) -> + case get_elem(Name, Es) of + [] -> []; + Elems -> + [Es1 || #xmlElement{content = Es1} <- Elems] + end. + get_content(Name, Es) -> case get_elem(Name, Es) of [#xmlElement{content = Es1}] -> diff --git a/lib/edoc/src/edoc_specs.erl b/lib/edoc/src/edoc_specs.erl index 7b451c43f8..19f890ed8b 100644 --- a/lib/edoc/src/edoc_specs.erl +++ b/lib/edoc/src/edoc_specs.erl @@ -21,7 +21,7 @@ -module(edoc_specs). --export([type/2, spec/2, dummy_spec/1, docs/2]). +-export([type/2, spec/1, dummy_spec/1, docs/2]). -export([add_data/4, tag/1, is_tag/1]). @@ -67,15 +67,14 @@ type(Form, TypeDocs) -> type = d2e(opaque2abstr(Name, Type))}, Doc}}. --spec spec(Form::syntaxTree(), ClauseN::pos_integer()) -> #tag{}. +-spec spec(Form::syntaxTree()) -> #tag{}. %% @doc Convert an Erlang spec to EDoc representation. -spec(Form, Clause) -> +spec(Form) -> {Name, _Arity, TypeSpecs} = get_spec(Form), - TypeSpec = lists:nth(Clause, TypeSpecs), - #tag{name = spec, line = get_line(element(2, TypeSpec)), + #tag{name = spec, line = get_line(element(2, lists:nth(1, TypeSpecs))), origin = code, - data = aspec(d2e(TypeSpec), Name)}. + data = [aspec(d2e(TypeSpec), Name) || TypeSpec <- TypeSpecs]}. -spec dummy_spec(Form::syntaxTree()) -> #tag{}. @@ -264,8 +263,9 @@ use_tags([#tag{origin = code}=T | Ts], E, TypeTable, NTs) -> use_tags([T | Ts], E, TypeTable, NTs) -> use_tags(Ts, E, TypeTable, [T | NTs]). -params(#tag{name = spec, data=#t_spec{type = #t_fun{args = As}}}, Default) -> - parms(As, Default). + +params(#tag{name = spec, data=Data}, Default) when is_list(Data) -> + [parms(As, Default) || #t_spec{type = #t_fun{args = As}} <- Data]. parms([], []) -> []; @@ -485,13 +485,17 @@ entries([E0 | Es], P, Opts) -> entries([], _P, _Opts) -> []. -specs([#tag{line = L, name = spec, origin = code, data = Spec}=Tag0 | Tags], +specs([#tag{line = L, name = spec, origin = code, data = Specs}=Tag0 | Tags], P0) -> - #t_spec{type = Type0, defs = Defs0} = Spec, P = P0#parms{line = L}, - Type = xrecs(Type0, P), - Defs = xrecs(Defs0, P), - Tag = Tag0#tag{data = Spec#t_spec{type = Type, defs = Defs}}, + Data = + [ begin + #t_spec{type = Type0, defs = Defs0} = Spec, + Type = xrecs(Type0, P), + Defs = xrecs(Defs0, P), + Spec#t_spec{type = Type, defs = Defs} + end || Spec <- Specs], + Tag = Tag0#tag{data = Data}, [Tag | specs(Tags, P)]; specs([Tag | Tags], P) -> [Tag | specs(Tags, P)]; diff --git a/lib/edoc/vsn.mk b/lib/edoc/vsn.mk index 3510fdfccf..5d2bbe769d 100644 --- a/lib/edoc/vsn.mk +++ b/lib/edoc/vsn.mk @@ -1 +1 @@ -EDOC_VSN = 0.11 +EDOC_VSN = 0.12 diff --git a/lib/erl_docgen/Makefile b/lib/erl_docgen/Makefile index 7e9cc824ec..acb1ea9776 100644 --- a/lib/erl_docgen/Makefile +++ b/lib/erl_docgen/Makefile @@ -36,6 +36,6 @@ SPECIAL_TARGETS = # include $(ERL_TOP)/make/otp_subdir.mk -DIA_PLT_APPS=edoc xmerl +DIA_PLT_APPS=edoc xmerl syntax_tools crypto include $(ERL_TOP)/make/app_targets.mk diff --git a/lib/erl_docgen/doc/src/doc_storage.xml b/lib/erl_docgen/doc/src/doc_storage.xml index 2fd804b522..2c719b3f20 100644 --- a/lib/erl_docgen/doc/src/doc_storage.xml +++ b/lib/erl_docgen/doc/src/doc_storage.xml @@ -21,7 +21,7 @@ limitations under the License. </legalnotice> - <title>Documentation Storage</title> + <title>EEP-48: Implementation in Erlang/OTP</title> <prepared></prepared> <docno></docno> <date></date> @@ -31,29 +31,103 @@ <section> <title>EEP-48: Documentation storage and format</title> - <p><url href="https://www.erlang.org/erlang-enhancement-proposals/eep-0048.html">EEP-48</url> + <p><seeguide marker="kernel:eep48_chapter">EEP-48</seeguide> defines a common documentation storage format for module documentation in the Erlang/OTP ecosystem. Erl_Docgen can generate documentation in this format from XML files following the DTD's descibed in the other User's Guides in this application.</p> - <p></p> <p>Some special considerations have to be taken when writing documentation that should also be available through EEP-48 style storage.</p> <list> - <item>The <c>#PCDATA</c> within <c><name></c> tags must be parseable to figure out the arity of the function.</item> + <item>The <c>#PCDATA</c> within <c><name></c> tags must be parseable + to figure out the arity of the function.</item> <item>It is not allowed to mix <c><name></c> tags with #PCDATA and attributes.</item> - <item>All <c><name></c> tags within <c><func></c> has to have a <c>since</c> attribute.</item> + <item>All <c><name></c> tags within <c><func></c> + has to have a <c>since</c> attribute.</item> <item>All callback function documentations have to start with a <c>Module</c> prefix.</item> </list> </section> <section> <title>Erlang Documentation Format</title> - <p>When generating documentation for generic storage</p> + <p>When generating documentation for EEP-48 Erl_Docgen uses the format mime type + <<"application/erlang+html">>. The documentation content is an Erlang + term that represents an HTML like structure.</p> + <code> +-type chunk_elements() :: [chunk_element()]. +-type chunk_element() :: {chunk_element_type(),chunk_element_attrs(), + chunk_elements()} | unicode:unicode_binary(). +-type chunk_element_attrs() :: [chunk_element_attr()]. +-type chunk_element_attr() :: {atom(),unicode:unicode_binary()}. +-type chunk_element_type() :: chunk_element_inline_type() | chunk_element_block_type(). +-type chunk_element_inline_type() :: a | code | em | i. +-type chunk_element_block_type() :: p | 'div' | br | pre | ul | + ol | li | dl | dt | dd | h1 | h2 | h3. + </code> + <p>The different element types follow their HTML meaning when rendered. + The following are some general rules for how the chunk elements are allowed + to be generated.</p> + <list> + <item>Inline and <c>pre</c> elements are not allowed to contain block elements.</item> + <item><c>p</c> elements are not allowed to be nested.</item> + </list> + <p>The attributes on some elements have a special meaning.</p> + <taglist> + <tag><c>{'div',[{class,unicode:unicode_binary()}],_}</c></tag> + <item>The class name will be used to provide styling to the content in the div. + The types of classes used by Erlang/OTP are: <c>warning</c>, <c>note</c>, <c>do</c>, + <c>dont</c> and <c>quote</c>.</item> + <tag><c><![CDATA[{ul,[{class,<<"types">>}],_}]]></c></tag> + <item>This is a list containing type documentation.</item> + <tag><c><![CDATA[{li,[{name,TypeName :: unicode:unicode_binary()}],_}]]></c></tag> + <item>A list item with a type specification located in the metadata of this modules + EEP-48 documentation. The implementation should look for the AST representation of + the type under the <c>types</c> key. This attribute is only valid under a <c>ul</c> + with class <<"types">>.</item> + <tag><c><![CDATA[{li,[{class,<<"type">>}],_}]]></c></tag> + <item>A list item with the type described in the Erlang Documentation Format. + This attribute is only valid under a <c>ul</c> with class <<"types">>.</item> + <tag><c><![CDATA[{li,[{class,<<"description">>}],_}]]></c></tag> + <item>A list item with the description of the type previous in the list. + This attribute is only valid under a <c>ul</c> with class <<"types">>.</item> + </taglist> + <p>The <seemfa marker="stdlib:shell_docs#validate/1"><c>shell_docs:validate/1</c></seemfa> + function can be used to do a validation of the Erlang Documentation Format.</p> + </section> + + <section> + <title>Erlang Documentation extra Metadata</title> + <p>Erlang/OTP uses some extra metadata fields to embed more information into the EEP-48 docs.</p> + <list> + <item>Fields on module level: + <taglist> + <tag><c>otp_doc_vsn := {non_neg_integer(),non_neg_integer(),non_neg_integer()}</c></tag> + <item>Describes the version of the Erlang Documentation Format used + within this module</item> + <tag><c>types := #{ TypeName :: unicode:unicode_binary() => TypeAST }</c></tag> + <item>A map containing the AST of the types that are part of this module. + This map is used to by functions and callbacks to render the types inline + into their documentation.</item> + </taglist> + </item> + <item>Fields on functions and types: + <taglist> + <tag><c>signature := SpecAST</c></tag> + <item>The spec AST associated with this function. It is used to render a more + descriptive slogan for the documentation entry.</item> + <tag><c>equiv := {Type,Name,Arity}</c></tag> + <item>The current function/type shares documentation with another function/type. + This means that if this and the target function/type are to be shown at the + same time only the prototype of this function/type should will be displayed + and the documentation will use a common body of text.</item> + </taglist> + </item> + </list> </section> <section> <title>See Also</title> <p> + <seeguide marker="kernel:eep48_chapter"></seeguide> <seeerl marker="stdlib:shell_docs"><c>shell_docs(3)</c></seeerl>, <seemfa marker="kernel:code#get_doc/1"><c>code:get_doc(3)</c></seemfa> </p> diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index e5076a4790..08bcc3d8da 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -31,7 +31,53 @@ </header> <p>This document describes the changes made to the <em>erl_docgen</em> application.</p> - <section><title>Erl_Docgen 0.11</title> + <section><title>Erl_Docgen 1.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Embedded documentation (also known as Documentation + Chunks) is now also available in the form of files + according to <url + href="https://www.erlang.org/erlang-enhancement-proposals/eep-0048.html">EEP-48</url>. + The Documentation Chunks are produced by default when + building the other Erlang/OTP documentation. If you want + to only build the embedded documentation you can pass the + <c>DOC_TARGETS=chunks</c> environment variable to make.</p> + <p> + Own Id: OTP-16406</p> + </item> + <item> + <p> + Minor DTD additions.</p> + <p> + Own Id: OTP-16497</p> + </item> + <item> + <p> + The <c>seealso</c> tag has been replaced with type aware + tags instead. The new tags are: + <c>seemfa|seeerl|seetype|seeapp|seecom|seecref|seefile|seeguide</c>.</p> + <p> + <c>fsdescription</c> has been added for adding a title to + groups of functions, for instance Module Callbacks.</p> + <p> + The <c>dtd</c>s of all documentation files have been + trimmed from all unused or rarely-used tags.</p> + <p> + Unused <c>dtd</c>s have been removed.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16503</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.11</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_docgen/priv/bin/specs_gen.escript b/lib/erl_docgen/priv/bin/specs_gen.escript index 96b63aa667..5b75a83a7e 100644 --- a/lib/erl_docgen/priv/bin/specs_gen.escript +++ b/lib/erl_docgen/priv/bin/specs_gen.escript @@ -89,8 +89,9 @@ call_edoc(FileSpec, InclFs, Dir) -> ok = write_text(Text, File, Dir), rename(Dir, File) catch - _:_ -> + E:R:ST -> io:format("EDoc could not process file '~s'\n", [File]), + io:format("~p:~p ~p\n", [E,R,ST]), clean_up(Dir), halt(3) end. diff --git a/lib/erl_docgen/priv/bin/validate_links.escript b/lib/erl_docgen/priv/bin/validate_links.escript index 41d533fb58..4d93d9c253 100755 --- a/lib/erl_docgen/priv/bin/validate_links.escript +++ b/lib/erl_docgen/priv/bin/validate_links.escript @@ -259,11 +259,6 @@ validate_link(Filename, LinkType = "seetype", Line, Link, CachedFiles) -> _ -> validate_type(Line,LinkType,read_link(Line, ParsedLink, CachedFiles)) end; -validate_link(Filename, "seeerl" = LinkType, Line, Link, CachedFiles) -> - ParsedLink = parse_link(Filename, maps:get(m2a,CachedFiles), Link), - TargetInfo = read_link(Line, ParsedLink, CachedFiles), - validate_type(Line,LinkType,TargetInfo), - validate_marker(Line,ParsedLink,TargetInfo); validate_link({"jinterface","jinterface_users_guide"},"seefile",_, _, _) -> %% Skip links to java documentation ok; diff --git a/lib/erl_docgen/priv/dtd/common.dtd b/lib/erl_docgen/priv/dtd/common.dtd index d4d5d989a5..cc186fe5d6 100644 --- a/lib/erl_docgen/priv/dtd/common.dtd +++ b/lib/erl_docgen/priv/dtd/common.dtd @@ -51,7 +51,7 @@ <!ELEMENT list (item+) > <!ATTLIST list type (ordered|bulleted) "bulleted" > -<!ELEMENT taglist (marker*,tag,item+)+ > +<!ELEMENT taglist (tag,item+)+ > <!ELEMENT tag (#PCDATA|c|i|em|br|%refs;|marker|anno)* > <!ELEMENT item (%inline;|%block;|warning|note|dont|do|quote|table)* > diff --git a/lib/erl_docgen/priv/xsl/db_html.xsl b/lib/erl_docgen/priv/xsl/db_html.xsl index 15c86adf2d..c14e7c9a71 100644 --- a/lib/erl_docgen/priv/xsl/db_html.xsl +++ b/lib/erl_docgen/priv/xsl/db_html.xsl @@ -2267,10 +2267,10 @@ </xsl:variable> <xsl:choose> <xsl:when test="string-length($fname2) > 0"> - <xsl:value-of select="$fname2"/> + <xsl:value-of select="normalize-space($fname2)"/> </xsl:when> <xsl:otherwise> - <xsl:value-of select="$fname1"/> + <xsl:value-of select="normalize-space($fname1)"/> </xsl:otherwise> </xsl:choose> </xsl:variable> diff --git a/lib/erl_docgen/src/docgen_edoc_xml_cb.erl b/lib/erl_docgen/src/docgen_edoc_xml_cb.erl index 60d73c279d..47ca1690e2 100644 --- a/lib/erl_docgen/src/docgen_edoc_xml_cb.erl +++ b/lib/erl_docgen/src/docgen_edoc_xml_cb.erl @@ -757,23 +757,21 @@ functions(Fs) -> function(_Name, E=#xmlElement{content = Es}) -> TypeSpec = get_content(typespec, Es), - [?NL,{func, [ ?NL, - {name, [{since,""}], - case funcheader(TypeSpec) of - [] -> - signature(get_content(args, Es), - get_attrval(name, E)); - Spec -> Spec - end - }, - ?NL,{fsummary, fsummary(Es)}, - ?NL,local_types(TypeSpec), - ?NL,{desc, - label_anchor(E)++ - deprecated(Es)++ - fulldesc(Es)++ - seealso_function(Es)} - ]}]. + FuncHeaders = + case funcheader(TypeSpec) of + [] -> + [signature(get_content(args, Es), get_attrval(name, E))]; + Specs -> + Specs + end, + [?NL, {func, [?NL]++ + [{name, [{since,""}], Spec} || Spec <- FuncHeaders]++ + [?NL, {fsummary, fsummary(Es)}, + ?NL, local_types(TypeSpec), + ?NL, {desc, label_anchor(E)++ + deprecated(Es)++ + fulldesc(Es)++ + seealso_function(Es)}]}]. fsummary([]) -> ["\s"]; fsummary(Es) -> @@ -817,7 +815,9 @@ arg(#xmlElement{content = Es}) -> funcheader([]) -> []; funcheader(Es) -> - [t_name(get_elem(erlangName, Es))] ++ t_utype(get_elem(type, Es)). + Name = t_name(get_elem(erlangName, Es)), + [ [Name] ++ t_utype([E]) || E <- get_elem(type, Es)]. + local_types([]) -> []; local_types(Es) -> @@ -1035,7 +1035,7 @@ author(E=#xmlElement{}) -> end, [?NL,{aname,[Name]},?NL,{email,[Mail]}]. -t_name([E]) -> +t_name([E | _]) -> N = get_attrval(name, E), case get_attrval(module, E) of "" -> N; @@ -1246,12 +1246,15 @@ get_attrval(Name, #xmlElement{attributes = As}) -> %% get_content(Tag, Es1) -> Es2 %% If there is one element in Es1 with name Tag, returns its contents, -%% otherwise [] +%% if there are no tags, return [], +%% if there are multiple, merge their contents. get_content(Name, Es) -> case get_elem(Name, Es) of - [#xmlElement{content = Es1}] -> - Es1; - [] -> [] + [#xmlElement{content = Es1}] -> + Es1; + [] -> []; + Elems -> + lists:append([Es1 || #xmlElement{content = Es1} <- Elems]) end. %% get_text(Tag, Es) -> string() diff --git a/lib/erl_docgen/src/docgen_xml_to_chunk.erl b/lib/erl_docgen/src/docgen_xml_to_chunk.erl index c57ecac213..6c9ae59996 100644 --- a/lib/erl_docgen/src/docgen_xml_to_chunk.erl +++ b/lib/erl_docgen/src/docgen_xml_to_chunk.erl @@ -525,30 +525,60 @@ func2func({func,Attr,Contents}) -> _ = VerifyNameList(NameList,fun([]) -> ok end), FAs = [TagsToFA(FAttr) || {name,FAttr,[]} <- NameList ], + SortedFAs = lists:usort(FAs), FAClauses = lists:usort([{TagsToFA(FAttr),proplists:get_value(clause_i,FAttr)} || {name,FAttr,[]} <- NameList ]), - Signature = [iolist_to_binary([F,"/",A]) || {F,A} <- FAs], - lists:map( - fun({F,A}) -> - Specs = [{func_to_atom(CF),list_to_integer(CA),C} - || {{CF,CA},C} <- FAClauses, - F =:= CF, A =:= CA], - {function,[{name,F},{arity,list_to_integer(A)}, - {signature,Signature}, - {meta,SinceMD#{ signature => Specs }}], - ContentsNoName} - end, lists:usort(FAs)); + + MakeFunc = fun({F,A}, MD, Doc) -> + Specs = [begin + {function,Name} = func_to_atom(CF), + {Name,list_to_integer(CA),C} + end || {{CF,CA},C} <- FAClauses, + F =:= CF, A =:= CA], + {function,[{name,F},{arity,list_to_integer(A)}, + {signature,[iolist_to_binary([F,"/",A])]}, + {meta,MD#{ signature => Specs }}], + Doc} + end, + + Base = MakeFunc(hd(SortedFAs), SinceMD, ContentsNoName), + + {BaseF,BaseA} = hd(SortedFAs), + MD = SinceMD#{ equiv => {function,list_to_atom(BaseF),list_to_integer(BaseA)}}, + Equiv = lists:map( + fun(FA) -> + MakeFunc(FA, MD, []) + end, tl(SortedFAs)), + [Base | Equiv]; NameList -> %% Manual style function docs - FAs = lists:flatten([func_to_tuple(NameString) || {name, _Attr, NameString} <- NameList]), + FAs = lists:foldl( + fun({name,_,NameString}, Acc) -> + FAs = func_to_tuple(NameString), + lists:foldl( + fun(FA, FAAcc) -> + Slogan = maps:get(FA, FAAcc, []), + FAAcc#{ FA => [strip_tags(NameString)|Slogan] } + end, Acc, FAs) + end, #{}, NameList), _ = VerifyNameList(NameList,fun([_|_]) -> ok end), - Signature = [strip_tags(NameString) || {name, _Attr, NameString} <- NameList], - [{function,[{name,F},{arity,A}, - {signature,Signature}, - {meta,SinceMD}],ContentsNoName} - || {F,A} <- lists:usort(FAs)] + SortedFAs = lists:usort(maps:to_list(FAs)), + + {{BaseF, BaseA}, BaseSig} = hd(SortedFAs), + + Base = {function,[{name,BaseF},{arity,BaseA}, + {signature,BaseSig}, + {meta,SinceMD}], + ContentsNoName}, + + Equiv = [{function, + [{name,F},{arity,A}, + {signature,Signature}, + {meta,SinceMD#{ equiv => {function,list_to_atom(BaseF),BaseA}}}],[]} + || {{F,A},Signature} <- tl(SortedFAs)], + [Base | Equiv] end, transform(Functions,[]). @@ -644,7 +674,7 @@ to_chunk(Dom, Source, Module, AST) -> TypeEntries = lists:map( fun({datatype,Attr,Descr}) -> - TypeName = func_to_atom(proplists:get_value(name,Attr)), + {function, TypeName} = func_to_atom(proplists:get_value(name,Attr)), TypeArity = case proplists:get_value(n_vars,Attr) of undefined -> find_type_arity(TypeName, TypeMap); @@ -662,24 +692,48 @@ to_chunk(Dom, Source, Module, AST) -> Sig -> #{ signature => [Sig] } end, - docs_v1_entry(type, Anno, TypeName, TypeArity, TypeSignature, MetaSig, Descr) + + MetaDepr + = case otp_internal:obsolete_type(Module, TypeName, TypeArity) of + {deprecated, Text} -> + MetaSig#{ deprecated => + unicode:characters_to_binary( + erl_lint:format_error({deprecated_type,{Module,TypeName,TypeArity}, Text})) }; + %% Commented out to make dialyzer happy + %% {deprecated, Replacement, Rel} -> + %% MetaSig#{ deprecated => + %% unicode:characters_to_binary( + %% erl_lint:format_error({deprecated_type,{Module,TypeName,TypeArity}, Replacement, Rel})) }; + no -> + MetaSig + end, + + docs_v1_entry(type, Anno, TypeName, TypeArity, TypeSignature, MetaDepr, Descr) end, Types), Functions = lists:flatten([Functions || {functions,[],Functions} <- Mcontent]), FuncEntrys = - lists:flatmap( + lists:map( fun({function,Attr,Fdoc}) -> - case func_to_atom(proplists:get_value(name,Attr)) of - callback -> - []; - Name -> - Arity = proplists:get_value(arity,Attr), - Signature = proplists:get_value(signature,Attr), - FMeta = proplists:get_value(meta,Attr), - MetaWSpec = add_spec(AST,FMeta), - [docs_v1_entry(function, Anno, Name, Arity, Signature, MetaWSpec, Fdoc)] - end + {Type, Name} = func_to_atom(proplists:get_value(name,Attr)), + Arity = proplists:get_value(arity,Attr), + Signature = proplists:get_value(signature,Attr), + FMeta = proplists:get_value(meta,Attr), + MetaWSpec = add_spec(AST,FMeta), + MetaDepr + = case otp_internal:obsolete(Module, Name, Arity) of + {deprecated, Text} -> + MetaWSpec#{ deprecated => + unicode:characters_to_binary( + erl_lint:format_error({deprecated,{Module,Name,Arity}, Text})) }; + {deprecated, Replacement, Rel} -> + MetaWSpec#{ deprecated => + unicode:characters_to_binary( + erl_lint:format_error({deprecated,{Module,Name,Arity}, Replacement, Rel})) }; + _ -> MetaWSpec + end, + docs_v1_entry(Type, Anno, Name, Arity, Signature, MetaDepr, Fdoc) end, Functions), docs_v1(ModuleDocs, Anno, TypeMeta, FuncEntrys ++ TypeEntries). @@ -691,6 +745,7 @@ docs_v1(DocContents, Anno, Metadata, Docs) -> docs = Docs }. docs_v1_entry(Kind, Anno, Name, Arity, Signature, Metadata, DocContents) -> + AnnoWLine = case Metadata of #{ signature := [Sig|_] } -> @@ -699,8 +754,16 @@ docs_v1_entry(Kind, Anno, Name, Arity, Signature, Metadata, DocContents) -> _NoSignature -> Anno end, - {{Kind, Name, Arity}, AnnoWLine, lists:flatten(Signature), - #{ <<"en">> => shell_docs:normalize(DocContents)}, Metadata}. + + Doc = + case DocContents of + [] -> + #{}; + DocContents -> + #{ <<"en">> => shell_docs:normalize(DocContents) } + end, + + {{Kind, Name, Arity}, AnnoWLine, lists:flatten(Signature), Doc, Metadata}. %% A special list_to_atom that handles %% 'and' @@ -708,11 +771,16 @@ docs_v1_entry(Kind, Anno, Name, Arity, Signature, Metadata, DocContents) -> %% 'begin' func_to_atom(List) -> case erl_scan:string(List) of - {ok,[{atom,_,Fn}],_} -> Fn; - {ok,[{var,_,Fn}],_} -> Fn; - {ok,[{Fn,_}],_} -> Fn; - {ok,[{var,_,_},{':',_},_],_} -> - callback + {ok,[{atom,_,Fn}],_} -> + {function, Fn}; + {ok,[{var,_,Fn}],_} -> + {function, Fn}; + {ok,[{Fn,_}],_} -> + {function, Fn}; + {ok,[{var,_,_},{':',_},{atom,_,Fn}],_} -> + {callback, Fn}; + {ok,[{var,_,_},{':',_},{var,_,Fn}],_} -> + {callback, Fn} end. -define(IS_TYPE(TO),(TO =:= type orelse TO =:= opaque)). diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index ebc9516da3..57b2fd10f4 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -1 +1 @@ -ERL_DOCGEN_VSN = 0.11 +ERL_DOCGEN_VSN = 1.0 diff --git a/lib/erl_interface/doc/src/ei.xml b/lib/erl_interface/doc/src/ei.xml index 361021bf50..a47b344c4c 100644 --- a/lib/erl_interface/doc/src/ei.xml +++ b/lib/erl_interface/doc/src/ei.xml @@ -35,9 +35,6 @@ <lib>ei</lib> <libsummary>Routines for handling the Erlang binary term format.</libsummary> <description> - <note><p>The support for VxWorks is deprecated as of OTP 22, and - will be removed in OTP 23.</p></note> - <p>The library <c>ei</c> contains macros and functions to encode and decode the Erlang binary term format.</p> @@ -85,7 +82,9 @@ <p>There are also encode functions that use a dynamic buffer. It is often more convenient to use these to encode data. All encode functions comes in two versions; those starting with - <c>ei_x</c> use a dynamic buffer.</p> + <c>ei_x_</c> use a dynamic buffer of type + <seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref>. + </p> <p>All functions return <c>0</c> if successful, otherwise <c>-1</c> (for example, if a term is not of the expected @@ -93,7 +92,7 @@ <p>Some of the decode functions need a pre-allocated buffer. This buffer must be allocated large enough, and for non-compound types - the <c>ei_get_type()</c> + the <seecref marker="#ei_get_type"><c>ei_get_type()</c></seecref> function returns the size required (notice that for strings an extra byte is needed for the <c>NULL</c>-terminator).</p> </description> @@ -101,7 +100,55 @@ <section> <title>Data Types</title> <taglist> - <tag><marker id="erlang_char_encoding"/>erlang_char_encoding</tag> + <tag><marker id="ei_term"/><c>ei_term</c></tag> + <item> + <code type="none"> +typedef struct { + char ei_type; + int arity; + int size; + union { + long i_val; + double d_val; + char atom_name[MAXATOMLEN_UTF8]; + erlang_pid pid; + erlang_port port; + erlang_ref ref; + } value; +} ei_term;</code> + <p>Structure written by + <seecref marker="#ei_decode_ei_term"><c>ei_decode_ei_term()</c></seecref>. + The <c>ei_type</c> field is the type of the term which equals to + what <seecref marker="#ei_get_type"><c>ei_get_type()</c></seecref> + sets <c>*type</c> to. + </p> + </item> + <tag><marker id="ei_x_buff"/><c>ei_x_buff</c></tag> + <item> + <p>A dynamically resized buffer. It is a <c>struct</c> with + two fields of interest for the user: + </p> + <taglist> + <tag><c>char *buff</c></tag> + <item> + <p>Pointer to the dynamically allocated buffer.</p> + </item> + <tag><c>int index</c></tag> + <item> + <p>Offset to the next byte to write which also equals the + amount of bytes currently written.</p> + </item> + </taglist> + <p> + An <c>ei_x_buff</c> is initialized by calling either + <seecref marker="#ei_x_new"><c>ei_x_new()</c></seecref> or + <seecref marker="#ei_x_new_with_version"><c>ei_x_new_with_version()</c></seecref>. + The memory used by an initialized <c>ei_x_buff</c> is released + by calling + <seecref marker="#ei_x_free"><c>ei_x_free()</c></seecref>. + </p> + </item> + <tag><marker id="erlang_char_encoding"/><c>erlang_char_encoding</c></tag> <item> <code type="none"> typedef enum { @@ -117,11 +164,91 @@ typedef enum { Notice that these constants are bit-flags and can be combined with bitwise OR.</p> </item> + <tag><marker id="erlang_fun"/><c>erlang_fun</c></tag> + <item> + <p>Opaque data type representing an Erlang fun.</p> + </item> + <tag><marker id="erlang_pid"/><c>erlang_pid</c></tag> + <item> + <p>Opaque data type representing an Erlang process identifier.</p> + </item> + <tag><marker id="erlang_port"/><c>erlang_port</c></tag> + <item> + <p>Opaque data type representing an Erlang port identifier.</p> + </item> + <tag><marker id="erlang_ref"/><c>erlang_ref</c></tag> + <item> + <p>Opaque data type representing an Erlang reference.</p> + </item> + <tag><marker id="erlang_trace"/><c>erlang_trace</c></tag> + <item> + <p>Opaque data type representing an Erlang sequential trace token.</p> + </item> </taglist> </section> <funcs> <func> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_cmp_pids(erlang_pid *a, erlang_pid *b)</nametext></name> + <fsummary>Compare two pids.</fsummary> + <type> + <v><seecref marker="#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> + <desc> + <p> + Compare two process identifiers. The comparison is done the same way + as Erlang does. + </p> + <p> + Returns <c>0</c> if <c>a</c> and <c>b</c> are equal. Returns a value + less than <c>0</c> if <c>a</c> compares as less than <c>b</c>. + Returns a value larger than <c>0</c> if <c>a</c> compares as larger + than <c>b</c>. + </p> + </desc> + </func> + + <func> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_cmp_ports(erlang_port *a, erlang_port *b)</nametext></name> + <fsummary>Compare two ports.</fsummary> + <type> + <v><seecref marker="#erlang_port"><c>erlang_port</c></seecref></v> + </type> + <desc> + <p> + Compare two port identifiers. The comparison is done the same way as + Erlang does. + </p> + <p> + Returns <c>0</c> if <c>a</c> and <c>b</c> are equal. Returns a value + less than <c>0</c> if <c>a</c> compares as less than <c>b</c>. + Returns a value larger than <c>0</c> if <c>a</c> compares as larger + than <c>b</c>. + </p> + </desc> + </func> + + <func> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_cmp_refs(erlang_ref *a, erlang_ref *b)</nametext></name> + <fsummary>Compare two references.</fsummary> + <type> + <v><seecref marker="#erlang_ref"><c>erlang_ref</c></seecref></v> + </type> + <desc> + <p> + Compare two references. The comparison is done the same way as Erlang + does. + </p> + <p> + Returns <c>0</c> if <c>a</c> and <c>b</c> are equal. Returns a value + less than <c>0</c> if <c>a</c> compares as less than <c>b</c>. + Returns a value larger than <c>0</c> if <c>a</c> compares as larger + than <c>b</c>. + </p> + </desc> + </func> + + <func> <name since=""><ret>int</ret><nametext>ei_decode_atom(const char *buf, int *index, char *p)</nametext></name> <fsummary>Decode an atom.</fsummary> <desc> @@ -134,6 +261,9 @@ typedef enum { <func> <name since="OTP R16B"><ret>int</ret><nametext>ei_decode_atom_as(const char *buf, int *index, char *p, int plen, erlang_char_encoding want, erlang_char_encoding* was, erlang_char_encoding* result)</nametext></name> <fsummary>Decode an atom.</fsummary> + <type> + <v><seecref marker="#erlang_char_encoding"><c>erlang_char_encoding</c></seecref></v> + </type> <desc> <p>Decodes an atom from the binary format. The <c>NULL</c>-terminated name of the atom is placed in buffer at <c>p</c> of length <c>plen</c> @@ -173,7 +303,8 @@ typedef enum { <c>len</c> is set to the actual size of the binary. Notice that <c>ei_decode_binary()</c> assumes that there is enough room for the binary. The size required can be - fetched by <c>ei_get_type()</c>.</p> + fetched by + <seecref marker="#ei_get_type"><c>ei_get_type()</c></seecref>.</p> </desc> </func> @@ -249,6 +380,9 @@ typedef enum { <func> <name since=""><ret>int</ret><nametext>ei_decode_ei_term(const char* buf, int* index, ei_term* term)</nametext></name> <fsummary>Decode a term, without previous knowledge of type.</fsummary> + <type> + <v><seecref marker="#ei_term"><c>ei_term</c></seecref></v> + </type> <desc> <p>Decodes any term, or at least tries to. If the term pointed at by <c>*index</c> in <c>buf</c> fits @@ -271,6 +405,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_decode_fun(const char *buf, int *index, erlang_fun *p)</nametext></name> <name since=""><ret>void</ret><nametext>free_fun(erlang_fun* f)</nametext></name> <fsummary>Decode a fun.</fsummary> + <type> + <v><seecref marker="#erlang_fun"><c>erlang_fun</c></seecref></v> + </type> <desc> <p>Decodes a fun from the binary format. Parameter <c>p</c> is to be <c>NULL</c> or point to an @@ -283,6 +420,37 @@ typedef enum { </func> <func> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_decode_iodata(const char *buf, int *index, int *size, char *outbuf)</nametext></name> + <fsummary>Decode iodata().</fsummary> + <desc> + <p>Decodes a term of the type <seeguide marker="system/reference_manual:typespec#builtin_types"><c>iodata()</c></seeguide>. The <c>iodata()</c> term will be + flattened an written into the buffer pointed to by the <c>outbuf</c> + argument. The byte size of the <c>iodata</c> is written into the + integer variable pointed to by the <c>size</c> argument. Both <c>size</c> + and <c>outbuf</c> can be set to <c>NULL</c>. The integer pointed to + by the <c>index</c> argument is updated to refer to the term + following after the <c>iodata()</c> term regardless of the the state + of the <c>size</c> and the <c>outbuf</c> arguments. + </p> + <p>Note that the buffer pointed to by the <c>outbuf</c> argument + must be large enough if a non <c>NULL</c> value is passed as + <c>outbuf</c>. You typically want to call <c>ei_decode_iodata()</c> + twice. First with a non <c>NULL</c> <c>size</c> argument and + a <c>NULL</c> <c>outbuf</c> argument in order to determine the + size of the buffer needed, and then once again in order to do + the actual decoding. Note that the integer pointed to by <c>index</c> + will be updated by the call determining the size as well, so you + need to reset it before the second call doing the actual decoding. + </p> + <p>Returns <c>0</c> on success and <c>-1</c> on failure. Failure + might be either due to invalid encoding of the term or due to + the term not being of the type <c>iodata()</c>. On failure, the + integer pointed to by the <c>index</c> argument will be updated + to refer to the sub term where the failure was detected.</p> + </desc> + </func> + + <func> <name since=""><ret>int</ret><nametext>ei_decode_list_header(const char *buf, int *index, int *arity)</nametext></name> <fsummary>Decode a list.</fsummary> <desc> @@ -315,8 +483,7 @@ typedef enum { <desc> <p>Decodes a GCC <c>long long</c> or Visual C++ <c>__int64</c> - (64-bit) integer from the binary format. This - function is missing in the VxWorks port.</p> + (64-bit) integer from the binary format.</p> </desc> </func> @@ -336,6 +503,9 @@ typedef enum { <func> <name since=""><ret>int</ret><nametext>ei_decode_pid(const char *buf, int *index, erlang_pid *p)</nametext></name> <fsummary>Decode a <c>pid</c>.</fsummary> + <type> + <v><seecref marker="#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Decodes a process identifier (pid) from the binary format.</p> </desc> @@ -344,6 +514,9 @@ typedef enum { <func> <name since=""><ret>int</ret><nametext>ei_decode_port(const char *buf, int *index, erlang_port *p)</nametext></name> <fsummary>Decode a port.</fsummary> + <type> + <v><seecref marker="#erlang_port"><c>erlang_port</c></seecref></v> + </type> <desc> <p>Decodes a port identifier from the binary format.</p> </desc> @@ -352,6 +525,9 @@ typedef enum { <func> <name since=""><ret>int</ret><nametext>ei_decode_ref(const char *buf, int *index, erlang_ref *p)</nametext></name> <fsummary>Decode a reference.</fsummary> + <type> + <v><seecref marker="#erlang_ref"><c>erlang_ref</c></seecref></v> + </type> <desc> <p>Decodes a reference from the binary format.</p> </desc> @@ -375,6 +551,9 @@ typedef enum { <func> <name since=""><ret>int</ret><nametext>ei_decode_trace(const char *buf, int *index, erlang_trace *p)</nametext></name> <fsummary>Decode a trace token.</fsummary> + <type> + <v><seecref marker="#erlang_trace"><c>erlang_trace</c></seecref></v> + </type> <desc> <p>Decodes an Erlang trace token from the binary format.</p> </desc> @@ -406,7 +585,7 @@ typedef enum { <desc> <p>Decodes a GCC <c>unsigned long long</c> or Visual C++ <c>unsigned __int64</c> (64-bit) integer from the binary - format. This function is missing in the VxWorks port.</p> + format.</p> </desc> </func> @@ -426,6 +605,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_x_encode_atom(ei_x_buff* x, const char *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_atom_len(ei_x_buff* x, const char *p, int len)</nametext></name> <fsummary>Encode an atom.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes an atom in the binary format. Parameter <c>p</c> is the name of the atom in Latin-1 encoding. Only up to @@ -441,6 +623,10 @@ typedef enum { <name since="OTP R16B"><ret>int</ret><nametext>ei_x_encode_atom_as(ei_x_buff* x, const char *p, erlang_char_encoding from_enc, erlang_char_encoding to_enc)</nametext></name> <name since="OTP R16B"><ret>int</ret><nametext>ei_x_encode_atom_len_as(ei_x_buff* x, const char *p, int len, erlang_char_encoding from_enc, erlang_char_encoding to_enc)</nametext></name> <fsummary>Encode an atom.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_char_encoding"><c>erlang_char_encoding</c></seecref></v> + </type> <desc> <p>Encodes an atom in the binary format. Parameter <c>p</c> is the name of the atom with character encoding @@ -459,6 +645,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_bignum(char *buf, int *index, mpz_t obj)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_bignum(ei_x_buff *x, mpz_t obj)</nametext></name> <fsummary>Encode an arbitrary precision integer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a GMP <c>mpz_t</c> integer to binary format. To use this function, the <c>ei</c> library must be configured and @@ -470,6 +659,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_binary(char *buf, int *index, const void *p, long len)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_binary(ei_x_buff* x, const void *p, long len)</nametext></name> <fsummary>Encode a binary.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a binary in the binary format. The data is at <c>p</c>, of <c>len</c> bytes length.</p> @@ -482,6 +674,9 @@ typedef enum { <name since="OTP 22.0"><ret>int</ret> <nametext>ei_x_encode_bitstring(ei_x_buff* x, const char *p, size_t bitoffs, size_t nbits)</nametext></name> <fsummary>Encode a bitstring.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a bit string in the binary format.</p> <p>The data is at <c>p</c>. The length of the bit string is <c>nbits</c> @@ -502,6 +697,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_boolean(char *buf, int *index, int p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_boolean(ei_x_buff* x, int p)</nametext></name> <fsummary>Encode a boolean.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a boolean value as the atom <c>true</c> if <c>p</c> is not zero, or <c>false</c> if <c>p</c> is @@ -513,6 +711,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_char(char *buf, int *index, char p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_char(ei_x_buff* x, char p)</nametext></name> <fsummary>Encode an 8-bit integer between 0-255.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a char (8-bit) as an integer between 0-255 in the binary format. For historical reasons the integer argument is of @@ -527,6 +728,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_double(char *buf, int *index, double p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_double(ei_x_buff* x, double p)</nametext></name> <fsummary>Encode a double float.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a double-precision (64-bit) floating point number in the binary format.</p> @@ -539,6 +743,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_empty_list(char* buf, int* index)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_empty_list(ei_x_buff* x)</nametext></name> <fsummary>Encode an empty list (<c>nil</c>).</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes an empty list. It is often used at the tail of a list.</p> </desc> @@ -548,6 +755,10 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_fun(char *buf, int *index, const erlang_fun *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_fun(ei_x_buff* x, const erlang_fun* fun)</nametext></name> <fsummary>Encode a fun.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_fun"><c>erlang_fun</c></seecref></v> + </type> <desc> <p>Encodes a fun in the binary format. Parameter <c>p</c> points to an <c>erlang_fun</c> structure. The @@ -561,6 +772,9 @@ typedef enum { <name since=""><ret>int</ret><nametext>ei_encode_list_header(char *buf, int *index, int arity)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_list_header(ei_x_buff* x, int arity)</nametext></name> <fsummary>Encode a list.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a list header, with a specified arity. The next <c>arity+1</c> terms are the elements @@ -598,6 +812,9 @@ ei_x_encode_empty_list(&x);</pre> <name since=""><ret>int</ret><nametext>ei_encode_long(char *buf, int *index, long p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_long(ei_x_buff* x, long p)</nametext></name> <fsummary>Encode integer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a long integer in the binary format. If the code is 64 bits, the function <c>ei_encode_long()</c> is @@ -609,10 +826,12 @@ ei_x_encode_empty_list(&x);</pre> <name since=""><ret>int</ret><nametext>ei_encode_longlong(char *buf, int *index, long long p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_longlong(ei_x_buff* x, long long p)</nametext></name> <fsummary>Encode integer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a GCC <c>long long</c> or Visual C++ - <c>__int64</c> (64-bit) integer in the binary format. - This function is missing in the VxWorks port.</p> + <c>__int64</c> (64-bit) integer in the binary format.</p> </desc> </func> @@ -620,6 +839,9 @@ ei_x_encode_empty_list(&x);</pre> <name since="OTP 17.0"><ret>int</ret><nametext>ei_encode_map_header(char *buf, int *index, int arity)</nametext></name> <name since="OTP 17.0"><ret>int</ret><nametext>ei_x_encode_map_header(ei_x_buff* x, int arity)</nametext></name> <fsummary>Encode a map.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a map header, with a specified arity. The next <c>arity*2</c> terms encoded will be the keys and values of the map @@ -641,11 +863,20 @@ ei_x_encode_string(&x, "Banana");</pre> <name since=""><ret>int</ret><nametext>ei_encode_pid(char *buf, int *index, const erlang_pid *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_pid(ei_x_buff* x, const erlang_pid *p)</nametext></name> <fsummary>Encode a pid.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Encodes an Erlang process identifier (pid) in the binary format. Parameter <c>p</c> points to an - <c>erlang_pid</c> structure (which should have been - obtained earlier with <c>ei_decode_pid()</c>).</p> + <c>erlang_pid</c> structure which should either have been + obtained earlier with + <seecref marker="#ei_decode_pid"><c>ei_decode_pid()</c></seecref>, + <seecref marker="ei_connect#ei_self"><c>ei_self()</c></seecref> or + created by + <seecref marker="ei_connect#ei_make_pid"><c>ei_make_pid()</c></seecref>. + </p> </desc> </func> @@ -653,11 +884,16 @@ ei_x_encode_string(&x, "Banana");</pre> <name since=""><ret>int</ret><nametext>ei_encode_port(char *buf, int *index, const erlang_port *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_port(ei_x_buff* x, const erlang_port *p)</nametext></name> <fsummary>Encode a port.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_port"><c>erlang_port</c></seecref></v> + </type> <desc> <p>Encodes an Erlang port in the binary format. Parameter - <c>p</c> points to a <c>erlang_port</c> - structure (which should have been obtained earlier with - <c>ei_decode_port()</c>).</p> + <c>p</c> points to an <c>erlang_port</c> structure which + should have been obtained earlier with + <seecref marker="#ei_decode_port"><c>ei_decode_port()</c></seecref>, + </p> </desc> </func> @@ -665,11 +901,17 @@ ei_x_encode_string(&x, "Banana");</pre> <name since=""><ret>int</ret><nametext>ei_encode_ref(char *buf, int *index, const erlang_ref *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_ref(ei_x_buff* x, const erlang_ref *p)</nametext></name> <fsummary>Encode a ref.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_ref"><c>erlang_ref</c></seecref></v> + </type> <desc> <p>Encodes an Erlang reference in the binary format. Parameter - <c>p</c> points to a <c>erlang_ref</c> - structure (which should have been obtained earlier with - <c>ei_decode_ref()</c>).</p> + <c>p</c> points to an <c>erlang_ref</c> + structure which either should have been obtained earlier with + <seecref marker="#ei_decode_ref"><c>ei_decode_ref()</c></seecref>, or + created by + <seecref marker="ei_connect#ei_make_ref"><c>ei_make_ref()</c></seecref>.</p> </desc> </func> @@ -679,6 +921,9 @@ ei_x_encode_string(&x, "Banana");</pre> <name since=""><ret>int</ret><nametext>ei_x_encode_string(ei_x_buff* x, const char *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_string_len(ei_x_buff* x, const char* s, int len)</nametext></name> <fsummary>Encode a string.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a string in the binary format. (A string in Erlang is a list, but is encoded as a character array in the binary @@ -690,11 +935,16 @@ ei_x_encode_string(&x, "Banana");</pre> <name since=""><ret>int</ret><nametext>ei_encode_trace(char *buf, int *index, const erlang_trace *p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_trace(ei_x_buff* x, const erlang_trace *p)</nametext></name> <fsummary>Encode a trace token.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_trace"><c>erlang_trace</c></seecref></v> + </type> <desc> <p>Encodes an Erlang trace token in the binary format. Parameter <c>p</c> points to a - <c>erlang_trace</c> structure (which should have been - obtained earlier with <c>ei_decode_trace()</c>).</p> + <c>erlang_trace</c> structure which should have been + obtained earlier with + <seecref marker="#ei_decode_trace"><c>ei_decode_trace()</c></seecref>.</p> </desc> </func> @@ -702,6 +952,9 @@ ei_x_encode_string(&x, "Banana");</pre> <name since=""><ret>int</ret><nametext>ei_encode_tuple_header(char *buf, int *index, int arity)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_tuple_header(ei_x_buff* x, int arity)</nametext></name> <fsummary>Encode a tuple.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a tuple header, with a specified arity. The next <c>arity</c> terms encoded will be the @@ -721,6 +974,9 @@ ei_encode_tuple_header(buf, &i, 0);</pre> <name since=""><ret>int</ret><nametext>ei_encode_ulong(char *buf, int *index, unsigned long p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_ulong(ei_x_buff* x, unsigned long p)</nametext></name> <fsummary>Encode unsigned integer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes an unsigned long integer in the binary format. If the code is 64 bits, the function <c>ei_encode_ulong()</c> is @@ -732,10 +988,13 @@ ei_encode_tuple_header(buf, &i, 0);</pre> <name since=""><ret>int</ret><nametext>ei_encode_ulonglong(char *buf, int *index, unsigned long long p)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_ulonglong(ei_x_buff* x, unsigned long long p)</nametext></name> <fsummary>Encode unsigned integer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a GCC <c>unsigned long long</c> or Visual C++ <c>unsigned __int64</c> (64-bit) integer in the binary - format. This function is missing in the VxWorks port.</p> + format.</p> </desc> </func> @@ -743,6 +1002,9 @@ ei_encode_tuple_header(buf, &i, 0);</pre> <name since=""><ret>int</ret><nametext>ei_encode_version(char *buf, int *index)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_encode_version(ei_x_buff* x)</nametext></name> <fsummary>Encode version.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Encodes a version magic number for the binary format. Must be the first token in a binary term.</p> @@ -760,6 +1022,105 @@ ei_encode_tuple_header(buf, &i, 0);</pre> the number of bytes. For lists, tuples and maps, <c>*size</c> is the arity of the object. For other types, <c>*size</c> is 0. In all cases, <c>index</c> is left unchanged.</p> + <p>Currently <c>*type</c> is one of:</p> + <taglist> + <tag>ERL_ATOM_EXT</tag> + <item><p> + Decode using either + <seecref marker="#ei_decode_atom"><c>ei_decode_atom()</c></seecref>, + <seecref marker="#ei_decode_atom_as"><c>ei_decode_atom_as()</c></seecref>, + or + <seecref marker="#ei_decode_boolean"><c>ei_decode_boolean()</c></seecref>. + </p></item> + + <tag>ERL_BINARY_EXT</tag> + <item><p> + Decode using either + <seecref marker="#ei_decode_binary"><c>ei_decode_binary()</c></seecref>, + <seecref marker="#ei_decode_bitstring"><c>ei_decode_bitstring()</c></seecref>, + or + <seecref marker="#ei_decode_iodata"><c>ei_decode_iodata()</c></seecref>. + </p></item> + + <tag>ERL_BIT_BINARY_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_bitstring"><c>ei_decode_bitstring()</c></seecref>. + </p></item> + + <tag>ERL_FLOAT_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_double"><c>ei_decode_double()</c></seecref>. + </p></item> + + <tag>ERL_NEW_FUN_EXT<br/>ERL_FUN_EXT<br/>ERL_EXPORT_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_fun"><c>ei_decode_fun()</c></seecref>. + </p></item> + + <tag>ERL_SMALL_INTEGER_EXT<br/>ERL_INTEGER_EXT<br/>ERL_SMALL_BIG_EXT<br/>ERL_LARGE_BIG_EXT</tag> + <item><p> + Decode using either + <seecref marker="#ei_decode_char"><c>ei_decode_char()</c></seecref>, + <seecref marker="#ei_decode_long"><c>ei_decode_long()</c></seecref>, + <seecref marker="#ei_decode_longlong"><c>ei_decode_longlong()</c></seecref>, + <seecref marker="#ei_decode_ulong"><c>ei_decode_ulong()</c></seecref>, + <seecref marker="#ei_decode_ulonglong"><c>ei_decode_ulonglong()</c></seecref>, + or + <seecref marker="#ei_decode_bignum"><c>ei_decode_bignum()</c></seecref>. + </p></item> + + <tag>ERL_LIST_EXT<br/>ERL_NIL_EXT</tag> + <item><p> + Decode using either + <seecref marker="#ei_decode_list_header"><c>ei_decode_list_header()</c></seecref>, + or + <seecref marker="#ei_decode_iodata"><c>ei_decode_iodata()</c></seecref>. + </p></item> + + <tag>ERL_STRING_EXT</tag> + <item><p> + Decode using either + <seecref marker="#ei_decode_string"><c>ei_decode_string()</c></seecref>, + or + <seecref marker="#ei_decode_iodata"><c>ei_decode_iodata()</c></seecref>. + </p></item> + + <tag>ERL_MAP_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_map_header"><c>ei_decode_map_header()</c></seecref>. + </p></item> + + <tag>ERL_PID_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_pid"><c>ei_decode_pid()</c></seecref>. + </p></item> + + <tag>ERL_PORT_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_port"><c>ei_decode_port()</c></seecref>. + </p></item> + + <tag>ERL_NEW_REFERENCE_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_ref"><c>ei_decode_ref()</c></seecref>. + </p></item> + + <tag>ERL_SMALL_TUPLE_EXT<br/>ERL_LARGE_TUPLE_EXT</tag> + <item><p> + Decode using + <seecref marker="#ei_decode_tuple_header"><c>ei_decode_tuple_header()</c></seecref>. + </p></item> + </taglist> + <p>Instead of decoding a term you can also skipped past it if you are + not interested in the data by usage of + <seecref marker="#ei_skip_term"><c>ei_skip_term()</c></seecref>.</p> </desc> </func> @@ -801,11 +1162,8 @@ ei_encode_tuple_header(buf, &i, 0);</pre> </func> <func> - <name since=""><ret>void</ret><nametext>ei_set_compat_rel(release_number)</nametext></name> + <name since=""><ret>void</ret><nametext>ei_set_compat_rel(unsigned release_number)</nametext></name> <fsummary>Set the ei library in compatibility mode.</fsummary> - <type> - <v>unsigned release_number;</v> - </type> <desc> <marker id="ei_set_compat_rel"></marker> <p>In general, the <c>ei</c> library is guaranteed @@ -878,6 +1236,9 @@ ei_encode_tuple_header(buf, &i, 0);</pre> <name since=""><ret>int</ret><nametext>ei_x_append(ei_x_buff* x, const ei_x_buff* x2)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_append_buf(ei_x_buff* x, const char* buf, int len)</nametext></name> <fsummary>Append a buffer at the end.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> <p>Appends data at the end of buffer <c>x</c>.</p> </desc> @@ -887,6 +1248,10 @@ ei_encode_tuple_header(buf, &i, 0);</pre> <name since=""><ret>int</ret><nametext>ei_x_format(ei_x_buff* x, const char* fmt, ...)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_format_wo_ver(ei_x_buff* x, const char *fmt, ... )</nametext></name> <fsummary>Format a term from a format string and parameters.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Formats a term, given as a string, to a buffer. Works like a sprintf for Erlang terms. @@ -915,9 +1280,13 @@ encodes the tuple {numbers,12,3.14159}</pre> <func> <name since=""><ret>int</ret><nametext>ei_x_free(ei_x_buff* x)</nametext></name> <fsummary>Free a buffer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> - <p>Frees an <c>ei_x_buff</c> buffer. - The memory used by the buffer is returned to the OS.</p> + <p>Deallocates the dynamically allocated content of the buffer + referred by <c>x</c>. After deallocation, the <c>buff</c> field + is set to <c>NULL</c>.</p> </desc> </func> @@ -925,10 +1294,14 @@ encodes the tuple {numbers,12,3.14159}</pre> <name since=""><ret>int</ret><nametext>ei_x_new(ei_x_buff* x)</nametext></name> <name since=""><ret>int</ret><nametext>ei_x_new_with_version(ei_x_buff* x)</nametext></name> <fsummary>Allocate a new buffer.</fsummary> + <type> + <v><seecref marker="#ei_x_buff"><c>ei_x_buff</c></seecref></v> + </type> <desc> - <p>Allocates a new <c>ei_x_buff</c> buffer. The - fields of the structure pointed to by parameter <c>x</c> - is filled in, and a default buffer is allocated. + <p>Initialize the dynamically realizable buffer referred to + by <c>x</c>. The fields of the structure pointed to by + parameter <c>x</c> is filled in, and a default buffer is + allocated. <c>ei_x_new_with_version()</c> also puts an initial version byte, which is used in the binary format (so that <c>ei_x_encode_version()</c> will not be needed.)</p> diff --git a/lib/erl_interface/doc/src/ei_connect.xml b/lib/erl_interface/doc/src/ei_connect.xml index f991165df7..c5ef9440c5 100644 --- a/lib/erl_interface/doc/src/ei_connect.xml +++ b/lib/erl_interface/doc/src/ei_connect.xml @@ -34,9 +34,6 @@ <lib>ei_connect</lib> <libsummary>Communicate with distributed Erlang.</libsummary> <description> - <note><p>The support for VxWorks is deprecated as of OTP 22, and - will be removed in OTP 23.</p></note> - <p>This module enables C-programs to communicate with Erlang nodes, using the Erlang distribution over TCP/IP.</p> @@ -112,27 +109,10 @@ only supports IPv4. That is, at this time <c>addr</c> always points to a <c>struct sockaddr_in</c> structure.</p> - <p>The <c>ei_socket_callbacks</c> structure may be enlarged in - the future. All fields not set, <em>needs</em> to be zeroed out.</p> - - <marker id="ei_socket_callbacks"/> - <code type="none"><![CDATA[ -typedef struct { - int flags; - int (*socket)(void **ctx, void *setup_ctx); - int (*close)(void *ctx); - int (*listen)(void *ctx, void *addr, int *len, int backlog); - int (*accept)(void **ctx, void *addr, int *len, unsigned tmo); - int (*connect)(void *ctx, void *addr, int len, unsigned tmo); - int (*writev)(void *ctx, const void *iov, int iovcnt, ssize_t *len, unsigned tmo); - int (*write)(void *ctx, const char *buf, ssize_t *len, unsigned tmo); - int (*read)(void *ctx, char *buf, ssize_t *len, unsigned tmo); - int (*handshake_packet_header_size)(void *ctx, int *sz); - int (*connect_handshake_complete)(void *ctx); - int (*accept_handshake_complete)(void *ctx); - int (*get_fd)(void *ctx, int *fd); -} ei_socket_callbacks; - ]]></code> + <p><marker id="ei_socket_callbacks_fields"/>The + <seecref marker="#ei_socket_callbacks"><c>ei_socket_callbacks</c></seecref> + structure may be enlarged in the future. All fields not set, <em>needs</em> + to be zeroed out. Currently the following fields exist:</p> <taglist> @@ -355,6 +335,79 @@ typedef struct { </item> </taglist> </section> + <section> + <title>Data Types</title> + <taglist> + <tag><marker id="ei_cnode"/><c>ei_cnode</c></tag> + <item><p> + Opaque data type representing a C-node. A <c>ei_cnode</c> + structure is initialized by calling + <seecref marker="#ei_connect_init"><c>ei_connect_init()</c></seecref> + or friends. + </p></item> + + <tag><marker id="ei_socket_callbacks"/><c>ei_socket_callbacks</c></tag> + <item><code type="none"> +typedef struct { + int flags; + int (*socket)(void **ctx, void *setup_ctx); + int (*close)(void *ctx); + int (*listen)(void *ctx, void *addr, int *len, int backlog); + int (*accept)(void **ctx, void *addr, int *len, unsigned tmo); + int (*connect)(void *ctx, void *addr, int len, unsigned tmo); + int (*writev)(void *ctx, const void *iov, int iovcnt, ssize_t *len, unsigned tmo); + int (*write)(void *ctx, const char *buf, ssize_t *len, unsigned tmo); + int (*read)(void *ctx, char *buf, ssize_t *len, unsigned tmo); + int (*handshake_packet_header_size)(void *ctx, int *sz); + int (*connect_handshake_complete)(void *ctx); + int (*accept_handshake_complete)(void *ctx); + int (*get_fd)(void *ctx, int *fd); +} ei_socket_callbacks;</code> + <p> + Callbacks functions for a <seecref marker="#ussi"><i>User + Supplied Socket Implementation</i></seecref>. + <seecref marker="#ei_socket_callbacks_fields">Documentation + of each field</seecref> can be found in the + <i>User Supplied Socket Implementation</i> section + above. + </p> + + </item> + + <tag><marker id="ErlConnect"/><c>ErlConnect</c></tag> + <item><code type="none"> +typedef struct { + char ipadr[4]; /* Ip v4 address in network byte order */ + char nodename[MAXNODELEN]; +} ErlConnect;</code> + <p>IP v4 address and nodename.</p> + </item> + + <tag><marker id="Erl_IpAddr"/><c>Erl_IpAddr</c></tag> + <item><code type="none"> +typedef struct { + unsigned s_addr; /* Ip v4 address in network byte order */ +} Erl_IpAddr;</code> + <p>IP v4 address.</p> + </item> + + <tag><marker id="erlang_msg"/><c>erlang_msg</c></tag> + <item> + <code type="none"> +typedef struct { + long msgtype; + erlang_pid from; + erlang_pid to; + char toname[MAXATOMLEN+1]; + char cookie[MAXATOMLEN+1]; + erlang_trace token; +} erlang_msg;</code> + <p>Information about a message received via + <seecref marker="#ei_receive_msg"><c>ei_receive_msg()</c></seecref> + or friends.</p> + </item> + </taglist> + </section> <funcs> <func> <name since=""><ret>struct hostent *</ret><nametext>ei_gethostbyaddr(const char *addr, int len, int type)</nametext></name> @@ -371,6 +424,10 @@ typedef struct { <func> <name since=""><ret>int</ret><nametext>ei_accept(ei_cnode *ec, int listensock, ErlConnect *conp)</nametext></name> <fsummary>Accept a connection from another node.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="#ErlConnect"><c>ErlConnect</c></seecref></v> + </type> <desc> <p>Used by a server process to accept a connection from a client process.</p> @@ -384,13 +441,8 @@ typedef struct { </item> <item> <p><c>conp</c> is a pointer to an - <c>ErlConnect</c> struct, described as follows:</p> - <code type="none"><![CDATA[ -typedef struct { - char ipadr[4]; - char nodename[MAXNODELEN]; -} ErlConnect; - ]]></code> + <seecref marker="#ErlConnect"><c>ErlConnect</c></seecref> + struct.</p> </item> </list> <p>On success, <c>conp</c> is filled in with the address and @@ -404,6 +456,10 @@ typedef struct { <name since=""><ret>int</ret><nametext>ei_accept_tmo(ei_cnode *ec, int listensock, ErlConnect *conp, unsigned timeout_ms)</nametext></name> <fsummary>Accept a connection from another node with optional time-out.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="#ErlConnect"><c>ErlConnect</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_accept</c> with an optional time-out argument, @@ -422,9 +478,13 @@ typedef struct { <func> <name since=""><ret>int</ret><nametext>ei_connect(ei_cnode* ec, char *nodename)</nametext></name> <name since=""><ret>int</ret><nametext>ei_xconnect(ei_cnode* ec, Erl_IpAddr adr, char *alivename)</nametext></name> - <name since="OTP @OTP-16251@"><ret>int</ret><nametext>ei_connect_host_port(ei_cnode* ec, char *hostname, int port)</nametext></name> - <name since="OTP @OTP-16251@"><ret>int</ret><nametext>ei_xconnect_host_port(ei_cnode* ec, Erl_IpAddr adr, int port)</nametext></name> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_connect_host_port(ei_cnode* ec, char *hostname, int port)</nametext></name> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_xconnect_host_port(ei_cnode* ec, Erl_IpAddr adr, int port)</nametext></name> <fsummary>Establish a connection to an Erlang node.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="#Erl_IpAddr"><c>Erl_IpAddr</c></seecref></v> + </type> <desc> <p>Sets up a connection to an Erlang node.</p> <p><c>ei_xconnect()</c> requires the IP address of the @@ -486,6 +546,11 @@ fd = ei_xconnect(&ec, &addr, ALIVE); <name since=""><ret>int</ret><nametext>ei_connect_xinit(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation)</nametext></name> <name since="OTP 21.3"><ret>int</ret><nametext>ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, const char *thisalivename, const char *thisnodename, Erl_IpAddr thisipaddr, const char *cookie, short creation, ei_socket_callbacks *cbs, int cbs_sz, void *setup_context)</nametext></name> <fsummary>Initialize for a connection.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="#Erl_IpAddr"><c>Erl_IpAddr</c></seecref></v> + <v><seecref marker="#ei_socket_callbacks"><c>ei_socket_callbacks</c></seecref></v> + </type> <desc> <p>Initializes the <c>ec</c> structure, to identify the node name and cookie of the server. One of them @@ -583,10 +648,14 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <func> <name since=""><ret>int</ret><nametext>ei_connect_tmo(ei_cnode* ec, char *nodename, unsigned timeout_ms)</nametext></name> <name since=""><ret>int</ret><nametext>ei_xconnect_tmo(ei_cnode* ec, Erl_IpAddr adr, char *alivename, unsigned timeout_ms)</nametext></name> - <name since="OTP @OTP-16251@"><ret>int</ret><nametext>ei_connect_host_port_tmo(ei_cnode* ec, char *hostname, int port, unsigned ms)</nametext></name> - <name since="OTP @OTP-16251@"><ret>int</ret><nametext>ei_xconnect_host_port_tmo(ei_cnode* ec, Erl_IpAddr adr, int port, unsigned ms)</nametext></name> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_connect_host_port_tmo(ei_cnode* ec, char *hostname, int port, unsigned ms)</nametext></name> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_xconnect_host_port_tmo(ei_cnode* ec, Erl_IpAddr adr, int port, unsigned ms)</nametext></name> <fsummary>Establish a connection to an Erlang node with optional time-out.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="#Erl_IpAddr"><c>Erl_IpAddr</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_connect</c>, <c>ei_xconnect</c>, <c>ei_connect_host_port</c> and @@ -613,6 +682,10 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <name since="OTP 21.3"><ret>int</ret><nametext>ei_listen(ei_cnode *ec, int *port, int backlog)</nametext></name> <name since="OTP 21.3"><ret>int</ret><nametext>ei_xlisten(ei_cnode *ec, Erl_IpAddr adr, int *port, int backlog)</nametext></name> <fsummary>Create a listen socket.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="#Erl_IpAddr"><c>Erl_IpAddr</c></seecref></v> + </type> <desc> <p>Used by a server process to setup a listen socket which later can be used for accepting connections from client processes. @@ -649,8 +722,60 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { </func> <func> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_make_pid(ei_cnode *ec, erlang_pid *pid)</nametext></name> + <fsummary>Create a new process identifier</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> + <desc> + <p> + Creates a new process identifier in the argument <c>pid</c>. This process identifier + refers to a conseptual process residing on the C-node identified by the argument + <c>ec</c>. On success <c>0</c> is returned. On failure <c>ERL_ERROR</c> is + returned and <c>erl_errno</c> is set. + </p> + <p> + The C-node identified by <c>ec</c> must have been initialized and must have + received a name prior to the call to <c>ei_make_pid()</c>. Initialization + of the C-node is done by a call to + <seecref marker="#ei_connect_init"><c>ei_connect_init()</c></seecref> + or friends. If the name is dynamically assigned from the peer node, the + C-node also has to be connected. + </p> + </desc> + </func> + + <func> + <name since="OTP 23.0"><ret>int</ret><nametext>ei_make_ref(ei_cnode *ec, erlang_ref *ref)</nametext></name> + <fsummary>Create a new reference</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="ei#erlang_ref"><c>erlang_ref</c></seecref></v> + </type> + <desc> + <p> + Creates a new reference in the argument <c>ref</c>. This reference originates + from the C-node identified by the argument <c>ec</c>. On success <c>0</c> is + returned. On failure <c>ERL_ERROR</c> is returned and <c>erl_errno</c> is set. + </p> + <p> + The C-node identified by <c>ec</c> must have been initialized and must have + received a name prior to the call to <c>ei_make_ref()</c>. Initialization + of the C-node is done by a call to + <seecref marker="#ei_connect_init"><c>ei_connect_init()</c></seecref> + or friends. If the name is dynamically assigned from the peer node, the + C-node also has to be connected. + </p> + </desc> + </func> + + <func> <name since=""><ret>int</ret><nametext>ei_publish(ei_cnode *ec, int port)</nametext></name> <fsummary>Publish a node name.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Used by a server process to register with the local name server EPMD, thereby allowing @@ -688,6 +813,9 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <func> <name since=""><ret>int</ret><nametext>ei_publish_tmo(ei_cnode *ec, int port, unsigned timeout_ms)</nametext></name> <fsummary>Publish a node name with optional time-out.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_publish</c> with an optional time-out argument, @@ -739,6 +867,9 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <func> <name since=""><ret>int</ret><nametext>ei_receive_encoded(int fd, char **mbufp, int *bufsz, erlang_msg *msg, int *msglen)</nametext></name> <fsummary>Obsolete function for receiving a message.</fsummary> + <type> + <v><seecref marker="#erlang_msg"><c>erlang_msg</c></seecref></v> + </type> <desc> <p>This function is retained for compatibility with code generated by the interface compiler and with code following @@ -770,6 +901,9 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <name since=""><ret>int</ret><nametext>ei_receive_encoded_tmo(int fd, char **mbufp, int *bufsz, erlang_msg *msg, int *msglen, unsigned timeout_ms)</nametext></name> <fsummary>Obsolete function for receiving a message with time-out. </fsummary> + <type> + <v><seecref marker="#erlang_msg"><c>erlang_msg</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_receive_encoded</c> with an optional time-out argument, @@ -781,6 +915,10 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <name since=""><ret>int</ret><nametext>ei_receive_msg(int fd, erlang_msg* msg, ei_x_buff* x)</nametext></name> <name since=""><ret>int</ret><nametext>ei_xreceive_msg(int fd, erlang_msg* msg, ei_x_buff* x)</nametext></name> <fsummary>Receive a message.</fsummary> + <type> + <v><seecref marker="ei#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_msg"><c>erlang_msg</c></seecref></v> + </type> <desc> <p>Receives a message to the buffer in <c>x</c>. <c>ei_xreceive_msg</c> allows the buffer in @@ -797,18 +935,8 @@ if (ei_connect_init(&ec, "madonna", "cookie...", n++) < 0) { <c>ei_x_new</c>.</item> </list> <p>On success, the functions return <c>ERL_MSG</c> and the - <c>msg</c> struct is initialized. - <c>erlang_msg</c> is defined as follows:</p> - <code type="none"><![CDATA[ -typedef struct { - long msgtype; - erlang_pid from; - erlang_pid to; - char toname[MAXATOMLEN+1]; - char cookie[MAXATOMLEN+1]; - erlang_trace token; -} erlang_msg; - ]]></code> + <seecref marker="#erlang_msg"><c>msg</c></seecref> struct + is initialized.</p> <p><c>msgtype</c> identifies the type of message, and is one of the following:</p> <taglist> @@ -846,6 +974,10 @@ typedef struct { <name since=""><ret>int</ret><nametext>ei_receive_msg_tmo(int fd, erlang_msg* msg, ei_x_buff* x, unsigned imeout_ms)</nametext></name> <name since=""><ret>int</ret><nametext>ei_xreceive_msg_tmo(int fd, erlang_msg* msg, ei_x_buff* x, unsigned timeout_ms)</nametext></name> <fsummary>Receive a message with optional time-out.</fsummary> + <type> + <v><seecref marker="ei#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_msg"><c>erlang_msg</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_receive_msg</c> and <c>ei_xreceive_msg</c> with an optional time-out argument, @@ -866,6 +998,9 @@ typedef struct { <func> <name since=""><ret>int</ret><nametext>ei_reg_send(ei_cnode* ec, int fd, char* server_name, char* buf, int len)</nametext></name> <fsummary>Send a message to a registered name.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Sends an Erlang term to a registered process.</p> <list type="bulleted"> @@ -899,6 +1034,9 @@ if (ei_reg_send(&ec, fd, x.buff, x.index) < 0) <name since=""><ret>int</ret><nametext>ei_reg_send_tmo(ei_cnode* ec, int fd, char* server_name, char* buf, int len, unsigned timeout_ms)</nametext></name> <fsummary>Send a message to a registered name with optional time-out </fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_reg_send</c> with an optional time-out argument, @@ -911,6 +1049,11 @@ if (ei_reg_send(&ec, fd, x.buff, x.index) < 0) <name since=""><ret>int</ret><nametext>ei_rpc_to(ei_cnode *ec, int fd, char *mod, char *fun, const char *argbuf, int argbuflen)</nametext></name> <name since=""><ret>int</ret><nametext>ei_rpc_from(ei_cnode *ec, int fd, int timeout, erlang_msg *msg, ei_x_buff *x)</nametext></name> <fsummary>Remote Procedure Call from C to Erlang.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="ei#ei_x_buff"><c>ei_x_buff</c></seecref></v> + <v><seecref marker="#erlang_msg"><c>erlang_msg</c></seecref></v> + </type> <desc> <p>Supports calling Erlang functions on remote nodes. <c>ei_rpc_to()</c> sends an RPC request to a remote node @@ -1010,19 +1153,40 @@ if (ei_decode_version(result.buff, &index) < 0 <func> <name since=""><ret>erlang_pid *</ret><nametext>ei_self(ei_cnode *ec)</nametext></name> <fsummary>Retrieve the pid of the C-node.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> - <p>Retrieves the pid of the C-node. Every C-node + <p>Retrieves a generic pid of the C-node. Every C-node has a (pseudo) pid used in <c>ei_send_reg</c>, - <c>ei_rpc</c>, + <c>ei_rpc()</c>, and others. This is contained in a field in the <c>ec</c> - structure. It will be safe for a long time to fetch this - field directly from the <c>ei_cnode</c> structure.</p> + structure. Do <em>not</em> modify this structure. + </p> + <p> + On success a pointer to the process identifier is returned. + On failure <c>NULL</c> is returned and <c>erl_errno</c> is + set. + </p> + <p> + The C-node identified by <c>ec</c> must have been initialized + and must have received a name prior to the call to <c>ei_self()</c>. + Initialization of the C-node is done by a call to + <seecref marker="#ei_connect_init"><c>ei_connect_init()</c></seecref> + or friends. If the name is dynamically assigned from the peer node, the + C-node also has to be connected. + </p> + </desc> </func> <func> <name since=""><ret>int</ret><nametext>ei_send(int fd, erlang_pid* to, char* buf, int len)</nametext></name> <fsummary>Send a message.</fsummary> + <type> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Sends an Erlang term to a process.</p> <list type="bulleted"> @@ -1044,6 +1208,9 @@ if (ei_decode_version(result.buff, &index) < 0 <func> <name since=""><ret>int</ret><nametext>ei_send_encoded(int fd, erlang_pid* to, char* buf, int len)</nametext></name> <fsummary>Obsolete function to send a message.</fsummary> + <type> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Works exactly as <c>ei_send</c>, the alternative name is retained for backward compatibility. The function will <em>not</em> be @@ -1055,6 +1222,9 @@ if (ei_decode_version(result.buff, &index) < 0 <name since=""><ret>int</ret><nametext>ei_send_encoded_tmo(int fd, erlang_pid* to, char* buf, int len, unsigned timeout_ms)</nametext></name> <fsummary>Obsolete function to send a message with optional time-out. </fsummary> + <type> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_send_encoded</c> with an optional time-out argument, @@ -1066,6 +1236,9 @@ if (ei_decode_version(result.buff, &index) < 0 <name since=""><ret>int</ret><nametext>ei_send_reg_encoded(int fd, const erlang_pid *from, const char *to, const char *buf, int len)</nametext></name> <fsummary>Obsolete function to send a message to a registered name. </fsummary> + <type> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>This function is retained for compatibility with code generated by the interface compiler and with code following @@ -1076,17 +1249,8 @@ if (ei_decode_version(result.buff, &index) < 0 <c>erlang_pid</c>, which is to be the process identifier of the sending process (in the Erlang distribution protocol).</p> - <p>A suitable <c>erlang_pid</c> can be constructed from the - <c>ei_cnode</c> structure by the following example - code:</p> - <code type="none"><![CDATA[ -ei_cnode ec; -erlang_pid *self; -int fd; /* the connection fd */ -... -self = ei_self(&ec); -self->num = fd; - ]]></code> + <p>A suitable <c>erlang_pid</c> can be retrieved from the + <c>ei_cnode</c> structure by calling <c>ei_self(cnode_pointer)</c>.</p> </desc> </func> @@ -1094,6 +1258,9 @@ self->num = fd; <name since=""><ret>int</ret><nametext>ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, const char *to, const char *buf, int len)</nametext></name> <fsummary>Obsolete function to send a message to a registered name with time-out.</fsummary> + <type> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_send_reg_encoded</c> with an optional time-out argument, @@ -1104,6 +1271,9 @@ self->num = fd; <func> <name since=""><ret>int</ret><nametext>ei_send_tmo(int fd, erlang_pid* to, char* buf, int len, unsigned timeout_ms)</nametext></name> <fsummary>Send a message with optional time-out.</fsummary> + <type> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_send</c> with an optional time-out argument, @@ -1116,6 +1286,9 @@ self->num = fd; <name since=""><ret>const char *</ret><nametext>ei_thishostname(ei_cnode *ec)</nametext></name> <name since=""><ret>const char *</ret><nametext>ei_thisalivename(ei_cnode *ec)</nametext></name> <fsummary>Retrieve some values.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Can be used to retrieve information about the C-node. These values are initially set with @@ -1131,6 +1304,9 @@ self->num = fd; <func> <name since=""><ret>int</ret><nametext>ei_unpublish(ei_cnode *ec)</nametext></name> <fsummary>Forcefully unpublish a node name.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Can be called by a process to unregister a specified node from EPMD on the local host. This is, however, usually @@ -1154,6 +1330,9 @@ self->num = fd; <func> <name since=""><ret>int</ret><nametext>ei_unpublish_tmo(ei_cnode *ec, unsigned timeout_ms)</nametext></name> <fsummary>Unpublish a node name with optional time-out.</fsummary> + <type> + <v><seecref marker="#ei_cnode"><c>ei_cnode</c></seecref></v> + </type> <desc> <p>Equivalent to <c>ei_unpublish</c> with an optional time-out argument, diff --git a/lib/erl_interface/doc/src/ei_global.xml b/lib/erl_interface/doc/src/ei_global.xml index ecca1aad68..148068490e 100644 --- a/lib/erl_interface/doc/src/ei_global.xml +++ b/lib/erl_interface/doc/src/ei_global.xml @@ -35,9 +35,6 @@ <lib>ei_global</lib> <libsummary>Access globally registered names.</libsummary> <description> - <note><p>The support for VxWorks is deprecated as of OTP 22, and - will be removed in OTP 23.</p></note> - <p>This module provides support for registering, looking up, and unregistering names in the <c>global</c> module. For more information, see @@ -54,14 +51,14 @@ <name since=""><ret>char **</ret><nametext>ei_global_names(ec,fd,count)</nametext></name> <fsummary>Obtain list of global names.</fsummary> <type> - <v>ei_cnode *ec;</v> + <v><seecref marker="ei_connect#ei_cnode"><c>ei_cnode</c></seecref> *ec;</v> <v>int fd;</v> <v>int *count;</v> </type> <desc> <p>Retrieves a list of all known global names.</p> <list type="bulleted"> - <item><c>ec</c> is the ei_cnode representing the current cnode.</item> + <item><c>ec</c> is the <c>ei_cnode</c> representing the current cnode.</item> <item><c>fd</c> is an open descriptor to an Erlang connection.</item> <item><c>count</c> is the address of an integer, or @@ -89,7 +86,7 @@ <type> <v>int fd;</v> <v>const char *name;</v> - <v>erlang_pid *pid;</v> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref> *pid;</v> </type> <desc> <p>Registers a name in <c>global</c>.</p> @@ -111,14 +108,14 @@ <name since=""><ret>int</ret><nametext>ei_global_unregister(ec,fd,name)</nametext></name> <fsummary>Unregister a name from global.</fsummary> <type> - <v>ei_cnode *ec;</v> + <v><seecref marker="ei_connect#ei_cnode"><c>ei_cnode</c></seecref> *ec;</v> <v>int fd;</v> <v>const char *name;</v> </type> <desc> <p>Unregisters a name from <c>global</c>.</p> <list type="bulleted"> - <item><c>ec</c> is the ei_cnode representing the current cnode.</item> + <item><c>ec</c> is the <c>ei_cnode</c> representing the current cnode.</item> <item><c>fd</c> is an open descriptor to an Erlang connection.</item> <item><c>name</c> is the name to unregister from @@ -132,16 +129,16 @@ <name since=""><ret>int</ret><nametext>ei_global_whereis(ec,fd,name,pid,node)</nametext></name> <fsummary>Look up a name in global.</fsummary> <type> - <v>ei_cnode *ec;</v> + <v><seecref marker="ei_connect#ei_cnode"><c>ei_cnode</c></seecref> *ec;</v> <v>int fd;</v> <v>const char *name;</v> - <v>erlang_pid* pid;</v> + <v><seecref marker="ei#erlang_pid"><c>erlang_pid</c></seecref> *pid;</v> <v>char *node;</v> </type> <desc> <p>Looks up a name in <c>global</c>.</p> <list type="bulleted"> - <item><c>ec</c> is the ei_cnode representing the current cnode.</item> + <item><c>ec</c> is the <c>ei_cnode</c> representing the current cnode.</item> <item><c>fd</c> is an open descriptor to an Erlang connection.</item> <item><c>name</c> is the name that is to be looked up in diff --git a/lib/erl_interface/doc/src/ei_users_guide.xml b/lib/erl_interface/doc/src/ei_users_guide.xml index 25952c2025..5dfaf556da 100644 --- a/lib/erl_interface/doc/src/ei_users_guide.xml +++ b/lib/erl_interface/doc/src/ei_users_guide.xml @@ -34,12 +34,6 @@ </header> <section> - <title>Deprecation and Removal</title> - <note><p>The support for VxWorks is deprecated as of OTP 22, and - will be removed in OTP 23.</p></note> - </section> - - <section> <title>Introduction</title> <p>The <c>Erl_Interface</c> library contains functions that help you integrate programs written in C and Erlang. The functions in @@ -536,6 +530,12 @@ ei_global_unregister(ec,fd,servicename); ]]></code> <section> <title>Using the Registry</title> + + <note><p>This functionality is deprecated as of OTP 23, and will be + removed in OTP 24. Reasonably new <c>gcc</c> compilers will issue + deprecation warnings. In order to disable these warnings, define the + macro <c>EI_NO_DEPR_WARN</c>.</p></note> + <p>This section describes the use of the registry, a simple mechanism for storing key-value pairs in a C-node, as well as backing them up or restoring them from an <c>Mnesia</c> table on an Erlang node. For more diff --git a/lib/erl_interface/doc/src/erl_call_cmd.xml b/lib/erl_interface/doc/src/erl_call_cmd.xml index 4a77aa954e..04b5ec74bf 100644 --- a/lib/erl_interface/doc/src/erl_call_cmd.xml +++ b/lib/erl_interface/doc/src/erl_call_cmd.xml @@ -183,6 +183,13 @@ specified, an Erlang node is started (if necessary) with <c>erl -sname</c>.</p> </item> + <tag><c>-timeout Seconds</c></tag> + <item> + <p>(<em>Optional.</em>) Aborts the <c>erl_call</c> process after + the timeout expires. Note that this does not abort commands that + have already been started with <c>-a</c>, <c>-e</c>, or similar. + </p> + </item> <tag><c>-v</c></tag> <item> <p>(<em>Optional.</em>) Prints a lot of <c>verbose</c> diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index ba5f501e85..37eb379d95 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -31,6 +31,178 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 4.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix various compiler warnings on 64-bit Windows.</p> + <p> + Own Id: OTP-15800</p> + </item> + <item> + <p><c>erl_call</c> will now work properly on systems that + cannot resolve their own hostname.</p> + <p> + Own Id: OTP-16604</p> + </item> + <item> + <p>Various bug fixes:</p> <list> <item>Internal error + checking in various functions.</item> <item><seecref + marker="ei_connect#ei_rpc"><c>ei_rpc()</c></seecref> + accepted any 2-tuple message as an rpc response.</item> + <item><seecref + marker="ei#ei_decode_ref"><c>ei_decode_ref()</c></seecref> + now refuse to write outside of allocated memory in case a + huge reference is decoded.</item> <item><seecref + marker="ei#ei_decode_ei_term"><c>ei_decode_ei_term()</c></seecref> + now reports the same term types as <seecref + marker="ei#ei_get_type"><c>ei_get_type()</c></seecref>.</item> + </list> + <p> + Own Id: OTP-16623</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>A client node can receive its node name dynamically + from the node that it first connects to. This featured + can by used by</p> <list> <item><p>starting with <c>erl + -sname undefined</c></p></item> <item><p>erl_interface + functions <c>ei_connect_init</c> and friends</p></item> + <item><p><c>erl_call -R</c></p></item> </list> + <p> + Own Id: OTP-13812</p> + </item> + <item> + <p> + Increased size of node incarnation numbers (aka + "creation"), from 2 bits to 32 bits. This will reduce the + risk of pids/ports/refs, from different node incarnation + with the same name, being mixed up.</p> + <p> + Own Id: OTP-15603</p> + </item> + <item> + <p> + Fix various build issues when compiling Erlang/OTP to the + IBM AIX platform.</p> + <p> + Own Id: OTP-15866 Aux Id: PR-2110 </p> + </item> + <item> + <p> + Improved node connection setup handshake protocol. Made + possible to agree on protocol version without dependence + on <c>epmd</c> or other prior knowledge of peer node + version. Also added exchange of node incarnation + ("creation") values and expanded the distribution + capability flag field from 32 to 64 bits.</p> + <p> + Own Id: OTP-16229</p> + </item> + <item> + <p> + New <c>erl_call</c> option <c>-address [Host]:Port</c> to + connect directly to a node without being dependent on + <c>epmd</c> to resolve the node name.</p> + <p> + Own Id: OTP-16251</p> + </item> + <item> + <p> + As announced in OTP 22.0, the deprecated parts of + <c>erl_interface</c> have now been removed (essentially + all C functions with prefix <c>erl_</c>).</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16328</p> + </item> + <item> + <p> + As announced in OTP 22.0, the previously existing limited + support for VxWorks has now been removed.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16329 Aux Id: OTP-15621 </p> + </item> + <item> + <p> + New function <c>ei_connect_host_port</c> and friends to + allow node connection without being dependent on + <c>epmd</c> for node name resolution.</p> + <p> + Own Id: OTP-16496 Aux Id: OTP-16251 </p> + </item> + <item> + <p>A number of new functions have been added to the + <c>erl_interface</c> API:</p> <list> <item><seecref + marker="erl_interface:ei#ei_cmp_pids"><c>ei_cmp_pids()</c></seecref></item> + <item><seecref + marker="erl_interface:ei#ei_cmp_ports"><c>ei_cmp_ports()</c></seecref></item> + <item><seecref + marker="erl_interface:ei#ei_cmp_refs"><c>ei_cmp_refs()</c></seecref></item> + <item><seecref + marker="erl_interface:ei#ei_decode_iodata"><c>ei_decode_iodata()</c></seecref></item> + <item><seecref + marker="erl_interface:ei_connect#ei_make_pid"><c>ei_make_pid()</c></seecref></item> + <item><seecref + marker="erl_interface:ei_connect#ei_make_ref"><c>ei_make_ref()</c></seecref></item> + </list> + <p> + Own Id: OTP-16594</p> + </item> + <item> + <p>Added a <c>-timeout</c> option to <c>erl_call</c>.</p> + <p> + Own Id: OTP-16624</p> + </item> + <item> + <p>The <c>erl_interface</c> <seecref + marker="erl_interface:registry"><c>registry</c></seecref> + functionality is deprecated as of OTP 23, and will be + removed in OTP 24. Reasonably new <c>gcc</c> compilers + will issue deprecation warnings when using this + functionality. In order to disable these warnings, define + the macro <c>EI_NO_DEPR_WARN</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16630</p> + </item> + <item> + <p> + Documentation improvements.</p> + <p> + Own Id: OTP-16633</p> + </item> + </list> + </section> + + + <section><title>Known Bugs and Problems</title> + <list> + <item> + <p> + The <c>ei</c> API for decoding/encoding terms is not + fully 64-bit compatible since terms that have a + representation on the external term format larger than 2 + GB cannot be handled.</p> + <p> + Own Id: OTP-16607 Aux Id: OTP-16608 </p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.13.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/erl_interface/doc/src/ref_man.xml b/lib/erl_interface/doc/src/ref_man.xml index 4900c0f5f3..064a83a4ba 100644 --- a/lib/erl_interface/doc/src/ref_man.xml +++ b/lib/erl_interface/doc/src/ref_man.xml @@ -29,8 +29,6 @@ <file>ref_man.xml</file> </header> <description> - <note><p>The support for VxWorks is deprecated as of OTP 22, and - will be removed in OTP 23.</p></note> </description> <xi:include href="ei.xml"/> <xi:include href="ei_connect.xml"/> diff --git a/lib/erl_interface/doc/src/registry.xml b/lib/erl_interface/doc/src/registry.xml index 28c79ad0fe..92858516d9 100644 --- a/lib/erl_interface/doc/src/registry.xml +++ b/lib/erl_interface/doc/src/registry.xml @@ -35,6 +35,11 @@ <lib>registry</lib> <libsummary>Store and back up key-value pairs.</libsummary> <description> + <note><p>This functionality is deprecated as of OTP 23, and will be + removed in OTP 24. Reasonably new <c>gcc</c> compilers will issue + deprecation warnings. In order to disable these warnings, define the + macro <c>EI_NO_DEPR_WARN</c>.</p></note> + <p>This module provides support for storing key-value pairs in a table known as a registry, backing up registries to <seeerl marker="mnesia:mnesia">Mnesia</seeerl> diff --git a/lib/erl_interface/include/ei.h b/lib/erl_interface/include/ei.h index 6b75a213d0..803e22d4b2 100644 --- a/lib/erl_interface/include/ei.h +++ b/lib/erl_interface/include/ei.h @@ -366,6 +366,7 @@ typedef struct ei_cnode_s { erlang_pid self; ei_socket_callbacks *cbs; void *setup_context; + unsigned int pidsn; } ei_cnode; typedef struct in_addr *Erl_IpAddr; @@ -446,6 +447,8 @@ const char *ei_thishostname(const ei_cnode* ec); const char *ei_thisalivename(const ei_cnode* ec); erlang_pid *ei_self(ei_cnode* ec); +int ei_make_pid(ei_cnode *ec, erlang_pid *pid); +int ei_make_ref(ei_cnode *ec, erlang_ref *ref); /* * settings @@ -568,6 +571,7 @@ int ei_decode_trace(const char *buf, int *index, erlang_trace *p); int ei_decode_tuple_header(const char *buf, int *index, int *arity); int ei_decode_list_header(const char *buf, int *index, int *arity); int ei_decode_map_header(const char *buf, int *index, int *arity); +int ei_decode_iodata(const char *buf, int* index, int *szp, char *out_buf); /* * ei_decode_ei_term() returns 1 if term is decoded, 0 if term is OK, @@ -599,6 +603,10 @@ int ei_x_append(ei_x_buff* x, const ei_x_buff* x2); int ei_x_append_buf(ei_x_buff* x, const char* buf, int len); int ei_skip_term(const char* buf, int* index); +int ei_cmp_refs(erlang_ref *a, erlang_ref *b); +int ei_cmp_pids(erlang_pid *a, erlang_pid *b); +int ei_cmp_ports(erlang_port *a, erlang_port *b); + /*************************************************************************** * * Hash types needed by registry types @@ -691,9 +699,9 @@ int ei_init(void); * be specified in all subsequent calls to registry functions. You can * open as many registries as you like. */ -ei_reg *ei_reg_open(int size); -int ei_reg_resize(ei_reg *oldreg, int newsize); -int ei_reg_close(ei_reg *reg); +ei_reg *ei_reg_open(int size) EI_DEPRECATED_ATTR; +int ei_reg_resize(ei_reg *oldreg, int newsize) EI_DEPRECATED_ATTR; +int ei_reg_close(ei_reg *reg) EI_DEPRECATED_ATTR; /* set values... these routines assign values to keys. If the key * exists, the previous value is discarded and the new one replaces @@ -710,10 +718,10 @@ int ei_reg_close(ei_reg *reg); * On success the function returns 0, otherwise a value * indicating the reason for failure will be returned. */ -int ei_reg_setival(ei_reg *reg, const char *key, long i); -int ei_reg_setfval(ei_reg *reg, const char *key, double f); -int ei_reg_setsval(ei_reg *reg, const char *key, const char *s); -int ei_reg_setpval(ei_reg *reg, const char *key, const void *p, int size); +int ei_reg_setival(ei_reg *reg, const char *key, long i) EI_DEPRECATED_ATTR; +int ei_reg_setfval(ei_reg *reg, const char *key, double f) EI_DEPRECATED_ATTR; +int ei_reg_setsval(ei_reg *reg, const char *key, const char *s) EI_DEPRECATED_ATTR; +int ei_reg_setpval(ei_reg *reg, const char *key, const void *p, int size) EI_DEPRECATED_ATTR; /* general set function (specifiy type via flags) * optional arguments are as for equivalent type-specific function, @@ -723,16 +731,16 @@ int ei_reg_setpval(ei_reg *reg, const char *key, const void *p, int size); * ei_reg_setval(fd, path, EI_STR, const char *s); * ei_reg_setval(fd, path, EI_BIN, const void *p, int size); */ -int ei_reg_setval(ei_reg *reg, const char *key, int flags, ...); +int ei_reg_setval(ei_reg *reg, const char *key, int flags, ...) EI_DEPRECATED_ATTR; /* get value of specific type object */ /* warning: it may be difficult to detect errors when using these * functions, since the error values are returned "in band" */ -long ei_reg_getival(ei_reg *reg, const char *key); -double ei_reg_getfval(ei_reg *reg, const char *key); -const char *ei_reg_getsval(ei_reg *reg, const char *key); -const void *ei_reg_getpval(ei_reg *reg, const char *key, int *size); +long ei_reg_getival(ei_reg *reg, const char *key) EI_DEPRECATED_ATTR; +double ei_reg_getfval(ei_reg *reg, const char *key) EI_DEPRECATED_ATTR; +const char *ei_reg_getsval(ei_reg *reg, const char *key) EI_DEPRECATED_ATTR; +const void *ei_reg_getpval(ei_reg *reg, const char *key, int *size) EI_DEPRECATED_ATTR; /* get value of any type object (must specify) * Retrieve a value from an object. The type of value expected and a @@ -748,7 +756,7 @@ const void *ei_reg_getpval(ei_reg *reg, const char *key, int *size); * for BIN objects an int* is needed to return the size of the object, i.e. * int ei_reg_getval(ei_reg *reg, const char *path, int flags, void **p, int *size); */ -int ei_reg_getval(ei_reg *reg, const char *key, int flags, ...); +int ei_reg_getval(ei_reg *reg, const char *key, int flags, ...) EI_DEPRECATED_ATTR; /* mark the object as dirty. Normally this operation will not be * necessary, as it is done automatically by all of the above 'set' @@ -758,26 +766,26 @@ int ei_reg_getval(ei_reg *reg, const char *key, int flags, ...); * backup operation. Use this function to set the dirty bit on the * object. */ -int ei_reg_markdirty(ei_reg *reg, const char *key); +int ei_reg_markdirty(ei_reg *reg, const char *key) EI_DEPRECATED_ATTR; /* remove objects. The value, if any, is discarded. For STR and BIN * objects, the object itself is removed using free(). */ -int ei_reg_delete(ei_reg *reg, const char *key); +int ei_reg_delete(ei_reg *reg, const char *key) EI_DEPRECATED_ATTR; /* get information about an object */ -int ei_reg_stat(ei_reg *reg, const char *key, struct ei_reg_stat *obuf); +int ei_reg_stat(ei_reg *reg, const char *key, struct ei_reg_stat *obuf) EI_DEPRECATED_ATTR; /* get information about table */ -int ei_reg_tabstat(ei_reg *reg, struct ei_reg_tabstat *obuf); +int ei_reg_tabstat(ei_reg *reg, struct ei_reg_tabstat *obuf) EI_DEPRECATED_ATTR; /* dump to / restore from backup */ /* fd is open descriptor to Erlang, mntab is Mnesia table name */ /* flags here: */ #define EI_FORCE 0x1 /* dump all records (not just dirty ones) */ #define EI_NOPURGE 0x2 /* don't purge deleted records */ -int ei_reg_dump(int fd, ei_reg *reg, const char *mntab, int flags); -int ei_reg_restore(int fd, ei_reg *reg, const char *mntab); -int ei_reg_purge(ei_reg *reg); +int ei_reg_dump(int fd, ei_reg *reg, const char *mntab, int flags) EI_DEPRECATED_ATTR; +int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) EI_DEPRECATED_ATTR; +int ei_reg_purge(ei_reg *reg) EI_DEPRECATED_ATTR; /* -------------------------------------------------------------------- */ /* The ei_global functions */ diff --git a/lib/erl_interface/src/Makefile.in b/lib/erl_interface/src/Makefile.in index 55827ce097..7ff3f09abb 100644 --- a/lib/erl_interface/src/Makefile.in +++ b/lib/erl_interface/src/Makefile.in @@ -331,6 +331,7 @@ DECODESRC = \ decode/decode_double.c \ decode/decode_fun.c \ decode/decode_intlist.c \ + decode/decode_iodata.c \ decode/decode_list_header.c \ decode/decode_long.c \ decode/decode_pid.c \ @@ -393,7 +394,8 @@ MISCSRC = \ misc/get_type.c \ misc/show_msg.c \ misc/ei_compat.c \ - misc/ei_init.c + misc/ei_init.c \ + misc/ei_cmp_nc.c REGISTRYSRC = \ registry/hash_dohash.c \ @@ -744,9 +746,7 @@ release: opt $(INSTALL_DATA) $(HEADERS) "$(RELEASE_PATH)/usr/include" $(INSTALL_DATA) $(OBJ_TARGETS) "$(RELSYSDIR)/lib" $(INSTALL_DATA) $(OBJ_TARGETS) "$(RELEASE_PATH)/usr/lib" -ifneq ($(EXE_TARGETS),) $(INSTALL_PROGRAM) $(EXE_TARGETS) "$(RELSYSDIR)/bin" -endif $(INSTALL_DATA) $(EXTRA) "$(RELSYSDIR)/src" $(INSTALL_DATA) connect/*.[ch] "$(RELSYSDIR)/src/connect" $(INSTALL_DATA) decode/*.[ch] "$(RELSYSDIR)/src/decode" diff --git a/lib/erl_interface/src/connect/ei_connect.c b/lib/erl_interface/src/connect/ei_connect.c index 40efdd101a..a5db73799d 100644 --- a/lib/erl_interface/src/connect/ei_connect.c +++ b/lib/erl_interface/src/connect/ei_connect.c @@ -525,11 +525,272 @@ const char *ei_thiscookie(const ei_cnode* ec) return (const char *)ec->ei_connect_cookie; } +static int +check_initialized_node(ei_cnode *ec) +{ + /* + * Try to guard against returning garbage pids and refs + * by verifying that the node has got its name... + */ + int i, at, end; + char *nodename = &ec->thisnodename[0]; + + for (i = at = end = 0; i < sizeof(ec->thisnodename); i++) { + if (!nodename[i]) { + end = !0; + break; + } + if (nodename[i] == '@') + at = !0; + } + + if (!at || !end) { + erl_errno = EINVAL; + return ERL_ERROR; + } + + return 0; +} + erlang_pid *ei_self(ei_cnode* ec) { + int err = check_initialized_node(ec); + if (err) + return NULL; return &ec->self; } +/* + * ei_make_pid() + */ + +#undef EI_MAKE_PID_ATOMIC__ +#ifdef _REENTRANT +# if (SIZEOF_INT == 4 \ + && (ETHR_HAVE___atomic_compare_exchange_n & 4) \ + && (ETHR_HAVE___atomic_load_n & 4)) +# define EI_MAKE_PID_ATOMIC__ +# else /* !EI_MAKE_PID_ATOMIC__ */ +static ei_mutex_t *pid_mtx = NULL; +# endif /* !EI_MAKE_PID_ATOMIC__ */ +#endif /* _REENTRANT */ + +static int +init_make_pid(int late) +{ +#if defined(_REENTRANT) && !defined(EI_MAKE_PID_ATOMIC__) + + if (late) + return ENOTSUP; /* Refuse doing unsafe initialization... */ + + pid_mtx = ei_mutex_create(); + if (!pid_mtx) + return ENOMEM; + +#endif /* _REENTRANT */ + + return 0; +} + +int ei_make_pid(ei_cnode *ec, erlang_pid *pid) +{ + unsigned int new; + int err; + + if (!ei_connect_initialized) { + fprintf(stderr,"<ERROR> erl_interface not initialized\n"); + exit(1); + } + + err = check_initialized_node(ec); + if (err) { + /* + * write invalid utf8 in nodename which will make + * ei_encode_pid() fail if used... + */ + pid->node[0] = 0xff; + pid->node[1] = 0; + pid->serial = -1; + pid->num = -1; + return err; + } + + strcpy(pid->node, ec->thisnodename); + pid->creation = ec->creation; + + /* + * We avoid creating pids with serial set to 0 since the + * documentation previously gave some really bad advise + * of modifying the 'num' field in the pid returned by + * ei_self(). Since 'serial' field in pid returned by + * ei_self() is initialized to 0, pids created by + * ei_make_pid() wont clash with such badly created pids + * using ei_self() unless user also modified serial, but + * that has at least never been suggested by the + * documentation. + */ + +#ifdef EI_MAKE_PID_ATOMIC__ + { + unsigned int xchg = __atomic_load_n(&ec->pidsn, __ATOMIC_RELAXED); + do { + new = xchg + 1; + if ((new & 0x0fff8000) == 0) + new = 0x8000; /* serial==0 -> serial=1 num=0 */ + } while(!__atomic_compare_exchange_n(&ec->pidsn, &xchg, new, 0, + __ATOMIC_ACQ_REL, + __ATOMIC_RELAXED)); + } +#else /* !EI_MAKE_PID_ATOMIC__ */ + +#ifdef _REENTRANT + ei_mutex_lock(pid_mtx, 0); +#endif + + new = ec->pidsn + 1; + if ((new & 0x0fff8000) == 0) + new = 0x8000; /* serial==0 -> serial=1 num=0 */ + + ec->pidsn = new; + +#ifdef _REENTRANT + ei_mutex_unlock(pid_mtx); +#endif + +#endif /* !EI_MAKE_PID_ATOMIC__ */ + + pid->num = new & 0x7fff; /* 15-bits */ + pid->serial = (new >> 15) & 0x1fff; /* 13-bits */ + + return 0; +} + +/* + * ei_make_ref() + */ + +#undef EI_MAKE_REF_ATOMIC__ +#ifdef _REENTRANT +# if ((SIZEOF_LONG == 8 || SIZEOF_LONGLONG == 8) \ + && (ETHR_HAVE___atomic_compare_exchange_n & 8) \ + && (ETHR_HAVE___atomic_load_n & 8)) +# define EI_MAKE_REF_ATOMIC__ +# if SIZEOF_LONG == 8 +typedef unsigned long ei_atomic_ref__; +# else +typedef unsigned long long ei_atomic_ref__; +# endif +# else /* !EI_MAKE_REF_ATOMIC__ */ +static ei_mutex_t *ref_mtx = NULL; +# endif /* !EI_MAKE_REF_ATOMIC__ */ +#endif /* _REENTRANT */ + +/* + * We use a global counter for all c-nodes in this process. + * We wont wrap anyway due to the enormous amount of values + * available. + */ +#ifdef EI_MAKE_REF_ATOMIC__ +static ei_atomic_ref__ ref_count; +#else +static unsigned int ref_count[3]; +#endif + +static int +init_make_ref(int late) +{ + +#ifdef EI_MAKE_REF_ATOMIC__ + ref_count = 0; +#else /* !EI_MAKE_REF_ATOMIC__ */ + +#ifdef _REENTRANT + + if (late) + return ENOTSUP; /* Refuse doing unsafe initialization... */ + + ref_mtx = ei_mutex_create(); + if (!ref_mtx) + return ENOMEM; + +#endif /* _REENTRANT */ + + ref_count[0] = 0; + ref_count[1] = 0; + ref_count[2] = 0; + +#endif /* !EI_MAKE_REF_ATOMIC__ */ + + return 0; +} + +int ei_make_ref(ei_cnode *ec, erlang_ref *ref) +{ + int err; + if (!ei_connect_initialized) { + fprintf(stderr,"<ERROR> erl_interface not initialized\n"); + exit(1); + } + + err = check_initialized_node(ec); + if (err) { + /* + * write invalid utf8 in nodename which will make + * ei_encode_ref() fail if used... + */ + ref->node[0] = 0xff; + ref->node[1] = 0; + ref->len = -1; + return err; + } + + strcpy(ref->node, ec->thisnodename); + ref->creation = ec->creation; + ref->len = 3; + +#ifdef EI_MAKE_REF_ATOMIC__ + { + ei_atomic_ref__ xchg, new; + xchg = __atomic_load_n(&ref_count, __ATOMIC_RELAXED); + do { + new = xchg + 1; + } while(!__atomic_compare_exchange_n(&ref_count, &xchg, new, 0, + __ATOMIC_ACQ_REL, + __ATOMIC_RELAXED)); + ref->n[0] = (unsigned int) (new & 0x3ffff); + ref->n[1] = (unsigned int) ((new >> 18) & 0xffffffff); + ref->n[2] = (unsigned int) ((new >> (18+32)) & 0xffffffff); + } +#else /* !EI_MAKE_REF_ATOMIC__ */ + +#ifdef _REENTRANT + ei_mutex_lock(ref_mtx, 0); +#endif + + ref->n[0] = ref_count[0]; + ref->n[1] = ref_count[1]; + ref->n[2] = ref_count[2]; + + ref_count[0]++; + ref_count[0] &= 0x3ffff; + if (ref_count[0] == 0) { + ref_count[1]++; + ref_count[1] &= 0xffffffff; + if (ref_count[1] == 0) { + ref_count[2]++; + ref_count[2] &= 0xffffffff; + } + } + +#ifdef _REENTRANT + ei_mutex_unlock(ref_mtx); +#endif + +#endif /* !EI_MAKE_REF_ATOMIC__ */ + + return 0; +} + /* two internal functions that will let us support different cookies * (to be able to connect to other nodes that don't have the same * cookie as each other or us) @@ -616,6 +877,18 @@ static int init_connect(int late) return error; } + error = init_make_ref(late); + if (error) { + EI_TRACE_ERR0("ei_init_connect","can't initiate ei_make_ref()"); + return error; + } + + error = init_make_pid(late); + if (error) { + EI_TRACE_ERR0("ei_init_connect","can't initiate ei_make_pid()"); + return error; + } + ei_connect_initialized = !0; return 0; } @@ -650,6 +923,7 @@ int ei_connect_xinit_ussi(ei_cnode* ec, const char *thishostname, } ec->creation = creation; + ec->pidsn = 0; if (cookie) { if (strlen(cookie) >= sizeof(ec->ei_connect_cookie)) { @@ -1475,11 +1749,8 @@ int ei_receive(int fd, unsigned char *bufp, int bufsize) int ei_reg_send_tmo(ei_cnode* ec, int fd, char *server_name, char* buf, int len, unsigned ms) { - erlang_pid *self = ei_self(ec); - self->num = fd; - /* erl_errno and return code is set by ei_reg_send_encoded_tmo() */ - return ei_send_reg_encoded_tmo(fd, self, server_name, buf, len, ms); + return ei_send_reg_encoded_tmo(fd, ei_self(ec), server_name, buf, len, ms); } @@ -1588,27 +1859,45 @@ int ei_rpc_to(ei_cnode *ec, int fd, char *mod, char *fun, ei_x_buff x; erlang_pid *self = ei_self(ec); - self->num = fd; + int err = ERL_ERROR; /* encode header */ - ei_x_new_with_version(&x); - ei_x_encode_tuple_header(&x, 2); /* A */ - - self->num = fd; - ei_x_encode_pid(&x, self); /* A 1 */ - - ei_x_encode_tuple_header(&x, 5); /* B A 2 */ - ei_x_encode_atom(&x, "call"); /* B 1 */ - ei_x_encode_atom(&x, mod); /* B 2 */ - ei_x_encode_atom(&x, fun); /* B 3 */ - ei_x_append_buf(&x, buf, len); /* B 4 */ - ei_x_encode_atom(&x, "user"); /* B 5 */ - - /* ei_x_encode_atom(&x,"user"); */ - ei_send_reg_encoded(fd, self, "rex", x.buff, x.index); - ei_x_free(&x); - + if (ei_x_new_with_version(&x) < 0) + goto einval; + if (ei_x_encode_tuple_header(&x, 2) < 0) /* A */ + goto einval; + + if (ei_x_encode_pid(&x, self) < 0) /* A 1 */ + goto einval; + + if (ei_x_encode_tuple_header(&x, 5) < 0) /* B A 2 */ + goto einval; + if (ei_x_encode_atom(&x, "call") < 0) /* B 1 */ + goto einval; + if (ei_x_encode_atom(&x, mod) < 0) /* B 2 */ + goto einval; + if (ei_x_encode_atom(&x, fun) < 0) /* B 3 */ + goto einval; + if (ei_x_append_buf(&x, buf, len) < 0) /* B 4 */ + goto einval; + if (ei_x_encode_atom(&x, "user") < 0) /* B 5 */ + goto einval; + + err = ei_send_reg_encoded(fd, self, "rex", x.buff, x.index); + if (err) + goto error; + + ei_x_free(&x); + return 0; + +einval: + EI_CONN_SAVE_ERRNO__(EINVAL); + +error: + if (x.buff != NULL) + ei_x_free(&x); + return err; } /* rpc_to */ /* @@ -1626,11 +1915,6 @@ int ei_rpc_from(ei_cnode *ec, int fd, int timeout, erlang_msg *msg, return res; } /* rpc_from */ - /* - * A true RPC. It return a NULL pointer - * in case of failure, otherwise a valid - * (ETERM *) pointer containing the reply - */ int ei_rpc(ei_cnode* ec, int fd, char *mod, char *fun, const char* inbuf, int inbuflen, ei_x_buff* x) { @@ -1640,27 +1924,45 @@ int ei_rpc(ei_cnode* ec, int fd, char *mod, char *fun, char rex[MAXATOMLEN]; if (ei_rpc_to(ec, fd, mod, fun, inbuf, inbuflen) < 0) { - return -1; + return ERL_ERROR; } - /* FIXME are we not to reply to the tick? */ + + /* ei_rpc_from() responds with a tick if it gets one... */ while ((i = ei_rpc_from(ec, fd, ERL_NO_TIMEOUT, &msg, x)) == ERL_TICK) ; - if (i == ERL_ERROR) return -1; - /*ep = 'erl'_element(2,emsg.msg);*/ /* {RPC_Tag, RPC_Reply} */ + if (i == ERL_ERROR) return i; + + /* Expect: {rex, RPC_Reply} */ + index = 0; - if (ei_decode_version(x->buff, &index, &i) < 0 - || ei_decode_ei_term(x->buff, &index, &t) < 0) - return -1; /* FIXME ei_decode_version don't set erl_errno as before */ - /* FIXME this is strange, we don't check correct "rex" atom - and we let it pass if not ERL_SMALL_TUPLE_EXT and arity == 2 */ - if (t.ei_type == ERL_SMALL_TUPLE_EXT && t.arity == 2) - if (ei_decode_atom(x->buff, &index, rex) < 0) - return -1; + if (ei_decode_version(x->buff, &index, &i) < 0) + goto ebadmsg; + + if (ei_decode_ei_term(x->buff, &index, &t) < 0) + goto ebadmsg; + + if (t.ei_type != ERL_SMALL_TUPLE_EXT && t.ei_type != ERL_LARGE_TUPLE_EXT) + goto ebadmsg; + + if (t.arity != 2) + goto ebadmsg; + + if (ei_decode_atom(x->buff, &index, rex) < 0) + goto ebadmsg; + + if (strcmp("rex", rex) != 0) + goto ebadmsg; + /* remove header */ x->index -= index; memmove(x->buff, &x->buff[index], x->index); return 0; + +ebadmsg: + + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } diff --git a/lib/erl_interface/src/connect/eirecv.c b/lib/erl_interface/src/connect/eirecv.c index 0673d25d38..96732db5b6 100644 --- a/lib/erl_interface/src/connect/eirecv.c +++ b/lib/erl_interface/src/connect/eirecv.c @@ -70,7 +70,7 @@ ei_recv_internal (int fd, err = EI_GET_CBS_CTX__(&cbs, &ctx, fd); if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } /* get length field */ @@ -80,7 +80,7 @@ ei_recv_internal (int fd, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } len = get32be(s); @@ -91,7 +91,7 @@ ei_recv_internal (int fd, ssize_t wlen = sizeof(tock); ei_write_fill_ctx_t__(cbs, ctx, tock, &wlen, tmo); /* Failure no problem */ *msglenp = 0; - return 0; /* maybe flag ERL_EAGAIN [sverkerw] */ + return ERL_TICK; } /* turn off tracing on each receive. it will be turned back on if @@ -106,7 +106,7 @@ ei_recv_internal (int fd, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } /* now decode header */ @@ -119,8 +119,8 @@ ei_recv_internal (int fd, || ei_decode_tuple_header(header,&index,&arity) || ei_decode_long(header,&index,&msg->msgtype)) { - erl_errno = EIO; /* Maybe another code for decoding errors */ - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } switch (msg->msgtype) { @@ -129,8 +129,8 @@ ei_recv_internal (int fd, if (ei_decode_atom_as(header,&index,msg->cookie,sizeof(msg->cookie),ERLANG_UTF8,NULL,NULL) || ei_decode_pid(header,&index,&msg->to)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } break; @@ -141,8 +141,8 @@ ei_recv_internal (int fd, || ei_decode_atom_as(header,&index,msg->cookie,sizeof(msg->cookie),ERLANG_UTF8,NULL,NULL) || ei_decode_atom_as(header,&index,msg->toname,sizeof(msg->toname),ERLANG_UTF8,NULL,NULL)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } /* actual message is remaining part of headerbuf, plus any unread bytes */ @@ -155,8 +155,8 @@ ei_recv_internal (int fd, if (ei_decode_pid(header,&index,&msg->from) || ei_decode_pid(header,&index,&msg->to)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } break; @@ -167,8 +167,8 @@ ei_recv_internal (int fd, if (ei_decode_pid(header,&index,&msg->from) || ei_decode_pid(header,&index,&msg->to)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } break; @@ -179,8 +179,8 @@ ei_recv_internal (int fd, || ei_decode_pid(header,&index,&msg->to) || ei_decode_trace(header,&index,&msg->token)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } ei_trace(1,&msg->token); /* turn on tracing */ @@ -193,8 +193,8 @@ ei_recv_internal (int fd, || ei_decode_atom_as(header,&index,msg->toname,sizeof(msg->toname),ERLANG_UTF8,NULL,NULL) || ei_decode_trace(header,&index,&msg->token)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } ei_trace(1,&msg->token); /* turn on tracing */ @@ -207,8 +207,8 @@ ei_recv_internal (int fd, || ei_decode_pid(header,&index,&msg->to) || ei_decode_trace(header,&index,&msg->token)) { - erl_errno = EIO; - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } ei_trace(1,&msg->token); /* turn on tracing */ @@ -235,14 +235,14 @@ ei_recv_internal (int fd, err = ei_read_fill_ctx_t__(cbs, ctx, header, &rlen, tmo); if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } if (rlen == 0) break; remain -= rlen; } erl_errno = EMSGSIZE; - return -1; + return ERL_ERROR; } else { /* Dynamic buffer --- grow it. */ @@ -253,7 +253,7 @@ ei_recv_internal (int fd, if ((mbuf = realloc(*mbufp, msglen)) == NULL) { erl_errno = ENOMEM; - return -1; + return ERL_ERROR; } *mbufp = mbuf; @@ -276,7 +276,7 @@ ei_recv_internal (int fd, if (err) { *msglenp = bytesread-index+1; /* actual bytes in users buffer */ EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } } diff --git a/lib/erl_interface/src/connect/send.c b/lib/erl_interface/src/connect/send.c index f9337187ab..df2e3e023d 100644 --- a/lib/erl_interface/src/connect/send.c +++ b/lib/erl_interface/src/connect/send.c @@ -67,19 +67,31 @@ int ei_send_encoded_tmo(int fd, const erlang_pid *to, /* check that he can receive trace tokens first */ if (ei_distversion(fd) > 0) token = ei_trace(0,NULL); + EI_CONN_SAVE_ERRNO__(EINVAL); + /* header = SEND, cookie, to max sizes: */ - ei_encode_version(header,&index); /* 1 */ + if (ei_encode_version(header,&index) < 0) /* 1 */ + return ERL_ERROR; if (token) { - ei_encode_tuple_header(header,&index,4); /* 2 */ - ei_encode_long(header,&index,ERL_SEND_TT); /* 2 */ + if (ei_encode_tuple_header(header,&index,4) < 0) /* 2 */ + return ERL_ERROR; + if (ei_encode_long(header,&index,ERL_SEND_TT) < 0) /* 2 */ + return ERL_ERROR; } else { - ei_encode_tuple_header(header,&index,3); - ei_encode_long(header,&index,ERL_SEND); + if (ei_encode_tuple_header(header,&index,3) < 0) + return ERL_ERROR; + if (ei_encode_long(header,&index,ERL_SEND) < 0) + return ERL_ERROR; } - ei_encode_atom(header,&index,ei_getfdcookie(fd)); /* 258 */ - ei_encode_pid(header,&index,to); /* 268 */ + if (ei_encode_atom(header,&index,ei_getfdcookie(fd)) < 0) /* 258 */ + return ERL_ERROR; + if (ei_encode_pid(header,&index,to) < 0) /* 268 */ + return ERL_ERROR; - if (token) ei_encode_trace(header,&index,token); /* 534 */ + if (token) { + if (ei_encode_trace(header,&index,token) < 0) /* 534 */ + return ERL_ERROR; + } /* control message (precedes header actually) */ /* length = 1 ('p') + header len + message len */ @@ -107,9 +119,11 @@ int ei_send_encoded_tmo(int fd, const erlang_pid *to, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } + erl_errno = 0; + return 0; } #endif /* EI_HAVE_STRUCT_IOVEC__ */ @@ -121,7 +135,7 @@ int ei_send_encoded_tmo(int fd, const erlang_pid *to, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } len = tot_len = (ssize_t) msglen; @@ -130,9 +144,10 @@ int ei_send_encoded_tmo(int fd, const erlang_pid *to, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } + erl_errno = 0; return 0; } diff --git a/lib/erl_interface/src/connect/send_exit.c b/lib/erl_interface/src/connect/send_exit.c index af0fb2af88..231aad0ae0 100644 --- a/lib/erl_interface/src/connect/send_exit.c +++ b/lib/erl_interface/src/connect/send_exit.c @@ -67,9 +67,12 @@ int ei_send_exit_tmo(int fd, const erlang_pid *from, const erlang_pid *to, return ERL_ERROR; } - if (len > EISMALLBUF) - if (!(dbuf = malloc(len))) - return -1; + if (len > EISMALLBUF) { + if (!(dbuf = malloc(len))) { + EI_CONN_SAVE_ERRNO__(ENOMEM); + return ERL_ERROR; + } + } msgbuf = (dbuf ? dbuf : sbuf); @@ -77,31 +80,46 @@ int ei_send_exit_tmo(int fd, const erlang_pid *from, const erlang_pid *to, /* check that he can receive trace tokens first */ if (ei_distversion(fd) > 0) token = ei_trace(0,NULL); + EI_CONN_SAVE_ERRNO__(EINVAL); + index = 5; /* max sizes: */ - ei_encode_version(msgbuf,&index); /* 1 */ + if (ei_encode_version(msgbuf,&index) < 0) /* 1 */ + return ERL_ERROR; if (token) { - ei_encode_tuple_header(msgbuf,&index,5); /* 2 */ - ei_encode_long(msgbuf,&index,ERL_EXIT_TT); /* 2 */ + if (ei_encode_tuple_header(msgbuf,&index,5) < 0) /* 2 */ + return ERL_ERROR; + if (ei_encode_long(msgbuf,&index,ERL_EXIT_TT) < 0)/* 2 */ + return ERL_ERROR; } else { - ei_encode_tuple_header(msgbuf,&index,4); - ei_encode_long(msgbuf,&index,ERL_EXIT); + if (ei_encode_tuple_header(msgbuf,&index,4) < 0) + return ERL_ERROR; + if (ei_encode_long(msgbuf,&index,ERL_EXIT) < 0) + return ERL_ERROR; } - ei_encode_pid(msgbuf,&index,from); /* 268 */ - ei_encode_pid(msgbuf,&index,to); /* 268 */ + if (ei_encode_pid(msgbuf,&index,from) < 0) /* 268 */ + return ERL_ERROR; + if (ei_encode_pid(msgbuf,&index,to) < 0) /* 268 */ + return ERL_ERROR; - if (token) ei_encode_trace(msgbuf,&index,token); /* 534 */ + if (token) { + if (ei_encode_trace(msgbuf,&index,token) < 0) /* 534 */ + return ERL_ERROR; + } /* Reason */ - ei_encode_string(msgbuf,&index,reason); /* len */ + if (ei_encode_string(msgbuf,&index,reason) < 0) /* len */ + return ERL_ERROR; /* 5 byte header missing */ s = msgbuf; put32be(s, index - 4); /* 4 */ - put8(s, ERL_PASS_THROUGH); /* 1 */ + put8(s, ERL_PASS_THROUGH); /* 1 */ /*** sum: len + 1080 */ - if (ei_tracelevel >= 4) - ei_show_sendmsg(stderr,msgbuf,NULL); + if (ei_tracelevel >= 4) { + if (ei_show_sendmsg(stderr,msgbuf,NULL) < 0) + return ERL_ERROR; + } wlen = (ssize_t) index; err = ei_write_fill_ctx_t__(cbs, ctx, msgbuf, &wlen, tmo); @@ -113,6 +131,8 @@ int ei_send_exit_tmo(int fd, const erlang_pid *from, const erlang_pid *to, EI_CONN_SAVE_ERRNO__(err); return ERL_ERROR; } + + erl_errno = 0; return 0; } diff --git a/lib/erl_interface/src/connect/send_reg.c b/lib/erl_interface/src/connect/send_reg.c index b5c9e6a6f8..1d9c85f210 100644 --- a/lib/erl_interface/src/connect/send_reg.c +++ b/lib/erl_interface/src/connect/send_reg.c @@ -64,21 +64,33 @@ int ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, if (ei_distversion(fd) > 0) token = ei_trace(0,NULL); + EI_CONN_SAVE_ERRNO__(EINVAL); + /* header = REG_SEND, from, cookie, toname max sizes: */ - ei_encode_version(header,&index); /* 1 */ + if (ei_encode_version(header,&index) < 0) /* 1 */ + return ERL_ERROR; if (token) { - ei_encode_tuple_header(header,&index,5); /* 2 */ - ei_encode_long(header,&index,ERL_REG_SEND_TT); /* 2 */ + if (ei_encode_tuple_header(header,&index,5) < 0) /* 2 */ + return ERL_ERROR; + if (ei_encode_long(header,&index,ERL_REG_SEND_TT) < 0) /* 2 */ + return ERL_ERROR; } else { - ei_encode_tuple_header(header,&index,4); - ei_encode_long(header,&index,ERL_REG_SEND); + if (ei_encode_tuple_header(header,&index,4) < 0) + return ERL_ERROR; + if (ei_encode_long(header,&index,ERL_REG_SEND) < 0) + return ERL_ERROR; } - ei_encode_pid(header, &index, from); /* 268 */ - ei_encode_atom(header, &index, ei_getfdcookie(fd)); /* 258 */ - ei_encode_atom(header, &index, to); /* 268 */ - - if (token) ei_encode_trace(header,&index,token); /* 534 */ + if (ei_encode_pid(header, &index, from) < 0) /* 268 */ + return ERL_ERROR; + if (ei_encode_atom(header, &index, ei_getfdcookie(fd)) < 0) /* 258 */ + return ERL_ERROR; + if (ei_encode_atom(header, &index, to) < 0) /* 268 */ + return ERL_ERROR; + if (token) { + if (ei_encode_trace(header,&index,token) < 0) /* 534 */ + return ERL_ERROR; + } /* control message (precedes header actually) */ /* length = 1 ('p') + header len + message len */ s = header; @@ -103,8 +115,11 @@ int ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } + + erl_errno = 0; + return 0; } #endif /* EI_HAVE_STRUCT_IOVEC__ */ @@ -116,7 +131,7 @@ int ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } len = tot_len = (ssize_t) msglen; @@ -125,8 +140,10 @@ int ei_send_reg_encoded_tmo(int fd, const erlang_pid *from, err = EIO; if (err) { EI_CONN_SAVE_ERRNO__(err); - return -1; + return ERL_ERROR; } + + erl_errno = 0; return 0; } diff --git a/lib/erl_interface/src/decode/decode_iodata.c b/lib/erl_interface/src/decode/decode_iodata.c new file mode 100644 index 0000000000..88d23d8a1e --- /dev/null +++ b/lib/erl_interface/src/decode/decode_iodata.c @@ -0,0 +1,166 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2020. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + * + */ + +#include <string.h> +#include "eidef.h" +#include "eiext.h" +#include "putget.h" + +static int decode_list_ext_iodata(const char *buf, int* index, + int *szp, unsigned char **pp); +static void decode_string(const char *buf, int* index, + int *szp, unsigned char **pp); + +int ei_decode_iodata(const char *buf, int* index, int *szp, char *out_buf) +{ + int type, len; + + if (szp) + *szp = 0; + + if (ei_get_type(buf, index, &type, &len) < 0) + return -1; + + /* Top level of iodata is either a list or a binary... */ + + switch (type) { + + case ERL_BINARY_EXT: { + long llen; + if (ei_decode_binary(buf, index, out_buf, &llen) < 0) + return -1; + if (llen != (long) len) + return -1; /* general 64-bit issue with ei api... */ + if (szp) + *szp += len; + return 0; + } + + case ERL_STRING_EXT: { + unsigned char *p = (unsigned char *) out_buf; + decode_string(buf, index, szp, p ? &p : NULL); + return 0; + } + case ERL_NIL_EXT: + return ei_decode_list_header(buf, index, NULL); + + case ERL_LIST_EXT: { + unsigned char *ptr = (unsigned char *) out_buf; + len = 0; + return decode_list_ext_iodata(buf, index, + szp ? szp : &len, + ptr ? &ptr : NULL); + } + + default: + return -1; /* Not a list nor a binary... */ + } +} + +static int decode_list_ext_iodata(const char *buf, int* index, + int *szp, unsigned char **pp) +{ + int type, len, i, conses; + + if (ei_decode_list_header(buf, index, &conses) < 0) + return -1; + + for (i = 0; i <= conses; i++) { + + if (ei_get_type(buf, index, &type, &len) < 0) + return -1; + + switch (type) { + case ERL_SMALL_INTEGER_EXT: + case ERL_INTEGER_EXT: { + long val; + if (i == conses) + return -1; /* int not allowed in cdr of cons */ + if (ei_decode_long(buf, index, &val) < 0) + return -1; + if (val < 0 || 255 < val) + return -1; + if (pp) + *((*pp)++) = (unsigned char) val; + *szp += 1; + break; + } + case ERL_BINARY_EXT: { + void *p = pp ? *pp : NULL; + long llen; + if (ei_decode_binary(buf, index, p, &llen) < 0) + return -1; + if (llen != (long) len) + return -1; /* general 64-bit issue with ei api... */ + if (pp) + *pp += len; + *szp += len; + break; + } + case ERL_STRING_EXT: + decode_string(buf, index, szp, pp); + break; + case ERL_LIST_EXT: + if (decode_list_ext_iodata(buf, index, szp, pp) < 0) + return -1; + break; + case ERL_NIL_EXT: + if (ei_decode_list_header(buf, index, NULL) < 0) + return -1; + break; + default: + /* Not a list, a binary, nor a byte sized integer... */ + return -1; + } + } + + return 0; +} + +static void +decode_string(const char *buf, int* index, int *szp, unsigned char **pp) +{ + /* + * ei_decode_string() null-terminates the string + * which we do not want, so we decode it ourselves + * here instead... + */ + int len; + char *s = (char *) buf + *index; + char *s0 = s; + + /* ASSERT(*s == ERL_STRING_EXT); */ + s++; + + len = get16be(s); + + if (pp) { + memcpy(*pp, s, len); + *pp += len; + } + + if (szp) + *szp += len; + + s += len; + *index += s-s0; + +} diff --git a/lib/erl_interface/src/decode/decode_ref.c b/lib/erl_interface/src/decode/decode_ref.c index c9b38c1c3b..c10a02094e 100644 --- a/lib/erl_interface/src/decode/decode_ref.c +++ b/lib/erl_interface/src/decode/decode_ref.c @@ -54,6 +54,9 @@ int ei_decode_ref(const char *buf, int *index, erlang_ref *p) /* first the integer count */ count = get16be(s); + if (count > sizeof(p->n)/sizeof(p->n[0])) + return -1; /* Not enough space in struct... */ + if (p) { p->len = count; if (get_atom(&s, p->node, NULL) < 0) return -1; diff --git a/lib/erl_interface/src/encode/encode_trace.c b/lib/erl_interface/src/encode/encode_trace.c index d0e6aec474..4c51e3d84b 100644 --- a/lib/erl_interface/src/encode/encode_trace.c +++ b/lib/erl_interface/src/encode/encode_trace.c @@ -23,12 +23,18 @@ int ei_encode_trace(char *buf, int *index, const erlang_trace *p) { /* { Flags, Label, Serial, FromPid, Prev } */ - ei_encode_tuple_header(buf,index,5); - ei_encode_long(buf,index,p->flags); - ei_encode_long(buf,index,p->label); - ei_encode_long(buf,index,p->serial); - ei_encode_pid(buf,index,&p->from); - ei_encode_long(buf,index,p->prev); + if (ei_encode_tuple_header(buf,index,5) < 0) + return -1; + if (ei_encode_long(buf,index,p->flags) < 0) + return -1; + if (ei_encode_long(buf,index,p->label) < 0) + return -1; + if (ei_encode_long(buf,index,p->serial) < 0) + return -1; + if (ei_encode_pid(buf,index,&p->from) < 0) + return -1; + if (ei_encode_long(buf,index,p->prev) < 0) + return -1; /* index is updated by the functions we called */ diff --git a/lib/erl_interface/src/global/global_names.c b/lib/erl_interface/src/global/global_names.c index bcbe740223..4524439467 100644 --- a/lib/erl_interface/src/global/global_names.c +++ b/lib/erl_interface/src/global/global_names.c @@ -26,6 +26,7 @@ #include "ei_connect_int.h" #include "ei.h" #include "ei_connect.h" +#include "ei_internal.h" #define GLOBALNAMEBUF (16*1024) /* not very small actually */ @@ -52,16 +53,18 @@ char **ei_global_names(ei_cnode *ec, int fd, int *count) char **names; char *s; - self->num = fd; - ei_encode_version(buf,&index); - ei_encode_tuple_header(buf,&index,2); - ei_encode_pid(buf,&index,self); /* PidFrom */ - ei_encode_tuple_header(buf,&index,5); - ei_encode_atom(buf,&index,"call"); /* call */ - ei_encode_atom(buf,&index,"global"); /* Mod */ - ei_encode_atom(buf,&index,"registered_names"); /* Fun */ - ei_encode_list_header(buf,&index,0); /* Args: [ ] */ - ei_encode_atom(buf,&index,"user"); /* user */ + if (ei_encode_version(buf,&index) + || ei_encode_tuple_header(buf,&index,2) + || ei_encode_pid(buf,&index,self) /* PidFrom */ + || ei_encode_tuple_header(buf,&index,5) + || ei_encode_atom(buf,&index,"call") /* call */ + || ei_encode_atom(buf,&index,"global") /* Mod */ + || ei_encode_atom(buf,&index,"registered_names")/* Fun */ + || ei_encode_list_header(buf,&index,0) /* Args: [ ] */ + || ei_encode_atom(buf,&index,"user")) { /* user */ + EI_CONN_SAVE_ERRNO__(EINVAL); + return NULL; + } /* make the rpc call */ if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return NULL; @@ -72,7 +75,10 @@ char **ei_global_names(ei_cnode *ec, int fd, int *count) else break; } - if (i != ERL_SEND) return NULL; + if (i != ERL_SEND) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return NULL; + } /* expecting { rex, [name1, name2, ...] } */ size = msglen; @@ -83,7 +89,10 @@ char **ei_global_names(ei_cnode *ec, int fd, int *count) || (arity != 2) || ei_decode_atom(buf,&index,tmpbuf) || strcmp(tmpbuf,"rex") - || ei_decode_list_header(buf,&index,&arity)) return NULL; + || ei_decode_list_header(buf,&index,&arity)) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return NULL; + } /* we use the size of the rest of the received message to estimate @@ -92,7 +101,10 @@ char **ei_global_names(ei_cnode *ec, int fd, int *count) * a little less than the atoms themselves needed in the reply. */ arity++; /* we will need a terminating NULL as well */ - if (!(names = malloc((arity * sizeof(char**)) + (size-index)))) return NULL; + if (!(names = malloc((arity * sizeof(char**)) + (size-index)))) { + EI_CONN_SAVE_ERRNO__(ENOMEM); + return NULL; + } /* arity pointers first, followed by s */ s = (char *)(names+arity); diff --git a/lib/erl_interface/src/global/global_register.c b/lib/erl_interface/src/global/global_register.c index c260ce091c..dbdab8dbc5 100644 --- a/lib/erl_interface/src/global/global_register.c +++ b/lib/erl_interface/src/global/global_register.c @@ -23,6 +23,7 @@ #include "eisend.h" #include "eirecv.h" #include "ei.h" +#include "ei_internal.h" int ei_global_register(int fd, const char *name, erlang_pid *self) { @@ -40,24 +41,27 @@ int ei_global_register(int fd, const char *name, erlang_pid *self) /* set up rpc arguments */ /* { PidFrom, { call, Mod, Fun, Args, user }} */ index = 0; - ei_encode_version(buf,&index); - ei_encode_tuple_header(buf,&index,2); - ei_encode_pid(buf,&index,self); /* PidFrom */ - ei_encode_tuple_header(buf,&index,5); - ei_encode_atom(buf,&index,"call"); /* call */ - ei_encode_atom(buf,&index,"global"); /* Mod */ - ei_encode_atom(buf,&index,"register_name_external"); /* Fun */ - ei_encode_list_header(buf,&index,3); /* Args: [ name, self(), cnode ] */ - ei_encode_atom(buf,&index,name); - ei_encode_pid(buf,&index,self); - ei_encode_tuple_header(buf,&index,2); - ei_encode_atom(buf,&index,"global"); /* special "resolve" treatment */ - ei_encode_atom(buf,&index,"cnode"); /* i.e. we get a SEND when conflict */ - ei_encode_empty_list(buf,&index); - ei_encode_atom(buf,&index,"user"); /* user */ + if (ei_encode_version(buf,&index) + || ei_encode_tuple_header(buf,&index,2) + || ei_encode_pid(buf,&index,self) /* PidFrom */ + || ei_encode_tuple_header(buf,&index,5) + || ei_encode_atom(buf,&index,"call") /* call */ + || ei_encode_atom(buf,&index,"global") /* Mod */ + || ei_encode_atom(buf,&index,"register_name_external")/* Fun */ + || ei_encode_list_header(buf,&index,3) /* Args: [ name, self(), cnode ] */ + || ei_encode_atom(buf,&index,name) + || ei_encode_pid(buf,&index,self) + || ei_encode_tuple_header(buf,&index,2) + || ei_encode_atom(buf,&index,"global") /* special "resolve" treatment */ + || ei_encode_atom(buf,&index,"cnode") /* i.e. we get a SEND when conflict */ + || ei_encode_empty_list(buf,&index) + || ei_encode_atom(buf,&index,"user")) { /* user */ + EI_CONN_SAVE_ERRNO__(EINVAL); + return ERL_ERROR; + } /* make the rpc call */ - if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return -1; + if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return ERL_ERROR; /* get the reply: expect link and an atom, or just an atom */ needlink = needatom = needmonitor = 1; @@ -72,19 +76,28 @@ int ei_global_register(int fd, const char *name, erlang_pid *self) switch (i) { case ERL_LINK: /* got link */ - if (!needlink) return -1; + if (!needlink) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; + } needlink = 0; break; case ERL_MONITOR_P-10: /* got monitor */ - if (!needmonitor) { return -1;} + if (!needmonitor) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; + } needmonitor = 0; break; case ERL_SEND: /* got message - does it contain our atom? */ - if (!needatom) return -1; + if (!needatom) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; + } else { /* expecting { rex, yes } */ index = 0; @@ -94,8 +107,10 @@ int ei_global_register(int fd, const char *name, erlang_pid *self) || ei_decode_atom(buf,&index,tmpbuf) || strcmp(tmpbuf,"rex") || ei_decode_atom(buf,&index,tmpbuf) - || strcmp(tmpbuf,"yes")) - return -1; /* bad response from other side */ + || strcmp(tmpbuf,"yes")) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; /* bad response from other side */ + } /* we're done */ return 0; @@ -103,7 +118,8 @@ int ei_global_register(int fd, const char *name, erlang_pid *self) break; default: - return -1; /* something else */ + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; /* something else */ } } return 0; diff --git a/lib/erl_interface/src/global/global_unregister.c b/lib/erl_interface/src/global/global_unregister.c index ee785a2abd..4743fe65d7 100644 --- a/lib/erl_interface/src/global/global_unregister.c +++ b/lib/erl_interface/src/global/global_unregister.c @@ -25,6 +25,7 @@ #include "ei_connect_int.h" #include "ei_connect.h" #include "ei.h" +#include "ei_internal.h" /* remove the association between name and its pid */ /* global:unregister_name(name) -> ok */ @@ -40,22 +41,23 @@ int ei_global_unregister(ei_cnode *ec, int fd, const char *name) int version,arity,msglen; int needunlink, needatom, needdemonitor; - /* make a self pid */ - self->num = fd; - ei_encode_version(buf,&index); - ei_encode_tuple_header(buf,&index,2); - ei_encode_pid(buf,&index,self); /* PidFrom */ - ei_encode_tuple_header(buf,&index,5); - ei_encode_atom(buf,&index,"call"); /* call */ - ei_encode_atom(buf,&index,"global"); /* Mod */ - ei_encode_atom(buf,&index,"unregister_name_external"); /* Fun */ - ei_encode_list_header(buf,&index,1); /* Args: [ name ] */ - ei_encode_atom(buf,&index,name); - ei_encode_empty_list(buf,&index); - ei_encode_atom(buf,&index,"user"); /* user */ + if (ei_encode_version(buf,&index) + || ei_encode_tuple_header(buf,&index,2) + || ei_encode_pid(buf,&index,self) /* PidFrom */ + || ei_encode_tuple_header(buf,&index,5) + || ei_encode_atom(buf,&index,"call") /* call */ + || ei_encode_atom(buf,&index,"global") /* Mod */ + || ei_encode_atom(buf,&index,"unregister_name_external") /* Fun */ + || ei_encode_list_header(buf,&index,1) /* Args: [ name ] */ + || ei_encode_atom(buf,&index,name) + || ei_encode_empty_list(buf,&index) + || ei_encode_atom(buf,&index,"user")) { /* user */ + EI_CONN_SAVE_ERRNO__(EINVAL); + return ERL_ERROR; + } /* make the rpc call */ - if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return -1; + if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return ERL_ERROR; /* get the reply: expect unlink and an atom, or just an atom */ needunlink = needatom = needdemonitor = 1; @@ -70,19 +72,28 @@ int ei_global_unregister(ei_cnode *ec, int fd, const char *name) switch (i) { case ERL_UNLINK: /* got unlink */ - if (!needunlink) return -1; + if (!needunlink) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; + } needunlink = 0; break; case ERL_DEMONITOR_P-10: /* got demonitor */ - if (!needdemonitor) return -1; + if (!needdemonitor) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; + } needdemonitor = 0; break; case ERL_SEND: /* got message - does it contain our atom? */ - if (!needatom) return -1; + if (!needatom) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; + } else { /* expecting { rex, ok } */ index = 0; @@ -92,8 +103,10 @@ int ei_global_unregister(ei_cnode *ec, int fd, const char *name) || ei_decode_atom(buf,&index,tmpbuf) || strcmp(tmpbuf,"rex") || ei_decode_atom(buf,&index,tmpbuf) - || strcmp(tmpbuf,"ok")) - return -1; /* bad response from other side */ + || strcmp(tmpbuf,"ok")) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; /* bad response from other side */ + } /* we're done here */ return 0; @@ -101,7 +114,8 @@ int ei_global_unregister(ei_cnode *ec, int fd, const char *name) break; default: - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } } diff --git a/lib/erl_interface/src/global/global_whereis.c b/lib/erl_interface/src/global/global_whereis.c index afedc98030..5a6134c2ab 100644 --- a/lib/erl_interface/src/global/global_whereis.c +++ b/lib/erl_interface/src/global/global_whereis.c @@ -26,6 +26,7 @@ #include "ei_connect_int.h" #include "ei.h" #include "ei_connect.h" +#include "ei_internal.h" /* return the ETERM pid corresponding to name. If caller * provides non-NULL node, nodename will be returned there @@ -44,22 +45,22 @@ int ei_global_whereis(ei_cnode *ec, int fd, const char *name, erlang_pid* pid, c int i; int version,arity,msglen; - self->num = fd; /* FIXME looks strange to change something?! */ - - ei_encode_version(buf,&index); - ei_encode_tuple_header(buf,&index,2); - ei_encode_pid(buf,&index,self); /* PidFrom */ - ei_encode_tuple_header(buf,&index,5); - ei_encode_atom(buf,&index,"call"); /* call */ - ei_encode_atom(buf,&index,"global"); /* Mod */ - ei_encode_atom(buf,&index,"whereis_name"); /* Fun */ - ei_encode_list_header(buf,&index,1); /* Args: [ name ] */ - ei_encode_atom(buf,&index,name); - ei_encode_empty_list(buf,&index); - ei_encode_atom(buf,&index,"user"); /* user */ - + if (ei_encode_version(buf,&index) + || ei_encode_tuple_header(buf,&index,2) + || ei_encode_pid(buf,&index,self) /* PidFrom */ + || ei_encode_tuple_header(buf,&index,5) + || ei_encode_atom(buf,&index,"call") /* call */ + || ei_encode_atom(buf,&index,"global") /* Mod */ + || ei_encode_atom(buf,&index,"whereis_name") /* Fun */ + || ei_encode_list_header(buf,&index,1) /* Args: [ name ] */ + || ei_encode_atom(buf,&index,name) + || ei_encode_empty_list(buf,&index) + || ei_encode_atom(buf,&index,"user")) { /* user */ + EI_CONN_SAVE_ERRNO__(EINVAL); + return ERL_ERROR; + } /* make the rpc call */ - if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return -1; + if (ei_send_reg_encoded(fd,self,"rex",buf,index)) return ERL_ERROR; while (1) { index = EISMALLBUF; @@ -67,7 +68,7 @@ int ei_global_whereis(ei_cnode *ec, int fd, const char *name, erlang_pid* pid, c else break; } - if (i != ERL_SEND) return -1; + if (i != ERL_SEND) return ERL_ERROR; /* expecting { rex, pid } */ index = 0; @@ -76,8 +77,10 @@ int ei_global_whereis(ei_cnode *ec, int fd, const char *name, erlang_pid* pid, c || (arity != 2) || ei_decode_atom(buf,&index,tmpbuf) || strcmp(tmpbuf,"rex") - || ei_decode_pid(buf,&index,&epid)) - return -1; /* bad response from other side */ + || ei_decode_pid(buf,&index,&epid)) { + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; /* bad response from other side */ + } /* extract the nodename for the caller */ if (node) { @@ -86,7 +89,8 @@ int ei_global_whereis(ei_cnode *ec, int fd, const char *name, erlang_pid* pid, c strcpy(node, node_str); } else { - return -1; + EI_CONN_SAVE_ERRNO__(EBADMSG); + return ERL_ERROR; } } *pid = epid; diff --git a/lib/erl_interface/src/misc/ei_cmp_nc.c b/lib/erl_interface/src/misc/ei_cmp_nc.c new file mode 100644 index 0000000000..73650f429f --- /dev/null +++ b/lib/erl_interface/src/misc/ei_cmp_nc.c @@ -0,0 +1,117 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2020. All Rights Reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * %CopyrightEnd% + * + */ + +#include <string.h> +#include "eidef.h" + +/* + * Comparison of node container types (pid, port, ref). Comparison + * done the same way as ERTS compare them. In ref and port case, + * node info is compared before other data, and in pid case node + * info is compared after other data. + */ + +static int cmp_nodes(char *aname, unsigned int acreation, + char *bname, unsigned int bcreation) +{ + int differ = strcmp(aname, bname); + if (differ) + return differ; + if (acreation == bcreation) + return 0; + if (acreation < bcreation) + return -1; + return 1; +} + +int +ei_cmp_refs(erlang_ref *a, erlang_ref *b) +{ + int differ; + int i, alen, blen; + unsigned int *anum, *bnum; + + differ = cmp_nodes(a->node, a->creation, b->node, b->creation); + if (differ) + return differ; + + alen = a->len; + blen = b->len; + + anum = &a->n[0]; + bnum = &b->n[0]; + + if (alen != blen) { + if (alen > blen) { + do { + if (anum[alen - 1] != 0) + return 1; + alen--; + } while (alen > blen); + } + else { + do { + if (bnum[blen - 1] != 0) + return -1; + blen--; + } while (alen < blen); + } + } + + for (i = alen - 1; i >= 0; i--) + if (anum[i] != bnum[i]) + return anum[i] < bnum[i] ? -1 : 1; + + return 0; +} + +int +ei_cmp_pids(erlang_pid *a, erlang_pid *b) +{ + int differ; + + if (a->serial != b->serial) + return a->serial < b->serial ? -1 : 1; + + if (a->num != b->num) + return a->num < b->num ? -1 : 1; + + differ = cmp_nodes(a->node, a->creation, b->node, b->creation); + if (differ) + return differ; + + return 0; +} + +int +ei_cmp_ports(erlang_port *a, erlang_port *b) +{ + int differ; + + differ = cmp_nodes(a->node, a->creation, b->node, b->creation); + if (differ) + return differ; + + if (a->id != b->id) + return a->id < b->id ? -1 : 1; + + return 0; +} diff --git a/lib/erl_interface/src/misc/ei_decode_term.c b/lib/erl_interface/src/misc/ei_decode_term.c index 79c8fb4388..72f1e0d511 100644 --- a/lib/erl_interface/src/misc/ei_decode_term.c +++ b/lib/erl_interface/src/misc/ei_decode_term.c @@ -47,25 +47,30 @@ int ei_decode_ei_term(const char* buf, int* index, ei_term* term) break; case ERL_FLOAT_EXT: case NEW_FLOAT_EXT: + term->ei_type = ERL_FLOAT_EXT; return (ei_decode_double(buf, index, &term->value.d_val) < 0 ? -1 : 1); case ERL_ATOM_EXT: case ERL_ATOM_UTF8_EXT: case ERL_SMALL_ATOM_EXT: case ERL_SMALL_ATOM_UTF8_EXT: + term->ei_type = ERL_ATOM_EXT; return (ei_decode_atom(buf, index, term->value.atom_name) < 0 ? -1 : 1); case ERL_REFERENCE_EXT: case ERL_NEW_REFERENCE_EXT: case ERL_NEWER_REFERENCE_EXT: + term->ei_type = ERL_NEW_REFERENCE_EXT; return (ei_decode_ref(buf, index, &term->value.ref) < 0 ? -1 : 1); case ERL_PORT_EXT: case ERL_NEW_PORT_EXT: + term->ei_type = ERL_PORT_EXT; return (ei_decode_port(buf, index, &term->value.port) < 0 ? -1 : 1); case ERL_PID_EXT: case ERL_NEW_PID_EXT: + term->ei_type = ERL_PID_EXT; return (ei_decode_pid(buf, index, &term->value.pid) < 0 ? -1 : 1); case ERL_SMALL_TUPLE_EXT: diff --git a/lib/erl_interface/src/misc/ei_portio.h b/lib/erl_interface/src/misc/ei_portio.h index d881a51398..1805411804 100644 --- a/lib/erl_interface/src/misc/ei_portio.h +++ b/lib/erl_interface/src/misc/ei_portio.h @@ -22,6 +22,8 @@ #ifndef _EI_PORTIO_H #define _EI_PORTIO_H +#include <stdint.h> + #undef EI_HAVE_STRUCT_IOVEC__ #if !defined(__WIN32__) && defined(HAVE_SYS_UIO_H) /* Declaration of struct iovec *iov should be visible in this scope. */ @@ -50,12 +52,12 @@ int ei_socket_callbacks_have_writev__(ei_socket_callbacks *cbs); extern ei_socket_callbacks ei_default_socket_callbacks; #define EI_FD_AS_CTX__(FD) \ - ((void *) (long) (FD)) + ((void *) (intptr_t) (FD)) #define EI_DFLT_CTX_TO_FD__(CTX, FD) \ - ((int) (long) (CTX) < 0 \ + ((intptr_t) (CTX) < 0 \ ? EBADF \ - : (*(FD) = (int) (long) (CTX), 0)) + : (*(FD) = (int) (intptr_t) (CTX), 0)) #define EI_GET_FD__(CBS, CTX, FD) \ ((CBS) == &ei_default_socket_callbacks \ diff --git a/lib/erl_interface/src/misc/show_msg.c b/lib/erl_interface/src/misc/show_msg.c index 518d9dd595..c87e73c545 100644 --- a/lib/erl_interface/src/misc/show_msg.c +++ b/lib/erl_interface/src/misc/show_msg.c @@ -129,9 +129,9 @@ int ei_show_sendmsg(FILE *stream, const char *header, const char *msgbuf) /* skip five bytes */ index = 5; - ei_decode_version(header,&index,&version); - ei_decode_tuple_header(header,&index,&arity); - ei_decode_long(header,&index,&msg.msgtype); + if (ei_decode_version(header,&index,&version) + || ei_decode_tuple_header(header,&index,&arity) + || ei_decode_long(header,&index,&msg.msgtype)) return -1; switch (msg.msgtype) { case ERL_SEND: diff --git a/lib/erl_interface/src/prog/erl_call.c b/lib/erl_interface/src/prog/erl_call.c index b0566a0db7..1c9bd69a96 100644 --- a/lib/erl_interface/src/prog/erl_call.c +++ b/lib/erl_interface/src/prog/erl_call.c @@ -80,6 +80,7 @@ struct call_flags { int randomp; int dynamic_name; int use_long_name; /* indicates if -name was used, else -sname or -n */ + int use_localhost_fallback; int debugp; int verbosep; int haltp; @@ -105,6 +106,23 @@ static void* ei_chk_calloc(size_t nmemb, size_t size); static void* ei_chk_realloc(void *old, size_t size); static char* ei_chk_strdup(char *s); +/* Converts the given hostname to a shortname, if required. */ +static void format_node_hostname(const struct call_flags *flags, + const char *hostname, + char dst[EI_MAXHOSTNAMELEN + 1]) +{ + char *ct; + + strncpy(dst, hostname, EI_MAXHOSTNAMELEN); + dst[EI_MAXHOSTNAMELEN] = '\0'; + + /* If shortnames, cut off the name at first '.' */ + if (flags->use_long_name == 0 && (ct = strchr(dst, '.'))) { + *ct = '\0'; + } +} + +static void start_timeout(int timeout); /*************************************************************************** * @@ -119,7 +137,6 @@ int main(int argc, char *argv[]) char host_name[EI_MAXHOSTNAMELEN+1]; char nodename[MAXNODELEN+1]; char *p = NULL; - char *ct = NULL; /* temporary used when truncating nodename */ int modsize = 0; char *host = NULL; char *module = NULL; @@ -177,6 +194,24 @@ int main(int argc, char *argv[]) } i++; } + } else if (strcmp(argv[i], "-timeout") == 0) { + long timeout; + + if (i+1 >= argc) { + usage_arg(progname, "-timeout "); + } + + timeout = strtol(argv[i+1], NULL, 10); + if (timeout <= 0 || timeout >= (1 << 20)) { + usage_error(progname, "-timeout"); + } + + start_timeout(timeout); + i++; + } else if (strcmp(argv[i], "-__uh_test__") == 0) { + /* Fakes a failure in the call to ei_gethostbyname(h_hostname) so + * we can test the localhost fallback. */ + flags.use_localhost_fallback = 1; } else { if (strlen(argv[i]) != 2) { usage_error(progname, argv[i]); @@ -312,23 +347,24 @@ int main(int argc, char *argv[]) char *h_nodename = h_nodename_buf; char *h_alivename = flags.hidden; struct in_addr h_ipadr; - char* ct; /* gethostname requires len to be max(hostname) + 1 */ if (gethostname(h_hostname, EI_MAXHOSTNAMELEN+1) < 0) { - fprintf(stderr,"erl_call: failed to get host name: %d\n", errno); - exit(1); + fprintf(stderr,"erl_call: failed to get host name: %d\n", errno); + exit(1); } - if ((hp = ei_gethostbyname(h_hostname)) == 0) { - fprintf(stderr,"erl_call: can't resolve hostname %s\n", h_hostname); - exit(1); - } - /* If shortnames, cut off the name at first '.' */ - if (flags.use_long_name == 0 && (ct = strchr(hp->h_name, '.')) != NULL) { - *ct = '\0'; + + if (flags.use_localhost_fallback || (hp = ei_gethostbyname(h_hostname)) == 0) { + /* Failed to resolve our own hostname; try binding to loopback and + * hope for the best. */ + hp = ei_gethostbyname("localhost"); + flags.use_localhost_fallback = 1; + + format_node_hostname(&flags, h_hostname, h_hostname); + } else { + format_node_hostname(&flags, hp->h_name, h_hostname); } - strncpy(h_hostname, hp->h_name, EI_MAXHOSTNAMELEN); - h_hostname[EI_MAXHOSTNAMELEN] = '\0'; + memcpy(&h_ipadr.s_addr, *hp->h_addr_list, sizeof(struct in_addr)); if (h_alivename) { if (strlen(h_alivename) + strlen(h_hostname) + 2 > sizeof(h_nodename_buf)) { @@ -364,20 +400,21 @@ int main(int argc, char *argv[]) host = p+1; } - /* - * Expand name to a real name (may be ip-address) - */ - /* FIXME better error string */ - if ((hp = ei_gethostbyname(host)) == 0) { - fprintf(stderr,"erl_call: can't ei_gethostbyname(%s)\n", host); - exit(1); - } - /* If shortnames, cut off the name at first '.' */ - if (flags.use_long_name == 0 && (ct = strchr(hp->h_name, '.')) != NULL) { - *ct = '\0'; + if (flags.use_localhost_fallback && strcmp(host, ei_thishostname(&ec)) == 0) { + /* We're on the same host *and* have used the localhost fallback, so we + * skip canonical name resolution since it's bound to fail. + * + * `ei_connect` will do the right thing later on. */ + strcpy(host_name, ei_thishostname(&ec)); + } else { + if ((hp = ei_gethostbyname(host)) == 0) { + fprintf(stderr,"erl_call: can't ei_gethostbyname(%s)\n", host); + exit(1); + } + + format_node_hostname(&flags, hp->h_name, host_name); } - strncpy(host_name, hp->h_name, EI_MAXHOSTNAMELEN); - host_name[EI_MAXHOSTNAMELEN] = '\0'; + if (flags.port == -1) { if (strlen(flags.node) + strlen(host_name) + 2 > sizeof(nodename)) { fprintf(stderr,"erl_call: nodename too long: %s\n", flags.node); @@ -832,6 +869,25 @@ static int get_module(char **mbuf, char **mname) } /* get_module */ +#ifdef __WIN32__ +static DWORD WINAPI timer_thread(void *data) { + DWORD_PTR timeout = (DWORD_PTR)data * 1000; + + Sleep(timeout); + exit(1); +} + +static void start_timeout(int timeout) { + if (CreateThread(NULL, 0, timer_thread, (void*)timeout, 0, NULL) == NULL) { + fprintf(stderr,"erl_call: Failed to start timer thread\n"); + exit(1); + } +} +#else +static void start_timeout(int timeout) { + alarm(timeout); +} +#endif /*************************************************************************** * @@ -841,7 +897,7 @@ static int get_module(char **mbuf, char **mname) static void usage_noexit(const char *progname) { fprintf(stderr,"\nUsage: %s [-[demqrsv]] [-c Cookie] [-h HiddenName] \n", progname); - fprintf(stderr," [-x ErlScript] [-a [Mod [Fun [Args]]]]\n"); + fprintf(stderr," [-x ErlScript] [-a [Mod [Fun [Args]]]] [-timeout Secs]\n"); fprintf(stderr," (-n Node | -sname Node | -name Node | -address [HOSTNAME:]PORT)\n\n"); #ifdef __WIN32__ fprintf(stderr," where: -a apply(Mod,Fun,Args) (e.g -a \"erlang length [[a,b,c]]\"\n"); @@ -862,6 +918,7 @@ static void usage_noexit(const char *progname) { " (e.g., %s -address my_host:36303 ...)\n" " (cannot be combinated with -s, -n, -name and -sname)\n", progname); + fprintf(stderr," -timeout command timeout, in seconds\n"); fprintf(stderr," -q halt the Erlang node (overrides the -s switch)\n"); fprintf(stderr," -r use a random name for the erl_call client node\n"); fprintf(stderr," -s start a new Erlang node if necessary\n"); diff --git a/lib/erl_interface/src/prog/erl_start.c b/lib/erl_interface/src/prog/erl_start.c index 8eca2dfa10..9c876feb5e 100644 --- a/lib/erl_interface/src/prog/erl_start.c +++ b/lib/erl_interface/src/prog/erl_start.c @@ -80,7 +80,7 @@ static struct in_addr *get_addr(const char *hostname, struct in_addr *oaddr); static int wait_for_erlang(int sockd, int magic, struct timeval *timeout); #if defined(__WIN32__) static int unique_id(void); -static unsigned long spawn_erlang_epmd(ei_cnode *ec, +static HANDLE spawn_erlang_epmd(ei_cnode *ec, char *alive, Erl_IpAddr adr, int flags, @@ -116,9 +116,9 @@ int erl_start_sys(ei_cnode *ec, char *alive, Erl_IpAddr adr, int flags, int sockd = 0; int one = 1; #if defined(__WIN32__) - unsigned long pid = 0; + HANDLE pid; #else - int pid = 0; + int pid; #endif int r = 0; @@ -145,8 +145,8 @@ int erl_start_sys(ei_cnode *ec, char *alive, Erl_IpAddr adr, int flags, listen(sockd,5); #if defined(__WIN32__) - if((pid = spawn_erlang_epmd(ec,alive,adr,flags,erl,args,port,1)) - == 0) + pid = spawn_erlang_epmd(ec,alive,adr,flags,erl,args,port,1); + if (pid == INVALID_HANDLE_VALUE) return ERL_SYS_ERROR; timeout.tv_usec = 0; timeout.tv_sec = 10; /* ignoring ERL_START_TIME */ @@ -154,7 +154,7 @@ int erl_start_sys(ei_cnode *ec, char *alive, Erl_IpAddr adr, int flags, == ERL_TIMEOUT) { /* Well, this is not a nice way to do it, and it does not always kill the emulator, but the alternatives are few.*/ - TerminateProcess((HANDLE) pid,1); + TerminateProcess(pid,1); } #else /* Unix */ switch ((pid = fork())) { @@ -269,7 +269,7 @@ static void free_args(char **args){ /* In NT we cannot fork(), Erlang and Epmd gets spawned by this function instead. */ -static unsigned long spawn_erlang_epmd(ei_cnode *ec, +static HANDLE spawn_erlang_epmd(ei_cnode *ec, char *alive, Erl_IpAddr adr, int flags, @@ -290,18 +290,18 @@ static unsigned long spawn_erlang_epmd(ei_cnode *ec, struct in_addr myaddr; struct in_addr *hisaddr = (struct in_addr *)adr; char iaddrbuf[IP_ADDR_CHARS + 1]; - int ret; + HANDLE ret; if(is_erlang){ get_addr(ei_thishostname(ec), &myaddr); if((ptr = inet_ntoa(myaddr)) == NULL) - return 0; + return INVALID_HANDLE_VALUE; else strcpy(iaddrbuf,ptr); } if ((flags & ERL_START_REMOTE) || (is_erlang && (hisaddr->s_addr != myaddr.s_addr))) { - return 0; + return INVALID_HANDLE_VALUE; } else { num_args = enquote_args(args, &args); for(cmdlen = i = 0; args[i] != NULL; ++i) @@ -364,9 +364,9 @@ static unsigned long spawn_erlang_epmd(ei_cnode *ec, NULL, &sinfo, &pinfo)) - ret = 0; + ret = INVALID_HANDLE_VALUE; else - ret = (unsigned long) pinfo.hProcess; + ret = pinfo.hProcess; free(cmdbuf); return ret; } diff --git a/lib/erl_interface/src/registry/reg_dump.c b/lib/erl_interface/src/registry/reg_dump.c index a027c4cdf5..f90fd4d4b6 100644 --- a/lib/erl_interface/src/registry/reg_dump.c +++ b/lib/erl_interface/src/registry/reg_dump.c @@ -209,11 +209,11 @@ static int mn_send_write(int fd, erlang_pid *mnesia, const char *key, ei_reg_obj break; case EI_STR: if (obj->size > 0) ei_encode_string(msgbuf,&index,obj->val.s); - else ei_encode_long(msgbuf,&index, (long)NULL); /* just the NULL pointer */ + else ei_encode_long(msgbuf,&index, 0); /* just the NULL pointer */ break; case EI_BIN: if (obj->size > 0) ei_encode_binary(msgbuf,&index,obj->val.p,obj->size); - else ei_encode_long(msgbuf,&index,(long)(obj->val.p)); /* just the pointer */ + else ei_encode_long(msgbuf,&index, obj->val.i); /* just the pointer */ break; default: if (dbuf) free(dbuf); @@ -255,7 +255,7 @@ static int mn_get_unlink(int fd) int ei_reg_dump(int fd, ei_reg *reg, const char *mntab, int flags) { ei_hash *tab; - erlang_pid self; + erlang_pid *self; erlang_pid mnesia; ei_bucket *b; ei_reg_obj *obj; @@ -271,12 +271,10 @@ int ei_reg_dump(int fd, ei_reg *reg, const char *mntab, int flags) if ((ec = ei_fd_to_cnode(fd)) == NULL) { return -1; } - strcpy(self.node,ei_thisnodename(ec)); - self.num = fd; - self.serial = 0; - self.creation = ei_thiscreation(ec); - if (mn_start_dump(fd,&self,&mnesia,mntab)) return -1; + self = ei_self(ec); + + if (mn_start_dump(fd,self,&mnesia,mntab)) return -1; /* traverse the table, passing objects to mnesia */ for (i=0; i<tab->size; i++) { @@ -288,13 +286,13 @@ int ei_reg_dump(int fd, ei_reg *reg, const char *mntab, int flags) if ((flags & EI_FORCE) || (obj->attr & EI_DIRTY)) { if (obj->attr & EI_DELET) { if (mn_send_delete(fd,&mnesia,key)) { - ei_send_exit(fd,&self,&mnesia,"delete failed"); + ei_send_exit(fd,self,&mnesia,"delete failed"); return -1; } } else { if (mn_send_write(fd,&mnesia,key,obj)) { - ei_send_exit(fd,&self,&mnesia,"update failed"); + ei_send_exit(fd,self,&mnesia,"update failed"); return -1; } } @@ -304,8 +302,8 @@ int ei_reg_dump(int fd, ei_reg *reg, const char *mntab, int flags) } /* end the transaction */ - if (mn_send_commit(fd,&mnesia,&self)) { - ei_send_exit(fd,&self,&mnesia,"commit failed"); + if (mn_send_commit(fd,&mnesia,self)) { + ei_send_exit(fd,self,&mnesia,"commit failed"); return -1; } diff --git a/lib/erl_interface/src/registry/reg_restore.c b/lib/erl_interface/src/registry/reg_restore.c index 75d073303f..030bab19b9 100644 --- a/lib/erl_interface/src/registry/reg_restore.c +++ b/lib/erl_interface/src/registry/reg_restore.c @@ -227,7 +227,7 @@ int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) char *dbuf = NULL; char *msgbuf = NULL; char *keybuf = NULL; - erlang_pid self; + erlang_pid *self; erlang_pid mnesia = {"",0,0,0}; erlang_msg msg; int index = 0; @@ -247,20 +247,18 @@ int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) if ((ec = ei_fd_to_cnode(fd)) == NULL) { return -1; } - strcpy(self.node,ei_thisnodename(ec)); - self.num = fd; - self.serial = 0; - self.creation = ei_thiscreation(ec); + + self = ei_self(ec); - if (mn_start_restore(fd,&self,&mnesia,mntab,&count,&maxkey,&maxobj)) { + if (mn_start_restore(fd,self,&mnesia,mntab,&count,&maxkey,&maxobj)) { /* send exit *only* if we have pid */ - if (mnesia.node[0]) ei_send_exit(fd,&self,&mnesia,"bad response from rpc start"); + if (mnesia.node[0]) ei_send_exit(fd,self,&mnesia,"bad response from rpc start"); return -1; } if (count <= 0) { - ei_send_exit(fd,&self,&mnesia,"nothing to do"); + ei_send_exit(fd,self,&mnesia,"nothing to do"); return 0; } @@ -268,7 +266,7 @@ int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) len = maxkey + maxobj + 512; if (len > EISMALLBUF) if (!(dbuf = malloc(len))) { - ei_send_exit(fd,&self,&mnesia,"cannot allocate space for incoming data"); + ei_send_exit(fd,self,&mnesia,"cannot allocate space for incoming data"); return -1; } msgbuf = (dbuf ? dbuf : sbuf); @@ -281,7 +279,7 @@ int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) ei_encode_version(msgbuf,&index); ei_encode_tuple_header(msgbuf,&index,2); ei_encode_atom(msgbuf,&index,"send_records"); - ei_encode_pid(msgbuf,&index,&self); + ei_encode_pid(msgbuf,&index,self); if (ei_send_encoded(fd,&mnesia,msgbuf,index)) goto restore_failure; /* read as much as possible, until count or EXIT */ @@ -317,7 +315,7 @@ int ei_reg_restore(int fd, ei_reg *reg, const char *mntab) return 0; restore_failure: - ei_send_exit(fd,&self,&mnesia,"restore failure"); + ei_send_exit(fd,self,&mnesia,"restore failure"); if (keybuf) free(keybuf); if (dbuf) free(dbuf); return -1; diff --git a/lib/erl_interface/test/Makefile b/lib/erl_interface/test/Makefile index 1816c73d4b..bdfedecc66 100644 --- a/lib/erl_interface/test/Makefile +++ b/lib/erl_interface/test/Makefile @@ -24,7 +24,7 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk # Target Specs # ---------------------------------------------------- -MODULES= \ +EI_MODULES= \ ei_accept_SUITE \ ei_connect_SUITE \ ei_decode_SUITE \ @@ -38,12 +38,16 @@ MODULES= \ port_call_SUITE \ runner +ERTS_MODULES= erts_test_utils + +MODULES=$(EI_MODULES) $(ERTS_MODULES) + SPEC_FILES = \ erl_interface.spec erl_interface_smoke.spec COVER_FILE = erl_interface.cover -ERL_FILES = $(MODULES:%=%.erl) +ERL_FILES = $(EI_MODULES:%=%.erl) $(ERTS_MODULES:%=$(ERL_TOP)/erts/emulator/test/%.erl) # ---------------------------------------------------- # Release directory specification diff --git a/lib/erl_interface/test/ei_connect_SUITE.erl b/lib/erl_interface/test/ei_connect_SUITE.erl index 224608ba71..0506359b71 100644 --- a/lib/erl_interface/test/ei_connect_SUITE.erl +++ b/lib/erl_interface/test/ei_connect_SUITE.erl @@ -34,7 +34,9 @@ ei_send_funs/1, ei_threaded_send/1, ei_set_get_tracelevel/1, - ei_connect_host_port_test/1]). + ei_connect_host_port_test/1, + ei_make_ref/1, + ei_make_pid/1]). -import(runner, [get_term/1,send_term/2]). @@ -54,7 +56,9 @@ groups() -> ei_send_funs, ei_set_get_tracelevel, ei_reg_send, - ei_rpc], + ei_rpc, + ei_make_ref, + ei_make_pid], [{default, [], Members}, {ussi, [], Members}]. @@ -207,6 +211,100 @@ ei_connect_host_port_test(Config) when is_list(Config) -> runner:recv_eot(P), ok. +ei_make_ref(Config) when is_list(Config) -> + P = runner:start(Config, ?interpret), + 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0, get_group(Config)), + {ok,Fd} = ei_connect(P, node()), + + %% Call ei_make_ref() enough times for it to + %% wrap the first internal integer.. + + N = 270, + {CNode, Refs} = make_refs(N, undefined, P, Fd, []), + io:format("Last Ref ~p~n", [hd(Refs)]), + + io:format("CNode = ~p", [CNode]), + + true = lists:member(CNode, nodes(hidden)), + + %% Ensure that all references are + %% unique... + RefsLen = N*1000, + RefsLen = length(lists:usort(Refs)), + + runner:send_eot(P), + runner:recv_eot(P), + ok. + +make_refs(0, CNode, _P, _Fd, Refs) -> + {CNode, Refs}; +make_refs(N, CNode, P, Fd, Refs) -> + ok = ei_make_refs(P, Fd, self()), + receive + {Node, NewRefs} -> + NewNode = if CNode == undefined -> + Node; + true -> + CNode = Node + end, + make_refs(N-1, NewNode, P, Fd, + chk_refs(NewRefs, NewNode, Refs)) + end. + +chk_refs([], _CNode, Refs) -> + Refs; +chk_refs([NewRef|NewRefs], CNode, Refs) -> + true = is_reference(NewRef), + CNode = node(NewRef), + chk_refs(NewRefs, CNode, [NewRef|Refs]). + +ei_make_pid(Config) when is_list(Config) -> + P = runner:start(Config, ?interpret), + 0 = ei_connect_init(P, 42, erlang:get_cookie(), 0, get_group(Config)), + {ok,Fd} = ei_connect(P, node()), + + %% + %% Ensure to wrap all num values... + %% + N = 200, + {CNode, Pids} = make_pids(N, undefined, P, Fd, []), + io:format("Last Pid ~p~n", [hd(Pids)]), + + io:format("CNode = ~p", [CNode]), + + true = lists:member(CNode, nodes(hidden)), + + %% Ensure that all pid created by ei_make_pid() + %% are unique. Note that ei_self() is passed + %% along in each call as well... + PidsLen = N*1000 + 1, + PidsLen = length(lists:usort(Pids)), + + runner:send_eot(P), + runner:recv_eot(P), + ok. + +make_pids(0, CNode, _P, _Fd, Pids) -> + {CNode, Pids}; +make_pids(N, CNode, P, Fd, Pids) -> + ok = ei_make_pids(P, Fd, self()), + receive + {Node, NewPids} -> + NewNode = if CNode == undefined -> + Node; + true -> + CNode = Node + end, + make_pids(N-1, NewNode, P, Fd, + chk_pids(NewPids, NewNode, Pids)) + end. + +chk_pids([], _CNode, Pids) -> + Pids; +chk_pids([NewPid|NewPids], CNode, Pids) -> + true = is_pid(NewPid), + CNode = node(NewPid), + chk_pids(NewPids, CNode, [NewPid|Pids]). %%% Interface functions for ei (erl_interface) functions. @@ -256,6 +354,14 @@ ei_rpc(P, Fd, To, Func, Msg) -> send_command(P, ei_rpc, [Fd, To, Func, Msg]), get_term(P). +ei_make_refs(P, Fd, To) -> + send_command(P, ei_make_refs, [Fd,To]), + get_send_result(P). + +ei_make_pids(P, Fd, To) -> + send_command(P, ei_make_pids, [Fd,To]), + get_send_result(P). + get_send_result(P) -> case get_term(P) of diff --git a/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c b/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c index 1dcd62400a..0aa6879adf 100644 --- a/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c +++ b/lib/erl_interface/test/ei_connect_SUITE_data/ei_connect_test.c @@ -40,6 +40,8 @@ static void cmd_ei_send_funs(char* buf, int len); static void cmd_ei_reg_send(char* buf, int len); static void cmd_ei_rpc(char* buf, int len); static void cmd_ei_set_get_tracelevel(char* buf, int len); +static void cmd_ei_make_refs(char* buf, int len); +static void cmd_ei_make_pids(char* buf, int len); static void send_errno_result(int value); @@ -60,6 +62,8 @@ static struct { "ei_rpc", 4, cmd_ei_rpc, "ei_set_get_tracelevel", 1, cmd_ei_set_get_tracelevel, "ei_format_pid", 2, cmd_ei_format_pid, + "ei_make_refs", 2, cmd_ei_make_refs, + "ei_make_pids", 2, cmd_ei_make_pids, }; @@ -210,6 +214,80 @@ static void cmd_ei_send(char* buf, int len) ei_x_free(&x); } +static void cmd_ei_make_refs(char* buf, int len) +{ + int index = 0; + long fd; + erlang_pid pid; + ei_x_buff x; + int i; + int nref = 1000; + + if (ei_decode_long(buf, &index, &fd) < 0) + fail("expected long"); + if (ei_decode_pid(buf, &index, &pid) < 0) + fail("expected pid (node)"); + if (ei_x_new_with_version(&x) < 0) + fail("ei_x_new_with_version"); + if (ei_x_encode_tuple_header(&x, 2) < 0) + fail("ei_x_encode_tuple_header() failed"); + if (ei_x_encode_atom(&x, ei_thisnodename(&ec)) < 0) + fail("ei_x_encode_atom() failed"); + if (ei_x_encode_list_header(&x, nref) < 0) + fail("ei_x_encode_list_header() failed"); + for (i = 0; i < nref; i++) { + erlang_ref ref; + if (ei_make_ref(&ec, &ref)) + fail("ei_make_ref() failed"); + if (ei_x_encode_ref(&x, &ref)) + fail("ei_x_encode_ref() failed"); + } + if (ei_x_encode_empty_list(&x) < 0) + fail("ei_x_encode_empty_list() failed"); + send_errno_result(ei_send(fd, &pid, x.buff, x.index)); + ei_x_free(&x); +} + +static void cmd_ei_make_pids(char* buf, int len) +{ + int index = 0; + long fd; + erlang_pid from_pid; + erlang_pid *self; + ei_x_buff x; + int i; + int npid = 1000; + + if (ei_decode_long(buf, &index, &fd) < 0) + fail("expected long"); + if (ei_decode_pid(buf, &index, &from_pid) < 0) + fail("expected pid (node)"); + if (ei_x_new_with_version(&x) < 0) + fail("ei_x_new_with_version"); + if (ei_x_encode_tuple_header(&x, 2) < 0) + fail("ei_x_encode_tuple_header() failed"); + if (ei_x_encode_atom(&x, ei_thisnodename(&ec)) < 0) + fail("ei_x_encode_atom() failed"); + if (ei_x_encode_list_header(&x, 1+npid) < 0) + fail("ei_x_encode_list_header() failed"); + self = ei_self(&ec); + if (!self) + fail("ei_self() failed"); + if (ei_x_encode_pid(&x, self)) + fail("ei_x_encode_pid() failed"); + for (i = 0; i < npid; i++) { + erlang_pid pid; + if (ei_make_pid(&ec, &pid)) + fail("ei_make_pid() failed"); + if (ei_x_encode_pid(&x, &pid)) + fail("ei_x_encode_pid() failed"); + } + if (ei_x_encode_empty_list(&x) < 0) + fail("ei_x_encode_empty_list() failed"); + send_errno_result(ei_send(fd, &from_pid, x.buff, x.index)); + ei_x_free(&x); +} + static void cmd_ei_format_pid(char* buf, int len) { int index = 0; diff --git a/lib/erl_interface/test/ei_decode_SUITE.erl b/lib/erl_interface/test/ei_decode_SUITE.erl index 475e68ced2..fee99aba7c 100644 --- a/lib/erl_interface/test/ei_decode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_SUITE.erl @@ -33,7 +33,9 @@ test_ei_decode_char/1, test_ei_decode_nonoptimal/1, test_ei_decode_misc/1, - test_ei_decode_utf8_atom/1]). + test_ei_decode_utf8_atom/1, + test_ei_decode_iodata/1, + test_ei_cmp_nc/1]). suite() -> [{ct_hooks,[ts_install_cth]}]. @@ -41,7 +43,8 @@ all() -> [test_ei_decode_long, test_ei_decode_ulong, test_ei_decode_longlong, test_ei_decode_ulonglong, test_ei_decode_char, test_ei_decode_nonoptimal, - test_ei_decode_misc, test_ei_decode_utf8_atom]. + test_ei_decode_misc, test_ei_decode_utf8_atom, + test_ei_decode_iodata, test_ei_cmp_nc]. init_per_testcase(Case, Config) -> runner:init_per_testcase(?MODULE, Case, Config). @@ -215,6 +218,160 @@ test_ei_decode_utf8_atom(Config) -> %% ######################################################################## %% +test_ei_decode_iodata(Config) when is_list(Config) -> + P = runner:start(Config, ?test_ei_decode_iodata), + + check_decode_iodata(P, [], true), + check_decode_iodata(P, $a, false), + check_decode_iodata(P, an_atom, false), + check_decode_iodata(P, self(), false), + check_decode_iodata(P, [$a,$a], true), + check_decode_iodata(P, [$a|$a], false), + check_decode_iodata(P, [[$a|$a],$a], false), + check_decode_iodata(P, "hej", true), + check_decode_iodata(P, ["hej", " ", "hopp"], true), + check_decode_iodata(P, <<"hopp san sa">>, true), + check_decode_iodata(P, [$a | <<"a">>], true), + check_decode_iodata(P, [[[["hej"]]], [$ , <<"hopp">>, $ , "san" | <<" sa">>]], true), + check_decode_iodata(P, [[[["hej"]]], [$ , <<"hopp">>, 0, "san" | <<" sa">>]], true), + check_decode_iodata(P, [[[["hej"]]], [$ , <<"hopp">>, 256, "san" | <<" sa">>]], false), + check_decode_iodata(P, [[[["hej"]]], [$ , <<"hopp">>, -2, "san" | <<" sa">>]], false), + check_decode_iodata(P, [[[["hej"]]], [$ , <<"hopp">>, $ , san | <<" sa">>]], false), + check_decode_iodata(P, [[[["hej"]]], [$ , <<"hopp">>, $ , "san s" | $a], " "], false), + check_decode_iodata(P, [[[[[[[]|<<"a">>]]]]]], true), + check_decode_iodata(P, [[[[[[[[]]]]]]],[[[],[],[]]]], true), + + send_raw(P, <<"done">>), + runner:recv_eot(P), + ok. + +check_decode_iodata(P, Data, Valid) -> + io:format("~n~nChecking: ~p~n", [Data]), + Expect = case Valid of + true -> + io:format("Expecting decode SUCCESS... ", []), + iolist_to_binary(Data); + false -> + io:format("Expecting decode FAILURE... ", []), + badarg = try + iolist_to_binary(Data) + catch + error:badarg -> + badarg + end, + decode_size_failed + end, + send_term_as_binary(P, Data), + Actual = case runner:get_term(P) of + {bytes, B} when is_binary(B) -> + B; + {bytes, L} when is_list(L) -> + list_to_binary(L); + {term, T} -> + T + end, + case Expect =:= Actual of + true -> + io:format("Expected result!~n",[]), + ok; + false -> + io:format("Expect: ~w~nActual: ~w~n", [Expect, Actual]), + ct:fail(unexpected_result) + end. + +%% ######################################################################## %% + +%% Should be moved to its own suite... + +test_ei_cmp_nc(Config) when is_list(Config) -> + P = runner:start(Config, ?test_ei_cmp_nc), + R0 = make_ref(), + R1 = make_ref(), + check_cmp(P, R0, R0), + check_cmp(P, R0, R1), + check_cmp(P, R1, R0), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17, 17]), mk_ref({a@c, 4711}, [17, 17, 17])), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17, 17]), mk_ref({a@d, 4711}, [17, 17, 17])), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17, 17]), mk_ref({a@bc, 4711}, [17, 17, 17])), + check_cmp(P, mk_ref({a@b, 4712}, [17, 17, 17]), mk_ref({a@b, 4711}, [17, 17, 17])), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17, 17]), mk_ref({a@b, 4711}, [18, 17, 17])), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17, 17]), mk_ref({a@b, 4711}, [17, 18, 17])), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17, 17]), mk_ref({a@b, 4711}, [17, 17, 18])), + check_cmp(P, mk_ref({a@b, 4711}, [0, 17]), mk_ref({a@b, 4711}, [18])), + check_cmp(P, mk_ref({a@b, 4711}, [17, 17]), mk_ref({a@b, 4711}, [17, 18])), + check_cmp(P, mk_ref({a@b, 4711}, [0, 0, 0]), mk_ref({a@b, 4711}, [17])), + check_cmp(P, mk_ref({a@b, 4711}, [0, 0, 17]), mk_ref({a@b, 4711}, [18, 17])), + check_cmp(P, mk_ref({a@b, 4711}, [0, 17, 17]), mk_ref({a@b, 4711}, [18])), + + check_cmp(P, self(), self()), + check_cmp(P, self(), whereis(file_server_2)), + check_cmp(P, whereis(file_server_2), self()), + check_cmp(P, mk_pid({a@b, 4711}, 17, 17), mk_pid({a@c, 4711}, 17, 17)), + check_cmp(P, mk_pid({a@b, 4711}, 17, 17), mk_pid({a@d, 4711}, 17, 17)), + check_cmp(P, mk_pid({a@b, 4711}, 17, 17), mk_pid({a@bc, 4711}, 17, 17)), + check_cmp(P, mk_pid({a@b, 4712}, 17, 17), mk_pid({a@b, 4711}, 17, 17)), + check_cmp(P, mk_pid({a@b, 4711}, 17, 17), mk_pid({a@b, 4711}, 18, 17)), + check_cmp(P, mk_pid({a@b, 4711}, 17, 17), mk_pid({a@b, 4711}, 17, 18)), + + Prt0 = open_port({spawn, "true"},[]), + Prt1 = open_port({spawn, "true"},[]), + + check_cmp(P, Prt0, Prt0), + check_cmp(P, Prt1, Prt0), + check_cmp(P, Prt0, Prt1), + check_cmp(P, mk_port({a@b, 4711}, 17), mk_port({a@b, 4711}, 17)), + check_cmp(P, mk_port({a@b, 4711}, 17), mk_port({a@d, 4711}, 17)), + check_cmp(P, mk_port({a@b, 4711}, 17), mk_port({a@bc, 4711}, 17)), + check_cmp(P, mk_port({a@b, 4712}, 17), mk_port({a@b, 4711}, 17)), + check_cmp(P, mk_port({a@b, 4711}, 17), mk_port({a@b, 4711}, 18)), + + send_raw(P, <<"done">>), + runner:recv_eot(P), + ok. + +mk_pid(Node, Num, Ser) -> + erts_test_utils:mk_ext_pid(Node, Num, Ser). + +mk_port(Node, Id) -> + erts_test_utils:mk_ext_port(Node, Id). + +mk_ref(Node, Numbers) -> + erts_test_utils:mk_ext_ref(Node, Numbers). + +check_cmp(P, A, B) when is_pid(A), is_pid(B) -> + check_cmp(P, {cmp_pids, A, B}); +check_cmp(P, A, B) when is_port(A), is_port(B) -> + check_cmp(P, {cmp_ports, A, B}); +check_cmp(P, A, B) when is_reference(A), is_reference(B) -> + check_cmp(P, {cmp_refs, A, B}). + +check_cmp(P, {_, A, B} = Data) -> + io:format("~n~nChecking: ~p~n", [Data]), + send_term_as_binary(P, Data), + {term, Res} = runner:get_term(P), + io:format("Res = ~p~n", [Res]), + case {{ei_cmp, Res}, {erlang, cmp_nc(A, B)}} of + {{ei_cmp, 0}, {erlang, equal}} -> + ok; + {{ei_cmp, Cmp}, {erlang, less_than}} when is_integer(Cmp), + Cmp < 0 -> + ok; + {{ei_cmp, Cmp}, {erlang, larger_than}} when is_integer(Cmp), + Cmp > 0 -> ok; + Fail -> + ct:fail(Fail) + end. + +cmp_nc(A, A) -> + equal; +cmp_nc(A, B) when A < B -> + less_than; +cmp_nc(_, _) -> + larger_than. + + +%% ######################################################################## %% + send_term_as_binary(Port, Term) when is_port(Port) -> Port ! {self(), {command, term_to_binary(Term)}}. diff --git a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c index 29542b1c6c..9eed34b0dd 100644 --- a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c +++ b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c @@ -19,6 +19,7 @@ */ #include <string.h> +#include <stdlib.h> #include "ei_runner.h" @@ -753,6 +754,192 @@ TESTCASE(test_ei_decode_utf8_atom) /* ******************************************************************** */ +TESTCASE(test_ei_decode_iodata) +{ + char *buf = NULL, *data = NULL; + ei_init(); + + while (1) { + int unexpected_write = 0; + int i; + int len, index, saved_index, err; + + if (buf) + free_packet(buf); + buf = read_packet(&len); + + if (len == 4 + && buf[0] == 'd' + && buf[1] == 'o' + && buf[2] == 'n' + && buf[3] == 'e') { + break; + } + + index = 0; + err = ei_decode_version(buf, &index, NULL); + if (err != 0) { + free_packet(buf); + fail1("ei_decode_version returned %d", err); + } + saved_index = index; + err = ei_decode_iodata(buf, &index, &len, NULL); + if (err != 0) { + ei_x_buff x; + ei_x_new_with_version(&x); + ei_x_encode_atom(&x, "decode_size_failed"); + send_bin_term(&x); + ei_x_free(&x); + continue; + } + if (data) { + data -= 100; + free(data); + } + data = malloc(len + 200); + if (!data) { + ei_x_buff x; + ei_x_new_with_version(&x); + ei_x_encode_atom(&x, "malloc_failed"); + send_bin_term(&x); + ei_x_free(&x); + continue; + } + for (i = 0; i < len + 200; i++) + data[i] = 'Y'; + data += 100; + err = ei_decode_iodata(buf, &saved_index, NULL, (unsigned char *) data); + if (err != 0) { + ei_x_buff x; + ei_x_new_with_version(&x); + ei_x_encode_atom(&x, "decode_data_failed"); + send_bin_term(&x); + ei_x_free(&x); + continue; + } + + for (i = -100; i < 0; i++) { + if (data[i] != 'Y') { + ei_x_buff x; + ei_x_new_with_version(&x); + ei_x_encode_atom(&x, "unexpected_write_before_data"); + send_bin_term(&x); + ei_x_free(&x); + unexpected_write = !0; + break; + } + } + + if (!unexpected_write) { + for (i = len; i < len + 100; i++) { + if (data[i] != 'Y') { + ei_x_buff x; + ei_x_new_with_version(&x); + ei_x_encode_atom(&x, "unexpected_write_after_data"); + send_bin_term(&x); + ei_x_free(&x); + unexpected_write = !0; + break; + } + } + } + + if (!unexpected_write) + send_buffer(data, len); + } + + if (buf) + free_packet(buf); + if (data) { + data -= 100; + free(data); + } + report(1); +} + +/* ******************************************************************** */ + +/* + * Does not belong here move to its own suite... + */ +TESTCASE(test_ei_cmp_nc) +{ + char *buf = NULL; + ei_init(); + + while (1) { + int len, index, arity; + char atom[MAXATOMLEN_UTF8]; + ei_x_buff x; + + if (buf) + free_packet(buf); + buf = read_packet(&len); + + if (len == 4 + && buf[0] == 'd' + && buf[1] == 'o' + && buf[2] == 'n' + && buf[3] == 'e') { + break; + } + + ei_x_new_with_version(&x); + index = 0; + if (ei_decode_version(buf, &index, NULL) + || ei_decode_tuple_header(buf, &index, &arity) + || (arity != 3) + || ei_decode_atom(buf, &index, atom)) { + ei_x_encode_atom(&x, "decode_tuple_failed"); + } + else if (strcmp(atom, "cmp_pids") == 0) { + erlang_pid a, b; + if (ei_decode_pid(buf, &index, &a) + || ei_decode_pid(buf, &index, &b)) { + ei_x_encode_atom(&x, "decode_pids_failed"); + } + else { + long res = (long) ei_cmp_pids(&a, &b); + ei_x_encode_long(&x, res); + } + } + else if (strcmp(atom, "cmp_ports") == 0) { + erlang_port a, b; + if (ei_decode_port(buf, &index, &a) + || ei_decode_port(buf, &index, &b)) { + ei_x_encode_atom(&x, "decode_ports_failed"); + } + else { + long res = (long) ei_cmp_ports(&a, &b); + ei_x_encode_long(&x, res); + } + } + else if (strcmp(atom, "cmp_refs") == 0) { + erlang_ref a, b; + if (ei_decode_ref(buf, &index, &a) + || ei_decode_ref(buf, &index, &b)) { + ei_x_encode_atom(&x, "decode_refs_failed"); + } + else { + long res = (long) ei_cmp_refs(&a, &b); + ei_x_encode_long(&x, res); + } + } + else { + ei_x_encode_atom(&x, "unexpected_operation"); + } + + send_bin_term(&x); + ei_x_free(&x); + } + + if (buf) + free_packet(buf); + report(1); +} + +/* ******************************************************************** */ + int ei_decode_my_atom_as(const char *buf, int *index, char *to, struct my_atom *atom) { erlang_char_encoding was,result; diff --git a/lib/erl_interface/test/ei_global_SUITE_data/ei_global_test.c b/lib/erl_interface/test/ei_global_SUITE_data/ei_global_test.c index 4c018667fe..dd41afe77e 100644 --- a/lib/erl_interface/test/ei_global_SUITE_data/ei_global_test.c +++ b/lib/erl_interface/test/ei_global_SUITE_data/ei_global_test.c @@ -211,6 +211,7 @@ cmd_ei_global_names(char* buf, int len) send_bin_term(&x); ei_x_free(&x); } + free(names); } static void diff --git a/lib/erl_interface/test/erl_call_SUITE.erl b/lib/erl_interface/test/erl_call_SUITE.erl index a35e027cd7..339131fed6 100644 --- a/lib/erl_interface/test/erl_call_SUITE.erl +++ b/lib/erl_interface/test/erl_call_SUITE.erl @@ -25,12 +25,16 @@ -export([all/0, smoke/1, random_cnode_name/1, - test_connect_to_host_port/1]). + test_connect_to_host_port/1, + unresolvable_hostname/1, + timeout/1]). all() -> [smoke, random_cnode_name, - test_connect_to_host_port]. + test_connect_to_host_port, + unresolvable_hostname, + timeout]. smoke(Config) when is_list(Config) -> Name = atom_to_list(?MODULE) @@ -115,6 +119,40 @@ test_connect_to_host_port_do(Name) -> end, ok. +%% OTP-16604: Tests that erl_call works even when the local hostname cannot be +%% resolved. +unresolvable_hostname(_Config) -> + Name = atom_to_list(?MODULE) + ++ "-" + ++ integer_to_list(erlang:system_time(microsecond)), + Opt = "-__uh_test__", + + try + CNodeName = start_node_and_get_c_node_name(Name, [Opt]), + [_, Hostname] = string:lexemes(atom_to_list(node()), "@"), + DefaultName = list_to_atom("c17@" ++ Hostname), + check_eq(CNodeName, DefaultName) + after + halt_node(Name) + end, + + ok. + +%% OTP-16604: Test the -timeout option +timeout(_Config) -> + Name = atom_to_list(?MODULE) + ++ "-" + ++ integer_to_list(erlang:system_time(microsecond)), + Opts = ["-timeout", "3"], + + try + [] = start_node_and_apply(Name, "timer sleep [10000]", Opts) + after + halt_node(Name) + end, + + ok. + % % Utility functions... % diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 31e3d1c0a0..67aac42e4e 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1,2 +1,2 @@ -EI_VSN = 3.13.2 +EI_VSN = 4.0 ERL_INTERFACE_VSN = $(EI_VSN) diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index 200ffc3714..b015fe1f89 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -33,6 +33,35 @@ </header> <p>This document describes the changes made to the EUnit application.</p> +<section><title>Eunit 2.5</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Let <c>eunit_surefire</c> skip invalid XML 1.0 + characters. </p> + <p> + Own Id: OTP-15950 Aux Id: PR-2316, ERL-991 </p> + </item> + <item> + <p> + Add new macro ?capturedOutput for enabling to write test + cases that verify data printed to standard out</p> + <p> + Own Id: OTP-16275 Aux Id: PR-2424 </p> + </item> + <item> + <p> + Add option to limit print depth of exceptions generated + by eunit test suites.</p> + <p> + Own Id: OTP-16549 Aux Id: PR-2532 </p> + </item> + </list> + </section> + +</section> + <section><title>Eunit 2.4.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/eunit/vsn.mk b/lib/eunit/vsn.mk index f96db657cf..b8410e4071 100644 --- a/lib/eunit/vsn.mk +++ b/lib/eunit/vsn.mk @@ -1 +1 @@ -EUNIT_VSN = 2.4.1 +EUNIT_VSN = 2.5 diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index f13551db83..1f9ab57727 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -31,6 +31,38 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 4.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Fixed a rare miss-compilation of tuple matching.</p> + <p> + Own Id: OTP-16470</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>The deprecated <c>erlang:get_stacktrace/0</c> BIF now + returns an empty list instead of a stacktrace. To + retrieve the stacktrace, use the extended try/catch + syntax that was introduced in OTP 21. + <c>erlang:get_stacktrace/0</c> is scheduled for removal + in OTP 24.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16484</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.19.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 5d34c61169..9a612e9063 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.19.3 +HIPE_VSN = 4.0 diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index fe6d7c8c83..0dfd01eab3 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -33,7 +33,31 @@ <file>notes.xml</file> </header> - <section><title>Inets 7.1.3</title> + <section><title>Inets 7.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Remove support for deprecated functionality. Support for + mod_esi eval scheme, mod_htacess, mod_browser, apache + config files and deprecated httpd_conf functions are + dropped. Module http_uri is deprecated.</p> + <p> + Own Id: OTP-16252</p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 7.1.3</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index b0ae8655d2..2eaef40a7a 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = inets -INETS_VSN = 7.1.3 +INETS_VSN = 7.2 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/jinterface/doc/src/notes.xml b/lib/jinterface/doc/src/notes.xml index 9eeca8e963..fca0b46878 100644 --- a/lib/jinterface/doc/src/notes.xml +++ b/lib/jinterface/doc/src/notes.xml @@ -31,6 +31,35 @@ </header> <p>This document describes the changes made to the Jinterface application.</p> +<section><title>Jinterface 1.11</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Increased size of node incarnation numbers (aka + "creation"), from 2 bits to 32 bits. This will reduce the + risk of pids/ports/refs, from different node incarnation + with the same name, being mixed up.</p> + <p> + Own Id: OTP-15603</p> + </item> + <item> + <p> + Improved node connection setup handshake protocol. Made + possible to agree on protocol version without dependence + on <c>epmd</c> or other prior knowledge of peer node + version. Also added exchange of node incarnation + ("creation") values and expanded the distribution + capability flag field from 32 to 64 bits.</p> + <p> + Own Id: OTP-16229</p> + </item> + </list> + </section> + +</section> + <section><title>Jinterface 1.10.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/jinterface/vsn.mk b/lib/jinterface/vsn.mk index f15a3f323b..f253cf7431 100644 --- a/lib/jinterface/vsn.mk +++ b/lib/jinterface/vsn.mk @@ -1 +1 @@ -JINTERFACE_VSN = 1.10.1 +JINTERFACE_VSN = 1.11 diff --git a/lib/kernel/doc/src/Makefile b/lib/kernel/doc/src/Makefile index 9b004b3781..d02e954ba9 100644 --- a/lib/kernel/doc/src/Makefile +++ b/lib/kernel/doc/src/Makefile @@ -67,9 +67,11 @@ XML_REF3_FILES = application.xml \ pg2.xml \ rpc.xml \ seq_trace.xml \ + socket.xml \ wrap_log_reader.xml \ user.xml \ - zlib_stub.xml + zlib_stub.xml \ + $(XML_REF3_ESOCK_EFILES) XML_REF4_FILES = app.xml config.xml @@ -79,8 +81,10 @@ XML_PART_FILES = part.xml XML_CHAPTER_FILES = \ notes.xml \ introduction_chapter.xml \ + socket_usage.xml \ logger_chapter.xml \ - logger_cookbook.xml + logger_cookbook.xml \ + eep48_chapter.xml BOOK_FILES = book.xml diff --git a/lib/kernel/doc/src/code.xml b/lib/kernel/doc/src/code.xml index 421c1ca254..81186b6876 100644 --- a/lib/kernel/doc/src/code.xml +++ b/lib/kernel/doc/src/code.xml @@ -705,7 +705,7 @@ ok = code:finish_loading(Prepared), </desc> </func> <func> - <name name="all_available" arity="0" since="OTP @OTP-16494@"/> + <name name="all_available" arity="0" since="OTP 23.0"/> <fsummary>Get all available modules.</fsummary> <type name="loaded_filename"/> <type name="loaded_ret_atoms"/> @@ -769,14 +769,13 @@ rpc:call(Node, code, load_binary, [Module, Filename, Binary]), </desc> </func> <func> - <name name="get_doc" arity="1" since="OTP @OTP-16406@"/> + <name name="get_doc" arity="1" since="OTP 23.0"/> <fsummary>Gets the documentation for a module.</fsummary> <desc> - <p>Searches the code path for a documentation chunk - and returns ut if available. If no documentation chunk - can be found the function tries to generate documentation - from the debug information in the module. If no debug - information is available, this function will return + <p>Searches the code path for EEP-48 style documentation and returns it + if available. If no documentation can be found the function tries to + generate documentation from the debug information in the module. + If no debug information is available, this function will return <c>{error,missing}</c>. </p> <p>For more information about the documentation chunk see diff --git a/lib/kernel/doc/src/eep48_chapter.xml b/lib/kernel/doc/src/eep48_chapter.xml new file mode 100644 index 0000000000..db5d13eb7a --- /dev/null +++ b/lib/kernel/doc/src/eep48_chapter.xml @@ -0,0 +1,188 @@ +<?xml version="1.0" encoding="utf-8" ?> +<!DOCTYPE chapter SYSTEM "chapter.dtd"> + +<chapter> + <header> + <copyright> + <year>2020</year><year>2020</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + </legalnotice> + + <title>EEP-48: Documentation storage and format</title> + <prepared></prepared> + <docno></docno> + <date></date> + <rev></rev> + <file>eep48_chapter.xml</file> + </header> + + <p>This User's Guide describes the documentation storage format initially described in + <url href="https://www.erlang.org/erlang-enhancement-proposals/eep-0048.html">EEP-48</url>. + By standardizing how API documentation is stored, it will be possible to write tools that + work across languages.</p> + + <p>To fetch the EEP-48 documentation for a module you can use + <seemfa marker="code#get_doc/1"><c>code:get_doc/1</c></seemfa>.</p> + + <section> + <title>the "Docs" storage</title> + <p>To look for documentation for a module name example, a tool should:</p> + + <p>Look for <c>example.beam</c> in the code path, parse the BEAM file and + retrieve the <c>Docs</c> chunk. If the chunk is not available, it should look + for "example.beam" in the code path and find the <c>doc/chunks/example.chunk</c> + file in the application that defines the <c>example</c> module. If a .chunk file is not + available, then documentation is not available.</p> + + <p>The choice of using a chunk or the filesystem is completely up to the language or library. + In both cases, the documentation can be added or removed at any moment by stripping + the <c>Docs</c> chunk or by removing the doc/chunks directory.</p> + + <p>For example, languages like Elixir and LFE attach the <c>Docs</c> chunk at + compilation time, which can be controlled via a compiler flag. On the other hand, + projects like OTP itself will likely generate the doc/chunks entries on a separate + command, completely unrelated from code compilation.</p> + </section> + + <section> + <title>the "Docs" format</title> + <p>In both storages, the documentation is written in the exactly same format: + an Erlang term serialized to binary via <seemfa marker="erts:erlang#term_to_binary/1"> + <c>term_to_binary/1</c></seemfa>. The term may be optionally compressed when serialized. + It must follow the type specification below:</p> + + <code> +{docs_v1, + Anno :: erl_anno:anno(), + BeamLanguage :: atom(), + Format :: binary(), + ModuleDoc :: #{DocLanguage := DocValue} | none | hidden, + Metadata :: map(), + Docs :: + [{{Kind, Name, Arity}, + Anno :: erl_anno:anno(), + Signature :: [binary()], + Doc :: #{DocLanguage := DocValue} | none | hidden, + Metadata :: map() + }]} when DocLanguage :: binary(), + DocValue :: binary() | term() + </code> + <p>where in the root tuple we have:</p> + <taglist> + <tag>Anno</tag> + <item>annotation (line, column, file) of the definition itself (see + <seeerl marker="stdlib:erl_anno"><c>erl_anno(3)</c></seeerl>)</item> + + <tag>BeamLanguage</tag> + <item>an atom representing the language, for example: erlang, elixir, lfe, alpaca, etc</item> + + <tag>Format</tag> + <item>the mime type of the documentation, such as <<"text/markdown">> or + <<"application/erlang+html">>. For details of the format used by Erlang + see the <seeguide marker="erl_docgen:doc_storage"><c>EEP-48 Chapter</c></seeguide> + in Erl_Docgen's User's Guide.</item> + + <tag>ModuleDoc</tag> + <item>a map with the documentation language as key, such as <c><<"en">></c> or + <c><<"pt_BR">></c>, and the documentation as a binary value. It may be the + atom <c>none</c> in case there is no documentation or the atom <c>hidden</c> if documentation + has been explicitly disabled for this entry.</item> + + <tag>Metadata</tag> + <item>a map of atom keys with any term as value. This can be used to add annotations like the + <c>authors</c> of a module, <c>deprecated</c>, or anything else a language or documentation + tool may find relevant.</item> + + <tag>Docs</tag> + <item>a list of documentation for other entities (such as functions and types) + in the module.</item> + </taglist> + + <p>For each entry in Docs, we have:</p> + + <taglist> + <tag>{Kind, Name, Arity}</tag> + <item>the kind, name and arity identifying the function, callback, type, etc. + The official entities are: <c>function</c>, <c>type</c> and <c>callback</c>. + Other languages will add their own. For instance, Elixir and LFE may add macro.</item> + + <tag>Anno</tag> + <item>annotation (line, column, file) of the module documentation or of the definition itself + (see <seeerl marker="stdlib:erl_anno"><c>erl_anno(3)</c></seeerl>).</item> + + <tag>Signature</tag> + <item>the signature of the entity. It is is a list of binaries. Each entry represents a + binary in the signature that can be joined with a whitespace or a newline. For example, + <c>[<<"binary_to_atom(Binary, Encoding)">>, + <<"when is_binary(Binary)">>]</c> may be rendered as + a single line or two lines. It exists exclusively for exhibition purposes.</item> + + <tag>Doc</tag> + <item>a map with the documentation language as key, such as <<"en">> or + <<"pt_BR">>, and the documentation as a value. The documentation may + either be a binary or any Erlang term, both described by <c>Format</c>. If it is an + Erlang term, then the Format must be <<"application/erlang+SUFFIX",>> + such as <<"application/erlang+html">> when the documentation is an Erlang + representation of an HTML document. The Doc may also be atom <c>none</c> in case there is + no documentation or the atom <c>hidden</c> if documentation has been explicitly + disabled for this entry.</item> + + <tag>Metadata</tag> + <item>a map of atom keys with any term as value.</item> + </taglist> + + <p>This shared format is the heart of the EEP as it is what effectively + allows cross-language collaboration.</p> + + <p>The Metadata field exists to allow languages, tools and libraries to add + custom information to each entry. This EEP documents the following metadata keys:</p> + + <taglist> + <tag>authors := [binary()]</tag> + <item>a list of authors as binaries.</item> + + <tag>cross_references := [module() | {module(), {Kind, Name, Arity}}]</tag> + <item>a list of modules or module entries that can be used as cross references + when generating documentation.</item> + + <tag>deprecated := binary()</tag> + <item>when present, it means the current entry is deprecated with a binary + that represents the reason for deprecation and a recommendation to replace + the deprecated code.</item> + + <tag>since := binary()</tag> + <item>a binary representing the version such entry was added, such + as <<"1.3.0">> or <<"20.0">>.</item> + + <tag>edit_url := binary()</tag> + <item>a binary representing a URL to change to change the documentation itself.</item> + </taglist> + + <p>Any key may be added to Metadata at any time. Keys that are frequently + used by the community can be standardized in future versions.</p> + </section> + + <section> + <title>See Also</title> + <p> + <seeerl marker="stdlib:erl_anno"><c>erl_anno(3)</c></seeerl>, + <seeerl marker="stdlib:shell_docs"><c>shell_docs(3)</c></seeerl>, + <seeguide marker="erl_docgen:doc_storage"><c>EEP-48 Chapter in Erl_Docgen's User's Guide</c></seeguide>, + <seemfa marker="code#get_doc/1"><c>code:get_doc/1</c></seemfa> + </p> + </section> +</chapter> diff --git a/lib/kernel/doc/src/erl_epmd.xml b/lib/kernel/doc/src/erl_epmd.xml index fbb316bbfc..03aa949516 100644 --- a/lib/kernel/doc/src/erl_epmd.xml +++ b/lib/kernel/doc/src/erl_epmd.xml @@ -73,7 +73,7 @@ </func> <func> - <name name="listen_port_please" arity="2" since="OTP @OTP-16250@"/> + <name name="listen_port_please" arity="2" since="OTP 23.0"/> <fsummary>Returns the port number for the local node.</fsummary> <desc> <p>Called by the distribution module to get which port the diff --git a/lib/kernel/doc/src/erpc.xml b/lib/kernel/doc/src/erpc.xml index d609bc3fd9..64032a7f94 100644 --- a/lib/kernel/doc/src/erpc.xml +++ b/lib/kernel/doc/src/erpc.xml @@ -28,7 +28,7 @@ <date>2020-02-20</date> <rev>A</rev> </header> - <module since="OTP @OTP-13450@">erpc</module> + <module since="OTP 23.0">erpc</module> <modulesummary>Enhanced Remote Procedure Call</modulesummary> <description> <p> @@ -75,8 +75,8 @@ <funcs> <func> - <name name="call" arity="2" since="OTP @OTP-13450@"/> - <name name="call" arity="3" since="OTP @OTP-13450@"/> + <name name="call" arity="2" since="OTP 23.0"/> + <name name="call" arity="3" since="OTP 23.0"/> <fsummary>Evaluate a function call on a node.</fsummary> <desc> <p> @@ -96,8 +96,8 @@ </func> <func> - <name name="call" arity="4" since="OTP @OTP-13450@"/> - <name name="call" arity="5" since="OTP @OTP-13450@"/> + <name name="call" arity="4" since="OTP 23.0"/> + <name name="call" arity="5" since="OTP 23.0"/> <fsummary>Evaluate a function call on a node.</fsummary> <desc> <p> @@ -249,7 +249,7 @@ </func> <func> - <name name="cast" arity="2" since="OTP @OTP-13450@"/> + <name name="cast" arity="2" since="OTP 23.0"/> <fsummary>Evaluate a function call on a node.</fsummary> <desc> <p> @@ -266,7 +266,7 @@ </func> <func> - <name name="cast" arity="4" since="OTP @OTP-13450@"/> + <name name="cast" arity="4" since="OTP 23.0"/> <fsummary>Evaluate a function call on a node ignoring the result.</fsummary> <desc> <p> @@ -298,7 +298,7 @@ </func> <func> - <name name="check_response" arity="2" since="OTP @OTP-13450@"/> + <name name="check_response" arity="2" since="OTP 23.0"/> <fsummary>Check if a message is a response corresponding to a previously sent call request.</fsummary> <desc> @@ -341,8 +341,8 @@ </func> <func> - <name name="multicall" arity="2" since="OTP @OTP-13450@"/> - <name name="multicall" arity="3" since="OTP @OTP-13450@"/> + <name name="multicall" arity="2" since="OTP 23.0"/> + <name name="multicall" arity="3" since="OTP 23.0"/> <fsummary>Evaluate a function call on a node.</fsummary> <desc> <p> @@ -362,8 +362,8 @@ </func> <func> - <name name="multicall" arity="4" since="OTP @OTP-13450@"/> - <name name="multicall" arity="5" since="OTP @OTP-13450@"/> + <name name="multicall" arity="4" since="OTP 23.0"/> + <name name="multicall" arity="5" since="OTP 23.0"/> <fsummary>Evaluate a function call on a number of nodes.</fsummary> <type name="caught_call_exception"/> <type name="stack_item"/> @@ -466,7 +466,7 @@ my_multicall(Nodes, Module, Function, Args) -> </func> <func> - <name name="multicast" arity="2" since="OTP @OTP-13450@"/> + <name name="multicast" arity="2" since="OTP 23.0"/> <fsummary>Evaluate a function call on a set nodes.</fsummary> <desc> <p> @@ -483,7 +483,7 @@ my_multicall(Nodes, Module, Function, Args) -> </func> <func> - <name name="multicast" arity="4" since="OTP @OTP-13450@"/> + <name name="multicast" arity="4" since="OTP 23.0"/> <fsummary>Evaluate a function call on a set of nodes ignoring the result.</fsummary> <desc> <p> @@ -517,8 +517,8 @@ my_multicall(Nodes, Module, Function, Args) -> </func> <func> - <name name="receive_response" arity="1" since="OTP @OTP-13450@"/> - <name name="receive_response" arity="2" since="OTP @OTP-13450@"/> + <name name="receive_response" arity="1" since="OTP 23.0"/> + <name name="receive_response" arity="2" since="OTP 23.0"/> <fsummary>Receive a call response corresponding to a previously sent call request.</fsummary> <desc> @@ -581,7 +581,7 @@ my_call(Node, Module, Function, Args, Timeout) -> </func> <func> - <name name="send_request" arity="2" since="OTP @OTP-13450@"/> + <name name="send_request" arity="2" since="OTP 23.0"/> <fsummary>Send a request to evaluate a function call on a node.</fsummary> <desc> <p> @@ -606,7 +606,7 @@ my_call(Node, Module, Function, Args, Timeout) -> </func> <func> - <name name="send_request" arity="4" since="OTP @OTP-13450@"/> + <name name="send_request" arity="4" since="OTP 23.0"/> <fsummary>Send a request to evaluate a function call on a node.</fsummary> <desc> <p> @@ -633,8 +633,8 @@ my_call(Node, Module, Function, Args, Timeout) -> </func> <func> - <name name="wait_response" arity="1" since="OTP @OTP-13450@"/> - <name name="wait_response" arity="2" since="OTP @OTP-13450@"/> + <name name="wait_response" arity="1" since="OTP 23.0"/> + <name name="wait_response" arity="2" since="OTP 23.0"/> <fsummary>Wait or poll for a call response corresponding to a previously sent call request.</fsummary> <desc> diff --git a/lib/kernel/doc/src/gen_tcp.xml b/lib/kernel/doc/src/gen_tcp.xml index fdde6e40b4..b90cc9d104 100644 --- a/lib/kernel/doc/src/gen_tcp.xml +++ b/lib/kernel/doc/src/gen_tcp.xml @@ -292,7 +292,7 @@ do_recv(Sock, Bs) -> If any other process is interacting with the socket while the transfer is happening, the transfer may not work correctly and messages may remain in the caller's mailbox. For instance - changing the sockets active mode before the transfere is complete + changing the sockets active mode before the transfer is complete may cause this.</p> </desc> </func> diff --git a/lib/kernel/doc/src/kernel_app.xml b/lib/kernel/doc/src/kernel_app.xml index cdfafd90eb..bbe4526f0c 100644 --- a/lib/kernel/doc/src/kernel_app.xml +++ b/lib/kernel/doc/src/kernel_app.xml @@ -134,7 +134,7 @@ <p>The parameter is described in <seemfa marker="application#load/2"><c>application:load/2</c></seemfa>.</p> </item> - <tag><c>dist_auto_connect = Value</c></tag> + <tag><marker id="dist_auto_connect"/><c>dist_auto_connect = Value</c></tag> <item> <p>Specifies when nodes are automatically connected. If this parameter is not specified, a node is always @@ -152,6 +152,12 @@ <seeerl marker="net_kernel"><c>net_kernel(3)</c></seeerl>.</p></item> </taglist> </item> + <tag><marker id="dist_listen"/><c>dist_listen = boolean()</c></tag> + <item> + <p>Specifies whether this node should be listening for incoming + distribution connections. Using this option implies that the node + also is <seecom marker="erts:erl#hidden"><c>-hidden</c></seecom>.</p> + </item> <tag><c>permissions = [Perm]</c></tag> <item> <p>Specifies the default permission for applications when they diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 624ccb6591..478607d0c8 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -31,6 +31,406 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 7.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix race condition during shutdown when + <c>shell_history</c> is enabled. The race condition would + trigger crashes in <c>disk_log</c>.</p> + <p> + Own Id: OTP-16008 Aux Id: PR-2302 </p> + </item> + <item> + <p> + Fix the Erlang distribution to handle the scenario when a + node connects that can handle message fragmentation but + can not handle the atom cache. This bug only affects + users that have implemented a custom distribution + carrier. It has been present since OTP-21.</p> + <p> + The <c>DFLAG_FRAGMENT</c> distribution flag was added to + the set of flags that can be rejected by a distribution + implementation.</p> + <p> + Own Id: OTP-16284</p> + </item> + <item> + <p> + Fix bug where a binary was not allowed to be the format + string in calls to <c>logger:log</c>.</p> + <p> + Own Id: OTP-16395 Aux Id: PR-2444 </p> + </item> + <item> + <p> + Fix bug where <c>logger</c> would end up in an infinite + loop when trying to log the crash of a handler or + formatter.</p> + <p> + Own Id: OTP-16489 Aux Id: ERL-1134 </p> + </item> + <item> + <p> + <c>code:lib_dir/1</c> has been fixed to also return the + lib dir for <c>erts</c>.</p> + <p> + This is been marked as an incompatibility for any + application that depended on <c>{error,bad_name}</c> to + be returned for <c>erts</c>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16502</p> + </item> + <item> + <p> + The application <c>stop/1</c> callback was not called if + the application master of the application terminated.</p> + <p> + Own Id: OTP-16504 Aux Id: PR-2328 </p> + </item> + <item> + <p> + Fix bug in <c>application:loaded_applications/0</c> that + could cause it to fail with <c>badarg</c> if for example + a concurrent upgrade/downgrade is running.</p> + <p> + Own Id: OTP-16627 Aux Id: PR-2601 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>A new module <seeerl + marker="kernel:erpc"><c>erpc</c></seeerl> has been + introduced in the <c>kernel</c> application. The + <c>erpc</c> module implements an enhanced subset of the + operations provided by the <seeerl + marker="kernel:rpc"><c>rpc</c></seeerl> module. Enhanced + in the sense that it makes it possible to distinguish + between returned value, raised exceptions, and other + errors. <c>erpc</c> also has better performance and + scalability than the original <c>rpc</c> implementation. + This by utilizing the newly introduced <seemfa + marker="erts:erlang#spawn_request/5"><c>spawn_request()</c></seemfa> + BIF. Also the <c>rpc</c> module benefits from these + improvements by utilizing <c>erpc</c> when it is + possible. </p><p> This change has been marked as a + potential incompatibility since <seemfa + marker="kernel:rpc#block_call/5"><c>rpc:block_call()</c></seemfa> + now only is guaranteed to block other <c>block_call()</c> + operations. The documentation previously claimed that it + would block all <c>rpc</c> operations. This has however + never been the case. It previously did not block + node-local <c>block_call()</c> operations.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-13450 Aux Id: OTP-15251 </p> + </item> + <item> + <p>A client node can receive its node name dynamically + from the node that it first connects to. This featured + can by used by</p> <list> <item><p>starting with <c>erl + -sname undefined</c></p></item> <item><p>erl_interface + functions <c>ei_connect_init</c> and friends</p></item> + <item><p><c>erl_call -R</c></p></item> </list> + <p> + Own Id: OTP-13812</p> + </item> + <item> + <p> + Improved the printout of single line logger events for + most of the OTP behaviours in STDLIB and Kernel. This + includes <c>proc_lib</c>, <c>gen_server</c>, + <c>gen_event</c>, <c>gen_statem</c>, <c>gen_fsm</c>, + <c>supervisor</c>, <c>supervisor_bridge</c> and + <c>application</c>.</p> + <p> + Improved the <seeerl + marker="kernel:logger_formatter#chars_limit"><c>chars_limit</c></seeerl> + and <seeerl + marker="kernel:logger_formatter#depth"><c>depth</c></seeerl> + handling in <c>proc_lib</c> and when formatting of + exceptions.</p> + <p> + Own Id: OTP-15299</p> + </item> + <item> + <p> + Remove usage and documentation of old requests of the + I/O-protocol.</p> + <p> + Own Id: OTP-15695</p> + </item> + <item> + <p>Directories can now be opened by <c>file:open/2</c> + when passing the <c>directory</c> option.</p> + <p> + Own Id: OTP-15835 Aux Id: PR-2212 </p> + </item> + <item> + <p> + The check of whether to log or not based on the log level + in <c>logger</c> has been optimized by using + <c>persistent_term</c> to store the log level.</p> + <p> + Own Id: OTP-15948 Aux Id: PR-2356 </p> + </item> + <item> + <p><c>file:read_file_info/2</c> can now be used on opened + files and directories.</p> + <p> + Own Id: OTP-15956 Aux Id: PR-2231 </p> + </item> + <item> + <p> + The <c>-config</c> option to <c>erl</c> now can take + multiple config files without repeating the + <c>-config</c> option. Example:</p> + <p> + erl -config sys local</p> + <p> + Own Id: OTP-16148 Aux Id: PR-2373 </p> + </item> + <item> + <p> + Improved node connection setup handshake protocol. Made + possible to agree on protocol version without dependence + on <c>epmd</c> or other prior knowledge of peer node + version. Also added exchange of node incarnation + ("creation") values and expanded the distribution + capability flag field from 32 to 64 bits.</p> + <p> + Own Id: OTP-16229</p> + </item> + <item> + <p>The possibility to run Erlang distribution without + relying on EPMD has been extended. To achieve this a + couple of new options to the inet distribution has been + added.</p> <taglist> <tag>-dist_listen false</tag> + <item>Setup the distribution channel, but do not listen + for incoming connection. This is useful when you want to + use the current node to interact with another node on the + same machine without it joining the entire + cluster.</item> <tag>-erl_epmd_port Port</tag> + <item>Configure a default port that the built-in EPMD + client should return. This allows the local node to know + the port to connect to for any other node in the + cluster.</item> </taglist> <p>The <c>erl_epmd</c> + callback API has also been extended to allow returning + <c>-1</c> as the creation which means that a random + creation will be created by the node.</p> + <p>In addition a new callback function called + <c>listen_port_please</c> has been added that allows the + callback to return which listen port the distribution + should use. This can be used instead of + <c>inet_dist_listen_min/max</c> if the listen port is to + be fetched from an external service.</p> + <p> + Own Id: OTP-16250</p> + </item> + <item> + <p> + A first EXPERIMENTAL module that is a <c>socket</c> + backend to <c>gen_tcp</c> and <c>inet</c> has been + implemented. Others will follow. Feedback will be + appreciated.</p> + <p> + Own Id: OTP-16260 Aux Id: OTP-15403 </p> + </item> + <item> + <p> + The new experimental <c>socket</c> module has been moved + to the Kernel application.</p> + <p> + Own Id: OTP-16312</p> + </item> + <item> + <p> + Replace usage of deprecated function in the <c>group</c> + module.</p> + <p> + Own Id: OTP-16345</p> + </item> + <item> + <p> + Minor updates due to the new spawn improvements made.</p> + <p> + Own Id: OTP-16368 Aux Id: OTP-15251 </p> + </item> + <item> + <p> + Update of <seeerl + marker="kernel:seq_trace#whatis">sequential + tracing</seeerl> to also support other information + transfers than message passing.</p> + <p> + Own Id: OTP-16370 Aux Id: OTP-15251, OTP-15232 </p> + </item> + <item> + <p><c>code:module_status/1</c> now accepts a list of + modules. <c>code:module_status/0</c>, which returns the + statuses for all loaded modules, has been added.</p> + <p> + Own Id: OTP-16402</p> + </item> + <item> + <p><c>filelib:wildcard/1,2</c> is now twice as fast when + a double star (<c>**</c>) is part of the pattern.</p> + <p> + Own Id: OTP-16419</p> + </item> + <item> + <p> A new implementation of distributed named process + groups has been introduced. It is available in the + <seeerl marker="kernel:pg"><c>pg</c></seeerl> module. + </p><p> Note that this <c>pg</c> module only has the name + in common with the experimental <c>pg</c> module that was + present in <c>stdlib</c> up until OTP 17. </p><p> Thanks + to Maxim Fedorov for the implementation. </p> + <p> + Own Id: OTP-16453 Aux Id: PR-2524 </p> + </item> + <item> + <p> The <seeerl marker="kernel:pg2"><c>pg2</c></seeerl> + module has been deprecated. It has also been scheduled + for removal in OTP 24. </p><p> You are advised to replace + the usage of <c>pg2</c> with the newly introduced <seeerl + marker="kernel:pg"><c>pg</c></seeerl> module. <c>pg</c> + has a similar API, but with a more scalable + implementation. </p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16455</p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p> + The internal hosts file resolver cache <c>inet_hosts</c> + has been rewritten to behave better when the hosts file + changes. For example the cache is updated per entry + instead of cleared and reloaded so lookups do not + temporarily fail during reloading, and; when multiple + processes simultaneously request reload these are now + folded into one instead of all done in sequence. Reported + and first solution suggestion by Maxim Fedorov.</p> + <p> + Own Id: OTP-16487 Aux Id: PR-2516 </p> + </item> + <item> + <p> + Add <c>code:all_available/0</c> that can be used to get + all available modules.</p> + <p> + Own Id: OTP-16494</p> + </item> + <item> + <p> + As of OTP 23, the distributed <seeerl + marker="kernel:disk_log"><c>disk_log</c></seeerl> feature + has been deprecated. It has also been scheduled for + removal in OTP 24.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16495</p> + </item> + <item> + <p> + Add the function <c>code:fetch_docs/1</c> for fetching + embedded documentation for aa Erlang module.</p> + <p> + Own Id: OTP-16499</p> + </item> + <item> + <p> + Improve configure for the net nif, which should increase + portability.</p> + <p> + Own Id: OTP-16530 Aux Id: OTP-16464 </p> + </item> + <item> + <p> + socket: Socket counters and socket global counters are + now represented as maps (instead of property lists).</p> + <p> + Own Id: OTP-16535</p> + </item> + <item> + <p> + The experimental socket module has gotten restrictions + removed so now the 'seqpacket' socket type should work + for any communication domain (protocol family) where the + OS supports it, typically the Unix Domain.</p> + <p> + Own Id: OTP-16550 Aux Id: ERIERL-476 </p> + </item> + <item> + <p> + Allow using custom IO devices in <c>logger_std_h</c>.</p> + <p> + Own Id: OTP-16563 Aux Id: PR-2523 </p> + </item> + <item> + <p>Added <c>file:del_dir_r/1</c> which deletes a + directory together with all of its contents, similar to + <c>rm -rf</c> on Unix systems.</p> + <p> + Own Id: OTP-16570 Aux Id: PR-2565 </p> + </item> + <item> + <p> + socket: By default the socket options rcvtimeo and + sndtimeo are now disabled. To enable these, OTP now has + to be built with the configure option + --enable-esock-rcvsndtimeo</p> + <p> + Own Id: OTP-16620</p> + </item> + <item> + <p> + The experimental gen_tcp compatibility code utilizing the + socket module could loose buffered data when receiving a + specified number of bytes. This bug has been fixed. + Reported by Maksim Lapshin on bugs.erlang.org ERL-1234</p> + <p> + Own Id: OTP-16632 Aux Id: ERL-1234 </p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 6.5.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix bug in <c>application:loaded_applications/0</c> that + could cause it to fail with <c>badarg</c> if for example + a concurrent upgrade/downgrade is running.</p> + <p> + Own Id: OTP-16627 Aux Id: PR-2601 </p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 6.5.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/doc/src/part.xml b/lib/kernel/doc/src/part.xml index 7a7a92b067..dd5d955a78 100644 --- a/lib/kernel/doc/src/part.xml +++ b/lib/kernel/doc/src/part.xml @@ -32,6 +32,8 @@ <p></p> </description> <xi:include href="introduction_chapter.xml"/> + <xi:include href="socket_usage.xml"/> <xi:include href="logger_chapter.xml"/> <xi:include href="logger_cookbook.xml"/> + <xi:include href="eep48_chapter.xml"/> </part> diff --git a/lib/kernel/doc/src/ref_man.xml b/lib/kernel/doc/src/ref_man.xml index 9127157eb5..333cb83bee 100644 --- a/lib/kernel/doc/src/ref_man.xml +++ b/lib/kernel/doc/src/ref_man.xml @@ -69,6 +69,7 @@ <xi:include href="pg2.xml"/> <xi:include href="rpc.xml"/> <xi:include href="seq_trace.xml"/> + <xi:include href="socket.xml"/> <xi:include href="user.xml"/> <xi:include href="wrap_log_reader.xml"/> <xi:include href="zlib_stub.xml"/> diff --git a/erts/doc/src/socket.xml b/lib/kernel/doc/src/socket.xml index 53cf9ed0c8..5a9fdf98f7 100644 --- a/erts/doc/src/socket.xml +++ b/lib/kernel/doc/src/socket.xml @@ -117,9 +117,6 @@ <name name="socket_counters"/> </datatype> <datatype> - <name name="socket_counter"/> - </datatype> - <datatype> <name name="socket_info"/> </datatype> <datatype> @@ -129,9 +126,6 @@ <name name="ip6_address"/> </datatype> <datatype> - <name name="ip_address"/> - </datatype> - <datatype> <name name="sockaddr"/> </datatype> <datatype> @@ -302,31 +296,29 @@ <name name="int32"/> </datatype> <datatype> - <name name="supports_options_socket"/> - </datatype> - <datatype> - <name name="supports_options_ip"/> - </datatype> - <datatype> - <name name="supports_options_ipv6"/> - </datatype> - <datatype> - <name name="supports_options_tcp"/> - </datatype> - <datatype> - <name name="supports_options_udp"/> - </datatype> - <datatype> - <name name="supports_options_sctp"/> - </datatype> - <datatype> - <name name="supports_options"/> - </datatype> - <datatype> - <name name="supports_send_flags"/> - </datatype> - <datatype> - <name name="supports_recv_flags"/> + <name name="errcode"/> + <desc> + <p> + The POSIX error codes are mostly come from the + OS level socket interface, + but this module may generate some appropriate + POSIX codes. + </p> + <p> + The other values come from this module's lower levels + and are all fairly fatal internal errors: + </p> + <taglist> + <tag><c>exalloc</c></tag> + <item>Memory allocation failed</item> + <tag><c>exmonitor</c></tag> + <item>Failed to set a monitor on a process</item> + <tag><c>exselect</c></tag> + <item>Select operation failed</item> + <tag><c>exself</c></tag> + <item>Failed to get current process</item> + </taglist> + </desc> </datatype> </datatypes> @@ -513,8 +505,8 @@ </func> <func> - <name name="open" arity="1" since="OTP @OTP-16398@"/> - <name name="open" arity="2" clause_i="1" since="OTP @OTP-16398@"/> + <name name="open" arity="1" since="OTP 23.0"/> + <name name="open" arity="2" clause_i="1" since="OTP 23.0"/> <fsummary>Create an endpoint for communication.</fsummary> <desc> <p>Create an endpoint (socket) for communication based on an @@ -903,10 +895,11 @@ <name name="setopt" arity="4" clause_i="5" since="OTP 22.0"/> <name name="setopt" arity="4" clause_i="6" since="OTP 22.0"/> <name name="setopt" arity="4" clause_i="7" since="OTP 22.0"/> - <fsummary>Set options on a socket.</fsummary> + <name name="setopt" arity="4" clause_i="8" since="OTP 22.0"/> + <fsummary>Set an option on a socket.</fsummary> <desc> - <p>Set options on a socket.</p> - <p>What properties are valid depend both on <c>Level</c> and on + <p>Set an option on a socket.</p> + <p>What options are valid depend both on <c>Level</c> and on what kind of socket it is (<c>domain</c>, <c>type</c> and <c>protocol</c>).</p> @@ -956,36 +949,178 @@ </func> <func> + <name since="OTP 22.0"> + supports() -> Supports + </name> + <name since="OTP 22.0"> + supports(Key1 :: options) -> SupportsOptions + </name> + <name since="OTP 22.0"> + supports(Key1 :: send_flags) -> SupportsSendFlags + </name> + <name since="OTP 22.0"> + supports(Key1 :: recv_flags) -> SupportsRecvFlags + </name> + <name since="OTP 22.0"> + supports(Key1 :: options, Key2 :: socket) -> SupportsOptionsSocket + </name> + <name since="OTP 22.0"> + supports(Key1 :: options, Key2 :: ip) -> SupportsOptionsIP + </name> + <name since="OTP 22.0"> + supports(Key1 :: options, Key2 :: ipv6) -> SupportsOptionsIPv6 + </name> + <name since="OTP 22.0"> + supports(Key1 :: options, Key2 :: tcp) -> SupportsOptionsTCP + </name> + <name since="OTP 22.0"> + supports(Key1 :: options, Key2 :: udp) -> SupportsOptionsUDP + </name> + <name since="OTP 22.0"> + supports(Key1 :: options, Key2 :: sctp) -> SupportsOptionsSCTP + </name> + <!-- <name name="supports" arity="0" since="OTP 22.0"/> - <name name="supports" arity="1" clause_i="1" since="OTP 22.0"/> - <name name="supports" arity="1" clause_i="2" since="OTP 22.0"/> - <name name="supports" arity="1" clause_i="3" since="OTP 22.0"/> - <name name="supports" arity="1" clause_i="4" since="OTP 22.0"/> - <name name="supports" arity="1" clause_i="5" since="OTP 22.3"/> - <name name="supports" arity="1" clause_i="6" since="OTP 22.3"/> - <name name="supports" arity="1" clause_i="7" since="OTP @OTP-16432@"/> - <name name="supports" arity="1" clause_i="8" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="1" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="2" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="3" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="4" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="5" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="6" since="OTP 22.0"/> - <name name="supports" arity="2" clause_i="7" since="OTP 22.3"/> - <name name="supports" arity="2" clause_i="8" since="OTP 22.3"/> - <name name="supports" arity="2" clause_i="9" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="1" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="2" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="3" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="4" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="5" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="6" since="OTP 22.0"/> - <name name="supports" arity="3" clause_i="7" since="OTP 22.0"/> + <name name="supports" arity="1" since="OTP 22.0"/> + <name name="supports" arity="2" since="OTP 22.0"/> + --> + <fsummary>Report info about what the platform supports.</fsummary> + <type> + <v> + Supports :: [{Feature, boolean()} + | {send_flags, SupportsSendFlags} + | {recv_flags, SupportsRecvFlags} + | {options, SupportsOptions}] + </v> + <v>Feature :: sctp | ipv6 | local | netns</v> + <v> + SupportsSendFlags :: + [{<seetype marker="#send_flag">send_flag()</seetype>, boolean()}] + </v> + <v> + SupportsRecvFlags :: + [{<seetype marker="#recv_flag">recv_flag()</seetype>, boolean()}] + </v> + <v> + SupportsOptions :: + [{socket, SupportsOptionsSocket} + | {ip, SupportsOptionsIP} + | {ipv6, SupportsOptionsIPv6} + | {tcp, SupportsOptionsTCP} + | {udp, SupportsOptionsUDP} + | {sctp, SupportsOptionsSCTP}] + </v> + <v> + SupportsOptionsSocket :: + [{<seetype marker="#socket_option">socket_option()</seetype>, boolean()}] + </v> + <v> + SupportsOptionsIP :: + [{<seetype marker="#ip_socket_option">ip_socket_option()</seetype>, boolean()}] + </v> + <v> + SupportsOptionsIPv6 :: + [{<seetype marker="#ipv6_socket_option">ipv6_socket_option()</seetype>, boolean()}] + </v> + <v> + SupportsOptionsTCP :: + [{<seetype marker="#tcp_socket_option">tcp_socket_option()</seetype>, boolean()}] + </v> + <v> + SupportsOptionsUDP :: + [{<seetype marker="#udp_socket_option">udp_socket_option()</seetype>, boolean()}] + </v> + <v> + SupportsOptionsSCTP :: + [{<seetype marker="#sctp_socket_option">sctp_socket_option()</seetype>, boolean()}] + </v> + </type> + <desc> + <p> + This function retreives information about what the + platform supports, such as if SCTP is supported, + or which socket options are supported. + </p> + <p> + For keys other than the known the empty list is returned, + Note that in a future version or on a different platform + there might be more supported items. + </p> + </desc> + </func> + + <func> + <name since="OTP 23.0"> + is_supported(Key1 :: sctp | ipv6 | local | netns) -> boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: send_flags, Key2 :: SendFlag) -> boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: recv_flags, Key2 :: RecvFlag) -> boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: options, Key2 :: socket, Key3 :: SocketOption) -> + boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: options, Key2 :: ip, Key3 :: IPSocketOption) -> + boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: options, Key2 :: ipv6, Key3 :: IPv6SocketOption) -> + boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: options, Key2 :: tcp, Key3 :: TCPSocketOption) -> + boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: options, Key2 :: udp, Key3 :: UDPSocketOption) -> + boolean() + </name> + <name since="OTP 23.0"> + is_supported(Key1 :: options, Key2 :: sctp, Key3 :: SCTPSocketOption) -> + boolean() + </name> <fsummary>Report info about what the platform supports.</fsummary> + <type> + <v> + SocketOption :: + <seetype marker="#socket_option">socket_option()</seetype> + </v> + <v> + IPSocketOption :: + <seetype marker="#ip_socket_option">ip_socket_option()</seetype> + </v> + <v> + IPv6SocketOption :: + <seetype marker="#ipv6_socket_option">ipv6_socket_option()</seetype> + </v> + <v> + TCPSocketOption :: + <seetype marker="#tcp_socket_option">tcp_socket_option()</seetype> + </v> + <v> + UDPSocketOption :: + <seetype marker="#udp_socket_option">udp_socket_option()</seetype> + </v> + <v> + SCTPSocketOption :: + <seetype marker="#sctp_socket_option">sctp_socket_option()</seetype> + </v> + </type> <desc> - <p>This function intends to retreive information about what the - platform supports. Such as if SCTP is supported. Or which socket - options are supported. </p> + <p> + This function retreives information about what the + platform supports, such as if SCTP is supported, + or which socket options are supported. + </p> + <p> + For keys other than the known <c>false</c> is returned. + Note that in a future version or on a different platform + there might be more supported items. + </p> </desc> </func> diff --git a/erts/doc/src/socket_usage.xml b/lib/kernel/doc/src/socket_usage.xml index e25c995098..580ff4cbd0 100644 --- a/erts/doc/src/socket_usage.xml +++ b/lib/kernel/doc/src/socket_usage.xml @@ -302,6 +302,10 @@ <cell>yes</cell> <cell>yes</cell> <cell> + This option is not normally supported (see why below). + OTP has to be explicitly built with the + <c>--enable-esock-rcvsndtime</c> configure option for this + to be available. Since our implementation is <em>nonblocking</em>, its unknown if and how this option works, or even if it may cause malfunctions. @@ -346,6 +350,10 @@ <cell>yes</cell> <cell>yes</cell> <cell> + This option is not normally supported (see why below). + OTP has to be explicitly built with the + <c>--enable-esock-rcvsndtime</c> configure option for this + to be available. Since our implementation is <em>nonblocking</em>, its unknown if and how this option works, or even if it may cause malfunctions. diff --git a/lib/kernel/doc/src/specs.xml b/lib/kernel/doc/src/specs.xml index 00f6f04218..990bc85782 100644 --- a/lib/kernel/doc/src/specs.xml +++ b/lib/kernel/doc/src/specs.xml @@ -35,6 +35,7 @@ <xi:include href="../specs/specs_pg2.xml"/> <xi:include href="../specs/specs_rpc.xml"/> <xi:include href="../specs/specs_seq_trace.xml"/> + <xi:include href="../specs/specs_socket.xml"/> <xi:include href="../specs/specs_user.xml"/> <xi:include href="../specs/specs_wrap_log_reader.xml"/> <xi:include href="../specs/specs_zlib_stub.xml"/> diff --git a/lib/kernel/src/Makefile b/lib/kernel/src/Makefile index 07f4f2304c..6c75bcffee 100644 --- a/lib/kernel/src/Makefile +++ b/lib/kernel/src/Makefile @@ -135,6 +135,7 @@ MODULES = \ ram_file \ rpc \ seq_trace \ + socket \ standard_error \ user \ user_drv \ diff --git a/lib/kernel/src/application_controller.erl b/lib/kernel/src/application_controller.erl index 8db0646dae..869e9be1e8 100644 --- a/lib/kernel/src/application_controller.erl +++ b/lib/kernel/src/application_controller.erl @@ -271,13 +271,12 @@ which_applications(Timeout) -> gen_server:call(?AC, which_applications, Timeout). loaded_applications() -> - ets:filter(ac_tab, - fun([{{loaded, AppName}, #appl{descr = Descr, vsn = Vsn}}]) -> - {true, {AppName, Descr, Vsn}}; - (_) -> - false - end, - []). + ets:select(ac_tab, + [{ + {{loaded, '$1'}, #appl{descr = '$2', vsn = '$3', _ = '_'}}, + [], + [{{'$1', '$2', '$3'}}] + }]). %% Returns some debug info info() -> diff --git a/lib/kernel/src/code.erl b/lib/kernel/src/code.erl index 9eb0e80af4..7b0ef8cf38 100644 --- a/lib/kernel/src/code.erl +++ b/lib/kernel/src/code.erl @@ -862,7 +862,7 @@ get_doc_chunk(Filename, Mod) when is_atom(Mod) -> {error,beam_lib,{file_error,_Filename,enoent}} -> get_doc_chunk(Filename, atom_to_list(Mod)); {ok, {Mod, [{"Docs",Bin}]}} -> - binary_to_term(Bin) + {ok,binary_to_term(Bin)} end; get_doc_chunk(Filename, Mod) -> case filename:dirname(Filename) of @@ -901,24 +901,22 @@ get_function_docs_from_ast(AST) -> lists:flatmap(fun(E) -> get_function_docs_from_ast(E, AST) end, AST). get_function_docs_from_ast({function,Anno,Name,Arity,_Code}, AST) -> Signature = io_lib:format("~p/~p",[Name,Arity]), - Specs = lists:filter(fun({attribute,_Ln,spec,{FA,_}}) -> - case FA of - {F,A} -> - F =:= Name andalso A =:= Arity; - {_, F, A} -> - F =:= Name andalso A =:= Arity - end; - (_) -> false - end, AST), + Specs = lists:filter( + fun({attribute,_Ln,spec,{FA,_}}) -> + case FA of + {F,A} -> + F =:= Name andalso A =:= Arity; + {_, F, A} -> + F =:= Name andalso A =:= Arity + end; + (_) -> false + end, AST), SpecMd = case Specs of - [S] -> #{ spec => [S] }; + [S] -> #{ signature => [S] }; [] -> #{} end, - FnDocs = [], - Md = SpecMd#{}, - [{{function, Name, Arity}, Anno, [unicode:characters_to_binary(Signature)], - #{ <<"en">> => FnDocs }, - Md#{}}]; + [{{function, Name, Arity}, Anno, + [unicode:characters_to_binary(Signature)], none, SpecMd}]; get_function_docs_from_ast(_, _) -> []. diff --git a/lib/kernel/src/dist_util.erl b/lib/kernel/src/dist_util.erl index 0b43377821..75a48e8ac4 100644 --- a/lib/kernel/src/dist_util.erl +++ b/lib/kernel/src/dist_util.erl @@ -353,7 +353,7 @@ do_mark_pending(Kernel, MyNode, Node, Flags) -> receive {Kernel,{accept_pending,Ret}} -> ?trace("do_mark_pending(~p,~p,~p,~p) -> ~p~n", - [Kernel,Node,Address,Flags,Ret]), + [Kernel, MyNode, Node, Flags, Ret]), Ret end. @@ -674,7 +674,10 @@ send_name(#hs_data{socket = Socket, this_node = Node, ?ERL_DIST_VER_5; is_integer(Version), Version >= ?ERL_DIST_VER_6 -> - Creation = erts_internal:get_creation(), + Creation = case name_type(Flags) of + static -> erts_internal:get_creation(); + dynamic -> 0 + end, NameLen = byte_size(NameBin), ?trace("send_name: 'N' node=~p creation=~w\n", [Node, Creation]), @@ -781,7 +784,7 @@ recv_name_new(HSData, <<Creation:32>> = <<Cr3,Cr2,Cr1,Cr0>>, <<NameLen:16>> = <<NL1,NL0>>, {Name, _Residue} = lists:split(NameLen, Rest), - ?trace("recv_name: 'N' node=~p creation=~w\n", [Node, Creation]), + ?trace("recv_name: 'N' name=~p creation=~w\n", [Name, Creation]), case is_name_ok(Name, Flags) of true -> check_allowed(HSData, Name), @@ -1135,7 +1138,7 @@ send_status(#hs_data{socket = Socket, other_creation = Creation, f_send = FSend}, named) -> - ?debug({dist_util,self(),send_status, Node, Stat}), + ?debug({dist_util, self(), send_status, Node}), NameBin = atom_to_binary(Node, utf8), NameLen = byte_size(NameBin), case FSend(Socket, [$s, "named:", diff --git a/lib/kernel/src/gen_tcp.erl b/lib/kernel/src/gen_tcp.erl index 0c98f62cdf..b44144d88a 100644 --- a/lib/kernel/src/gen_tcp.erl +++ b/lib/kernel/src/gen_tcp.erl @@ -130,7 +130,7 @@ {netns, file:filename_all()} | {bind_to_device, binary()} | option(). --type socket() :: port(). +-type socket() :: inet:socket(). -export_type([option/0, option_name/0, connect_option/0, listen_option/0, socket/0, pktoptions_value/0]). diff --git a/lib/kernel/src/gen_tcp_socket.erl b/lib/kernel/src/gen_tcp_socket.erl index fe58cc0cba..68733ce8e6 100644 --- a/lib/kernel/src/gen_tcp_socket.erl +++ b/lib/kernel/src/gen_tcp_socket.erl @@ -291,10 +291,10 @@ send(?module_socket(Server, Socket), Data) -> Result = socket_send(Socket, Data, SendTimeout), send_result(Server, Meta, Result) end; - {ok, _Meta} -> + {ok, _BadMeta} -> exit(badarg); {error, _} = Error -> - ?badarg_exit(Error) + Error end. %% send_result(Server, Meta, Result) -> @@ -1227,9 +1227,7 @@ handle_event( {call, From}, {recv, Length, Timeout}, State, {P, D}) -> case State of 'connected' -> - handle_recv( - P, recv_start(D, From, Length), - [{{timeout, recv}, Timeout, recv}]); + handle_recv_start(P, D, From, Length, Timeout); #recv{} -> %% Receive in progress {keep_state_and_data, @@ -1352,6 +1350,32 @@ handle_connected(P, D, ActionsR) -> handle_recv(P, recv_start(D), ActionsR) end. +handle_recv_start( + P, #{packet := Packet, buffer := Buffer} = D, From, Length, Timeout) + when Packet =:= raw, 0 < Length; + Packet =:= 0, 0 < Length -> + Size = iolist_size(Buffer), + if + Length =< Size -> + {Data, NewBuffer} = + split_binary(condense_buffer(Buffer), Length), + handle_recv_deliver( + P, + D#{recv_length => Length, % Redundant + recv_from => From, + buffer := NewBuffer}, + [], Data); + true -> + N = Length - Size, + handle_recv( + P, D#{recv_length => N, recv_from => From}, + [{{timeout, recv}, Timeout, recv}]) + end; +handle_recv_start(P, D, From, _Length, Timeout) -> + handle_recv( + P, D#{recv_length => 0, recv_from => From}, + [{{timeout, recv}, Timeout, recv}]). + handle_recv(P, #{packet := Packet, recv_length := Length} = D, ActionsR) -> if 0 < Length -> @@ -1661,16 +1685,6 @@ cleanup_recv_reply( end}. %% Initialize packet recv state -recv_start(#{packet := Packet} = D, From, Length) -> - %% - D#{recv_length => - case Packet of - raw -> Length; - 0 -> Length; - _ -> 0 - end, - recv_from => From}. - recv_start(D) -> D#{recv_length => 0}. @@ -1699,6 +1713,10 @@ recv_data_deliver( [{reply, From, {ok, DeliverData}}, {{timeout, recv}, cancel} | ActionsR]}; + #{active := false} -> + D_1 = D#{buffer := unrecv_buffer(Data, maps:get(buffer, D))}, + {recv_stop(next_packet(D_1, Packet, Data)), + ActionsR}; #{active := Active} -> ModuleSocket = module_socket(P), Owner ! @@ -1767,6 +1785,16 @@ catbin(Bin, <<>>) when is_binary(Bin) -> Bin; catbin(Bin1, Bin2) when is_binary(Bin1), is_binary(Bin2) -> <<Bin1/binary, Bin2/binary>>. +unrecv_buffer(Data, Buffer) -> + case Buffer of + <<>> -> + Data; + _ when is_binary(Buffer) -> + [Data, Buffer]; + _ -> + [Data | Buffer] + end. + condense_buffer([Bin]) when is_binary(Bin) -> Bin; condense_buffer(Buffer) -> iolist_to_binary(reverse_improper(Buffer, [])). @@ -1993,7 +2021,7 @@ getstat_avg(SumTag, D, C, CntTag) -> socket_info_counters(Socket) -> #{counters := Counters} = socket:info(Socket), - maps:from_list(Counters). + Counters. receive_counter_wrap(Socket, D, Wrapped) -> receive diff --git a/lib/kernel/src/inet.erl b/lib/kernel/src/inet.erl index 8b944a1049..6ae5351652 100644 --- a/lib/kernel/src/inet.erl +++ b/lib/kernel/src/inet.erl @@ -128,7 +128,8 @@ 'etimedout' | 'ewouldblock' | 'exbadport' | 'exbadseq' | file:posix(). --type socket() :: port(). +-type module_socket() :: {'$inet', Handler :: module(), Handle :: term()}. +-type socket() :: port() | module_socket(). -type socket_setopt() :: gen_sctp:option() | gen_tcp:option() | gen_udp:option(). @@ -1433,7 +1434,7 @@ gethostbyaddr_tm_native(Addr, Timer, Opts) -> Family :: address_family(), Type :: socket_type(), Module :: atom()) -> - {'ok', socket()} | {'error', posix()}. + {'ok', port()} | {'error', posix()}. open(FdO, Addr, Port, Opts, Protocol, Family, Type, Module) when is_integer(FdO), FdO < 0; diff --git a/lib/kernel/src/inet6_udp.erl b/lib/kernel/src/inet6_udp.erl index 7ebdb3d301..b8c33e91ec 100644 --- a/lib/kernel/src/inet6_udp.erl +++ b/lib/kernel/src/inet6_udp.erl @@ -44,10 +44,10 @@ getaddr(Address, Timer) -> inet:getaddr(Address, ?FAMILY, Timer). %% inet_udp special this side addresses translate_ip(IP) -> inet:translate_ip(IP, ?FAMILY). --spec open(_) -> {ok, inet:socket()} | {error, atom()}. +-spec open(_) -> {ok, port()} | {error, atom()}. open(Port) -> open(Port, []). --spec open(_, _) -> {ok, inet:socket()} | {error, atom()}. +-spec open(_, _) -> {ok, port()} | {error, atom()}. open(Port, Opts) -> case inet:udp_options( [{port,Port} | Opts], @@ -91,7 +91,7 @@ recv(S, Len) -> recv(S, Len, Time) -> prim_inet:recvfrom(S, Len, Time). --spec close(inet:socket()) -> ok. +-spec close(port()) -> ok. close(S) -> inet:udp_close(S). diff --git a/lib/kernel/src/inet_udp.erl b/lib/kernel/src/inet_udp.erl index ace095f30c..00ebd4bb1d 100644 --- a/lib/kernel/src/inet_udp.erl +++ b/lib/kernel/src/inet_udp.erl @@ -45,10 +45,10 @@ getaddr(Address, Timer) -> inet:getaddr(Address, ?FAMILY, Timer). %% inet_udp special this side addresses translate_ip(IP) -> inet:translate_ip(IP, ?FAMILY). --spec open(_) -> {ok, inet:socket()} | {error, atom()}. +-spec open(_) -> {ok, port()} | {error, atom()}. open(Port) -> open(Port, []). --spec open(_, _) -> {ok, inet:socket()} | {error, atom()}. +-spec open(_, _) -> {ok, port()} | {error, atom()}. open(Port, Opts) -> case inet:udp_options( [{port,Port}, {recbuf, ?RECBUF} | Opts], @@ -92,7 +92,7 @@ recv(S, Len) -> recv(S, Len, Time) -> prim_inet:recvfrom(S, Len, Time). --spec close(inet:socket()) -> ok. +-spec close(port()) -> ok. close(S) -> inet:udp_close(S). diff --git a/lib/kernel/src/kernel.app.src b/lib/kernel/src/kernel.app.src index 48c77c49a9..e9f6049d5f 100644 --- a/lib/kernel/src/kernel.app.src +++ b/lib/kernel/src/kernel.app.src @@ -115,6 +115,7 @@ raw_file_io_list, raw_file_io_raw, seq_trace, + socket, standard_error, wrap_log_reader]}, {registered, [application_controller, @@ -154,6 +155,6 @@ {shell_docs_ansi,auto} ]}, {mod, {kernel, []}}, - {runtime_dependencies, ["erts-@OTP-15251@", "stdlib-@OTP-15251@", "sasl-3.0"]} + {runtime_dependencies, ["erts-11.0", "stdlib-3.13", "sasl-3.0"]} ] }. diff --git a/lib/kernel/src/kernel.appup.src b/lib/kernel/src/kernel.appup.src index f42dd8ca6e..09a55d6f0a 100644 --- a/lib/kernel/src/kernel.appup.src +++ b/lib/kernel/src/kernel.appup.src @@ -21,6 +21,7 @@ %% versions from the following OTP releases: %% - OTP 21 %% - OTP 22 +%% - OTP 23 %% %% We also allow upgrade from, and downgrade to all %% versions that have branched off from the above @@ -44,7 +45,8 @@ {<<"^6\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^6\\.5$">>,[restart_new_emulator]}, {<<"^6\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^6\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], + {<<"^6\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^6\\.5\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^6\\.0$">>,[restart_new_emulator]}, {<<"^6\\.0\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^6\\.0\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -62,4 +64,5 @@ {<<"^6\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^6\\.5$">>,[restart_new_emulator]}, {<<"^6\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^6\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. + {<<"^6\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^6\\.5\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/kernel/src/net.erl b/lib/kernel/src/net.erl index aceb903bb6..905114ab7c 100644 --- a/lib/kernel/src/net.erl +++ b/lib/kernel/src/net.erl @@ -152,7 +152,6 @@ relay(X) -> slave:relay(X). info() -> prim_net:info(). -else. --dialyzer({nowarn_function, info/0}). info() -> erlang:error(notsup). -endif. @@ -164,7 +163,6 @@ info() -> command(Cmd) -> prim_net:command(Cmd). -else. --dialyzer({nowarn_function, command/1}). command(_Cmd) -> erlang:error(notsup). -endif. @@ -191,7 +189,6 @@ command(_Cmd) -> gethostname() -> prim_net:gethostname(). -else. --dialyzer({nowarn_function, gethostname/0}). gethostname() -> erlang:error(notsup). -endif. @@ -208,6 +205,8 @@ gethostname() -> Info :: name_info(), Reason :: term(). +-dialyzer({no_return, getnameinfo/1}). + getnameinfo(SockAddr) -> getnameinfo(SockAddr, undefined). @@ -226,25 +225,18 @@ getnameinfo(SockAddr) -> -endif. -ifdef(USE_ESOCK). -getnameinfo(SockAddr, [] = _Flags) -> - getnameinfo(SockAddr, undefined); -getnameinfo(#{family := Fam, addr := _Addr} = SockAddr, Flags) - when ((Fam =:= inet) orelse (Fam =:= inet6)) andalso - (is_list(Flags) orelse (Flags =:= undefined)) -> - prim_net:getnameinfo(socket:ensure_sockaddr(SockAddr), Flags); -getnameinfo(#{family := Fam, path := _Path} = SockAddr, Flags) - when (Fam =:= local) andalso (is_list(Flags) orelse (Flags =:= undefined)) -> +getnameinfo(SockAddr, Flags) + when is_map(SockAddr), is_list(Flags); + is_map(SockAddr), Flags =:= undefined -> prim_net:getnameinfo(SockAddr, Flags). + -else. --dialyzer({nowarn_function, getnameinfo/2}). -getnameinfo(SockAddr, [] = _Flags) -> - getnameinfo(SockAddr, undefined); -getnameinfo(#{family := Fam, addr := _Addr} = _SockAddr, Flags) - when ((Fam =:= inet) orelse (Fam =:= inet6)) andalso - (is_list(Flags) orelse (Flags =:= undefined)) -> - erlang:error(notsup); -getnameinfo(#{family := Fam, path := _Path} = _SockAddr, Flags) - when (Fam =:= local) andalso (is_list(Flags) orelse (Flags =:= undefined)) -> + +-dialyzer({no_return, getnameinfo/2}). + +getnameinfo(SockAddr, Flags) + when is_map(SockAddr), is_list(Flags); + is_map(SockAddr), Flags =:= undefined -> erlang:error(notsup). -endif. @@ -260,6 +252,8 @@ getnameinfo(#{family := Fam, path := _Path} = _SockAddr, Flags) Info :: [address_info()], Reason :: term(). +-dialyzer({no_return, getaddrinfo/1}). + getaddrinfo(Host) when is_list(Host) -> getaddrinfo(Host, undefined). @@ -285,7 +279,6 @@ getaddrinfo(Host, Service) (not ((Service =:= undefined) andalso (Host =:= undefined))) -> prim_net:getaddrinfo(Host, Service). -else. --dialyzer({nowarn_function, getaddrinfo/2}). getaddrinfo(Host, Service) when (is_list(Host) orelse (Host =:= undefined)) andalso (is_list(Service) orelse (Service =:= undefined)) andalso @@ -332,7 +325,6 @@ getifaddrs(Filter) when is_function(Filter, 1) -> getifaddrs(Namespace) when is_list(Namespace) -> prim_net:getifaddrs(#{netns => Namespace}). -else. --dialyzer({nowarn_function, getifaddrs/1}). getifaddrs(Filter) when is_atom(Filter) orelse is_map(Filter) orelse is_function(Filter) -> @@ -348,6 +340,8 @@ getifaddrs(Namespace) when is_list(Namespace) -> IfAddrs :: [ifaddrs()], Reason :: term(). +-dialyzer({no_return, getifaddrs/2}). + getifaddrs(Filter, Namespace) when (is_atom(Filter) orelse is_map(Filter)) andalso is_list(Namespace) -> do_getifaddrs(getifaddrs_filter_map(Filter), @@ -356,6 +350,8 @@ getifaddrs(Filter, Namespace) when is_function(Filter, 1) andalso is_list(Namespace) -> do_getifaddrs(Filter, fun() -> getifaddrs(Namespace) end). +-dialyzer({no_return, do_getifaddrs/2}). + do_getifaddrs(Filter, GetIfAddrs) -> case GetIfAddrs() of {ok, IfAddrs0} when is_function(Filter) -> @@ -395,6 +391,8 @@ getifaddrs_filter_map_inet6() -> getifaddrs_filter_map_packet() -> #{family => packet, flags => any}. +-compile({nowarn_unused_function, getifaddrs_filter/2}). + getifaddrs_filter(#{family := FFamily, flags := FFlags}, #{addr := #{family := Family}, flags := Flags} = _Entry) when (FFamily =:= default) andalso @@ -419,6 +417,8 @@ getifaddrs_filter(#{family := FFamily, flags := FFlags}, getifaddrs_filter(_Filter, _Entry) -> false. +-compile({nowarn_unused_function, getifaddrs_filter_flags/2}). + getifaddrs_filter_flags(any, _Flags) -> true; getifaddrs_filter_flags(FilterFlags, Flags) -> @@ -442,7 +442,6 @@ getifaddrs_filter_flags(FilterFlags, Flags) -> if_name2index(If) when is_list(If) -> prim_net:if_name2index(If). -else. --dialyzer({nowarn_function, if_name2index/1}). if_name2index(If) when is_list(If) -> erlang:error(notsup). -endif. @@ -465,7 +464,6 @@ if_name2index(If) when is_list(If) -> if_index2name(Idx) when is_integer(Idx) -> prim_net:if_index2name(Idx). -else. --dialyzer({nowarn_function, if_index2name/1}). if_index2name(Idx) when is_integer(Idx) -> erlang:error(notsup). -endif. @@ -488,7 +486,6 @@ if_index2name(Idx) when is_integer(Idx) -> if_names() -> prim_net:if_names(). -else. --dialyzer({nowarn_function, if_names/0}). if_names() -> erlang:error(notsup). -endif. diff --git a/lib/kernel/src/net_kernel.erl b/lib/kernel/src/net_kernel.erl index c93862e602..b0e6127520 100644 --- a/lib/kernel/src/net_kernel.erl +++ b/lib/kernel/src/net_kernel.erl @@ -1679,14 +1679,18 @@ epmd_module() -> %% dist_listen() -> whether the erlang distribution should listen for connections %% dist_listen() -> - case init:get_argument(dist_listen) of - {ok,[[DoListen]]} -> - list_to_atom(DoListen) =/= false; - _ -> - true + case persistent_term:get(net_kernel, undefined) of + dynamic_node_name -> + false; + _ -> + case init:get_argument(dist_listen) of + {ok,[[DoListen]]} -> + list_to_atom(DoListen) =/= false; + _ -> + true + end end. - %% %% Start all protocols %% @@ -1761,8 +1765,12 @@ wrap_creation(Cr) -> start_protos_listen(Node, Ps, CleanHalt) -> - {Name, "@"++Host} = split_node(Node), - start_protos_listen(list_to_atom(Name), Host, Node, Ps, [], CleanHalt). + case split_node(Node) of + {"undefined", _} -> + start_protos_no_listen(Node, Ps, [], CleanHalt); + {Name, "@"++Host} -> + start_protos_listen(list_to_atom(Name), Host, Node, Ps, [], CleanHalt) + end. start_protos_listen(Name, Host, Node, [Proto | Ps], Ls, CleanHalt) -> Mod = list_to_atom(Proto ++ "_dist"), try try Mod:listen(Name,Host) diff --git a/lib/kernel/src/socket.erl b/lib/kernel/src/socket.erl new file mode 100644 index 0000000000..9533066c6b --- /dev/null +++ b/lib/kernel/src/socket.erl @@ -0,0 +1,2570 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2020. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +-module(socket). + +-compile({no_auto_import,[error/1]}). + +%% Administrative and "global" utility functions +-export([ + number_of/0, + which_sockets/0, which_sockets/1, + + debug/1, socket_debug/1, + info/0, info/1, + supports/0, supports/1, supports/2, + is_supported/1, is_supported/2, is_supported/3 + ]). + +-export([ + open/1, open/2, open/3, open/4, + bind/2, bind/3, + connect/1, connect/2, connect/3, + listen/1, listen/2, + accept/1, accept/2, + + send/2, send/3, send/4, + sendto/3, sendto/4, sendto/5, + sendmsg/2, sendmsg/3, sendmsg/4, + + recv/1, recv/2, recv/3, recv/4, + recvfrom/1, recvfrom/2, recvfrom/3, recvfrom/4, + recvmsg/1, recvmsg/2, recvmsg/3, recvmsg/5, + + close/1, + shutdown/2, + + setopt/4, + getopt/3, + + sockname/1, + peername/1, + + cancel/2 + ]). + +-export_type([ + socket/0, + + select_tag/0, + select_ref/0, + select_info/0, + + socket_counters/0, + socket_info/0, + + domain/0, + type/0, + protocol/0, + + port_number/0, + ip4_address/0, + ip6_address/0, + sockaddr/0, + sockaddr_in4/0, + sockaddr_in6/0, + sockaddr_un/0, + sockaddr_ll/0, + + send_flags/0, + send_flag/0, + + recv_flags/0, + recv_flag/0, + + shutdown_how/0, + + sockopt_level/0, + otp_socket_option/0, + socket_option/0, + ip_socket_option/0, + ipv6_socket_option/0, + tcp_socket_option/0, + udp_socket_option/0, + sctp_socket_option/0, + raw_socket_option/0, + + timeval/0, + ip_tos/0, + ip_mreq/0, + ip_mreq_source/0, + ip_pmtudisc/0, + ip_msfilter_mode/0, + ip_msfilter/0, + ip_pktinfo/0, + ipv6_mreq/0, + ipv6_pmtudisc/0, + ipv6_pktinfo/0, + in6_flow_info/0, + in6_scope_id/0, + sctp_assoc_id/0, + sctp_sndrcvinfo/0, + sctp_event_subscribe/0, + sctp_assocparams/0, + sctp_initmsg/0, + sctp_rtoinfo/0, + + + msghdr_flag/0, + msghdr_flags/0, + msghdr/0, + cmsghdr_level/0, + cmsghdr_type/0, + %% cmsghdr_data/0, + cmsghdr_recv/0, cmsghdr_send/0, + + ee_origin/0, + icmp_dest_unreach/0, + icmpv6_dest_unreach/0, + extended_err/0, + + uint8/0, + uint16/0, + uint20/0, + uint32/0, + int32/0 + ]). + +%% Also in prim_socket +-define(REGISTRY, socket_registry). + + +-type socket_counters() :: #{read_byte := non_neg_integer(), + read_fails := non_neg_integer(), + read_pkg := non_neg_integer(), + read_pkg_max := non_neg_integer(), + read_tries := non_neg_integer(), + read_waits := non_neg_integer(), + write_byte := non_neg_integer(), + write_fails := non_neg_integer(), + write_pkg := non_neg_integer(), + write_pkg_max := non_neg_integer(), + write_tries := non_neg_integer(), + write_waits := non_neg_integer(), + acc_success := non_neg_integer(), + acc_fails := non_neg_integer(), + acc_tries := non_neg_integer(), + acc_waits := non_neg_integer()}. +-type socket_info() :: #{domain := domain(), + type := type(), + protocol := protocol(), + ctrl := pid(), + ctype := normal | fromfd | {fromfd, integer()}, + counters := socket_counters(), + num_readers := non_neg_integer(), + num_writers := non_neg_integer(), + num_acceptors := non_neg_integer(), + writable := boolean(), + readable := boolean()}. + +-type uint8() :: 0..16#FF. +-type uint16() :: 0..16#FFFF. +-type uint20() :: 0..16#FFFFF. +-type uint32() :: 0..16#FFFFFFFF. +-type int32() :: -2147483648..2147483647. + + +%% We support only a subset of all domains. +-type domain() :: local | inet | inet6. + +%% We support only a subset of all types. +%% RDM - Reliably Delivered Messages +-type type() :: stream | dgram | raw | rdm | seqpacket. + +%% We support only a subset of all protocols: +%% Note that the '{raw, integer()}' construct is intended +%% to be used with type = raw. +%% Note also that only the "superuser" can create a raw socket. +-type protocol() :: ip | tcp | udp | sctp | icmp | igmp | {raw, integer()}. + +-type port_number() :: 0..65535. + +-type ip4_address() :: {0..255, 0..255, 0..255, 0..255}. + +-type in6_flow_info() :: uint20(). +-type in6_scope_id() :: uint32(). + +-type ip6_address() :: + {0..65535, + 0..65535, + 0..65535, + 0..65535, + 0..65535, + 0..65535, + 0..65535, + 0..65535}. + +-type timeval() :: #{sec := integer(), + usec := integer()}. + +-type ip_pktinfo() :: #{ + ifindex := non_neg_integer(), % Interface Index + spec_dst := ip4_address(), % Local Address + addr := ip4_address() % Header Destination address + }. + +%% If the integer value is used, its up to the caller to ensure its valid! +-type ip_tos() :: lowdelay | + throughput | + reliability | + mincost | + integer(). + +%% This type is used when requesting to become member of a multicast +%% group with a call to setopt. Example: +%% +%% socket:setopt(Socket, ip, add_membership, #{multiaddr => Addr, +%% interface => any}). +%% +%% Its also used when removing from a multicast group. Example: +%% +%% socket:setopt(Socket, ip, drop_membership, #{multiaddr => Addr, +%% interface => any}). +%% + +-type ip_mreq() :: #{multiaddr := ip4_address(), + interface := any | ip4_address()}. +%% -type ip_mreqn() :: #{multiaddr := ip4_address(), +%% address := any | ip4_address(), +%% ifindex := integer()}. + +-type ip_mreq_source() :: #{multiaddr := ip4_address(), + interface := ip4_address(), + sourceaddr := ip4_address()}. + +-type ip_pmtudisc() :: want | dont | do | probe. + +%% multiaddr: Multicast group address +%% interface: Address of local interface +%% mode: Filter mode +%% slist: List of source addresses +-type ip_msfilter_mode() :: include | exclude. + +-type ip_msfilter() :: #{multiaddr := ip4_address(), + interface := ip4_address(), + mode := ip_msfilter_mode(), + slist := [ip4_address()]}. + +-type ipv6_mreq() :: #{multiaddr := ip6_address(), + interface := non_neg_integer()}. + +-type ipv6_pmtudisc() :: ip_pmtudisc(). + +-type ipv6_pktinfo() :: #{ + addr := ip6_address(), + ifindex := integer() + }. + + +-type sctp_assoc_id() :: int32(). +-type sctp_sndrcvinfo() :: #{ + stream := uint16(), + ssn := uint16(), + flags := uint16(), + ppid := uint16(), + context := uint16(), + timetolive := uint16(), + tsn := uint16(), + cumtsn := uint16(), + assoc_id := sctp_assoc_id() + }. + +-type sctp_event_subscribe() :: #{data_in := boolean(), + association := boolean(), + address := boolean(), + send_failure := boolean(), + peer_error := boolean(), + shutdown := boolean(), + partial_delivery := boolean(), + adaptation_layer := boolean(), + authentication := boolean(), + sender_dry := boolean()}. + +-type sctp_assocparams() :: #{assoc_id := sctp_assoc_id(), + max_rxt := uint16(), + num_peer_dests := uint16(), + peer_rwnd := uint32(), + local_rwnd := uint32(), + cookie_life := uint32()}. + +-type sctp_initmsg() :: #{num_outstreams := uint16(), + max_instreams := uint16(), + max_attempts := uint16(), + max_init_timeo := uint16() + }. + +-type sctp_rtoinfo() :: #{assoc_id := sctp_assoc_id(), + initial := uint32(), + max := uint32(), + min := uint32()}. + +-type sockaddr_un() :: #{family := local, + path := binary() | string()}. +-type sockaddr_in4() :: #{family := inet, + port := port_number(), + %% The 'broadcast' here is the "limited broadcast" + addr := any | broadcast | loopback | ip4_address()}. +-type sockaddr_in6() :: #{family := inet6, + port := port_number(), + addr := any | loopback | ip6_address(), + flowinfo := in6_flow_info(), + scope_id := in6_scope_id()}. +-type sockaddr_ll() :: #{family := packet, + protocol := non_neg_integer(), + ifindex := integer(), + pkttype := packet_type(), + hatype := non_neg_integer(), + addr := binary()}. +-type packet_type() :: host | broadcast | multicast | otherhost | + outgoing | loopback | user | kernel | fastroute | + non_neg_integer(). +-type sockaddr() :: sockaddr_in4() | + sockaddr_in6() | + sockaddr_un() | + sockaddr_ll(). + +%% otp - This option is internal to our (OTP) implementation. +%% socket - The socket layer (SOL_SOCKET). +%% ip - The IP layer (SOL_IP or is it IPPROTO_IP?). +%% ipv6 - The IPv6 layer (SOL_IPV6). +%% tcp - The TCP (Transport Control Protocol) layer (IPPROTO_TCP). +%% udp - The UDP (User Datagram Protocol) layer (IPPROTO_UDP). +%% sctp - The SCTP (Stream Control Transmission Protocol) layer (IPPROTO_SCTP). +%% Int - Raw level, sent down and used "as is". +%% Its up to the caller to make sure this is correct! +-type sockopt_level() :: otp | + socket | + ip | ipv6 | tcp | udp | sctp | + non_neg_integer(). + +%% There are some options that are 'read-only'. +%% Should those be included here or in a special list? +%% Should we just document it and leave it to the user? +%% Or catch it in the encode functions? +%% A setopt for a readonly option leads to einval? +%% Do we really need a sndbuf? + +-type otp_socket_option() :: debug | + iow | + controlling_process | + rcvbuf | % sndbuf | + rcvctrlbuf | + sndctrlbuf | + meta | + fd. +%% Shall we have special treatment of linger?? +%% read-only options: +%% domain | protocol | type. +%% FreeBSD (only?): acceptfilter +-type socket_option() :: acceptconn | + acceptfilter | + bindtodevice | + broadcast | + busy_poll | + debug | + domain | + dontroute | + error | + keepalive | + linger | + mark | + oobinline | + passcred | + peek_off | + peercred | + priority | + protocol | + rcvbuf | + rcvbufforce | + rcvlowat | + rcvtimeo | + reuseaddr | + reuseport | + rxq_ovfl | + setfib | + sndbuf | + sndbufforce | + sndlowat | + sndtimeo | + timestamp | + type. + +%% Read-only options: +%% mtu +%% +%% Options only valid on FreeBSD?: +%% dontfrag +%% Options only valid for RAW sockets: +%% nodefrag (linux only?) +-type ip_socket_option() :: add_membership | + add_source_membership | + block_source | + dontfrag | + drop_membership | + drop_source_membership | + freebind | + hdrincl | + minttl | + msfilter | + mtu | + mtu_discover | + multicast_all | + multicast_if | + multicast_loop | + multicast_ttl | + nodefrag | + options | + pktinfo | + recverr | + recvif | + recvdstaddr | + recvopts | + recvorigdstaddr | + recvtos | + recvttl | + retopts | + router_alert | + sndsrcaddr | + tos | + transparent | + ttl | + unblock_source. +-type ipv6_socket_option() :: + addrform | + add_membership | + authhdr | + auth_level | + checksum | + drop_membership | + dstopts | + esp_trans_level | + esp_network_level | + faith | + flowinfo | + hopopts | + ipcomp_level | + join_group | + leave_group | + mtu | + mtu_discover | + multicast_hops | + multicast_if | + multicast_loop | + portrange | + pktoptions | + recverr | + recvhoplimit | hoplimit | + recvpktinfo | pktinfo | + recvtclass | + router_alert | + rthdr | + tclass | + unicast_hops | + use_min_mtu | + v6only. + +-type tcp_socket_option() :: congestion | + cork | + info | + keepcnt | + keepidle | + keepintvl | + maxseg | + md5sig | + nodelay | + noopt | + nopush | + syncnt | + user_timeout. + +-type udp_socket_option() :: cork. + +-type sctp_socket_option() :: + adaption_layer | + associnfo | + auth_active_key | + auth_asconf | + auth_chunk | + auth_key | + auth_delete_key | + autoclose | + context | + default_send_params | + delayed_ack_time | + disable_fragments | + hmac_ident | + events | + explicit_eor | + fragment_interleave | + get_peer_addr_info | + initmsg | + i_want_mapped_v4_addr | + local_auth_chunks | + maxseg | + maxburst | + nodelay | + partial_delivery_point | + peer_addr_params | + peer_auth_chunks | + primary_addr | + reset_streams | + rtoinfo | + set_peer_primary_addr | + status | + use_ext_recvinfo. + +-type raw_socket_option() :: filter. + +%% -type plain_socket_option() :: integer(). +%% -type sockopt() :: otp_socket_option() | +%% socket_option() | +%% ip_socket_option() | +%% ipv6_socket_option() | +%% tcp_socket_option() | +%% udp_socket_option() | +%% sctp_socket_option() | +%% raw_socket_option() | +%% plain_socket_option(). + +%% The names of these macros match the names of corresponding +%%C functions in the NIF code, so a search will match both +%% +-define(socket_tag, '$socket'). +%% +%% Our socket abstract data type +-define(mk_socket(Ref), {?socket_tag, (Ref)}). +%% +%% Messages sent from the nif-code to erlang processes: +-define(mk_socket_msg(Socket, Tag, Info), {?socket_tag, (Socket), (Tag), (Info)}). + +-opaque socket() :: ?mk_socket(reference()). + +-type send_flags() :: [send_flag()]. +-type send_flag() :: confirm | + dontroute | + eor | + more | + nosignal | + oob. + +%% Note that not all of these flags are useful for every recv function! +%% +-type recv_flags() :: [recv_flag()]. +-type recv_flag() :: cmsg_cloexec | + errqueue | + oob | + peek | + trunc. + +-type shutdown_how() :: read | write | read_write. + +-type msghdr_flag() :: ctrunc | eor | errqueue | oob | trunc. +-type msghdr_flags() :: [msghdr_flag()]. +-type msghdr() :: #{ + %% *Optional* target address + %% Used on an unconnected socket to specify the + %% target address for a datagram. + addr := sockaddr(), + + iov := [binary()], + + %% The maximum size of the control buffer is platform + %% specific. It is the users responsibility to ensure + %% that its not exceeded. + ctrl := [cmsghdr_recv()] | [cmsghdr_send()], + + %% Only valid with recvmsg + flags := msghdr_flags() + }. +%% We are able to (completely) decode *some* control message headers. +%% Even if we are able to decode both level and type, we may not be +%% able to decode the data, in which case it will be a binary. + +-type cmsghdr_level() :: socket | ip | ipv6 | integer(). +-type cmsghdr_type() :: credentials | + hoplevel | + origdstaddr | + pktinfo | + recvtos | + rights | + timestamp | + tos | + ttl | + integer(). +-type cmsghdr_recv() :: + #{level := socket, type := timestamp, data := timeval()} | + #{level := socket, type := rights, data := binary()} | + #{level := socket, type := credentials, data := binary()} | + #{level := socket, type := integer(), data := binary()} | + #{level := ip, type := tos, data := ip_tos()} | + #{level := ip, type := recvtos, data := ip_tos()} | + #{level := ip, type := ttl, data := integer()} | + #{level := ip, type := recvttl, data := integer()} | + #{level := ip, type := pktinfo, data := ip_pktinfo()} | + #{level := ip, type := origdstaddr, data := sockaddr_in4()} | + #{level := ip, type := recverr, data := extended_err() | binary()} | + #{level := ip, type := integer(), data := binary()} | + #{level := ipv6, type := hoplevel, data := integer()} | + #{level := ipv6, type := pktinfo, data := ipv6_pktinfo()} | + #{level := ipv6, type := recverr, data := extended_err() | binary()} | + #{level := ipv6, type := tclass, data := integer()} | + #{level := ipv6, type := integer(), data := binary()} | + #{level := integer(), type := integer(), data := binary()}. +-type cmsghdr_send() :: + #{level := socket, type := timestamp, data := binary()} | + #{level := socket, type := rights, data := binary()} | + #{level := socket, type := credentials, data := binary()} | + #{level := socket, type := integer(), data := binary()} | + #{level := ip, type := tos, data := ip_tos() | binary()} | + #{level := ip, type := ttl, data := integer() | binary()} | + #{level := ip, type := integer(), data := binary()} | + #{level := ipv6, type := tclass, data := integer()} | + #{level := ipv6, type := integer(), data := binary()} | + #{level := udp, type := integer(), data := binary()} | + #{level := integer(), type := integer(), data := binary()}. + +-type ee_origin() :: none | local | icmp | icmp6 | uint8(). +-type icmp_dest_unreach() :: net_unreach | host_unreach | port_unreach | frag_needed | + net_unknown | host_unknown | uint8(). +-type icmpv6_dest_unreach() :: noroute | adm_prohibited | not_neighbour | addr_unreach | + port_unreach | policy_fail | reject_route | uint8(). +-type extended_err() :: + #{error := term(), + origin := icmp, + type := dest_unreach, + code := icmp_dest_unreach(), + info := uint32(), + data := uint32(), + offender := undefined | sockaddr()} | + #{error := term(), + origin := icmp, + type := time_exceeded | uint8(), + code := uint8(), + info := uint32(), + data := uint32(), + offender := undefined | sockaddr()} | + #{error := term(), + origin := icmp6, + type := dest_unreach, + code := icmpv6_dest_unreach(), + info := uint32(), + data := uint32(), + offender := undefined | sockaddr()} | + #{error := term(), + origin := icmp6, + type := pkt_toobig | time_exceeded | uint8(), + code := uint8(), + info := uint32(), + data := uint32(), + offender := undefined | sockaddr()} | + #{error := term(), + origin := ee_origin(), + type := uint8(), + code := uint8(), + info := uint32(), + data := uint32(), + offender := undefined | sockaddr()}. + +-type errcode() :: + inet:posix() | % closed | timeout | not_owner | + exalloc | exmonitor | exselect | exself. + +%% =========================================================================== +%% +%% Interface term formats +%% + +-opaque select_tag() :: atom(). +-opaque select_ref() :: reference(). + +-type select_info() :: {select_info, select_tag(), select_ref()}. + +-define(SELECT_INFO(T, R), {select_info, T, R}). +-define(SELECT(T, R), {select, ?SELECT_INFO(T, R)}). + + +%% =========================================================================== +%% +%% Defaults +%% + +-define(ESOCK_LISTEN_BACKLOG_DEFAULT, 5). + +-define(ESOCK_ACCEPT_TIMEOUT_DEFAULT, infinity). + +-define(ESOCK_SEND_FLAGS_DEFAULT, []). +-define(ESOCK_SEND_TIMEOUT_DEFAULT, infinity). +-define(ESOCK_SENDTO_FLAGS_DEFAULT, []). +-define(ESOCK_SENDTO_TIMEOUT_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). +-define(ESOCK_SENDMSG_FLAGS_DEFAULT, []). +-define(ESOCK_SENDMSG_TIMEOUT_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). + +-define(ESOCK_RECV_FLAGS_DEFAULT, []). +-define(ESOCK_RECV_TIMEOUT_DEFAULT, infinity). + + +%% =========================================================================== +%% +%% Administrative and utility API +%% +%% =========================================================================== + +%% *** number_of *** +%% +%% Interface function to the socket registry +%% returns the number of existing (and "alive") sockets. +%% +-spec number_of() -> non_neg_integer(). + +number_of() -> + ?REGISTRY:number_of(). + + +%% *** which_sockets/0,1 *** +%% +%% Interface function to the socket registry +%% Returns a list of all the sockets, accoring to the filter rule. +%% +-spec which_sockets() -> [socket()]. + +which_sockets() -> + ?REGISTRY:which_sockets(fun(_) -> true end). + +-spec which_sockets(FilterRule) -> [socket()] when + FilterRule :: inet | inet6 | + stream | dgram | seqpacket | + sctp | tcp | udp | + pid() | + fun((socket_info()) -> boolean()). + +which_sockets(Domain) + when ((Domain =:= inet) orelse (Domain =:= inet6)) -> + ?REGISTRY:which_sockets(fun(#{domain := D}) when (D =:= Domain) -> true; + (_) -> false end); +which_sockets(Type) + when ((Type =:= stream) orelse (Type =:= dgram) orelse (Type =:= seqpacket)) -> + ?REGISTRY:which_sockets(fun(#{type := T}) when (T =:= Type) -> true; + (_) -> false end); +which_sockets(Proto) + when ((Proto =:= sctp) orelse (Proto =:= tcp) orelse (Proto =:= udp)) -> + ?REGISTRY:which_sockets(fun(#{protocol := P}) when (P =:= Proto) -> true; + (_) -> false end); +which_sockets(CTRL) + when is_pid(CTRL) -> + ?REGISTRY:which_sockets(fun(#{ctrl := C}) when (C =:= CTRL) -> true; + (_) -> false end); +which_sockets(Filter) when is_function(Filter, 1) -> + ?REGISTRY:which_sockets(Filter). + + +%% =========================================================================== +%% +%% Debug features +%% +%% =========================================================================== + + +-spec info() -> map(). +%% +info() -> + prim_socket:info(). + + +-spec debug(D :: boolean()) -> ok. +%% +debug(D) when is_boolean(D) -> + prim_socket:debug(D). + + +-spec socket_debug(D :: boolean()) -> ok. +%% +socket_debug(D) when is_boolean(D) -> + prim_socket:socket_debug(D). + + +%% =========================================================================== +%% +%% info - Get miscellaneous information about a socket. +%% +%% Generates a list of various info about the socket, such as counter values. +%% +%% Do *not* call this function often. +%% +%% =========================================================================== + +-spec info(Socket) -> socket_info() when + Socket :: socket(). +%% +info(?mk_socket(SockRef)) when is_reference(SockRef) -> + prim_socket:info(SockRef); +info(Socket) -> + erlang:error(badarg, [Socket]). + + +%% =========================================================================== +%% +%% supports - get information about what the platform "supports". +%% +%% Generates a list of various info about what the plaform can support. +%% The most obvious case is 'options'. +%% +%% Each item in a 'supports'-list will appear only *one* time. +%% +%% =========================================================================== + +-spec supports() -> [{Key1 :: term(), + boolean() | [{Key2 :: term(), + boolean() | [{Key3 :: term(), + boolean()}]}]}]. +supports() -> + [{Key1, supports(Key1)} + || Key1 <- [options, send_flags, recv_flags]] + ++ prim_socket:supports(). + +-spec supports(Key1 :: term()) -> + [{Key2 :: term(), + boolean() | [{Key3 :: term(), + boolean()}]}]. +%% +supports(options) -> + [{Level, supports(options, Level)} + || Level <- [socket, ip, ipv6, tcp, udp, sctp]]; +supports(Key) -> + prim_socket:supports(Key). + +-spec supports(Key1 :: term(), Key2 :: term()) -> + [{Key3 :: term(), + boolean()}]. +%% +supports(Key1, Key2) -> + prim_socket:supports(Key1, Key2). + + +-spec is_supported(Key1 :: term()) -> + boolean(). +is_supported(Key1) -> + get_is_supported(Key1, supports()). +%% +-spec is_supported(Key1 :: term(), Key2 :: term()) -> + boolean(). +is_supported(Key1, Key2) -> + get_is_supported(Key2, supports(Key1)). +%% +-spec is_supported(Key1 :: term(), Key2 :: term(), Key3 :: term()) -> + boolean(). +is_supported(Key1, Key2, Key3) -> + get_is_supported(Key3, supports(Key1, Key2)). + + +get_is_supported(Key, Supported) -> + case lists:keyfind(Key, 1, Supported) of + false -> + false; + {_, Value} -> + if + is_boolean(Value) -> + Value; + is_list(Value) -> + false + end + end. + + +%% =========================================================================== +%% +%% The proper socket API +%% +%% =========================================================================== + +%% =========================================================================== +%% +%% <KOLLA> +%% +%% The nif sets up a monitor to this process, and if it dies the socket +%% is closed. It is also used if someone wants to monitor the socket. +%% +%% We may therefor need monitor function(s): +%% +%% socket:monitor(Socket) +%% socket:demonitor(Socket) +%% +%% </KOLLA> +%% + +%% =========================================================================== +%% +%% open - create an endpoint for communication +%% + +-spec open(FD) -> {ok, Socket} | {error, Reason} when + FD :: integer(), + Socket :: socket(), + Reason :: errcode(). + +open(FD) when is_integer(FD) -> + open(FD, #{}); +open(FD) -> + erlang:error(badarg, [FD]). + +-spec open(FD, Opts) -> {ok, Socket} | {error, Reason} when + FD :: integer(), + Opts :: + #{domain => domain(), + type => type(), + protocol => protocol(), + dup => boolean()}, + Socket :: socket(), + Reason :: errcode(); + + (Domain, Type) -> {ok, Socket} | {error, Reason} when + Domain :: domain(), + Type :: type(), + Socket :: socket(), + Reason :: errcode(). + +open(FD, Opts) when is_integer(FD), is_map(Opts) -> + case prim_socket:open(FD, Opts) of + {ok, SockRef} -> + Socket = ?mk_socket(SockRef), + {ok, Socket}; + {error, _} = ERROR -> + ERROR + end; +open(Domain, Type) -> + open(Domain, Type, default). + +-spec open(Domain, Type, Protocol) -> {ok, Socket} | {error, Reason} when + Domain :: domain(), + Type :: type(), + Protocol :: default | protocol(), + Socket :: socket(), + Reason :: errcode(). + +open(Domain, Type, Protocol) -> + open(Domain, Type, Protocol, #{}). + +-spec open(Domain, Type, Protocol, Opts) -> {ok, Socket} | {error, Reason} when + Domain :: domain(), + Type :: type(), + Protocol :: default | protocol(), + Opts :: map(), + Socket :: socket(), + Reason :: errcode(). + +open(Domain, Type, Protocol, Opts) when is_map(Opts) -> + case prim_socket:open(Domain, Type, Protocol, Opts) of + {ok, SockRef} -> + Socket = ?mk_socket(SockRef), + {ok, Socket}; + {error, _} = ERROR -> + ERROR + end; +open(Domain, Type, Protocol, Opts) -> + erlang:error(badarg, [Domain, Type, Protocol, Opts]). + + +%% =========================================================================== +%% +%% bind - bind a name (an address) to a socket +%% +%% Note that the short (atom) addresses only work for some domains, +%% and that the nif will reject 'broadcast' for other domains than 'inet' +%% + +-spec bind(Socket, Addr) -> {ok, Port} | {error, Reason} when + Socket :: socket(), + Addr :: sockaddr() | any | broadcast | loopback, + Port :: port_number(), + Reason :: inet:posix() | closed. + +bind(?mk_socket(SockRef) = Socket, Addr) when is_reference(SockRef) -> + if + is_map(Addr) -> + prim_socket:bind(SockRef, Addr); + %% + Addr =:= any; + Addr =:= broadcast; + Addr =:= loopback -> + case prim_socket:getopt(SockRef, otp, domain) of + {ok, Domain} + when Domain =:= inet; + Domain =:= inet6 -> + prim_socket:bind( + SockRef, #{family => Domain, addr => Addr}); + {ok, _Domain} -> + {error, eafnosupport}; + {error, _} = ERROR -> + ERROR + end; + %% + true -> + erlang:error(badarg, [Socket, Addr]) + end; +bind(Socket, Addr) -> + erlang:error(badarg, [Socket, Addr]). + + +%% =========================================================================== +%% +%% bind - Add or remove a bind addresses on a socket +%% +%% Calling this function is only valid if the socket is: +%% type = seqpacket +%% protocol = sctp +%% +%% If the domain is inet, then all addresses *must* be IPv4. +%% If the domain is inet6, the addresses can be aither IPv4 or IPv6. +%% + +-spec bind(Socket, Addrs, Action) -> ok | {error, Reason} when + Socket :: socket(), + Addrs :: [sockaddr()], + Action :: add | remove, + Reason :: inet:posix() | closed. + +bind(?mk_socket(SockRef), Addrs, Action) + when is_reference(SockRef) + andalso is_list(Addrs) + andalso (Action =:= add + orelse Action =:= remove) -> + prim_socket:bind(SockRef, Addrs, Action); +bind(Socket, Addrs, Action) -> + erlang:error(badarg, [Socket, Addrs, Action]). + + +%% =========================================================================== +%% +%% connect - initiate a connection on a socket +%% + +-spec connect(Socket) -> ok | {error, Reason} when + Socket :: socket(), + Reason :: errcode() | closed. + +%% Finalize connect after connect(,, nowait) and received +%% select message - see connect_deadline/3 +%% +connect(?mk_socket(SockRef)) + when is_reference(SockRef) -> + prim_socket:connect(SockRef); +connect(Socket) -> + erlang:error(badarg, [Socket]). + + +-spec connect(Socket, SockAddr) -> ok | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + Reason :: errcode() | closed. + +connect(Socket, SockAddr) -> + connect(Socket, SockAddr, infinity). + + +-spec connect(Socket, SockAddr, nowait) -> + ok | {select, SelectInfo} | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, SockAddr, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + Timeout :: timeout(), + Reason :: errcode() | closed | timeout. + +%% <KOLLA> +%% Is it possible to connect with family = local for the (dest) sockaddr? +%% </KOLLA> +connect(?mk_socket(SockRef) = Socket, SockAddr, Timeout) + when is_reference(SockRef) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, SockAddr, Timeout]); + nowait -> + connect_nowait(SockRef, SockAddr); + Deadline -> + connect_deadline(SockRef, SockAddr, Deadline) + end; +connect(Socket, SockAddr, Timeout) -> + erlang:error(badarg, [Socket, SockAddr, Timeout]). + +connect_nowait(SockRef, SockAddr) -> + case prim_socket:connect(SockRef, SockAddr) of + {select, Ref} -> + ?SELECT(connect, Ref); + Result -> + Result + end. + +connect_deadline(SockRef, SockAddr, Deadline) -> + case prim_socket:connect(SockRef, SockAddr) of + {select, Ref} -> + %% Connecting... + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(_Socket, select, Ref) -> + prim_socket:connect(SockRef); + ?mk_socket_msg(_Socket, abort, {Ref, Reason}) -> + {error, Reason} + after Timeout -> + cancel(SockRef, connect, Ref), + {error, timeout} + end; + Result -> + Result + end. + + +%% =========================================================================== +%% +%% listen - listen for connections on a socket +%% + +-spec listen(Socket) -> ok | {error, Reason} when + Socket :: socket(), + Reason :: inet:posix() | closed. + +listen(Socket) -> + listen(Socket, ?ESOCK_LISTEN_BACKLOG_DEFAULT). + +-spec listen(Socket, Backlog) -> ok | {error, Reason} when + Socket :: socket(), + Backlog :: integer(), + Reason :: inet:posix() | closed. + +listen(?mk_socket(SockRef), Backlog) + when is_reference(SockRef), is_integer(Backlog) -> + prim_socket:listen(SockRef, Backlog); +listen(Socket, Backlog) -> + erlang:error(badarg, [Socket, Backlog]). + + +%% =========================================================================== +%% +%% accept, accept4 - accept a connection on a socket +%% + +-spec accept(LSocket) -> {ok, Socket} | {error, Reason} when + LSocket :: socket(), + Socket :: socket(), + Reason :: errcode() | closed. + +accept(Socket) -> + accept(Socket, ?ESOCK_ACCEPT_TIMEOUT_DEFAULT). + +-spec accept(LSocket, nowait) -> + {ok, Socket} | + {select, SelectInfo} | + {error, Reason} when + LSocket :: socket(), + Socket :: socket(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (LSocket, Timeout) -> {ok, Socket} | {error, Reason} when + LSocket :: socket(), + Timeout :: timeout(), + Socket :: socket(), + Reason :: errcode() | closed | timeout. + +accept(?mk_socket(LSockRef) = Socket, Timeout) + when is_reference(LSockRef) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, Timeout]); + nowait -> + accept_nowait(LSockRef); + Deadline -> + accept_deadline(LSockRef, Deadline) + end; +accept(Socket, Timeout) -> + erlang:error(badarg, [Socket, Timeout]). + +accept_nowait(LSockRef) -> + AccRef = make_ref(), + case prim_socket:accept(LSockRef, AccRef) of + select -> + ?SELECT(accept, AccRef); + Result -> + accept_result(LSockRef, AccRef, Result) + end. + +accept_deadline(LSockRef, Deadline) -> + AccRef = make_ref(), + case prim_socket:accept(LSockRef, AccRef) of + select -> + %% Each call is non-blocking, but even then it takes + %% *some* time, so just to be sure, recalculate before + %% the receive. + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(?mk_socket(LSockRef), select, AccRef) -> + accept_deadline(LSockRef, Deadline); + ?mk_socket_msg(_Socket, abort, {AccRef, Reason}) -> + {error, Reason} + after Timeout -> + cancel(LSockRef, accept, AccRef), + {error, timeout} + end; + Result -> + accept_result(LSockRef, AccRef, Result) + end. + +accept_result(LSockRef, AccRef, Result) -> + case Result of + {ok, SockRef} -> + Socket = ?mk_socket(SockRef), + {ok, Socket}; + {error, _} = ERROR -> + cancel(LSockRef, accept, AccRef), % Just to be on the safe side... + ERROR + end. + + +%% =========================================================================== +%% +%% send, sendto, sendmsg - send a message on a socket +%% + +-spec send(Socket, Data) -> ok | {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Reason :: term(). + +send(Socket, Data) -> + send(Socket, Data, ?ESOCK_SEND_FLAGS_DEFAULT, ?ESOCK_SEND_TIMEOUT_DEFAULT). + +-spec send(Socket, Data, Flags) -> ok | {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Flags :: send_flags(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}; + + (Socket, Data, Timeout :: nowait) -> + ok | + {ok, {binary(), SelectInfo}} | + {select, SelectInfo} | + {ok, {RestData, SelectInfo}} | + {error, Reason} when + Socket :: socket(), + Data :: iodata(), + RestData :: binary(), + SelectInfo :: select_info(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}; + + (Socket, Data, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Timeout :: timeout(), + Reason :: {errcode() | closed | timeout, + Remaining :: pos_integer()}. + +send(Socket, Data, Flags) when is_list(Flags) -> + send(Socket, Data, Flags, ?ESOCK_SEND_TIMEOUT_DEFAULT); +send(Socket, Data, Timeout) -> + send(Socket, Data, ?ESOCK_SEND_FLAGS_DEFAULT, Timeout). + +-spec send(Socket, Data, Flags, nowait) -> ok | + {select, SelectInfo} | + {ok, {RestData, SelectInfo}} | + {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Flags :: send_flags(), + RestData :: binary(), + SelectInfo :: select_info(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}; + + (Socket, Data, Flags, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Flags :: send_flags(), + Timeout :: timeout(), + Reason :: {errcode() | closed | timeout, + Remaining :: pos_integer()}. + +send(Socket, Data, Flags, Timeout) when is_list(Data) -> + Bin = erlang:list_to_binary(Data), + send(Socket, Bin, Flags, Timeout); +send(?mk_socket(SockRef) = Socket, Data, Flags, Timeout) + when is_reference(SockRef), is_binary(Data), is_list(Flags) -> + To = undefined, + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, Data, Flags, Timeout]); + nowait -> + send_common_nowait(SockRef, Data, To, Flags, send); + Deadline -> + send_common_deadline(SockRef, Data, To, Flags, Deadline, send) + end; +send(Socket, Data, Flags, Timeout) -> + erlang:error(badarg, [Socket, Data, Flags, Timeout]). + +send_common_nowait(SockRef, Data, To, Flags, SendName) -> + SendRef = make_ref(), + case + case SendName of + send -> + prim_socket:send(SockRef, SendRef, Data, Flags); + sendto -> + prim_socket:sendto(SockRef, SendRef, Data, To, Flags) + end + of + {ok, Written} -> + %% We are partially done, but the user don't want to wait (here) + %% for completion + <<_:Written/binary, Rest/binary>> = Data, + {ok, {Rest, ?SELECT_INFO(SendName, SendRef)}}; + select -> + ?SELECT(SendName, SendRef); + Result -> + send_common_result(Data, Result) + end. + +send_common_deadline(SockRef, Data, To, Flags, Deadline, SendName) -> + SendRef = make_ref(), + case + case SendName of + send -> + prim_socket:send(SockRef, SendRef, Data, Flags); + sendto -> + prim_socket:sendto(SockRef, SendRef, Data, To, Flags) + end + of + {ok, Written} -> + %% We are partially done, wait for continuation + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(_Socket, select, SendRef) + when (Written > 0) -> + <<_:Written/binary, Rest/binary>> = Data, + send_common_deadline( + SockRef, Rest, To, Flags, Deadline, SendName); + ?mk_socket_msg(_Socket, select, SendRef) -> + send_common_deadline( + SockRef, Data, To, Flags, Deadline, SendName); + ?mk_socket_msg(_Socket, abort, {SendRef, Reason}) -> + {error, {Reason, byte_size(Data)}} + after Timeout -> + _ = cancel(SockRef, SendName, SendRef), + {error, {timeout, byte_size(Data)}} + end; + select -> + %% Wait for continuation + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(_Socket, select, SendRef) -> + send_common_deadline( + SockRef, Data, To, Flags, Deadline, SendName); + ?mk_socket_msg(_Socket, abort, {SendRef, Reason}) -> + {error, {Reason, byte_size(Data)}} + after Timeout -> + _ = cancel(SockRef, SendName, SendRef), + {error, {timeout, byte_size(Data)}} + end; + %% + {error, ealready = Reason} -> + %% Internal error: + %% we called send, got eagain, and called send again + %% - without waiting for select message + erlang:error(Reason); + Result -> + send_common_result(Data, Result) + end. + +send_common_result(Data, Result) -> + case Result of + ok -> + ok; + {error, Reason} -> + {error, {Reason, byte_size(Data)}} + end. + + +%% --------------------------------------------------------------------------- +%% + +-spec sendto(Socket, Data, Dest) -> + ok | {error, Reason} when + Socket :: socket(), + Data :: binary(), + Dest :: sockaddr(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}. + +sendto(Socket, Data, Dest) -> + sendto(Socket, Data, Dest, ?ESOCK_SENDTO_FLAGS_DEFAULT). + +-spec sendto(Socket, Data, Dest, Flags) -> + ok | {error, Reason} when + Socket :: socket(), + Data :: binary(), + Dest :: sockaddr(), + Flags :: send_flags(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}; + + (Socket, Data, Dest, Timeout :: nowait) -> + ok | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Dest :: sockaddr(), + SelectInfo :: select_info(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}; + + (Socket, Data, Dest, Timeout) -> + ok | {error, Reason} when + Socket :: socket(), + Data :: iodata(), + Dest :: sockaddr(), + Timeout :: timeout(), + Reason :: {errcode() | closed | timeout, + Remaining :: pos_integer()}. + +sendto(Socket, Data, Dest, Flags) when is_list(Flags) -> + sendto(Socket, Data, Dest, Flags, ?ESOCK_SENDTO_TIMEOUT_DEFAULT); +sendto(Socket, Data, Dest, Timeout) -> + sendto(Socket, Data, Dest, ?ESOCK_SENDTO_FLAGS_DEFAULT, Timeout). + + +-spec sendto(Socket, Data, Dest, Flags, nowait) -> + ok | + {ok, {binary(), SelectInfo}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Data :: binary(), + Dest :: sockaddr(), + Flags :: send_flags(), + SelectInfo :: select_info(), + Reason :: {errcode() | closed, + Remaining :: pos_integer()}; + + (Socket, Data, Dest, Flags, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + Data :: binary(), + Dest :: sockaddr(), + Flags :: send_flags(), + Timeout :: timeout(), + Reason :: {errcode() | closed | timeout, + Remaining :: pos_integer()}. + +sendto(Socket, Data, Dest, Flags, Timeout) when is_list(Data) -> + Bin = erlang:list_to_binary(Data), + sendto(Socket, Bin, Dest, Flags, Timeout); +sendto(?mk_socket(SockRef) = Socket, Data, Dest, Flags, Timeout) + when is_reference(SockRef), is_binary(Data), is_list(Flags) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, Data, Dest, Flags, Timeout]); + nowait -> + send_common_nowait(SockRef, Data, Dest, Flags, sendto); + Deadline -> + send_common_deadline( + SockRef, Data, Dest, Flags, Deadline, sendto) + end; +sendto(Socket, Data, Dest, Flags, Timeout) -> + erlang:error(badarg, [Socket, Data, Dest, Flags, Timeout]). + + +%% --------------------------------------------------------------------------- +%% +%% The only part of the msghdr() that *must* exist (a connected +%% socket need not specify the addr field) is the iov. +%% The ctrl field is optional, and the addr and flags are not +%% used when sending. +%% + +-spec sendmsg(Socket, MsgHdr) -> + ok | + {ok, Remaining} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Remaining :: erlang:iovec(), + Reason :: term(). + +sendmsg(Socket, MsgHdr) -> + sendmsg(Socket, MsgHdr, + ?ESOCK_SENDMSG_FLAGS_DEFAULT, ?ESOCK_SENDMSG_TIMEOUT_DEFAULT). + + +-spec sendmsg(Socket, MsgHdr, Flags) -> ok | {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Flags :: send_flags(), + Reason :: errcode() | closed; + + (Socket, MsgHdr, Timeout :: nowait) -> + ok | + {ok, Remaining} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Remaining :: erlang:iovec(), + Reason :: errcode() | closed; + + (Socket, MsgHdr, Timeout) -> ok | {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Timeout :: timeout(), + Reason :: errcode() | closed | timeout. + +sendmsg(Socket, MsgHdr, Flags) when is_list(Flags) -> + sendmsg(Socket, MsgHdr, Flags, ?ESOCK_SENDMSG_TIMEOUT_DEFAULT); +sendmsg(Socket, MsgHdr, Timeout) -> + sendmsg(Socket, MsgHdr, ?ESOCK_SENDMSG_FLAGS_DEFAULT, Timeout). + + +-spec sendmsg(Socket, MsgHdr, Flags, nowait) -> + ok | + {ok, Remaining} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Flags :: send_flags(), + Remaining :: erlang:iovec(), + Reason :: errcode() | closed; + + (Socket, MsgHdr, Flags, Timeout) -> + ok | + {ok, Remaining} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Flags :: send_flags(), + Timeout :: timeout(), + Remaining :: erlang:iovec(), + Reason :: errcode() | closed | timeout. + +sendmsg(?mk_socket(SockRef) = Socket, MsgHdr, Flags, Timeout) + when is_reference(SockRef), is_map(MsgHdr), is_list(Flags) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, MsgHdr, Flags, Timeout]); + Deadline -> + sendmsg_loop(SockRef, MsgHdr, Flags, Deadline) + end; +sendmsg(Socket, MsgHdr, Flags, Timeout) -> + erlang:error(badarg, [Socket, MsgHdr, Flags, Timeout]). + +sendmsg_loop(SockRef, MsgHdr, Flags, Deadline) -> + SendRef = make_ref(), + case prim_socket:sendmsg(SockRef, SendRef, MsgHdr, Flags) of + ok -> + %% We are done + ok; + %% + {ok, Written} when is_integer(Written) andalso (Written > 0) -> + %% We should not retry here since the protocol may not + %% be able to handle a message being split. Leave it to + %% the caller to figure out (call again with the rest). + %% + %% We need to cancel this partial write. + %% + _ = cancel(SockRef, sendmsg, SendRef), + {ok, sendmsg_rest(maps:get(iov, MsgHdr), Written)}; + %% + select when (Deadline =:= nowait) -> + ?SELECT(sendmsg, SendRef); + select -> + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(?mk_socket(SockRef), select, SendRef) -> + sendmsg_loop(SockRef, MsgHdr, Flags, Deadline); + ?mk_socket_msg(_Socket, abort, {SendRef, Reason}) -> + {error, Reason} + after Timeout -> + _ = cancel(SockRef, sendmsg, SendRef), + {error, timeout} + end; + %% + {error, ealready = Reason} when Deadline =/= nowait -> + %% Internal error: + %% we called send, got eagain, and called send again + %% - without waiting for select message + erlang:error(Reason); + {error, _} = ERROR -> + ERROR + end. + +sendmsg_rest([B|IOVec], Written) when Written >= byte_size(B) -> + sendmsg_rest(IOVec, Written - byte_size(B)); +sendmsg_rest([B|IOVec], Written) -> + <<_:Written/binary, Rest/binary>> = B, + [Rest|IOVec]. + + +%% =========================================================================== +%% +%% recv, recvfrom, recvmsg - receive a message from a socket +%% +%% Description: +%% There is a special case for the argument Length. If its set to zero (0), +%% it means "give me everything you have". +%% +%% Returns: {ok, Binary} | {error, Reason} +%% Binary - The received data as a binary +%% Reason - The error reason: +%% timeout | {timeout, AccData} | +%% posix() | {posix(), AccData} | +%% atom() | {atom(), AccData} +%% AccData - The data (as a binary) that we did manage to receive +%% before the timeout. +%% +%% Arguments: +%% Socket - The socket to read from. +%% Length - The number of bytes to read. +%% Flags - A list of "options" for the read. +%% Timeout - Time-out in milliseconds. + +-spec recv(Socket) -> + {ok, Data} | + {error, Reason} when + Socket :: socket(), + Data :: binary(), + Reason :: + errcode() | closed | + {errcode() | closed, Data :: binary()}. + +recv(Socket) -> + recv(Socket, 0). + +-spec recv(Socket, Length) -> + {ok, Data} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Data :: binary(), + Reason :: + errcode() | closed | + {errcode() | closed, Data :: binary()}. + +recv(Socket, Length) -> + recv(Socket, Length, + ?ESOCK_RECV_FLAGS_DEFAULT, + ?ESOCK_RECV_TIMEOUT_DEFAULT). + +-spec recv(Socket, Length, Flags) -> + {ok, Data} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Flags :: recv_flags(), + Data :: binary(), + Reason :: + errcode() | closed | + {errcode() | closed, Data :: binary()}; + + (Socket, Length, Timeout :: nowait) -> + {ok, Data} | + {ok, {Data, SelectInfo}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Data :: binary(), + SelectInfo :: select_info(), + Reason :: + errcode() | closed | + {errcode() | closed, Data :: binary()}; + + (Socket, Length, Timeout) -> + {ok, Data} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Timeout :: timeout(), + Data :: binary(), + Reason :: + errcode() | closed | timeout | + {errcode() | closed | timeout, Data :: binary()}. + +recv(Socket, Length, Flags) when is_list(Flags) -> + recv(Socket, Length, Flags, ?ESOCK_RECV_TIMEOUT_DEFAULT); +recv(Socket, Length, Timeout) -> + recv(Socket, Length, ?ESOCK_RECV_FLAGS_DEFAULT, Timeout). + +-spec recv(Socket, Length, Flags, nowait) -> + {ok, Data} | + {ok, {Data, SelectInfo}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Flags :: recv_flags(), + Data :: binary(), + SelectInfo :: select_info(), + Reason :: + errcode() | closed | + {errcode() | closed, Data :: binary()}; + + (Socket, Length, Flags, Timeout) -> + {ok, Data} | + {error, Reason} when + Socket :: socket(), + Length :: non_neg_integer(), + Flags :: recv_flags(), + Timeout :: timeout(), + Data :: binary(), + Reason :: + errcode() | closed | timeout | + {errcode() | closed | timeout, Data :: binary()}. + +recv(?mk_socket(SockRef) = Socket, Length, Flags, Timeout) + when is_reference(SockRef), + is_integer(Length), Length >= 0, + is_list(Flags) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, Length, Flags, Timeout]); + nowait -> + recv_nowait(SockRef, Length, Flags, <<>>); + Deadline -> + recv_deadline(SockRef, Length, Flags, Deadline, <<>>) + end; +recv(Socket, Length, Flags, Timeout) -> + erlang:error(badarg, [Socket, Length, Flags, Timeout]). + +%% We will only recurse with Length == 0 if Length is 0, +%% so Length == 0 means to return all available data also when recursing + +recv_nowait(SockRef, Length, Flags, Acc) -> + RecvRef = make_ref(), + case prim_socket:recv(SockRef, RecvRef, Length, Flags) of + {more, Bin} -> + %% We got what we requested but will not waste more time + %% although there might be more data available + {ok, bincat(Acc, Bin)}; + {select, Bin} -> + %% We got less than requested so the caller will + %% get a select message when there might be more to read + {ok, {bincat(Acc, Bin), ?SELECT_INFO(recv, RecvRef)}}; + select -> + %% The caller will get a select message when there + %% might me data to read + if + byte_size(Acc) =:= 0 -> + ?SELECT(recv, RecvRef); + true -> + {ok, {Acc, ?SELECT_INFO(recv, RecvRef)}} + end; + Result -> + recv_result(Acc, Result) + end. + +recv_deadline(SockRef, Length, Flags, Deadline, Acc) -> + RecvRef = make_ref(), + case prim_socket:recv(SockRef, RecvRef, Length, Flags) of + {more, Bin} -> + %% There is more data readily available + %% - repeat unless time's up + Timeout = timeout(Deadline), + if + 0 < Timeout -> + %% Recv more + recv_deadline( + SockRef, Length, Flags, Deadline, bincat(Acc, Bin)); + true -> + {ok, bincat(Acc, Bin)} + end; + %% + {select, Bin} -> + %% We got less than requested + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(?mk_socket(SockRef), select, RecvRef) -> + if + 0 < Timeout -> + %% Recv more + recv_deadline( + SockRef, Length - byte_size(Bin), Flags, + Deadline, bincat(Acc, Bin)); + true -> + {error, {timeout, bincat(Acc, Bin)}} + end; + ?mk_socket_msg(_Socket, abort, {RecvRef, Reason}) -> + {error, {Reason, bincat(Acc, Bin)}} + after Timeout -> + cancel(SockRef, recv, RecvRef), + {error, {timeout, bincat(Acc, Bin)}} + end; + %% + select when Length =:= 0, 0 < byte_size(Acc) -> + %% We first got some data and are then asked to wait, + %% but we only want the first that comes + %% - cancel and return what we have + cancel(SockRef, recv, RecvRef), + {ok, Acc}; + select -> + %% There is nothing just now, but we will be notified when there + %% is something to read (a select message). + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(?mk_socket(SockRef), select, RecvRef) -> + if + 0 < Timeout -> + %% Retry + recv_deadline( + SockRef, Length, Flags, Deadline, Acc); + true -> + recv_error(Acc, timeout) + end; + ?mk_socket_msg(_Socket, abort, {RecvRef, Reason}) -> + recv_error(Acc, Reason) + after Timeout -> + cancel(SockRef, recv, RecvRef), + recv_error(Acc, timeout) + end; + %% + {error, ealready = Reason} -> + %% Internal error: + %% we called recv, got eagain, and called recv again + %% - without waiting for select message + erlang:error(Reason); + Result -> + recv_result(Acc, Result) + end. + +recv_result(Acc, Result) -> + case Result of + {ok, Bin} -> + {ok, bincat(Acc, Bin)}; + {error, _} = ERROR when byte_size(Acc) =:= 0 -> + ERROR; + {error, Reason} -> + {error, {Reason, Acc}} + end. + +recv_error(Acc, Reason) -> + if + byte_size(Acc) =:= 0 -> + {error, Reason}; + true -> + {error, {Reason, Acc}} + end. + +%% --------------------------------------------------------------------------- +%% +%% With recvfrom we get messages, which means that regardless of how +%% much we want to read, we return when we get a message. +%% The MaxSize argument basically defines the size of our receive +%% buffer. By setting the size to zero (0), we use the configured +%% size (see setopt). +%% It may be impossible to know what (buffer) size is appropriate +%% "in advance", and in those cases it may be convenient to use the +%% (recv) 'peek' flag. When this flag is provided the message is *not* +%% "consumed" from the underlying (OS) buffers, so another recvfrom call +%% is needed, possibly with a then adjusted buffer size. +%% + +-spec recvfrom(Socket) -> {ok, {Source, Data}} | {error, Reason} when + Socket :: socket(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: errcode() | closed. + +recvfrom(Socket) -> + recvfrom(Socket, 0). + +-spec recvfrom(Socket, BufSz) -> {ok, {Source, Data}} | {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: errcode() | closed. + +recvfrom(Socket, BufSz) -> + recvfrom(Socket, BufSz, + ?ESOCK_RECV_FLAGS_DEFAULT, + ?ESOCK_RECV_TIMEOUT_DEFAULT). + +-spec recvfrom(Socket, Flags, nowait) -> + {ok, {Source, Data}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + Source :: sockaddr() | undefined, + Data :: binary(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, Flags, Timeout) -> + {ok, {Source, Data}} | + {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + Timeout :: timeout(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: errcode() | closed | timeout; + + (Socket, BufSz, Flags) -> + {ok, {Source, Data}} | {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Flags :: recv_flags(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: errcode() | closed; + + (Socket, BufSz, nowait) -> + {ok, {Source, Data}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Source :: sockaddr() | undefined, + Data :: binary(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, BufSz, Timeout) -> + {ok, {Source, Data}} | {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Timeout :: timeout(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: errcode() | closed | timeout. + +recvfrom(Socket, Flags, Timeout) when is_list(Flags) -> + recvfrom(Socket, 0, Flags, Timeout); +recvfrom(Socket, BufSz, Flags) when is_list(Flags) -> + recvfrom(Socket, BufSz, Flags, ?ESOCK_RECV_TIMEOUT_DEFAULT); +recvfrom(Socket, BufSz, Timeout) -> + recvfrom(Socket, BufSz, ?ESOCK_RECV_FLAGS_DEFAULT, Timeout). + +-spec recvfrom(Socket, BufSz, Flags, nowait) -> + {ok, {Source, Data}} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Flags :: recv_flags(), + Source :: sockaddr() | undefined, + Data :: binary(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, BufSz, Flags, Timeout) -> + {ok, {Source, Data}} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + Flags :: recv_flags(), + Timeout :: timeout(), + Source :: sockaddr() | undefined, + Data :: binary(), + Reason :: errcode() | closed | timeout. + +recvfrom(?mk_socket(SockRef) = Socket, BufSz, Flags, Timeout) + when is_reference(SockRef), + is_integer(BufSz), 0 =< BufSz, + is_list(Flags) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, BufSz, Flags, Timeout]); + nowait -> + recvfrom_nowait(SockRef, BufSz, Flags); + Deadline -> + recvfrom_deadline(SockRef, BufSz, Flags, Deadline) + end; +recvfrom(Socket, BufSz, Flags, Timeout) -> + erlang:error(badarg, [Socket, BufSz, Flags, Timeout]). + +recvfrom_nowait(SockRef, BufSz, Flags) -> + RecvRef = make_ref(), + case prim_socket:recvfrom(SockRef, RecvRef, BufSz, Flags) of + select -> + ?SELECT(recvfrom, RecvRef); + Result -> + recvfrom_result(Result) + end. + +recvfrom_deadline(SockRef, BufSz, Flags, Deadline) -> + RecvRef = make_ref(), + case prim_socket:recvfrom(SockRef, RecvRef, BufSz, Flags) of + select -> + %% There is nothing just now, but we will be notified when there + %% is something to read (a select message). + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(?mk_socket(SockRef), select, RecvRef) -> + recvfrom_deadline(SockRef, BufSz, Flags, Deadline); + ?mk_socket_msg(_Socket, abort, {RecvRef, Reason}) -> + {error, Reason} + after Timeout -> + cancel(SockRef, recvfrom, RecvRef), + {error, timeout} + end; + {error, ealready = Reason} -> + %% Internal error: + %% we called recvfrom, got eagain, and called recvfrom again + %% - without waiting for select message + erlang:error(Reason); + Result -> + recvfrom_result(Result) + end. + +recvfrom_result(Result) -> + case Result of + {ok, {_Source, _NewData}} = OK -> + OK; + {error, _Reason} = ERROR -> + ERROR + end. + + +%% --------------------------------------------------------------------------- +%% + +-spec recvmsg(Socket) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + Reason :: errcode() | closed. + +recvmsg(Socket) -> + recvmsg(Socket, 0, 0, + ?ESOCK_RECV_FLAGS_DEFAULT, ?ESOCK_RECV_TIMEOUT_DEFAULT). + +-spec recvmsg(Socket, Flags) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + MsgHdr :: msghdr(), + Reason :: errcode() | closed; + + (Socket, Timeout :: nowait) -> {ok, MsgHdr} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, Timeout) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + Timeout :: timeout(), + MsgHdr :: msghdr(), + Reason :: errcode() | closed | timeout. + +recvmsg(Socket, Flags) when is_list(Flags) -> + recvmsg(Socket, 0, 0, Flags, ?ESOCK_RECV_TIMEOUT_DEFAULT); +recvmsg(Socket, Timeout) -> + recvmsg(Socket, 0, 0, ?ESOCK_RECV_FLAGS_DEFAULT, Timeout). + +-spec recvmsg(Socket, Flags, nowait) -> {ok, MsgHdr} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, Flags, Timeout) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + Flags :: recv_flags(), + Timeout :: timeout(), + MsgHdr :: msghdr(), + Reason :: errcode() | closed | timeout; + + (Socket, BufSz, CtrlSz) -> {ok, MsgHdr} | {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + CtrlSz :: non_neg_integer(), + MsgHdr :: msghdr(), + Reason :: errcode() | closed. + +recvmsg(Socket, Flags, Timeout) when is_list(Flags) -> + recvmsg(Socket, 0, 0, Flags, Timeout); +recvmsg(Socket, BufSz, CtrlSz) when is_integer(BufSz), is_integer(CtrlSz) -> + recvmsg(Socket, BufSz, CtrlSz, + ?ESOCK_RECV_FLAGS_DEFAULT, ?ESOCK_RECV_TIMEOUT_DEFAULT). + + +-spec recvmsg(Socket, BufSz, CtrlSz, Flags, nowait) -> + {ok, MsgHdr} | + {select, SelectInfo} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + CtrlSz :: non_neg_integer(), + Flags :: recv_flags(), + MsgHdr :: msghdr(), + SelectInfo :: select_info(), + Reason :: errcode() | closed; + + (Socket, BufSz, CtrlSz, Flags, Timeout) -> + {ok, MsgHdr} | + {error, Reason} when + Socket :: socket(), + BufSz :: non_neg_integer(), + CtrlSz :: non_neg_integer(), + Flags :: recv_flags(), + Timeout :: timeout(), + MsgHdr :: msghdr(), + Reason :: errcode() | closed | timeout. + +recvmsg(?mk_socket(SockRef) = Socket, BufSz, CtrlSz, Flags, Timeout) + when is_reference(SockRef), + is_integer(BufSz), 0 =< BufSz, + is_integer(CtrlSz), 0 =< CtrlSz, + is_list(Flags) -> + case deadline(Timeout) of + badarg = Reason -> + erlang:error(Reason, [Socket, BufSz, CtrlSz, Flags, Timeout]); + nowait -> + recvmsg_nowait(SockRef, BufSz, CtrlSz, Flags); + Deadline -> + recvmsg_deadline(SockRef, BufSz, CtrlSz, Flags, Deadline) + end; +recvmsg(Socket, BufSz, CtrlSz, Flags, Timeout) -> + erlang:error(badarg, [Socket, BufSz, CtrlSz, Flags, Timeout]). + +recvmsg_nowait(SockRef, BufSz, CtrlSz, Flags) -> + RecvRef = make_ref(), + case prim_socket:recvmsg(SockRef, RecvRef, BufSz, CtrlSz, Flags) of + select -> + ?SELECT(recvmsg, RecvRef); + Result -> + recvmsg_result(Result) + end. + +recvmsg_deadline(SockRef, BufSz, CtrlSz, Flags, Deadline) -> + RecvRef = make_ref(), + case prim_socket:recvmsg(SockRef, RecvRef, BufSz, CtrlSz, Flags) of + select -> + %% There is nothing just now, but we will be notified when there + %% is something to read (a select message). + Timeout = timeout(Deadline), + receive + ?mk_socket_msg(?mk_socket(SockRef), select, RecvRef) -> + recvmsg_deadline( + SockRef, BufSz, CtrlSz, Flags, Deadline); + ?mk_socket_msg(_Socket, abort, {RecvRef, Reason}) -> + {error, Reason} + after Timeout -> + cancel(SockRef, recvmsg, RecvRef), + {error, timeout} + end; + %% + {error, ealready = Reason} -> + %% Internal error: + %% we called recvmsg, got eagain, and called recvmsg again + %% - without waiting for select message + erlang:error(Reason); + Result -> + recvmsg_result(Result) + end. + +recvmsg_result(Result) -> + case Result of + {ok, _MsgHdr} = OK -> + OK; + {error, _Reason} = ERROR -> + ERROR + end. + + +%% =========================================================================== +%% +%% close - close a file descriptor +%% +%% Closing a socket is a two stage rocket (because of linger). +%% We need to perform the actual socket close while in BLOCKING mode. +%% But that would hang the entire VM, so what we do is divide the +%% close in two steps: +%% 1) prim_socket:nif_close + the socket_stop (nif) callback function +%% This is for everything that can be done safely NON-BLOCKING. +%% 2) prim_socket:nif_finalize_close which is executed by a *dirty* scheduler +%% Before we call the socket close function, we set the socket +%% BLOCKING. Thereby linger is handled properly. + +-spec close(Socket) -> ok | {error, Reason} when + Socket :: socket(), + Reason :: errcode() | closed | timeout. + +close(?mk_socket(SockRef)) + when is_reference(SockRef) -> + case prim_socket:close(SockRef) of + ok -> + prim_socket:finalize_close(SockRef); + {ok, CloseRef} -> + %% We must wait for the socket_stop callback function to + %% complete its work + receive + ?mk_socket_msg(?mk_socket(SockRef), close, CloseRef) -> + prim_socket:finalize_close(SockRef) + end; + {error, _} = ERROR -> + ERROR + end; +close(Socket) -> + erlang:error(badarg, [Socket]). + + + +%% =========================================================================== +%% +%% shutdown - shut down part of a full-duplex connection +%% + +-spec shutdown(Socket, How) -> ok | {error, Reason} when + Socket :: socket(), + How :: shutdown_how(), + Reason :: inet:posix() | closed. + +shutdown(?mk_socket(SockRef), How) + when is_reference(SockRef) -> + prim_socket:shutdown(SockRef, How); +shutdown(Socket, How) -> + erlang:error(badarg, [Socket, How]). + + +%% =========================================================================== +%% +%% setopt - manipulate individual properties of a socket +%% +%% What properties are valid depend on what kind of socket it is +%% (domain, type and protocol) +%% If its an "invalid" option (or value), we should not crash but return some +%% useful error... +%% +%% <KOLLA> +%% +%% WE NEED TO MAKE SURE THAT THE USER DOES NOT MAKE US BLOCKING +%% AS MUCH OF THE CODE EXPECTS TO BE NON-BLOCKING!! +%% +%% </KOLLA> + +-spec setopt(Socket, otp, otp_socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: errcode() | closed | not_owner; + + (Socket, socket, socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, ip, ip_socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, ipv6, ipv6_socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, tcp, tcp_socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, udp, udp_socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, sctp, sctp_socket_option(), Value) -> + ok | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, Level, Key, Value) -> + ok | {error, Reason} when + Socket :: socket(), + Level :: non_neg_integer(), + Key :: non_neg_integer(), + Value :: binary(), + Reason :: inet:posix() | closed. + +setopt(?mk_socket(SockRef), Level, Key, Value) + when is_reference(SockRef) -> + prim_socket:setopt(SockRef, Level, Key, Value); +setopt(Socket, Level, Key, Value) -> + erlang:error(badarg, [Socket, Level, Key, Value]). + + +%% =========================================================================== +%% +%% getopt - retrieve individual properties of a socket +%% +%% What properties are valid depend on what kind of socket it is +%% (domain, type and protocol). +%% If its an "invalid" option, we should not crash but return some +%% useful error... +%% +%% When specifying level as an integer, and therefor using "native mode", +%% we should make it possible to specify common types instead of the +%% value size. Example: int | bool | {string, pos_integer()} | non_neg_integer() +%% + +-spec getopt(Socket, otp, otp_socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: einval | closed; + + (Socket, socket, socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, ip, ip_socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, ipv6, ipv6_socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, tcp, tcp_socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, udp, udp_socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, sctp, sctp_socket_option()) -> + {ok, Value} | {error, Reason} when + Socket :: socket(), + Value :: term(), + Reason :: inet:posix() | closed; + + (Socket, Level, Key) -> + ok | {ok, Value} | {error, Reason} when + Socket :: socket(), + Level :: integer(), + Key :: {NativeOpt, ValueSize}, + NativeOpt :: integer(), + ValueSize :: int | bool | non_neg_integer(), + Value :: term(), + Reason :: inet:posix() | closed. + +getopt(?mk_socket(SockRef), Level, Key) + when is_reference(SockRef) -> + prim_socket:getopt(SockRef, Level, Key); +getopt(Socket, Level, Key) -> + erlang:error(badarg, [Socket, Level, Key]). + + +%% =========================================================================== +%% +%% sockname - return the current address of the socket. +%% +%% + +-spec sockname(Socket) -> {ok, SockAddr} | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + Reason :: inet:posix() | closed. + +sockname(?mk_socket(SockRef)) + when is_reference(SockRef) -> + prim_socket:sockname(SockRef); +sockname(Socket) -> + erlang:error(badarg, [Socket]). + + +%% =========================================================================== +%% +%% peername - return the address of the peer *connected* to the socket. +%% +%% + +-spec peername(Socket) -> {ok, SockAddr} | {error, Reason} when + Socket :: socket(), + SockAddr :: sockaddr(), + Reason :: inet:posix() | closed. + +peername(?mk_socket(SockRef)) + when is_reference(SockRef) -> + prim_socket:peername(SockRef); +peername(Socket) -> + erlang:error(badarg, [Socket]). + + +%% =========================================================================== +%% +%% cancel - cancel an operation resulting in a select +%% +%% A call to accept, recv/recvfrom/recvmsg and send/sendto/sendmsg +%% can result in a select if they are called with the Timeout argument +%% set to nowait. This is indicated by the return of the select-info. +%% Such a operation can be cancelled by calling this function. +%% + +-spec cancel(Socket, SelectInfo) -> ok | {error, Reason} when + Socket :: socket(), + SelectInfo :: select_info(), + Reason :: einval | closed | exself. + +cancel(?mk_socket(SockRef), ?SELECT_INFO(Tag, Ref)) + when is_reference(SockRef) -> + cancel(SockRef, Tag, Ref); +cancel(Socket, SelectInfo) -> + erlang:error(badarg, [Socket, SelectInfo]). + + +cancel(SockRef, Op, OpRef) -> + case prim_socket:cancel(SockRef, Op, OpRef) of + %% The select has already completed + {error, select_sent} -> + flush_select_msg(SockRef, OpRef), + _ = flush_abort_msg(SockRef, OpRef), + ok; + {error, not_found} -> + _ = flush_abort_msg(SockRef, OpRef), + {error, einval}; + Other -> + _ = flush_abort_msg(SockRef, OpRef), + Other + end. + +flush_select_msg(SockRef, Ref) -> + receive + ?mk_socket_msg(?mk_socket(SockRef), select, Ref) -> + ok + after 0 -> + ok + end. + +flush_abort_msg(SockRef, Ref) -> + receive + ?mk_socket_msg(?mk_socket(SockRef), abort, {Ref, Reason}) -> + Reason + after 0 -> + ok + end. + + +%% =========================================================================== +%% +%% Misc utility functions +%% +%% =========================================================================== + +%% formated_timestamp() -> +%% format_timestamp(os:timestamp()). + +%% format_timestamp(Now) -> +%% N2T = fun(N) -> calendar:now_to_local_time(N) end, +%% format_timestamp(Now, N2T, true). + +%% format_timestamp({_N1, _N2, N3} = N, N2T, true) -> +%% FormatExtra = ".~.2.0w", +%% ArgsExtra = [N3 div 10000], +%% format_timestamp(N, N2T, FormatExtra, ArgsExtra); +%% format_timestamp({_N1, _N2, _N3} = N, N2T, false) -> +%% FormatExtra = "", +%% ArgsExtra = [], +%% format_timestamp(N, N2T, FormatExtra, ArgsExtra). + +%% format_timestamp(N, N2T, FormatExtra, ArgsExtra) -> +%% {Date, Time} = N2T(N), +%% {YYYY,MM,DD} = Date, +%% {Hour,Min,Sec} = Time, +%% FormatDate = +%% io_lib:format("~.4w-~.2.0w-~.2.0w ~.2.0w:~.2.0w:~.2.0w" ++ FormatExtra, +%% [YYYY, MM, DD, Hour, Min, Sec] ++ ArgsExtra), +%% lists:flatten(FormatDate). + + +deadline(Timeout) -> + case Timeout of + nowait -> + Timeout; + infinity -> + Timeout; + 0 -> + zero; + _ when is_integer(Timeout), 0 < Timeout -> + timestamp() + Timeout; + _ -> + badarg + end. + +timeout(Deadline) -> + case Deadline of + infinity -> + Deadline; + zero -> + 0; + _ -> + Now = timestamp(), + if + Deadline > Now -> + Deadline - Now; + true -> + 0 + end + end. + +timestamp() -> + erlang:monotonic_time(milli_seconds). + + +-compile({inline, [bincat/2]}). +bincat(<<>>, <<_/binary>> = B) -> B; +bincat(<<_/binary>> = A, <<>>) -> A; +bincat(<<_/binary>> = A, <<_/binary>> = B) -> + <<A/binary, B/binary>>. + + +%% p(F) -> +%% p(F, []). + +%% p(F, A) -> +%% p(get(sname), F, A). + +%% p(undefined, F, A) -> +%% p("***", F, A); +%% p(SName, F, A) -> +%% TS = formated_timestamp(), +%% io:format(user,"[~s][~s,~p] " ++ F ++ "~n", [TS, SName, self()|A]), +%% io:format("[~s][~s,~p] " ++ F ++ "~n", [TS, SName, self()|A]). diff --git a/lib/kernel/src/user_drv.erl b/lib/kernel/src/user_drv.erl index 3ef6bc5533..f420b4faea 100644 --- a/lib/kernel/src/user_drv.erl +++ b/lib/kernel/src/user_drv.erl @@ -121,6 +121,13 @@ server1(Iport, Oport, Shell) -> case init:get_argument(remsh) of {ok,[[Node]]} -> ANode = list_to_atom(append_hostname(Node)), + %% We try to connect to the node if the current node is not + %% a distributed node yet. If this succeeds it means that we + %% are running using "-sname undefined". + [begin + _ = net_kernel:start([undefined, shortnames]), + net_kernel:connect_node(ANode) + end || node() =:= nonode@nohost], RShell = {ANode,shell,start,[]}, RGr = group:start(self(), RShell, rem_sh_opts(ANode)), {RGr,RShell}; diff --git a/lib/kernel/test/Makefile b/lib/kernel/test/Makefile index dcc892ec50..6a3696f92e 100644 --- a/lib/kernel/test/Makefile +++ b/lib/kernel/test/Makefile @@ -24,6 +24,21 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk # Target Specs # ---------------------------------------------------- +SOCKET_MODULES = \ + socket_test_lib \ + socket_test_logger \ + socket_test_evaluator \ + socket_test_ttest_lib \ + socket_test_ttest_tcp_gen \ + socket_test_ttest_tcp_socket \ + socket_test_ttest_tcp_client \ + socket_test_ttest_tcp_client_gen \ + socket_test_ttest_tcp_client_socket \ + socket_test_ttest_tcp_server \ + socket_test_ttest_tcp_server_gen \ + socket_test_ttest_tcp_server_socket \ + socket_SUITE + MODULES= \ erpc_SUITE \ rpc_SUITE \ @@ -89,6 +104,7 @@ MODULES= \ pg_SUITE \ pg2_SUITE \ seq_trace_SUITE \ + $(SOCKET_MODULES) \ wrap_log_reader_SUITE \ cleanup \ ignore_cores \ @@ -113,6 +129,10 @@ APP_FILES = \ topApp3.app ERL_FILES= $(MODULES:%=%.erl) code_a_test.erl +HRL_FILES= \ + socket_test_evaluator.hrl \ + socket_test_ttest.hrl \ + socket_test_ttest_client.hrl TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) INSTALL_PROGS= $(TARGET_FILES) @@ -135,6 +155,7 @@ ERL_COMPILE_FLAGS += EBIN = . TARGETS = $(MODULES:%=$(EBIN)/%.$(EMULATOR)) +SOCKET_TARGETS = $(SOCKET_MODULES:%=$(EBIN)/%.$(EMULATOR)) # ---------------------------------------------------- @@ -161,6 +182,7 @@ clean: docs: targets: $(TARGETS) +socket_targets: $(SOCKET_TARGETS) # ---------------------------------------------------- @@ -172,7 +194,7 @@ release_spec: opt release_tests_spec: make_emakefile $(INSTALL_DIR) "$(RELSYSDIR)" - $(INSTALL_DATA) $(ERL_FILES) "$(RELSYSDIR)" + $(INSTALL_DATA) $(ERL_FILES) $(HRL_FILES) "$(RELSYSDIR)" $(INSTALL_DATA) $(APP_FILES) "$(RELSYSDIR)" $(INSTALL_DATA) \ kernel.spec kernel_smoke.spec kernel_bench.spec logger.spec \ diff --git a/lib/kernel/test/erl_boot_server_SUITE.erl b/lib/kernel/test/erl_boot_server_SUITE.erl index 1eaa2cf500..491176678a 100644 --- a/lib/kernel/test/erl_boot_server_SUITE.erl +++ b/lib/kernel/test/erl_boot_server_SUITE.erl @@ -238,7 +238,7 @@ responses(Config) when is_list(Config) -> {ok,BootPid} = erl_boot_server:start_link([Host]), %% Send junk - S1 = open_udp(), + S1 = open_udp(Ip), prim_inet:sendto(S1, Ip, EBOOT_PORT, ["0"]), receive What -> @@ -249,10 +249,10 @@ responses(Config) when is_list(Config) -> end, %% Req from a slave with same erlang vsn. - S2 = open_udp(), + S2 = open_udp(Ip), prim_inet:sendto(S2, Ip, EBOOT_PORT, [EBOOT_REQUEST,ThisVer]), receive - {udp,S2,Ip,_Port1,Resp1} -> + {udp,S2,_Ip,_Port1,Resp1} -> close_udp(S2), EBOOT_REPLY = string:substr(Resp1, 1, length(EBOOT_REPLY)), Rest1 = string:substr(Resp1, length(EBOOT_REPLY)+1, length(Resp1)), @@ -263,7 +263,7 @@ responses(Config) when is_list(Config) -> end, %% Req from a slave with other erlang vsn. - S3 = open_udp(), + S3 = open_udp(Ip), prim_inet:sendto(S3, Ip, EBOOT_PORT, [EBOOT_REQUEST,"1.0"]), receive Anything -> @@ -284,7 +284,7 @@ responses(Config) when is_list(Config) -> {ok,BootPid2} = erl_boot_server:start_link(["127.0.0.1"]), %% Req from slave with invalid ip address. - S4 = open_udp(), + S4 = open_udp(Ip), Ret = case Ip of {127,0,0,1} -> @@ -336,11 +336,11 @@ good_hosts(_Config) -> GoodHost3 = "sauron", [GoodHost1, GoodHost2, GoodHost3]. -open_udp() -> +open_udp(Ip) -> {ok, S} = prim_inet:open(udp, inet, dgram), ok = prim_inet:setopts(S, [{mode,list},{active,true}, {deliver,term},{broadcast,true}]), - {ok,_} = prim_inet:bind(S, {0,0,0,0}, 0), + {ok,_} = prim_inet:bind(S, Ip, 0), S. close_udp(S) -> diff --git a/lib/kernel/test/file_name_SUITE.erl b/lib/kernel/test/file_name_SUITE.erl index 91dce39d12..7166064558 100644 --- a/lib/kernel/test/file_name_SUITE.erl +++ b/lib/kernel/test/file_name_SUITE.erl @@ -634,9 +634,9 @@ hopeless_darwin() -> case {os:type(),os:version()} of {{unix,darwin},{Major,_,_}} -> %% icky file names worked between 10 and 17, but started returning - %% EILSEQ in 18. The check against 18 is exact in case newer + %% EILSEQ in 18. The check against 18..19 is exact in case newer %% versions of Darwin support them again. - Major < 9 orelse Major =:= 18; + Major < 9 orelse (Major >= 18 andalso Major =< 19); _ -> false end. diff --git a/lib/kernel/test/gen_sctp_SUITE.erl b/lib/kernel/test/gen_sctp_SUITE.erl index 65183d83cc..55109a5178 100644 --- a/lib/kernel/test/gen_sctp_SUITE.erl +++ b/lib/kernel/test/gen_sctp_SUITE.erl @@ -41,7 +41,8 @@ peeloff_active_once/1, peeloff_active_true/1, peeloff_active_n/1, buffers/1, names_unihoming_ipv4/1, names_unihoming_ipv6/1, - names_multihoming_ipv4/1, names_multihoming_ipv6/1]). + names_multihoming_ipv4/1, names_multihoming_ipv6/1, + recv_close/1]). suite() -> [{ct_hooks,[ts_install_cth]}, @@ -56,10 +57,25 @@ all() -> {group,G}]. groups() -> - [{smoke,[],[basic,basic_stream]}, - {old_solaris,[],[skip_old_solaris]}, - {extensive,[], - [api_open_close, api_listen, api_connect_init, + [ + {smoke, [], smoke_cases()}, + {old_solaris, [], old_solaris_cases()}, + {extensive, [], extensive_cases()} + ]. + +smoke_cases() -> + [ + basic, + basic_stream + ]. + +old_solaris_cases() -> + [ + skip_old_solaris + ]. + +extensive_cases() -> + [api_open_close, api_listen, api_connect_init, api_opts, xfer_min, xfer_active, def_sndrcvinfo, implicit_inet6, open_multihoming_ipv4_socket, open_unihoming_ipv6_socket, @@ -68,7 +84,8 @@ groups() -> xfer_stream_min, peeloff_active_once, peeloff_active_true, peeloff_active_n, buffers, names_unihoming_ipv4, names_unihoming_ipv6, - names_multihoming_ipv4, names_multihoming_ipv6]}]. + names_multihoming_ipv4, names_multihoming_ipv6, + recv_close]. init_per_suite(_Config) -> case gen_sctp:open() of @@ -1511,6 +1528,111 @@ recv_comm_up_eventually(S) -> recv_comm_up_eventually(S) end. + +%% +recv_close(Config) when is_list(Config) -> + p("create server socket (and listen)"), + {ok, S} = gen_sctp:open(), + gen_sctp:listen(S, true), + {ok, SPort} = inet:port(S), + + p("create client socket (and connect)"), + {ok, C} = gen_sctp:open(), + {ok, _} = gen_sctp:connect(C, localhost, SPort, []), + + TC = self(), + RECV = fun() -> + p("try setup recv(s)"), + ok = recv_close_setup_recv(S), + p("announce ready"), + TC ! {self(), ready}, + p("try data recv"), + Res = gen_sctp:recv(S), + p("recv res: " + "~n ~p", [Res]), + exit(Res) + end, + p("spawn reader - then await reader ready"), + {Pid, MRef} = spawn_monitor(RECV), + receive + {'DOWN', MRef, process, Pid, PreReason} -> + %% Make sure it does not die for some other reason... + p("unexpected reader termination:" + "~n ~p", [PreReason]), + (catch gen_sctp:close(S)), + (catch gen_sctp:close(C)), + ?line ct:fail("Unexpected pre close from reader (~p): ~p", + [Pid, PreReason]); + {Pid, ready} -> + p("reader ready"), + ok + after 30000 -> % Just in case... + %% This is **extreme**, but there is no way to know + %% how long it will take to iterate through all the + %% addresses of a host... + p("reader ready timeout"), + (catch gen_sctp:close(S)), + (catch gen_sctp:close(C)), + ?line ct:fail("Unexpected pre close timeout (~p)", [Pid]) + end, + + p("\"ensure\" reader reading..."), + receive + Any -> + p("Received unexpected message: " + "~n ~p", [Any]), + (catch gen_sctp:close(S)), + (catch gen_sctp:close(C)), + ?line ct:fail("Unexpected message: ~p", [Any]) + after 5000 -> + ok + end, + + p("close server socket"), + ok = gen_sctp:close(S), + p("await reader termination"), + receive + {'DOWN', MRef, process, Pid, {error, closed}} -> + p("expected reader termination result"), + (catch gen_sctp:close(C)), + ok; + {'DOWN', MRef, process, Pid, PostReason} -> + p("unexpected reader termination: " + "~n ~p", [PostReason]), + (catch gen_sctp:close(C)), + ?line ct:fail("Unexpected post close from reader (~p): ~p", + [Pid, PostReason]) + after 5000 -> + p("unexpected reader termination timeout"), + demonitor(MRef, [flush]), + (catch gen_sctp:close(C)), + exit(Pid, kill), + ?line ct:fail("Reader (~p) termination timeout", [Pid]) + end, + p("close client socket"), + (catch gen_sctp:close(C)), + p("done"), + ok. + + +recv_close_setup_recv(S) -> + recv_close_setup_recv(S, 1). + +recv_close_setup_recv(S, N) -> + p("try setup recv ~w", [N]), + case gen_sctp:recv(S, 5000) of + {ok, {Addr, + Port, + _AncData, + Data}} when is_tuple(Addr) andalso is_integer(Port) -> + p("setup recv ~w: " + "~n ~p", [N, Data]), + recv_close_setup_recv(S, N+1); + {error, timeout} -> + ok + end. + + %%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%% socket gen_server ultra light @@ -1745,3 +1867,21 @@ match_unless_solaris(A, B) -> timestamp() -> erlang:monotonic_time(). + +formated_timestamp() -> + format_timestamp(os:timestamp()). + +format_timestamp({_N1, _N2, N3} = TS) -> + {_Date, Time} = calendar:now_to_local_time(TS), + {Hour, Min, Sec} = Time, + FormatTS = io_lib:format("~.2.0w:~.2.0w:~.2.0w.~.3.0w", + [Hour, Min, Sec, N3 div 1000]), + lists:flatten(FormatTS). + +p(F) -> + p(F, []). + +p(F, A) -> + io:format("~s ~p " ++ F ++ "~n", [formated_timestamp(), self() | A]). + + diff --git a/lib/kernel/test/gen_tcp_echo_SUITE.erl b/lib/kernel/test/gen_tcp_echo_SUITE.erl index 1cf67a374a..e2b17d374b 100644 --- a/lib/kernel/test/gen_tcp_echo_SUITE.erl +++ b/lib/kernel/test/gen_tcp_echo_SUITE.erl @@ -246,6 +246,16 @@ echo_packet0(Echo, Type, EchoFun, SlowEcho, Opts) -> echo_packet1(Echo, Type, EchoFun, infinite); true -> ok end, + PacketSize =:= 0 andalso + begin + %% Switch to raw mode and echo one byte + ok = inet:setopts(Echo, [{packet, raw}, {active, false}]), + ok = gen_tcp:send(Echo, <<"$">>), + case gen_tcp:recv(Echo, 1) of + {ok, <<"$">>} -> ok; + {ok, "$"} -> ok + end + end, _CloseResult = gen_tcp:close(Echo), ct:log("echo_packet0[~w] close: ~p", [self(), _CloseResult]), ok. diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index 2720d3cc77..7ec553ec51 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -40,25 +40,45 @@ recvtos/1, recvtosttl/1, recvttl/1, recvtclass/1, sendtos/1, sendtosttl/1, sendttl/1, sendtclass/1, local_basic/1, local_unbound/1, - local_fdopen/1, local_fdopen_unbound/1, local_abstract/1]). + local_fdopen/1, local_fdopen_unbound/1, local_abstract/1, + recv_close/1]). suite() -> [{ct_hooks,[ts_install_cth]}, {timetrap,{minutes,1}}]. all() -> - [send_to_closed, buffer_size, binary_passive_recv, max_buffer_size, - bad_address, read_packets, recv_poll_after_active_once, - open_fd, connect, - implicit_inet6, active_n, + [ + send_to_closed, + buffer_size, + binary_passive_recv, + max_buffer_size, + bad_address, + read_packets, + recv_poll_after_active_once, + open_fd, + connect, + implicit_inet6, + active_n, recvtos, recvtosttl, recvttl, recvtclass, sendtos, sendtosttl, sendttl, sendtclass, - {group, local}]. + {group, local}, + recv_close + ]. groups() -> - [{local, [], - [local_basic, local_unbound, - local_fdopen, local_fdopen_unbound, local_abstract]}]. + [ + {local, [], local_cases()} + ]. + +local_cases() -> + [ + local_basic, + local_unbound, + local_fdopen, + local_fdopen_unbound, + local_abstract + ]. init_per_suite(Config) -> Config. @@ -969,6 +989,52 @@ local_handshake(S, SAddr, C, CAddr) -> end. + + +%%------------------------------------------------------------- +%% Open a passive socket. Create a socket that reads from it. +%% Then close the socket. +recv_close(Config) when is_list(Config) -> + {ok, Sock} = gen_udp:open(0, [{active, false}]), + RECV = fun() -> + io:format("~p try recv~n", [self()]), + Res = gen_udp:recv(Sock, 0), + io:format("~p recv res: ~p~n", [self(), Res]), + exit(Res) + end, + io:format("~p spawn reader", [self()]), + {Pid, MRef} = spawn_monitor(RECV), + receive + {'DOWN', MRef, process, Pid, PreReason} -> + %% Make sure id does not die for some other reason... + ?line ct:fail("Unexpected pre close from reader (~p): ~p", + [Pid, PreReason]) + after 5000 -> % Just in case... + ok + end, + io:format("~p close socket", [self()]), + ok = gen_udp:close(Sock), + io:format("~p await reader termination", [self()]), + receive + {'DOWN', MRef, process, Pid, {error, closed}} -> + io:format("~p expected reader termination result", [self()]), + ok; + {'DOWN', MRef, process, Pid, PostReason} -> + io:format("~p unexpected reader termination: ~p", + [self(), PostReason]), + ?line ct:fail("Unexpected post close from reader (~p): ~p", + [Pid, PostReason]) + after 5000 -> + io:format("~p unexpected reader termination timeout", [self()]), + demonitor(MRef, [flush]), + exit(Pid, kill), + ?line ct:fail("Reader (~p) termination timeout", [Pid]) + end, + ok. + + + + %% %% Utils %% diff --git a/lib/kernel/test/global_SUITE.erl b/lib/kernel/test/global_SUITE.erl index 3c4654b44c..22db756d97 100644 --- a/lib/kernel/test/global_SUITE.erl +++ b/lib/kernel/test/global_SUITE.erl @@ -3738,13 +3738,17 @@ start_node(Name, How, Config) -> start_node(Name0, How, Args, Config) -> Name = node_name(Name0, Config), Pa = filename:dirname(code:which(?MODULE)), - R = test_server:start_node(Name, How, [{args, - Args ++ " " ++ - "-kernel net_setuptime 100 " - %% "-noshell " - "-pa " ++ Pa}, - {linked, false} - ]), + R = test_server:start_node( + Name, How, [{args, + Args ++ + " -kernel net_setuptime 100 " ++ + %% Limit the amount of threads so that we + %% don't run into the maximum allowed + " +S 1 +SDio 1 " ++ + %% "-noshell " + "-pa " ++ Pa}, + {linked, false} + ]), %% {linked,false} only seems to work for slave nodes. %% ct:sleep(1000), record_started_node(R). @@ -3761,12 +3765,16 @@ start_node_rel(Name0, Rel, Config) -> end, Env = [], Pa = filename:dirname(code:which(?MODULE)), - Res = test_server:start_node(Name, peer, - [{args, - Compat ++ - " -kernel net_setuptime 100 " - " -pa " ++ Pa}, - {erl, Release}] ++ Env), + Res = test_server:start_node( + Name, peer, + [{args, + Compat ++ + " -kernel net_setuptime 100 " ++ + %% Limit the amount of threads so that we + %% don't run into the maximum allowed + " +S 1 +SDio 1 " ++ + "-pa " ++ Pa}, + {erl, Release}] ++ Env), record_started_node(Res). record_started_node({ok, Node}) -> diff --git a/lib/kernel/test/init_SUITE.erl b/lib/kernel/test/init_SUITE.erl index ceee387289..47e44200bf 100644 --- a/lib/kernel/test/init_SUITE.erl +++ b/lib/kernel/test/init_SUITE.erl @@ -354,11 +354,18 @@ restart_with_mode(Config) when is_list(Config) -> {ok,[[Erl]]} = init:get_argument(progname), ModPath = filename:dirname(code:which(?MODULE)), - Eval1 = "'Mode=code:get_mode(), io:fwrite(Mode), case Mode of interactive -> init:restart([{mode,embedded}]); embedded -> erlang:halt() end'", + Quote = case os:type() of + {win32,_} -> + [$"]; + {unix,_} -> + [$'] + end, + + Eval1 = Quote ++ "Mode=code:get_mode(), io:fwrite(Mode), case Mode of interactive -> init:restart([{mode,embedded}]); embedded -> erlang:halt() end" ++ Quote, Cmd1 = Erl ++ " -mode interactive -noshell -eval " ++ Eval1, "interactiveembedded" = os:cmd(Cmd1), - Eval2 = "'Mode=code:get_mode(), io:fwrite(Mode), case Mode of embedded -> init:restart([{mode,interactive}]); interactive -> erlang:halt() end'", + Eval2 = Quote ++ "Mode=code:get_mode(), io:fwrite(Mode), case Mode of embedded -> init:restart([{mode,interactive}]); interactive -> erlang:halt() end" ++ Quote, Cmd2 = Erl ++ " -mode embedded -noshell -eval " ++ Eval2, "embeddedinteractive" = os:cmd(Cmd2), diff --git a/lib/kernel/test/logger_std_h_SUITE.erl b/lib/kernel/test/logger_std_h_SUITE.erl index d65305384f..8ba2f946e7 100644 --- a/lib/kernel/test/logger_std_h_SUITE.erl +++ b/lib/kernel/test/logger_std_h_SUITE.erl @@ -291,9 +291,16 @@ errors(Config) -> _ -> NoDir = lists:concat(["/",?MODULE,"_dir"]), {error, - {handler_not_added,{open_failed,NoDir,eacces}}} = + {handler_not_added,{open_failed,NoDir,Error}}} = logger:add_handler(myh2,logger_std_h, - #{config=>#{type=>{file,NoDir}}}) + #{config=>#{type=>{file,NoDir}}}), + case Error of + erofs -> + %% Happens on OS X + ok; + eacces -> + ok + end end, {error, @@ -1768,7 +1775,7 @@ rotation_opts_restart_handler(Config) -> HConfig3#{config=>StdHConfig3#{max_no_bytes=>75, max_no_files=>1, compress_on_rotate=>true}}), - timer:sleep(100), + timer:sleep(500), {ok,#file_info{size=0}} = file:read_file_info(Log), {ok,#file_info{size=29}} = file:read_file_info(Log++".0.gz"), [_] = filelib:wildcard(Log++".*"), diff --git a/lib/kernel/test/net_SUITE.erl b/lib/kernel/test/net_SUITE.erl index b3226f7e8c..5e6ee054f7 100644 --- a/lib/kernel/test/net_SUITE.erl +++ b/lib/kernel/test/net_SUITE.erl @@ -132,18 +132,13 @@ api_basic_cases() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init_per_suite(Config) -> - %% We test on the socket module for simplicity - case lists:member(socket, erlang:loaded()) of - true -> - case os:type() of - {win32, _} -> - not_yet_implemented(); - _ -> - %% ?LOGGER:start(), - Config - end; - false -> - {skip, "esock disabled"} + try net:info() of + #{} -> + %% ?LOGGER:start(), + Config + catch + error : notsup -> + {skip, "esock not supported"} end. end_per_suite(_) -> diff --git a/erts/emulator/test/socket_SUITE.erl b/lib/kernel/test/socket_SUITE.erl index 266175d947..7dd9b04662 100644 --- a/erts/emulator/test/socket_SUITE.erl +++ b/lib/kernel/test/socket_SUITE.erl @@ -83,6 +83,7 @@ api_b_open_and_close_tcp6/1, api_b_open_and_close_udpL/1, api_b_open_and_close_tcpL/1, + api_b_open_and_close_seqpL/1, api_b_open_and_close_sctp4/1, api_b_open_and_maybe_close_raw/1, api_b_sendto_and_recvfrom_udp4/1, @@ -91,8 +92,10 @@ api_b_sendmsg_and_recvmsg_udpL/1, api_b_send_and_recv_tcp4/1, api_b_send_and_recv_tcpL/1, + api_b_send_and_recv_seqpL/1, api_b_sendmsg_and_recvmsg_tcp4/1, api_b_sendmsg_and_recvmsg_tcpL/1, + api_b_sendmsg_and_recvmsg_seqpL/1, api_b_sendmsg_and_recvmsg_sctp4/1, %% *** API socket from FD *** @@ -820,6 +823,7 @@ api_basic_cases() -> api_b_open_and_close_tcp6, api_b_open_and_close_udpL, api_b_open_and_close_tcpL, + api_b_open_and_close_seqpL, api_b_open_and_close_sctp4, api_b_open_and_maybe_close_raw, api_b_sendto_and_recvfrom_udp4, @@ -828,8 +832,10 @@ api_basic_cases() -> api_b_sendmsg_and_recvmsg_udpL, api_b_send_and_recv_tcp4, api_b_send_and_recv_tcpL, + api_b_send_and_recv_seqpL, api_b_sendmsg_and_recvmsg_tcp4, api_b_sendmsg_and_recvmsg_tcpL, + api_b_sendmsg_and_recvmsg_seqpL, api_b_sendmsg_and_recvmsg_sctp4 ]. @@ -1905,25 +1911,22 @@ otp16359_cases() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% init_per_suite(Config) -> + ct:timetrap(?MINS(2)), Factor = analyze_and_print_host_info(), - case lists:member(socket, erlang:loaded()) of - true -> - case os:type() of - {win32, _} -> - (catch not_yet_implemented()); - _ -> - case quiet_mode(Config) of - default -> - ?LOGGER:start(), - [{esock_factor, Factor} | Config]; - Quiet -> - ?LOGGER:start(Quiet), - [{esock_factor, Factor}, - {esock_test_quiet, Quiet} | Config] - end - end; - false -> - {skip, "esock disabled"} + try socket:info() of + #{} -> + case quiet_mode(Config) of + default -> + ?LOGGER:start(), + [{esock_factor, Factor} | Config]; + Quiet -> + ?LOGGER:start(Quiet), + [{esock_factor, Factor}, + {esock_test_quiet, Quiet} | Config] + end + catch + error : notsup -> + {skip, "esock not supported"} end. end_per_suite(_) -> @@ -1931,349 +1934,6 @@ end_per_suite(_) -> ok. -%% This function prints various host info, which might be usefull -%% when analyzing the test suite (results). -%% It also returns a "factor" that can be used to when deciding -%% the load for some tst cases (traffic). Such as run time or -%% number of iteraions. This only works for some OSes (linux). -%% Also, mostly it just returns the factor 1. -%% At this time we just look at BogoMIPS! -analyze_and_print_host_info() -> - {OsFam, OsName} = os:type(), - Version = - case os:version() of - {Maj, Min, Rel} -> - ?F("~w.~w.~w", [Maj, Min, Rel]); - VStr -> - VStr - end, - case {OsFam, OsName} of - {unix, linux} -> - analyze_and_print_linux_host_info(Version); - {unix, openbsd} -> - analyze_and_print_openbsd_host_info(Version); - {unix, freebsd} -> - analyze_and_print_freebsd_host_info(Version); - {unix, sunos} -> - io:format("Solaris: ~s" - "~n", [Version]), - 1; - _ -> - io:format("OS Family: ~p" - "~n OS Type: ~p" - "~n Version: ~p" - "~n", [OsFam, OsName, Version]), - 1 - end. - -analyze_and_print_linux_host_info(Version) -> - case file:read_file_info("/etc/issue") of - {ok, _} -> - io:format("Linux: ~s" - "~n ~s" - "~n", - [Version, string:trim(os:cmd("cat /etc/issue"))]); - _ -> - io:format("Linux: ~s" - "~n", [Version]) - end, - Factor = - case (catch linux_which_cpuinfo()) of - {ok, {CPU, BogoMIPS}} -> - io:format("CPU: " - "~n Model: ~s" - "~n BogoMIPS: ~s" - "~n", [CPU, BogoMIPS]), - %% We first assume its a float, and if not try integer - try list_to_float(string:trim(BogoMIPS)) of - F when F > 1000 -> - 1; - F when F > 1000 -> - 2; - _ -> - 3 - catch - _:_:_ -> - %% - try list_to_integer(string:trim(BogoMIPS)) of - I when I > 1000 -> - 1; - I when I > 1000 -> - 2; - _ -> - 3 - catch - _:_:_ -> - 1 - end - end; - _ -> - 1 - end, - %% Check if we need to adjust the factor because of the memory - try linux_which_meminfo() of - AddFactor -> - Factor + AddFactor - catch - _:_:_ -> - Factor - end. - -linux_which_cpuinfo() -> - %% Check for x86 (Intel or AMD) - CPU = - try [string:trim(S) || S <- string:tokens(os:cmd("grep \"model name\" /proc/cpuinfo"), [$:,$\n])] of - ["model name", ModelName | _] -> - ModelName; - _ -> - %% ARM (at least some distros...) - try [string:trim(S) || S <- string:tokens(os:cmd("grep \"Processor\" /proc/cpuinfo"), [$:,$\n])] of - ["Processor", Proc | _] -> - Proc; - _ -> - %% Ok, we give up - throw(noinfo) - catch - _:_:_ -> - throw(noinfo) - end - catch - _:_:_ -> - throw(noinfo) - end, - try [string:trim(S) || S <- string:tokens(os:cmd("grep -i \"bogomips\" /proc/cpuinfo"), [$:,$\n])] of - [_, BMips | _] -> - {ok, {CPU, BMips}}; - _ -> - {ok, CPU} - catch - _:_:_ -> - {ok, CPU} - end. - -%% We *add* the value this return to the Factor. -linux_which_meminfo() -> - try [string:trim(S) || S <- string:tokens(os:cmd("grep MemTotal /proc/meminfo"), [$:])] of - [_, MemTotal] -> - io:format("Memory:" - "~n ~s" - "~n", [MemTotal]), - case string:tokens(MemTotal, [$ ]) of - [MemSzStr, MemUnit] -> - MemSz2 = list_to_integer(MemSzStr), - MemSz3 = - case string:to_lower(MemUnit) of - "kb" -> - MemSz2; - "mb" -> - MemSz2*1024; - "gb" -> - MemSz2*1024*1024; - _ -> - throw(noinfo) - end, - if - (MemSz3 >= 8388608) -> - 0; - (MemSz3 >= 4194304) -> - 1; - (MemSz3 >= 2097152) -> - 2; - true -> - 3 - end; - _X -> - 0 - end; - _ -> - 0 - catch - _:_:_ -> - 0 - end. - - -%% Just to be clear: This is ***not*** scientific... -analyze_and_print_openbsd_host_info(Version) -> - io:format("OpenBSD:" - "~n Version: ~p" - "~n", [Version]), - Extract = - fun(Key) -> - string:tokens(string:trim(os:cmd("sysctl " ++ Key)), [$=]) - end, - try - begin - CPU = - case Extract("hw.model") of - ["hw.model", Model] -> - string:trim(Model); - _ -> - "-" - end, - CPUSpeed = - case Extract("hw.cpuspeed") of - ["hw.cpuspeed", Speed] -> - list_to_integer(Speed); - _ -> - -1 - end, - NCPU = - case Extract("hw.ncpufound") of - ["hw.ncpufound", N] -> - list_to_integer(N); - _ -> - -1 - end, - Memory = - case Extract("hw.physmem") of - ["hw.physmem", PhysMem] -> - list_to_integer(PhysMem) div 1024; - _ -> - -1 - end, - io:format("CPU:" - "~n Model: ~s" - "~n Speed: ~w" - "~n N: ~w" - "~nMemory:" - "~n ~w KB" - "~n", [CPU, CPUSpeed, NCPU, Memory]), - CPUFactor = - if - (CPUSpeed =:= -1) -> - 1; - (CPUSpeed >= 2000) -> - if - (NCPU >= 4) -> - 1; - (NCPU >= 2) -> - 2; - true -> - 3 - end; - true -> - if - (NCPU >= 4) -> - 2; - (NCPU >= 2) -> - 3; - true -> - 4 - end - end, - MemAddFactor = - if - (Memory =:= -1) -> - 0; - (Memory >= 8388608) -> - 0; - (Memory >= 4194304) -> - 1; - (Memory >= 2097152) -> - 2; - true -> - 3 - end, - CPUFactor + MemAddFactor - end - catch - _:_:_ -> - 1 - end. - - -analyze_and_print_freebsd_host_info(Version) -> - io:format("FreeBSD:" - "~n Version: ~p" - "~n", [Version]), - Extract = - fun(Key) -> - string:tokens(string:trim(os:cmd("sysctl " ++ Key)), [$:]) - end, - try - begin - CPU = - case Extract("hw.model") of - ["hw.model", Model] -> - string:trim(Model); - _ -> - "-" - end, - CPUSpeed = - case Extract("hw.clockrate") of - ["hw.clockrate", Speed] -> - list_to_integer(string:trim(Speed)); - _ -> - -1 - end, - NCPU = - case Extract("hw.ncpu") of - ["hw.ncpu", N] -> - list_to_integer(string:trim(N)); - _ -> - -1 - end, - Memory = - case Extract("hw.physmem") of - ["hw.physmem", PhysMem] -> - list_to_integer(string:trim(PhysMem)) div 1024; - _ -> - -1 - end, - io:format("CPU:" - "~n Model: ~s" - "~n Speed: ~w" - "~n N: ~w" - "~nMemory:" - "~n ~w KB" - "~n", [CPU, CPUSpeed, NCPU, Memory]), - CPUFactor = - if - (CPUSpeed =:= -1) -> - 1; - (CPUSpeed >= 2000) -> - if - (NCPU >= 4) -> - 1; - (NCPU >= 2) -> - 2; - true -> - 3 - end; - true -> - if - (NCPU >= 4) -> - 2; - (NCPU >= 2) -> - 3; - true -> - 4 - end - end, - MemAddFactor = - if - (Memory =:= -1) -> - 0; - (Memory >= 8388608) -> - 0; - (Memory >= 4194304) -> - 1; - (Memory >= 2097152) -> - 2; - true -> - 3 - end, - CPUFactor + MemAddFactor - end - catch - _:_:_ -> - 1 - end. - - - - init_per_group(GroupName, Config) when (GroupName =:= sc_remote_close) orelse (GroupName =:= sc_remote_shutdown) orelse @@ -2726,6 +2386,26 @@ api_b_open_and_close_tcpL(_Config) when is_list(_Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% Basically open (create) and close an Unix Domain dgram (UDP) socket. +%% With some extra checks... +api_b_open_and_close_seqpL(suite) -> + []; +api_b_open_and_close_seqpL(doc) -> + []; +api_b_open_and_close_seqpL(_Config) when is_list(_Config) -> + ?TT(?SECS(5)), + tc_try(?FUNCTION_NAME, + fun() -> has_support_unix_domain_socket() end, + fun() -> + InitState = #{domain => local, + type => seqpacket, + protocol => default}, + ok = api_b_open_and_close(InitState) + end). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + %% Basically open (create) and close an IPv4 SCTP (seqpacket) socket. %% With some extra checks... api_b_open_and_close_sctp4(suite) -> @@ -2760,8 +2440,12 @@ api_b_open_and_close(InitState) -> cmd => fun({S, {ok, Sock}}) -> NewS = S#{socket => Sock}, {ok, NewS}; + ({_, {error, epfnosupport = Reason}}) -> + {skip, Reason}; ({_, {error, eprotonosupport = Reason}}) -> {skip, Reason}; + ({_, {error, esocktnosupport = Reason}}) -> + {skip, Reason}; ({_, {error, _} = ERROR}) -> ERROR end}, @@ -2796,7 +2480,7 @@ api_b_open_and_close(InitState) -> end}, #{desc => "get protocol", cmd => fun(#{socket := Sock} = State) -> - case socket:supports(options, socket, protocol) of + case socket:is_supported(options, socket, protocol) of true -> Res = socket:getopt(Sock, socket, protocol), {ok, {State, Res}}; @@ -3236,10 +2920,11 @@ api_b_send_and_recv_tcp4(_Config) when is_list(_Config) -> socket:recv(Sock) end, InitState = #{domain => inet, + type => stream, proto => tcp, send => Send, recv => Recv}, - ok = api_b_send_and_recv_tcp(InitState) + ok = api_b_send_and_recv_conn(InitState) end). @@ -3263,10 +2948,39 @@ api_b_send_and_recv_tcpL(_Config) when is_list(_Config) -> socket:recv(Sock) end, InitState = #{domain => local, + type => stream, + proto => default, + send => Send, + recv => Recv}, + ok = api_b_send_and_recv_conn(InitState) + end). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +%% Basically send and receive using the "common" functions (send and recv) +%% on an Unix Domain seqpacket socket. +api_b_send_and_recv_seqpL(suite) -> + []; +api_b_send_and_recv_seqpL(doc) -> + []; +api_b_send_and_recv_seqpL(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(?FUNCTION_NAME, + fun() -> has_support_unix_domain_socket() end, + fun() -> + Send = fun(Sock, Data) -> + socket:send(Sock, Data) + end, + Recv = fun(Sock) -> + socket:recv(Sock) + end, + InitState = #{domain => local, + type => seqpacket, proto => default, send => Send, recv => Recv}, - ok = api_b_send_and_recv_tcp(InitState) + ok = api_b_send_and_recv_conn(InitState) end). @@ -3296,10 +3010,11 @@ api_b_sendmsg_and_recvmsg_tcp4(_Config) when is_list(_Config) -> end end, InitState = #{domain => inet, + type => stream, proto => tcp, send => Send, recv => Recv}, - ok = api_b_send_and_recv_tcp(InitState) + ok = api_b_send_and_recv_conn(InitState) end). @@ -3345,16 +3060,68 @@ api_b_sendmsg_and_recvmsg_tcpL(_Config) when is_list(_Config) -> end end, InitState = #{domain => local, + type => stream, proto => default, send => Send, recv => Recv}, - ok = api_b_send_and_recv_tcp(InitState) + ok = api_b_send_and_recv_conn(InitState) end). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -api_b_send_and_recv_tcp(InitState) -> +%% Basically send and receive using the msg functions (sendmsg and recvmsg) +%% on an Unix Domain (stream) socket (TCP). +api_b_sendmsg_and_recvmsg_seqpL(suite) -> + []; +api_b_sendmsg_and_recvmsg_seqpL(doc) -> + []; +api_b_sendmsg_and_recvmsg_seqpL(_Config) when is_list(_Config) -> + ?TT(?SECS(10)), + tc_try(?FUNCTION_NAME, + fun() -> has_support_unix_domain_socket() end, + fun() -> + Send = + fun(Sock, Data) -> + MsgHdr = + #{iov => [Data]}, + socket:sendmsg(Sock, MsgHdr) + end, + Recv = fun(Sock) -> + case socket:recvmsg(Sock) of + %% On some platforms, the address + %% is *not* provided (e.g. FreeBSD) + {ok, + #{addr := undefined, + iov := [Data]}} -> + {ok, Data}; + %% On some platforms, the address + %% *is* provided (e.g. linux) + {ok, + #{addr := #{family := local}, + iov := [Data]}} -> + socket:setopt( + Sock, otp, debug, false), + {ok, Data}; + {error, _} = ERROR -> + socket:setopt( + Sock, otp, debug, false), + ERROR + end + end, + InitState = + #{domain => local, + type => seqpacket, + proto => default, + send => Send, + recv => Recv}, + ok = api_b_send_and_recv_conn(InitState) + end). + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +api_b_send_and_recv_conn(InitState) -> process_flag(trap_exit, true), ServerSeq = [ @@ -3378,10 +3145,13 @@ api_b_send_and_recv_tcp(InitState) -> end}, #{desc => "create listen socket", cmd => fun(#{domain := Domain, + type := Type, proto := Proto} = State) -> - case socket:open(Domain, stream, Proto) of + case socket:open(Domain, Type, Proto) of {ok, Sock} -> {ok, State#{lsock => Sock}}; + {error, eprotonosupport = Reason} -> + {skip, Reason}; {error, _} = ERROR -> ERROR end @@ -3539,8 +3309,9 @@ api_b_send_and_recv_tcp(InitState) -> end}, #{desc => "create socket", cmd => fun(#{domain := Domain, + type := Type, proto := Proto} = State) -> - case socket:open(Domain, stream, Proto) of + case socket:open(Domain, Type, Proto) of {ok, Sock} -> {ok, State#{sock => Sock}}; {error, _} = ERROR -> @@ -18926,7 +18697,7 @@ api_opt_ip_recvopts_udp(InitState) -> #{desc => "test for ip:recvtos", cmd => fun(State) -> ?SEV_IPRINT("test for ip:recvtos"), - case socket:supports(options, ip, recvtos) of + case socket:is_supported(options, ip, recvtos) of true -> ?SEV_IPRINT("use ip:recvtos"), {ok, State#{recvtos => true}}; @@ -18938,7 +18709,7 @@ api_opt_ip_recvopts_udp(InitState) -> #{desc => "test for socket:timestamp", cmd => fun(State) -> ?SEV_IPRINT("test for socket:timestamp"), - case socket:supports(options, socket, timestamp) of + case socket:is_supported(options, socket, timestamp) of true -> ?SEV_IPRINT("use socket:timestamp"), {ok, State#{timestamp => true}}; @@ -20884,25 +20655,25 @@ api_opt_ip_mopts_udp4(_Config) when is_list(_Config) -> %% 'control message header type', %% default | value()}] Opts = - case socket:supports(options, socket, timestamp) of + case socket:is_supported(options, socket, timestamp) of true -> [{socket, timestamp, timestamp, default}]; false -> [] end ++ - case socket:supports(options, ip, pktinfo) of + case socket:is_supported(options, ip, pktinfo) of true -> [{ip, pktinfo, pktinfo, default}]; false -> [] end ++ - case socket:supports(options, ip, recvorigdstaddr) of + case socket:is_supported(options, ip, recvorigdstaddr) of true -> [{ip, recvorigdstaddr, origdstaddr, default}]; false -> [] end ++ - case socket:supports(options, ip, recvtos) of + case socket:is_supported(options, ip, recvtos) of true -> %% It seems that sending any of the %% TOS or TTL values will fail on: @@ -20939,7 +20710,7 @@ api_opt_ip_mopts_udp4(_Config) when is_list(_Config) -> {unix, darwin} -> []; _ -> - case socket:supports(options, ip, recvttl) of + case socket:is_supported(options, ip, recvttl) of true -> %% It seems that sending any of the %% TOS or TTL values will fail on: @@ -21735,7 +21506,7 @@ api_opt_ipv6_hoplimit_udp6(_Config) when is_list(_Config) -> end, fun() -> %% Begin by choosing which of the options we shall use - Opt = case socket:supports(options, ipv6, recvhoplimit) of + Opt = case socket:is_supported(options, ipv6, recvhoplimit) of true -> recvhoplimit; false -> hoplimit end, @@ -22024,7 +21795,7 @@ api_opt_ipv6_tclass_udp6(_Config) when is_list(_Config) -> end, fun() -> %% Begin by choosing which of the options we shall use - Opt = case socket:supports(options, ipv6, recvtclass) of + Opt = case socket:is_supported(options, ipv6, recvtclass) of true -> recvtclass; false -> tclass end, @@ -22402,40 +22173,40 @@ api_opt_ipv6_mopts_udp6(_Config) when is_list(_Config) -> %% control message header type(s): %% [{'ipv6 socket option', 'control message header type'}] Opts = - case socket:supports(options, socket, timestamp) of + case socket:is_supported(options, socket, timestamp) of true -> [{socket, timestamp, timestamp, default}]; false -> [] end ++ - case socket:supports(options, ipv6, recvpktinfo) of + case socket:is_supported(options, ipv6, recvpktinfo) of true -> [{ipv6, recvpktinfo, pktinfo, default}]; false -> [] end ++ - case socket:supports(options, ipv6, flowinfo) of + case socket:is_supported(options, ipv6, flowinfo) of true -> [{ipv6, flowinfo, flowinfo, default}]; false -> [] end ++ - case socket:supports(options, ipv6, recvhoplimit) of + case socket:is_supported(options, ipv6, recvhoplimit) of true -> [{ipv6, recvhoplimit, hoplimit, default}]; false -> - case socket:supports(options, ipv6, hoplimit) of + case socket:is_supported(options, ipv6, hoplimit) of true -> [{ipv6, hoplimit, hoplimit, default}]; false -> [] end end ++ - case socket:supports(options, ipv6, recvtclass) of + case socket:is_supported(options, ipv6, recvtclass) of true -> [{ipv6, recvtclass, tclass, 42}]; false -> - case socket:supports(options, ipv6, tclass) of + case socket:is_supported(options, ipv6, tclass) of true -> [{ipv6, tclass, tclass, 42}]; false -> @@ -24534,7 +24305,7 @@ api_to_recv_tcp6(_Config) when is_list(_Config) -> tc_try(api_to_recv_tcp6, fun() -> has_support_ipv6() end, fun() -> - case socket:supports(ipv6) of + case socket:is_supported(ipv6) of true -> Recv = fun(Sock, To) -> socket:recv(Sock, 0, To) @@ -30496,18 +30267,18 @@ format_counters(Type, Counters) when (Type =:= listen) orelse (Type =:= traffic) format_counters(" ", Type, Counters). format_counters(Prefix, traffic, Counters) -> - ReadByte = proplists:get_value(read_byte, Counters, -1), - ReadFails = proplists:get_value(read_fails, Counters, -1), - ReadPkg = proplists:get_value(read_pkg, Counters, -1), - ReadPkgMax = proplists:get_value(read_pkg_max, Counters, -1), - ReadTries = proplists:get_value(read_tries, Counters, -1), - ReadWaits = proplists:get_value(read_waits, Counters, -1), - WriteByte = proplists:get_value(write_byte, Counters, -1), - WriteFails = proplists:get_value(write_fails, Counters, -1), - WritePkg = proplists:get_value(write_pkg, Counters, -1), - WritePkgMax = proplists:get_value(write_pkg_max, Counters, -1), - WriteTries = proplists:get_value(write_tries, Counters, -1), - WriteWaits = proplists:get_value(write_waits, Counters, -1), + ReadByte = maps:get(read_byte, Counters, -1), + ReadFails = maps:get(read_fails, Counters, -1), + ReadPkg = maps:get(read_pkg, Counters, -1), + ReadPkgMax = maps:get(read_pkg_max, Counters, -1), + ReadTries = maps:get(read_tries, Counters, -1), + ReadWaits = maps:get(read_waits, Counters, -1), + WriteByte = maps:get(write_byte, Counters, -1), + WriteFails = maps:get(write_fails, Counters, -1), + WritePkg = maps:get(write_pkg, Counters, -1), + WritePkgMax = maps:get(write_pkg_max, Counters, -1), + WriteTries = maps:get(write_tries, Counters, -1), + WriteWaits = maps:get(write_waits, Counters, -1), f("~n~sNumber Of Read Bytes: ~p" "~n~sNumber Of Read Fails: ~p" "~n~sNumber Of Read Packages: ~p" @@ -30534,10 +30305,10 @@ format_counters(Prefix, traffic, Counters) -> Prefix, WritePkgMax]); format_counters(Prefix, listen, Counters) -> - AccSuccess = proplists:get_value(acc_success, Counters, -1), - AccFails = proplists:get_value(acc_fails, Counters, -1), - AccTries = proplists:get_value(acc_tries, Counters, -1), - AccWaits = proplists:get_value(acc_waits, Counters, -1), + AccSuccess = maps:get(acc_success, Counters, -1), + AccFails = maps:get(acc_fails, Counters, -1), + AccTries = maps:get(acc_tries, Counters, -1), + AccWaits = maps:get(acc_waits, Counters, -1), f("~n~sNumber Of Successful Accepts: ~p" "~n~sNumber Of Failed Accepts: ~p" "~n~sNumber Of Accept Attempts: ~p" @@ -30592,10 +30363,17 @@ ensure_counters([{Cnt, Val}|DefCounters], Counters, Acc) -> end. traffic_sar_counters_validation(Counters) -> - traffic_sar_counters_validation2(Counters, zero_counters()). + %% ?SEV_IPRINT("traffic_sar_counters_validation -> entry with" + %% "~n Counters: ~p", [Counters]), + traffic_sar_counters_validation2(maps:to_list(Counters), + zero_counters()). traffic_sar_counters_validation(Counters, ValidateCounters) -> - traffic_sar_counters_validation2(Counters, ensure_counters(ValidateCounters)). + %% ?SEV_IPRINT("traffic_sar_counters_validation -> entry with" + %% "~n Counters: ~p" + %% "~n Validate Counters: ~p", [Counters, ValidateCounters]), + traffic_sar_counters_validation2(maps:to_list(Counters), + ensure_counters(ValidateCounters)). traffic_sar_counters_validation2(Counters, []) -> %% ?SEV_IPRINT("traffic_sar_counters_validation2 -> Remaining Counters: " @@ -30622,7 +30400,8 @@ traffic_sar_counters_validation2(Counters, [{Cnt, Val}|ValidateCounters]) -> Counters2 = lists:keydelete(Cnt, 1, Counters), traffic_sar_counters_validation2(Counters2, ValidateCounters); {value, {Cnt, InvVal}} -> - ?SEV_EPRINT("traffic_sar_counters_validation2 -> ~w validation failed: " + ?SEV_EPRINT("traffic_sar_counters_validation2 -> " + "~w validation failed: " "~n Expected Value: ~p" "~n Actual Value: ~p", [Cnt, Val, InvVal]), {error, {invalid_counter, Cnt, InvVal, Val}}; @@ -43074,8 +42853,8 @@ has_support_ip_recvtos() -> has_support_socket_option_ip(recvtos). has_support_ip_recvtos_and_or_sock_timestamp() -> - case (socket:supports(options, ip, recvtos) orelse - socket:supports(options, socket, timestamp)) of + case (socket:is_supported(options, ip, recvtos) orelse + socket:is_supported(options, socket, timestamp)) of true -> ok; false -> @@ -43099,8 +42878,8 @@ has_support_ipv6_flowinfo() -> has_support_socket_option_ipv6(flowinfo). has_support_ipv6_hoplimit_or_recvhoplimit() -> - %% case (socket:supports(options, ipv6, recvhoplimit) orelse - %% socket:supports(options, ipv6, hoplimit)) of + %% case (socket:is_supported(options, ipv6, recvhoplimit) orelse + %% socket:is_supported(options, ipv6, hoplimit)) of case is_any_options_supported([{ipv6, recvhoplimit}, {ipv6, hoplimit}]) of true -> ok; @@ -43165,7 +42944,7 @@ has_support_socket_option_udp(Opt) -> has_support_socket_option(Level, Option) -> - case socket:supports(options, Level, Option) of + case socket:is_supported(options, Level, Option) of true -> ok; false -> @@ -43173,7 +42952,7 @@ has_support_socket_option(Level, Option) -> end. is_any_options_supported(Options) -> - Pred = fun({Level, Option}) -> socket:supports(options, Level, Option) end, + Pred = fun({Level, Option}) -> socket:is_supported(options, Level, Option) end, lists:any(Pred, Options). @@ -43195,7 +42974,7 @@ has_support_recv_flag(Flag) -> has_support_send_or_recv_flag("Recv", recv_flags, Flag). has_support_send_or_recv_flag(Pre, Key, Flag) -> - case socket:supports(Key, Flag) of + case socket:is_supported(Key, Flag) of true -> ok; false -> @@ -43260,7 +43039,7 @@ has_support_unix_domain_socket() -> {win32, _} -> skip("Not supported"); _ -> - case socket:supports(local) of + case socket:is_supported(local) of true -> ok; false -> @@ -43273,7 +43052,7 @@ has_support_sctp() -> {win32, _} -> skip("Not supported"); _ -> - case socket:supports(sctp) of + case socket:is_supported(sctp) of true -> ok; false -> @@ -43482,6 +43261,1020 @@ tc_which_name() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% This function prints various host info, which might be usefull +%% when analyzing the test suite (results). +%% It also returns a "factor" that can be used when deciding +%% the load for some test cases (traffic). Such as run time or +%% number of iterations. This only works for some OSes (linux). +%% Also, mostly it just returns the factor 1. +%% At this time we just look at BogoMIPS! +analyze_and_print_host_info() -> + {OsFam, OsName} = os:type(), + Version = + case os:version() of + {Maj, Min, Rel} -> + f("~w.~w.~w", [Maj, Min, Rel]); + VStr -> + VStr + end, + case {OsFam, OsName} of + {unix, linux} -> + analyze_and_print_linux_host_info(Version); + {unix, openbsd} -> + analyze_and_print_openbsd_host_info(Version); + {unix, freebsd} -> + analyze_and_print_freebsd_host_info(Version); + {unix, netbsd} -> + analyze_and_print_netbsd_host_info(Version); + {unix, sunos} -> + analyze_and_print_solaris_host_info(Version); + {win32, nt} -> + analyze_and_print_win_host_info(Version); + _ -> + io:format("OS Family: ~p" + "~n OS Type: ~p" + "~n Version: ~p" + "~n Num Schedulers: ~s" + "~n", [OsFam, OsName, Version, str_num_schedulers()]), + num_schedulers_to_factor() + end. + +str_num_schedulers() -> + try erlang:system_info(schedulers) of + N -> f("~w", [N]) + catch + _:_:_ -> "-" + end. + +num_schedulers_to_factor() -> + try erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + N when (N =< 6) -> + 2; + _ -> + 1 + catch + _:_:_ -> + 10 + end. + + + +%% --- Linux --- + +linux_which_distro(Version) -> + case file:read_file_info("/etc/issue") of + {ok, _} -> + case [string:trim(S) || + S <- string:tokens(os:cmd("cat /etc/issue"), [$\n])] of + [DistroStr|_] -> + io:format("Linux: ~s" + "~n ~s" + "~n", + [Version, DistroStr]), + case DistroStr of + "Wind River Linux" ++ _ -> + wind_river; + "MontaVista" ++ _ -> + montavista; + "Yellow Dog" ++ _ -> + yellow_dog; + _ -> + other + end; + X -> + io:format("Linux: ~s" + "~n ~p" + "~n", + [Version, X]), + other + end; + _ -> + io:format("Linux: ~s" + "~n", [Version]), + other + end. + + +analyze_and_print_linux_host_info(Version) -> + Distro = linux_which_distro(Version), + Factor = + case (catch linux_which_cpuinfo(Distro)) of + {ok, {CPU, BogoMIPS}} -> + io:format("CPU: " + "~n Model: ~s" + "~n BogoMIPS: ~w" + "~n Num Schedulers: ~s" + "~n", [CPU, BogoMIPS, str_num_schedulers()]), + if + (BogoMIPS > 20000) -> + 1; + (BogoMIPS > 10000) -> + 2; + (BogoMIPS > 5000) -> + 3; + (BogoMIPS > 2000) -> + 5; + (BogoMIPS > 1000) -> + 8; + true -> + 10 + end; + {ok, CPU} -> + io:format("CPU: " + "~n Model: ~s" + "~n Num Schedulers: ~s" + "~n", [CPU, str_num_schedulers()]), + num_schedulers_to_factor(); + _ -> + 5 + end, + %% Check if we need to adjust the factor because of the memory + try linux_which_meminfo() of + AddFactor -> + Factor + AddFactor + catch + _:_:_ -> + Factor + end. + + +linux_cpuinfo_lookup(Key) when is_list(Key) -> + linux_info_lookup(Key, "/proc/cpuinfo"). + +linux_cpuinfo_cpu() -> + case linux_cpuinfo_lookup("cpu") of + [Model] -> + Model; + _ -> + "-" + end. + +linux_cpuinfo_motherboard() -> + case linux_cpuinfo_lookup("motherboard") of + [MB] -> + MB; + _ -> + "-" + end. + +linux_cpuinfo_bogomips() -> + case linux_cpuinfo_lookup("bogomips") of + BMips when is_list(BMips) -> + try lists:sum([bogomips_to_int(BM) || BM <- BMips]) + catch + _:_:_ -> + "-" + end; + _ -> + "-" + end. + +linux_cpuinfo_total_bogomips() -> + case linux_cpuinfo_lookup("total bogomips") of + [TBM] -> + try bogomips_to_int(TBM) + catch + _:_:_ -> + "-" + end; + _ -> + "-" + end. + +bogomips_to_int(BM) -> + try list_to_float(BM) of + F -> + floor(F) + catch + _:_:_ -> + try list_to_integer(BM) of + I -> + I + catch + _:_:_ -> + throw(noinfo) + end + end. + +linux_cpuinfo_model() -> + case linux_cpuinfo_lookup("model") of + [M] -> + M; + _ -> + "-" + end. + +linux_cpuinfo_platform() -> + case linux_cpuinfo_lookup("platform") of + [P] -> + P; + _ -> + "-" + end. + +linux_cpuinfo_model_name() -> + case linux_cpuinfo_lookup("model name") of + [P|_] -> + P; + _X -> + "-" + end. + +linux_cpuinfo_processor() -> + case linux_cpuinfo_lookup("Processor") of + [P] -> + P; + _ -> + "-" + end. + +linux_which_cpuinfo(montavista) -> + CPU = + case linux_cpuinfo_cpu() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_motherboard() of + "-" -> + Model; + MB -> + Model ++ " (" ++ MB ++ ")" + end + end, + case linux_cpuinfo_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end; + +linux_which_cpuinfo(yellow_dog) -> + CPU = + case linux_cpuinfo_cpu() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_motherboard() of + "-" -> + Model; + MB -> + Model ++ " (" ++ MB ++ ")" + end + end, + {ok, CPU}; + +linux_which_cpuinfo(wind_river) -> + CPU = + case linux_cpuinfo_model() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_platform() of + "-" -> + Model; + Platform -> + Model ++ " (" ++ Platform ++ ")" + end + end, + case linux_cpuinfo_total_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end; + +linux_which_cpuinfo(other) -> + %% Check for x86 (Intel or AMD) + CPU = + case linux_cpuinfo_model_name() of + "-" -> + %% ARM (at least some distros...) + case linux_cpuinfo_processor() of + "-" -> + %% Ok, we give up + throw(noinfo); + Proc -> + Proc + end; + ModelName -> + ModelName + end, + case linux_cpuinfo_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end. + +linux_meminfo_lookup(Key) when is_list(Key) -> + linux_info_lookup(Key, "/proc/meminfo"). + +linux_meminfo_memtotal() -> + case linux_meminfo_lookup("MemTotal") of + [X] -> + X; + _ -> + "-" + end. + +%% We *add* the value this return to the Factor. +linux_which_meminfo() -> + case linux_meminfo_memtotal() of + "-" -> + 0; + MemTotal -> + io:format("Memory:" + "~n ~s" + "~n", [MemTotal]), + case string:tokens(MemTotal, [$ ]) of + [MemSzStr, MemUnit] -> + MemSz2 = list_to_integer(MemSzStr), + MemSz3 = + case string:to_lower(MemUnit) of + "kb" -> + MemSz2; + "mb" -> + MemSz2*1024; + "gb" -> + MemSz2*1024*1024; + _ -> + throw(noinfo) + end, + if + (MemSz3 >= 8388608) -> + 0; + (MemSz3 >= 4194304) -> + 1; + (MemSz3 >= 2097152) -> + 3; + true -> + 5 + end; + _X -> + 0 + end + end. + + +linux_info_lookup(Key, File) -> + try [string:trim(S) || S <- string:tokens(os:cmd("grep " ++ "\"" ++ Key ++ "\"" ++ " " ++ File), [$:,$\n])] of + Info -> + linux_info_lookup_collect(Key, Info, []) + catch + _:_:_ -> + "-" + end. + +linux_info_lookup_collect(_Key, [], Values) -> + lists:reverse(Values); +linux_info_lookup_collect(Key, [Key, Value|Rest], Values) -> + linux_info_lookup_collect(Key, Rest, [Value|Values]); +linux_info_lookup_collect(_, _, Values) -> + lists:reverse(Values). + + +%% --- OpenBSD --- + +%% Just to be clear: This is ***not*** scientific... +analyze_and_print_openbsd_host_info(Version) -> + io:format("OpenBSD:" + "~n Version: ~p" + "~n", [Version]), + Extract = + fun(Key) -> + string:tokens(string:trim(os:cmd("sysctl " ++ Key)), [$=]) + end, + try + begin + CPU = + case Extract("hw.model") of + ["hw.model", Model] -> + string:trim(Model); + _ -> + "-" + end, + CPUSpeed = + case Extract("hw.cpuspeed") of + ["hw.cpuspeed", Speed] -> + list_to_integer(Speed); + _ -> + -1 + end, + NCPU = + case Extract("hw.ncpufound") of + ["hw.ncpufound", N] -> + list_to_integer(N); + _ -> + -1 + end, + Memory = + case Extract("hw.physmem") of + ["hw.physmem", PhysMem] -> + list_to_integer(PhysMem) div 1024; + _ -> + -1 + end, + io:format("CPU:" + "~n Model: ~s" + "~n Speed: ~w" + "~n N: ~w" + "~nMemory:" + "~n ~w KB" + "~n", [CPU, CPUSpeed, NCPU, Memory]), + CPUFactor = + if + (CPUSpeed =:= -1) -> + 1; + (CPUSpeed >= 2000) -> + if + (NCPU >= 4) -> + 1; + (NCPU >= 2) -> + 2; + true -> + 3 + end; + true -> + if + (NCPU >= 4) -> + 2; + (NCPU >= 2) -> + 3; + true -> + 4 + end + end, + MemAddFactor = + if + (Memory =:= -1) -> + 0; + (Memory >= 8388608) -> + 0; + (Memory >= 4194304) -> + 1; + (Memory >= 2097152) -> + 2; + true -> + 3 + end, + CPUFactor + MemAddFactor + end + catch + _:_:_ -> + 1 + end. + + +%% --- FreeBSD --- + +analyze_and_print_freebsd_host_info(Version) -> + io:format("FreeBSD:" + "~n Version: ~p" + "~n", [Version]), + %% This test require that the program 'sysctl' is in the path. + %% First test with 'which sysctl', if that does not work + %% try with 'which /sbin/sysctl'. If that does not work either, + %% we skip the test... + try + begin + SysCtl = + case string:trim(os:cmd("which sysctl")) of + [] -> + case string:trim(os:cmd("which /sbin/sysctl")) of + [] -> + throw(sysctl); + SC2 -> + SC2 + end; + SC1 -> + SC1 + end, + Extract = + fun(Key) -> + string:tokens(string:trim(os:cmd(SysCtl ++ " " ++ Key)), + [$:]) + end, + CPU = analyze_freebsd_cpu(Extract), + CPUSpeed = analyze_freebsd_cpu_speed(Extract), + NCPU = analyze_freebsd_ncpu(Extract), + Memory = analyze_freebsd_memory(Extract), + io:format("CPU:" + "~n Model: ~s" + "~n Speed: ~w" + "~n N: ~w" + "~n Num Schedulers: ~w" + "~nMemory:" + "~n ~w KB" + "~n", + [CPU, CPUSpeed, NCPU, + erlang:system_info(schedulers), Memory]), + CPUFactor = + if + (CPUSpeed =:= -1) -> + 1; + (CPUSpeed >= 2000) -> + if + (NCPU >= 4) -> + 1; + (NCPU >= 2) -> + 2; + true -> + 3 + end; + true -> + if + (NCPU =:= -1) -> + 1; + (NCPU >= 4) -> + 2; + (NCPU >= 2) -> + 3; + true -> + 4 + end + end, + MemAddFactor = + if + (Memory =:= -1) -> + 0; + (Memory >= 8388608) -> + 0; + (Memory >= 4194304) -> + 1; + (Memory >= 2097152) -> + 2; + true -> + 3 + end, + CPUFactor + MemAddFactor + end + catch + _:_:_ -> + io:format("CPU:" + "~n Num Schedulers: ~w" + "~n", [erlang:system_info(schedulers)]), + case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end + end. + +analyze_freebsd_cpu(Extract) -> + analyze_freebsd_item(Extract, "hw.model", fun(X) -> X end, "-"). + +analyze_freebsd_cpu_speed(Extract) -> + analyze_freebsd_item(Extract, + "hw.clockrate", + fun(X) -> list_to_integer(X) end, + -1). + +analyze_freebsd_ncpu(Extract) -> + analyze_freebsd_item(Extract, + "hw.ncpu", + fun(X) -> list_to_integer(X) end, + -1). + +analyze_freebsd_memory(Extract) -> + analyze_freebsd_item(Extract, + "hw.physmem", + fun(X) -> list_to_integer(X) div 1024 end, + -1). + +analyze_freebsd_item(Extract, Key, Process, Default) -> + try + begin + case Extract(Key) of + [Key, Model] -> + Process(string:trim(Model)); + _ -> + Default + end + end + catch + _:_:_ -> + Default + end. + + +%% --- NetBSD --- + +analyze_and_print_netbsd_host_info(Version) -> + io:format("NetBSD:" + "~n Version: ~p" + "~n", [Version]), + %% This test require that the program 'sysctl' is in the path. + %% First test with 'which sysctl', if that does not work + %% try with 'which /sbin/sysctl'. If that does not work either, + %% we skip the test... + try + begin + SysCtl = + case string:trim(os:cmd("which sysctl")) of + [] -> + case string:trim(os:cmd("which /sbin/sysctl")) of + [] -> + throw(sysctl); + SC2 -> + SC2 + end; + SC1 -> + SC1 + end, + Extract = + fun(Key) -> + [string:trim(S) || + S <- + string:tokens(string:trim(os:cmd(SysCtl ++ " " ++ Key)), + [$=])] + end, + CPU = analyze_netbsd_cpu(Extract), + Machine = analyze_netbsd_machine(Extract), + Arch = analyze_netbsd_machine_arch(Extract), + CPUSpeed = analyze_netbsd_cpu_speed(Extract), + NCPU = analyze_netbsd_ncpu(Extract), + Memory = analyze_netbsd_memory(Extract), + io:format("CPU:" + "~n Model: ~s (~s, ~s)" + "~n Speed: ~w MHz" + "~n N: ~w" + "~n Num Schedulers: ~w" + "~nMemory:" + "~n ~w KB" + "~n", + [CPU, Machine, Arch, CPUSpeed, NCPU, + erlang:system_info(schedulers), Memory]), + CPUFactor = + if + (CPUSpeed =:= -1) -> + 1; + (CPUSpeed >= 2000) -> + if + (NCPU >= 4) -> + 1; + (NCPU >= 2) -> + 2; + true -> + 3 + end; + true -> + if + (NCPU =:= -1) -> + 1; + (NCPU >= 4) -> + 2; + (NCPU >= 2) -> + 3; + true -> + 4 + end + end, + MemAddFactor = + if + (Memory =:= -1) -> + 0; + (Memory >= 8388608) -> + 0; + (Memory >= 4194304) -> + 1; + (Memory >= 2097152) -> + 2; + true -> + 3 + end, + CPUFactor + MemAddFactor + end + catch + _:_:_ -> + io:format("CPU:" + "~n Num Schedulers: ~w" + "~n", [erlang:system_info(schedulers)]), + case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end + end. + +analyze_netbsd_cpu(Extract) -> + analyze_netbsd_item(Extract, "hw.model", fun(X) -> X end, "-"). + +analyze_netbsd_machine(Extract) -> + analyze_netbsd_item(Extract, "hw.machine", fun(X) -> X end, "-"). + +analyze_netbsd_machine_arch(Extract) -> + analyze_netbsd_item(Extract, "hw.machine_arch", fun(X) -> X end, "-"). + +analyze_netbsd_cpu_speed(Extract) -> + analyze_netbsd_item(Extract, "machdep.dmi.processor-frequency", + fun(X) -> case string:tokens(X, [$\ ]) of + [MHz, "MHz"] -> + list_to_integer(MHz); + _ -> + -1 + end + end, "-"). + +analyze_netbsd_ncpu(Extract) -> + analyze_netbsd_item(Extract, + "hw.ncpu", + fun(X) -> list_to_integer(X) end, + -1). + +analyze_netbsd_memory(Extract) -> + analyze_netbsd_item(Extract, + "hw.physmem64", + fun(X) -> list_to_integer(X) div 1024 end, + -1). + +analyze_netbsd_item(Extract, Key, Process, Default) -> + analyze_freebsd_item(Extract, Key, Process, Default). + + + +%% --- Solaris --- + +analyze_and_print_solaris_host_info(Version) -> + Release = + case file:read_file_info("/etc/release") of + {ok, _} -> + case [string:trim(S) || S <- string:tokens(os:cmd("cat /etc/release"), [$\n])] of + [Rel | _] -> + Rel; + _ -> + "-" + end; + _ -> + "-" + end, + %% Display the firmware device tree root properties (prtconf -b) + Props = [list_to_tuple([string:trim(PS) || PS <- Prop]) || + Prop <- [string:tokens(S, [$:]) || + S <- string:tokens(os:cmd("prtconf -b"), [$\n])]], + BannerName = case lists:keysearch("banner-name", 1, Props) of + {value, {_, BN}} -> + string:trim(BN); + _ -> + "-" + end, + InstructionSet = + case string:trim(os:cmd("isainfo -k")) of + "Pseudo-terminal will not" ++ _ -> + "-"; + IS -> + IS + end, + PtrConf = [list_to_tuple([string:trim(S) || S <- Items]) || Items <- [string:tokens(S, [$:]) || S <- string:tokens(os:cmd("prtconf"), [$\n])], length(Items) > 1], + SysConf = + case lists:keysearch("System Configuration", 1, PtrConf) of + {value, {_, SC}} -> + SC; + _ -> + "-" + end, + NumPhysProc = + begin + NPPStr = string:trim(os:cmd("psrinfo -p")), + try list_to_integer(NPPStr) of + _ -> + NPPStr + catch + _:_:_ -> + "-" + end + end, + NumProc = try integer_to_list(length(string:tokens(os:cmd("psrinfo"), [$\n]))) of + NPStr -> + NPStr + catch + _:_:_ -> + "-" + end, + MemSz = + case lists:keysearch("Memory size", 1, PtrConf) of + {value, {_, MS}} -> + MS; + _ -> + "-" + end, + io:format("Solaris: ~s" + "~n Release: ~s" + "~n Banner Name: ~s" + "~n Instruction Set: ~s" + "~n CPUs: ~s (~s)" + "~n System Config: ~s" + "~n Memory Size: ~s" + "~n Num Schedulers: ~s" + "~n~n", [Version, Release, BannerName, InstructionSet, + NumPhysProc, NumProc, + SysConf, MemSz, + str_num_schedulers()]), + MemFactor = + try string:tokens(MemSz, [$ ]) of + [SzStr, "Mega" ++ _] -> + try list_to_integer(SzStr) of + Sz when Sz > 8192 -> + 0; + Sz when Sz > 4096 -> + 1; + Sz when Sz > 2048 -> + 2; + _ -> + 5 + catch + _:_:_ -> + 10 + end; + [SzStr, "Giga" ++ _] -> + try list_to_integer(SzStr) of + Sz when Sz > 8 -> + 0; + Sz when Sz > 4 -> + 1; + Sz when Sz > 2 -> + 2; + _ -> + 5 + catch + _:_:_ -> + 10 + end; + _ -> + 10 + catch + _:_:_ -> + 10 + end, + try erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + N when (N =< 6) -> + 2; + _ -> + 1 + catch + _:_:_ -> + 10 + end + MemFactor. + + +%% --- Windows --- + +analyze_and_print_win_host_info(Version) -> + SysInfo = which_win_system_info(), + OsName = win_sys_info_lookup(os_name, SysInfo), + OsVersion = win_sys_info_lookup(os_version, SysInfo), + SysMan = win_sys_info_lookup(system_manufacturer, SysInfo), + SysMod = win_sys_info_lookup(system_model, SysInfo), + NumProcs = win_sys_info_lookup(num_processors, SysInfo), + TotPhysMem = win_sys_info_lookup(total_phys_memory, SysInfo), + io:format("Windows: ~s" + "~n OS Version: ~s (~p)" + "~n System Manufacturer: ~s" + "~n System Model: ~s" + "~n Number of Processor(s): ~s" + "~n Total Physical Memory: ~s" + "~n Num Schedulers: ~s" + "~n", [OsName, OsVersion, Version, + SysMan, SysMod, NumProcs, TotPhysMem, + str_num_schedulers()]), + MemFactor = + try + begin + [MStr, MUnit|_] = + string:tokens(lists:delete($,, TotPhysMem), [$\ ]), + case string:to_lower(MUnit) of + "gb" -> + try list_to_integer(MStr) of + M when M > 8 -> + 0; + M when M > 4 -> + 1; + M when M > 2 -> + 2; + _ -> + 5 + catch + _:_:_ -> + 10 + end; + "mb" -> + try list_to_integer(MStr) of + M when M > 8192 -> + 0; + M when M > 4096 -> + 1; + M when M > 2048 -> + 2; + _ -> + 5 + catch + _:_:_ -> + 10 + end; + _ -> + 10 + end + end + catch + _:_:_ -> + 10 + end, + CPUFactor = + case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end, + CPUFactor + MemFactor. + +win_sys_info_lookup(Key, SysInfo) -> + win_sys_info_lookup(Key, SysInfo, "-"). + +win_sys_info_lookup(Key, SysInfo, Def) -> + case lists:keysearch(Key, 1, SysInfo) of + {value, {Key, Value}} -> + Value; + false -> + Def + end. + +%% This function only extracts the prop(s) we actually care about! +%% On some hosts this (systeminfo) takes a *long time* (several minutes). +%% And since there is no way to provide a timeout to the os command call, +%% we have to wrap it in a process. +which_win_system_info() -> + F = fun() -> + try + begin + SysInfo = os:cmd("systeminfo"), + process_win_system_info( + string:tokens(SysInfo, [$\r, $\n]), []) + end + catch + C:E:S -> + io:format("Failed get or process System info: " + " Error Class: ~p" + " Error: ~p" + " Stack: ~p" + "~n", [C, E, S]), + [] + end + end, + ?LIB:pcall(F, ?MINS(1), []). + +process_win_system_info([], Acc) -> + Acc; +process_win_system_info([H|T], Acc) -> + case string:tokens(H, [$:]) of + [Key, Value] -> + case string:to_lower(Key) of + "os name" -> + process_win_system_info(T, + [{os_name, string:trim(Value)}|Acc]); + "os version" -> + process_win_system_info(T, + [{os_version, string:trim(Value)}|Acc]); + "system manufacturer" -> + process_win_system_info(T, + [{system_manufacturer, string:trim(Value)}|Acc]); + "system model" -> + process_win_system_info(T, + [{system_model, string:trim(Value)}|Acc]); + "processor(s)" -> + [NumProcStr|_] = string:tokens(Value, [$\ ]), + T2 = lists:nthtail(list_to_integer(NumProcStr), T), + process_win_system_info(T2, + [{num_processors, NumProcStr}|Acc]); + "total physical memory" -> + process_win_system_info(T, + [{total_phys_memory, string:trim(Value)}|Acc]); + _ -> + process_win_system_info(T, Acc) + end; + _ -> + process_win_system_info(T, Acc) + end. + + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + l2a(S) when is_list(S) -> list_to_atom(S). @@ -43494,46 +44287,6 @@ b2l(B) when is_binary(B) -> f(F, A) -> lists:flatten(io_lib:format(F, A)). -%% p(F) -> -%% p(F, []). - -%% p(F, A) -> -%% p(F, A, "", ""). - -%% p(F, A, Before, After) when is_list(Before) andalso is_list(After) -> -%% TcName = -%% case get(tc_name) of -%% undefined -> -%% case get(sname) of -%% undefined -> -%% ""; -%% SName when is_list(SName) -> -%% SName -%% end; -%% Name when is_list(Name) -> -%% Name -%% end, -%% FStr = f("*** [~s][~s][~p] " ++ F ++ "~n", -%% [formated_timestamp(),TcName,self()|A]), -%% i(Before ++ FStr ++ After, []). - - -%% d(F, A) -> -%% d(get(dbg_fd), F, A). - -%% d(undefined, F, A) -> -%% [NodeNameStr|_] = string:split(atom_to_list(node()), [$@]), -%% DbgFileName = f("~s-dbg.txt", [NodeNameStr]), -%% case file:open(DbgFileName, [write]) of -%% {ok, FD} -> -%% put(dbg_fd, FD), -%% d(FD, F, A); -%% {error, Reason} -> -%% exit({failed_open_dbg_file, Reason}) -%% end; -%% d(FD, F, A) -> -%% io:format(FD, "~s~n", [f("[~s] " ++ F, [formated_timestamp()|A])]). - i(F) -> i(F, []). diff --git a/erts/emulator/test/socket_test_evaluator.erl b/lib/kernel/test/socket_test_evaluator.erl index 694f0d5f1e..694f0d5f1e 100644 --- a/erts/emulator/test/socket_test_evaluator.erl +++ b/lib/kernel/test/socket_test_evaluator.erl diff --git a/erts/emulator/test/socket_test_evaluator.hrl b/lib/kernel/test/socket_test_evaluator.hrl index 5be49dc022..5be49dc022 100644 --- a/erts/emulator/test/socket_test_evaluator.hrl +++ b/lib/kernel/test/socket_test_evaluator.hrl diff --git a/erts/emulator/test/socket_test_lib.erl b/lib/kernel/test/socket_test_lib.erl index 6440788651..13d4a4ba8e 100644 --- a/erts/emulator/test/socket_test_lib.erl +++ b/lib/kernel/test/socket_test_lib.erl @@ -21,8 +21,12 @@ -module(socket_test_lib). -export([ + %% Process info pi/1, pi/2, pi/3, + %% Proxy call + pcall/3, + %% Time stuff timestamp/0, tdiff/2, @@ -67,6 +71,21 @@ pi(Node, Pid, Item) when is_pid(Pid) andalso is_atom(Item) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +pcall(F, Timeout, Default) + when is_function(F, 0) andalso is_integer(Timeout) andalso (Timeout > 0) -> + {P, M} = erlang:spawn_monitor(fun() -> exit(F()) end), + receive + {'DOWN', M, process, P, Reply} -> + Reply + after Timeout -> + erlang:demonitor(M, [flush]), + exit(P, kill), + Default + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + timestamp() -> os:timestamp(). @@ -100,7 +119,7 @@ f(F, A) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% has_support_ipv6() -> - case socket:supports(ipv6) of + case socket:is_supported(ipv6) of true -> ok; false -> diff --git a/erts/emulator/test/socket_test_logger.erl b/lib/kernel/test/socket_test_logger.erl index f5d4c8c7b2..f5d4c8c7b2 100644 --- a/erts/emulator/test/socket_test_logger.erl +++ b/lib/kernel/test/socket_test_logger.erl diff --git a/erts/emulator/test/socket_test_ttest.hrl b/lib/kernel/test/socket_test_ttest.hrl index 1a004a9a7a..1a004a9a7a 100644 --- a/erts/emulator/test/socket_test_ttest.hrl +++ b/lib/kernel/test/socket_test_ttest.hrl diff --git a/erts/emulator/test/socket_test_ttest_client.hrl b/lib/kernel/test/socket_test_ttest_client.hrl index 84e736cc34..84e736cc34 100644 --- a/erts/emulator/test/socket_test_ttest_client.hrl +++ b/lib/kernel/test/socket_test_ttest_client.hrl diff --git a/erts/emulator/test/socket_test_ttest_lib.erl b/lib/kernel/test/socket_test_ttest_lib.erl index ebce16dcfa..ebce16dcfa 100644 --- a/erts/emulator/test/socket_test_ttest_lib.erl +++ b/lib/kernel/test/socket_test_ttest_lib.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_client.erl b/lib/kernel/test/socket_test_ttest_tcp_client.erl index f28819ca69..f28819ca69 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_client.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_client.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_client_gen.erl b/lib/kernel/test/socket_test_ttest_tcp_client_gen.erl index 65a3a94d38..65a3a94d38 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_client_gen.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_client_gen.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_client_socket.erl b/lib/kernel/test/socket_test_ttest_tcp_client_socket.erl index 2fb1242028..2fb1242028 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_client_socket.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_client_socket.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_gen.erl b/lib/kernel/test/socket_test_ttest_tcp_gen.erl index 5d20e49359..5d20e49359 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_gen.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_gen.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_server.erl b/lib/kernel/test/socket_test_ttest_tcp_server.erl index 2394dc7924..2394dc7924 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_server.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_server.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_server_gen.erl b/lib/kernel/test/socket_test_ttest_tcp_server_gen.erl index fdf40f1369..fdf40f1369 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_server_gen.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_server_gen.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_server_socket.erl b/lib/kernel/test/socket_test_ttest_tcp_server_socket.erl index 4045bf4e4e..4045bf4e4e 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_server_socket.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_server_socket.erl diff --git a/erts/emulator/test/socket_test_ttest_tcp_socket.erl b/lib/kernel/test/socket_test_ttest_tcp_socket.erl index a1e08e605c..a1e08e605c 100644 --- a/erts/emulator/test/socket_test_ttest_tcp_socket.erl +++ b/lib/kernel/test/socket_test_ttest_tcp_socket.erl diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index e578f3dde4..013cc28c40 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 6.5.2 +KERNEL_VSN = 7.0 diff --git a/lib/megaco/doc/src/notes.xml b/lib/megaco/doc/src/notes.xml index 9f9d5a2c23..c2f72e8b2f 100644 --- a/lib/megaco/doc/src/notes.xml +++ b/lib/megaco/doc/src/notes.xml @@ -37,7 +37,47 @@ section is the version number of Megaco.</p> - <section><title>Megaco 3.18.8</title> + <section><title>Megaco 3.19</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p> + The preliminary version 3 codec(s) prev3a, prev3b and + prev3c has been deprecated and will be *removed* in OTP + 24. The encoding config option 'version3' will continue + to work until OTP 24.</p> + <p> + Own Id: OTP-16531</p> + </item> + </list> + </section> + +</section> + + <section><title>Megaco 3.18.8.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The mini parser could not properly decode some IPv6 + addresses.</p> + <p> + Own Id: OTP-16631 Aux Id: ERIERL-491 </p> + </item> + </list> + </section> + +</section> + +<section><title>Megaco 3.18.8</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/megaco/src/text/megaco_text_mini_parser.yrl b/lib/megaco/src/text/megaco_text_mini_parser.yrl index af3050a05b..2a4041867e 100644 --- a/lib/megaco/src/text/megaco_text_mini_parser.yrl +++ b/lib/megaco/src/text/megaco_text_mini_parser.yrl @@ -77,7 +77,7 @@ Nonterminals Terminals - %% 'AddToken' + 'AddToken' %% 'AndAUDITselectToken' 'AuditCapToken' 'AuditToken' @@ -106,7 +106,7 @@ Terminals %% 'EmergencyValueToken' 'ErrorToken' %% 'EventBufferToken' - %% 'EventsToken' + 'EventsToken' %% 'ExternalToken' 'FailoverToken' 'ForcedToken' @@ -273,7 +273,7 @@ pathName -> safeToken : ensure_pathName('$1') . safeToken -> safeToken2 : make_safe_token('$1') . -%% safeToken2 -> 'AddToken' : '$1' . +safeToken2 -> 'AddToken' : '$1' . safeToken2 -> 'AuditToken' : '$1' . safeToken2 -> 'AuditCapToken' : '$1' . safeToken2 -> 'AuditValueToken' : '$1' . @@ -298,7 +298,7 @@ safeToken2 -> 'EmbedToken' : '$1' . %% safeToken2 -> 'EmergencyOffToken' : '$1' . safeToken2 -> 'ErrorToken' : '$1' . %% safeToken2 -> 'EventBufferToken' : '$1' . -%% safeToken2 -> 'EventsToken' : '$1' . +safeToken2 -> 'EventsToken' : '$1' . %% safeToken2 -> 'ExternalToken' : '$1' . % v3 safeToken2 -> 'FailoverToken' : '$1' . safeToken2 -> 'ForcedToken' : '$1' . diff --git a/lib/megaco/test/megaco_codec_mini_SUITE.erl b/lib/megaco/test/megaco_codec_mini_SUITE.erl index 12113aae70..6c7f60db05 100644 --- a/lib/megaco/test/megaco_codec_mini_SUITE.erl +++ b/lib/megaco/test/megaco_codec_mini_SUITE.erl @@ -39,7 +39,32 @@ init_per_testcase/2, end_per_testcase/2, otp7672_msg01/1, - otp7672_msg02/1 + otp7672_msg02/1, + + otp16631_msg01/1, + otp16631_msg02/1, + otp16631_msg03/1, + otp16631_msg04/1, + otp16631_msg05/1, + otp16631_msg06/1, + otp16631_msg11/1, + otp16631_msg12/1, + otp16631_msg13/1, + otp16631_msg14/1, + otp16631_msg15/1, + otp16631_msg16/1, + otp16631_msg21/1, + otp16631_msg22/1, + otp16631_msg23/1, + otp16631_msg24/1, + otp16631_msg25/1, + otp16631_msg26/1, + otp16631_msg31/1, + otp16631_msg32/1, + otp16631_msg33/1, + otp16631_msg34/1, + otp16631_msg35/1, + otp16631_msg36/1 ]). @@ -62,15 +87,51 @@ all() -> groups() -> [ - {tickets, [], tickets_cases()} + {tickets, [], tickets_cases()}, + {otp7672, [], otp7672_cases()}, + {otp16631, [], otp16631_cases()} ]. tickets_cases() -> [ + {group, otp7672}, + {group, otp16631} + ]. + +otp7672_cases() -> + [ otp7672_msg01, otp7672_msg02 ]. +otp16631_cases() -> + [ + otp16631_msg01, + otp16631_msg02, + otp16631_msg03, + otp16631_msg04, + otp16631_msg05, + otp16631_msg06, + otp16631_msg11, + otp16631_msg12, + otp16631_msg13, + otp16631_msg14, + otp16631_msg15, + otp16631_msg16, + otp16631_msg21, + otp16631_msg22, + otp16631_msg23, + otp16631_msg24, + otp16631_msg25, + otp16631_msg26, + otp16631_msg31, + otp16631_msg32, + otp16631_msg33, + otp16631_msg34, + otp16631_msg35, + otp16631_msg36 + ]. + %% @@ -200,6 +261,342 @@ otp7672(Msg) -> end. + +%% -------------------------------------------------------------- +%% + +otp16631_msg01(suite) -> + []; +otp16631_msg01(Config) when is_list(Config) -> + d("otp16631_msg01 -> entry", []), + ok = otp16631( otp16631_msg01() ), + ok. + +otp16631_msg01() -> + otp16631_msg("a"). + + +%% -- + +otp16631_msg02(suite) -> + []; +otp16631_msg02(Config) when is_list(Config) -> + d("otp16631_msg02 -> entry", []), + ok = otp16631( otp16631_msg02() ), + ok. + +otp16631_msg02() -> + otp16631_msg("b"). + + +%% -- + +otp16631_msg03(suite) -> + []; +otp16631_msg03(Config) when is_list(Config) -> + d("otp16631_msg03 -> entry", []), + ok = otp16631( otp16631_msg03() ), + ok. + +otp16631_msg03() -> + otp16631_msg("c"). + + +%% -- + +otp16631_msg04(suite) -> + []; +otp16631_msg04(Config) when is_list(Config) -> + d("otp16631_msg04 -> entry", []), + ok = otp16631( otp16631_msg04() ), + ok. + +otp16631_msg04() -> + otp16631_msg("d"). + + +%% -- + +otp16631_msg05(suite) -> + []; +otp16631_msg05(Config) when is_list(Config) -> + d("otp16631_msg05 -> entry", []), + ok = otp16631( otp16631_msg05() ), + ok. + +otp16631_msg05() -> + otp16631_msg("e"). + + +%% -- + +otp16631_msg06(suite) -> + []; +otp16631_msg06(Config) when is_list(Config) -> + d("otp16631_msg06 -> entry", []), + ok = otp16631( otp16631_msg06() ), + ok. + +otp16631_msg06() -> + otp16631_msg("f"). + + +%% -- + +otp16631_msg11(suite) -> + []; +otp16631_msg11(Config) when is_list(Config) -> + d("otp16631_msg11 -> entry", []), + ok = otp16631( otp16631_msg11() ), + ok. + +otp16631_msg11() -> + otp16631_msg("000a"). + + +%% -- + +otp16631_msg12(suite) -> + []; +otp16631_msg12(Config) when is_list(Config) -> + d("otp16631_msg12 -> entry", []), + ok = otp16631( otp16631_msg12() ), + ok. + +otp16631_msg12() -> + otp16631_msg("000b"). + + +%% -- + +otp16631_msg13(suite) -> + []; +otp16631_msg13(Config) when is_list(Config) -> + d("otp16631_msg13 -> entry", []), + ok = otp16631( otp16631_msg13() ), + ok. + +otp16631_msg13() -> + otp16631_msg("000c"). + + +%% -- + +otp16631_msg14(suite) -> + []; +otp16631_msg14(Config) when is_list(Config) -> + d("otp16631_msg14 -> entry", []), + ok = otp16631( otp16631_msg14() ), + ok. + +otp16631_msg14() -> + otp16631_msg("000d"). + + +%% -- + +otp16631_msg15(suite) -> + []; +otp16631_msg15(Config) when is_list(Config) -> + d("otp16631_msg15 -> entry", []), + ok = otp16631( otp16631_msg15() ), + ok. + +otp16631_msg15() -> + otp16631_msg("000e"). + + +%% -- + +otp16631_msg16(suite) -> + []; +otp16631_msg16(Config) when is_list(Config) -> + d("otp16631_msg16 -> entry", []), + ok = otp16631( otp16631_msg16() ), + ok. + +otp16631_msg16() -> + otp16631_msg("000f"). + + +%% -- + +otp16631_msg21(suite) -> + []; +otp16631_msg21(Config) when is_list(Config) -> + d("otp16631_msg21 -> entry", []), + ok = otp16631( otp16631_msg21() ), + ok. + +otp16631_msg21() -> + otp16631_msg("0a12"). + + +%% -- + +otp16631_msg22(suite) -> + []; +otp16631_msg22(Config) when is_list(Config) -> + d("otp16631_msg22 -> entry", []), + ok = otp16631( otp16631_msg22() ), + ok. + +otp16631_msg22() -> + otp16631_msg("0b12"). + + +%% -- + +otp16631_msg23(suite) -> + []; +otp16631_msg23(Config) when is_list(Config) -> + d("otp16631_msg23 -> entry", []), + ok = otp16631( otp16631_msg23() ), + ok. + +otp16631_msg23() -> + otp16631_msg("0c12"). + + +%% -- + +otp16631_msg24(suite) -> + []; +otp16631_msg24(Config) when is_list(Config) -> + d("otp16631_msg24 -> entry", []), + ok = otp16631( otp16631_msg24() ), + ok. + +otp16631_msg24() -> + otp16631_msg("0d12"). + + +%% -- + +otp16631_msg25(suite) -> + []; +otp16631_msg25(Config) when is_list(Config) -> + d("otp16631_msg25 -> entry", []), + ok = otp16631( otp16631_msg25() ), + ok. + +otp16631_msg25() -> + otp16631_msg("0e12"). + + +%% -- + +otp16631_msg26(suite) -> + []; +otp16631_msg26(Config) when is_list(Config) -> + d("otp16631_msg26 -> entry", []), + ok = otp16631( otp16631_msg26() ), + ok. + +otp16631_msg26() -> + otp16631_msg("0f12"). + + +%% -- + +otp16631_msg31(suite) -> + []; +otp16631_msg31(Config) when is_list(Config) -> + d("otp16631_msg31 -> entry", []), + ok = otp16631( otp16631_msg31() ), + ok. + +otp16631_msg31() -> + otp16631_msg("a123"). + + +%% -- + +otp16631_msg32(suite) -> + []; +otp16631_msg32(Config) when is_list(Config) -> + d("otp16631_msg32 -> entry", []), + ok = otp16631( otp16631_msg32() ), + ok. + +otp16631_msg32() -> + otp16631_msg("b123"). + + +%% -- + +otp16631_msg33(suite) -> + []; +otp16631_msg33(Config) when is_list(Config) -> + d("otp16631_msg33 -> entry", []), + ok = otp16631( otp16631_msg33() ), + ok. + +otp16631_msg33() -> + otp16631_msg("c123"). + + +%% -- + +otp16631_msg34(suite) -> + []; +otp16631_msg34(Config) when is_list(Config) -> + d("otp16631_msg34 -> entry", []), + ok = otp16631( otp16631_msg34() ), + ok. + +otp16631_msg34() -> + otp16631_msg("d123"). + + +%% -- + +otp16631_msg35(suite) -> + []; +otp16631_msg35(Config) when is_list(Config) -> + d("otp16631_msg35 -> entry", []), + ok = otp16631( otp16631_msg35() ), + ok. + +otp16631_msg35() -> + otp16631_msg("e123"). + + +%% -- + +otp16631_msg36(suite) -> + []; +otp16631_msg36(Config) when is_list(Config) -> + d("otp16631_msg36 -> entry", []), + ok = otp16631( otp16631_msg36() ), + ok. + +otp16631_msg36() -> + otp16631_msg("f123"). + + +%% ----- + +otp16631( Msg ) -> + Bin = erlang:list_to_binary(Msg), + try megaco_compact_text_encoder:decode_mini_message([], dynamic, Bin) of + {ok, _} -> + ok; + {error, _} = ERROR -> + ERROR + catch + C:E:S -> + {error, {C, E, S}} + end. + + +otp16631_msg(X) when is_list(X) -> + "!/1 [2409:8050:5005:1243:1011::" ++ X ++ + "] T=2523{C=-{SC=ROOT{SV{MT=RS,RE=901,PF=ETSI_BGF/2,V=3}}}}". + + + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% p(F, A) -> diff --git a/lib/megaco/test/megaco_mess_SUITE.erl b/lib/megaco/test/megaco_mess_SUITE.erl index 8fb5c4a982..be523f42a2 100644 --- a/lib/megaco/test/megaco_mess_SUITE.erl +++ b/lib/megaco/test/megaco_mess_SUITE.erl @@ -266,6 +266,7 @@ ]). -endif. +-include_lib("common_test/include/ct.hrl"). -include_lib("megaco/include/megaco.hrl"). -include_lib("megaco/include/megaco_message_v1.hrl"). -include("megaco_test_lib.hrl"). @@ -511,18 +512,19 @@ init_per_testcase(Case, Config) -> init_per_testcase2(otp_7189 = Case, Config) -> C = lists:keydelete(tc_timeout, 1, Config), - init_per_testcase3(Case, [{tc_timeout, min(2)} |C]); + init_per_testcase3(Case, [{tc_timeout, min(tfactor(2, Config))} |C]); init_per_testcase2(request_and_no_reply = Case, Config) -> C = lists:keydelete(tc_timeout, 1, Config), - init_per_testcase3(Case, [{tc_timeout, min(2)} |C]); + init_per_testcase3(Case, [{tc_timeout, min(tfactor(2, Config))} |C]); init_per_testcase2(Case, Config) -> C = lists:keydelete(tc_timeout, 1, Config), - init_per_testcase3(Case, [{tc_timeout, min(1)} |C]). + init_per_testcase3(Case, [{tc_timeout, min(tfactor(1, Config))} |C]). init_per_testcase3(Case, Config) -> megaco_test_global_sys_monitor:reset_events(), megaco_test_lib:init_per_testcase(Case, Config). - + + end_per_testcase(Case, Config) -> @@ -539,6 +541,13 @@ end_per_testcase(Case, Config) -> min(M) -> ?MINS(M). +tfactor(T, Config) -> + case ?config(megaco_factor, Config) of + Factor when is_integer(Factor) andalso (Factor > 1) -> + T * Factor; + _ -> + T + end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -10560,6 +10569,7 @@ otp_6442_resend_request2_mg_notify_request_ar(Rid, Tid, Cid) -> otp_6442_resend_reply1(suite) -> []; otp_6442_resend_reply1(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), Pre = fun() -> MgNode = make_node_name(mg), d("start (MG) node: ~p", [MgNode]), @@ -10567,14 +10577,14 @@ otp_6442_resend_reply1(Config) when is_list(Config) -> ok = ?START_NODES(Nodes, true), Nodes end, - Case = fun do_otp_6442_resend_reply1/1, + Case = fun(Nodes) -> do_otp_6442_resend_reply1(Nodes, Factor) end, Post = fun(Nodes) -> d("stop nodes"), ?STOP_NODES(lists:reverse(Nodes)) end, try_tc(request_and_no_reply, Pre, Case, Post). -do_otp_6442_resend_reply1([MgNode]) -> +do_otp_6442_resend_reply1([MgNode], Factor) -> d("[MG] start the simulator "), {ok, Mg} = megaco_test_megaco_generator:start_link("MG", MgNode), @@ -10591,7 +10601,7 @@ do_otp_6442_resend_reply1([MgNode]) -> {ok, MgId} = megaco_test_megaco_generator:exec(Mg, MgEvSeq), i("await the transport module service change send_message event"), - Pid = otp_6442_expect(fun otp_6442_rsrp1_verify_scr_msg/1, 5000), + Pid = otp_6442_expect(fun otp_6442_rsrp1_verify_scr_msg/1, Factor * 5000), i("wait some before issuing the service change reply"), sleep(500), @@ -10610,7 +10620,7 @@ do_otp_6442_resend_reply1([MgNode]) -> i("await the transport module first notify-reply send_message event from MG: " "ignore"), - otp_6442_expect(fun otp_6442_rsrp1_verify_first_nr_msg/1, 5000), + otp_6442_expect(fun otp_6442_rsrp1_verify_first_nr_msg/1, Factor * 5000), i("await the transport module second notify-reply send_message event from MG: " "ack"), @@ -10955,6 +10965,7 @@ otp_6442_resend_reply1_err_desc(T) -> otp_6442_resend_reply2(suite) -> []; otp_6442_resend_reply2(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), Pre = fun() -> MgNode = make_node_name(mg), d("start (MG) node: ~p", [MgNode]), @@ -10962,14 +10973,14 @@ otp_6442_resend_reply2(Config) when is_list(Config) -> ok = ?START_NODES(Nodes, true), Nodes end, - Case = fun do_otp_6442_resend_reply2/1, + Case = fun(Nodes) -> do_otp_6442_resend_reply2(Nodes, Factor) end, Post = fun(Nodes) -> d("stop nodes"), ?STOP_NODES(lists:reverse(Nodes)) end, try_tc(otp6442rrep2, Pre, Case, Post). -do_otp_6442_resend_reply2([MgNode]) -> +do_otp_6442_resend_reply2([MgNode], Factor) -> d("[MG] start the simulator "), {ok, Mg} = megaco_test_megaco_generator:start_link("MG", MgNode), @@ -10986,7 +10997,7 @@ do_otp_6442_resend_reply2([MgNode]) -> {ok, MgId} = megaco_test_megaco_generator:exec(Mg, MgEvSeq), i("await the transport module service change send_message event"), - Pid = otp_6442_expect(fun otp_6442_rsrp2_verify_scr_msg/1, 5000), + Pid = otp_6442_expect(fun otp_6442_rsrp2_verify_scr_msg/1, Factor * 5000), i("wait some before issuing the service change reply"), sleep(500), @@ -11004,7 +11015,7 @@ do_otp_6442_resend_reply2([MgNode]) -> megaco_test_generic_transport:incomming_message(Pid, NotifyRequest), i("await the transport module notify-reply send_message event from MG: ignore"), - otp_6442_expect(otp_6442_rsrp2_verify_first_nr_msg_fun(), 5000), + otp_6442_expect(otp_6442_rsrp2_verify_first_nr_msg_fun(), Factor * 5000), i("await the transport module notify-reply resend_message event from MG: ack"), {TransId, _, _} = diff --git a/lib/megaco/test/megaco_segment_SUITE.erl b/lib/megaco/test/megaco_segment_SUITE.erl index 08b86606de..a403c3309d 100644 --- a/lib/megaco/test/megaco_segment_SUITE.erl +++ b/lib/megaco/test/megaco_segment_SUITE.erl @@ -165,7 +165,7 @@ end_per_group(_Group, Config) -> init_per_testcase(Case, Config) -> process_flag(trap_exit, true), - p("init_per_suite -> entry with" + p("init_per_testcase -> entry with" "~n Config: ~p" "~n Nodes: ~p", [Config, erlang:nodes()]), @@ -175,7 +175,7 @@ init_per_testcase(Case, Config) -> end_per_testcase(Case, Config) -> process_flag(trap_exit, false), - p("end_per_suite -> entry with" + p("end_per_testcase -> entry with" "~n Config: ~p" "~n Nodes: ~p", [Config, erlang:nodes()]), @@ -806,17 +806,9 @@ send_segmented_msg_plain2(doc) -> "Second plain test that it is possible to send segmented messages. " "Send window = infinity. "; send_segmented_msg_plain2(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), + ct:timetrap(?MINS(1) + Factor * ?MINS(1)), Pre = fun() -> - %% We leave it commented out as test - %% All the other changes to the framework - %% may have "solved" the issues... - - %% <CONDITIONAL-SKIP> - %% Skippable = [{unix, [linux]}], - %% Condition = fun() -> ?OS_BASED_SKIP(Skippable) end, - %% ?NON_PC_TC_MAYBE_SKIP(Config, Condition), - %% </CONDITIONAL-SKIP> - MgcNode = make_node_name(mgc), MgNode = make_node_name(mg), d("start nodes: " @@ -827,20 +819,20 @@ send_segmented_msg_plain2(Config) when is_list(Config) -> ok = ?START_NODES(Nodes), Nodes end, - Case = fun do_send_segmented_msg_plain2/1, + Case = fun(X) -> do_send_segmented_msg_plain2(Factor, X) end, Post = fun(Nodes) -> d("stop nodes"), ?STOP_NODES(lists:reverse(Nodes)) end, try_tc(ssmp2, Pre, Case, Post). -do_send_segmented_msg_plain2([MgcNode, MgNode]) -> +do_send_segmented_msg_plain2(Factor, [MgcNode, MgNode]) -> d("[MGC] start the simulator "), {ok, Mgc} = megaco_test_tcp_generator:start_link("MGC", MgcNode), d("[MGC] create the event sequence"), - MgcEvSeq = ssmp2_mgc_event_sequence(text, tcp), + MgcEvSeq = ssmp2_mgc_event_sequence(Factor, text, tcp), i("wait some time before starting the MGC simulation"), sleep(1000), @@ -855,7 +847,7 @@ do_send_segmented_msg_plain2([MgcNode, MgNode]) -> {ok, Mg} = megaco_test_megaco_generator:start_link("MG", MgNode), d("[MG] create the event sequence"), - MgEvSeq = ssmp2_mg_event_sequence(text, tcp), + MgEvSeq = ssmp2_mg_event_sequence(Factor, text, tcp), i("wait some time before starting the MG simulation"), sleep(1000), @@ -883,7 +875,7 @@ do_send_segmented_msg_plain2([MgcNode, MgNode]) -> %% MGC generator stuff %% -ssmp2_mgc_event_sequence(text, tcp) -> +ssmp2_mgc_event_sequence(Factor, text, tcp) -> DecodeFun = ssmp2_mgc_decode_msg_fun(megaco_pretty_text_encoder, []), EncodeFun = ssmp2_mgc_encode_msg_fun(megaco_pretty_text_encoder, []), Mid = {deviceName,"mgc"}, @@ -908,6 +900,7 @@ ssmp2_mgc_event_sequence(text, tcp) -> SegmentRep1 = ssmp2_mgc_segment_reply_msg(Mid, TransId, 1, false), SegmentRep2 = ssmp2_mgc_segment_reply_msg(Mid, TransId, 2, true), TransAck = ssmp2_mgc_trans_ack_msg(Mid, TransId), + TO = fun(T) -> Factor * T end, EvSeq = [{debug, true}, {decode, DecodeFun}, {encode, EncodeFun}, @@ -915,15 +908,17 @@ ssmp2_mgc_event_sequence(text, tcp) -> {expect_accept, any}, {expect_receive, "service-change-request", {ScrVerifyFun, 5000}}, {send, "service-change-reply", ServiceChangeRep}, - {expect_nothing, timer:seconds(1)}, + {expect_nothing, ?SECS(1)}, {send, "notify request", NotifyReq}, - {expect_receive, "notify reply: segment 1", {NrVerifyFun1, 2000}}, + {expect_receive, "notify reply: segment 1", + {NrVerifyFun1, TO(?SECS(2))}}, {send, "segment reply 1", SegmentRep1}, - {expect_receive, "notify reply: segment 2", {NrVerifyFun2, 1000}}, + {expect_receive, "notify reply: segment 2", + {NrVerifyFun2, TO(?SECS(1))}}, {send, "segment reply 2", SegmentRep2}, {sleep, 100}, % {expect_nothing, 500}, {send, "transaction-ack", TransAck}, - {expect_closed, timer:seconds(5)}, + {expect_closed, TO(?SECS(5))}, disconnect ], EvSeq. @@ -1036,14 +1031,13 @@ ssmp2_mgc_verify_notify_reply_segment_msg_fun(SN, Last, ssmp2_mgc_verify_notify_reply_segment(#'MegacoMessage'{mess = Mess} = M, SN, Last, TransId, TermId, Cid) -> - io:format("ssmp2_mgc_verify_notify_reply_segment -> entry with" - "~n M: ~p" - "~n SN: ~p" - "~n Last: ~p" - "~n TransId: ~p" - "~n TermId: ~p" - "~n Cid: ~p" - "~n", [M, SN, Last, TransId, TermId, Cid]), + p("ssmp2_mgc_verify_notify_reply_segment -> entry with" + "~n M: ~p" + "~n SN: ~p" + "~n Last: ~p" + "~n TransId: ~p" + "~n TermId: ~p" + "~n Cid: ~p", [M, SN, Last, TransId, TermId, Cid]), Body = case Mess of #'Message'{version = ?VERSION, @@ -1173,7 +1167,7 @@ ssmp2_mgc_trans_ack_msg(Mid, TransId) -> %% %% MG generator stuff %% -ssmp2_mg_event_sequence(text, tcp) -> +ssmp2_mg_event_sequence(Factor, text, tcp) -> Mid = {deviceName,"mg"}, RI = [ {port, 2944}, @@ -1187,7 +1181,8 @@ ssmp2_mg_event_sequence(text, tcp) -> Tid1 = #megaco_term_id{id = ["00000000","00000000","00000001"]}, Tid2 = #megaco_term_id{id = ["00000000","00000000","00000002"]}, NotifyReqVerify = ssmp2_mg_verify_notify_request_fun(Tid1, Tid2), - AckVerify = ssmp2_mg_verify_ack_fun(), + AckVerify = ssmp2_mg_verify_ack_fun(), + SECS = fun(T) -> ?SECS(Factor * T) end, EvSeq = [ {debug, true}, {megaco_trace, disable}, @@ -1206,12 +1201,12 @@ ssmp2_mg_event_sequence(text, tcp) -> {megaco_update_conn_info, protocol_version, ?VERSION}, {megaco_update_conn_info, segment_send, infinity}, {megaco_update_conn_info, max_pdu_size, 128}, - {sleep, 1000}, + {sleep, ?SECS(1)}, {megaco_callback, handle_trans_request, NotifyReqVerify}, - {megaco_callback, handle_trans_ack, AckVerify, 5000}, + {megaco_callback, handle_trans_ack, AckVerify, SECS(5)}, megaco_stop_user, megaco_stop, - {sleep, 1000} + {sleep, ?SECS(1)} ], EvSeq. @@ -1220,12 +1215,12 @@ ssmp2_mg_verify_handle_connect_fun() -> fun(Ev) -> ssmp2_mg_verify_handle_connect(Ev) end. ssmp2_mg_verify_handle_connect({handle_connect, CH, 1}) -> - io:format("ssmp2_mg_verify_handle_connect -> ok" - "~n CH: ~p~n", [CH]), + p("ssmp2_mg_verify_handle_connect -> ok" + "~n CH: ~p", [CH]), {ok, CH, ok}; ssmp2_mg_verify_handle_connect(Else) -> - io:format("ssmp2_mg_verify_handle_connect -> unknown" - "~n Else: ~p~n", [Else]), + p("ssmp2_mg_verify_handle_connect -> unknown" + "~n Else: ~p", [Else]), {error, Else, ok}. @@ -1235,14 +1230,13 @@ ssmp2_mg_verify_service_change_reply_fun() -> ssmp2_mg_verify_scr({handle_trans_reply, _CH, 1, {ok, [AR]}, _}) -> (catch ssmp2_mg_do_verify_scr(AR)); ssmp2_mg_verify_scr(Crap) -> - io:format("ssmp2_mg_verify_scr -> error: " - "~n Crap: ~p" - "~n", [Crap]), + p("ssmp2_mg_verify_scr -> error: " + "~n Crap: ~p", [Crap]), {error, Crap, ok}. ssmp2_mg_do_verify_scr(AR) -> - io:format("ssmp2_mg_do_verify_scr -> ok: " - "~n AR: ~p~n", [AR]), + p("ssmp2_mg_do_verify_scr -> ok: " + "~n AR: ~p", [AR]), CR = case AR of #'ActionReply'{commandReply = [CmdRep]} -> @@ -1304,30 +1298,27 @@ ssmp2_mg_verify_notify_request( {handle_trans_request, CH, V, ARs}, _Tid1, _Tid2) -> {error, {invalid_trans_request, {CH, V, ARs}}, ok}; ssmp2_mg_verify_notify_request(Crap, _Tid1, _Tid2) -> - io:format("ssmp2_mg_verify_notify_request -> unknown request" - "~n Tid1: ~p" - "~n Tid2: ~p" - "~n Crap: ~p" - "~n", [_Tid1, _Tid2, Crap]), + p("ssmp2_mg_verify_notify_request -> unknown request" + "~n Tid1: ~p" + "~n Tid2: ~p" + "~n Crap: ~p", [_Tid1, _Tid2, Crap]), {error, {unexpected_event, Crap}, ok}. ssmp2_mg_do_verify_notify_request(Tid1, Tid2, AR1, AR2) -> - io:format("ssmp2_mg_do_verify_notify_request -> ok" - "~n Tid1: ~p" - "~n Tid2: ~p" - "~n AR1: ~p" - "~n AR2: ~p" - "~n", [Tid1, Tid2, AR1, AR2]), + p("ssmp2_mg_do_verify_notify_request -> ok" + "~n Tid1: ~p" + "~n Tid2: ~p" + "~n AR1: ~p" + "~n AR2: ~p", [Tid1, Tid2, AR1, AR2]), ActionReply1 = ssmp2_mg_do_verify_notify_request(Tid1, AR1), ActionReply2 = ssmp2_mg_do_verify_notify_request(Tid2, AR2), Reply = {{handle_ack, ssmp2}, [ActionReply1, ActionReply2]}, {ok, [AR1, AR2], Reply}. ssmp2_mg_do_verify_notify_request(Tid, AR) -> - io:format("ssmp2_mg_do_verify_notify_request -> ok" - "~n Tid: ~p" - "~n AR: ~p" - "~n", [Tid, AR]), + p("ssmp2_mg_do_verify_notify_request -> ok" + "~n Tid: ~p" + "~n AR: ~p", [Tid, AR]), {Cid, CR} = case AR of #'ActionRequest'{contextId = CtxId, @@ -1375,9 +1366,8 @@ ssmp2_mg_verify_ack_fun() -> fun(Event) -> ssmp2_mg_verify_ack(Event) end. ssmp2_mg_verify_ack({handle_trans_ack, CH, ?VERSION, ok, ssmp2}) -> - io:format("ssmp2_mg_verify_ack -> ok" - "~n CH: ~p" - "~n", [CH]), + p("ssmp2_mg_verify_ack -> ok" + "~n CH: ~p", [CH]), {ok, CH, ok}; ssmp2_mg_verify_ack({handle_trans_ack, CH, ?VERSION, ok, CrapAckData}) -> {error, {unknown_ack_data, CrapAckData, CH}, ok}; @@ -1412,6 +1402,8 @@ send_segmented_msg_plain3(doc) -> "Third plain test that it is possible to send segmented messages. " "Send window = 1. "; send_segmented_msg_plain3(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), + ct:timetrap(?MINS(1) + Factor * ?MINS(1)), Pre = fun() -> MgcNode = make_node_name(mgc), MgNode = make_node_name(mg), @@ -1612,9 +1604,8 @@ ssmp3_mgc_verify_service_change_req_msg_fun() -> end. ssmp3_mgc_verify_service_change_req(#'MegacoMessage'{mess = Mess} = M) -> - io:format("ssmp3_mgc_verify_service_change_req -> entry with" - "~n M: ~p" - "~n", [M]), + p("ssmp3_mgc_verify_service_change_req -> entry with" + "~n M: ~p", [M]), Body = case Mess of #'Message'{version = 1, @@ -1704,14 +1695,13 @@ ssmp3_mgc_verify_notify_reply_segment_msg_fun(SN, Last, ssmp3_mgc_verify_notify_reply_segment(#'MegacoMessage'{mess = Mess} = M, SN, Last, TransId, TermId, Cid) -> - io:format("ssmp3_mgc_verify_notify_reply_segment -> entry with" - "~n M: ~p" - "~n SN: ~p" - "~n Last: ~p" - "~n TransId: ~p" - "~n TermId: ~p" - "~n Cid: ~p" - "~n", [M, SN, Last, TransId, TermId, Cid]), + p("ssmp3_mgc_verify_notify_reply_segment -> entry with" + "~n M: ~p" + "~n SN: ~p" + "~n Last: ~p" + "~n TransId: ~p" + "~n TermId: ~p" + "~n Cid: ~p", [M, SN, Last, TransId, TermId, Cid]), Body = case Mess of #'Message'{version = ?VERSION, @@ -1903,12 +1893,12 @@ ssmp3_mg_verify_handle_connect_fun() -> fun(Ev) -> ssmp3_mg_verify_handle_connect(Ev) end. ssmp3_mg_verify_handle_connect({handle_connect, CH, 1}) -> - io:format("ssmp3_mg_verify_handle_connect -> ok" - "~n CH: ~p~n", [CH]), + p("ssmp3_mg_verify_handle_connect -> ok" + "~n CH: ~p", [CH]), {ok, CH, ok}; ssmp3_mg_verify_handle_connect(Else) -> - io:format("ssmp3_mg_verify_handle_connect -> unknown" - "~n Else: ~p~n", [Else]), + p("ssmp3_mg_verify_handle_connect -> unknown" + "~n Else: ~p", [Else]), {error, Else, ok}. @@ -1918,14 +1908,13 @@ ssmp3_mg_verify_service_change_reply_fun() -> ssmp3_mg_verify_scr({handle_trans_reply, _CH, 1, {ok, [AR]}, _}) -> (catch ssmp3_mg_do_verify_scr(AR)); ssmp3_mg_verify_scr(Crap) -> - io:format("ssmp3_mg_verify_scr -> error: " - "~n Crap: ~p" - "~n", [Crap]), + p("ssmp3_mg_verify_scr -> error: " + "~n Crap: ~p", [Crap]), {error, Crap, ok}. ssmp3_mg_do_verify_scr(AR) -> - io:format("ssmp3_mg_do_verify_scr -> ok: " - "~n AR: ~p~n", [AR]), + p("ssmp3_mg_do_verify_scr -> ok: " + "~n AR: ~p", [AR]), CR = case AR of #'ActionReply'{commandReply = [CmdRep]} -> @@ -1988,21 +1977,18 @@ ssmp3_mg_verify_notify_request( {handle_trans_request, CH, V, ARs}, _Tids) -> {error, {invalid_trans_request, {CH, V, ARs}}, ok}; ssmp3_mg_verify_notify_request(Crap, _Tids) -> - io:format("ssmp3_mg_verify_notify_request -> unknown request" - "~n Crap: ~p" - "~n Tids: ~p" - "~n", [Crap, _Tids]), + p("ssmp3_mg_verify_notify_request -> unknown request" + "~n Crap: ~p" + "~n Tids: ~p", [Crap, _Tids]), {error, {unexpected_event, Crap}, ok}. ssmp3_mg_do_verify_notify_request(Tids, ARs) -> - io:format("ssmp3_mg_do_verify_notify_request -> ok" - "~n Tids: ~p" - "~n ARs: ~p" - "~n", [Tids, ARs]), + p("ssmp3_mg_do_verify_notify_request -> ok" + "~n Tids: ~p" + "~n ARs: ~p", [Tids, ARs]), ActionReplies = ssmp3_mg_do_verify_notify_request_ars(Tids, ARs), - io:format("ssmp3_mg_do_verify_notify_request -> ok" - "~n ActionReplies: ~p" - "~n", [ActionReplies]), + p("ssmp3_mg_do_verify_notify_request -> ok" + "~n ActionReplies: ~p", [ActionReplies]), Reply = {{handle_ack, ssmp3}, ActionReplies}, {ok, ARs, Reply}. @@ -2016,10 +2002,9 @@ ssmp3_mg_do_verify_notify_request_ars([Tid|Tids], [AR|ARs], Acc) -> ssmp3_mg_do_verify_notify_request_ars(Tids, ARs, [ActionReply|Acc]). ssmp3_mg_do_verify_notify_request_ar(Tid, AR) -> - io:format("ssmp3_mg_do_verify_notify_request_ar -> ok" - "~n Tid: ~p" - "~n AR: ~p" - "~n", [Tid, AR]), + p("ssmp3_mg_do_verify_notify_request_ar -> ok" + "~n Tid: ~p" + "~n AR: ~p", [Tid, AR]), {Cid, CR} = case AR of #'ActionRequest'{contextId = CtxId, @@ -2067,9 +2052,8 @@ ssmp3_mg_verify_ack_fun() -> fun(Event) -> ssmp3_mg_verify_ack(Event) end. ssmp3_mg_verify_ack({handle_trans_ack, CH, ?VERSION, ok, ssmp3}) -> - io:format("ssmp3_mg_verify_ack -> ok" - "~n CH: ~p" - "~n", [CH]), + p("ssmp3_mg_verify_ack -> ok" + "~n CH: ~p", [CH]), {ok, CH, ok}; ssmp3_mg_verify_ack({handle_trans_ack, CH, ?VERSION, ok, CrapAckData}) -> {error, {unknown_ack_data, CrapAckData, CH}, ok}; @@ -2105,7 +2089,7 @@ send_segmented_msg_plain4(doc) -> "Send window = 3. "; send_segmented_msg_plain4(Config) when is_list(Config) -> Factor = ?config(megaco_factor, Config), - ct:timetrap(Factor * ?SECS(60)), + ct:timetrap(Factor * ?MINS(1)), Pre = fun() -> MgcNode = make_node_name(mgc), MgNode = make_node_name(mg), @@ -2302,9 +2286,8 @@ ssmp4_mgc_verify_service_change_req_msg_fun() -> end. ssmp4_mgc_verify_service_change_req(#'MegacoMessage'{mess = Mess} = M) -> - io:format("ssmp4_mgc_verify_service_change_req -> entry with" - "~n M: ~p" - "~n", [M]), + p("ssmp4_mgc_verify_service_change_req -> entry with" + "~n M: ~p", [M]), Body = case Mess of #'Message'{version = 1, @@ -2394,14 +2377,13 @@ ssmp4_mgc_verify_notify_reply_segment_msg_fun(SN, Last, ssmp4_mgc_verify_notify_reply_segment(#'MegacoMessage'{mess = Mess} = M, SN, Last, TransId, TermId, Cid) -> - io:format("ssmp4_mgc_verify_notify_reply_segment -> entry with" - "~n M: ~p" - "~n SN: ~p" - "~n Last: ~p" - "~n TransId: ~p" - "~n TermId: ~p" - "~n Cid: ~p" - "~n", [M, SN, Last, TransId, TermId, Cid]), + p("ssmp4_mgc_verify_notify_reply_segment -> entry with" + "~n M: ~p" + "~n SN: ~p" + "~n Last: ~p" + "~n TransId: ~p" + "~n TermId: ~p" + "~n Cid: ~p", [M, SN, Last, TransId, TermId, Cid]), Body = case Mess of #'Message'{version = ?VERSION, @@ -2582,7 +2564,7 @@ ssmp4_mg_event_sequence(Factor, text, tcp) -> {megaco_update_conn_info, max_pdu_size, 128}, {sleep, 1000}, {megaco_callback, handle_trans_request, NotifyReqVerify}, - {megaco_callback, handle_trans_ack, AckVerify, TO(15000)}, + {megaco_callback, handle_trans_ack, AckVerify, TO(?SECS(15))}, megaco_stop_user, megaco_stop, {sleep, 1000} @@ -2594,12 +2576,12 @@ ssmp4_mg_verify_handle_connect_fun() -> fun(Ev) -> ssmp4_mg_verify_handle_connect(Ev) end. ssmp4_mg_verify_handle_connect({handle_connect, CH, 1}) -> - io:format("ssmp4_mg_verify_handle_connect -> ok" - "~n CH: ~p~n", [CH]), + p("ssmp4_mg_verify_handle_connect -> ok" + "~n CH: ~p", [CH]), {ok, CH, ok}; ssmp4_mg_verify_handle_connect(Else) -> - io:format("ssmp4_mg_verify_handle_connect -> unknown" - "~n Else: ~p~n", [Else]), + p("ssmp4_mg_verify_handle_connect -> unknown" + "~n Else: ~p", [Else]), {error, Else, ok}. @@ -2609,14 +2591,13 @@ ssmp4_mg_verify_service_change_reply_fun() -> ssmp4_mg_verify_scr({handle_trans_reply, _CH, 1, {ok, [AR]}, _}) -> (catch ssmp4_mg_do_verify_scr(AR)); ssmp4_mg_verify_scr(Crap) -> - io:format("ssmp4_mg_verify_scr -> error: " - "~n Crap: ~p" - "~n", [Crap]), + p("ssmp4_mg_verify_scr -> error: " + "~n Crap: ~p", [Crap]), {error, Crap, ok}. ssmp4_mg_do_verify_scr(AR) -> - io:format("ssmp4_mg_do_verify_scr -> ok: " - "~n AR: ~p~n", [AR]), + p("ssmp4_mg_do_verify_scr -> ok: " + "~n AR: ~p", [AR]), CR = case AR of #'ActionReply'{commandReply = [CmdRep]} -> @@ -2761,9 +2742,8 @@ ssmp4_mg_verify_ack_fun() -> fun(Event) -> ssmp4_mg_verify_ack(Event) end. ssmp4_mg_verify_ack({handle_trans_ack, CH, ?VERSION, ok, ssmp4}) -> - io:format("ssmp4_mg_verify_ack -> ok" - "~n CH: ~p" - "~n", [CH]), + p("ssmp4_mg_verify_ack -> ok" + "~n CH: ~p", [CH]), {ok, CH, ok}; ssmp4_mg_verify_ack({handle_trans_ack, CH, ?VERSION, ok, CrapAckData}) -> {error, {unknown_ack_data, CrapAckData, CH}, ok}; @@ -2992,9 +2972,8 @@ ssmo1_mgc_verify_service_change_req_msg_fun() -> end. ssmo1_mgc_verify_service_change_req(#'MegacoMessage'{mess = Mess} = M) -> - io:format("ssmo1_mgc_verify_service_change_req -> entry with" - "~n M: ~p" - "~n", [M]), + p("ssmo1_mgc_verify_service_change_req -> entry with" + "~n M: ~p", [M]), Body = case Mess of #'Message'{version = 1, @@ -7896,12 +7875,12 @@ await_completion(Ids) -> ok; {error, {OK, ERROR}} -> d("ERROR => " - "~n OK: ~p" - "~n ERROR: ~p", [OK, ERROR]), + "~n OK: ~p" + "~n ERROR: ~p", [OK, ERROR]), ?ERROR({failed, ERROR}); {error, Reply} -> d("ERROR => " - "~n Reply: ~p", [Reply]), + "~n Reply: ~p", [Reply]), ?ERROR({failed, Reply}) end. diff --git a/lib/megaco/test/megaco_test_lib.erl b/lib/megaco/test/megaco_test_lib.erl index d73ed45add..97d408bee8 100644 --- a/lib/megaco/test/megaco_test_lib.erl +++ b/lib/megaco/test/megaco_test_lib.erl @@ -29,6 +29,7 @@ %% -compile(export_all). -export([ + proxy_call/3, log/4, error/3, @@ -71,6 +72,31 @@ %% ---------------------------------------------------------------- +%% Proxy Call +%% This is used when we need to assign a timeout to a call, but the +%% call itself does not provide such an argument. +%% +%% This has nothing to to with the proxy_start and proxy_init +%% functions below. + +proxy_call(F, Timeout, Default) + when is_function(F, 0) andalso + is_integer(Timeout) andalso (Timeout > 0) andalso + is_function(Default, 0) -> + {P, M} = erlang:spawn_monitor(fun() -> exit(F()) end), + receive + {'DOWN', M, process, P, Reply} -> + Reply + after Timeout -> + erlang:demonitor(M, [flush]), + exit(P, kill), + Default() + end; +proxy_call(F, Timeout, Default) -> + proxy_call(F, Timeout, fun() -> Default end). + + +%% ---------------------------------------------------------------- %% Time related function %% @@ -164,6 +190,10 @@ os_base_skip(Skippable, OsFam, OsName) -> case lists:keysearch(OsFam, 1, Skippable) of {value, {OsFam, OsName}} -> true; + {value, {OsFam, Check}} when is_function(Check, 0) -> + Check(); + {value, {OsFam, Check}} when is_function(Check, 1) -> + Check(os:version()); {value, {OsFam, OsNames}} when is_list(OsNames) -> %% OsNames is a list of: %% [atom()|{atom(), function/0 | function/1}] @@ -443,11 +473,31 @@ pprint(F, A) -> init_per_suite(Config) -> + ct:timetrap(minutes(3)), + + try analyze_and_print_host_info() of + {Factor, HostInfo} when is_integer(Factor) -> + try maybe_skip(HostInfo) of + true -> + {skip, "Unstable host and/or os (or combo thererof)"}; + false -> + maybe_start_global_sys_monitor(Config), + [{megaco_factor, Factor} | Config] + catch + throw:{skip, _} = SKIP -> + SKIP + end + catch + throw:{skip, _} = SKIP -> + SKIP + end. + +maybe_skip(HostInfo) -> + %% We have some crap machines that causes random test case failures %% for no obvious reason. So, attempt to identify those without actually %% checking for the host name... - %% We have two "machines" we are checking for. Both are old installations - %% running on really slow VMs (the host machines are old and tired). + LinuxVersionVerify = fun(V) when (V > {3,6,11}) -> false; % OK - No skip @@ -458,6 +508,28 @@ init_per_suite(Config) -> _ -> false end; + (V) when (V =:= {3,4,20}) -> + case string:trim(os:cmd("cat /etc/issue")) of + "Wind River Linux 5.0.1.0" ++ _ -> % *Old* Wind River => skip + true; + _ -> + false + end; + (V) when (V =:= {2,6,32}) -> + case string:trim(os:cmd("cat /etc/issue")) of + "Debian GNU/Linux 6.0 " ++ _ -> % Stone age Debian => Skip + true; + _ -> + false + end; + (V) when (V =:= {2,6,10}) -> + case string:trim(os:cmd("cat /etc/issue")) of + "MontaVista" ++ _ -> % Stone age MontaVista => Skip + %% The real problem is that the machine is *very* slow + true; + _ -> + false + end; (V) when (V > {2,6,24}) -> false; % OK - No skip (_) -> @@ -478,37 +550,22 @@ init_per_suite(Config) -> %% This version is *not* ok: Skip true end, - %% We are "looking" for a specific machine (a VM) - %% which are *old and crappy" and slow, because it - %% causes a bunch of test cases to fail randomly. - %% But we don not want to test for the host name... - %% WinVersionVerify = - %% fun(V) when (V =:= {6,2,9200}) -> - %% try erlang:system_info(schedulers) of - %% 2 -> - %% true; - %% _ -> - %% false - %% catch - %% _:_:_ -> - %% true - %% end; - %% (_) -> - %% false - %% end, + SkipWindowsOnVirtual = + fun() -> + SysMan = win_sys_info_lookup(system_manufacturer, HostInfo), + case string:to_lower(SysMan) of + "vmware" ++ _ -> + true; + _ -> + false + end + end, COND = [ {unix, [{linux, LinuxVersionVerify}, - {darwin, DarwinVersionVerify}]}%% , - %% {win32, [{nt, WinVersionVerify}]} + {darwin, DarwinVersionVerify}]}, + {win32, SkipWindowsOnVirtual} ], - case os_based_skip(COND) of - true -> - {skip, "Unstable host and/or os (or combo thererof)"}; - false -> - Factor = analyze_and_print_host_info(), - maybe_start_global_sys_monitor(Config), - [{megaco_factor, Factor} | Config] - end. + os_based_skip(COND). %% We start the global system monitor unless explicitly disabled maybe_start_global_sys_monitor(Config) -> @@ -569,10 +626,6 @@ end_per_testcase(_Case, Config) -> %% the load for some test cases. Such as run time or number of %% iteraions. This only works for some OSes. %% -%% We make some calculations on Linux, OpenBSD and FreeBSD. -%% On SunOS we always set the factor to 2 (just to be on the safe side) -%% On all other os:es (mostly windows) we check the number of schedulers, -%% but at least the factor will be 2. analyze_and_print_host_info() -> {OsFam, OsName} = os:type(), Version = @@ -591,6 +644,8 @@ analyze_and_print_host_info() -> analyze_and_print_freebsd_host_info(Version); {unix, netbsd} -> analyze_and_print_netbsd_host_info(Version); + {unix, darwin} -> + analyze_and_print_darwin_host_info(Version); {unix, sunos} -> analyze_and_print_solaris_host_info(Version); {win32, nt} -> @@ -601,19 +656,7 @@ analyze_and_print_host_info() -> "~n Version: ~p" "~n Num Schedulers: ~s" "~n", [OsFam, OsName, Version, str_num_schedulers()]), - try erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - N when (N =< 6) -> - 2; - _ -> - 1 - catch - _:_:_ -> - 10 - end + {num_schedulers_to_factor(), []} end. str_num_schedulers() -> @@ -623,106 +666,285 @@ str_num_schedulers() -> _:_:_ -> "-" end. +num_schedulers_to_factor() -> + try erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + N when (N =< 6) -> + 2; + _ -> + 1 + catch + _:_:_ -> + 10 + end. -analyze_and_print_linux_host_info(Version) -> + + +linux_which_distro(Version) -> case file:read_file_info("/etc/issue") of {ok, _} -> - io:format("Linux: ~s" - "~n ~s" - "~n", - [Version, string:trim(os:cmd("cat /etc/issue"))]); + case [string:trim(S) || + S <- string:tokens(os:cmd("cat /etc/issue"), [$\n])] of + [DistroStr|_] -> + io:format("Linux: ~s" + "~n ~s" + "~n", + [Version, DistroStr]), + case DistroStr of + "Wind River Linux" ++ _ -> + wind_river; + "MontaVista" ++ _ -> + montavista; + "Yellow Dog" ++ _ -> + yellow_dog; + _ -> + other + end; + X -> + io:format("Linux: ~s" + "~n ~p" + "~n", + [Version, X]), + other + end; _ -> io:format("Linux: ~s" - "~n", [Version]) - end, + "~n", [Version]), + other + end. + + +analyze_and_print_linux_host_info(Version) -> + Distro = linux_which_distro(Version), Factor = - case (catch linux_which_cpuinfo()) of + case (catch linux_which_cpuinfo(Distro)) of {ok, {CPU, BogoMIPS}} -> io:format("CPU: " "~n Model: ~s" - "~n BogoMIPS: ~s" + "~n BogoMIPS: ~w" "~n Num Schedulers: ~s" "~n", [CPU, BogoMIPS, str_num_schedulers()]), - %% We first assume its a float, and if not try integer - try list_to_float(string:trim(BogoMIPS)) of - F when F > 4000 -> + if + (BogoMIPS > 20000) -> 1; - F when F > 1000 -> + (BogoMIPS > 10000) -> 2; - F when F > 500 -> + (BogoMIPS > 5000) -> 3; - _ -> - 5 - catch - _:_:_ -> - try list_to_integer(string:trim(BogoMIPS)) of - I when I > 4000 -> - 1; - I when I > 1000 -> - 2; - I when I > 500 -> - 3; - _ -> - 5 - catch - _:_:_ -> - 5 % Be a "bit" conservative... - end + (BogoMIPS > 2000) -> + 5; + (BogoMIPS > 1000) -> + 8; + true -> + 10 end; {ok, CPU} -> io:format("CPU: " "~n Model: ~s" "~n Num Schedulers: ~s" "~n", [CPU, str_num_schedulers()]), - 2; % Be a "bit" conservative... + num_schedulers_to_factor(); _ -> - 5 % Be a "bit" (more) conservative... + 5 end, %% Check if we need to adjust the factor because of the memory try linux_which_meminfo() of AddFactor -> - Factor + AddFactor + {Factor + AddFactor, []} catch _:_:_ -> - Factor + {Factor, []} + end. + + +linux_cpuinfo_lookup(Key) when is_list(Key) -> + linux_info_lookup(Key, "/proc/cpuinfo"). + +linux_cpuinfo_cpu() -> + case linux_cpuinfo_lookup("cpu") of + [Model] -> + Model; + _ -> + "-" + end. + +linux_cpuinfo_motherboard() -> + case linux_cpuinfo_lookup("motherboard") of + [MB] -> + MB; + _ -> + "-" + end. + +linux_cpuinfo_bogomips() -> + case linux_cpuinfo_lookup("bogomips") of + BMips when is_list(BMips) -> + try lists:sum([bogomips_to_int(BM) || BM <- BMips]) + catch + _:_:_ -> + "-" + end; + _ -> + "-" end. -linux_which_cpuinfo() -> +linux_cpuinfo_total_bogomips() -> + case linux_cpuinfo_lookup("total bogomips") of + [TBM] -> + try bogomips_to_int(TBM) + catch + _:_:_ -> + "-" + end; + _ -> + "-" + end. + +bogomips_to_int(BM) -> + try list_to_float(BM) of + F -> + floor(F) + catch + _:_:_ -> + try list_to_integer(BM) of + I -> + I + catch + _:_:_ -> + throw(noinfo) + end + end. + +linux_cpuinfo_model() -> + case linux_cpuinfo_lookup("model") of + [M] -> + M; + _ -> + "-" + end. + +linux_cpuinfo_platform() -> + case linux_cpuinfo_lookup("platform") of + [P] -> + P; + _ -> + "-" + end. + +linux_cpuinfo_model_name() -> + case linux_cpuinfo_lookup("model name") of + [P|_] -> + P; + _X -> + "-" + end. + +linux_cpuinfo_processor() -> + case linux_cpuinfo_lookup("Processor") of + [P] -> + P; + _ -> + "-" + end. + +linux_which_cpuinfo(montavista) -> + CPU = + case linux_cpuinfo_cpu() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_motherboard() of + "-" -> + Model; + MB -> + Model ++ " (" ++ MB ++ ")" + end + end, + case linux_cpuinfo_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end; + +linux_which_cpuinfo(yellow_dog) -> + CPU = + case linux_cpuinfo_cpu() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_motherboard() of + "-" -> + Model; + MB -> + Model ++ " (" ++ MB ++ ")" + end + end, + {ok, CPU}; + +linux_which_cpuinfo(wind_river) -> + CPU = + case linux_cpuinfo_model() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_platform() of + "-" -> + Model; + Platform -> + Model ++ " (" ++ Platform ++ ")" + end + end, + case linux_cpuinfo_total_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end; + +linux_which_cpuinfo(other) -> %% Check for x86 (Intel or AMD) CPU = - try [string:trim(S) || S <- string:tokens(os:cmd("grep \"model name\" /proc/cpuinfo"), [$:,$\n])] of - ["model name", ModelName | _] -> - ModelName; - _ -> + case linux_cpuinfo_model_name() of + "-" -> %% ARM (at least some distros...) - try [string:trim(S) || S <- string:tokens(os:cmd("grep \"Processor\" /proc/cpuinfo"), [$:,$\n])] of - ["Processor", Proc | _] -> - Proc; - _ -> + case linux_cpuinfo_processor() of + "-" -> %% Ok, we give up - throw(noinfo) - catch - _:_:_ -> - throw(noinfo) - end - catch - _:_:_ -> - throw(noinfo) + throw(noinfo); + Proc -> + Proc + end; + ModelName -> + ModelName end, - try [string:trim(S) || S <- string:tokens(os:cmd("grep -i \"bogomips\" /proc/cpuinfo"), [$:,$\n])] of - [_, BMips | _] -> - {ok, {CPU, BMips}}; + case linux_cpuinfo_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end. + +linux_meminfo_lookup(Key) when is_list(Key) -> + linux_info_lookup(Key, "/proc/meminfo"). + +linux_meminfo_memtotal() -> + case linux_meminfo_lookup("MemTotal") of + [X] -> + X; _ -> - {ok, CPU} - catch - _:_:_ -> - {ok, CPU} + "-" end. %% We *add* the value this return to the Factor. linux_which_meminfo() -> - try [string:trim(S) || S <- string:tokens(os:cmd("grep MemTotal /proc/meminfo"), [$:])] of - [_, MemTotal] -> + case linux_meminfo_memtotal() of + "-" -> + 0; + MemTotal -> io:format("Memory:" "~n ~s" "~n", [MemTotal]), @@ -752,12 +974,7 @@ linux_which_meminfo() -> end; _X -> 0 - end; - _ -> - 0 - catch - _:_:_ -> - 0 + end end. @@ -843,11 +1060,11 @@ analyze_and_print_openbsd_host_info(Version) -> true -> 3 end, - CPUFactor + MemAddFactor + {CPUFactor + MemAddFactor, []} end catch _:_:_ -> - 1 + {5, []} end. @@ -930,21 +1147,22 @@ analyze_and_print_freebsd_host_info(Version) -> true -> 3 end, - CPUFactor + MemAddFactor + {CPUFactor + MemAddFactor, []} end catch _:_:_ -> io:format("CPU:" "~n Num Schedulers: ~w" "~n", [erlang:system_info(schedulers)]), - case erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - _ -> - 2 - end + Factor = case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end, + {Factor, []} end. analyze_freebsd_cpu(Extract) -> @@ -1067,21 +1285,22 @@ analyze_and_print_netbsd_host_info(Version) -> true -> 3 end, - CPUFactor + MemAddFactor + {CPUFactor + MemAddFactor, []} end catch _:_:_ -> io:format("CPU:" "~n Num Schedulers: ~w" "~n", [erlang:system_info(schedulers)]), - case erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - _ -> - 2 - end + Factor = case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end, + {Factor, []} end. analyze_netbsd_cpu(Extract) -> @@ -1120,6 +1339,289 @@ analyze_netbsd_item(Extract, Key, Process, Default) -> +%% Model Identifier: Macmini7,1 +%% Processor Name: Intel Core i5 +%% Processor Speed: 2,6 GHz +%% Number of Processors: 1 +%% Total Number of Cores: 2 +%% L2 Cache (per Core): 256 KB +%% L3 Cache: 3 MB +%% Hyper-Threading Technology: Enabled +%% Memory: 16 GB + +analyze_and_print_darwin_host_info(Version) -> + %% This stuff is for macOS. + %% If we ever tested on a pure darwin machine, + %% we need to find some other way to find some info... + %% Also, I suppose its possible that we for some other + %% reason *fail* to get the info... + case analyze_darwin_software_info() of + [] -> + io:format("Darwin:" + "~n Version: ~s" + "~n Num Schedulers: ~s" + "~n", [Version, str_num_schedulers()]), + {num_schedulers_to_factor(), []}; + SwInfo when is_list(SwInfo) -> + SystemVersion = analyze_darwin_sw_system_version(SwInfo), + KernelVersion = analyze_darwin_sw_kernel_version(SwInfo), + HwInfo = analyze_darwin_hardware_info(), + ModelName = analyze_darwin_hw_model_name(HwInfo), + ModelId = analyze_darwin_hw_model_identifier(HwInfo), + ProcName = analyze_darwin_hw_processor_name(HwInfo), + ProcSpeed = analyze_darwin_hw_processor_speed(HwInfo), + NumProc = analyze_darwin_hw_number_of_processors(HwInfo), + NumCores = analyze_darwin_hw_total_number_of_cores(HwInfo), + Memory = analyze_darwin_hw_memory(HwInfo), + io:format("Darwin:" + "~n System Version: ~s" + "~n Kernel Version: ~s" + "~n Model: ~s (~s)" + "~n Processor: ~s (~s, ~s, ~s)" + "~n Memory: ~s" + "~n Num Schedulers: ~s" + "~n", [SystemVersion, KernelVersion, + ModelName, ModelId, + ProcName, ProcSpeed, NumProc, NumCores, + Memory, + str_num_schedulers()]), + CPUFactor = analyze_darwin_cpu_to_factor(ProcName, + ProcSpeed, + NumProc, + NumCores), + MemFactor = analyze_darwin_memory_to_factor(Memory), + if (MemFactor =:= 1) -> + {CPUFactor, []}; + true -> + {CPUFactor + MemFactor, []} + end + end. + +analyze_darwin_sw_system_version(SwInfo) -> + proplists:get_value("system version", SwInfo, "-"). + +analyze_darwin_sw_kernel_version(SwInfo) -> + proplists:get_value("kernel version", SwInfo, "-"). + +analyze_darwin_software_info() -> + analyze_darwin_system_profiler("SPSoftwareDataType"). + +analyze_darwin_hw_model_name(HwInfo) -> + proplists:get_value("model name", HwInfo, "-"). + +analyze_darwin_hw_model_identifier(HwInfo) -> + proplists:get_value("model identifier", HwInfo, "-"). + +analyze_darwin_hw_processor_name(HwInfo) -> + proplists:get_value("processor name", HwInfo, "-"). + +analyze_darwin_hw_processor_speed(HwInfo) -> + proplists:get_value("processor speed", HwInfo, "-"). + +analyze_darwin_hw_number_of_processors(HwInfo) -> + proplists:get_value("number of processors", HwInfo, "-"). + +analyze_darwin_hw_total_number_of_cores(HwInfo) -> + proplists:get_value("total number of cores", HwInfo, "-"). + +analyze_darwin_hw_memory(HwInfo) -> + proplists:get_value("memory", HwInfo, "-"). + +analyze_darwin_hardware_info() -> + analyze_darwin_system_profiler("SPHardwareDataType"). + +%% This basically has the structure: "Key: Value" +%% But could also be (for example): +%% "Something:" (which we ignore) +%% "Key: Value1:Value2" +analyze_darwin_system_profiler(DataType) -> + %% First, make sure the program actually exist: + case os:cmd("which system_profiler") of + [] -> + []; + _ -> + D0 = os:cmd("system_profiler " ++ DataType), + D1 = string:tokens(D0, [$\n]), + D2 = [string:trim(S1) || S1 <- D1], + D3 = [string:tokens(S2, [$:]) || S2 <- D2], + analyze_darwin_system_profiler2(D3) + end. + +analyze_darwin_system_profiler2(L) -> + analyze_darwin_system_profiler2(L, []). + +analyze_darwin_system_profiler2([], Acc) -> + [{string:to_lower(K), V} || {K, V} <- lists:reverse(Acc)]; +analyze_darwin_system_profiler2([[_]|T], Acc) -> + analyze_darwin_system_profiler2(T, Acc); +analyze_darwin_system_profiler2([[H1,H2]|T], Acc) -> + analyze_darwin_system_profiler2(T, [{H1, string:trim(H2)}|Acc]); +analyze_darwin_system_profiler2([[H|TH0]|T], Acc) -> + %% Some value parts has ':' in them, so put them together + TH1 = colonize(TH0), + analyze_darwin_system_profiler2(T, [{H, string:trim(TH1)}|Acc]). + +%% This is only called if the length is at least 2 +colonize([L1, L2]) -> + L1 ++ ":" ++ L2; +colonize([H|T]) -> + H ++ ":" ++ colonize(T). + + +%% The memory looks like this "<size> <unit>". Example: "2 GB" +analyze_darwin_memory_to_factor(Mem) -> + case [string:to_lower(S) || S <- string:tokens(Mem, [$\ ])] of + [_SzStr, "tb"] -> + 1; + [SzStr, "gb"] -> + try list_to_integer(SzStr) of + Sz when Sz < 2 -> + 20; + Sz when Sz < 4 -> + 10; + Sz when Sz < 8 -> + 5; + Sz when Sz < 16 -> + 2; + _ -> + 1 + catch + _:_:_ -> + 20 + end; + [_SzStr, "mb"] -> + 20; + _ -> + 20 + end. + + +%% The speed is a string: "<speed> <unit>" +%% the speed may be a float, which we transforms into an integer of MHz. +%% To calculate a factor based on processor speed, number of procs +%% and number of cores is ... not an exact ... science ... +analyze_darwin_cpu_to_factor(_ProcName, + ProcSpeedStr, NumProcStr, NumCoresStr) -> + Speed = + case [string:to_lower(S) || S <- string:tokens(ProcSpeedStr, [$\ ])] of + [SpeedStr, "mhz"] -> + try list_to_integer(SpeedStr) of + SpeedI -> + SpeedI + catch + _:_:_ -> + try list_to_float(SpeedStr) of + SpeedF -> + trunc(SpeedF) + catch + _:_:_ -> + -1 + end + end; + [SpeedStr, "ghz"] -> + try list_to_float(SpeedStr) of + SpeedF -> + trunc(1000*SpeedF) + catch + _:_:_ -> + try list_to_integer(SpeedStr) of + SpeedI -> + 1000*SpeedI + catch + _:_:_ -> + -1 + end + end; + _ -> + -1 + end, + NumProc = try list_to_integer(NumProcStr) of + NumProcI -> + NumProcI + catch + _:_:_ -> + 1 + end, + NumCores = try list_to_integer(NumCoresStr) of + NumCoresI -> + NumCoresI + catch + _:_:_ -> + 1 + end, + if + (Speed > 3000) -> + if + (NumProc =:= 1) -> + if + (NumCores < 2) -> + 5; + (NumCores < 4) -> + 3; + (NumCores < 6) -> + 2; + true -> + 1 + end; + true -> + if + (NumCores < 4) -> + 2; + true -> + 1 + end + end; + (Speed > 2000) -> + if + (NumProc =:= 1) -> + if + (NumCores < 2) -> + 8; + (NumCores < 4) -> + 5; + (NumCores < 6) -> + 3; + true -> + 1 + end; + true -> + if + (NumCores < 4) -> + 5; + (NumCores < 8) -> + 2; + true -> + 1 + end + end; + true -> + if + (NumProc =:= 1) -> + if + (NumCores < 2) -> + 10; + (NumCores < 4) -> + 7; + (NumCores < 6) -> + 5; + (NumCores < 8) -> + 3; + true -> + 1 + end; + true -> + if + (NumCores < 4) -> + 8; + (NumCores < 8) -> + 4; + true -> + 1 + end + end + end. + + analyze_and_print_solaris_host_info(Version) -> Release = case file:read_file_info("/etc/release") of @@ -1231,19 +1733,19 @@ analyze_and_print_solaris_host_info(Version) -> _:_:_ -> 10 end, - try erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - N when (N =< 6) -> - 2; - _ -> - 1 - catch - _:_:_ -> - 10 - end + MemFactor. + {try erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + N when (N =< 6) -> + 2; + _ -> + 1 + catch + _:_:_ -> + 10 + end + MemFactor, []}. analyze_and_print_win_host_info(Version) -> @@ -1315,7 +1817,7 @@ analyze_and_print_win_host_info(Version) -> _ -> 2 end, - CPUFactor + MemFactor. + {CPUFactor + MemFactor, SysInfo}. win_sys_info_lookup(Key, SysInfo) -> win_sys_info_lookup(Key, SysInfo, "-"). @@ -1330,14 +1832,25 @@ win_sys_info_lookup(Key, SysInfo, Def) -> %% This function only extracts the prop we actually care about! which_win_system_info() -> - SysInfo = os:cmd("systeminfo"), - try process_win_system_info(string:tokens(SysInfo, [$\r, $\n]), []) - catch - _:_:_ -> - io:format("Failed process System info: " - "~s~n", [SysInfo]), - [] - end. + F = fun() -> + try + begin + SysInfo = os:cmd("systeminfo"), + process_win_system_info( + string:tokens(SysInfo, [$\r, $\n]), []) + end + catch + C:E:S -> + io:format("Failed get or process System info: " + " Error Class: ~p" + " Error: ~p" + " Stack: ~p" + "~n", [C, E, S]), + [] + end + end, + proxy_call(F, minutes(1), + fun() -> throw({skip, "System info timeout"}) end). process_win_system_info([], Acc) -> Acc; @@ -1373,6 +1886,22 @@ process_win_system_info([H|T], Acc) -> end. +linux_info_lookup(Key, File) -> + try [string:trim(S) || S <- string:tokens(os:cmd("grep " ++ "\"" ++ Key ++ "\"" ++ " " ++ File), [$:,$\n])] of + Info -> + linux_info_lookup_collect(Key, Info, []) + catch + _:_:_ -> + "-" + end. + +linux_info_lookup_collect(_Key, [], Values) -> + lists:reverse(Values); +linux_info_lookup_collect(Key, [Key, Value|Rest], Values) -> + linux_info_lookup_collect(Key, Rest, [Value|Values]); +linux_info_lookup_collect(_, _, Values) -> + lists:reverse(Values). + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Set kill timer diff --git a/lib/megaco/test/megaco_test_lib.hrl b/lib/megaco/test/megaco_test_lib.hrl index f6af199d4e..0777adbcd0 100644 --- a/lib/megaco/test/megaco_test_lib.hrl +++ b/lib/megaco/test/megaco_test_lib.hrl @@ -29,6 +29,8 @@ -define(LIB, megaco_test_lib). +-define(PCALL(F, T, D), ?LIB:proxy_call(F, T, D)). + -define(APPLY(Proxy, Fun), Proxy ! {apply, Fun}). diff --git a/lib/megaco/test/megaco_udp_SUITE.erl b/lib/megaco/test/megaco_udp_SUITE.erl index 05910e50a9..42d0060fcf 100644 --- a/lib/megaco/test/megaco_udp_SUITE.erl +++ b/lib/megaco/test/megaco_udp_SUITE.erl @@ -27,6 +27,7 @@ %%---------------------------------------------------------------------- %% Include files %%---------------------------------------------------------------------- +-include_lib("common_test/include/ct.hrl"). -include_lib("megaco/src/udp/megaco_udp.hrl"). -include("megaco_test_lib.hrl"). @@ -239,6 +240,8 @@ start_and_stop(doc) -> ["This test case sets up a connection and then cloises it. " "No data is sent. "]; start_and_stop(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), + ct:timetrap(Factor * ?SECS(45)), Pre = fun() -> p("create nodes"), ServerNode = make_node_name(server), @@ -247,20 +250,22 @@ start_and_stop(Config) when is_list(Config) -> ok = ?START_NODES(Nodes), Nodes end, - Case = fun do_start_and_stop/1, + Case = fun(X) -> do_start_and_stop(Factor, X) end, Post = fun(Nodes) -> p("stop nodes"), ?STOP_NODES(lists:reverse(Nodes)) end, try_tc(start_and_stop, Pre, Case, Post). -do_start_and_stop([ServerNode, ClientNode]) -> +do_start_and_stop(Factor, [ServerNode, ClientNode]) -> %% Create command sequences + TOCalc = fun(BaseTO) -> to_calc(Factor, BaseTO) end, + TO = TOCalc(?SECS(5)), p("create command sequences"), ServerPort = 2944, ServerCmds = start_and_stop_server_commands(ServerPort), {ok, ServerHost} = inet:gethostname(), - ClientCmds = start_and_stop_client_commands(ServerPort, ServerHost), + ClientCmds = start_and_stop_client_commands(TO, ServerPort, ServerHost), %% Start the test procs used in the test-case, one for each node p("start command handlers"), @@ -268,8 +273,8 @@ do_start_and_stop([ServerNode, ClientNode]) -> p("server command handler started: ~p", [Server]), Client = client_start_command_handler(ClientNode, ClientCmds), p("client command handler started: ~p", [Client]), - - ok = + + ok = receive {operational, Server} -> p("received listening message from server [~p] => " @@ -280,11 +285,11 @@ do_start_and_stop([ServerNode, ClientNode]) -> ?SKIP(Reason); {'EXIT', Client, {skip, Reason}} -> ?SKIP(Reason) - after 5000 -> + after TO -> {error, server_timeout} end, - ok = await_command_handler_completion([Server, Client], ?SECS(20)), + ok = await_command_handler_completion([Server, Client], TOCalc(?SECS(20))), p("done"), ok. @@ -337,7 +342,7 @@ start_and_stop_server_commands(Port) -> ]. -start_and_stop_client_commands(ServerPort, _ServerHost) -> +start_and_stop_client_commands(TO, ServerPort, _ServerHost) -> Opts = [{port, ServerPort}], Self = self(), [ @@ -362,7 +367,7 @@ start_and_stop_client_commands(ServerPort, _ServerHost) -> #{id => 4, desc => "Await continue", cmd => fun(State) -> - client_await_continue_signal(State, 5000) + client_await_continue_signal(State, TO) end}, #{id => 5, @@ -399,6 +404,8 @@ sendreceive(suite) -> sendreceive(doc) -> ["Test send and receive with the UDP transport. "]; sendreceive(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), + ct:timetrap(Factor * ?SECS(30)), Pre = fun() -> p("create nodes"), ServerNode = make_node_name(server), @@ -407,20 +414,22 @@ sendreceive(Config) when is_list(Config) -> ok = ?START_NODES(Nodes), Nodes end, - Case = fun do_sendreceive/1, + Case = fun(X) -> do_sendreceive(Factor, X) end, Post = fun(Nodes) -> p("stop nodes"), ?STOP_NODES(lists:reverse(Nodes)) end, try_tc(sendreceive, Pre, Case, Post). -do_sendreceive([ServerNode, ClientNode]) -> +do_sendreceive(Factor, [ServerNode, ClientNode]) -> %% Create command sequences p("create command sequences"), + TOCalc = fun(BaseTO) -> to_calc(Factor, BaseTO) end, + TO = TOCalc(?SECS(5)), ServerPort = 2944, - ServerCmds = sendreceive_server_commands(ServerPort), + ServerCmds = sendreceive_server_commands(TO, ServerPort), {ok, ServerHost} = inet:gethostname(), - ClientCmds = sendreceive_client_commands(ServerPort, ServerHost), + ClientCmds = sendreceive_client_commands(TO, ServerPort, ServerHost), %% Start the test procs used in the test-case, one for each node p("start command handlers"), @@ -440,16 +449,16 @@ do_sendreceive([ServerNode, ClientNode]) -> ?SKIP(Reason); {'EXIT', Client, {skip, Reason}} -> ?SKIP(Reason) - after 5000 -> + after TO -> {error, server_timeout} end, - ok = await_command_handler_completion([Server, Client], ?SECS(20)), + ok = await_command_handler_completion([Server, Client], TOCalc(?SECS(20))), p("done"), ok. -sendreceive_server_commands(Port) -> +sendreceive_server_commands(TO, Port) -> Opts = [{port, Port}], Self = self(), [ @@ -480,7 +489,7 @@ sendreceive_server_commands(Port) -> #{id => 5, desc => "Await initial message (ping)", cmd => fun(State) -> - server_await_initial_message(State, "ping", 5000) + server_await_initial_message(State, "ping", TO) end}, #{id => 6, @@ -492,7 +501,7 @@ sendreceive_server_commands(Port) -> #{id => 7, desc => "Await nothing before sending a message (hejsan)", cmd => fun(State) -> - server_await_nothing(State, 1000) + server_await_nothing(State, TO div 5) end}, #{id => 8, @@ -504,13 +513,13 @@ sendreceive_server_commands(Port) -> #{id => 9, desc => "Await reply (hoppsan) to message", cmd => fun(State) -> - server_await_message(State, "hoppsan", 1000) + server_await_message(State, "hoppsan", TO div 5) end}, #{id => 10, desc => "Await nothing before closing", cmd => fun(State) -> - server_await_nothing(State, 1000) + server_await_nothing(State, TO div 5) end}, #{id => 11, @@ -522,7 +531,7 @@ sendreceive_server_commands(Port) -> #{id => 12, desc => "Await nothing before stopping transport", cmd => fun(State) -> - server_await_nothing(State, 1000) + server_await_nothing(State, TO div 5) end}, #{id => 13, @@ -532,7 +541,7 @@ sendreceive_server_commands(Port) -> end} ]. -sendreceive_client_commands(ServerPort, ServerHost) -> +sendreceive_client_commands(TO, ServerPort, ServerHost) -> OwnPort = ServerPort+1, Opts = [{port, OwnPort}], Self = self(), @@ -558,7 +567,7 @@ sendreceive_client_commands(ServerPort, ServerHost) -> #{id => 4, desc => "Await continue", cmd => fun(State) -> - client_await_continue_signal(State, 5000) + client_await_continue_signal(State, TO) end}, #{id => 5, @@ -576,13 +585,13 @@ sendreceive_client_commands(ServerPort, ServerHost) -> #{id => 7, desc => "Await reply (pong) to initial message", cmd => fun(State) -> - client_await_message(State, "pong", 1000) + client_await_message(State, "pong", TO div 5) end}, #{id => 8, desc => "Await message (hejsan)", cmd => fun(State) -> - client_await_message(State, "hejsan", 5000) + client_await_message(State, "hejsan", TO) end}, #{id => 9, @@ -594,7 +603,7 @@ sendreceive_client_commands(ServerPort, ServerHost) -> #{id => 10, desc => "Await nothing before closing", cmd => fun(State) -> - client_await_nothing(State, 1000) + client_await_nothing(State, TO div 5) end}, #{id => 11, @@ -606,7 +615,7 @@ sendreceive_client_commands(ServerPort, ServerHost) -> #{id => 12, desc => "Await nothing before stopping transport", cmd => fun(State) -> - client_await_nothing(State, 1000) + client_await_nothing(State, TO div 5) end}, #{id => 13, @@ -624,6 +633,8 @@ block_unblock(suite) -> block_unblock(doc) -> ["Test the block/unblock functions of the UDP transport. "]; block_unblock(Config) when is_list(Config) -> + Factor = ?config(megaco_factor, Config), + ct:timetrap(Factor * ?MINS(1)), Pre = fun() -> p("create nodes"), ServerNode = make_node_name(server), @@ -632,20 +643,22 @@ block_unblock(Config) when is_list(Config) -> ok = ?START_NODES(Nodes), Nodes end, - Case = fun do_block_unblock/1, + Case = fun(X) -> do_block_unblock(Factor, X) end, Post = fun(Nodes) -> p("stop nodes"), ?STOP_NODES(lists:reverse(Nodes)) end, try_tc(block_unblock, Pre, Case, Post). -do_block_unblock([ServerNode, ClientNode]) -> +do_block_unblock(Factor, [ServerNode, ClientNode]) -> %% Create command sequences p("create command sequences"), + TOCalc = fun(BaseTO) -> to_calc(Factor, BaseTO) end, + TO = TOCalc(?SECS(5)), ServerPort = 2944, - ServerCmds = block_unblock_server_commands(ServerPort), + ServerCmds = block_unblock_server_commands(TO, ServerPort), {ok, ServerHost} = inet:gethostname(), - ClientCmds = block_unblock_client_commands(ServerPort, ServerHost), + ClientCmds = block_unblock_client_commands(TO, ServerPort, ServerHost), %% Start the test procs used in the test-case, one for each node p("start command handlers"), @@ -667,7 +680,7 @@ do_block_unblock([ServerNode, ClientNode]) -> ?SKIP(Reason1); {'EXIT', Client, {skip, Reason2}} -> ?SKIP(Reason2) - after 5000 -> + after TO -> {error, server_timeout} end, @@ -684,16 +697,16 @@ do_block_unblock([ServerNode, ClientNode]) -> ?SKIP(Reason3); {'EXIT', Client, {skip, Reason4}} -> ?SKIP(Reason4) - after 5000 -> + after TO -> {error, timeout} end, - ok = await_command_handler_completion([Server, Client], ?SECS(20)), + ok = await_command_handler_completion([Server, Client], TOCalc(?SECS(20))), p("done"), ok. -block_unblock_server_commands(Port) -> +block_unblock_server_commands(TO, Port) -> Opts = [{port, Port}], Self = self(), [ @@ -724,7 +737,7 @@ block_unblock_server_commands(Port) -> #{id => 5, desc => "Await initial message (ping)", cmd => fun(State) -> - server_await_initial_message(State, "ping", 5000) + server_await_initial_message(State, "ping", TO) end}, #{id => 6, @@ -736,7 +749,7 @@ block_unblock_server_commands(Port) -> #{id => 7, desc => "Await continue", cmd => fun(State) -> - server_await_continue_signal(State, 5000) + server_await_continue_signal(State, TO) end}, #{id => 8, @@ -748,19 +761,19 @@ block_unblock_server_commands(Port) -> #{id => 9, desc => "Await nothing before receiving (hoppsan) reply", cmd => fun(State) -> - server_await_nothing(State, 4000) + server_await_nothing(State, TO) end}, #{id => 10, desc => "Await reply (hoppsan) to message", cmd => fun(State) -> - server_await_message(State, "hoppsan", 2000) + server_await_message(State, "hoppsan", TO div 2) end}, #{id => 11, desc => "Await nothing before closing", cmd => fun(State) -> - server_await_nothing(State, 1000) + server_await_nothing(State, TO div 5) end}, #{id => 12, @@ -772,7 +785,7 @@ block_unblock_server_commands(Port) -> #{id => 13, desc => "Await nothing before stopping transport", cmd => fun(State) -> - server_await_nothing(State, 1000) + server_await_nothing(State, TO div 5) end}, #{id => 14, @@ -783,7 +796,7 @@ block_unblock_server_commands(Port) -> ]. -block_unblock_client_commands(ServerPort, ServerHost) -> +block_unblock_client_commands(TO, ServerPort, ServerHost) -> OwnPort = ServerPort+1, Opts = [{port, OwnPort}], Self = self(), @@ -809,7 +822,7 @@ block_unblock_client_commands(ServerPort, ServerHost) -> #{id => 4, desc => "Await continue", cmd => fun(State) -> - client_await_continue_signal(State, 5000) + client_await_continue_signal(State, TO) end}, #{id => 5, @@ -827,7 +840,7 @@ block_unblock_client_commands(ServerPort, ServerHost) -> #{id => 7, desc => "Await reply (pong) to initial message", cmd => fun(State) -> - client_await_message(State, "pong", 1000) + client_await_message(State, "pong", TO div 5) end}, #{id => 8, @@ -845,7 +858,7 @@ block_unblock_client_commands(ServerPort, ServerHost) -> #{id => 10, desc => "Await nothing before unblocking", cmd => fun(State) -> - client_await_nothing(State, 5000) + client_await_nothing(State, TO) end}, #{id => 11, @@ -857,7 +870,7 @@ block_unblock_client_commands(ServerPort, ServerHost) -> #{id => 8, desc => "Await message (hejsan)", cmd => fun(State) -> - client_await_message(State, "hejsan", 5000) + client_await_message(State, "hejsan", TO) end}, #{id => 9, @@ -869,7 +882,7 @@ block_unblock_client_commands(ServerPort, ServerHost) -> #{id => 10, desc => "Await nothing before closing", cmd => fun(State) -> - client_await_nothing(State, 1000) + client_await_nothing(State, TO) end}, #{id => 11, @@ -881,7 +894,7 @@ block_unblock_client_commands(ServerPort, ServerHost) -> #{id => 12, desc => "Await nothing before stopping transport", cmd => fun(State) -> - client_await_nothing(State, 1000) + client_await_nothing(State, TO) end}, #{id => 13, @@ -974,14 +987,17 @@ server_start_transport(State) when is_map(State) -> server_open(#{transport_ref := Ref} = State, Options) when is_list(Options) -> Opts = [{receive_handle, self()}, {module, ?MODULE} | Options], - case (catch megaco_udp:open(Ref, Opts)) of + try megaco_udp:open(Ref, Opts) of {ok, Socket, ControlPid} -> {ok, State#{handle => {socket, Socket}, % Temporary control_pid => ControlPid}}; {error, {could_not_open_udp_port, eaddrinuse}} -> {skip, {server, eaddrinuse}}; - Error -> - Error + {error, _} = ERROR -> + ERROR + catch + C:E:S -> + {error, {catched, C, E, S}} end. server_notify_operational(#{parent := Parent} = State) -> @@ -1080,14 +1096,17 @@ client_start_transport(State) when is_map(State) -> client_open(#{transport_ref := Ref} = State, Options) when is_list(Options) -> Opts = [{receive_handle, self()}, {module, ?MODULE} | Options], - case (catch megaco_udp:open(Ref, Opts)) of + try megaco_udp:open(Ref, Opts) of {ok, Socket, ControlPid} -> {ok, State#{handle => {socket, Socket}, control_pid => ControlPid}}; {error, {could_not_open_udp_port, eaddrinuse}} -> {skip, {client, eaddrinuse}}; - Error -> - Error + {error, _} = ERROR -> + ERROR + catch + C:E:S -> + {error, {catched, C, E, S}} end. client_await_continue_signal(#{parent := Parent} = State, Timeout) -> @@ -1196,6 +1215,13 @@ make_node_name(Name) -> end. +to_calc(1 = _Factor, BaseTO) when is_integer(BaseTO) andalso (BaseTO > 0) -> + BaseTO; +to_calc(Factor, BaseTO) when is_integer(Factor) andalso (Factor > 0) andalso + is_integer(BaseTO) andalso (BaseTO > 0) -> + trunc( ((Factor + 1) / 2) * BaseTO ). + + p(F) -> p(F, []). diff --git a/lib/megaco/vsn.mk b/lib/megaco/vsn.mk index e9c21389bc..3a35c5d125 100644 --- a/lib/megaco/vsn.mk +++ b/lib/megaco/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = megaco -MEGACO_VSN = 3.18.8 +MEGACO_VSN = 3.19 PRE_VSN = APP_VSN = "$(APPLICATION)-$(MEGACO_VSN)$(PRE_VSN)" diff --git a/lib/mnesia/doc/specs/.gitignore b/lib/mnesia/doc/specs/.gitignore new file mode 100644 index 0000000000..322eebcb06 --- /dev/null +++ b/lib/mnesia/doc/specs/.gitignore @@ -0,0 +1 @@ +specs_*.xml diff --git a/lib/mnesia/doc/src/Makefile b/lib/mnesia/doc/src/Makefile index f14fd33c7a..486993e76d 100644 --- a/lib/mnesia/doc/src/Makefile +++ b/lib/mnesia/doc/src/Makefile @@ -68,6 +68,7 @@ XML_GEN_FILES = $(XML_CHAPTER_GEN_FILES:%=$(XMLDIR)/%) IMAGE_FILES = \ company.gif -# ---------------------------------------------------- +SPECS_FILES = $(XML_REF3_FILES:%.xml=$(SPECDIR)/specs_%.xml) +TOP_SPECS_FILE = specs.xml include $(ERL_TOP)/make/doc.mk diff --git a/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc b/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc index a53bac020b..3c41945285 100644 --- a/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc +++ b/lib/mnesia/doc/src/Mnesia_chap2.xmlsrc @@ -298,7 +298,7 @@ <p>The function <c>insert_emp/3</c> creates a Functional Object (Fun). <c>Fun</c> is passed as a single argument to the function - <seemfa marker="mnesia#transaction/2">mnesia:transaction(Fun)</seemfa>. + <seemfa marker="mnesia#transaction/1">mnesia:transaction(Fun)</seemfa>. This means that <c>Fun</c> is run as a transaction with the following properties:</p> <list type="bulleted"> diff --git a/lib/mnesia/doc/src/Mnesia_chap4.xmlsrc b/lib/mnesia/doc/src/Mnesia_chap4.xmlsrc index c722646d33..335f09dd30 100644 --- a/lib/mnesia/doc/src/Mnesia_chap4.xmlsrc +++ b/lib/mnesia/doc/src/Mnesia_chap4.xmlsrc @@ -220,7 +220,7 @@ forced to release all its locks and sleep for a while. The Fun in the transaction is evaluated once more.</p> <p>It is therefore important that the code inside the Fun given to - <seemfa marker="mnesia#transaction/2"><c>mnesia:transaction/1</c></seemfa> + <seemfa marker="mnesia#transaction/1"><c>mnesia:transaction/1</c></seemfa> is pure. Some strange results can occur if, for example, messages are sent by the transaction Fun. The following example illustrates this situation:</p> @@ -252,7 +252,7 @@ transaction. If no enclosing transaction (or other enclosing <c>Mnesia</c> activity) exists, they all fail.</p> <list type="bulleted"> - <item><seemfa marker="mnesia#transaction/2">mnesia:transaction(Fun) -> {aborted, Reason} |{atomic, Value}</seemfa> + <item><seemfa marker="mnesia#transaction/1">mnesia:transaction(Fun) -> {aborted, Reason} |{atomic, Value}</seemfa> executes one transaction with the functional object <c>Fun</c> as the single parameter. </item> @@ -498,14 +498,6 @@ <c>mnesia:dirty_prev/2</c> are synonyms. </item> <item> - <p><seemfa marker="mnesia#dirty_slot/2">mnesia:dirty_slot(Tab, Slot)</seemfa> - returns the list of records that are associated with <c>Slot</c> - in a table. It can be used to traverse a table in a manner - similar to the function <c>dirty_next/2</c>. A table has a - number of slots that range from zero to some unknown upper - bound. The function <c>dirty_slot/2</c> returns the special - atom <c>'$end_of_table'</c> when the end of the table is - reached.</p> <p>The behavior of this function is undefined if the table is written on while being traversed. The function @@ -664,7 +656,7 @@ mnesia:all_keys/1</seemfa>. <p>As previously described, a Functional Object (Fun) performing table access operations, as listed here, can be passed on as arguments to the function - <seemfa marker="mnesia#transaction/2">mnesia:transaction/1,2,3</seemfa>: + <seemfa marker="mnesia#transaction/1">mnesia:transaction/1,2,3</seemfa>: </p> <list type="bulleted"> <item> @@ -732,7 +724,7 @@ mnesia:all_keys/1</seemfa>. <item><c>ets</c></item> </list> <p>By passing the same "fun" as argument to the function - <seemfa marker="mnesia#sync_transaction/3">mnesia:sync_transaction(Fun [, Args])</seemfa> + <seemfa marker="mnesia#sync_transaction/1">mnesia:sync_transaction(Fun [, Args])</seemfa> it is performed in synced transaction context. Synced transactions wait until all active replicas has committed the transaction (to disc) before diff --git a/lib/mnesia/doc/src/mnesia.xml b/lib/mnesia/doc/src/mnesia.xml index 9097724012..1932b7695e 100644 --- a/lib/mnesia/doc/src/mnesia.xml +++ b/lib/mnesia/doc/src/mnesia.xml @@ -11,7 +11,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - + http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software @@ -181,9 +181,67 @@ transaction before iterating over the table.</p> </description> + <datatypes> + <datatype> + <name name="table"/> + </datatype> + <datatype> + <name name="activity"/> + </datatype> + <datatype> + <name name="create_option"/> + </datatype> + <datatype> + <name name="storage_type"/> + </datatype> + <datatype> + <name name="t_result" n_vars="1"/> + </datatype> + <datatype> + <name name="result"/> + </datatype> + <datatype> + <name name="index_attr"/> + </datatype> + <datatype> + <name name="write_locks"/> + </datatype> + <datatype> + <name name="read_locks"/> + </datatype> + <datatype> + <name name="lock_kind"/> + </datatype> + <datatype> + <name name="select_continuation"/> + </datatype> + <datatype> + <name name="snmp_struct"/> + </datatype> + <datatype> + <name name="snmp_type"/> + </datatype> + <datatype> + <name name="tuple_of" n_vars="1"/> + </datatype> + <datatype> + <name name="config_key"/> + </datatype> + <datatype> + <name name="config_value"/> + </datatype> + <datatype> + <name name="config_result"/> + </datatype> + <datatype> + <name name="debug_level"/> + </datatype> + + </datatypes> + <funcs> <func> - <name since="">abort(Reason) -> transaction abort</name> + <name name="abort" arity="1" since=""/> <fsummary>Terminates the current transaction.</fsummary> <desc> <p>Makes the transaction silently @@ -195,7 +253,7 @@ </desc> </func> <func> - <name since="">activate_checkpoint(Args) -> {ok,Name,Nodes} | {error,Reason}</name> + <name name="activate_checkpoint" arity="1" since=""/> <fsummary>Activates a checkpoint.</fsummary> <desc> <marker id="activate_checkpoint"></marker> @@ -259,7 +317,8 @@ </desc> </func> <func> - <name since="">activity(AccessContext, Fun [, Args]) -> ResultOfFun | exit(Reason)</name> + <name name="activity" arity="2" since=""/> + <!--<name name="activity" arity="3" since=""/>--> <fsummary>Executes <c>Fun</c> in <c>AccessContext</c>.</fsummary> <desc> <marker id="activity_2_3"></marker> @@ -271,7 +330,7 @@ </desc> </func> <func> - <name since="">activity(AccessContext, Fun, Args, AccessMod) -> ResultOfFun | exit(Reason)</name> + <name name="activity" arity="4" since=""/> <fsummary>Executes <c>Fun</c> in <c>AccessContext</c>.</fsummary> <desc> <marker id="activity_4"></marker> @@ -403,7 +462,7 @@ </desc> </func> <func> - <name since="">add_table_copy(Tab, Node, Type) -> {aborted, R} | {atomic, ok}</name> + <name name="add_table_copy" arity="3" since=""/> <fsummary>Copies a table to a remote node.</fsummary> <desc> <marker id="add_table_copy"></marker> @@ -420,7 +479,7 @@ mnesia:add_table_copy(person, Node, disc_copies)</code> </desc> </func> <func> - <name since="">add_table_index(Tab, AttrName) -> {aborted, R} | {atomic, ok}</name> + <name name="add_table_index" arity="2" since=""/> <fsummary>Creates an index for a table.</fsummary> <desc> <marker id="add_table_index"></marker> @@ -441,7 +500,7 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">all_keys(Tab) -> KeyList | transaction abort</name> + <name name="all_keys" arity="1" since=""/> <fsummary>Returns all keys in a table.</fsummary> <desc> <marker id="all_keys"></marker> @@ -453,7 +512,8 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">async_dirty(Fun [, Args]) -> ResultOfFun | exit(Reason)</name> + <name name="async_dirty" arity="1" since=""/> + <name name="async_dirty" arity="2" since=""/> <fsummary>Calls the <c>Fun</c> in a context that is not protected by a transaction.</fsummary> <desc> <marker id="async_dirty"></marker> @@ -493,7 +553,8 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">backup(Opaque [, BackupMod]) -> ok | {error,Reason}</name> + <name name="backup" arity="1" since=""/> + <name name="backup" arity="2" since=""/> <fsummary>Backs up all tables in the database.</fsummary> <desc> <marker id="backup"></marker> @@ -505,7 +566,8 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">backup_checkpoint(Name, Opaque [, BackupMod]) -> ok | {error,Reason}</name> + <name name="backup_checkpoint" arity="2" since=""/> + <name name="backup_checkpoint" arity="3" since=""/> <fsummary>Backs up all tables in a checkpoint.</fsummary> <desc> <marker id="backup_checkpoint"></marker> @@ -520,7 +582,7 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">change_config(Config, Value) -> {error, Reason} | {ok, ReturnValue}</name> + <name name="change_config" arity="2" since=""/> <fsummary>Changes a configuration parameter.</fsummary> <desc> <marker id="change_config"></marker> @@ -554,7 +616,7 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">change_table_access_mode(Tab, AccessMode) -> {aborted, R} | {atomic, ok}</name> + <name name="change_table_access_mode" arity="2" since=""/> <fsummary>Changes the access mode for the table.</fsummary> <desc> <marker id="change_table_access_mode"></marker> @@ -568,7 +630,7 @@ mnesia:add_table_index(person, age)</code> </desc> </func> <func> - <name since="">change_table_copy_type(Tab, Node, To) -> {aborted, R} | {atomic, ok}</name> + <name name="change_table_copy_type" arity="3" since=""/> <fsummary>Changes the storage type of a table.</fsummary> <desc> <marker id="change_table_copy_type"></marker> @@ -585,7 +647,7 @@ mnesia:change_table_copy_type(person, node(), disc_copies)</code> </desc> </func> <func> - <name since="">change_table_load_order(Tab, LoadOrder) -> {aborted, R} | {atomic, ok}</name> + <name name="change_table_load_order" arity="2" since=""/> <fsummary>Changes the load order priority for the table.</fsummary> <desc> <marker id="change_table_load_order"></marker> @@ -595,7 +657,7 @@ mnesia:change_table_copy_type(person, node(), disc_copies)</code> </desc> </func> <func> - <name since="OTP R14B03">change_table_majority(Tab, Majority) -> {aborted, R} | {atomic, ok}</name> + <name name="change_table_majority" arity="2" since="OTP R14B03"/> <fsummary>Changes the majority check setting for the table.</fsummary> <desc> <p><c>Majority</c> must be a boolean. Default is <c>false</c>. @@ -607,7 +669,7 @@ mnesia:change_table_copy_type(person, node(), disc_copies)</code> </desc> </func> <func> - <name since="">clear_table(Tab) -> {aborted, R} | {atomic, ok}</name> + <name name="clear_table" arity="1" since=""/> <fsummary>Deletes all entries in a table.</fsummary> <desc> <marker id="clear_table"></marker> @@ -615,7 +677,7 @@ mnesia:change_table_copy_type(person, node(), disc_copies)</code> </desc> </func> <func> - <name since="">create_schema(DiscNodes) -> ok | {error,Reason}</name> + <name name="create_schema" arity="1" since=""/> <fsummary>Creates a new schema on the specified nodes.</fsummary> <desc> <marker id="create_schema"></marker> @@ -637,7 +699,7 @@ mnesia:change_table_copy_type(person, node(), disc_copies)</code> </desc> </func> <func> - <name since="">create_table(Name, TabDef) -> {atomic, ok} | {aborted, Reason}</name> + <name name="create_table" arity="2" since=""/> <fsummary>Creates a Mnesia table called <c>Name</c>with properties as described by argument <c>TabDef</c>.</fsummary> <desc> <marker id="create_table"></marker> @@ -799,7 +861,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">deactivate_checkpoint(Name) -> ok | {error, Reason}</name> + <name name="deactivate_checkpoint" arity="1" since=""/> <fsummary>Deactivates a checkpoint.</fsummary> <desc> <marker id="deactivate_checkpoint"></marker> @@ -811,7 +873,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">del_table_copy(Tab, Node) -> {aborted, R} | {atomic, ok}</name> + <name name="del_table_copy" arity="2" since=""/> <fsummary>Deletes the replica of table <c>Tab</c> at node <c>Node</c>.</fsummary> <desc> <marker id="del_table_copy"></marker> @@ -825,7 +887,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">del_table_index(Tab, AttrName) -> {aborted, R} | {atomic, ok}</name> + <name name="del_table_index" arity="2" since=""/> <fsummary>Deletes an index in a table.</fsummary> <desc> <marker id="del_table_index"></marker> @@ -834,16 +896,16 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">delete({Tab, Key}) -> transaction abort | ok</name> + <name name="delete" arity="1" since=""/> <fsummary>Deletes all records in table <c>Tab</c> with the key <c>Key</c>.</fsummary> <desc> - <marker id="delete_2"></marker> + <marker id="delete_1"></marker> <p>Calls <c>mnesia:delete(Tab, Key, write)</c>.</p> </desc> </func> <func> - <name since="">delete(Tab, Key, LockKind) -> transaction abort | ok</name> - <fsummary>Deletes all records in table <c>Tab</c>with the key <c>Key</c>.</fsummary> + <name name="delete" arity="3" since=""/> + <fsummary>Deletes all records in table <c>Tab</c> with the key <c>Key</c>.</fsummary> <desc> <marker id="delete_3"></marker> <p>Deletes all records in table <c>Tab</c> with the key @@ -857,7 +919,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">delete_object(Record) -> transaction abort | ok</name> + <name name="delete_object" arity="1" since=""/> <fsummary>Delete a record.</fsummary> <desc> <marker id="delete_object_1"></marker> @@ -866,7 +928,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">delete_object(Tab, Record, LockKind) -> transaction abort | ok</name> + <name name="delete_object" arity="3" since=""/> <fsummary>Deletes a record.</fsummary> <desc> <marker id="delete_object_3"></marker> @@ -883,7 +945,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">delete_schema(DiscNodes) -> ok | {error,Reason}</name> + <name name="delete_schema" arity="1" since=""/> <fsummary>Deletes the schema on the given nodes.</fsummary> <desc> <marker id="delete_schema"></marker> @@ -904,7 +966,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">delete_table(Tab) -> {aborted, Reason} | {atomic, ok}</name> + <name name="delete_table" arity="1" since=""/> <fsummary>Deletes permanently all replicas of table <c>Tab</c>.</fsummary> <desc> <marker id="delete_table"></marker> @@ -912,7 +974,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_all_keys(Tab) -> KeyList | exit({aborted, Reason})</name> + <name name="dirty_all_keys" arity="1" since=""/> <fsummary>Dirty search for all record keys in table.</fsummary> <desc> <marker id="delete_all_keys"></marker> @@ -920,7 +982,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_delete({Tab, Key}) -> ok | exit({aborted, Reason})</name> + <name name="dirty_delete" arity="1" since=""/> <fsummary>Dirty delete of a record.</fsummary> <desc> <marker id="dirty_delete"></marker> @@ -928,14 +990,14 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_delete(Tab, Key) -> ok | exit({aborted, Reason})</name> + <name name="dirty_delete" arity="2" since=""/> <fsummary>Dirty delete of a record.</fsummary> <desc> <p>Dirty equivalent of the function <c>mnesia:delete/3</c>.</p> </desc> </func> <func> - <name since="">dirty_delete_object(Record)</name> + <name name="dirty_delete_object" arity="1" since=""/> <fsummary>Dirty delete of a record.</fsummary> <desc> <marker id="dirty_delete_object_1"></marker> @@ -944,18 +1006,18 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_delete_object(Tab, Record)</name> + <name name="dirty_delete_object" arity="2" since=""/> <fsummary>Dirty delete of a record.</fsummary> <desc> <p>Dirty equivalent of the function <c>mnesia:delete_object/3</c>.</p> </desc> </func> <func> - <name since="">dirty_first(Tab) -> Key | exit({aborted, Reason})</name> + <name name="dirty_first" arity="1" since=""/> <fsummary>Returns the key for the first record in a table.</fsummary> <desc> <marker id="dirty_first"></marker> - <p>Records in <c>set</c> or <c>bag</c> tables are not ordered. + <p>Records in <c>set</c> or <c>bag</c> tables are not ordered. However, there is an ordering of the records that is unknown to the user. Therefore, a table can be traversed by this function with the function <c>mnesia:dirty_next/2</c>. @@ -967,7 +1029,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_index_match_object(Pattern, Pos)</name> + <name name="dirty_index_match_object" arity="2" since=""/> <fsummary>Dirty pattern match using index.</fsummary> <desc> <marker id="dirty_index_match_object_2"></marker> @@ -977,7 +1039,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_index_match_object(Tab, Pattern, Pos)</name> + <name name="dirty_index_match_object" arity="3" since=""/> <fsummary>Dirty pattern match using index.</fsummary> <desc> <p>Dirty equivalent of the function @@ -985,7 +1047,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_index_read(Tab, SecondaryKey, Pos)</name> + <name name="dirty_index_read" arity="3" since=""/> <fsummary>Dirty read using index.</fsummary> <desc> <marker id="dirty_index_read"></marker> @@ -994,7 +1056,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_last(Tab) -> Key | exit({aborted, Reason})</name> + <name name="dirty_last" arity="1" since=""/> <fsummary>Returns the key for the last record in a table.</fsummary> <desc> <marker id="dirty_last"></marker> @@ -1006,7 +1068,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_match_object(Pattern) -> RecordList | exit({aborted, Reason})</name> + <name name="dirty_match_object" arity="1" since=""/> <fsummary>Dirty pattern match pattern.</fsummary> <desc> <marker id="dirty_match_object_1"></marker> @@ -1015,7 +1077,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_match_object(Tab, Pattern) -> RecordList | exit({aborted, Reason})</name> + <name name="dirty_match_object" arity="2" since=""/> <fsummary>Dirty pattern match pattern.</fsummary> <desc> <p>Dirty equivalent of the function @@ -1023,7 +1085,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_next(Tab, Key) -> Key | exit({aborted, Reason})</name> + <name name="dirty_next" arity="2" since=""/> <fsummary>Return the next key in a table.</fsummary> <desc> <marker id="dirty_next"></marker> @@ -1038,7 +1100,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_prev(Tab, Key) -> Key | exit({aborted, Reason})</name> + <name name="dirty_prev" arity="2" since=""/> <fsummary>Returns the previous key in a table.</fsummary> <desc> <marker id="dirty_prev"></marker> @@ -1050,7 +1112,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_read({Tab, Key}) -> ValueList | exit({aborted, Reason}</name> + <name name="dirty_read" arity="1" since=""/> <fsummary>Dirty read of records.</fsummary> <desc> <marker id="dirty_read"></marker> @@ -1058,14 +1120,14 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_read(Tab, Key) -> ValueList | exit({aborted, Reason}</name> + <name name="dirty_read" arity="2" since=""/> <fsummary>Dirty read of records.</fsummary> <desc> <p>Dirty equivalent of the function <c>mnesia:read/3</c>.</p> </desc> </func> <func> - <name since="">dirty_select(Tab, MatchSpec) -> ValueList | exit({aborted, Reason}</name> + <name name="dirty_select" arity="2" since=""/> <fsummary>Dirty matches the objects in <c>Tab</c> against <c>MatchSpec</c>.</fsummary> <desc> <marker id="dirty_select"></marker> @@ -1073,23 +1135,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_slot(Tab, Slot) -> RecordList | exit({aborted, Reason})</name> - <fsummary>Returns the list of records that are associated with <c>Slot</c> in a table.</fsummary> - <desc> - <marker id="dirty_slot"></marker> - <p>Traverses a table in a - manner similar to the function <c>mnesia:dirty_next/2</c>. - A table has a number of slots that range from 0 (zero) to - an unknown upper bound. The function - <c>mnesia:dirty_slot/2</c> returns the special atom - <c>'$end_of_table'</c> when the end of the table is reached. - The behavior of this function is undefined if a write - operation is performed on the table while it is being - traversed.</p> - </desc> - </func> - <func> - <name since="">dirty_update_counter({Tab, Key}, Incr) -> NewVal | exit({aborted, Reason})</name> + <name name="dirty_update_counter" arity="2" since=""/> <fsummary>Dirty update of a counter record.</fsummary> <desc> <marker id="dirty_update_counter"></marker> @@ -1097,7 +1143,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_update_counter(Tab, Key, Incr) -> NewVal | exit({aborted, Reason})</name> + <name name="dirty_update_counter" arity="3" since=""/> <fsummary>Dirty update of a counter record.</fsummary> <desc> <p>Mnesia has no special counter records. However, @@ -1126,7 +1172,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_write(Record) -> ok | exit({aborted, Reason})</name> + <name name="dirty_write" arity="1" since=""/> <fsummary>Dirty write of a record.</fsummary> <desc> <marker id="dirty_write_1"></marker> @@ -1135,14 +1181,14 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dirty_write(Tab, Record) -> ok | exit({aborted, Reason})</name> + <name name="dirty_write" arity="2" since=""/> <fsummary>Dirty write of a record.</fsummary> <desc> <p>Dirty equivalent of the function <c>mnesia:write/3</c>.</p> </desc> </func> <func> - <name since="">dump_log() -> dumped</name> + <name name="dump_log" arity="0" since=""/> <fsummary>Performs a user-initiated dump of the local log file.</fsummary> <desc> <marker id="dump_log"></marker> @@ -1156,7 +1202,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dump_tables(TabList) -> {atomic, ok} | {aborted, Reason}</name> + <name name="dump_tables" arity="1" since=""/> <fsummary>Dumps all RAM tables to disc.</fsummary> <desc> <marker id="dump_tables"></marker> @@ -1168,7 +1214,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">dump_to_textfile(Filename)</name> + <name name="dump_to_textfile" arity="1" since=""/> <fsummary>Dumps local tables into a text file.</fsummary> <desc> <marker id="dump_to_textfile"></marker> @@ -1181,7 +1227,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">error_description(Error) -> String</name> + <name name="error_description" arity="1" since=""/> <fsummary>Returns a string describing a particular Mnesia error.</fsummary> <desc> <marker id="error_description"></marker> @@ -1259,7 +1305,8 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">ets(Fun [, Args]) -> ResultOfFun | exit(Reason)</name> + <name name="ets" arity="1" since=""/> + <name name="ets" arity="2" since=""/> <fsummary>Calls the <c>Fun</c> in a raw context that is not protected by a transaction.</fsummary> <desc> <marker id="ets"></marker> @@ -1278,7 +1325,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">first(Tab) -> Key | transaction abort</name> + <name name="first" arity="1" since=""/> <fsummary>Returns the key for the first record in a table.</fsummary> <desc> <marker id="first"></marker> @@ -1293,7 +1340,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">foldl(Function, Acc, Table) -> NewAcc | transaction abort</name> + <name name="foldl" arity="3" since=""/> <fsummary>Calls <c>Function</c> for each record in <c>Table</c>.</fsummary> <desc> <marker id="foldl"></marker> @@ -1306,7 +1353,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">foldr(Function, Acc, Table) -> NewAcc | transaction abort</name> + <name name="foldr" arity="3" since=""/> <fsummary>Calls <c>Function</c> for each record in <c>Table</c>.</fsummary> <desc> <marker id="foldr"></marker> @@ -1317,7 +1364,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">force_load_table(Tab) -> yes | ErrorDescription</name> + <name name="force_load_table" arity="1" since=""/> <fsummary>Forces a table to be loaded into the system.</fsummary> <desc> <marker id="force_load_table"></marker> @@ -1335,7 +1382,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">index_match_object(Pattern, Pos) -> transaction abort | ObjList</name> + <name name="index_match_object" arity="2" since=""/> <fsummary>Matches records and uses index information.</fsummary> <desc> <marker id="index_match_object_2"></marker> @@ -1345,7 +1392,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">index_match_object(Tab, Pattern, Pos, LockKind) -> transaction abort | ObjList</name> + <name name="index_match_object" arity="4" since=""/> <fsummary>Matches records and uses index information.</fsummary> <desc> <marker id="index_match_object_4"></marker> @@ -1377,7 +1424,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">index_read(Tab, SecondaryKey, Pos) -> transaction abort | RecordList</name> + <name name="index_read" arity="3" since=""/> <fsummary>Reads records through index table.</fsummary> <desc> <marker id="index_read"></marker> @@ -1397,7 +1444,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">info() -> ok</name> + <name name="info" arity="0" since=""/> <fsummary>Prints system information on the terminal.</fsummary> <desc> <marker id="info"></marker> @@ -1408,7 +1455,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">install_fallback(Opaque) -> ok | {error,Reason}</name> + <name name="install_fallback" arity="1" since=""/> <fsummary>Installs a backup as fallback.</fsummary> <desc> <marker id="install_fallback_1"></marker> @@ -1417,7 +1464,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">install_fallback(Opaque), BackupMod) -> ok | {error,Reason}</name> + <name name="install_fallback" arity="1" since=""/> <fsummary>Installs a backup as fallback.</fsummary> <desc> <p>Calls <c>mnesia:install_fallback(Opaque, Args)</c>, where @@ -1425,7 +1472,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">install_fallback(Opaque, Args) -> ok | {error,Reason}</name> + <name name="install_fallback" arity="2" since=""/> <fsummary>Installs a backup as fallback.</fsummary> <desc> <p>Installs a backup as fallback. The fallback is used to @@ -1483,7 +1530,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">is_transaction() -> boolean</name> + <name name="is_transaction" arity="0" since=""/> <fsummary>Checks if code is running in a transaction.</fsummary> <desc> <marker id="is_transaction"></marker> @@ -1492,7 +1539,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">last(Tab) -> Key | transaction abort</name> + <name name="last" arity="1" since=""/> <fsummary>Returns the key for the last record in a table.</fsummary> <desc> <p>Works exactly like @@ -1503,7 +1550,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">load_textfile(Filename)</name> + <name name="load_textfile" arity="1" since=""/> <fsummary>Loads tables from a text file.</fsummary> <desc> <marker id="load_textfile"></marker> @@ -1516,7 +1563,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">lock(LockItem, LockKind) -> Nodes | ok | transaction abort</name> + <name name="lock" arity="2" since=""/> <fsummary>Explicit grab lock.</fsummary> <desc> <marker id="lock"></marker> @@ -1605,7 +1652,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">match_object(Pattern) -> transaction abort | RecList</name> + <name name="match_object" arity="1" since=""/> <fsummary>Matches <c>Pattern</c> for records.</fsummary> <desc> <marker id="match_object_1"></marker> @@ -1614,7 +1661,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">match_object(Tab, Pattern, LockKind) -> transaction abort | RecList</name> + <name name="match_object" arity="3" since=""/> <fsummary>Matches <c>Pattern</c> for records.</fsummary> <desc> <marker id="match_object_3"></marker> @@ -1639,7 +1686,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">move_table_copy(Tab, From, To) -> {aborted, Reason} | {atomic, ok}</name> + <name name="move_table_copy" arity="3" since=""/> <fsummary>Moves the copy of table <c>Tab</c> from node <c>From</c> to node <c>To</c>.</fsummary> <desc> <marker id="move_table_copy"></marker> @@ -1653,7 +1700,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">next(Tab, Key) -> Key | transaction abort</name> + <name name="next" arity="2" since=""/> <fsummary>Returns the next key in a table.</fsummary> <desc> <marker id="next"></marker> @@ -1665,7 +1712,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">prev(Tab, Key) -> Key | transaction abort</name> + <name name="prev" arity="2" since=""/> <fsummary>Returns the previous key in a table.</fsummary> <desc> <p>Works exactly like @@ -1676,7 +1723,8 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">read({Tab, Key}) -> transaction abort | RecordList</name> + <name name="read" arity="1" since=""/> + <name name="read" arity="2" since=""/> <fsummary>Reads records(s) with a given key.</fsummary> <desc> <marker id="read_2"></marker> @@ -1684,14 +1732,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">read(Tab, Key) -> transaction abort | RecordList</name> - <fsummary>Reads records(s) with a given key.</fsummary> - <desc> - <p>Calls function <c>mnesia:read(Tab, Key, read)</c>.</p> - </desc> - </func> - <func> - <name since="">read(Tab, Key, LockKind) -> transaction abort | RecordList</name> + <name name="read" arity="3" since=""/> <fsummary>Reads records(s) with a given key.</fsummary> <desc> <marker id="read_3"></marker> @@ -1716,7 +1757,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">read_lock_table(Tab) -> ok | transaction abort</name> + <name name="read_lock_table" arity="1" since=""/> <fsummary>Sets a read lock on an entire table.</fsummary> <desc> <marker id="read_lock_table"></marker> @@ -1725,7 +1766,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">report_event(Event) -> ok</name> + <name name="report_event" arity="1" since=""/> <fsummary>Reports a user event to the Mnesia event handler.</fsummary> <desc> <marker id="report_event"></marker> @@ -1743,7 +1784,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">restore(Opaque, Args) -> {atomic, RestoredTabs} |{aborted, Reason}</name> + <name name="restore" arity="2" since=""/> <fsummary>Online restore of backup.</fsummary> <desc> <marker id="restore"></marker> @@ -1803,7 +1844,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">s_delete({Tab, Key}) -> ok | transaction abort</name> + <name name="s_delete" arity="1" since=""/> <fsummary>Sets sticky lock and delete records.</fsummary> <desc> <marker id="s_delete"></marker> @@ -1812,7 +1853,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">s_delete_object(Record) -> ok | transaction abort</name> + <name name="s_delete_object" arity="1" since=""/> <fsummary>Sets sticky lock and delete record.</fsummary> <desc> <marker id="s_delete_object"></marker> @@ -1822,7 +1863,7 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">s_write(Record) -> ok | transaction abort</name> + <name name="s_write" arity="1" since=""/> <fsummary>Writes <c>Record</c> and sets sticky lock.</fsummary> <desc> <marker id="s_write"></marker> @@ -1832,21 +1873,22 @@ mnesia:create_table(person, </desc> </func> <func> - <name since="">schema() -> ok</name> + <name name="schema" arity="0" since=""/> <fsummary>Prints information about all table definitions on the terminal.</fsummary> <desc> <p>Prints information about all table definitions on the terminal.</p> </desc> </func> <func> - <name since="">schema(Tab) -> ok</name> + <name name="schema" arity="1" since=""/> <fsummary>Prints information about one table definition on the terminal.</fsummary> <desc> <p>Prints information about one table definition on the terminal.</p> </desc> </func> <func> - <name since="">select(Tab, MatchSpec [, Lock]) -> transaction abort | [Object]</name> + <name name="select" arity="2" since=""/> + <name name="select" arity="3" since=""/> <fsummary>Matches the objects in <c>Tab</c> against <c>MatchSpec</c>.</fsummary> <desc> <marker id="select_2_3"></marker> @@ -1884,7 +1926,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">select(Tab, MatchSpec, NObjects, Lock) -> transaction abort | {[Object],Cont} | '$end_of_table'</name> + <name name="select" arity="4" since=""/> <fsummary>Matches the objects in <c>Tab</c> against <c>MatchSpec</c>.</fsummary> <desc> <marker id="select_4"></marker> @@ -1907,7 +1949,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">select(Cont) -> transaction abort | {[Object],Cont} | '$end_of_table'</name> + <name name="select" arity="1" since=""/> <fsummary>Continues selecting objects.</fsummary> <desc> <p>Selects more objects with the match specification initiated @@ -1919,7 +1961,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">set_debug_level(Level) -> OldLevel</name> + <name name="set_debug_level" arity="1" since=""/> <fsummary>Changes the internal debug level of Mnesia.</fsummary> <desc> <marker id="set_debug_level"></marker> @@ -1930,7 +1972,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">set_master_nodes(MasterNodes) -> ok | {error, Reason}</name> + <name name="set_master_nodes" arity="1" since=""/> <fsummary>Sets the master nodes for all tables.</fsummary> <desc> <marker id="set_master_nodes_1"></marker> @@ -1943,7 +1985,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">set_master_nodes(Tab, MasterNodes) -> ok | {error, Reason}</name> + <name name="set_master_nodes" arity="2" since=""/> <fsummary>Sets the master nodes for a table.</fsummary> <desc> <marker id="set_master_nodes_2"></marker> @@ -1968,14 +2010,14 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">snmp_close_table(Tab) -> {aborted, R} | {atomic, ok}</name> + <name name="snmp_close_table" arity="1" since=""/> <fsummary>Removes the possibility for SNMP to manipulate the table.</fsummary> <desc> <p>Removes the possibility for SNMP to manipulate the table.</p> </desc> </func> <func> - <name since="">snmp_get_mnesia_key(Tab, RowIndex) -> {ok, Key} | undefined</name> + <name name="snmp_get_mnesia_key" arity="2" since=""/> <fsummary>Gets the corresponding Mnesia key from an SNMP index.</fsummary> <type> <v>Tab ::= atom()</v> @@ -1990,7 +2032,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">snmp_get_next_index(Tab, RowIndex) -> {ok, NextIndex} | endOfTable</name> + <name name="snmp_get_next_index" arity="2" since=""/> <fsummary>Gets the index of the next lexicographical row.</fsummary> <type> <v>Tab ::= atom()</v> @@ -2006,7 +2048,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">snmp_get_row(Tab, RowIndex) -> {ok, Row} | undefined</name> + <name name="snmp_get_row" arity="2" since=""/> <fsummary>Retrieves a row indexed by an SNMP index.</fsummary> <type> <v>Tab ::= atom()</v> @@ -2019,7 +2061,7 @@ mnesia:select(Tab,[{MatchHead, [Guard], [Result]}]),</code> </desc> </func> <func> - <name since="">snmp_open_table(Tab, SnmpStruct) -> {aborted, R} | {atomic, ok}</name> + <name name="snmp_open_table" arity="2" since=""/> <fsummary>Organizes a Mnesia table as an SNMP table.</fsummary> <type> <v>Tab ::= atom()</v> @@ -2073,7 +2115,7 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">start() -> ok | {error, Reason}</name> + <name name="start" arity="0" since=""/> <fsummary>Starts a local Mnesia system.</fsummary> <desc> <marker id="start"></marker> @@ -2115,7 +2157,7 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">stop() -> stopped</name> + <name name="stop" arity="0" since=""/> <fsummary>Stops Mnesia locally.</fsummary> <desc> <marker id="stop"></marker> @@ -2124,7 +2166,7 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">subscribe(EventCategory) -> {ok, Node} | {error, Reason}</name> + <name name="subscribe" arity="1" since=""/> <fsummary>Subscribes to events of type <c>EventCategory</c>.</fsummary> <desc> <marker id="subscribe"></marker> @@ -2134,7 +2176,8 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">sync_dirty(Fun [, Args]) -> ResultOfFun | exit(Reason)</name> + <name name="sync_dirty" arity="1" since=""/> + <name name="sync_dirty" arity="2" since=""/> <fsummary>Calls the <c>Fun</c> in a context that is not protected by a transaction.</fsummary> <desc> <marker id="sync_dirty"></marker> @@ -2150,7 +2193,7 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="OTP 17.0">sync_log() -> ok | {error, Reason}</name> + <name name="sync_log" arity="0" since="OTP 17.0"/> <fsummary>Performs a file sync of the local log file.</fsummary> <desc> <p>Ensures that the local transaction log file is synced to disk. @@ -2160,7 +2203,10 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">sync_transaction(Fun [, Args [, Retries]]) -> {aborted, Reason} | {atomic, ResultOfFun}</name> + <name name="sync_transaction" arity="1" since=""/> + <name name="sync_transaction" arity="2" clause_i="1" since=""/> + <name name="sync_transaction" arity="2" clause_i="2" since=""/> + <name name="sync_transaction" arity="3" since=""/> <fsummary>Synchronously executes a transaction.</fsummary> <desc> <marker id="sync_transaction"></marker> @@ -2173,7 +2219,7 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">system_info(InfoKey) -> Info | exit({aborted, Reason})</name> + <name name="system_info" arity="1" since=""/> <fsummary>Returns information about the Mnesia system.</fsummary> <desc> <marker id="system_info"></marker> @@ -2360,7 +2406,8 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">table(Tab [,[Option]]) -> QueryHandle</name> + <name name="table" arity="1" since=""/> + <name name="table" arity="2" since=""/> <fsummary>Return a QLC query handle.</fsummary> <desc> <marker id="table"></marker> @@ -2415,7 +2462,7 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">table_info(Tab, InfoKey) -> Info | exit({aborted, Reason})</name> + <name name="table_info" arity="2" since=""/> <fsummary>Returns local information about table.</fsummary> <desc> <marker id="table_info"></marker> @@ -2571,7 +2618,11 @@ mnesia:create_table(employee, </desc> </func> <func> - <name since="">transaction(Fun [, Args [, Retries]]) -> {aborted, Reason} | {atomic, ResultOfFun}</name> + <name name="transaction" arity="1" since=""/> + <name name="transaction" arity="2" clause_i="1" since=""/> + <name name="transaction" arity="2" clause_i="2" since=""/> + <name name="transaction" arity="3" since=""/> + <fsummary>Executes a transaction.</fsummary> <desc> <marker id="transaction"></marker> @@ -2593,8 +2644,8 @@ mnesia:create_table(employee, <code type="none"> add_family({family, F, M, Children}) -> ChildOids = lists:map(fun oid/1, Children), - Trans = fun() -> - mnesia:write(F#person{children = ChildOids}, + Trans = fun() -> + mnesia:write(F#person{children = ChildOids}, mnesia:write(M#person{children = ChildOids}, Write = fun(Child) -> mnesia:write(Child) end, lists:foreach(Write, Children) @@ -2639,7 +2690,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">transform_table(Tab, Fun, NewAttributeList, NewRecordName) -> {aborted, R} | {atomic, ok}</name> + <name name="transform_table" arity="4" since=""/> <fsummary>Changes format on all records in table <c>Tab</c>.</fsummary> <desc> <marker id="transform_table_4"></marker> @@ -2660,7 +2711,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">transform_table(Tab, Fun, NewAttributeList) -> {aborted, R} | {atomic, ok}</name> + <name name="transform_table" arity="3" since=""/> <fsummary>Changes format on all records in table <c>Tab</c>.</fsummary> <desc> <p>Calls <c>mnesia:transform_table(Tab, Fun, @@ -2669,7 +2720,8 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">traverse_backup(Source, [SourceMod,] Target, [TargetMod,] Fun, Acc) -> {ok, LastAcc} | {error, Reason}</name> + <name name="traverse_backup" arity="4" since=""/> + <name name="traverse_backup" arity="6" since=""/> <fsummary>Traversal of a backup.</fsummary> <desc> <marker id="traverse_backup"></marker> @@ -2691,7 +2743,7 @@ raise(Name, Amount) -> <c>{BackupItems,NewAcc}</c>, where <c>BackupItems</c> is a list of valid backup items, and <c>NewAcc</c> is a new accumulator value. The returned backup items are written - in the target backup. + in the target backup. </item> <item><c>LastAcc</c> is the last accumulator value. This is the last <c>NewAcc</c> value that was returned by <c>Fun</c>. @@ -2700,7 +2752,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">uninstall_fallback() -> ok | {error,Reason}</name> + <name name="uninstall_fallback" arity="0" since=""/> <fsummary>Uninstalls a fallback.</fsummary> <desc> <marker id="uninstall_fallback_0"></marker> @@ -2709,7 +2761,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">uninstall_fallback(Args) -> ok | {error,Reason}</name> + <name name="uninstall_fallback" arity="1" since=""/> <fsummary>Uninstalls a fallback.</fsummary> <desc> <p>Deinstalls a fallback before it @@ -2736,7 +2788,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">unsubscribe(EventCategory) -> {ok, Node} | {error, Reason}</name> + <name name="unsubscribe" arity="1" since=""/> <fsummary>Subscribes to events of type <c>EventCategory</c>.</fsummary> <desc> <marker id="unsubscribe"></marker> @@ -2746,7 +2798,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">wait_for_tables(TabList, Timeout) -> ok | {timeout, BadTabList} | {error, Reason}</name> + <name name="wait_for_tables" arity="2" since=""/> <fsummary>Waits for tables to be accessible.</fsummary> <desc> <marker id="wait_for_tables"></marker> @@ -2757,7 +2809,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">wread({Tab, Key}) -> transaction abort | RecordList</name> + <name name="wread" arity="1" since=""/> <fsummary>Reads records with given key.</fsummary> <desc> <marker id="wread"></marker> @@ -2765,7 +2817,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">write(Record) -> transaction abort | ok</name> + <name name="write" arity="1" since=""/> <fsummary>Writes a record into the database.</fsummary> <desc> <marker id="write_1"></marker> @@ -2774,7 +2826,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">write(Tab, Record, LockKind) -> transaction abort | ok</name> + <name name="write" arity="3" since=""/> <fsummary>Writes a record into the database.</fsummary> <desc> <marker id="write_3"></marker> @@ -2790,7 +2842,7 @@ raise(Name, Amount) -> </desc> </func> <func> - <name since="">write_lock_table(Tab) -> ok | transaction abort</name> + <name name="write_lock_table" arity="1" since=""/> <fsummary>Sets write lock on an entire table.</fsummary> <desc> <marker id="write_lock_table"></marker> @@ -2874,7 +2926,7 @@ raise(Name, Amount) -> <item> <p><c>-mnesia dc_dump_limit Number</c>. Controls how often <c>disc_copies</c> tables are dumped from memory. - Tables are dumped when + Tables are dumped when <c>filesize(Log) > (filesize(Tab)/Dc_dump_limit)</c>. Lower values reduce CPU overhead but increase disk space and startup times. Default is 4.</p> @@ -3037,6 +3089,6 @@ raise(Name, Amount) -> <seeerl marker="mnesia:mnesia_registry">mnesia_registry(3)</seeerl>, <seeerl marker="stdlib:qlc">qlc(3)</seeerl></p> </section> - + </erlref> diff --git a/lib/mnesia/doc/src/notes.xml b/lib/mnesia/doc/src/notes.xml index 5cda5ed72e..2fd59e6496 100644 --- a/lib/mnesia/doc/src/notes.xml +++ b/lib/mnesia/doc/src/notes.xml @@ -39,7 +39,51 @@ thus constitutes one section in this document. The title of each section is the version number of Mnesia.</p> - <section><title>Mnesia 4.16.3</title> + <section><title>Mnesia 4.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Make <c>mnesia:create_table/2</c> return correct badarg + value.</p> + <p> + Own Id: OTP-16072 Aux Id: PR-2320 </p> + </item> + <item> + <p> + Fixed a bug where mnesia was sometimes not waiting during + start for a commit decision on asymmetric transactions.</p> + <p> + Own Id: OTP-16634 Aux Id: PR-2610 ERL-1227 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Remove usage and documentation of old requests of the + I/O-protocol.</p> + <p> + Own Id: OTP-15695</p> + </item> + <item> + <p> + Avoid using <c>rpc</c> calls to do table reads, which + will reduce the load on rpc server and improve + performance.</p> + <p> + Own Id: OTP-16189</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.16.3</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/mnesia/doc/src/specs.xml b/lib/mnesia/doc/src/specs.xml new file mode 100644 index 0000000000..3d33a1e611 --- /dev/null +++ b/lib/mnesia/doc/src/specs.xml @@ -0,0 +1,6 @@ +<?xml version="1.0" encoding="utf-8" ?> +<specs xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="../specs/specs_mnesia.xml"/> + <xi:include href="../specs/specs_mnesia_frag_hash.xml"/> + <xi:include href="../specs/specs_mnesia_registry.xml"/> +</specs> diff --git a/lib/mnesia/src/mnesia.erl b/lib/mnesia/src/mnesia.erl index 560ebca824..3f1b173e37 100644 --- a/lib/mnesia/src/mnesia.erl +++ b/lib/mnesia/src/mnesia.erl @@ -155,6 +155,7 @@ {'user_properties', proplists:proplist()}. -type t_result(Res) :: {'atomic', Res} | {'aborted', Reason::term()}. +-type result() :: ok | {'error', Reason::term()}. -type activity() :: 'ets' | 'async_dirty' | 'sync_dirty' | 'transaction' | 'sync_transaction' | {'transaction', Retries::non_neg_integer()} | {'sync_transaction', Retries::non_neg_integer()}. @@ -171,6 +172,7 @@ -type config_key() :: extra_db_nodes | dc_dump_limit. -type config_value() :: [node()] | number(). -type config_result() :: {ok, config_value()} | {error, term()}. +-type debug_level() :: 'none' | 'verbose' | 'debug' | 'trace'. -define(DEFAULT_ACCESS, ?MODULE). @@ -230,7 +232,7 @@ e_has_var(X, Pos) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Start and stop --spec start() -> 'ok' | {'error', term()}. +-spec start() -> result(). start() -> start([]). @@ -250,7 +252,7 @@ start_() -> {error, R} end. --spec start([{Option::atom(), Value::_}]) -> 'ok' | {'error', term()}. +-spec start([{Option::atom(), Value::_}]) -> result(). start(ExtraEnv) when is_list(ExtraEnv) -> case mnesia_lib:ensure_loaded(?APPLICATION) of ok -> @@ -281,8 +283,8 @@ stop() -> Other -> Other end. --spec change_config(Config::config_key(), Value::config_value()) -> - config_result(). +-spec change_config(Config, Value) -> config_result() when + Config :: config_key(), Value :: config_value(). change_config(extra_db_nodes, Ns) when is_list(Ns) -> mnesia_controller:connect_nodes(Ns); change_config(dc_dump_limit, N) when is_number(N), N > 0 -> @@ -299,6 +301,10 @@ change_config(BadKey, _BadVal) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Debugging + +-spec set_debug_level(Level :: debug_level()) -> + OldLevel :: debug_level(). + set_debug_level(Level) -> mnesia_subscr:set_debug_level(Level). @@ -371,7 +377,7 @@ transaction(Fun) -> -spec transaction(Fun, Retries) -> t_result(Res) when Fun :: fun(() -> Res), Retries :: non_neg_integer() | 'infinity'; - (Fun, [Arg::_]) -> t_result(Res) when + (Fun, Args::[Arg::_]) -> t_result(Res) when Fun :: fun((...) -> Res). transaction(Fun, Retries) when is_integer(Retries), Retries >= 0 -> transaction(get(mnesia_activity_state), Fun, [], Retries, ?DEFAULT_ACCESS, async); @@ -392,9 +398,9 @@ sync_transaction(Fun) -> transaction(get(mnesia_activity_state), Fun, [], infinity, ?DEFAULT_ACCESS, sync). -spec sync_transaction(Fun, Retries) -> t_result(Res) when - Fun :: fun(() -> Res), + Fun :: fun(() -> Res) | fun((...) -> Res), Retries :: non_neg_integer() | 'infinity'; - (Fun, [Arg::_]) -> t_result(Res) when + (Fun, Args :: [Arg::_]) -> t_result(Res) when Fun :: fun((...) -> Res). sync_transaction(Fun, Retries) when is_integer(Retries), Retries >= 0 -> transaction(get(mnesia_activity_state), Fun, [], Retries, ?DEFAULT_ACCESS, sync); @@ -2637,18 +2643,18 @@ load_mnesia_or_abort() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Database mgt --spec create_schema(Ns::[node()]) -> 'ok' | {'error', Reason::term()}. +-spec create_schema(Ns::[node()]) -> result(). create_schema(Ns) -> create_schema(Ns, []). --spec create_schema(Ns::[node()], [Prop]) -> 'ok' | {'error', Reason::term()} when +-spec create_schema(Ns::[node()], [Prop]) -> result() when Prop :: BackendType | IndexPlugin, BackendType :: {backend_types, [{Name::atom(), Module::module()}]}, IndexPlugin :: {index_plugins, [{{Name::atom()}, Module::module(), Function::atom()}]}. create_schema(Ns, Properties) -> mnesia_bup:create_schema(Ns, Properties). --spec delete_schema(Ns::[node()]) -> 'ok' | {'error', Reason::term()}. +-spec delete_schema(Ns::[node()]) -> result(). delete_schema(Ns) -> mnesia_schema:delete_schema(Ns). @@ -2656,12 +2662,12 @@ delete_schema(Ns) -> add_backend_type(Alias, Module) -> mnesia_schema:add_backend_type(Alias, Module). --spec backup(Dest::term()) -> 'ok' | {'error', Reason::term()}. +-spec backup(Dest::term()) -> result(). backup(Opaque) -> mnesia_log:backup(Opaque). -spec backup(Dest::term(), Mod::module()) -> - 'ok' | {'error', Reason::term()}. + result(). backup(Opaque, Mod) -> mnesia_log:backup(Opaque, Mod). @@ -2679,12 +2685,12 @@ traverse_backup(S, T, Fun, Acc) -> traverse_backup(S, SM, T, TM, F, A) -> mnesia_bup:traverse_backup(S, SM, T, TM, F, A). --spec install_fallback(Src::term()) -> 'ok' | {'error', Reason::term()}. +-spec install_fallback(Src::term()) -> result(). install_fallback(Opaque) -> mnesia_bup:install_fallback(Opaque). -spec install_fallback(Src::term(), Mod::module()|[Opt]) -> - 'ok' | {'error', Reason::term()} when + result() when Opt :: Module | Scope | Dir, Module :: {'module', Mod::module()}, Scope :: {'scope', 'global' | 'local'}, @@ -2692,11 +2698,11 @@ install_fallback(Opaque) -> install_fallback(Opaque, Mod) -> mnesia_bup:install_fallback(Opaque, Mod). --spec uninstall_fallback() -> 'ok' | {'error', Reason::term()}. +-spec uninstall_fallback() -> result(). uninstall_fallback() -> mnesia_bup:uninstall_fallback(). --spec uninstall_fallback(Args) -> 'ok' | {'error', Reason::term()} when +-spec uninstall_fallback(Args) -> result() when Args :: [{'mnesia_dir', Dir::string()}]. uninstall_fallback(Args) -> mnesia_bup:uninstall_fallback(Args). @@ -2707,16 +2713,17 @@ uninstall_fallback(Args) -> activate_checkpoint(Args) -> mnesia_checkpoint:activate(Args). --spec deactivate_checkpoint(Name::_) -> 'ok' | {'error', Reason::term()}. +-spec deactivate_checkpoint(Name::_) -> result(). deactivate_checkpoint(Name) -> mnesia_checkpoint:deactivate(Name). --spec backup_checkpoint(Name::_, Dest::_) -> 'ok' | {'error', Reason::term()}. +-spec backup_checkpoint(Name, Dest) -> result() when + Name :: term(), Dest :: term(). backup_checkpoint(Name, Opaque) -> mnesia_log:backup_checkpoint(Name, Opaque). --spec backup_checkpoint(Name::_, Dest::_, Mod::module()) -> - 'ok' | {'error', Reason::term()}. +-spec backup_checkpoint(Name, Dest, Mod) -> result() when + Name :: term(), Dest :: term(), Mod :: module(). backup_checkpoint(Name, Opaque, Mod) -> mnesia_log:backup_checkpoint(Name, Opaque, Mod). @@ -2743,7 +2750,8 @@ create_table(Name, Arg) -> delete_table(Tab) -> mnesia_schema:delete_table(Tab). --spec add_table_copy(Tab::table(), N::node(), ST::storage_type()) -> t_result(ok). +-spec add_table_copy(Tab, N, ST) -> t_result(ok) when + Tab :: table(), N::node(), ST::storage_type(). add_table_copy(Tab, N, S) -> mnesia_schema:add_table_copy(Tab, N, S). @@ -2755,10 +2763,12 @@ del_table_copy(Tab, N) -> move_table_copy(Tab, From, To) -> mnesia_schema:move_table(Tab, From, To). --spec add_table_index(Tab::table(), I::index_attr()) -> t_result(ok). +-spec add_table_index(Tab, I) -> t_result(ok) when + Tab :: table(), I :: index_attr(). add_table_index(Tab, Ix) -> mnesia_schema:add_table_index(Tab, Ix). --spec del_table_index(Tab::table(), I::index_attr()) -> t_result(ok). +-spec del_table_index(Tab, I) -> t_result(ok) when + Tab::table(), I::index_attr(). del_table_index(Tab, Ix) -> mnesia_schema:del_table_index(Tab, Ix). @@ -2842,7 +2852,7 @@ dump_tables(Tabs) -> %% allow the user to wait for some tables to be loaded -spec wait_for_tables([Tab::table()], TMO::timeout()) -> - 'ok' | {'timeout', [table()]} | {'error', Reason::term()}. + result() | {'timeout', [table()]}. wait_for_tables(Tabs, Timeout) -> mnesia_controller:wait_for_tables(Tabs, Timeout). @@ -2867,7 +2877,7 @@ change_table_load_order(T, O) -> change_table_majority(T, M) -> mnesia_schema:change_table_majority(T, M). --spec set_master_nodes(Ns::[node()]) -> 'ok' | {'error', Reason::term()}. +-spec set_master_nodes(Ns::[node()]) -> result(). set_master_nodes(Nodes) when is_list(Nodes) -> UseDir = system_info(use_dir), IsRunning = system_info(is_running), @@ -2906,8 +2916,7 @@ log_valid_master_nodes(Cstructs, Nodes, UseDir, IsRunning) -> Args = lists:map(Fun, Cstructs), mnesia_recover:log_master_nodes(Args, UseDir, IsRunning). --spec set_master_nodes(Tab::table(), Ns::[node()]) -> - 'ok' | {'error', Reason::term()}. +-spec set_master_nodes(Tab::table(), Ns::[node()]) -> result(). set_master_nodes(Tab, Nodes) when is_list(Nodes) -> UseDir = system_info(use_dir), IsRunning = system_info(is_running), @@ -2962,7 +2971,7 @@ set_master_nodes(Tab, Nodes) -> dump_log() -> mnesia_controller:sync_dump_log(user). --spec sync_log() -> 'ok' | {'error', Reason::term()}. +-spec sync_log() -> result(). sync_log() -> mnesia_monitor:sync_log(latest_log). @@ -3136,7 +3145,7 @@ snmp_filter_key(undefined, RowIndex, Tab, Store) -> load_textfile(F) -> mnesia_text:load_textfile(F). --spec dump_to_textfile(File :: file:filename()) -> 'ok' | 'error' | {'error', term()}. +-spec dump_to_textfile(File :: file:filename()) -> result() | 'error'. dump_to_textfile(F) -> mnesia_text:dump_to_textfile(F). diff --git a/lib/mnesia/src/mnesia_recover.erl b/lib/mnesia/src/mnesia_recover.erl index 2ccea1ea6d..8749b625a1 100644 --- a/lib/mnesia/src/mnesia_recover.erl +++ b/lib/mnesia/src/mnesia_recover.erl @@ -449,6 +449,9 @@ wait_for_decision(D, InitBy, N) -> if Outcome =:= committed -> {Tid, committed}; Outcome =:= aborted -> {Tid, aborted}; + InitBy == startup -> + {ok, Res} = call({wait_for_decision, D}), + {Tid, Res}; Outcome =:= presume_abort -> case N > Max of true -> {Tid, aborted}; @@ -460,10 +463,7 @@ wait_for_decision(D, InitBy, N) -> %% Wait a while for active transactions %% to end and try again timer:sleep(100), - wait_for_decision(D, InitBy, N); - InitBy == startup -> - {ok, Res} = call({wait_for_decision, D}), - {Tid, Res} + wait_for_decision(D, InitBy, N) end. still_pending([Tid | Pending]) -> diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index ae849f2771..ba006208cd 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.16.3 +MNESIA_VSN = 4.17 diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index b27de66984..2533f99bd5 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -32,6 +32,21 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 2.9.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Minor updates due to the new spawn improvements made.</p> + <p> + Own Id: OTP-16368 Aux Id: OTP-15251 </p> + </item> + </list> + </section> + +</section> + <section><title>Observer 2.9.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/observer/src/observer.app.src b/lib/observer/src/observer.app.src index 55de6de0c6..ca65ee703e 100644 --- a/lib/observer/src/observer.app.src +++ b/lib/observer/src/observer.app.src @@ -66,7 +66,7 @@ {registered, []}, {applications, [kernel, stdlib]}, {env, []}, - {runtime_dependencies, ["wx-1.2","stdlib-@OTP-15251@","runtime_tools-1.8.14", - "kernel-@OTP-15251@","et-1.5","erts-@OTP-15251@"]}]}. + {runtime_dependencies, ["wx-1.2","stdlib-3.13","runtime_tools-1.8.14", + "kernel-7.0","et-1.5","erts-11.0"]}]}. diff --git a/lib/observer/src/observer_alloc_wx.erl b/lib/observer/src/observer_alloc_wx.erl index 1b144e05dc..a7b55f0f72 100644 --- a/lib/observer/src/observer_alloc_wx.erl +++ b/lib/observer/src/observer_alloc_wx.erl @@ -158,7 +158,7 @@ handle_info({refresh, Seq}, State#state.active andalso (catch wxWindow:refresh(Panel)), erlang:send_after(1000 div ?DISP_FREQ, self(), {refresh, Next}), if Seq =:= (trunc(DispF)-1) -> - Req = rpc:async_call(Node, observer_backend, sys_info, []), + Req = request_info(Node), {noreply, State#state{time=Ti#ti{tick=Next}, async=Req}}; true -> {noreply, State#state{time=Ti#ti{tick=Next}}} @@ -193,6 +193,13 @@ code_change(_, _, State) -> %%%%%%%%%% +request_info(Node) -> + ReplyTo = self(), + spawn(fun() -> + Res = rpc:call(Node, observer_backend, sys_info, []), + ReplyTo ! {self(), {promise_reply, Res}} + end). + restart_fetcher(Node, #state{panel=Panel, wins=Wins0, time=Ti} = State) -> case rpc:call(Node, observer_backend, sys_info, []) of {badrpc, _} -> State; diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index 6b733687b8..4de0cc113f 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 2.9.3 +OBSERVER_VSN = 2.9.4 diff --git a/lib/odbc/c_src/odbcserver.c b/lib/odbc/c_src/odbcserver.c index fb4f61417e..996c122199 100644 --- a/lib/odbc/c_src/odbcserver.c +++ b/lib/odbc/c_src/odbcserver.c @@ -304,7 +304,7 @@ int main(void) static void spawn_sup(const char *port) { DWORD threadId; - (HANDLE)_beginthreadex(NULL, 0, supervise, port, 0, &threadId); + _beginthreadex(NULL, 0, supervise, port, 0, &threadId); } #elif defined(UNIX) static void spawn_sup(const char *port) diff --git a/lib/odbc/c_src/odbcserver.h b/lib/odbc/c_src/odbcserver.h index 0461c57d1f..28bb2b9030 100644 --- a/lib/odbc/c_src/odbcserver.h +++ b/lib/odbc/c_src/odbcserver.h @@ -120,7 +120,7 @@ /*------------------------ TYPDEFS ----------------------------------*/ -typedef char byte; +typedef unsigned char byte; typedef int Boolean; typedef struct { diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index 8d708162e4..d8c6126496 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -32,7 +32,35 @@ <p>This document describes the changes made to the odbc application. </p> - <section><title>ODBC 2.12.4</title> + <section><title>ODBC 2.13</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix various compiler warnings on 64-bit Windows.</p> + <p> + Own Id: OTP-15800</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Rewrite due to the removal of <c>erl_interface</c> legacy + functions.</p> + <p> + Own Id: OTP-16544 Aux Id: OTP-16328 </p> + </item> + </list> + </section> + +</section> + +<section><title>ODBC 2.12.4</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index df6db09f2f..cf271f3505 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.12.4 +ODBC_VSN = 2.13 diff --git a/lib/os_mon/c_src/nteventlog/elog_main.c b/lib/os_mon/c_src/nteventlog/elog_main.c index c31e3ef1ff..f1baaa4084 100644 --- a/lib/os_mon/c_src/nteventlog/elog_main.c +++ b/lib/os_mon/c_src/nteventlog/elog_main.c @@ -266,11 +266,11 @@ BOOL output_record(char *category, EVENTLOGRECORD *event){ strlen(PIPE_LOG_FORMAT) + PIPE_LOG_EXTRA); sprintf(buff,PIPE_LOG_FORMAT, - strlen(tbuff),tbuff, - strlen(category), category, - strlen(fac), fac, - strlen(sev), sev, - strlen(bigbuff), bigbuff); + (int)strlen(tbuff), tbuff, + (int)strlen(category), category, + (int)strlen(fac), fac, + (int)strlen(sev), sev, + (int)strlen(bigbuff), bigbuff); ret = data_to_pipe(buff,ackbuff, ACK_MAX); if(ret && strcmp(ackbuff,PIPE_LOG_ACK)) ret = FALSE; diff --git a/lib/os_mon/doc/src/notes.xml b/lib/os_mon/doc/src/notes.xml index 63efa96e2f..b08ade9938 100644 --- a/lib/os_mon/doc/src/notes.xml +++ b/lib/os_mon/doc/src/notes.xml @@ -31,6 +31,33 @@ </header> <p>This document describes the changes made to the OS_Mon application.</p> +<section><title>Os_Mon 2.5.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix various compiler warnings on 64-bit Windows.</p> + <p> + Own Id: OTP-15800</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + </list> + </section> + +</section> + <section><title>Os_Mon 2.5.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/os_mon/vsn.mk b/lib/os_mon/vsn.mk index 6081e181ff..e4b7574e02 100644 --- a/lib/os_mon/vsn.mk +++ b/lib/os_mon/vsn.mk @@ -1 +1 @@ -OS_MON_VSN = 2.5.1 +OS_MON_VSN = 2.5.2 diff --git a/lib/parsetools/doc/src/notes.xml b/lib/parsetools/doc/src/notes.xml index f8cd9b972d..3975c55c6c 100644 --- a/lib/parsetools/doc/src/notes.xml +++ b/lib/parsetools/doc/src/notes.xml @@ -31,6 +31,22 @@ </header> <p>This document describes the changes made to the Parsetools application.</p> +<section><title>Parsetools 2.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Remove usage and documentation of old requests of the + I/O-protocol.</p> + <p> + Own Id: OTP-15695</p> + </item> + </list> + </section> + +</section> + <section><title>Parsetools 2.1.8</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/parsetools/vsn.mk b/lib/parsetools/vsn.mk index 1a5201ce5d..c18fcbe762 100644 --- a/lib/parsetools/vsn.mk +++ b/lib/parsetools/vsn.mk @@ -1 +1 @@ -PARSETOOLS_VSN = 2.1.8 +PARSETOOLS_VSN = 2.2 diff --git a/lib/public_key/asn1/OTP-PKIX.asn1 b/lib/public_key/asn1/OTP-PKIX.asn1 index ff3250b383..e1d8f2e121 100644 --- a/lib/public_key/asn1/OTP-PKIX.asn1 +++ b/lib/public_key/asn1/OTP-PKIX.asn1 @@ -122,7 +122,9 @@ IMPORTS sha224WithRSAEncryption, sha256WithRSAEncryption, sha384WithRSAEncryption, - sha512WithRSAEncryption + sha512WithRSAEncryption, + id-RSASSA-PSS, + RSASSA-PSS-params FROM PKCS-1 { iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) pkcs-1(1) @@ -341,6 +343,7 @@ SupportedSignatureAlgorithms SIGNATURE-ALGORITHM-CLASS ::= { sha256-with-rsa-encryption | sha384-with-rsa-encryption | sha512-with-rsa-encryption | + rsassa-pss | ecdsa-with-sha1 | ecdsa-with-sha224 | ecdsa-with-sha256 | @@ -348,7 +351,7 @@ SupportedSignatureAlgorithms SIGNATURE-ALGORITHM-CLASS ::= { ecdsa-with-sha512 } SupportedPublicKeyAlgorithms PUBLIC-KEY-ALGORITHM-CLASS ::= { - dsa | rsa-encryption | dh | kea | ec-public-key } + dsa | rsa-encryption | rsa-pss | dh | kea | ec-public-key } -- DSA Keys and Signatures @@ -430,6 +433,11 @@ SupportedPublicKeyAlgorithms PUBLIC-KEY-ALGORITHM-CLASS ::= { ID sha512WithRSAEncryption TYPE NULL } + rsassa-pss SIGNATURE-ALGORITHM-CLASS ::= { + ID id-RSASSA-PSS + TYPE RSASSA-PSS-params } + + -- Certificate.signature -- See PKCS #1 (RFC 2313). XXX @@ -440,6 +448,11 @@ SupportedPublicKeyAlgorithms PUBLIC-KEY-ALGORITHM-CLASS ::= { TYPE NULL PUBLIC-KEY-TYPE RSAPublicKey } + rsa-pss PUBLIC-KEY-ALGORITHM-CLASS ::= { + ID id-RSASSA-PSS + TYPE RSASSA-PSS-params + PUBLIC-KEY-TYPE RSAPublicKey } + -- -- Diffie-Hellman Keys -- @@ -494,6 +507,7 @@ SupportedPublicKeyAlgorithms PUBLIC-KEY-ALGORITHM-CLASS ::= { ID ecdsa-with-SHA512 TYPE EcpkParameters } -- XXX Must be empty and not NULL + FIELD-ID-CLASS ::= CLASS { &id OBJECT IDENTIFIER UNIQUE, &Type } diff --git a/lib/public_key/asn1/PKCS-1.asn1 b/lib/public_key/asn1/PKCS-1.asn1 index 117eacd8ad..6fb7ccb981 100644 --- a/lib/public_key/asn1/PKCS-1.asn1 +++ b/lib/public_key/asn1/PKCS-1.asn1 @@ -1,10 +1,15 @@ +-- PKCS #1 v2.2 ASN.1 Module +-- Revised October 27, 2012 +-- (plain merged with previous version to support all that we need) + +-- This module has been checked for conformance with the +-- ASN.1 standard by the OSS ASN.1 Tools + PKCS-1 { iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) pkcs-1(1) modules(0) pkcs-1(1) } --- $Revision: 1.1 $ - DEFINITIONS EXPLICIT TAGS ::= BEGIN @@ -15,43 +20,74 @@ BEGIN -- gov(101) csor(3) nistalgorithm(4) modules(0) sha2(1) -- }; +-- ============================ +-- Basic object identifiers +-- ============================ +-- The DER encoding of this in hexadecimal is: +-- (0x)06 08 +-- 2A 86 48 86 F7 0D 01 01 + pkcs-1 OBJECT IDENTIFIER ::= { iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) 1 } +-- +-- When rsaEncryption is used in an AlgorithmIdentifier the parameters +-- MUST be present and MUST be NULL. +-- rsaEncryption OBJECT IDENTIFIER ::= { pkcs-1 1 } +-- +-- When id-RSAES-OAEP is used in an AlgorithmIdentifier the parameters MUST +-- be present and MUST be RSAES-OAEP-params. +-- id-RSAES-OAEP OBJECT IDENTIFIER ::= { pkcs-1 7 } -id-pSpecified OBJECT IDENTIFIER ::= { pkcs-1 9 } +-- +-- When id-pSpecified is used in an AlgorithmIdentifier the parameters MUST +-- be an OCTET STRING. +-- +id-pSpecified OBJECT IDENTIFIER ::= { pkcs-1 9 } +-- +-- When id-RSASSA-PSS is used in an AlgorithmIdentifier the parameters MUST +-- be present and MUST be RSASSA-PSS-params. +-- id-RSASSA-PSS OBJECT IDENTIFIER ::= { pkcs-1 10 } -md2WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 2 } -md5WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 4 } -sha1WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 5 } -sha256WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 11 } -sha384WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 12 } -sha512WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 13 } -sha224WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 14 } +-- +-- When the following OIDs are used in an AlgorithmIdentifier the parameters +-- MUST be present and MUST be NULL. +-- +md2WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 2 } +md5WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 4 } +sha1WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 5 } +sha224WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 14 } +sha256WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 11 } +sha384WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 12 } +sha512WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 13 } +sha512-224WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 15 } +sha512-256WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 16 } -- ISO oid - equvivalent to sha1WithRSAEncryption sha-1WithRSAEncryption OBJECT IDENTIFIER ::= { iso(1) identified-organization(3) oiw(14) secsig(3) algorithms(2) sha-1WithRSAEncryption(29)} - -id-sha1 OBJECT IDENTIFIER ::= { - iso(1) identified-organization(3) oiw(14) secsig(3) - algorithms(2) 26 +-- +-- This OID really belongs in a module with the secsig OIDs. +-- +id-sha1 OBJECT IDENTIFIER ::= { + iso(1) identified-organization(3) oiw(14) secsig(3) algorithms(2) 26 } - +-- +-- OIDs for MD2 and MD5, allowed only in EMSA-PKCS1-v1_5. +-- id-md2 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) rsadsi(113549) digestAlgorithm(2) 2 -} + iso(1) member-body(2) us(840) rsadsi(113549) digestAlgorithm(2) 2} id-md5 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) rsadsi(113549) digestAlgorithm(2) 5 -} + iso(1) member-body(2) us(840) rsadsi(113549) digestAlgorithm(2) 5} +-- Additional oids that where included previously... id-hmacWithSHA224 OBJECT IDENTIFIER ::= { iso(1) member-body(2) us(840) rsadsi(113549) digestAlgorithm(2) 8 } @@ -68,8 +104,6 @@ id-hmacWithSHA512 OBJECT IDENTIFIER ::= { iso(1) member-body(2) us(840) rsadsi(113549) digestAlgorithm(2) 11 } -id-mgf1 OBJECT IDENTIFIER ::= { pkcs-1 8 } - id-sha224 OBJECT IDENTIFIER ::= { joint-iso-itu-t(2) country(16) us(840) organization(1) gov(101) csor(3) nistalgorithm(4) hashalgs(2) 4 } @@ -86,68 +120,255 @@ id-sha512 OBJECT IDENTIFIER ::= { joint-iso-itu-t(2) country(16) us(840) organization(1) gov(101) csor(3) nistalgorithm(4) hashalgs(2) 3 } +-- +-- When id-mgf1 is used in an AlgorithmIdentifier the parameters MUST be +-- present and MUST be a HashAlgorithm, for example sha1. +-- +id-mgf1 OBJECT IDENTIFIER ::= { pkcs-1 8 } + +-- ================ +-- Useful types +-- ================ +ALGORITHM-IDENTIFIER ::= CLASS { + &id OBJECT IDENTIFIER UNIQUE, + &Type OPTIONAL +} -RSAPublicKey ::= SEQUENCE { - modulus INTEGER, -- n - publicExponent INTEGER -- e + WITH SYNTAX { OID &id [PARAMETERS &Type] } +-- Note: the parameter InfoObjectSet in the following definitions allows a +-- distinct information object set to be specified for sets of algorithms +-- such as: +-- DigestAlgorithms ALGORITHM-IDENTIFIER ::= { +-- { OID id-md2 PARAMETERS NULL }| +-- { OID id-md5 PARAMETERS NULL }| +-- { OID id-sha1 PARAMETERS NULL } +-- } +-- +AlgorithmIdentifier-PKCS1 { ALGORITHM-IDENTIFIER:InfoObjectSet } ::= SEQUENCE { + algorithm + ALGORITHM-IDENTIFIER.&id({InfoObjectSet}), + parameters + ALGORITHM-IDENTIFIER.&Type({InfoObjectSet}{@.algorithm}) OPTIONAL +} + +-- ============== +-- Algorithms +-- ============== +-- +-- Allowed EME-OAEP and EMSA-PSS digest algorithms. +-- +OAEP-PSSDigestAlgorithms ALGORITHM-IDENTIFIER ::= { + { OID id-sha1 PARAMETERS NULL }| + { OID id-sha256 PARAMETERS NULL }| + { OID id-sha384 PARAMETERS NULL }| + { OID id-sha512 PARAMETERS NULL }, + ... -- Allows for future expansion -- +} +-- +-- Allowed EMSA-PKCS1-v1_5 digest algorithms. +-- +PKCS1-v1-5DigestAlgorithms ALGORITHM-IDENTIFIER ::= { + { OID id-md2 PARAMETERS NULL }| + { OID id-md5 PARAMETERS NULL }| + { OID id-sha1 PARAMETERS NULL }| + { OID id-sha256 PARAMETERS NULL }| + { OID id-sha384 PARAMETERS NULL }| + { OID id-sha512 PARAMETERS NULL } +} + +-- When id-md2 and id-md5 are used in an AlgorithmIdentifier, the +-- parameters field shall have a value of type NULL. + +-- When id-sha1, id-sha224, id-sha256, id-sha384, id-sha512, +-- id-sha512-224, and id-sha512-256 are used in an +-- AlgorithmIdentifier, the parameters (which are optional) SHOULD be +-- omitted, but if present, they SHALL have a value of type NULL. +-- However, implementations MUST accept AlgorithmIdentifier values +-- both without parameters and with NULL parameters. + +-- Exception: When formatting the DigestInfoValue in EMSA-PKCS1-v1_5 +-- (see Section 9.2), the parameters field associated with id-sha1, +-- id-sha224, id-sha256, id-sha384, id-sha512, id-sha512-224, and +-- id-sha512-256 SHALL have a value of type NULL. This is to +-- maintain compatibility with existing implementations and with the +-- numeric information values already published for EMSA-PKCS1-v1_5, +-- which are also reflected in IEEE 1363a. + +sha1 HashAlgorithm ::= { + algorithm id-sha1, + parameters SHA1Parameters : NULL +} + +HashAlgorithm ::= AlgorithmIdentifier-PKCS1 { {OAEP-PSSDigestAlgorithms} } +SHA1Parameters ::= NULL + +-- +-- Allowed mask generation function algorithms. +-- If the identifier is id-mgf1, the parameters are a HashAlgorithm. +-- +PKCS1MGFAlgorithms ALGORITHM-IDENTIFIER ::= { + { OID id-mgf1 PARAMETERS HashAlgorithm }, + ... -- Allows for future expansion -- +} + +-- +-- Default AlgorithmIdentifier for id-RSAES-OAEP.maskGenAlgorithm and +-- id-RSASSA-PSS.maskGenAlgorithm. +-- +mgf1SHA1 MaskGenAlgorithm ::= { + algorithm id-mgf1, + parameters HashAlgorithm : sha1 } +MaskGenAlgorithm ::= AlgorithmIdentifier-PKCS1 { {PKCS1MGFAlgorithms} } + +-- +-- Allowed algorithms for pSourceAlgorithm. +-- +PKCS1PSourceAlgorithms ALGORITHM-IDENTIFIER ::= { + { OID id-pSpecified PARAMETERS EncodingParameters }, + ... -- Allows for future expansion -- +} + +EncodingParameters ::= OCTET STRING(SIZE(0..MAX)) + +-- +-- This identifier means that the label L is an empty string, so the digest +-- of the empty string appears in the RSA block before masking. +-- +pSpecifiedEmpty PSourceAlgorithm ::= { + algorithm id-pSpecified, + parameters EncodingParameters : emptyString +} +PSourceAlgorithm ::= AlgorithmIdentifier-PKCS1 { {PKCS1PSourceAlgorithms} } + +emptyString EncodingParameters ::= ''H + +-- +-- Type identifier definitions for the PKCS #1 OIDs. +-- +PKCS1Algorithms ALGORITHM-IDENTIFIER ::= { + { OID rsaEncryption PARAMETERS NULL } | + { OID md2WithRSAEncryption PARAMETERS NULL } | + { OID md5WithRSAEncryption PARAMETERS NULL } | + { OID sha1WithRSAEncryption PARAMETERS NULL } | + { OID sha256WithRSAEncryption PARAMETERS NULL } | + { OID sha384WithRSAEncryption PARAMETERS NULL } | + { OID sha512WithRSAEncryption PARAMETERS NULL } | + { OID id-RSAES-OAEP PARAMETERS RSAES-OAEP-params } | + PKCS1PSourceAlgorithms | + { OID id-RSASSA-PSS PARAMETERS RSASSA-PSS-params } , + ... -- Allows for future expansion -- +} +-- =================== +-- Main structures +-- =================== +RSAPublicKey ::= SEQUENCE { + modulus INTEGER, -- n + publicExponent INTEGER -- e +} +-- +-- Representation of RSA private key with information for the CRT algorithm. +-- RSAPrivateKey ::= SEQUENCE { - version Version, - modulus INTEGER, -- n - publicExponent INTEGER, -- e - privateExponent INTEGER, -- d - prime1 INTEGER, -- p - prime2 INTEGER, -- q - exponent1 INTEGER, -- d mod (p-1) - exponent2 INTEGER, -- d mod (q-1) - coefficient INTEGER, -- (inverse of q) mod p - otherPrimeInfos OtherPrimeInfos OPTIONAL + version Version, + modulus INTEGER, -- n + publicExponent INTEGER, -- e + privateExponent INTEGER, -- d + prime1 INTEGER, -- p + prime2 INTEGER, -- q + exponent1 INTEGER, -- d mod (p-1) + exponent2 INTEGER, -- d mod (q-1) + coefficient INTEGER, -- (inverse of q) mod p + otherPrimeInfos OtherPrimeInfos OPTIONAL } Version ::= INTEGER { two-prime(0), multi(1) } - (CONSTRAINED BY { - -- version must be multi if otherPrimeInfos present -- - }) + (CONSTRAINED BY {-- version must be multi if otherPrimeInfos present --}) OtherPrimeInfos ::= SEQUENCE SIZE(1..MAX) OF OtherPrimeInfo OtherPrimeInfo ::= SEQUENCE { - prime INTEGER, -- ri - exponent INTEGER, -- di - coefficient INTEGER -- ti + prime INTEGER, -- ri + exponent INTEGER, -- di + coefficient INTEGER -- ti } -Algorithm ::= SEQUENCE { - algorithm OBJECT IDENTIFIER, - parameters ANY DEFINED BY algorithm OPTIONAL +-- +-- AlgorithmIdentifier.parameters for id-RSAES-OAEP. +-- Note that the tags in this Sequence are explicit. +-- + +RSAES-OAEP-params ::= SEQUENCE { + hashAlgorithm [0] HashAlgorithm DEFAULT sha1, + maskGenAlgorithm [1] MaskGenAlgorithm DEFAULT mgf1SHA1, + pSourceAlgorithm [2] PSourceAlgorithm DEFAULT pSpecifiedEmpty } -AlgorithmNull ::= SEQUENCE { - algorithm OBJECT IDENTIFIER, - parameters NULL +-- +-- Identifier for default RSAES-OAEP algorithm identifier. +-- The DER Encoding of this is in hexadecimal: +-- (0x)30 0D +-- 06 09 +-- 2A 86 48 86 F7 0D 01 01 07 +-- 30 00 +-- Notice that the DER encoding of default values is "empty". +-- +rSAES-OAEP-Default-Identifier RSAES-AlgorithmIdentifier ::= { + algorithm id-RSAES-OAEP, + parameters RSAES-OAEP-params : { + hashAlgorithm sha1, + maskGenAlgorithm mgf1SHA1, + pSourceAlgorithm pSpecifiedEmpty + } } +RSAES-AlgorithmIdentifier ::= AlgorithmIdentifier-PKCS1 { + {PKCS1Algorithms} +} +-- +-- AlgorithmIdentifier.parameters for id-RSASSA-PSS. +-- Note that the tags in this Sequence are explicit. +-- RSASSA-PSS-params ::= SEQUENCE { - hashAlgorithm [0] Algorithm, -- DEFAULT sha1, - maskGenAlgorithm [1] Algorithm, -- DEFAULT mgf1SHA1, - saltLength [2] INTEGER DEFAULT 20, - trailerField [3] TrailerField DEFAULT trailerFieldBC + hashAlgorithm [0] HashAlgorithm DEFAULT sha1, + maskGenAlgorithm [1] MaskGenAlgorithm DEFAULT mgf1SHA1, + saltLength [2] INTEGER DEFAULT 20, + trailerField [3] TrailerField DEFAULT trailerFieldBC } TrailerField ::= INTEGER { trailerFieldBC(1) } -DigestInfo ::= SEQUENCE { - digestAlgorithm Algorithm, - digest OCTET STRING +-- +-- Identifier for default RSASSA-PSS algorithm identifier +-- The DER Encoding of this is in hexadecimal: +-- (0x)30 0D +-- 06 09 +-- 2A 86 48 86 F7 0D 01 01 0A +-- 30 00 +-- Notice that the DER encoding of default values is "empty". +-- +rSASSA-PSS-Default-Identifier RSASSA-AlgorithmIdentifier ::= { + algorithm id-RSASSA-PSS, + parameters RSASSA-PSS-params : { + hashAlgorithm sha1, + maskGenAlgorithm mgf1SHA1, + saltLength 20,trailerField trailerFieldBC + } +} +RSASSA-AlgorithmIdentifier ::= AlgorithmIdentifier-PKCS1 { + {PKCS1Algorithms} } -DigestInfoNull ::= SEQUENCE { - digestAlgorithm AlgorithmNull, +-- +-- Syntax for the EMSA-PKCS1-v1_5 hash identifier. +-- +DigestInfo ::= SEQUENCE { + digestAlgorithm DigestAlgorithm, digest OCTET STRING } +DigestAlgorithm ::= AlgorithmIdentifier-PKCS1 { {PKCS1-v1-5DigestAlgorithms} } -END -- PKCS1Definitions - +END -- PKCS1Definitions diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index c182a28c53..b15e745252 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -35,6 +35,30 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 1.8</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Added support for RSA-PSS signature schemes</p> + <p> + Own Id: OTP-15247</p> + </item> + <item> + <p> + Calls of deprecated functions in the <seeguide + marker="crypto:new_api#the-old-api">Old Crypto + API</seeguide> are replaced by calls of their <seeguide + marker="crypto:new_api#the-new-api">substitutions</seeguide>.</p> + <p> + Own Id: OTP-16346</p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 1.7.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/doc/src/public_key.xml b/lib/public_key/doc/src/public_key.xml index 9c5aaa9812..3e72f88894 100644 --- a/lib/public_key/doc/src/public_key.xml +++ b/lib/public_key/doc/src/public_key.xml @@ -100,6 +100,7 @@ <datatype> <name name="public_key"/> <name name="rsa_public_key"/> + <name name="rsa_pss_public_key"/> <name name="dsa_public_key"/> <name name="ec_public_key"/> <name name="ecpk_parameters"/> @@ -118,6 +119,7 @@ <datatype> <name name="private_key"/> <name name="rsa_private_key"/> + <name name="rsa_pss_private_key"/> <name name="dsa_private_key"/> <name name="ec_private_key"/> <desc> @@ -405,8 +407,8 @@ <v>CertChain = [der_encoded()]</v> <d>A list of DER-encoded certificates in trust order ending with the peer certificate.</d> <v>Options = proplists:proplist()</v> - <v>PublicKeyInfo = {?'rsaEncryption' | ?'id-dsa', - rsa_public_key() | integer(), 'NULL' | 'Dss-Parms'{}}</v> + <v>PublicKeyInfo = {?'rsaEncryption' | ?'id-RSASSA-PSS'| ?'id-dsa', + rsa_public_key() | integer(), 'NULL' | 'RSASSA-PSS-params'{} | 'Dss-Parms'{}}</v> <v>PolicyTree = term()</v> <d>At the moment this is always an empty list as policies are not currently supported.</d> <v>Reason = cert_expired | invalid_issuer | invalid_signature | name_not_permitted | @@ -582,7 +584,15 @@ fun(#'DistributionPoint'{}, #'CertificateList'{}, <p> Extracts distribution points from the certificates extensions.</p> </desc> </func> - + + <func> + <name name="pkix_hash_type" arity="1" since="@master@"/> + <fsummary>Translates OID to Erlang digest type</fsummary> + <desc> + <p>Translates OID to Erlang digest type</p> + </desc> + </func> + <func> <name name="pkix_match_dist_point" arity="2" since="OTP 19.0"/> <fsummary>Checks whether the given distribution point matches the diff --git a/lib/public_key/doc/src/public_key_records.xml b/lib/public_key/doc/src/public_key_records.xml index 8075ecc4d1..3036d45c00 100644 --- a/lib/public_key/doc/src/public_key_records.xml +++ b/lib/public_key/doc/src/public_key_records.xml @@ -138,7 +138,22 @@ prime, % integer() exponent, % integer() coefficient % integer() - }. </code> + }. + +#'RSASSA-PSS-params'{hashAlgorithm, % #'HashAlgorithm'{}}, + maskGenAlgorithm, % #'MaskGenAlgorithm'{}}, + saltLength, % integer(), + trailerField, % integer() + }. + +#'HashAlgorithm'{algorithm, % oid() + parameters % defaults to asn1_NOVALUE + }. + +#'MaskGenAlgorithm'{algorithm, % oid() + parameters, % defaults to asn1_NOVALUE + }. + </code> </section> diff --git a/lib/public_key/src/pubkey_cert.erl b/lib/public_key/src/pubkey_cert.erl index a5836f6d07..1680845075 100644 --- a/lib/public_key/src/pubkey_cert.erl +++ b/lib/public_key/src/pubkey_cert.erl @@ -24,16 +24,29 @@ -include("public_key.hrl"). --export([init_validation_state/3, prepare_for_next_cert/2, - validate_time/3, validate_signature/6, - validate_issuer/4, validate_names/6, +-export([init_validation_state/3, + prepare_for_next_cert/2, + validate_time/3, + validate_signature/6, + validate_issuer/4, + validate_names/6, validate_extensions/4, - normalize_general_name/1, is_self_signed/1, - is_issuer/2, issuer_id/2, distribution_points/1, - is_fixed_dh_cert/1, verify_data/1, verify_fun/4, - select_extension/2, match_name/3, - extensions_list/1, cert_auth_key_id/1, time_str_2_gregorian_sec/1, - gen_test_certs/1, root_cert/2]). + normalize_general_name/1, + is_self_signed/1, + is_issuer/2, + issuer_id/2, + distribution_points/1, + is_fixed_dh_cert/1, + verify_data/1, + verify_fun/4, + select_extension/2, + match_name/3, + extensions_list/1, + cert_auth_key_id/1, + time_str_2_gregorian_sec/1, + gen_test_certs/1, + x509_pkix_sign_types/1, + root_cert/2]). -define(NULL, 0). @@ -503,6 +516,17 @@ gen_test_certs( DERCAs = ca_config(RootCert, CAsKeys), [{cert, DERCert}, {key, DERKey}, {cacerts, DERCAs}]. + +x509_pkix_sign_types(#'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = #'RSASSA-PSS-params'{hashAlgorithm = #'HashAlgorithm'{algorithm = Alg}}}) -> + Hash = public_key:pkix_hash_type(Alg), + {Hash, rsa_pss_pss, [{rsa_padding, rsa_pkcs1_pss_padding}, + {rsa_pss_saltlen, -1}, + {rsa_mgf1_md, Hash}]}; +x509_pkix_sign_types(#'SignatureAlgorithm'{algorithm = Alg}) -> + {Hash, Sign} = public_key:pkix_sign_types(Alg), + {Hash, Sign, []}. + %%% -spec root_cert(string(), [cert_opt()]) -> test_root_cert(). %% @@ -511,13 +535,16 @@ root_cert(Name, Opts) -> PrivKey = gen_key(proplists:get_value(key, Opts, default_key_gen())), TBS = cert_template(), Issuer = subject("root", Name), + SignatureId = sign_algorithm(PrivKey, Opts), + SPI = public_key(PrivKey, SignatureId), + OTPTBS = TBS#'OTPTBSCertificate'{ - signature = sign_algorithm(PrivKey, Opts), + signature = SignatureId, issuer = Issuer, validity = validity(Opts), subject = Issuer, - subjectPublicKeyInfo = public_key(PrivKey), + subjectPublicKeyInfo = SPI, extensions = extensions(undefined, ca, Opts) }, #{cert => public_key:pkix_sign(OTPTBS, PrivKey), @@ -552,17 +579,21 @@ extensions_list(Extensions) -> extract_verify_data(OtpCert, DerCert) -> Signature = OtpCert#'OTPCertificate'.signature, - SigAlgRec = OtpCert#'OTPCertificate'.signatureAlgorithm, - SigAlg = SigAlgRec#'SignatureAlgorithm'.algorithm, + SigAlg = OtpCert#'OTPCertificate'.signatureAlgorithm, PlainText = encoded_tbs_cert(DerCert), - {DigestType,_} = public_key:pkix_sign_types(SigAlg), + {DigestType,_,_} = x509_pkix_sign_types(SigAlg), {DigestType, PlainText, Signature}. verify_signature(OtpCert, DerCert, Key, KeyParams) -> {DigestType, PlainText, Signature} = extract_verify_data(OtpCert, DerCert), case Key of #'RSAPublicKey'{} -> - public_key:verify(PlainText, DigestType, Signature, Key); + case KeyParams of + #'RSASSA-PSS-params'{} -> + public_key:verify(PlainText, DigestType, Signature, Key, verify_options(KeyParams)); + 'NULL' -> + public_key:verify(PlainText, DigestType, Signature, Key) + end; _ -> public_key:verify(PlainText, DigestType, Signature, {Key, KeyParams}) end. @@ -1119,6 +1150,8 @@ is_key(#'DSAPrivateKey'{}) -> true; is_key(#'RSAPrivateKey'{}) -> true; +is_key({#'RSAPrivateKey'{}, _}) -> + true; is_key(#'ECPrivateKey'{}) -> true; is_key(_) -> @@ -1174,10 +1207,18 @@ validity(Opts) -> #'Validity'{notBefore={generalTime, Format(DefFrom)}, notAfter ={generalTime, Format(DefTo)}}. -sign_algorithm(#'RSAPrivateKey'{}, Opts) -> - Type = rsa_digest_oid(proplists:get_value(digest, Opts, sha1)), - #'SignatureAlgorithm'{algorithm = Type, - parameters = 'NULL'}; +sign_algorithm(#'RSAPrivateKey'{} = Key , Opts) -> + case proplists:get_value(rsa_padding, Opts, rsa_pkcs1_pss_padding) of + rsa_pkcs1_pss_padding -> + DigestId = rsa_digest_oid(proplists:get_value(digest, Opts, sha1)), + rsa_sign_algo(Key, DigestId, 'NULL'); + rsa_pss_rsae -> + DigestId = rsa_digest_oid(proplists:get_value(digest, Opts, sha256)), + rsa_sign_algo(Key, DigestId, 'NULL') + end; +sign_algorithm({#'RSAPrivateKey'{} = Key,#'RSASSA-PSS-params'{} = Params}, _Opts) -> + rsa_sign_algo(Key, ?'id-RSASSA-PSS', Params); + sign_algorithm(#'DSAPrivateKey'{p=P, q=Q, g=G}, _Opts) -> #'SignatureAlgorithm'{algorithm = ?'id-dsa-with-sha1', parameters = {params,#'Dss-Parms'{p=P, q=Q, g=G}}}; @@ -1185,6 +1226,16 @@ sign_algorithm(#'ECPrivateKey'{parameters = Parms}, Opts) -> Type = ecdsa_digest_oid(proplists:get_value(digest, Opts, sha1)), #'SignatureAlgorithm'{algorithm = Type, parameters = Parms}. + +rsa_sign_algo(#'RSAPrivateKey'{}, ?'id-RSASSA-PSS' = Type, #'RSASSA-PSS-params'{} = Params) -> + #'SignatureAlgorithm'{algorithm = Type, + parameters = Params}; +rsa_sign_algo(#'RSAPrivateKey'{}, Type, Parms) -> + #'SignatureAlgorithm'{algorithm = Type, + parameters = Parms}. + +rsa_digest_oid(Oid) when is_tuple(Oid) -> + Oid; rsa_digest_oid(sha1) -> ?'sha1WithRSAEncryption'; rsa_digest_oid(sha) -> @@ -1196,8 +1247,10 @@ rsa_digest_oid(sha384) -> rsa_digest_oid(sha256) -> ?'sha256WithRSAEncryption'; rsa_digest_oid(md5) -> - ?'md5WithRSAEncryption'. + ?'md5WithRSAEncryption'. +ecdsa_digest_oid(Oid) when is_tuple(Oid) -> + Oid; ecdsa_digest_oid(sha1) -> ?'ecdsa-with-SHA1'; ecdsa_digest_oid(sha) -> @@ -1229,12 +1282,13 @@ cert_chain(Role, IssuerCert, IssuerKey, [CAOpts | Rest], N, Acc) -> cert(Role, #'OTPCertificate'{tbsCertificate = #'OTPTBSCertificate'{subject = Issuer}}, PrivKey, Key, Contact, Name, Opts, Type) -> TBS = cert_template(), + SignAlgoId = sign_algorithm(PrivKey, Opts), OTPTBS = TBS#'OTPTBSCertificate'{ - signature = sign_algorithm(PrivKey, Opts), + signature = SignAlgoId, issuer = Issuer, validity = validity(Opts), subject = subject(Contact, atom_to_list(Role) ++ Name), - subjectPublicKeyInfo = public_key(Key), + subjectPublicKeyInfo = public_key(Key, SignAlgoId), extensions = extensions(Role, Type, Opts) }, public_key:pkix_sign(OTPTBS, PrivKey). @@ -1251,19 +1305,33 @@ default_key_gen() -> {namedCurve, Oid} end. -public_key(#'RSAPrivateKey'{modulus=N, publicExponent=E}) -> +public_key(#'RSAPrivateKey'{modulus=N, publicExponent=E}, + #'SignatureAlgorithm'{algorithm = ?rsaEncryption, + parameters = #'RSASSA-PSS-params'{} = Params}) -> + Public = #'RSAPublicKey'{modulus=N, publicExponent=E}, + Algo = #'PublicKeyAlgorithm'{algorithm= ?rsaEncryption, parameters = Params}, + #'OTPSubjectPublicKeyInfo'{algorithm = Algo, + subjectPublicKey = Public}; +public_key({#'RSAPrivateKey'{modulus=N, publicExponent=E}, #'RSASSA-PSS-params'{} = Params}, + #'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = #'RSASSA-PSS-params'{} = Params}) -> + Public = #'RSAPublicKey'{modulus=N, publicExponent=E}, + Algo = #'PublicKeyAlgorithm'{algorithm= ?'id-RSASSA-PSS', parameters= Params}, + #'OTPSubjectPublicKeyInfo'{algorithm = Algo, + subjectPublicKey = Public}; +public_key(#'RSAPrivateKey'{modulus=N, publicExponent=E}, _) -> Public = #'RSAPublicKey'{modulus=N, publicExponent=E}, Algo = #'PublicKeyAlgorithm'{algorithm= ?rsaEncryption, parameters='NULL'}, #'OTPSubjectPublicKeyInfo'{algorithm = Algo, subjectPublicKey = Public}; -public_key(#'DSAPrivateKey'{p=P, q=Q, g=G, y=Y}) -> +public_key(#'DSAPrivateKey'{p=P, q=Q, g=G, y=Y}, _) -> Algo = #'PublicKeyAlgorithm'{algorithm= ?'id-dsa', parameters={params, #'Dss-Parms'{p=P, q=Q, g=G}}}, #'OTPSubjectPublicKeyInfo'{algorithm = Algo, subjectPublicKey = Y}; public_key(#'ECPrivateKey'{version = _Version, privateKey = _PrivKey, parameters = Params, - publicKey = PubKey}) -> + publicKey = PubKey}, _) -> Algo = #'PublicKeyAlgorithm'{algorithm= ?'id-ecPublicKey', parameters=Params}, #'OTPSubjectPublicKeyInfo'{algorithm = Algo, subjectPublicKey = #'ECPoint'{point = PubKey}}. @@ -1306,6 +1374,9 @@ add_default_extensions(Defaults0, Exts) -> end, Defaults0), Exts ++ Defaults. +encode_key({#'RSAPrivateKey'{}, #'RSASSA-PSS-params'{}} = Key) -> + {Asn1Type, DER, _} = public_key:pem_entry_encode('PrivateKeyInfo', Key), + {Asn1Type, DER}; encode_key(#'RSAPrivateKey'{} = Key) -> {'RSAPrivateKey', public_key:der_encode('RSAPrivateKey', Key)}; encode_key(#'ECPrivateKey'{} = Key) -> @@ -1313,3 +1384,11 @@ encode_key(#'ECPrivateKey'{} = Key) -> encode_key(#'DSAPrivateKey'{} = Key) -> {'DSAPrivateKey', public_key:der_encode('DSAPrivateKey', Key)}. +verify_options(#'RSASSA-PSS-params'{saltLength = SaltLen, + maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = HashOid}}}) -> + HashAlgo = public_key:pkix_hash_type(HashOid), + [{rsa_padding, rsa_pkcs1_pss_padding}, + {rsa_pss_saltlen, SaltLen}, + {rsa_mgf1_md, HashAlgo}]. diff --git a/lib/public_key/src/pubkey_cert_records.erl b/lib/public_key/src/pubkey_cert_records.erl index cdb3723bf3..b556314ad1 100644 --- a/lib/public_key/src/pubkey_cert_records.erl +++ b/lib/public_key/src/pubkey_cert_records.erl @@ -110,7 +110,8 @@ supportedPublicKeyAlgorithms(?'rsaEncryption') -> 'RSAPublicKey'; supportedPublicKeyAlgorithms(?'id-dsa') -> 'DSAPublicKey'; supportedPublicKeyAlgorithms(?'dhpublicnumber') -> 'DHPublicKey'; supportedPublicKeyAlgorithms(?'id-keyExchangeAlgorithm') -> 'KEA-PublicKey'; -supportedPublicKeyAlgorithms(?'id-ecPublicKey') -> 'ECPoint'. +supportedPublicKeyAlgorithms(?'id-ecPublicKey') -> 'ECPoint'; +supportedPublicKeyAlgorithms(?'id-RSASSA-PSS') -> 'RSAPublicKey'. supportedCurvesTypes(?'characteristic-two-field') -> characteristic_two_field; supportedCurvesTypes(?'prime-field') -> prime_field; diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index a47b7148e7..e1f5f7576e 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -40,7 +40,8 @@ sign/3, sign/4, verify/4, verify/5, generate_key/1, compute_key/2, compute_key/3, - pkix_sign/2, pkix_verify/2, + pkix_sign/2, pkix_verify/2, + pkix_hash_type/1, pkix_sign_types/1, pkix_is_self_signed/1, pkix_is_fixed_dh_cert/1, @@ -68,11 +69,12 @@ pki_asn1_type/0, asn1_type/0, ssh_file/0, der_encoded/0, key_params/0, digest_type/0, issuer_name/0, oid/0]). --type public_key() :: rsa_public_key() | dsa_public_key() | ec_public_key() | ed_public_key() . --type private_key() :: rsa_private_key() | dsa_private_key() | ec_private_key() | ed_private_key() . - +-type public_key() :: rsa_public_key() | rsa_pss_public_key() | dsa_public_key() | ec_public_key() | ed_public_key() . +-type private_key() :: rsa_private_key() | rsa_pss_private_key() | dsa_private_key() | ec_private_key() | ed_private_key() . -type rsa_public_key() :: #'RSAPublicKey'{}. --type rsa_private_key() :: #'RSAPrivateKey'{}. +-type rsa_private_key() :: #'RSAPrivateKey'{}. +-type rsa_pss_public_key() :: {#'RSAPublicKey'{}, #'RSASSA-PSS-params'{}}. +-type rsa_pss_private_key() :: { #'RSAPrivateKey'{}, #'RSASSA-PSS-params'{}}. -type dsa_private_key() :: #'DSAPrivateKey'{}. -type dsa_public_key() :: {integer(), #'Dss-Parms'{}}. -type ecpk_parameters() :: {ecParameters, #'ECParameters'{}} | {namedCurve, Oid::tuple()}. @@ -293,6 +295,12 @@ der_priv_key_decode({'PrivateKeyInfo', v1, {'PrivateKeyInfo_privateKeyAlgorithm', ?'rsaEncryption', _}, PrivKey, _}) -> der_decode('RSAPrivateKey', PrivKey); der_priv_key_decode({'PrivateKeyInfo', v1, + {'PrivateKeyInfo_privateKeyAlgorithm', ?'id-RSASSA-PSS', + {asn1_OPENTYPE, Parameters}}, PrivKey, _}) -> + Key = der_decode('RSAPrivateKey', PrivKey), + Params = der_decode('RSASSA-PSS-params', Parameters), + {Key, Params}; +der_priv_key_decode({'PrivateKeyInfo', v1, {'PrivateKeyInfo_privateKeyAlgorithm', ?'id-dsa', {asn1_OPENTYPE, Parameters}}, PrivKey, _}) -> {params, #'Dss-Parms'{p=P, q=Q, g=G}} = der_decode('DSAParams', Parameters), X = der_decode('Prime-p', PrivKey), @@ -307,34 +315,40 @@ der_priv_key_decode(PKCS8Key) -> %% %% Description: Encodes a public key entity with asn1 DER encoding. %%-------------------------------------------------------------------- - der_encode('PrivateKeyInfo', #'DSAPrivateKey'{p=P, q=Q, g=G, x=X}) -> der_encode('PrivateKeyInfo', - {'PrivateKeyInfo', v1, - {'PrivateKeyInfo_privateKeyAlgorithm', ?'id-dsa', - {asn1_OPENTYPE, der_encode('Dss-Parms', #'Dss-Parms'{p=P, q=Q, g=G})}}, + {'PrivateKeyInfo', v1, + {'PrivateKeyInfo_privateKeyAlgorithm', ?'id-dsa', + {asn1_OPENTYPE, der_encode('Dss-Parms', #'Dss-Parms'{p=P, q=Q, g=G})}}, der_encode('Prime-p', X), asn1_NOVALUE}); der_encode('PrivateKeyInfo', #'RSAPrivateKey'{} = PrivKey) -> der_encode('PrivateKeyInfo', - {'PrivateKeyInfo', v1, - {'PrivateKeyInfo_privateKeyAlgorithm', ?'rsaEncryption', {asn1_OPENTYPE, ?DER_NULL}}, - der_encode('RSAPrivateKey', PrivKey), asn1_NOVALUE}); + {'PrivateKeyInfo', v1, + {'PrivateKeyInfo_privateKeyAlgorithm', ?'rsaEncryption', + {asn1_OPENTYPE, ?DER_NULL}}, + der_encode('RSAPrivateKey', PrivKey), asn1_NOVALUE}); +der_encode('PrivateKeyInfo', {#'RSAPrivateKey'{} = PrivKey, Parameters}) -> + der_encode('PrivateKeyInfo', + {'PrivateKeyInfo', v1, + {'PrivateKeyInfo_privateKeyAlgorithm', ?'id-RSASSA-PSS', + {asn1_OPENTYPE, der_encode('RSASSA-PSS-params', Parameters)}}, + der_encode('RSAPrivateKey', PrivKey), asn1_NOVALUE}); der_encode('PrivateKeyInfo', #'ECPrivateKey'{parameters = Parameters} = PrivKey) -> der_encode('PrivateKeyInfo', - {'PrivateKeyInfo', v1, + {'PrivateKeyInfo', v1, {'PrivateKeyInfo_privateKeyAlgorithm', ?'id-ecPublicKey', - {asn1_OPENTYPE, der_encode('EcpkParameters', Parameters)}}, - der_encode('ECPrivateKey', PrivKey#'ECPrivateKey'{parameters = asn1_NOVALUE}), asn1_NOVALUE}); + {asn1_OPENTYPE, der_encode('EcpkParameters', Parameters)}}, + der_encode('ECPrivateKey', PrivKey#'ECPrivateKey'{parameters = asn1_NOVALUE}), + asn1_NOVALUE}); der_encode(Asn1Type, Entity) when (Asn1Type == 'PrivateKeyInfo') or (Asn1Type == 'EncryptedPrivateKeyInfo') -> try - {ok, Encoded} = 'PKCS-FRAME':encode(Asn1Type, Entity), - Encoded - catch + {ok, Encoded} = 'PKCS-FRAME':encode(Asn1Type, Entity), + Encoded + catch error:{badmatch, {error, _}} = Error -> - erlang:error(Error) - end; - + erlang:error(Error) + end; der_encode(Asn1Type, Entity) when is_atom(Asn1Type) -> try {ok, Encoded} = 'OTP-PUB-KEY':encode(Asn1Type, Entity), @@ -625,6 +639,22 @@ pkix_sign_types(?'ecdsa-with-SHA512') -> {sha512, ecdsa}. %%-------------------------------------------------------------------- +-spec pkix_hash_type(HashOid::oid()) -> DigestType:: md5 | crypto:sha1() | crypto:sha2(). + +pkix_hash_type(?'id-sha1') -> + sha; +pkix_hash_type(?'id-sha512') -> + sha512; +pkix_hash_type(?'id-sha384') -> + sha384; +pkix_hash_type(?'id-sha256') -> + sha256; +pkix_hash_type('id-sha224') -> + sha224; +pkix_hash_type('id-md5') -> + md5. + +%%-------------------------------------------------------------------- %% Description: Create digital signature. %%-------------------------------------------------------------------- -spec sign(Msg, DigestType, Key) -> @@ -768,12 +798,11 @@ pkix_match_dist_point(#'CertificateList'{ %% der encoded 'Certificate'{} %%-------------------------------------------------------------------- pkix_sign(#'OTPTBSCertificate'{signature = - #'SignatureAlgorithm'{algorithm = Alg} + #'SignatureAlgorithm'{} = SigAlg} = TBSCert, Key) -> - Msg = pkix_encode('OTPTBSCertificate', TBSCert, otp), - {DigestType, _} = pkix_sign_types(Alg), - Signature = sign(Msg, DigestType, Key), + {DigestType, _, Opts} = pubkey_cert:x509_pkix_sign_types(SigAlg), + Signature = sign(Msg, DigestType, format_pkix_sign_key(Key), Opts), Cert = #'OTPCertificate'{tbsCertificate= TBSCert, signatureAlgorithm = SigAlg, signature = Signature @@ -796,6 +825,11 @@ pkix_verify(DerCert, #'RSAPublicKey'{} = RSAKey) {DigestType, PlainText, Signature} = pubkey_cert:verify_data(DerCert), verify(PlainText, DigestType, Signature, RSAKey); +pkix_verify(DerCert, {#'RSAPublicKey'{} = RSAKey, #'RSASSA-PSS-params'{} = Params}) + when is_binary(DerCert) -> + {DigestType, PlainText, Signature} = pubkey_cert:verify_data(DerCert), + verify(PlainText, DigestType, Signature, RSAKey, rsa_opts(Params)); + pkix_verify(DerCert, Key = {#'ECPoint'{}, _}) when is_binary(DerCert) -> {DigestType, PlainText, Signature} = pubkey_cert:verify_data(DerCert), @@ -1257,7 +1291,11 @@ set_padding(Pad, Opts) -> T =/= rsa_pad] ]. - +format_pkix_sign_key({#'RSAPrivateKey'{} = Key, _}) -> + %% Params are handled in option arg + Key; +format_pkix_sign_key(Key) -> + Key. format_sign_key(Key = #'RSAPrivateKey'{}) -> {rsa, format_rsa_private_key(Key)}; format_sign_key(#'DSAPrivateKey'{p = P, q = Q, g = G, x = X}) -> @@ -1289,6 +1327,15 @@ format_verify_key(#'DSAPrivateKey'{y=Y, p=P, q=Q, g=G}) -> format_verify_key(_) -> badarg. +rsa_opts(#'RSASSA-PSS-params'{maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = HashAlgoOid} + }}) -> + HashAlgo = pkix_hash_type(HashAlgoOid), + [{rsa_padding, rsa_pkcs1_pss_padding}, + {rsa_pss_saltlen, -1}, + {rsa_mgf1_md, HashAlgo}]. + do_pem_entry_encode(Asn1Type, Entity, CipherInfo, Password) -> Der = der_encode(Asn1Type, Entity), DecryptDer = pubkey_pem:cipher(Der, CipherInfo, Password), @@ -1776,3 +1823,4 @@ format_details([]) -> no_relevant_crls; format_details(Details) -> Details. + diff --git a/lib/public_key/test/public_key_SUITE.erl b/lib/public_key/test/public_key_SUITE.erl index 1fd1d2fa76..3b2f1b7184 100644 --- a/lib/public_key/test/public_key_SUITE.erl +++ b/lib/public_key/test/public_key_SUITE.erl @@ -45,7 +45,9 @@ all() -> pkix, pkix_countryname, pkix_emailaddress, pkix_path_validation, pkix_iso_rsa_oid, pkix_iso_dsa_oid, pkix_dsa_sha2_oid, - pkix_crl, general_name, + pkix_crl, + pkix_hash_type, + general_name, pkix_verify_hostname_cn, pkix_verify_hostname_subjAltName, pkix_verify_hostname_subjAltName_IP, @@ -56,13 +58,13 @@ all() -> ]. groups() -> - [{pem_decode_encode, [], [dsa_pem, rsa_pem, ec_pem, encrypted_pem, + [{pem_decode_encode, [], [dsa_pem, rsa_pem, rsa_pss_pss_pem, ec_pem, encrypted_pem, dh_pem, cert_pem, pkcs7_pem, pkcs10_pem, ec_pem2, rsa_priv_pkcs8, dsa_priv_pkcs8, ec_priv_pkcs8, ec_pem_encode_generated, gen_ec_param_prime_field, gen_ec_param_char_2_field ]}, - {sign_verify, [], [rsa_sign_verify, dsa_sign_verify]} + {sign_verify, [], [rsa_sign_verify, rsa_pss_sign_verify, dsa_sign_verify]} ]. %%------------------------------------------------------------------- init_per_suite(Config) -> @@ -101,6 +103,18 @@ init_per_testcase(gen_ec_param_prime_field=TC, Config) -> init_per_testcase(gen_ec_param_char_2_field=TC, Config) -> init_per_testcase_gen_ec_param(TC, sect571r1, Config); +init_per_testcase(rsa_pss_sign_verify, Config) -> + Supports = crypto:supports(), + RSAOpts = proplists:get_value(rsa_opts, Supports), + + case lists:member(rsa_pkcs1_pss_padding, RSAOpts) + andalso lists:member(rsa_pss_saltlen, RSAOpts) + andalso lists:member(rsa_mgf1_md, RSAOpts) of + true -> + Config; + false -> + {skip, not_supported_by_crypto} + end; init_per_testcase(TestCase, Config) -> case TestCase of ec_pem_encode_generated -> @@ -204,6 +218,19 @@ rsa_pem(Config) when is_list(Config) -> RSARawPemNoEndNewLines = strip_superfluous_newlines(RSARawPem), RSARawPemNoEndNewLines = strip_superfluous_newlines(public_key:pem_encode([PubEntry1])). +rsa_pss_pss_pem() -> + [{doc, "RSA PKCS8 RSASSA-PSS private key decode/encode"}]. +rsa_pss_pss_pem(Config) when is_list(Config) -> + Datadir = proplists:get_value(data_dir, Config), + {ok, RsaPem} = file:read_file(filename:join(Datadir, "rsa_pss_pss_key.pem")), + [{'PrivateKeyInfo', DerRSAKey, not_encrypted} = Entry0 ] = public_key:pem_decode(RsaPem), + {RSAKey, Parms} = public_key:der_decode('PrivateKeyInfo', DerRSAKey), + {RSAKey, Parms} = public_key:pem_entry_decode(Entry0), + true = check_entry_type(RSAKey, 'RSAPrivateKey'), + PrivEntry0 = public_key:pem_entry_encode('PrivateKeyInfo', {RSAKey, Parms}), + RSAPemNoEndNewLines = strip_superfluous_newlines(RsaPem), + RSAPemNoEndNewLines = strip_superfluous_newlines(public_key:pem_encode([PrivEntry0])). + rsa_priv_pkcs8() -> [{doc, "RSA PKCS8 private key decode/encode"}]. rsa_priv_pkcs8(Config) when is_list(Config) -> @@ -410,6 +437,24 @@ rsa_sign_verify(Config) when is_list(Config) -> true = public_key:verify(Msg, md5, RSASign1, PublicRSA). %%-------------------------------------------------------------------- +rsa_pss_sign_verify() -> + [{doc, "Checks that we can sign and verify rsa pss signatures."}]. +rsa_pss_sign_verify(Config) when is_list(Config) -> + CertChainConf = #{server_chain => + #{root => [{digest, sha256}, {hardcode_rsa_key(1), pss_params(sha256)}], + intermediates => [[]], + peer => [{digest, sha256}, {hardcode_rsa_key(2), pss_params(sha256)}]}, + client_chain => + #{root => [{digest, sha256}, {hardcode_rsa_key(3), pss_params(sha256)}], + intermediates => [[]], + peer => [{digest, sha256}, {hardcode_rsa_key(4), pss_params(sha256)}]}}, + #{client_config := ClientConf} = public_key:pkix_test_data(CertChainConf), + Cert = proplists:get_value(cert, ClientConf), + {#'RSAPrivateKey'{modulus=Mod, publicExponent=Exp}, Parms} = {hardcode_rsa_key(4), pss_params(sha256)}, + + public_key:pkix_verify(Cert, {#'RSAPublicKey'{modulus=Mod, publicExponent=Exp}, Parms}). + +%%-------------------------------------------------------------------- dsa_sign_verify() -> [{doc, "Checks that we can sign and verify dsa signatures."}]. @@ -848,6 +893,21 @@ general_name(Config) when is_list(Config) -> [{rfc822Name, DummyRfc822Name}], authorityCertSerialNumber = 1}). + +%%-------------------------------------------------------------------- + +pkix_hash_type() -> + [{doc, "Test API function pkix_hash_type/1"}]. + +pkix_hash_type(Config) when is_list(Config) -> + sha = public_key:pkix_hash_type(?'id-sha1'), + sha512 = public_key:pkix_hash_type(?'id-sha512'), + sha384 = public_key:pkix_hash_type(?'id-sha384'), + sha256 = public_key:pkix_hash_type(?'id-sha256'), + sha224 = public_key:pkix_hash_type('id-sha224'), + md5 = public_key:pkix_hash_type('id-md5'). + + %%-------------------------------------------------------------------- pkix_test_data_all_default() -> @@ -901,7 +961,7 @@ pkix_test_data(Config) when is_list(Config) -> public_key:pkix_test_data(#{server_chain => #{root => [], intermediates => [], - peer => [{key, hardcode_rsa_key()}]}, + peer => [{key, hardcode_rsa_key(1)}]}, client_chain => #{root => [{validity, {{Year-2, Month, Day}, {Year-1, Month, Day}}}], @@ -1089,9 +1149,7 @@ incorrect_countryname_pkix_cert() -> incorrect_emailaddress_pkix_cert() -> <<48,130,3,74,48,130,2,50,2,9,0,133,49,203,25,198,156,252,230,48,13,6,9,42,134, 72,134,247,13,1,1,5,5,0,48,103,49,11,48,9,6,3,85,4,6,19,2,65,85,49,19,48,17, 6,3,85,4,8,12,10,83,111,109,101,45,83,116,97,116,101,49,33,48,31,6,3,85,4,10, 12,24,73,110,116,101,114,110,101,116,32,87,105,100,103,105,116,115,32,80,116, 121,32,76,116,100,49,32,48,30,6,9,42,134,72,134,247,13,1,9,1,12,17,105,110, 118,97,108,105,100,64,101,109,97,105,108,46,99,111,109,48,30,23,13,49,51,49, 49,48,55,50,48,53,54,49,56,90,23,13,49,52,49,49,48,55,50,48,53,54,49,56,90, 48,103,49,11,48,9,6,3,85,4,6,19,2,65,85,49,19,48,17,6,3,85,4,8,12,10,83,111, 109,101,45,83,116,97,116,101,49,33,48,31,6,3,85,4,10,12,24,73,110,116,101, 114,110,101,116,32,87,105,100,103,105,116,115,32,80,116,121,32,76,116,100,49, 32,48,30,6,9,42,134,72,134,247,13,1,9,1,12,17,105,110,118,97,108,105,100,64, 101,109,97,105,108,46,99,111,109,48,130,1,34,48,13,6,9,42,134,72,134,247,13, 1,1,1,5,0,3,130,1,15,0,48,130,1,10,2,130,1,1,0,190,243,49,213,219,60,232,105, 1,127,126,9,130,15,60,190,78,100,148,235,246,223,21,91,238,200,251,84,55,212, 78,32,120,61,85,172,0,144,248,5,165,29,143,79,64,178,51,153,203,76,115,238, 192,49,173,37,121,203,89,62,157,13,181,166,30,112,154,40,202,140,104,211,157, 73,244,9,78,236,70,153,195,158,233,141,42,238,2,143,160,225,249,27,30,140, 151,176,43,211,87,114,164,108,69,47,39,195,123,185,179,219,28,218,122,53,83, 77,48,81,184,14,91,243,12,62,146,86,210,248,228,171,146,225,87,51,146,155, 116,112,238,212,36,111,58,41,67,27,6,61,61,3,84,150,126,214,121,57,38,12,87, 121,67,244,37,45,145,234,131,115,134,58,194,5,36,166,52,59,229,32,47,152,80, 237,190,58,182,248,98,7,165,198,211,5,31,231,152,116,31,108,71,218,64,188, 178,143,27,167,79,15,112,196,103,116,212,65,197,94,37,4,132,103,91,217,73, 223,207,185,7,153,221,240,232,31,44,102,108,82,83,56,242,210,214,74,71,246, 177,217,148,227,220,230,4,176,226,74,194,37,2,3,1,0,1,48,13,6,9,42,134,72, 134,247,13,1,1,5,5,0,3,130,1,1,0,89,247,141,154,173,123,123,203,143,85,28,79, 73,37,164,6,17,89,171,224,149,22,134,17,198,146,158,192,241,41,253,58,230, 133,71,189,43,66,123,88,15,242,119,227,249,99,137,61,200,54,161,0,177,167, 169,114,80,148,90,22,97,78,162,181,75,93,209,116,245,46,81,232,64,157,93,136, 52,57,229,113,197,218,113,93,42,161,213,104,205,137,30,144,183,58,10,98,47, 227,177,96,40,233,98,150,209,217,68,22,221,133,27,161,152,237,46,36,179,59, 172,97,134,194,205,101,137,71,192,57,153,20,114,27,173,233,166,45,56,0,61, 205,45,202,139,7,132,103,248,193,157,184,123,43,62,172,236,110,49,62,209,78, 249,83,219,133,1,213,143,73,174,16,113,143,189,41,84,60,128,222,30,177,104, 134,220,52,239,171,76,59,176,36,113,176,214,118,16,44,235,21,167,199,216,200, 76,219,142,248,13,70,145,205,216,230,226,148,97,223,216,179,68,209,222,63, 140,137,24,164,192,149,194,79,119,247,75,159,49,116,70,241,70,116,11,40,119, 176,157,36,160,102,140,255,34,248,25,231,136,59>>. - - -hardcode_rsa_key() -> +hardcode_rsa_key(1) -> #'RSAPrivateKey'{ version = 'two-prime', modulus = 23995666614853919027835084074500048897452890537492185072956789802729257783422306095699263934587064480357348855732149402060270996295002843755712064937715826848741191927820899197493902093529581182351132392364214171173881547273475904587683433713767834856230531387991145055273426806331200574039205571401702219159773947658558490957010003143162250693492642996408861265758000254664396313741422909188635443907373976005987612936763564996605457102336549804831742940035613780926178523017685712710473543251580072875247250504243621640157403744718833162626193206685233710319205099867303242759099560438381385658382486042995679707669, @@ -1102,4 +1160,51 @@ hardcode_rsa_key() -> exponent1 = 119556097830058336212015217380447172615655659108450823901745048534772786676204666783627059584226579481512852103690850928442711896738555003036938088452023283470698275450886490965004917644550167427154181661417665446247398284583687678213495921811770068712485038160606780733330990744565824684470897602653233516609, exponent2 = 41669135975672507953822256864985956439473391144599032012999352737636422046504414744027363535700448809435637398729893409470532385959317485048904982111185902020526124121798693043976273393287623750816484427009887116945685005129205106462566511260580751570141347387612266663707016855981760014456663376585234613993, coefficient = 76837684977089699359024365285678488693966186052769523357232308621548155587515525857011429902602352279058920284048929101483304120686557782043616693940283344235057989514310975192908256494992960578961614059245280827077951132083993754797053182279229469590276271658395444955906108899267024101096069475145863928441, + otherPrimeInfos = asn1_NOVALUE}; + +hardcode_rsa_key(2) -> + #'RSAPrivateKey'{ + version = 'two-prime', + modulus = 21343679768589700771839799834197557895311746244621307033143551583788179817796325695589283169969489517156931770973490560582341832744966317712674900833543896521418422508485833901274928542544381247956820115082240721897193055368570146764204557110415281995205343662628196075590438954399631753508888358737971039058298703003743872818150364935790613286541190842600031570570099801682794056444451081563070538409720109449780410837763602317050353477918147758267825417201591905091231778937606362076129350476690460157227101296599527319242747999737801698427160817755293383890373574621116766934110792127739174475029121017282777887777, + publicExponent = 17, + privateExponent = 18832658619343853622211588088997845201745658451136447382185486691577805721584993260814073385267196632785528033211903435807948675951440868570007265441362261636545666919252206383477878125774454042314841278013741813438699754736973658909592256273895837054592950290554290654932740253882028017801960316533503857992358685308186680144968293076156011747178275038098868263178095174694099811498968993700538293188879611375604635940554394589807673542938082281934965292051746326331046224291377703201248790910007232374006151098976879987912446997911775904329728563222485791845480864283470332826504617837402078265424772379987120023773, + prime1 = 146807662748886761089048448970170315054939768171908279335181627815919052012991509112344782731265837727551849787333310044397991034789843793140419387740928103541736452627413492093463231242466386868459637115999163097726153692593711599245170083315894262154838974616739452594203727376460632750934355508361223110419, + prime2 = 145385325050081892763917667176962991350872697916072592966410309213561884732628046256782356731057378829876640317801978404203665761131810712267778698468684631707642938779964806354584156202882543264893826268426566901882487709510744074274965029453915224310656287149777603803201831202222853023280023478269485417083, + exponent1 = 51814469205489445090252393754177758254684624060673510353593515699736136004585238510239335081623236845018299924941168250963996835808180162284853901555621683602965806809675350150634081614988136541809283687999704622726877773856604093851236499993845033701707873394143336209718962603456693912094478414715725803677, + exponent2 = 51312467664734785681382706062457526359131540440966797517556579722433606376221663384746714140373192528191755406283051201483646739222992016094510128871300458249756331334105225772206172777487956446433115153562317730076172132768497908567634716277852432109643395464627389577600646306666889302334125933506877206029, + coefficient = 30504662229874176232343608562807118278893368758027179776313787938167236952567905398252901545019583024374163153775359371298239336609182249464886717948407152570850677549297935773605431024166978281486607154204888016179709037883348099374995148481968169438302456074511782717758301581202874062062542434218011141540, + otherPrimeInfos = asn1_NOVALUE}; +hardcode_rsa_key(3) -> + #'RSAPrivateKey'{ + version = 'two-prime', + modulus = 25089040456112869869472694987833070928503703615633809313972554887193090845137746668197820419383804666271752525807484521370419854590682661809972833718476098189250708650325307850184923546875260207894844301992963978994451844985784504212035958130279304082438876764367292331581532569155681984449177635856426023931875082020262146075451989132180409962870105455517050416234175675478291534563995772675388370042873175344937421148321291640477650173765084699931690748536036544188863178325887393475703801759010864779559318631816411493486934507417755306337476945299570726975433250753415110141783026008347194577506976486290259135429, + publicExponent = 17, + privateExponent = 8854955455098659953931539407470495621824836570223697404931489960185796768872145882893348383311931058684147950284994536954265831032005645344696294253579799360912014817761873358888796545955974191021709753644575521998041827642041589721895044045980930852625485916835514940558187965584358347452650930302268008446431977397918214293502821599497633970075862760001650736520566952260001423171553461362588848929781360590057040212831994258783694027013289053834376791974167294527043946669963760259975273650548116897900664646809242902841107022557239712438496384819445301703021164043324282687280801738470244471443835900160721870265, + prime1 = 171641816401041100605063917111691927706183918906535463031548413586331728772311589438043965564336865070070922328258143588739626712299625805650832695450270566547004154065267940032684307994238248203186986569945677705100224518137694769557564475390859269797990555863306972197736879644001860925483629009305104925823, + prime2 =146170909759497809922264016492088453282310383272504533061020897155289106805616042710009332510822455269704884883705830985184223718261139908416790475825625309815234508695722132706422885088219618698987115562577878897003573425367881351537506046253616435685549396767356003663417208105346307649599145759863108910523, + exponent1 = 60579464612132153154728441333538327425711971378777222246428851853999433684345266860486105493295364142377972586444050678378691780811632637288529186629507258781295583787741625893888579292084087601124818789392592131211843947578009918667375697196773859928702549128225990187436545756706539150170692591519448797349, + exponent2 = 137572620950115585809189662580789132500998007785886619351549079675566218169991569609420548245479957900898715184664311515467504676010484619686391036071176762179044243478326713135456833024206699951987873470661533079532774988581535389682358631768109586527575902839864474036157372334443583670210960715165278974609, + coefficient = 15068630434698373319269196003209754243798959461311186548759287649485250508074064775263867418602372588394608558985183294561315208336731894947137343239541687540387209051236354318837334154993136528453613256169847839789803932725339395739618592522865156272771578671216082079933457043120923342632744996962853951612, + otherPrimeInfos = asn1_NOVALUE}; +hardcode_rsa_key(4) -> + #'RSAPrivateKey'{ + version ='two-prime', + modulus = 28617237755030755643854803617273584643843067580642149032833640135949799721163782522787597288521902619948688786051081993247908700824196122780349730169173433743054172191054872553484065655968335396052034378669869864779940355219732200954630251223541048434478476115391643898092650304645086338265930608997389611376417609043761464100338332976874588396803891301015812818307951159858145399281035705713082131199940309445719678087542976246147777388465712394062188801177717719764254900022006288880246925156931391594131839991579403409541227225173269459173129377291869028712271737734702830877034334838181789916127814298794576266389, + publicExponent = 17, + privateExponent = 26933870828264240605980991639786903194205240075898493207372837775011576208154148256741268036255908348187001210401018346586267012540419880263858569570986761169933338532757527109161473558558433313931326474042230460969355628442100895016122589386862163232450330461545076609969553227901257730132640573174013751883368376011370428995523268034111482031427024082719896108094847702954695363285832195666458915142143884210891427766607838346722974883433132513540317964796373298134261669479023445911856492129270184781873446960437310543998533283339488055776892320162032014809906169940882070478200435536171854883284366514852906334641, + prime1 = 177342190816702392178883147766999616783253285436834252111702533617098994535049411784501174309695427674025956656849179054202187436663487378682303508229883753383891163725167367039879190685255046547908384208614573353917213168937832054054779266431207529839577747601879940934691505396807977946728204814969824442867, + prime2 = 161367340863680900415977542864139121629424927689088951345472941851682581254789586032968359551717004797621579428672968948552429138154521719743297455351687337112710712475376510559020211584326773715482918387500187602625572442687231345855402020688502483137168684570635690059254866684191216155909970061793538842967, + exponent1 = 62591361464718491357252875682470452982324688977706206627659717747211409835899792394529826226951327414362102349476180842659595565881230839534930649963488383547255704844176717778780890830090016428673547367746320007264898765507470136725216211681602657590439205035957626212244060728285168687080542875871702744541, + exponent2 = 28476589564178982426348978152495139111074987239250991413906989738532220221433456358759122273832412611344984605059935696803369847909621479954699550944415412431654831613301737157474154985469430655673456186029444871051571607533040825739188591886206320553618003159523945304574388238386685203984112363845918619347, + coefficient = 34340318160575773065401929915821192439103777558577109939078671096408836197675640654693301707202885840826672396546056002756167635035389371579540325327619480512374920136684787633921441576901246290213545161954865184290700344352088099063404416346968182170720521708773285279884132629954461545103181082503707725012, otherPrimeInfos = asn1_NOVALUE}. + +pss_params(sha256) -> + #'RSASSA-PSS-params'{ + hashAlgorithm = #'HashAlgorithm'{algorithm = ?'id-sha256'}, + maskGenAlgorithm = #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = ?'id-sha256'} + }, + saltLength = 32, + trailerField = 1}. + diff --git a/lib/public_key/test/public_key_SUITE_data/rsa_pss_pss_key.pem b/lib/public_key/test/public_key_SUITE_data/rsa_pss_pss_key.pem new file mode 100644 index 0000000000..65032269c1 --- /dev/null +++ b/lib/public_key/test/public_key_SUITE_data/rsa_pss_pss_key.pem @@ -0,0 +1,29 @@ +-----BEGIN PRIVATE KEY----- +MIIE7wIBADA9BgkqhkiG9w0BAQowMKANMAsGCWCGSAFlAwQCAaEaMBgGCSqGSIb3 +DQEBCDALBglghkgBZQMEAgGiAwIBIASCBKkwggSlAgEAAoIBAQDDlygksUEAajpd +Vquo9XIAyTd9ZJ+55hNmhBfhn3lHz3ryPD+0XlgCE9qsKwfR7iYaqmnNilQnsxWp +MGXAgOlC1+w5zh8qHvrI5wX+A6U9N8leIOSgFuFNP0FMMG7I677QzRxGFqKX1o4V +73JWqnHCfnfHRyZY9xM0tYbJKNbRO7Hy4jKBPl3ptPHUoTltr4WYTOpgstcEamdi +iif+0U4bQvVltNg9pzFEjkAktTUGn92W5CgLnsbPXxBo6a/kUlHcgmhYbpOXEjCP +ufZLgsQo8iF2Bq8eWMEsByjr0chQjzrfZAUVtD8Hmh2uMVAPQFAHUkaLj2tHukL+ +s9tAaWKNAgMBAAECggEBAIzgfwWOtmb6HHfGSXY085wlUlZ696EKWsboNdtI5i4W +/1Mimi/sFC/K5SJFDCjlA4UJYZOuItdFYkCun1t8foaqx3cLQ98u2SuDWwmOzqG9 +YMjvoDy+viDJgtrBt8n4I0R5t/ezrgD3hPe/s/dAZRfVx6g9Ux2ZOLgqV57kT3X7 +6paEz3jrIMvuoXQCsi9Qh+eJQ23/sAcc7OHQ7uD8QJVudEBnSHQ+ttvOPXhr7tba +8NuNVa6E/KewkKHRAZqBTJolCVyPtWmvfaDwdJtunCvyR1w3Rv1adZLK4YRFz+vc +sOMK+K1c2aojA+/Fnba19inNq13j6Dwqmq8Ho7MZwHECgYEA6aSx7/93S1VGpxQ9 +KqFE4Fy9ylliC/hanc9qOcfEIo0tDus9lfpuPp+aOXML0msVkIfhCnaru32qtnaI +AQkIbPhSZFvC/i6BibpArXINbDzTS/46zZHehXskjWFGw+iRm/YI7MBuCmWzSnFO +YUwSKRIPKZKyXswFzP8RsQO/QbsCgYEA1k5SamQheuKdo/X40ShWTTOoDlpL4Sir +b2zTnEqlHyMv8c7w880hPf4P+0pqrKyf7jmEykJvp1qSAmyMUCWzrKTr8gQ2sMyb +zj90cEm++M5YIQh5lPJy4pGqmCliJXqkt+zT1xmnRASwMNQOnU2bBmXkve/ofb4M +dEwyig/nZFcCgYBLWPilTD6dhce+NBGxwMZkkKQIMKEk+RfIEs7QCXNgLSUdzZFT +36pT+caTxl1Go5AVxyw04qZpVZKLO1iK9O3Jrp9rjAgrTrYpw23+QWzAvjDqLfeq +ueMIKvlTus5GeacTo9mm+DvEkJ2sYTQEvrKQmilXn950IdmxDYUYD/xK5wKBgQDQ +5ON9BUGFUSQsUHVLG7CT7EhiRS41ubjyEfhrHm+53Ei9weQpIcjHbsERR8aXrmTu +h26i4QOI88XjSv+ymC19mfzLmcPdrnQpJL1RPvFCAZDyEhrBT1sg8rCBRcV/lv68 +scMEpuLecFt2HR5pwt3b7LJ9Wj8bYoctTaDt5va8XQKBgQDCr4hZB5haAcKmNm/g +PjlaLdrDEIuuBjxMzX1t3PXwsEene1cE731v6fbmrDUa8AuJyMY80xhGrTTDQfS3 +QOu/6wtcUv/JC/06OwEaUlT/kdYek+zYfBm3b1sKP3HVKSxCLTcPcC4aQoAFqbEy +3kuSVh03vVBdaP//qMPyeue17w== +-----END PRIVATE KEY----- diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 0008cf7a16..0a4bb38f70 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 1.7.2 +PUBLIC_KEY_VSN = 1.8 diff --git a/lib/reltool/test/reltool_test_lib.erl b/lib/reltool/test/reltool_test_lib.erl index 033d952d0a..88e5244b8b 100644 --- a/lib/reltool/test/reltool_test_lib.erl +++ b/lib/reltool/test/reltool_test_lib.erl @@ -238,7 +238,7 @@ wait_for_close() -> end. erl_libs() -> - lists:sort([filename:absname(P) || P<-reltool_utils:erl_libs()]). + [filename:absname(P) || P<-reltool_utils:erl_libs()]. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% A small test server, which can be run standalone in a shell diff --git a/lib/runtime_tools/doc/src/notes.xml b/lib/runtime_tools/doc/src/notes.xml index f6cc85b4a0..c259f890ba 100644 --- a/lib/runtime_tools/doc/src/notes.xml +++ b/lib/runtime_tools/doc/src/notes.xml @@ -32,6 +32,27 @@ <p>This document describes the changes made to the Runtime_Tools application.</p> +<section><title>Runtime_Tools 1.15</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Improved the presentation of allocations and carriers + in the <c>instrument</c> module.</p> + <p> + Own Id: OTP-16327</p> + </item> + <item> + <p> + Minor updates due to the new spawn improvements made.</p> + <p> + Own Id: OTP-16368 Aux Id: OTP-15251 </p> + </item> + </list> + </section> + +</section> + <section><title>Runtime_Tools 1.14</title> <section><title>Improvements and New Features</title> diff --git a/lib/runtime_tools/src/runtime_tools.app.src b/lib/runtime_tools/src/runtime_tools.app.src index b55d50d040..d7c2975a5b 100644 --- a/lib/runtime_tools/src/runtime_tools.app.src +++ b/lib/runtime_tools/src/runtime_tools.app.src @@ -29,5 +29,5 @@ {applications, [kernel, stdlib]}, {env, []}, {mod, {runtime_tools, []}}, - {runtime_dependencies, ["stdlib-@OTP-15251@","mnesia-4.12","kernel-@OTP-15251@", - "erts-@OTP-15251:OTP-16327@"]}]}. + {runtime_dependencies, ["stdlib-3.13","mnesia-4.12","kernel-7.0", + "erts-11.0"]}]}. diff --git a/lib/runtime_tools/vsn.mk b/lib/runtime_tools/vsn.mk index c01dd60009..4bacd1f571 100644 --- a/lib/runtime_tools/vsn.mk +++ b/lib/runtime_tools/vsn.mk @@ -1 +1 @@ -RUNTIME_TOOLS_VSN = 1.14 +RUNTIME_TOOLS_VSN = 1.15 diff --git a/lib/sasl/doc/specs/.gitignore b/lib/sasl/doc/specs/.gitignore new file mode 100644 index 0000000000..322eebcb06 --- /dev/null +++ b/lib/sasl/doc/specs/.gitignore @@ -0,0 +1 @@ +specs_*.xml diff --git a/lib/sasl/doc/src/Makefile b/lib/sasl/doc/src/Makefile index 684fd2b5e4..249b278f6b 100644 --- a/lib/sasl/doc/src/Makefile +++ b/lib/sasl/doc/src/Makefile @@ -53,6 +53,8 @@ XML_FILES = \ $(XML_PART_FILES) $(XML_REF3_FILES) $(XML_REF4_FILES) \ $(XML_REF6_FILES) $(XML_APPLICATION_FILES) +TOP_SPECS_FILE = specs.xml + # ---------------------------------------------------- include $(ERL_TOP)/make/doc.mk diff --git a/lib/sasl/doc/src/notes.xml b/lib/sasl/doc/src/notes.xml index 5982ce005d..71705080be 100644 --- a/lib/sasl/doc/src/notes.xml +++ b/lib/sasl/doc/src/notes.xml @@ -31,6 +31,45 @@ </header> <p>This document describes the changes made to the SASL application.</p> +<section><title>SASL 4.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Remove usage and documentation of old requests of the + I/O-protocol.</p> + <p> + Own Id: OTP-15695</p> + </item> + <item> + <p> + <c>systools:make_script/2</c> now accepts the name of the + boot file to create, it is not restricted to only + <c>RelName.boot</c> or <c>start.boot</c>.</p> + <p> + <c>systools:make_tar/2</c> now accepts the option + <c>extra_files</c> to add any extra non release related + files to the tar file.</p> + <p> + Own Id: OTP-16561 Aux Id: PR-2420 </p> + </item> + <item> + <p> + <seemfa + marker="systools#make_tar/1"><c>systools:make_tar/1,2</c></seemfa> + now filters out any tools from erts if included in the + release tar ball. See the documentation for more details.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16603</p> + </item> + </list> + </section> + +</section> + <section><title>SASL 3.4.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/sasl/doc/src/specs.xml b/lib/sasl/doc/src/specs.xml new file mode 100644 index 0000000000..f64b7df114 --- /dev/null +++ b/lib/sasl/doc/src/specs.xml @@ -0,0 +1,4 @@ +<?xml version="1.0" encoding="utf-8" ?> +<specs xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="../specs/specs_systools.xml"/> +</specs> diff --git a/lib/sasl/doc/src/systools.xml b/lib/sasl/doc/src/systools.xml index 483445f954..13878d379a 100644 --- a/lib/sasl/doc/src/systools.xml +++ b/lib/sasl/doc/src/systools.xml @@ -251,8 +251,8 @@ warnings are issued for calls to undefined functions.</p> <p>By default, errors and warnings are printed to tty and the function returns <c>ok</c> or <c>error</c>. If option - <c>silent</c> is specified, the function instead returns <c>{ok,Module,Warnings}</c> or <c>{error,Module,Error}</c>. + <c>silent</c> is specified, the function instead returns Warnings and errors can be converted to strings by calling <c>Module:format_warning(Warnings)</c> or <c>Module:format_error(Error)</c>.</p> @@ -265,25 +265,9 @@ </func> <func> - <name since="">make_tar(Name) -> Result</name> - <name since="">make_tar(Name, [Opt]) -> Result</name> + <name name="make_tar" arity="1" since=""/> + <name name="make_tar" arity="2" since=""/> <fsummary>Creates a release package.</fsummary> - <type> - <v>Name = string()</v> - <v>Opt = {dirs,[IncDir]} | {path,[Dir]} | {variables,[Var]} | {var_tar,VarTar} | {erts,Dir} | src_tests | exref | {exref,[App]} | silent | {outdir,Dir} | | no_warn_sasl | warnings_as_errors | {extra_files, ExtraFiles}</v> - <v> Dir = string()</v> - <v> IncDir = src | include | atom()</v> - <v> Var = {VarName,PreFix}</v> - <v> VarName = Prefix = string()</v> - <v> VarTar = include | ownfile | omit</v> - <v> Machine = atom()</v> - <v> App = atom()</v> - <v>Result = ok | error | {ok,Module,Warnings} | {error,Module,Error}</v> - <v> Module = atom()</v> - <v> Warning = Error = term()</v> - <v> ExtraFiles = [{NameInArchive, file:filename_all()}]</v> - <v> NameInArchive = string()</v> - </type> <desc> <p>Creates a release package file <c>Name.tar.gz</c>. This file must be uncompressed and unpacked on the target @@ -364,8 +348,10 @@ myapp-1/ebin/myapp.app specified using option <c>path</c>. In the case of <c>sys.config</c> it is not included if <c>sys.config.src</c> is found.</p> <p>If the release package is to contain a new Erlang runtime - system, the <c>bin</c> directory of the specified runtime - system <c>{erts,Dir}</c> is copied to <c>erts-ErtsVsn/bin</c>.</p> + system, the <c>erts-ErtsVsn/bin</c> directory of the specified runtime + system <c>{erts,Dir}</c> is copied to <c>erts-ErtsVsn/bin</c>. Some + erts executables are not copied by default, if you want to include all + executables you can give the <c>erts_all</c> option.</p> <p>All checks with function <seemfa marker="#make_script/1"><c>make_script</c></seemfa> are performed before the release package is created. diff --git a/lib/sasl/src/sasl.appup.src b/lib/sasl/src/sasl.appup.src index b795123645..621454d6a4 100644 --- a/lib/sasl/src/sasl.appup.src +++ b/lib/sasl/src/sasl.appup.src @@ -21,6 +21,7 @@ %% versions from the following OTP releases: %% - OTP 21 %% - OTP 22 +%% - OTP 23 %% %% We also allow upgrade from, and downgrade to all %% versions that have branched off from the above @@ -34,7 +35,8 @@ {<<"^3\\.3\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.4$">>,[restart_new_emulator]}, {<<"^3\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], + {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^3\\.4\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}], [{<<"^3\\.2$">>,[restart_new_emulator]}, {<<"^3\\.2\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.2\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -42,4 +44,5 @@ {<<"^3\\.3\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.4$">>,[restart_new_emulator]}, {<<"^3\\.4\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, - {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. + {<<"^3\\.4\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, + {<<"^3\\.4\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}]}. diff --git a/lib/sasl/src/systools.erl b/lib/sasl/src/systools.erl index 34eca6679f..a6aa4919ec 100644 --- a/lib/sasl/src/systools.erl +++ b/lib/sasl/src/systools.erl @@ -62,8 +62,32 @@ make_script(RelName, Opt) -> %% release package and erts specifies that the erts-Vsn/bin directory %% should be included in the release package and there it can be found. %%----------------------------------------------------------------- +-spec make_tar(Name) -> Result when + Name :: string(), + Result :: ok | error | {ok, Module :: module(), Warnings :: term()} | + {error, Module :: module(), Error :: term()}. make_tar(RelName) -> make_tar(RelName, []). +-spec make_tar(Name, Opts) -> Result when + Name :: string(), + Opts :: [Opt], + Opt :: {dirs,[IncDir]} | {path,[Dir]} | + {variables,[Var]} | {var_tar,VarTar} | + {erts,Dir} | erts_all | src_tests | exref | + {exref,[App]} | silent | {outdir,Dir} | + no_warn_sasl | warnings_as_errors | + {extra_files, ExtraFiles}, + Dir :: file:filename_all(), + IncDir :: src | include | atom(), + Var :: {VarName,PreFix}, + VarName :: string(), + PreFix :: string(), + VarTar :: include | ownfile | omit, + App :: atom(), + Result :: ok | error | {ok, Module :: module(), Warnings :: term()} | + {error, Module :: module(), Error :: term()}, + ExtraFiles :: [{NameInArchive, file:filename_all()}], + NameInArchive :: string(). make_tar(RelName, Opt) -> systools_make:make_tar(RelName, Opt). diff --git a/lib/sasl/src/systools_make.erl b/lib/sasl/src/systools_make.erl index 1565366d92..a371239823 100644 --- a/lib/sasl/src/systools_make.erl +++ b/lib/sasl/src/systools_make.erl @@ -27,8 +27,7 @@ -export([format_error/1, format_warning/1]). --export([read_release/2, get_release/2, get_release/3, - get_release/4, pack_app/1]). +-export([read_release/2, get_release/2, get_release/3, pack_app/1]). -export([read_application/4]). @@ -47,11 +46,9 @@ -compile({inline,[{badarg,2}]}). -ifdef(USE_ESOCK). --define(ESOCK_SOCKET_MODS, [socket, socket_registry]). --define(ESOCK_NET_MODS, [prim_net]). +-define(ESOCK_MODS, [prim_net,prim_socket,socket_registry]). -else. --define(ESOCK_SOCKET_MODS, []). --define(ESOCK_NET_MODS, []). +-define(ESOCK_MODS, []). -endif. @@ -67,7 +64,6 @@ %% New options: {path,Path} can contain wildcards %% src_tests %% {variables,[{Name,AbsString}]} -%% {machine, jam | beam | vee} %% exref | {exref, [AppName]} %% no_warn_sasl %%----------------------------------------------------------------- @@ -100,7 +96,7 @@ make_script(RelName, Output, Flags) when is_list(RelName), Path1 = mk_path(Path0), % expand wildcards etc. Path = make_set(Path1 ++ code:get_path()), ModTestP = {member(src_tests, Flags),xref_p(Flags)}, - case get_release(RelName, Path, ModTestP, machine(Flags)) of + case get_release(RelName, Path, ModTestP) of {ok, Release, Appls, Warnings0} -> Warnings = wsasl(Flags, Warnings0), case systools_lib:werror(Flags, Warnings) of @@ -139,12 +135,6 @@ wsasl(Options, Warnings) -> badarg(BadArg, Args) -> erlang:error({badarg,BadArg}, Args). -machine(Flags) -> - case get_flag(machine,Flags) of - {machine, Machine} when is_atom(Machine) -> Machine; - _ -> false - end. - get_script_name(RelName, Flags) -> case get_flag(script_name,Flags) of {script_name,ScriptName} when is_list(ScriptName) -> ScriptName; @@ -364,7 +354,6 @@ add_apply_upgrade(Script,Args) -> %% src_tests %% exref | {exref, [AppName]} %% {variables,[{Name,AbsString}]} -%% {machine, jam | beam | vee} %% {var_tar, include | ownfile | omit} %% no_warn_sasl %% warnings_as_errors @@ -400,7 +389,7 @@ make_tar(RelName, Flags) when is_list(RelName), is_list(Flags) -> Path1 = mk_path(Path0), Path = make_set(Path1 ++ code:get_path()), ModTestP = {member(src_tests, Flags),xref_p(Flags)}, - case get_release(RelName, Path, ModTestP, machine(Flags)) of + case get_release(RelName, Path, ModTestP) of {ok, Release, Appls, Warnings0} -> Warnings = wsasl(Flags, Warnings0), case systools_lib:werror(Flags, Warnings) of @@ -432,17 +421,13 @@ make_tar(RelName, Flags) -> %%______________________________________________________________________ %% get_release(File, Path) -> %% get_release(File, Path, ModTestP) -> -%% get_release(File, Path, ModTestP, Machine) -> %% {ok, #release, [{{Name,Vsn},#application}], Warnings} | {error, What} get_release(File, Path) -> - get_release(File, Path, {false,false}, false). + get_release(File, Path, {false,false}). get_release(File, Path, ModTestP) -> - get_release(File, Path, ModTestP, false). - -get_release(File, Path, ModTestP, Machine) -> - case catch get_release1(File, Path, ModTestP, Machine) of + case catch get_release1(File, Path, ModTestP) of {error, Error} -> {error, ?MODULE, Error}; {'EXIT', Why} -> @@ -451,12 +436,12 @@ get_release(File, Path, ModTestP, Machine) -> Answer end. -get_release1(File, Path, ModTestP, Machine) -> +get_release1(File, Path, ModTestP) -> {ok, Release, Warnings1} = read_release(File, Path), {ok, Appls0} = collect_applications(Release, Path), {ok, Appls1} = check_applications(Appls0), {ok, Appls2} = sort_used_and_incl_appls(Appls1, Release), % OTP-4121, OTP-9984 - {ok, Warnings2} = check_modules(Appls2, Path, ModTestP, Machine), + {ok, Warnings2} = check_modules(Appls2, Path, ModTestP), {ok, Appls} = sort_appls(Appls2), {ok, Release, Appls, Warnings1 ++ Warnings2}. @@ -976,13 +961,13 @@ find_pos(N, Name, [_OtherAppl|OrderedAppls]) -> find_pos(N+1, Name, OrderedAppls). %%______________________________________________________________________ -%% check_modules(Appls, Path, TestP, Machine) -> +%% check_modules(Appls, Path, TestP) -> %% {ok, Warnings} | throw({error, What}) %% where Appls = [{App,Vsn}, #application}] %% performs logical checking that we can find all the modules %% etc. -check_modules(Appls, Path, TestP, Machine) -> +check_modules(Appls, Path, TestP) -> %% first check that all the module names are unique %% Make a list M1 = [{Mod,App,Dir}] M1 = [{Mod,App,A#application.dir} || @@ -990,7 +975,7 @@ check_modules(Appls, Path, TestP, Machine) -> Mod <- A#application.modules], case duplicates(M1) of [] -> - case check_mods(M1, Appls, Path, TestP, Machine) of + case check_mods(M1, Appls, Path, TestP) of {error, Errors} -> throw({error, {modules, Errors}}); Return -> @@ -1006,8 +991,8 @@ check_modules(Appls, Path, TestP, Machine) -> %% Use the module extension of the running machine as extension for %% the checked modules. -check_mods(Modules, Appls, Path, {SrcTestP, XrefP}, Machine) -> - SrcTestRes = check_src(Modules, Appls, Path, SrcTestP, Machine), +check_mods(Modules, Appls, Path, {SrcTestP, XrefP}) -> + SrcTestRes = check_src(Modules, Appls, Path, SrcTestP), XrefRes = check_xref(Appls, Path, XrefP), Res = SrcTestRes ++ XrefRes, case filter(fun({error, _}) -> true; @@ -1023,8 +1008,8 @@ check_mods(Modules, Appls, Path, {SrcTestP, XrefP}, Machine) -> {error, Errors} end. -check_src(Modules, Appls, Path, true, Machine) -> - Ext = objfile_extension(Machine), +check_src(Modules, Appls, Path, true) -> + Ext = code:objfile_extension(), IncPath = create_include_path(Appls, Path), append(map(fun(ModT) -> {Mod,App,Dir} = ModT, @@ -1038,7 +1023,7 @@ check_src(Modules, Appls, Path, true, Machine) -> end end, Modules)); -check_src(_, _, _, _, _) -> +check_src(_, _, _, _) -> []. check_xref(_Appls, _Path, false) -> @@ -1136,11 +1121,6 @@ exists_xref(Flag) -> _ -> Flag end. -objfile_extension(false) -> - code:objfile_extension(); -objfile_extension(Machine) -> - "." ++ atom_to_list(Machine). - check_mod(Mod,App,Dir,Ext,IncPath) -> ObjFile = mod_to_filename(Dir, Mod, Ext), case file:read_file_info(ObjFile) of @@ -1578,12 +1558,23 @@ mandatory_modules() -> %% This is the modules that are preloaded into the Erlang system. preloaded() -> - %% Sorted - [atomics,counters,erl_init,erl_prim_loader,erl_tracer,erlang, - erts_code_purger,erts_dirty_process_signal_handler, - erts_internal,erts_literal_area_collector, - init,persistent_term,prim_buffer,prim_eval,prim_file, - prim_inet] ++ ?ESOCK_NET_MODS ++ [prim_zip] ++ ?ESOCK_SOCKET_MODS ++ [zlib]. + lists:sort( + ?ESOCK_MODS ++ + [atomics,counters,erl_init,erl_prim_loader,erl_tracer,erlang, + erts_code_purger,erts_dirty_process_signal_handler, + erts_internal,erts_literal_area_collector, + init,persistent_term,prim_buffer,prim_eval,prim_file, + prim_inet,prim_zip,zlib]). + +%%______________________________________________________________________ +%% This is the erts binaries that should *not* be part of a systool:make_tar package + +erts_binary_filter() -> + Cmds = ["typer", "dialyzer", "ct_run", "yielding_c_fun", "erlc"], + case os:type() of + {unix,_} -> Cmds; + {win32,_} -> [ [Cmd, ".exe"] || Cmd <- Cmds] + end. %%______________________________________________________________________ %% Kernel processes; processes that are specially treated by the init @@ -1895,7 +1886,7 @@ add_appl(Name, Vsn, App, Tar, Variables, Flags, Var) -> Tar, AppDir, BinDir, - objfile_extension(machine(Flags))) + code:objfile_extension()) end. %%______________________________________________________________________ @@ -1974,18 +1965,26 @@ add_priv(ADir, ToDir, Tar) -> end. add_erts_bin(Tar, Release, Flags) -> - case get_flag(erts,Flags) of - {erts,ErtsDir} -> - EVsn = Release#release.erts_vsn, - FromDir = filename:join([to_list(ErtsDir), - "erts-" ++ EVsn, "bin"]), - dirp(FromDir), - ToDir = filename:join("erts-" ++ EVsn, "bin"), - add_to_tar(Tar, FromDir, ToDir); + case {get_flag(erts,Flags),member(erts_all,Flags)} of + {{erts,ErtsDir},true} -> + add_erts_bin(Tar, Release, ErtsDir, []); + {{erts,ErtsDir},false} -> + add_erts_bin(Tar, Release, ErtsDir, erts_binary_filter()); _ -> ok end. +add_erts_bin(Tar, Release, ErtsDir, Filters) -> + FlattenedFilters = [filename:flatten(Filter) || Filter <- Filters], + EVsn = Release#release.erts_vsn, + FromDir = filename:join([to_list(ErtsDir), + "erts-" ++ EVsn, "bin"]), + ToDir = filename:join("erts-" ++ EVsn, "bin"), + {ok, Bins} = file:list_dir(FromDir), + [add_to_tar(Tar, filename:join(FromDir,Bin), filename:join(ToDir,Bin)) + || Bin <- Bins, not lists:member(Bin, FlattenedFilters)], + ok. + %%______________________________________________________________________ %% Tar functions. @@ -2179,9 +2178,6 @@ cas([{variables, V} | Args], X) when is_list(V) -> error -> cas(Args, X++[{variables, V}]) end; -%%% machine ------------------------------------------------------------ -cas([{machine, M} | Args], X) when is_atom(M) -> - cas(Args, X); %%% exref -------------------------------------------------------------- cas([exref | Args], X) -> cas(Args, X); @@ -2248,6 +2244,8 @@ cat([{dirs, D} | Args], X) -> %%% erts --------------------------------------------------------------- cat([{erts, E} | Args], X) when is_list(E)-> cat(Args, X); +cat([erts_all | Args], X) -> + cat(Args, X); %%% src_tests ---------------------------------------------------- cat([src_tests | Args], X) -> cat(Args, X); @@ -2264,9 +2262,6 @@ cat([{var_tar, VT} | Args], X) when VT == include; VT == ownfile; VT == omit -> cat(Args, X); -%%% machine ------------------------------------------------------------ -cat([{machine, M} | Args], X) when is_atom(M) -> - cat(Args, X); %%% exref -------------------------------------------------------------- cat([exref | Args], X) -> cat(Args, X); diff --git a/lib/sasl/test/systools_SUITE.erl b/lib/sasl/test/systools_SUITE.erl index fb1dc03dcd..3c60c0fa21 100644 --- a/lib/sasl/test/systools_SUITE.erl +++ b/lib/sasl/test/systools_SUITE.erl @@ -69,7 +69,7 @@ groups() -> [tar_options, relname_tar, normal_tar, no_mod_vsn_tar, system_files_tar, system_src_file_tar, invalid_system_files_tar, variable_tar, src_tests_tar, var_tar, exref_tar, link_tar, no_sasl_tar, - otp_9507_path_ebin, additional_files_tar]}, + otp_9507_path_ebin, additional_files_tar, erts_tar]}, {relup, [], [normal_relup, restart_relup, abnormal_relup, no_sasl_relup, no_appup_relup, bad_appup_relup, app_start_type_relup, regexp_relup @@ -1041,13 +1041,90 @@ additional_files_tar(Config) -> ok. - system_files_tar(cleanup,Config) -> Dir = ?privdir, file:delete(filename:join(Dir,"sys.config")), file:delete(filename:join(Dir,"relup")), ok. +erts_tar(Config) -> + + {ok, OldDir} = file:get_cwd(), + + {LatestDir, LatestName} = create_script(current_all,Config), + + ERTS_VSN = erlang:system_info(version), + ERTS_DIR = fname(["erts-" ++ ERTS_VSN,bin]), + + %% List of all expected executable files in erts/bin + %% This list needs to be kept up to date whenever a file is + %% added or removed. + {Default, Ignored} = + case os:type() of + {unix,_} -> + {["beam.smp","dyn_erl","epmd","erl","erl_call","erl_child_setup", + "erlexec","erl.src","escript","heart","inet_gethost","run_erl", + "start","start_erl.src","start.src","to_erl"], + ["ct_run","dialyzer","erlc","typer","yielding_c_fun"]}; + {win32, _} -> + {["beam.smp.pdb","erl.exe", + "erl.pdb","erl_log.exe","erlexec.dll","erlsrv.exe","heart.exe", + "start_erl.exe","werl.exe","beam.smp.dll", + "epmd.exe","erl.ini","erl_call.exe", + "erlexec.pdb","escript.exe","inet_gethost.exe","werl.pdb"], + ["dialyzer.exe","erlc.exe","yielding_c_fun.exe","ct_run.exe","typer.exe"]} + end, + + ErtsTarContent = + fun(TarName) -> + lists:sort( + [filename:basename(File) + || File <- tar_contents(TarName), + string:equal(filename:dirname(File),ERTS_DIR), + %% Filter out beam.*.smp.* + re:run(filename:basename(File), "beam\\.[^\\.]+\\.smp(\\.dll)?") == nomatch, + %% Filter out any erl_child_setup.* + re:run(filename:basename(File), "erl_child_setup\\..*") == nomatch + ]) + end, + + DataDir = filename:absname(?copydir), + LibDir = fname([DataDir, d_normal, lib]), + P = [fname([LibDir, 'db-2.1', ebin])], + + ok = file:set_cwd(LatestDir), + + {ok, _, []} = systools:make_script(LatestName, [silent, {path, P}, {script_name, "start"}]), + ok = systools:make_tar(LatestName, [{path, P}, {erts, code:root_dir()}]), + ErtsContent = ErtsTarContent(LatestName), + + case lists:sort(Default) of + ErtsContent -> + ok; + Expected -> + ct:pal("Content: ~p",[ErtsContent]), + ct:pal("Expected: ~p",[Expected]), + ct:fail("Incorrect erts bin content") + end, + + ok = systools:make_tar(LatestName, [{path, P}, + {erts, code:root_dir()}, + erts_all]), + ErtsAllContent = ErtsTarContent(LatestName), + + case lists:sort(Default ++ Ignored) of + ErtsAllContent -> + ok; + ExpectedIgn -> + ct:pal("Content: ~p",[ErtsAllContent]), + ct:pal("Expected: ~p",[ExpectedIgn]), + ct:fail("Incorrect erts bin content") + end, + + ok = file:set_cwd(OldDir), + ok. + + %% make_tar: Check that sys.config.src and not sys.config is included system_src_file_tar(Config) -> {ok, OldDir} = file:get_cwd(), @@ -2342,7 +2419,7 @@ delete_tree(Dir) -> end. tar_contents(Name) -> - {ok, Cont} = erl_tar:table(Name ++ ".tar.gz", [compressed]), + {ok, Cont} = erl_tar:table(tar_name(Name), [compressed]), Cont. tar_name(Name) -> diff --git a/lib/sasl/vsn.mk b/lib/sasl/vsn.mk index fd045e49d5..9cab3fd792 100644 --- a/lib/sasl/vsn.mk +++ b/lib/sasl/vsn.mk @@ -1 +1 @@ -SASL_VSN = 3.4.2 +SASL_VSN = 4.0 diff --git a/lib/snmp/doc/src/notes.xml b/lib/snmp/doc/src/notes.xml index d4d5dd2f35..52b0b26c84 100644 --- a/lib/snmp/doc/src/notes.xml +++ b/lib/snmp/doc/src/notes.xml @@ -34,7 +34,76 @@ </header> - <section><title>SNMP 5.5</title> + <section><title>SNMP 5.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + For manager, fix PrivParams for SNMPv3 USM with AES + privacy; * In `snmp_usm:do_decrypt/3`, pass full + UsmSecParams to `snmp_usm:try_decrypt/5` as expected by + AES clause. * Change `snmpm_usm:aes_encrypt/3` to use + EngineBoots and EngineTime as cached by + `snmpm_config:get_usm_eboots/1` and + `snmpm_config:get_usm_etime/1` instead of + `snmpm_config:get_engine_boots/0` and + `snmpm_config:get_engine_time/0`. This ensures correct + msgPrivacyParameters are sent when AES is used. * Add + test `snmp.snmp_manager_SUITE.usm_priv_aes/1` to avoid + regression.</p> + <p> + Own Id: OTP-16541 Aux Id: #2544 </p> + </item> + <item> + <p> + Invalid character in (manager) usm config entry generator + function.</p> + <p> + Own Id: OTP-16552 Aux Id: ERL-1196 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Remove usage and documentation of old requests of the + I/O-protocol.</p> + <p> + Own Id: OTP-15695</p> + </item> + <item> + <p> + Calls of deprecated functions in the <seeguide + marker="crypto:new_api#the-old-api">Old Crypto + API</seeguide> are replaced by calls of their <seeguide + marker="crypto:new_api#the-new-api">substitutions</seeguide>.</p> + <p> + Own Id: OTP-16346</p> + </item> + <item> + <p> + Finalize deprecation. Already deprecated functions has a + "remove version 24" set and "new" functions added to list + of deprecated functions.</p> + <p> + Own Id: OTP-16463</p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + </list> + </section> + +</section> + +<section><title>SNMP 5.5</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/snmp/test/snmp_agent_SUITE.erl b/lib/snmp/test/snmp_agent_SUITE.erl index 62129c2543..c676bc487e 100644 --- a/lib/snmp/test/snmp_agent_SUITE.erl +++ b/lib/snmp/test/snmp_agent_SUITE.erl @@ -944,25 +944,26 @@ end_per_testcase(Case, Config) when is_list(Config) -> %% already failed, we will want to get as much of the logs %% as possible. So, set no timeout (infinity) and let the %% test framework take care of things... - DisplayLogTimeout = - case ?config(tc_status, Config) of - ok -> - ?SECS(30); - _ -> - infinity + %% But also, *don't* bother with this unless the test case + %% has failed! + case ?config(tc_status, Config) of + ok -> + ok; + _ -> + To = ?SECS(30), + Flag = process_flag(trap_exit, true), + Pid = spawn_link(fun() -> display_log(Config), exit(normal) end), + receive + {'EXIT', Pid, _} -> + process_flag(trap_exit, Flag), + ok + after To -> + ?WPRINT("Display Log process fail to complete in time" + "(~w msec): kill it", [To]), + process_flag(trap_exit, Flag), + exit(Pid, kill) + end end, - Flag = process_flag(trap_exit, true), - Pid = spawn_link(fun() -> display_log(Config), exit(normal) end), - receive - {'EXIT', Pid, _} -> - process_flag(trap_exit, Flag), - ok - after DisplayLogTimeout -> - ?WPRINT("Display Log process fail to complete in time (~w msec): " - "kill it", [DisplayLogTimeout]), - process_flag(trap_exit, Flag), - exit(Pid, kill) - end, Result = end_per_testcase1(Case, Config), @@ -1443,6 +1444,7 @@ msd_varm_mib_start(X) -> msm_varm_mib_start(X) -> %% <CONDITIONAL-SKIP> + %% This is a bit radioactive but... Skippable = [win32], Condition = fun() -> ?OS_BASED_SKIP(Skippable) end, ?NON_PC_TC_MAYBE_SKIP(X, Condition), @@ -6307,27 +6309,34 @@ otp_1131_3(X) -> %% Montavista Linux looks like a Debian distro (/etc/issue) LinuxVersionVerify = fun() -> - case os:cmd("uname -m") of + case string:to_lower(os:cmd("uname -m")) of "ppc" ++ _ -> case file:read_file_info("/etc/issue") of {ok, _} -> - case os:cmd("grep -i montavista /etc/issue") of - Info when (is_list(Info) andalso - (length(Info) > 0)) -> + case string:to_lower( + os:cmd("grep -i montavista /etc/issue")) of + "montavista" ++ _ -> case os:version() of {2, 6, 10} -> + ?IPRINT("(PPC Linux) kernel version check: " + "{2, 6, 10} => SKIP"), true; - _ -> + V -> + ?IPRINT("(PPC Linux) kernel version check: " + "~p != {2, 6, 10} => *NO* SKIP", [V]), false end; _ -> % Maybe plain Debian or Ubuntu + ?IPRINT("(PPC Linux) Not MontaVista => *NO* SKIP"), false end; _ -> %% Not a Debian based distro + ?IPRINT("(PPC Linux) Unknown distro => *NO* SKIP"), false end; _ -> + ?IPRINT("(Linux) Not PPC => *NO* SKIP"), false end end, diff --git a/lib/snmp/test/snmp_agent_mibs_SUITE.erl b/lib/snmp/test/snmp_agent_mibs_SUITE.erl index 150e015554..39946ba7d1 100644 --- a/lib/snmp/test/snmp_agent_mibs_SUITE.erl +++ b/lib/snmp/test/snmp_agent_mibs_SUITE.erl @@ -172,29 +172,28 @@ init_per_testcase2(size_check_ets2_bad_file1, Config) when is_list(Config) -> %% Create a bad file ok = file:write_file(join(DbDir, "snmpa_symbolic_store.db"), "calvin and hoppes play chess"), + Factor = ?config(snmp_factor, Config), + ct:timetrap(?MINS(1 + (Factor div 2))), Config; init_per_testcase2(size_check_ets3_bad_file1, Config) when is_list(Config) -> DbDir = ?config(db_dir, Config), %% Create a bad file ok = file:write_file(join(DbDir, "snmpa_symbolic_store.db"), "calvin and hoppes play chess"), + Factor = ?config(snmp_factor, Config), + ct:timetrap(?MINS(1 + (Factor div 2))), Config; init_per_testcase2(size_check_mnesia, Config) when is_list(Config) -> + Factor = ?config(snmp_factor, Config), + ct:timetrap(?MINS(1 + (Factor div 2))), Config; init_per_testcase2(cache_test, Config) when is_list(Config) -> - Min = timer:minutes(5), - Timeout = - case lists:keysearch(tc_timeout, 1, Config) of - {value, {tc_timeout, TcTimeout}} when TcTimeout < Min -> - Min; - {value, {tc_timeout, TcTimeout}} -> - TcTimeout; - _ -> - Min - end, - Dog = test_server:timetrap(Timeout), - [{watchdog, Dog} | Config]; + Factor = ?config(snmp_factor, Config), + ct:timetrap(?MINS(5 + (Factor div 2))), + Config; init_per_testcase2(_Case, Config) when is_list(Config) -> + Factor = ?config(snmp_factor, Config), + ct:timetrap(?MINS(1 + (Factor div 3))), Config. @@ -220,6 +219,10 @@ end_per_testcase1(_Case, Config) when is_list(Config) -> start_and_stop(suite) -> []; start_and_stop(Config) when is_list(Config) -> + tc_try(start_and_start, + fun() -> do_start_and_stop(Config) end). + +do_start_and_stop(_Config) -> Prio = normal, Verbosity = trace, @@ -238,6 +241,10 @@ start_and_stop(Config) when is_list(Config) -> load_unload(suite) -> []; load_unload(Config) when is_list(Config) -> + tc_try(load_unload, + fun() -> do_load_unload(Config) end). + +do_load_unload(Config) -> ?DBG("load_unload -> start", []), Prio = normal, @@ -365,49 +372,8 @@ do_size_check(Name, Config) -> do_size_check(Name, Init, Config). do_size_check(Name, Init, Config) -> - Pre = fun() -> - {ok, Node} = ?ALIB:start_node(unique(Name)), - ok = run_on(Node, Init), - Node - end, - Case = fun(Node) -> - monitor_node(Node, true), - Pid = spawn_link(Node, fun() -> do_size_check(Config) end), - receive - {nodedown, Node} = N -> - exit(N); - {'EXIT', Pid, normal} -> - monitor_node(Node, false), - ok; - {'EXIT', Pid, ok} -> - monitor_node(Node, false), - ok; - {'EXIT', Pid, Reason} -> - monitor_node(Node, false), - exit(Reason) - end - end, - Post = fun({Node, _}) -> - ?STOP_NODE(Node) - end, - ?TC_TRY(Name, Pre, Case, Post). + tc_try(Name, Init, fun() -> do_size_check(Config) end). -run_on(Node, F) when is_atom(Node) andalso is_function(F, 0) -> - monitor_node(Node, true), - Pid = spawn_link(Node, F), - receive - {nodedown, Node} = N -> - exit(N); - {'EXIT', Pid, normal} -> - monitor_node(Node, false), - ok; - {'EXIT', Pid, Reason} -> - monitor_node(Node, false), - Reason - end. - -unique(PreName) -> - list_to_atom(?F("~w_~w", [PreName, erlang:system_time(millisecond)])). do_size_check(Config) -> ?IPRINT("do_size_check -> start with" @@ -465,6 +431,10 @@ do_size_check(Config) -> me_lookup(suite) -> []; me_lookup(Config) when is_list(Config) -> + tc_try(me_lookup, + fun() -> do_me_lookup(Config) end). + +do_me_lookup(Config) -> Prio = normal, Verbosity = trace, MibDir = ?config(data_dir, Config), @@ -518,6 +488,10 @@ me_lookup(Config) when is_list(Config) -> which_mib(suite) -> []; which_mib(Config) when is_list(Config) -> + tc_try(which_mib, + fun() -> do_which_mib(Config) end). + +do_which_mib(Config) -> Prio = normal, Verbosity = trace, MibDir = ?config(data_dir, Config), @@ -574,7 +548,11 @@ which_mib(Config) when is_list(Config) -> cache_test(suite) -> []; cache_test(Config) when is_list(Config) -> - ?DBG("cache_test -> start", []), + tc_try(cache_test, + fun() -> do_cache_test(Config) end). + +do_cache_test(Config) -> + ?DBG("do_cache_test -> start", []), Prio = normal, Verbosity = trace, MibStorage = [{module, snmpa_mib_storage_ets}], @@ -660,15 +638,17 @@ walk(MibsPid) -> do_walk(MibsPid, Oid, MibView) -> - io:format("do_walk -> entry with" - "~n Oid: ~p" - "~n", [Oid]), + ?IPRINT("do_walk -> entry with" + "~n Oid: ~p" + "~n", [Oid]), case snmpa_mib:next(MibsPid, Oid, MibView) of {table, _, _, #me{oid = Oid}} -> + ?IPRINT("do_walk -> done for table (~p)", [Oid]), ok; {table, _, _, #me{oid = Next}} -> do_walk(MibsPid, Next, MibView); {variable, #me{oid = Oid}, _} -> + ?IPRINT("do_walk -> done for variable (~p)", [Oid]), ok; {variable, #me{oid = Next}, _} -> do_walk(MibsPid, Next, MibView) @@ -897,6 +877,65 @@ mib_storage() -> [{module, snmpa_mib_storage_ets}]. +%% -- + +tc_try(Name, TC) -> + tc_try(Name, fun() -> ok end, TC). + +tc_try(Name, Init, TC) + when is_atom(Name) andalso is_function(Init, 0) andalso is_function(TC, 0) -> + Pre = fun() -> + {ok, Node} = ?ALIB:start_node(unique(Name)), + ok = run_on(Node, Init), + Node + end, + Case = fun(Node) -> + monitor_node(Node, true), + Pid = spawn_link(Node, TC), + receive + {nodedown, Node} = N -> + exit(N); + {'EXIT', Pid, normal} -> + monitor_node(Node, false), + ok; + {'EXIT', Pid, ok} -> + monitor_node(Node, false), + ok; + {'EXIT', Pid, Reason} -> + monitor_node(Node, false), + exit(Reason) + end + end, + Post = fun(Node) -> + monitor_node(Node, true), + ?NPRINT("try stop node ~p", [Node]), + ?STOP_NODE(Node), + receive + {nodedown, Node} -> + ?NPRINT("node ~p stopped", [Node]), + ok + end + end, + ?TC_TRY(Name, Pre, Case, Post). + +run_on(Node, F) when is_atom(Node) andalso is_function(F, 0) -> + monitor_node(Node, true), + Pid = spawn_link(Node, F), + receive + {nodedown, Node} = N -> + exit(N); + {'EXIT', Pid, normal} -> + monitor_node(Node, false), + ok; + {'EXIT', Pid, Reason} -> + monitor_node(Node, false), + Reason + end. + +unique(PreName) -> + list_to_atom(?F("~w_~w", [PreName, erlang:system_time(millisecond)])). + + %% -- display_memory_usage(MibsPid) -> diff --git a/lib/snmp/test/snmp_manager_SUITE.erl b/lib/snmp/test/snmp_manager_SUITE.erl index 6e695f5ddf..6cc0b0ebde 100644 --- a/lib/snmp/test/snmp_manager_SUITE.erl +++ b/lib/snmp/test/snmp_manager_SUITE.erl @@ -68,34 +68,24 @@ info/1, usm_priv_aes/1, - simple_sync_get2/1, simple_sync_get3/1, - simple_async_get2/1, simple_async_get3/1, - simple_sync_get_next2/1, simple_sync_get_next3/1, - simple_async_get_next2/1, simple_async_get_next3_cbp_def/1, simple_async_get_next3_cbp_temp/1, simple_async_get_next3_cbp_perm/1, - simple_sync_set2/1, simple_sync_set3/1, - simple_async_set2/1, simple_async_set3_cbp_def/1, simple_async_set3_cbp_temp/1, simple_async_set3_cbp_perm/1, - simple_sync_get_bulk2/1, simple_sync_get_bulk3/1, - simple_async_get_bulk2/1, simple_async_get_bulk3_cbp_def/1, simple_async_get_bulk3_cbp_temp/1, simple_async_get_bulk3_cbp_perm/1, - misc_async2/1, - discovery/1, trap1/1, @@ -206,8 +196,7 @@ groups() -> {group, get_tests}, {group, get_next_tests}, {group, set_tests}, - {group, bulk_tests}, - {group, misc_request_tests} + {group, bulk_tests} ] }, {request_tests_mt, [], @@ -215,23 +204,18 @@ groups() -> {group, get_tests}, {group, get_next_tests}, {group, set_tests}, - {group, bulk_tests}, - {group, misc_request_tests} + {group, bulk_tests} ] }, {get_tests, [], [ - simple_sync_get2, simple_sync_get3, - simple_async_get2, simple_async_get3 ] }, {get_next_tests, [], [ - simple_sync_get_next2, simple_sync_get_next3, - simple_async_get_next2, simple_async_get_next3_cbp_def, simple_async_get_next3_cbp_temp, simple_async_get_next3_cbp_perm @@ -239,9 +223,7 @@ groups() -> }, {set_tests, [], [ - simple_sync_set2, simple_sync_set3, - simple_async_set2, simple_async_set3_cbp_def, simple_async_set3_cbp_temp, simple_async_set3_cbp_perm @@ -249,19 +231,12 @@ groups() -> }, {bulk_tests, [], [ - simple_sync_get_bulk2, simple_sync_get_bulk3, - simple_async_get_bulk2, simple_async_get_bulk3_cbp_def, simple_async_get_bulk3_cbp_temp, simple_async_get_bulk3_cbp_perm ] }, - {misc_request_tests, [], - [ - misc_async2 - ] - }, {event_tests, [], [ trap1, @@ -315,16 +290,11 @@ ipv6_tests() -> [ register_agent_old, simple_sync_get_next3, - simple_async_get2, simple_sync_get3, - simple_async_get_next2, simple_sync_set3, - simple_async_set2, - simple_sync_get_bulk2, simple_async_get_bulk3_cbp_def, simple_async_get_bulk3_cbp_temp, simple_async_get_bulk3_cbp_perm, - misc_async2, inform1, inform_swarm_cbp_def, inform_swarm_cbp_temp, @@ -529,18 +499,16 @@ init_per_testcase2(Case, Config) -> Family = proplists:get_value(ipfamily, Config, inet), + Factor = ?config(snmp_factor, Config), TO = case Case of inform3 -> - ?MINS(2); + ?MINS(2 + (Factor div 2)); InformSwarm when (InformSwarm =:= inform_swarm_cbp_def) orelse (InformSwarm =:= inform_swarm_cbp_temp) orelse (InformSwarm =:= inform_swarm_cbp_perm) -> - case ?config(snmp_factor, Config) of - N when is_integer(N) -> ?MINS(2*N); - _ -> ?MINS(2) - end; + ?MINS(1 + Factor); _ -> - ?MINS(1) + ?MINS(1 + (Factor div 2)) end, ?IPRINT("Set test case timetrap: ~p", [TO]), ct:timetrap(TO), @@ -564,15 +532,6 @@ init_per_testcase2(Case, Config) -> init_per_testcase3(Case, Config) -> ApiCases02 = [ - simple_sync_get2, - simple_async_get2, - simple_sync_get_next2, - simple_async_get_next2, - simple_sync_set2, - simple_async_set2, - simple_sync_get_bulk2, - simple_async_get_bulk2, - misc_async2, otp8395_1 ], ApiCases03 = @@ -702,15 +661,6 @@ end_per_testcase(Case, Config) when is_list(Config) -> end_per_testcase2(Case, Config) -> ApiCases02 = [ - simple_sync_get2, - simple_async_get2, - simple_sync_get_next2, - simple_async_get_next2, - simple_sync_set2, - simple_async_set2, - simple_sync_get_bulk2, - simple_async_get_bulk2, - misc_async2, otp8395_1 ], ApiCases03 = @@ -1159,20 +1109,15 @@ notify_started02(Config) when is_list(Config) -> notify_started02_cond(Config) -> LinuxVersionVerify = fun() -> - case os:cmd("uname -m") of - "i686" ++ _ -> - case os:version() of - {2, 6, Rev} when Rev >= 16 -> - false; - {2, Min, _} when Min > 6 -> - false; - {Maj, _, _} when Maj > 2 -> - false; - _ -> - true - end; - _ -> - false + case os:version() of + V when V > {2, 6, 16} -> + ?IPRINT("(Linux) kernel version check: " + "~p > {2, 6, 16} => *NO* SKIP", [V]), + false; + V -> + ?IPRINT("(Linux) kernel version check: " + "~p =< {2, 6, 16} => *SKIP*", [V]), + true end end, Skippable = [{unix, [{linux, LinuxVersionVerify}]}], @@ -2142,25 +2087,40 @@ do_register_agent3([ManagerNode], Config) -> %%====================================================================== -simple_sync_get2(doc) -> - ["Simple sync get-request - Version 2 API (TargetName)"]; -simple_sync_get2(suite) -> []; -simple_sync_get2(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_get2, - fun() -> do_simple_sync_get2(Config) end). +simple_sync_get3(doc) -> + ["Simple sync get-request - Version 3 API (TargetName and send-opts)"]; +simple_sync_get3(suite) -> []; +simple_sync_get3(Config) when is_list(Config) -> + ?TC_TRY(simple_sync_get3, + fun() -> do_simple_sync_get3(Config) end). -do_simple_sync_get2(Config) -> +do_simple_sync_get3(Config) -> ?IPRINT("starting with Config: " "~n ~p", [Config]), + Self = self(), + Msg = simple_sync_get3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], Get = fun(Node, TargetName, Oids) -> - mgr_user_sync_get(Node, TargetName, Oids) - end, - PostVerify = fun() -> ok end, - Res = do_simple_sync_get2(Config, Get, PostVerify), + mgr_user_sync_get2(Node, TargetName, Oids, SendOpts) + end, + PostVerify = + fun() -> + receive + Msg -> + ok + end + end, + Res = do_simple_sync_get3(Config, Get, PostVerify), display_log(Config), Res. -do_simple_sync_get2(Config, Get, PostVerify) -> + +do_simple_sync_get3(Config, Get, PostVerify) -> ?IPRINT("starting with Config: " "~n ~p", [Config]), @@ -2169,15 +2129,15 @@ do_simple_sync_get2(Config, Get, PostVerify) -> ?IPRINT("issue get-request without loading the mib"), Oids1 = [?sysObjectID_instance, ?sysDescr_instance, ?sysUpTime_instance], - ?line ok = do_simple_sync_get2(Node, TargetName, Oids1, Get, PostVerify), + ?line ok = do_simple_sync_get3(Node, TargetName, Oids1, Get, PostVerify), ?IPRINT("issue get-request after first loading the mibs"), ?line ok = mgr_user_load_mib(Node, std_mib()), Oids2 = [[sysObjectID, 0], [sysDescr, 0], [sysUpTime, 0]], - ?line ok = do_simple_sync_get2(Node, TargetName, Oids2, Get, PostVerify), + ?line ok = do_simple_sync_get3(Node, TargetName, Oids2, Get, PostVerify), ok. -do_simple_sync_get2(Node, TargetName, Oids, Get, PostVerify) +do_simple_sync_get3(Node, TargetName, Oids, Get, PostVerify) when is_function(Get, 3) andalso is_function(PostVerify, 0) -> ?line {ok, Reply, _Rem} = Get(Node, TargetName, Oids), @@ -2213,43 +2173,6 @@ do_simple_sync_get2(Node, TargetName, Oids, Get, PostVerify) %%====================================================================== -simple_sync_get3(doc) -> - ["Simple sync get-request - Version 3 API (TargetName and send-opts)"]; -simple_sync_get3(suite) -> []; -simple_sync_get3(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_get3, - fun() -> do_simple_sync_get3(Config) end). - -do_simple_sync_get3(Config) -> - ?IPRINT("starting with Config: " - "~n ~p", [Config]), - Self = self(), - Msg = simple_sync_get3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - Get = fun(Node, TargetName, Oids) -> - mgr_user_sync_get2(Node, TargetName, Oids, SendOpts) - end, - PostVerify = - fun() -> - receive - Msg -> - ok - end - end, - Res = do_simple_sync_get2(Config, Get, PostVerify), - display_log(Config), - Res. - - - - -%%====================================================================== - sag_verify({noError, 0, _Vbs}, any) -> ?IPRINT("verified [any]"), ok; @@ -2283,35 +2206,46 @@ sag_verify_vbs([Vb|_], [E|_]) -> %%====================================================================== -simple_async_get2(doc) -> - ["Simple (async) get-request - Version 2 API (TargetName)"]; -simple_async_get2(suite) -> []; -simple_async_get2(Config) when is_list(Config) -> - ?TC_TRY(simple_async_get2, - fun() -> do_simple_async_get2(Config) end). +simple_async_get3(doc) -> + ["Simple (async) get-request - Version 3 API (TargetName and send-opts)"]; +simple_async_get3(suite) -> []; +simple_async_get3(Config) when is_list(Config) -> + ?TC_TRY(simple_async_get3, + fun() -> do_simple_async_get3(Config) end). -do_simple_async_get2(Config) -> +do_simple_async_get3(Config) -> ?IPRINT("starting with Config: " "~n ~p", [Config]), MgrNode = ?config(manager_node, Config), AgentNode = ?config(agent_node, Config), TargetName = ?config(manager_agent_target_name, Config), - Get = fun(Oids) -> async_g_exec2(MgrNode, TargetName, Oids) end, - PostVerify = fun(Res) -> Res end, - do_simple_async_sync_get2(Config, MgrNode, AgentNode, Get, PostVerify), + Self = self(), + Msg = simple_async_get3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], + Get = fun(Oids) -> async_g_exec3(MgrNode, TargetName, Oids, SendOpts) end, + PostVerify = fun(ok) -> receive Msg -> ok end; + (Error) -> Error + end, + Res = do_simple_async_sync_get3(Config, MgrNode, AgentNode, + Get, PostVerify), display_log(Config), - ok. + Res. -do_simple_async_sync_get2(Config, MgrNode, AgentNode, Get, PostVerify) -> +do_simple_async_sync_get3(Config, MgrNode, AgentNode, Get, PostVerify) -> ?line ok = mgr_user_load_mib(MgrNode, std_mib()), Test2Mib = test2_mib(Config), ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), ?line ok = agent_load_mib(AgentNode, Test2Mib), - do_simple_async_sync_get2(fun() -> mgr_info(MgrNode) end, + do_simple_async_sync_get3(fun() -> mgr_info(MgrNode) end, fun() -> agent_info(AgentNode) end, Get, PostVerify). -do_simple_async_sync_get2(MgrInfo, AgentInfo, Get, PostVerify) +do_simple_async_sync_get3(MgrInfo, AgentInfo, Get, PostVerify) when is_function(MgrInfo, 0) andalso is_function(AgentInfo, 0) andalso is_function(Get, 1) andalso @@ -2365,41 +2299,6 @@ do_simple_async_sync_get2(MgrInfo, AgentInfo, Get, PostVerify) ok. -async_g_exec2(Node, TargetName, Oids) -> - mgr_user_async_get(Node, TargetName, Oids). - - -%%====================================================================== - -simple_async_get3(doc) -> - ["Simple (async) get-request - Version 3 API (TargetName and send-opts)"]; -simple_async_get3(suite) -> []; -simple_async_get3(Config) when is_list(Config) -> - ?TC_TRY(simple_async_get3, - fun() -> do_simple_async_get3(Config) end). - -do_simple_async_get3(Config) -> - ?IPRINT("starting with Config: " - "~n ~p", [Config]), - MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), - TargetName = ?config(manager_agent_target_name, Config), - Self = self(), - Msg = simple_async_get3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - Get = fun(Oids) -> async_g_exec3(MgrNode, TargetName, Oids, SendOpts) end, - PostVerify = fun(ok) -> receive Msg -> ok end; - (Error) -> Error - end, - Res = do_simple_async_sync_get2(Config, MgrNode, AgentNode, Get, PostVerify), - display_log(Config), - Res. - async_g_exec3(Node, TargetName, Oids, SendOpts) -> mgr_user_async_get2(Node, TargetName, Oids, SendOpts). @@ -2440,27 +2339,35 @@ check_ssgn_vbs([Vb|_], [E|_]) -> %%====================================================================== -simple_sync_get_next2(doc) -> - ["Simple (sync) get_next-request - Version 2 API (TargetName)"]; -simple_sync_get_next2(suite) -> []; -simple_sync_get_next2(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_get_next2, - fun() -> do_simple_sync_get_next2(Config) end). - -do_simple_sync_get_next2(Config) -> +simple_sync_get_next3(doc) -> + ["Simple (sync) get_next-request - " + "Version 3 API (TargetName with send-opts)"]; +simple_sync_get_next3(suite) -> []; +simple_sync_get_next3(Config) when is_list(Config) -> + process_flag(trap_exit, true), + put(tname, ssgn3), ?IPRINT("starting with Config: " "~n ~p", [Config]), - + Self = self(), + Msg = simple_sync_get_next3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], GetNext = fun(Node, TargetName, Oids) -> - mgr_user_sync_get_next(Node, TargetName, Oids) + mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) end, - PostVerify = fun(Res) -> Res end, - Res = do_simple_sync_get_next2(Config, GetNext, PostVerify), + PostVerify = fun(ok) -> receive Msg -> ok end; + (Error) -> Error + end, + do_simple_sync_get_next3(Config, GetNext, PostVerify), display_log(Config), - Res. + ok. -do_simple_sync_get_next2(Config, GetNext, PostVerify) +do_simple_sync_get_next3(Config, GetNext, PostVerify) when is_function(GetNext, 3) andalso is_function(PostVerify, 1) -> MgrNode = ?config(manager_node, Config), @@ -2550,7 +2457,6 @@ do_simple_sync_get_next2(Config, GetNext, PostVerify) GetNext, PostVerify), ok. - do_simple_get_next(N, Node, TargetName, Oids, Verify, GetNext, PostVerify) -> ?IPRINT("issue get-next command ~w", [N]), case GetNext(Node, TargetName, Oids) of @@ -2565,46 +2471,36 @@ do_simple_get_next(N, Node, TargetName, Oids, Verify, GetNext, PostVerify) -> end. + %%====================================================================== -simple_sync_get_next3(doc) -> - ["Simple (sync) get_next-request - " +simple_async_get_next3_cbp_def(doc) -> + ["Simple (async) get_next-request - " "Version 3 API (TargetName with send-opts)"]; -simple_sync_get_next3(suite) -> []; -simple_sync_get_next3(Config) when is_list(Config) -> - process_flag(trap_exit, true), - put(tname, ssgn3), - ?IPRINT("starting with Config: " - "~n ~p", [Config]), - Self = self(), - Msg = simple_sync_get_next3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - GetNext = fun(Node, TargetName, Oids) -> - mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) - end, - PostVerify = fun(ok) -> receive Msg -> ok end; - (Error) -> Error - end, - do_simple_sync_get_next2(Config, GetNext, PostVerify), - display_log(Config), - ok. +simple_async_get_next3_cbp_def(suite) -> []; +simple_async_get_next3_cbp_def(Config) when is_list(Config) -> + simple_async_get_next3(ssgn2_cbp_def, Config). +simple_async_get_next3_cbp_temp(doc) -> + ["Simple (async) get_next-request - " + "Version 3 API (TargetName with send-opts)"]; +simple_async_get_next3_cbp_temp(suite) -> []; +simple_async_get_next3_cbp_temp(Config) when is_list(Config) -> + simple_async_get_next3(ssgn2_cbp_temp, Config). -%%====================================================================== +simple_async_get_next3_cbp_perm(doc) -> + ["Simple (async) get_next-request - " + "Version 3 API (TargetName with send-opts)"]; +simple_async_get_next3_cbp_perm(suite) -> []; +simple_async_get_next3_cbp_perm(Config) when is_list(Config) -> + simple_async_get_next3(ssgn2_cbp_perm, Config). -simple_async_get_next2(doc) -> - ["Simple (async) get_next-request - Version 2 API (TargetName)"]; -simple_async_get_next2(suite) -> []; -simple_async_get_next2(Config) when is_list(Config) -> - ?TC_TRY(simple_async_get_next2, - fun() -> do_simple_async_get_next2(Config) end). +simple_async_get_next3(Case, Config) when is_list(Config) -> + ?TC_TRY(Case, + fun() -> do_simple_async_get_next3(Config) end). -do_simple_async_get_next2(Config) -> +do_simple_async_get_next3(Config) -> + %% process_flag(trap_exit, true), ?IPRINT("starting with Config: " "~n ~p", [Config]), @@ -2616,15 +2512,28 @@ do_simple_async_get_next2(Config) -> Test2Mib = test2_mib(Config), ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), ?line ok = agent_load_mib(AgentNode, Test2Mib), + + Self = self(), + Msg = simple_async_get_next3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], + GetNext = fun(Oids) -> - async_gn_exec2(MgrNode, TargetName, Oids) + async_gn_exec3(MgrNode, TargetName, Oids, SendOpts) end, - PostVerify = fun(Res) -> Res end, - Res = do_simple_async_get_next2(MgrNode, AgentNode, GetNext, PostVerify), + PostVerify = fun(ok) -> receive Msg -> ok end; + (Error) -> Error + end, + + Res = do_simple_async_get_next3(MgrNode, AgentNode, GetNext, PostVerify), display_log(Config), Res. -do_simple_async_get_next2(MgrNode, AgentNode, GetNext, PostVerify) +do_simple_async_get_next3(MgrNode, AgentNode, GetNext, PostVerify) when is_function(GetNext, 1) andalso is_function(PostVerify, 1) -> ?line {ok, [TCnt2|_]} = mgr_user_name_to_oid(MgrNode, tCnt2), ?line {ok, [TGenErr1|_]} = mgr_user_name_to_oid(MgrNode, tGenErr1), @@ -2708,71 +2617,6 @@ do_simple_async_get_next2(MgrNode, AgentNode, GetNext, PostVerify) ok. -async_gn_exec2(Node, TargetName, Oids) -> - mgr_user_async_get_next(Node, TargetName, Oids). - - -%%====================================================================== - -simple_async_get_next3_cbp_def(doc) -> - ["Simple (async) get_next-request - " - "Version 3 API (TargetName with send-opts)"]; -simple_async_get_next3_cbp_def(suite) -> []; -simple_async_get_next3_cbp_def(Config) when is_list(Config) -> - simple_async_get_next3(ssgn2_cbp_def, Config). - -simple_async_get_next3_cbp_temp(doc) -> - ["Simple (async) get_next-request - " - "Version 3 API (TargetName with send-opts)"]; -simple_async_get_next3_cbp_temp(suite) -> []; -simple_async_get_next3_cbp_temp(Config) when is_list(Config) -> - simple_async_get_next3(ssgn2_cbp_temp, Config). - -simple_async_get_next3_cbp_perm(doc) -> - ["Simple (async) get_next-request - " - "Version 3 API (TargetName with send-opts)"]; -simple_async_get_next3_cbp_perm(suite) -> []; -simple_async_get_next3_cbp_perm(Config) when is_list(Config) -> - simple_async_get_next3(ssgn2_cbp_perm, Config). - -simple_async_get_next3(Case, Config) when is_list(Config) -> - ?TC_TRY(Case, - fun() -> do_simple_async_get_next3(Config) end). - -do_simple_async_get_next3(Config) -> - %% process_flag(trap_exit, true), - ?IPRINT("starting with Config: " - "~n ~p", [Config]), - - MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), - TargetName = ?config(manager_agent_target_name, Config), - - ?line ok = mgr_user_load_mib(MgrNode, std_mib()), - Test2Mib = test2_mib(Config), - ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), - ?line ok = agent_load_mib(AgentNode, Test2Mib), - - Self = self(), - Msg = simple_async_get_next3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - - GetNext = fun(Oids) -> - async_gn_exec3(MgrNode, TargetName, Oids, SendOpts) - end, - PostVerify = fun(ok) -> receive Msg -> ok end; - (Error) -> Error - end, - - Res = do_simple_async_get_next2(MgrNode, AgentNode, GetNext, PostVerify), - display_log(Config), - Res. - async_gn_exec3(Node, TargetName, Oids, SendOpts) -> mgr_user_async_get_next2(Node, TargetName, Oids, SendOpts). @@ -2792,27 +2636,36 @@ value_of_vavs([{_Oid, Val}|VAVs], Acc) -> %%====================================================================== -simple_sync_set2(doc) -> - ["Simple (sync) set-request - Version 2 API (TargetName)"]; -simple_sync_set2(suite) -> []; -simple_sync_set2(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_set2, - fun() -> do_simple_sync_set2(Config) end). +simple_sync_set3(doc) -> + ["Simple (sync) set-request - Version 3 API (TargetName with send-opts)"]; +simple_sync_set3(suite) -> []; +simple_sync_set3(Config) when is_list(Config) -> + ?TC_TRY(simple_sync_set3, + fun() -> do_simple_sync_set3(Config) end). -do_simple_sync_set2(Config) -> +do_simple_sync_set3(Config) -> ?IPRINT("starting with Config: " "~n ~p", [Config]), + Self = self(), + Msg = simple_sync_set3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], + Set = fun(Node, TargetName, VAVs) -> - mgr_user_sync_set(Node, TargetName, VAVs) + mgr_user_sync_set2(Node, TargetName, VAVs, SendOpts) end, - PostVerify = fun() -> ok end, + PostVerify = fun() -> receive Msg -> ok end end, - Res = do_simple_sync_set2(Config, Set, PostVerify), + Res = do_simple_sync_set3(Config, Set, PostVerify), display_log(Config), Res. -do_simple_sync_set2(Config, Set, PostVerify) +do_simple_sync_set3(Config, Set, PostVerify) when is_function(Set, 3) andalso is_function(PostVerify, 0) -> Node = ?config(manager_node, Config), @@ -2825,7 +2678,7 @@ do_simple_sync_set2(Config, Set, PostVerify) {?sysName_instance, s, Val11}, {?sysLocation_instance, s, Val12} ], - ?line ok = do_simple_set2(Node, TargetName, VAVs1, Set, PostVerify), + ?line ok = do_simple_set3(Node, TargetName, VAVs1, Set, PostVerify), ?IPRINT("issue set-request after first loading the mibs"), ?line ok = mgr_user_load_mib(Node, std_mib()), @@ -2835,10 +2688,10 @@ do_simple_sync_set2(Config, Set, PostVerify) {[sysName, 0], Val21}, {[sysLocation, 0], Val22} ], - ?line ok = do_simple_set2(Node, TargetName, VAVs2, Set, PostVerify), + ?line ok = do_simple_set3(Node, TargetName, VAVs2, Set, PostVerify), ok. -do_simple_set2(Node, TargetName, VAVs, Set, PostVerify) -> +do_simple_set3(Node, TargetName, VAVs, Set, PostVerify) -> [SysName, SysLoc] = value_of_vavs(VAVs), ?line {ok, Reply, _Rem} = Set(Node, TargetName, VAVs), @@ -2866,38 +2719,6 @@ do_simple_set2(Node, TargetName, VAVs, Set, PostVerify) -> %%====================================================================== -simple_sync_set3(doc) -> - ["Simple (sync) set-request - Version 3 API (TargetName with send-opts)"]; -simple_sync_set3(suite) -> []; -simple_sync_set3(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_set3, - fun() -> do_simple_sync_set3(Config) end). - -do_simple_sync_set3(Config) -> - ?IPRINT("starting with Config: " - "~n ~p", [Config]), - - Self = self(), - Msg = simple_sync_set3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - - Set = fun(Node, TargetName, VAVs) -> - mgr_user_sync_set2(Node, TargetName, VAVs, SendOpts) - end, - PostVerify = fun() -> receive Msg -> ok end end, - - Res = do_simple_sync_set2(Config, Set, PostVerify), - display_log(Config), - Res. - - -%%====================================================================== - sas_verify({noError, 0, _Vbs}, any) -> ?IPRINT("verified [any]"), ok; @@ -2930,17 +2751,31 @@ sas_verify_vbs([Vb|_], [E|_]) -> %%====================================================================== -simple_async_set2(doc) -> - ["Simple (async) set-request - Version 2 API (TargetName)"]; -simple_async_set2(suite) -> []; -simple_async_set2(Config) when is_list(Config) -> - ?TC_TRY(simple_async_set2, - fun() -> do_simple_async_set2(Config) end). +simple_async_set3_cbp_def(doc) -> + ["Simple (async) set-request - Version 3 API (TargetName with send-opts)"]; +simple_async_set3_cbp_def(suite) -> []; +simple_async_set3_cbp_def(Config) when is_list(Config) -> + simple_async_set3(sas3_cbp_def, Config). + +simple_async_set3_cbp_temp(doc) -> + ["Simple (async) set-request - Version 3 API (TargetName with send-opts)"]; +simple_async_set3_cbp_temp(suite) -> []; +simple_async_set3_cbp_temp(Config) when is_list(Config) -> + simple_async_set3(sas3_cbp_temp, Config). + +simple_async_set3_cbp_perm(doc) -> + ["Simple (async) set-request - Version 3 API (TargetName with send-opts)"]; +simple_async_set3_cbp_perm(suite) -> []; +simple_async_set3_cbp_perm(Config) when is_list(Config) -> + simple_async_set3(sas3_cbp_perm, Config). + +simple_async_set3(Case, Config) -> + ?TC_TRY(Case, + fun() -> do_simple_async_set3(Config) end). -do_simple_async_set2(Config) -> +do_simple_async_set3(Config) -> ?IPRINT("starting with Config: " - "~n ~p" - "~n", [Config]), + "~n ~p~n", [Config]), MgrNode = ?config(manager_node, Config), AgentNode = ?config(agent_node, Config), @@ -2951,17 +2786,28 @@ do_simple_async_set2(Config) -> ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), ?line ok = agent_load_mib(AgentNode, Test2Mib), + Self = self(), + Msg = simple_async_set3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], + Set = fun(Oids) -> - async_s_exec2(MgrNode, TargetName, Oids) + async_s_exec3(MgrNode, TargetName, Oids, SendOpts) end, - PostVerify = fun(Res) -> Res end, + PostVerify = fun(ok) -> receive Msg -> ok end; + (Res) -> Res + end, - Res = do_simple_async_set2(MgrNode, AgentNode, Set, PostVerify), + Res = do_simple_async_set3(MgrNode, AgentNode, Set, PostVerify), display_log(Config), Res. -do_simple_async_set2(MgrNode, AgentNode, Set, PostVerify) -> +do_simple_async_set3(MgrNode, AgentNode, Set, PostVerify) -> Requests = [ {1, @@ -3005,68 +2851,6 @@ do_simple_async_set2(MgrNode, AgentNode, Set, PostVerify) -> ok. -async_s_exec2(Node, TargetName, VAVs) -> - mgr_user_async_set(Node, TargetName, VAVs). - - -%%====================================================================== - -simple_async_set3_cbp_def(doc) -> - ["Simple (async) set-request - Version 3 API (TargetName with send-opts)"]; -simple_async_set3_cbp_def(suite) -> []; -simple_async_set3_cbp_def(Config) when is_list(Config) -> - simple_async_set3(sas3_cbp_def, Config). - -simple_async_set3_cbp_temp(doc) -> - ["Simple (async) set-request - Version 3 API (TargetName with send-opts)"]; -simple_async_set3_cbp_temp(suite) -> []; -simple_async_set3_cbp_temp(Config) when is_list(Config) -> - simple_async_set3(sas3_cbp_temp, Config). - -simple_async_set3_cbp_perm(doc) -> - ["Simple (async) set-request - Version 3 API (TargetName with send-opts)"]; -simple_async_set3_cbp_perm(suite) -> []; -simple_async_set3_cbp_perm(Config) when is_list(Config) -> - simple_async_set3(sas3_cbp_perm, Config). - -simple_async_set3(Case, Config) -> - ?TC_TRY(Case, - fun() -> do_simple_async_set3(Config) end). - -do_simple_async_set3(Config) -> - ?IPRINT("starting with Config: " - "~n ~p~n", [Config]), - - MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), - TargetName = ?config(manager_agent_target_name, Config), - - ?line ok = mgr_user_load_mib(MgrNode, std_mib()), - Test2Mib = test2_mib(Config), - ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), - ?line ok = agent_load_mib(AgentNode, Test2Mib), - - Self = self(), - Msg = simple_async_set3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - - Set = - fun(Oids) -> - async_s_exec3(MgrNode, TargetName, Oids, SendOpts) - end, - PostVerify = fun(ok) -> receive Msg -> ok end; - (Res) -> Res - end, - - Res = do_simple_async_set2(MgrNode, AgentNode, Set, PostVerify), - display_log(Config), - Res. - async_s_exec3(Node, TargetName, VAVs, SendOpts) -> mgr_user_async_set2(Node, TargetName, VAVs, SendOpts). @@ -3110,14 +2894,15 @@ check_ssgb_vbs([R|_], [E|_]) -> %%====================================================================== -simple_sync_get_bulk2(doc) -> - ["Simple (sync) get_bulk-request - Version 2 API (TargetName)"]; -simple_sync_get_bulk2(suite) -> []; -simple_sync_get_bulk2(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_get_bulk2, - fun() -> do_simple_sync_get_bulk2(Config) end). +simple_sync_get_bulk3(doc) -> + ["Simple (sync) get_bulk-request - " + "Version 3 API (TargetName with send-opts)"]; +simple_sync_get_bulk3(suite) -> []; +simple_sync_get_bulk3(Config) when is_list(Config) -> + ?TC_TRY(simple_sync_get_bulk3, + fun() -> do_simple_sync_get_bulk3(Config) end). -do_simple_sync_get_bulk2(Config) -> +do_simple_sync_get_bulk3(Config) -> ?IPRINT("starting with Config: " "~n ~p~n", [Config]), @@ -3125,32 +2910,43 @@ do_simple_sync_get_bulk2(Config) -> AgentNode = ?config(agent_node, Config), TargetName = ?config(manager_agent_target_name, Config), + Self = self(), + Msg = simple_async_set3, + Fun = fun() -> Self ! Msg end, + Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, + SendOpts = + [ + {extra, Extra} + ], + GetBulk = fun(NonRep, MaxRep, Oids) -> - mgr_user_sync_get_bulk(MgrNode, TargetName, - NonRep, MaxRep, Oids) + mgr_user_sync_get_bulk2(MgrNode, TargetName, + NonRep, MaxRep, Oids, SendOpts) end, - PostVerify = fun(Res) -> Res end, + PostVerify = fun(ok) -> receive Msg -> ok end; + (Res) -> Res + end, - Res = do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify), + Res = do_simple_sync_get_bulk3(Config, MgrNode, AgentNode, GetBulk, PostVerify), display_log(Config), Res. -do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> +do_simple_sync_get_bulk3(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> %% -- 1 -- - ?line ok = do_simple_get_bulk2(1, + ?line ok = do_simple_get_bulk3(1, 1, 1, [], fun verify_ssgb_reply1/1, GetBulk, PostVerify), %% -- 2 -- - ?line ok = do_simple_get_bulk2(2, + ?line ok = do_simple_get_bulk3(2, -1, 1, [], fun verify_ssgb_reply1/1, GetBulk, PostVerify), %% -- 3 -- - ?line ok = do_simple_get_bulk2(3, + ?line ok = do_simple_get_bulk3(3, -1, -1, [], fun verify_ssgb_reply1/1, GetBulk, PostVerify), @@ -3160,12 +2956,12 @@ do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> VF04 = fun(X) -> verify_ssgb_reply2(X, [?sysDescr_instance, endOfMibView]) end, - ?line ok = do_simple_get_bulk2(4, + ?line ok = do_simple_get_bulk3(4, 2, 0, [[sysDescr],[1,3,7,1]], VF04, GetBulk, PostVerify), %% -- 5 -- - ?line ok = do_simple_get_bulk2(5, + ?line ok = do_simple_get_bulk3(5, 1, 2, [[sysDescr],[1,3,7,1]], VF04, GetBulk, PostVerify), @@ -3175,7 +2971,7 @@ do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> [?sysDescr_instance, endOfMibView, ?sysObjectID_instance, endOfMibView]) end, - ?line ok = do_simple_get_bulk2(6, + ?line ok = do_simple_get_bulk3(6, 0, 2, [[sysDescr],[1,3,7,1]], VF06, GetBulk, PostVerify), @@ -3186,7 +2982,7 @@ do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> ?sysDescr_instance, endOfMibView, ?sysObjectID_instance, endOfMibView]) end, - ?line ok = do_simple_get_bulk2(7, + ?line ok = do_simple_get_bulk3(7, 2, 2, [[sysDescr],[1,3,7,1],[sysDescr],[1,3,7,1]], VF07, @@ -3202,14 +2998,14 @@ do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> [?sysDescr_instance, ?sysDescr_instance]) end, - ?line ok = do_simple_get_bulk2(8, + ?line ok = do_simple_get_bulk3(8, 1, 2, [[sysDescr],[sysDescr],[tTooBig]], VF08, GetBulk, PostVerify), %% -- 9 -- - ?line ok = do_simple_get_bulk2(9, + ?line ok = do_simple_get_bulk3(9, 1, 12, [[tDescr2], [sysDescr]], fun verify_ssgb_reply1/1, @@ -3223,7 +3019,7 @@ do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> {?tGenErr1, 'NULL'}, {?sysDescr, 'NULL'}]) end, - ?line ok = do_simple_get_bulk2(10, + ?line ok = do_simple_get_bulk3(10, 2, 2, [[sysDescr], [sysObjectID], @@ -3240,14 +3036,14 @@ do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify) -> [{fl([TCnt2,2]), 100}, {fl([TCnt2,2]), endOfMibView}]) end, - ?line ok = do_simple_get_bulk2(11, + ?line ok = do_simple_get_bulk3(11, 0, 2, [[TCnt2, 1]], VF11, GetBulk, PostVerify), ok. -do_simple_get_bulk2(N, +do_simple_get_bulk3(N, NonRep, MaxRep, Oids, Verify, GetBulk, PostVerify) when is_function(Verify, 1) andalso @@ -3268,15 +3064,19 @@ do_simple_get_bulk2(N, %%====================================================================== -simple_sync_get_bulk3(doc) -> - ["Simple (sync) get_bulk-request - " +simple_async_get_bulk3_cbp_def(doc) -> + ["Simple (async) get_bulk-request - " "Version 3 API (TargetName with send-opts)"]; -simple_sync_get_bulk3(suite) -> []; -simple_sync_get_bulk3(Config) when is_list(Config) -> - ?TC_TRY(simple_sync_get_bulk3, - fun() -> do_simple_sync_get_bulk3(Config) end). +simple_async_get_bulk3_cbp_def(suite) -> []; +simple_async_get_bulk3_cbp_def(Config) when is_list(Config) -> + simple_async_get_bulk3(sagb3_cbp_def, Config). -do_simple_sync_get_bulk3(Config) -> +simple_async_get_bulk3(Case, Config) -> + ?TC_TRY(Case, + fun() -> do_simple_async_get_bulk3(Config) end). + +do_simple_async_get_bulk3(Config) -> + process_flag(trap_exit, true), ?IPRINT("starting with Config: " "~n ~p~n", [Config]), @@ -3284,8 +3084,13 @@ do_simple_sync_get_bulk3(Config) -> AgentNode = ?config(agent_node, Config), TargetName = ?config(manager_agent_target_name, Config), + ?line ok = mgr_user_load_mib(MgrNode, std_mib()), + Test2Mib = test2_mib(Config), + ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), + ?line ok = agent_load_mib(AgentNode, Test2Mib), + Self = self(), - Msg = simple_async_set3, + Msg = simple_async_get_bulk3, Fun = fun() -> Self ! Msg end, Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, SendOpts = @@ -3294,52 +3099,18 @@ do_simple_sync_get_bulk3(Config) -> ], GetBulk = - fun(NonRep, MaxRep, Oids) -> - mgr_user_sync_get_bulk2(MgrNode, TargetName, - NonRep, MaxRep, Oids, SendOpts) + fun(Data) -> + async_gb_exec3(MgrNode, TargetName, Data, SendOpts) end, - PostVerify = fun(ok) -> receive Msg -> ok end; + PostVerify = fun(ok) -> receive Msg -> ok end; (Res) -> Res end, - Res = do_simple_sync_get_bulk2(Config, MgrNode, AgentNode, GetBulk, PostVerify), + Res = do_simple_async_get_bulk3(MgrNode, AgentNode, GetBulk, PostVerify), display_log(Config), Res. - -%%====================================================================== - -simple_async_get_bulk2(doc) -> - ["Simple (async) get_bulk-request - Version 2 API (TargetName)"]; -simple_async_get_bulk2(suite) -> []; -simple_async_get_bulk2(Config) when is_list(Config) -> - ?TC_TRY(simple_async_get_bulk2, - fun() -> do_simple_async_get_bulk2(Config) end). - -do_simple_async_get_bulk2(Config) -> - ?IPRINT("starting with Config: " - "~p ~n", [Config]), - - MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), - TargetName = ?config(manager_agent_target_name, Config), - - ?line ok = mgr_user_load_mib(MgrNode, std_mib()), - Test2Mib = test2_mib(Config), - ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), - ?line ok = agent_load_mib(AgentNode, Test2Mib), - - GetBulk = - fun(Data) -> - async_gb_exec2(MgrNode, TargetName, Data) - end, - PostVerify = fun(Res) -> Res end, - - Res = do_simple_async_get_bulk2(MgrNode, AgentNode, GetBulk, PostVerify), - display_log(Config), - Res. - -do_simple_async_get_bulk2(MgrNode, AgentNode, GetBulk, PostVerify) -> +do_simple_async_get_bulk3(MgrNode, AgentNode, GetBulk, PostVerify) -> %% We re-use the verification functions from the ssgb test-case VF04 = fun(X) -> PostVerify( @@ -3465,58 +3236,6 @@ do_simple_async_get_bulk2(MgrNode, AgentNode, GetBulk, PostVerify) -> ok. -async_gb_exec2(Node, TargetName, {NR, MR, Oids}) -> - mgr_user_async_get_bulk(Node, TargetName, NR, MR, Oids). - - -%%====================================================================== - -simple_async_get_bulk3_cbp_def(doc) -> - ["Simple (async) get_bulk-request - " - "Version 3 API (TargetName with send-opts)"]; -simple_async_get_bulk3_cbp_def(suite) -> []; -simple_async_get_bulk3_cbp_def(Config) when is_list(Config) -> - simple_async_get_bulk3(sagb3_cbp_def, Config). - -simple_async_get_bulk3(Case, Config) -> - ?TC_TRY(Case, - fun() -> do_simple_async_get_bulk3(Config) end). - -do_simple_async_get_bulk3(Config) -> - process_flag(trap_exit, true), - ?IPRINT("starting with Config: " - "~n ~p~n", [Config]), - - MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), - TargetName = ?config(manager_agent_target_name, Config), - - ?line ok = mgr_user_load_mib(MgrNode, std_mib()), - Test2Mib = test2_mib(Config), - ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), - ?line ok = agent_load_mib(AgentNode, Test2Mib), - - Self = self(), - Msg = simple_async_get_bulk3, - Fun = fun() -> Self ! Msg end, - Extra = {?SNMPM_EXTRA_INFO_TAG, Fun}, - SendOpts = - [ - {extra, Extra} - ], - - GetBulk = - fun(Data) -> - async_gb_exec3(MgrNode, TargetName, Data, SendOpts) - end, - PostVerify = fun(ok) -> receive Msg -> ok end; - (Res) -> Res - end, - - Res = do_simple_async_get_bulk2(MgrNode, AgentNode, GetBulk, PostVerify), - display_log(Config), - Res. - async_gb_exec3(Node, TargetName, {NR, MR, Oids}, SendOpts) -> mgr_user_async_get_bulk2(Node, TargetName, NR, MR, Oids, SendOpts). @@ -3543,203 +3262,6 @@ simple_async_get_bulk3_cbp_perm(Config) when is_list(Config) -> %%====================================================================== -misc_async2(doc) -> - ["Misc (async) request(s) - Version 2 API (TargetName)"]; -misc_async2(suite) -> []; -misc_async2(Config) when is_list(Config) -> - ?TC_TRY(misc_async2, - fun() -> do_misc_async2(Config) end). - -do_misc_async2(Config) -> - ?IPRINT("starting with Config: " - "~n ~p" - "~n", [Config]), - - MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), - TargetName = ?config(manager_agent_target_name, Config), - - ?line ok = mgr_user_load_mib(MgrNode, std_mib()), - Test2Mib = test2_mib(Config), - ?line ok = mgr_user_load_mib(MgrNode, Test2Mib), - ?line ok = agent_load_mib(AgentNode, Test2Mib), - - ExecG = fun(Data) -> - async_g_exec2(MgrNode, TargetName, Data) - end, - - ExecGN = fun(Data) -> - async_gn_exec2(MgrNode, TargetName, Data) - end, - - ExecS = fun(Data) -> - async_s_exec2(MgrNode, TargetName, Data) - end, - - ExecGB = fun(Data) -> - async_gb_exec2(MgrNode, TargetName, Data) - end, - - ?line {ok, [TCnt2|_]} = mgr_user_name_to_oid(MgrNode, tCnt2), - ?line {ok, [TGenErr1|_]} = mgr_user_name_to_oid(MgrNode, tGenErr1), - ?line {ok, [TGenErr2|_]} = mgr_user_name_to_oid(MgrNode, tGenErr2), - ?line {ok, [TGenErr3|_]} = mgr_user_name_to_oid(MgrNode, tGenErr3), - ?line {ok, [TTooBig|_]} = mgr_user_name_to_oid(MgrNode, tTooBig), - - Requests = - [ - { 1, - [?sysObjectID_instance], - ExecG, - fun(X) -> - sag_verify(X, [?sysObjectID_instance]) - end - }, - { 2, - {1, 1, []}, - ExecGB, - fun verify_ssgb_reply1/1}, - { 3, - {-1, 1, []}, - ExecGB, - fun verify_ssgb_reply1/1}, - { 4, - [{?sysLocation_instance, s, "Stockholm"}, - {?sysName_instance, s, "Arne Anka"}], - ExecS, - fun(X) -> - sas_verify(X, [?sysLocation_instance, ?sysName_instance]) - end}, - { 5, - [[sysDescr], [1,3,7,1]], - ExecGN, - fun(X) -> - verify_ssgn_reply1(X, [?sysDescr_instance, endOfMibView]) - end}, - { 6, - [[sysObjectID, 0], [sysDescr, 0], [sysUpTime, 0]], - ExecG, - fun(X) -> - sag_verify(X, [?sysObjectID_instance, - ?sysDescr_instance, - ?sysUpTime_instance]) - end}, - { 7, - [TGenErr2], - ExecGN, - fun(X) -> - verify_ssgn_reply2(X, {genErr, 1, [TGenErr2]}) - end}, - { 8, - {2, 0, [[sysDescr],[1,3,7,1]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply2(X, [?sysDescr_instance, endOfMibView]) - end}, - { 9, - {1, 2, [[sysDescr],[1,3,7,1]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply2(X, [?sysDescr_instance, endOfMibView]) - end}, - {10, - [TGenErr1], - ExecGN, - fun(X) -> - verify_ssgn_reply2(X, {genErr, 1, [TGenErr1]}) - end}, - {11, - {0, 2, [[sysDescr],[1,3,7,1]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply2(X, - [?sysDescr_instance, endOfMibView, - ?sysObjectID_instance, endOfMibView]) - end}, - {12, - [{[sysName, 0], "Gothenburg"}, - {[sysLocation, 0], "Sune Anka"}], - ExecS, - fun(X) -> - sas_verify(X, [?sysName_instance, ?sysLocation_instance]) - end}, - {13, - {2, 2, [[sysDescr],[1,3,7,1],[sysDescr],[1,3,7,1]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply2(X, - [?sysDescr_instance, endOfMibView, - ?sysDescr_instance, endOfMibView, - ?sysObjectID_instance, endOfMibView]) - end}, - {14, - {1, 2, [[sysDescr],[sysDescr],[tTooBig]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply2(X, - [?sysDescr_instance, - ?sysDescr_instance]) - end}, - {15, - {1, 12, [[tDescr2], [sysDescr]]}, - ExecGB, - fun verify_ssgb_reply1/1}, - {16, - {2, 2, [[sysDescr],[sysObjectID], [tGenErr1],[sysDescr]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply3(X, - [{?sysDescr, 'NULL'}, - {?sysObjectID, 'NULL'}, - {?tGenErr1, 'NULL'}, - {?sysDescr, 'NULL'}]) - end}, - {17, - [[sysDescr], TGenErr3], - ExecGN, - fun(X) -> - verify_ssgn_reply2(X, {genErr, 2, [TGenErr3]}) - end}, - {18, - {0, 2, [[TCnt2, 1]]}, - ExecGB, - fun(X) -> - verify_ssgb_reply2(X, - [{fl([TCnt2,2]), 100}, - {fl([TCnt2,2]), endOfMibView}]) - end}, - {19, - [TTooBig], - ExecGN, - fun(X) -> - verify_ssgn_reply2(X, {tooBig, 0, []}) - end}, - {20, - [TTooBig], - ExecGN, - fun(X) -> - verify_ssgn_reply2(X, {tooBig, 0, []}) - end} - ], - - ?IPRINT("manager info when starting test: " - "~n ~p", [mgr_info(MgrNode)]), - ?IPRINT("agent info when starting test: " - "~n ~p", [agent_info(AgentNode)]), - - ?line ok = async_exec(Requests, []), - - ?IPRINT("manager info when ending test: " - "~n ~p", [mgr_info(MgrNode)]), - ?IPRINT("agent info when ending test: " - "~n ~p", [agent_info(AgentNode)]), - - display_log(Config), - ok. - - -%%====================================================================== - discovery(suite) -> []; discovery(Config) when is_list(Config) -> ?SKIP(not_yet_implemented). @@ -4764,7 +4286,8 @@ do_inform_swarm(Config) -> "~p ~n", [Config]), MgrNode = ?config(manager_node, Config), - AgentNode = ?config(agent_node, Config), + AgentNode = ?config(agent_node, Config), + Factor = ?config(snmp_factor, Config), ?line ok = mgr_user_load_mib(MgrNode, snmpv2_mib()), Test2Mib = test2_mib(Config), @@ -4776,7 +4299,7 @@ do_inform_swarm(Config) -> ?line ok = agent_load_mib(AgentNode, Test2Mib), ?line ok = agent_load_mib(AgentNode, TestTrapMib), ?line ok = agent_load_mib(AgentNode, TestTrapv2Mib), - NumInforms = 2000, + NumInforms = 2000 div Factor, Collector = self(), @@ -4796,8 +4319,8 @@ do_inform_swarm(Config) -> {{inform2_tag1, N}, Collector}, "standard inform", []), - %% Sleep some [(N div 10)*100 ms] - %% every tenth notification + %% Sleep 1000 ms every 100th notif + %% Sleep 100 ms every 10th notif if N rem 100 == 0 -> Sleep = 1000, @@ -4836,11 +4359,11 @@ do_inform_swarm(Config) -> Commands = [ - {1, "Manager and agent info at start of test", Cmd1}, - {2, "Send notifcation(s) from agent", Cmd2}, - {3, "Await send-ack(s)/inform(s)/response(s)", Cmd3}, - {4, "Sleep some time (1 sec)", Cmd4}, - {5, "Manager and agent info after test completion", Cmd1} + {1, "Manager and agent info at start of test", Cmd1}, + {2, ?F("Send ~p notifcation(s) from agent", [NumInforms]), Cmd2}, + {3, "Await send-ack(s)/inform(s)/response(s)", Cmd3}, + {4, "Sleep some time (1 sec)", Cmd4}, + {5, "Manager and agent info after test completion", Cmd1} ], Res = command_handler(Commands), @@ -4885,7 +4408,7 @@ inform_swarm_collector(N, SentAckCnt, RecvCnt, RespCnt, Timeout) -> %% The manager has received the actual inform {async_event, From, {inform, Pid, Inform}} -> - ?IPRINT("received inform"), + ?IPRINT("received inform (~p of ~p)", [RecvCnt+1, N]), case Inform of {noError, 0, VBs} when is_list(VBs) -> Pid ! {handle_inform_response, From}, @@ -5001,7 +4524,7 @@ otp8395_1(Config) when is_list(Config) -> fun() -> do_otp8395_1(Config) end). do_otp8395_1(Config) -> - do_simple_sync_get2(Config). + do_simple_sync_get3(Config). %%====================================================================== @@ -5707,16 +5230,16 @@ mgr_user_load_mib(Node, Mib) -> %% mgr_user_sync_get(Node, Oids) -> %% mgr_user_sync_get(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_sync_get(Node, TargetName, Oids) when is_list(TargetName) -> - rcall(Node, snmp_manager_user, sync_get, [TargetName, Oids]). +%% mgr_user_sync_get(Node, TargetName, Oids) when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, sync_get, [TargetName, Oids]). mgr_user_sync_get2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_get2, [TargetName, Oids, SendOpts]). %% mgr_user_async_get(Node, Oids) -> %% mgr_user_async_get(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_async_get(Node, TargetName, Oids) when is_list(TargetName) -> - rcall(Node, snmp_manager_user, async_get, [TargetName, Oids]). +%% mgr_user_async_get(Node, TargetName, Oids) when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, async_get, [TargetName, Oids]). mgr_user_async_get2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> @@ -5724,8 +5247,8 @@ mgr_user_async_get2(Node, TargetName, Oids, SendOpts) %% mgr_user_sync_get_next(Node, Oids) -> %% mgr_user_sync_get_next(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_sync_get_next(Node, TargetName, Oids) when is_list(TargetName) -> - rcall(Node, snmp_manager_user, sync_get_next, [TargetName, Oids]). +%% mgr_user_sync_get_next(Node, TargetName, Oids) when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, sync_get_next, [TargetName, Oids]). mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> @@ -5733,8 +5256,8 @@ mgr_user_sync_get_next2(Node, TargetName, Oids, SendOpts) %% mgr_user_async_get_next(Node, Oids) -> %% mgr_user_async_get_next(Node, ?LOCALHOST(), ?AGENT_PORT, Oids). -mgr_user_async_get_next(Node, TargetName, Oids) when is_list(TargetName) -> - rcall(Node, snmp_manager_user, async_get_next, [TargetName, Oids]). +%% mgr_user_async_get_next(Node, TargetName, Oids) when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, async_get_next, [TargetName, Oids]). mgr_user_async_get_next2(Node, TargetName, Oids, SendOpts) when is_list(TargetName) -> @@ -5742,16 +5265,16 @@ mgr_user_async_get_next2(Node, TargetName, Oids, SendOpts) %% mgr_user_sync_set(Node, VAV) -> %% mgr_user_sync_set(Node, ?LOCALHOST(), ?AGENT_PORT, VAV). -mgr_user_sync_set(Node, TargetName, VAV) when is_list(TargetName) -> - rcall(Node, snmp_manager_user, sync_set, [TargetName, VAV]). +%% mgr_user_sync_set(Node, TargetName, VAV) when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, sync_set, [TargetName, VAV]). mgr_user_sync_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, sync_set2, [TargetName, VAV, SendOpts]). %% mgr_user_async_set(Node, VAV) -> %% mgr_user_async_set(Node, ?LOCALHOST(), ?AGENT_PORT, VAV). -mgr_user_async_set(Node, TargetName, VAV) when is_list(TargetName) -> - rcall(Node, snmp_manager_user, async_set, [TargetName, VAV]). +%% mgr_user_async_set(Node, TargetName, VAV) when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, async_set, [TargetName, VAV]). mgr_user_async_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> rcall(Node, snmp_manager_user, async_set2, [TargetName, VAV, SendOpts]). @@ -5759,10 +5282,10 @@ mgr_user_async_set2(Node, TargetName, VAV, SendOpts) when is_list(TargetName) -> %% mgr_user_sync_get_bulk(Node, NonRep, MaxRep, Oids) -> %% mgr_user_sync_get_bulk(Node, ?LOCALHOST(), ?AGENT_PORT, %% NonRep, MaxRep, Oids). -mgr_user_sync_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) - when is_list(TargetName) -> - rcall(Node, snmp_manager_user, sync_get_bulk, - [TargetName, NonRep, MaxRep, Oids]). +%% mgr_user_sync_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) +%% when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, sync_get_bulk, +%% [TargetName, NonRep, MaxRep, Oids]). mgr_user_sync_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) when is_list(TargetName) -> @@ -5772,10 +5295,10 @@ mgr_user_sync_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) %% mgr_user_async_get_bulk(Node, NonRep, MaxRep, Oids) -> %% mgr_user_async_get_bulk(Node, ?LOCALHOST(), ?AGENT_PORT, %% NonRep, MaxRep, Oids). -mgr_user_async_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) - when is_list(TargetName) -> - rcall(Node, snmp_manager_user, async_get_bulk, - [TargetName, NonRep, MaxRep, Oids]). +%% mgr_user_async_get_bulk(Node, TargetName, NonRep, MaxRep, Oids) +%% when is_list(TargetName) -> +%% rcall(Node, snmp_manager_user, async_get_bulk, +%% [TargetName, NonRep, MaxRep, Oids]). mgr_user_async_get_bulk2(Node, TargetName, NonRep, MaxRep, Oids, SendOpts) when is_list(TargetName) -> diff --git a/lib/snmp/test/snmp_manager_user.erl b/lib/snmp/test/snmp_manager_user.erl index 409f87cf40..60a6844875 100644 --- a/lib/snmp/test/snmp_manager_user.erl +++ b/lib/snmp/test/snmp_manager_user.erl @@ -51,14 +51,14 @@ update_agent_info/3, which_all_agents/0, which_own_agents/0, load_mib/1, unload_mib/1, - sync_get/1, sync_get/2, sync_get2/3, - async_get/1, async_get/2, async_get2/3, - sync_get_next/1, sync_get_next/2, sync_get_next2/3, - async_get_next/1, async_get_next/2, async_get_next2/3, - sync_set/1, sync_set/2, sync_set2/3, - async_set/1, async_set/2, async_set2/3, - sync_get_bulk/3, sync_get_bulk/4, sync_get_bulk2/5, - async_get_bulk/3, async_get_bulk/4, async_get_bulk2/5, + sync_get2/3, + async_get2/3, + sync_get_next2/3, + async_get_next2/3, + sync_set2/3, + async_set2/3, + sync_get_bulk2/5, + async_get_bulk2/5, name_to_oid/1, oid_to_name/1, purify_oid/1 ]). @@ -159,90 +159,42 @@ unload_mib(Mib) -> %% -- -sync_get(Oids) -> - call({sync_get, Oids}). - -sync_get(TargetName, Oids) -> - call({sync_get, TargetName, Oids}). - sync_get2(TargetName, Oids, SendOpts) -> call({sync_get2, TargetName, Oids, SendOpts}). %% -- -async_get(Oids) -> - call({async_get, Oids}). - -async_get(TargetName, Oids) -> - call({async_get, TargetName, Oids}). - async_get2(TargetName, Oids, SendOpts) -> call({async_get2, TargetName, Oids, SendOpts}). %% -- -sync_get_next(Oids) -> - call({sync_get_next, Oids}). - -sync_get_next(TargetName, Oids) -> - call({sync_get_next, TargetName, Oids}). - sync_get_next2(TargetName, Oids, SendOpts) -> call({sync_get_next2, TargetName, Oids, SendOpts}). %% -- -async_get_next(Oids) -> - call({async_get_next, Oids}). - -async_get_next(TargetName, Oids) -> - call({async_get_next, TargetName, Oids}). - async_get_next2(TargetName, Oids, SendOpts) -> call({async_get_next2, TargetName, Oids, SendOpts}). %% -- -sync_set(VAV) -> - call({sync_set, VAV}). - -sync_set(TargetName, VAV) -> - call({sync_set, TargetName, VAV}). - sync_set2(TargetName, VAV, SendOpts) -> call({sync_set2, TargetName, VAV, SendOpts}). %% -- -async_set(VAV) -> - call({async_set, VAV}). - -async_set(TargetName, VAV) -> - call({async_set, TargetName, VAV}). - async_set2(TargetName, VAV, SendOpts) -> call({async_set2, TargetName, VAV, SendOpts}). %% -- -sync_get_bulk(NonRep, MaxRep, Oids) -> - call({sync_get_bulk, NonRep, MaxRep, Oids}). - -sync_get_bulk(TargetName, NonRep, MaxRep, Oids) -> - call({sync_get_bulk, TargetName, NonRep, MaxRep, Oids}). - sync_get_bulk2(TargetName, NonRep, MaxRep, Oids, SendOpts) -> call({sync_get_bulk2, TargetName, NonRep, MaxRep, Oids, SendOpts}). %% -- -async_get_bulk(NonRep, MaxRep, Oids) -> - call({async_get_bulk, NonRep, MaxRep, Oids}). - -async_get_bulk(TargetName, NonRep, MaxRep, Oids) -> - call({async_get_bulk, TargetName, NonRep, MaxRep, Oids}). - async_get_bulk2(TargetName, NonRep, MaxRep, Oids, SendOpts) -> call({async_get_bulk2, TargetName, NonRep, MaxRep, Oids, SendOpts}). @@ -377,23 +329,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - %% No agent specified, so send it to all of them - {{sync_get, Oids}, From, Ref} -> - d("loop -> received sync_get request " - "(for every agent of this user)"), - Res = [snmpm:sync_get(Id, TargetName, Oids) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{sync_get, TargetName, Oids}, From, Ref} when is_list(TargetName) -> - d("loop -> received sync_get request with" - "~n TargetName: ~p" - "~n Oids: ~p", [TargetName, Oids]), - Res = snmpm:sync_get(Id, TargetName, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) get-request -- @@ -409,22 +344,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - %% No agent specified, so send it to all of them - {{async_get, Oids}, From, Ref} -> - d("loop -> received async_get request"), - Res = [snmpm:async_get(Id, TargetName, Oids) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{async_get, TargetName, Oids}, From, Ref} when is_list(TargetName) -> - d("loop -> received async_get request with" - "~n TargetName: ~p" - "~n Oids: ~p", [TargetName, Oids]), - Res = snmpm:async_get(Id, TargetName, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (sync) get_next-request -- @@ -440,22 +359,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - %% No agent specified, so send it to all of them - {{sync_get_next, Oids}, From, Ref} -> - d("loop -> received sync_get_next request"), - Res = [snmpm:sync_get_next(Id, TargetName, Oids) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{sync_get_next, TargetName, Oids}, From, Ref} when is_list(TargetName) -> - d("loop -> received sync_get_next request with" - "~n TargetName: ~p" - "~n Oids: ~p", [TargetName, Oids]), - Res = snmpm:sync_get_next(Id, TargetName, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) get_next-request -- @@ -471,22 +374,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - %% No agent specified, so send it to all of them - {{async_get_next, Oids}, From, Ref} -> - d("loop -> received async_get_next request"), - Res = [snmpm:async_get_next(Id, TargetName, Oids) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{async_get_next, TargetName, Oids}, From, Ref} when is_list(TargetName) -> - d("loop -> received async_get_next request with" - "~n TargetName: ~p" - "~n Oids: ~p", [TargetName, Oids]), - Res = snmpm:async_get_next(Id, TargetName, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (sync) set-request -- @@ -502,19 +389,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{sync_set, VAV}, From, Ref} -> - d("loop -> received sync_set request"), - Res = [snmpm:sync_set(Id, TargetName, VAV) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{sync_set, TargetName, VAV}, From, Ref} when is_list(TargetName) -> - d("loop -> received sync_set request"), - Res = snmpm:sync_set(Id, TargetName, VAV), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) set-request -- @@ -530,19 +404,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - {{async_set, VAV}, From, Ref} -> - d("loop -> received async_set request"), - Res = [snmpm:async_set(Id, TargetName, VAV) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{async_set, TargetName, VAV}, From, Ref} when is_list(TargetName) -> - d("loop -> received async_set request"), - Res = snmpm:async_set(Id, TargetName, VAV), - reply(From, Res, Ref), - loop(S); - %% %% -- (sync) get-bulk-request -- @@ -562,27 +423,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - %% No agent specified, so send it to all of them - {{sync_get_bulk, NonRep, MaxRep, Oids}, From, Ref} -> - d("loop -> received sync_get_bulk request with" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [NonRep, MaxRep, Oids]), - Res = [snmpm:sync_get_bulk(Id, TargetName, NonRep, MaxRep, Oids) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{sync_get_bulk, TargetName, NonRep, MaxRep, Oids}, From, Ref} when is_list(TargetName) -> - d("loop -> received sync_get_bulk request with" - "~n TargetName: ~p" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [TargetName, NonRep, MaxRep, Oids]), - Res = snmpm:sync_get_bulk(Id, TargetName, NonRep, MaxRep, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- (async) get-bulk-request -- @@ -602,27 +442,6 @@ loop(#state{parent = Parent, id = Id} = S) -> reply(From, Res, Ref), loop(S); - %% No agent specified, so send it to all of them - {{async_get_bulk, NonRep, MaxRep, Oids}, From, Ref} -> - d("loop -> received async_get_bulk request with" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [NonRep, MaxRep, Oids]), - Res = [snmpm:async_get_bulk(Id, TargetName, NonRep, MaxRep, Oids) || - TargetName <- snmpm:which_agents(Id)], - reply(From, Res, Ref), - loop(S); - - {{async_get_bulk, TargetName, NonRep, MaxRep, Oids}, From, Ref} when is_list(TargetName) -> - d("loop -> received async_get_bulk request with" - "~n TargetName: ~p" - "~n NonRep: ~w" - "~n MaxRep: ~w" - "~n Oids: ~p", [TargetName, NonRep, MaxRep, Oids]), - Res = snmpm:async_get_bulk(Id, TargetName, NonRep, MaxRep, Oids), - reply(From, Res, Ref), - loop(S); - %% %% -- logical name translation -- diff --git a/lib/snmp/test/snmp_manager_user_SUITE.erl b/lib/snmp/test/snmp_manager_user_SUITE.erl index eca0d8a4f9..bad746b29a 100644 --- a/lib/snmp/test/snmp_manager_user_SUITE.erl +++ b/lib/snmp/test/snmp_manager_user_SUITE.erl @@ -888,26 +888,17 @@ register_monitor_and_crash3(Conf) when is_list(Conf) -> put(tname,rlac3), %% <CONDITIONAL-SKIP> - %% The point of this is to catch machines running - %% SLES9 (2.6.5) LinuxVersionVerify = fun() -> - case os:cmd("uname -m") of - "i686" ++ _ -> -%% io:format("found an i686 machine, " -%% "now check version~n", []), - case os:version() of - {2, 6, Rev} when Rev >= 16 -> - true; - {2, Min, _} when Min > 6 -> - true; - {Maj, _, _} when Maj > 2 -> - true; - _ -> - false - end; - _ -> - true + case os:version() of + V when V > {2, 6, 16} -> + ?IPRINT("(Linux) kernel version check: " + "~p > {2, 6, 16} => *NO* SKIP", [V]), + false; + V -> + ?IPRINT("(Linux) kernel version check: " + "~p =< {2, 6, 16} => *SKIP*", [V]), + true end end, Skippable = [{unix, [{linux, LinuxVersionVerify}]}], @@ -922,10 +913,10 @@ register_monitor_and_crash3(Conf) when is_list(Conf) -> write_manager_conf(ConfDir), - Opts = [{server, [{verbosity, trace}]}, - {net_if, [{verbosity, trace}]}, + Opts = [{server, [{verbosity, trace}]}, + {net_if, [{verbosity, trace}]}, {note_store, [{verbosity, trace}]}, - {config, [{verbosity, trace}, {dir, ConfDir}, {db_dir, DbDir}]}], + {config, [{verbosity, trace}, {dir, ConfDir}, {db_dir, DbDir}]}], ?IPRINT("try starting manager"), ?line ok = snmpm:start_link(Opts), diff --git a/lib/snmp/test/snmp_test_lib.erl b/lib/snmp/test/snmp_test_lib.erl index fe6fef291d..76f424d25c 100644 --- a/lib/snmp/test/snmp_test_lib.erl +++ b/lib/snmp/test/snmp_test_lib.erl @@ -25,6 +25,7 @@ -export([tc_try/2, tc_try/3, tc_try/4, tc_try/5]). +-export([proxy_call/3]). -export([hostname/0, hostname/1, localhost/0, localhost/1, os_type/0, sz/1, display_suite_info/1]). -export([non_pc_tc_maybe_skip/4, @@ -257,6 +258,19 @@ tc_which_name() -> %% Misc functions %% +proxy_call(F, Timeout, Default) + when is_function(F, 0) andalso is_integer(Timeout) andalso (Timeout > 0) -> + {P, M} = erlang:spawn_monitor(fun() -> exit(F()) end), + receive + {'DOWN', M, process, P, Reply} -> + Reply + after Timeout -> + erlang:demonitor(M, [flush]), + exit(P, kill), + Default + end. + + hostname() -> hostname(node()). @@ -268,20 +282,6 @@ hostname(Node) -> [] end. -%% localhost() -> -%% {ok, Ip} = snmp_misc:ip(net_adm:localhost()), -%% Ip. -%% localhost(Family) -> -%% {ok, Ip} = snmp_misc:ip(net_adm:localhost(), Family), -%% Ip. - -%% localhost() -> -%% {ok, Ip} = snmp_misc:ip(net_adm:localhost()), -%% Ip. -%% localhost(Family) -> -%% {ok, Ip} = snmp_misc:ip(net_adm:localhost(), Family), -%% Ip. - localhost() -> localhost(inet). @@ -467,6 +467,10 @@ os_base_skip(Skippable, OsFam, OsName) -> case lists:keysearch(OsFam, 1, Skippable) of {value, {OsFam, OsName}} -> true; + {value, {OsFam, Check}} when is_function(Check, 0) -> + Check(); + {value, {OsFam, Check}} when is_function(Check, 1) -> + Check(os:version()); {value, {OsFam, OsNames}} when is_list(OsNames) -> %% OsNames is a list of: %% [atom()|{atom(), function/0 | function/1}] @@ -503,7 +507,7 @@ has_support_ipv6() -> %% so for windows we need to use the old style... old_has_support_ipv6(); _ -> - socket:supports(ipv6) andalso has_valid_ipv6_address() + socket:is_supported(ipv6) andalso has_valid_ipv6_address() end. has_valid_ipv6_address() -> @@ -619,11 +623,31 @@ old_is_ipv6_host(Hostname) -> init_per_suite(Config) -> + ct:timetrap(minutes(2)), + + try analyze_and_print_host_info() of + {Factor, HostInfo} when is_integer(Factor) -> + try maybe_skip(HostInfo) of + true -> + {skip, "Unstable host and/or os (or combo thererof)"}; + false -> + snmp_test_global_sys_monitor:start(), + [{snmp_factor, Factor} | Config] + catch + throw:{skip, _} = SKIP -> + SKIP + end + catch + throw:{skip, _} = SKIP -> + SKIP + end. + +maybe_skip(HostInfo) -> + %% We have some crap machines that causes random test case failures %% for no obvious reason. So, attempt to identify those without actually %% checking for the host name... - %% We have two "machines" we are checking for. Both are old installations - %% running on really slow VMs (the host machines are old and tired). + LinuxVersionVerify = fun(V) when (V > {3,6,11}) -> false; % OK - No skip @@ -634,8 +658,30 @@ init_per_suite(Config) -> _ -> false end; + (V) when (V =:= {3,4,20}) -> + case string:trim(os:cmd("cat /etc/issue")) of + "Wind River Linux 5.0.1.0" ++ _ -> % *Old* Wind River => skip + true; + _ -> + false + end; + (V) when (V =:= {2,6,32}) -> + case string:trim(os:cmd("cat /etc/issue")) of + "Debian GNU/Linux 6.0 " ++ _ -> % Stone age Debian => Skip + true; + _ -> + false + end; (V) when (V > {2,6,24}) -> false; % OK - No skip + (V) when (V =:= {2,6,10}) -> + case string:trim(os:cmd("cat /etc/issue")) of + "MontaVista" ++ _ -> % Stone age MontaVista => Skip + %% The real problem is that the machine is *very* slow + true; + _ -> + false + end; (_) -> %% We are specifically checking for %% a *really* old gento... @@ -646,15 +692,28 @@ init_per_suite(Config) -> true end end, - COND = [{unix, [{linux, LinuxVersionVerify}]}], - case os_based_skip(COND) of - true -> - {skip, "Unstable host and/or os (or combo thererof)"}; - false -> - Factor = analyze_and_print_host_info(), - snmp_test_global_sys_monitor:start(), - [{snmp_factor, Factor} | Config] - end. + DarwinVersionVerify = + fun(V) when (V > {9, 8, 0}) -> + %% This version is OK: No Skip + false; + (_V) -> + %% This version is *not* ok: Skip + true + end, + SkipWindowsOnVirtual = + fun() -> + SysMan = win_sys_info_lookup(system_manufacturer, HostInfo), + case string:to_lower(SysMan) of + "vmware" ++ _ -> + true; + _ -> + false + end + end, + COND = [{unix, [{linux, LinuxVersionVerify}, + {darwin, DarwinVersionVerify}]}, + {win32, SkipWindowsOnVirtual}], + os_based_skip(COND). end_per_suite(Config) when is_list(Config) -> @@ -795,7 +854,7 @@ skip(Reason, Module, Line) -> %% when analyzing the test suite (results). %% It also returns a "factor" that can be used when deciding %% the load for some test cases. Such as run time or number of -%% iteraions. This only works for some OSes. +%% iterations. This only works for some OSes. %% %% We make some calculations on Linux, OpenBSD and FreeBSD. %% On SunOS we always set the factor to 2 (just to be on the safe side) @@ -819,6 +878,8 @@ analyze_and_print_host_info() -> analyze_and_print_freebsd_host_info(Version); {unix, netbsd} -> analyze_and_print_netbsd_host_info(Version); + {unix, darwin} -> + analyze_and_print_darwin_host_info(Version); {unix, sunos} -> analyze_and_print_solaris_host_info(Version); {win32, nt} -> @@ -829,112 +890,288 @@ analyze_and_print_host_info() -> "~n Version: ~p" "~n Num Schedulers: ~s" "~n", [OsFam, OsName, Version, str_num_schedulers()]), - try erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - N when (N =< 6) -> - 2; - _ -> - 1 - catch - _:_:_ -> - 10 - end + {num_schedulers_to_factor(), []} end. - -analyze_and_print_linux_host_info(Version) -> + +linux_which_distro(Version) -> case file:read_file_info("/etc/issue") of {ok, _} -> - io:format("Linux: ~s" - "~n ~s" - "~n", - [Version, string:trim(os:cmd("cat /etc/issue"))]); + case [string:trim(S) || + S <- string:tokens(os:cmd("cat /etc/issue"), [$\n])] of + [DistroStr|_] -> + io:format("Linux: ~s" + "~n ~s" + "~n", + [Version, DistroStr]), + case DistroStr of + "Wind River Linux" ++ _ -> + wind_river; + "MontaVista" ++ _ -> + montavista; + "Yellow Dog" ++ _ -> + yellow_dog; + _ -> + other + end; + X -> + io:format("Linux: ~s" + "~n ~p" + "~n", + [Version, X]), + other + end; _ -> io:format("Linux: ~s" - "~n", [Version]) - end, + "~n", [Version]), + other + end. + +analyze_and_print_linux_host_info(Version) -> + Distro = + case file:read_file_info("/etc/issue") of + {ok, _} -> + linux_which_distro(Version); + _ -> + io:format("Linux: ~s" + "~n", [Version]), + other + end, Factor = - case (catch linux_which_cpuinfo()) of + case (catch linux_which_cpuinfo(Distro)) of {ok, {CPU, BogoMIPS}} -> io:format("CPU: " - "~n Model: ~s" - "~n BogoMIPS: ~s" - "~n", [CPU, BogoMIPS]), - %% We first assume its a float, and if not try integer - try list_to_float(string:trim(BogoMIPS)) of - F when F > 1000 -> + "~n Model: ~s" + "~n BogoMIPS: ~w" + "~n Num Schedulers: ~s" + "~n", [CPU, BogoMIPS, str_num_schedulers()]), + if + (BogoMIPS > 20000) -> 1; - F when F > 1000 -> + (BogoMIPS > 10000) -> 2; - _ -> - 3 - catch - _:_:_ -> - %% - try list_to_integer(string:trim(BogoMIPS)) of - I when I > 1000 -> - 1; - I when I > 1000 -> - 2; - _ -> - 3 - catch - _:_:_ -> - 1 - end + (BogoMIPS > 5000) -> + 3; + (BogoMIPS > 2000) -> + 5; + (BogoMIPS > 1000) -> + 8; + true -> + 10 + end; + {ok, CPU} -> + io:format("CPU: " + "~n Model: ~s" + "~n Num Schedulers: ~s" + "~n", [CPU, str_num_schedulers()]), + NumChed = erlang:system_info(schedulers), + if + (NumChed > 2) -> + 2; + true -> + 5 end; _ -> - 1 + 5 end, %% Check if we need to adjust the factor because of the memory try linux_which_meminfo() of AddFactor -> io:format("TS Scale Factor: ~w~n", [timetrap_scale_factor()]), - Factor + AddFactor + {Factor + AddFactor, []} catch _:_:_ -> io:format("TS Scale Factor: ~w~n", [timetrap_scale_factor()]), - Factor + {Factor, []} + end. + + + +linux_cpuinfo_lookup(Key) when is_list(Key) -> + linux_info_lookup(Key, "/proc/cpuinfo"). + +linux_cpuinfo_cpu() -> + case linux_cpuinfo_lookup("cpu") of + [Model] -> + Model; + _ -> + "-" + end. + +linux_cpuinfo_motherboard() -> + case linux_cpuinfo_lookup("motherboard") of + [MB] -> + MB; + _ -> + "-" + end. + +linux_cpuinfo_bogomips() -> + case linux_cpuinfo_lookup("bogomips") of + BMips when is_list(BMips) -> + try lists:sum([bogomips_to_int(BM) || BM <- BMips]) + catch + _:_:_ -> + "-" + end; + _ -> + "-" + end. + +linux_cpuinfo_total_bogomips() -> + case linux_cpuinfo_lookup("total bogomips") of + [TBM] -> + try bogomips_to_int(TBM) + catch + _:_:_ -> + "-" + end; + _ -> + "-" + end. + +bogomips_to_int(BM) -> + try list_to_float(BM) of + F -> + floor(F) + catch + _:_:_ -> + try list_to_integer(BM) of + I -> + I + catch + _:_:_ -> + throw(noinfo) + end + end. + +linux_cpuinfo_model() -> + case linux_cpuinfo_lookup("model") of + [M] -> + M; + _ -> + "-" + end. + +linux_cpuinfo_platform() -> + case linux_cpuinfo_lookup("platform") of + [P] -> + P; + _ -> + "-" + end. + +linux_cpuinfo_model_name() -> + case linux_cpuinfo_lookup("model name") of + [P|_] -> + P; + _ -> + "-" end. -linux_which_cpuinfo() -> - %% Check for x86 (Intel or AMD) +linux_cpuinfo_processor() -> + case linux_cpuinfo_lookup("Processor") of + [P] -> + P; + _ -> + "-" + end. + + +linux_which_cpuinfo(montavista) -> CPU = - try [string:trim(S) || S <- string:tokens(os:cmd("grep \"model name\" /proc/cpuinfo"), [$:,$\n])] of - ["model name", ModelName | _] -> - ModelName; - _ -> + case linux_cpuinfo_cpu() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_motherboard() of + "-" -> + Model; + MB -> + Model ++ " (" ++ MB ++ ")" + end + end, + case linux_cpuinfo_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end; + +linux_which_cpuinfo(yellow_dog) -> + CPU = + case linux_cpuinfo_cpu() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_motherboard() of + "-" -> + Model; + MB -> + Model ++ " (" ++ MB ++ ")" + end + end, + {ok, CPU}; + +linux_which_cpuinfo(wind_river) -> + CPU = + case linux_cpuinfo_model() of + "-" -> + throw(noinfo); + Model -> + case linux_cpuinfo_platform() of + "-" -> + Model; + Platform -> + Model ++ " (" ++ Platform ++ ")" + end + end, + case linux_cpuinfo_total_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} + end; + +%% Check for x86 (Intel or AMD) +linux_which_cpuinfo(other) -> + CPU = + case linux_cpuinfo_model_name() of + "-" -> %% ARM (at least some distros...) - try [string:trim(S) || S <- string:tokens(os:cmd("grep \"Processor\" /proc/cpuinfo"), [$:,$\n])] of - ["Processor", Proc | _] -> - Proc; - _ -> + case linux_cpuinfo_processor() of + "-" -> %% Ok, we give up - throw(noinfo) - catch - _:_:_ -> - throw(noinfo) - end - catch - _:_:_ -> - throw(noinfo) + throw(noinfo); + Proc -> + Proc + end; + ModelName -> + ModelName end, - try [string:trim(S) || S <- string:tokens(os:cmd("grep -i \"bogomips\" /proc/cpuinfo"), [$:,$\n])] of - [_, BMips | _] -> - {ok, {CPU, BMips}}; - _ -> - {ok, CPU} - catch - _:_:_ -> - {ok, CPU} + case linux_cpuinfo_bogomips() of + "-" -> + {ok, CPU}; + BMips -> + {ok, {CPU, BMips}} end. +linux_meminfo_lookup(Key) when is_list(Key) -> + linux_info_lookup(Key, "/proc/meminfo"). + +linux_meminfo_memtotal() -> + case linux_meminfo_lookup("MemTotal") of + [X] -> + X; + _ -> + "-" + end. + %% We *add* the value this return to the Factor. linux_which_meminfo() -> - try [string:trim(S) || S <- string:tokens(os:cmd("grep MemTotal /proc/meminfo"), [$:])] of - [_, MemTotal] -> + case linux_meminfo_memtotal() of + "-" -> + 0; + MemTotal -> io:format("Memory:" "~n ~s" "~n", [MemTotal]), @@ -958,18 +1195,13 @@ linux_which_meminfo() -> (MemSz3 >= 4194304) -> 1; (MemSz3 >= 2097152) -> - 2; + 3; true -> - 3 + 5 end; _X -> 0 - end; - _ -> - 0 - catch - _:_:_ -> - 0 + end end. @@ -1056,12 +1288,12 @@ analyze_and_print_openbsd_host_info(Version) -> true -> 3 end, - CPUFactor + MemAddFactor + {CPUFactor + MemAddFactor, []} end catch _:_:_ -> io:format("TS Scale Factor: ~w~n", [timetrap_scale_factor()]), - 1 + {2, []} end. @@ -1144,7 +1376,7 @@ analyze_and_print_freebsd_host_info(Version) -> true -> 3 end, - CPUFactor + MemAddFactor + {CPUFactor + MemAddFactor, []} end catch _:_:_ -> @@ -1152,14 +1384,15 @@ analyze_and_print_freebsd_host_info(Version) -> "~n Num Schedulers: ~s" "~n", [str_num_schedulers()]), io:format("TS Scale Factor: ~w~n", [timetrap_scale_factor()]), - case erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - _ -> - 2 - end + Factor = case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end, + {Factor, []} end. @@ -1283,21 +1516,22 @@ analyze_and_print_netbsd_host_info(Version) -> true -> 3 end, - CPUFactor + MemAddFactor + {CPUFactor + MemAddFactor, []} end catch _:_:_ -> io:format("CPU:" "~n Num Schedulers: ~w" "~n", [erlang:system_info(schedulers)]), - case erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - _ -> - 2 - end + Factor = case erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + _ -> + 2 + end, + {Factor, []} end. analyze_netbsd_cpu(Extract) -> @@ -1336,6 +1570,289 @@ analyze_netbsd_item(Extract, Key, Process, Default) -> +%% Model Identifier: Macmini7,1 +%% Processor Name: Intel Core i5 +%% Processor Speed: 2,6 GHz +%% Number of Processors: 1 +%% Total Number of Cores: 2 +%% L2 Cache (per Core): 256 KB +%% L3 Cache: 3 MB +%% Hyper-Threading Technology: Enabled +%% Memory: 16 GB + +analyze_and_print_darwin_host_info(Version) -> + %% This stuff is for macOS. + %% If we ever tested on a pure darwin machine, + %% we need to find some other way to find some info... + %% Also, I suppose its possible that we for some other + %% reason *fail* to get the info... + case analyze_darwin_software_info() of + [] -> + io:format("Darwin:" + "~n Version: ~s" + "~n Num Schedulers: ~s" + "~n", [Version, str_num_schedulers()]), + {num_schedulers_to_factor(), []}; + SwInfo when is_list(SwInfo) -> + SystemVersion = analyze_darwin_sw_system_version(SwInfo), + KernelVersion = analyze_darwin_sw_kernel_version(SwInfo), + HwInfo = analyze_darwin_hardware_info(), + ModelName = analyze_darwin_hw_model_name(HwInfo), + ModelId = analyze_darwin_hw_model_identifier(HwInfo), + ProcName = analyze_darwin_hw_processor_name(HwInfo), + ProcSpeed = analyze_darwin_hw_processor_speed(HwInfo), + NumProc = analyze_darwin_hw_number_of_processors(HwInfo), + NumCores = analyze_darwin_hw_total_number_of_cores(HwInfo), + Memory = analyze_darwin_hw_memory(HwInfo), + io:format("Darwin:" + "~n System Version: ~s" + "~n Kernel Version: ~s" + "~n Model: ~s (~s)" + "~n Processor: ~s (~s, ~s, ~s)" + "~n Memory: ~s" + "~n Num Schedulers: ~s" + "~n", [SystemVersion, KernelVersion, + ModelName, ModelId, + ProcName, ProcSpeed, NumProc, NumCores, + Memory, + str_num_schedulers()]), + CPUFactor = analyze_darwin_cpu_to_factor(ProcName, + ProcSpeed, + NumProc, + NumCores), + MemFactor = analyze_darwin_memory_to_factor(Memory), + if (MemFactor =:= 1) -> + {CPUFactor, []}; + true -> + {CPUFactor + MemFactor, []} + end + end. + +analyze_darwin_sw_system_version(SwInfo) -> + proplists:get_value("system version", SwInfo, "-"). + +analyze_darwin_sw_kernel_version(SwInfo) -> + proplists:get_value("kernel version", SwInfo, "-"). + +analyze_darwin_software_info() -> + analyze_darwin_system_profiler("SPSoftwareDataType"). + +analyze_darwin_hw_model_name(HwInfo) -> + proplists:get_value("model name", HwInfo, "-"). + +analyze_darwin_hw_model_identifier(HwInfo) -> + proplists:get_value("model identifier", HwInfo, "-"). + +analyze_darwin_hw_processor_name(HwInfo) -> + proplists:get_value("processor name", HwInfo, "-"). + +analyze_darwin_hw_processor_speed(HwInfo) -> + proplists:get_value("processor speed", HwInfo, "-"). + +analyze_darwin_hw_number_of_processors(HwInfo) -> + proplists:get_value("number of processors", HwInfo, "-"). + +analyze_darwin_hw_total_number_of_cores(HwInfo) -> + proplists:get_value("total number of cores", HwInfo, "-"). + +analyze_darwin_hw_memory(HwInfo) -> + proplists:get_value("memory", HwInfo, "-"). + +analyze_darwin_hardware_info() -> + analyze_darwin_system_profiler("SPHardwareDataType"). + +%% This basically has the structure: "Key: Value" +%% But could also be (for example): +%% "Something:" (which we ignore) +%% "Key: Value1:Value2" +analyze_darwin_system_profiler(DataType) -> + %% First, make sure the program actually exist: + case os:cmd("which system_profiler") of + [] -> + []; + _ -> + D0 = os:cmd("system_profiler " ++ DataType), + D1 = string:tokens(D0, [$\n]), + D2 = [string:trim(S1) || S1 <- D1], + D3 = [string:tokens(S2, [$:]) || S2 <- D2], + analyze_darwin_system_profiler2(D3) + end. + +analyze_darwin_system_profiler2(L) -> + analyze_darwin_system_profiler2(L, []). + +analyze_darwin_system_profiler2([], Acc) -> + [{string:to_lower(K), V} || {K, V} <- lists:reverse(Acc)]; +analyze_darwin_system_profiler2([[_]|T], Acc) -> + analyze_darwin_system_profiler2(T, Acc); +analyze_darwin_system_profiler2([[H1,H2]|T], Acc) -> + analyze_darwin_system_profiler2(T, [{H1, string:trim(H2)}|Acc]); +analyze_darwin_system_profiler2([[H|TH0]|T], Acc) -> + %% Some value parts has ':' in them, so put them together + TH1 = colonize(TH0), + analyze_darwin_system_profiler2(T, [{H, string:trim(TH1)}|Acc]). + +%% This is only called if the length is at least 2 +colonize([L1, L2]) -> + L1 ++ ":" ++ L2; +colonize([H|T]) -> + H ++ ":" ++ colonize(T). + + +%% The memory looks like this "<size> <unit>". Example: "2 GB" +analyze_darwin_memory_to_factor(Mem) -> + case [string:to_lower(S) || S <- string:tokens(Mem, [$\ ])] of + [_SzStr, "tb"] -> + 1; + [SzStr, "gb"] -> + try list_to_integer(SzStr) of + Sz when Sz < 2 -> + 20; + Sz when Sz < 4 -> + 10; + Sz when Sz < 8 -> + 5; + Sz when Sz < 16 -> + 2; + _ -> + 1 + catch + _:_:_ -> + 20 + end; + [_SzStr, "mb"] -> + 20; + _ -> + 20 + end. + + +%% The speed is a string: "<speed> <unit>" +%% the speed may be a float, which we transforms into an integer of MHz. +%% To calculate a factor based on processor speed, number of procs +%% and number of cores is ... not an exact ... science ... +analyze_darwin_cpu_to_factor(_ProcName, + ProcSpeedStr, NumProcStr, NumCoresStr) -> + Speed = + case [string:to_lower(S) || S <- string:tokens(ProcSpeedStr, [$\ ])] of + [SpeedStr, "mhz"] -> + try list_to_integer(SpeedStr) of + SpeedI -> + SpeedI + catch + _:_:_ -> + try list_to_float(SpeedStr) of + SpeedF -> + trunc(SpeedF) + catch + _:_:_ -> + -1 + end + end; + [SpeedStr, "ghz"] -> + try list_to_float(SpeedStr) of + SpeedF -> + trunc(1000*SpeedF) + catch + _:_:_ -> + try list_to_integer(SpeedStr) of + SpeedI -> + 1000*SpeedI + catch + _:_:_ -> + -1 + end + end; + _ -> + -1 + end, + NumProc = try list_to_integer(NumProcStr) of + NumProcI -> + NumProcI + catch + _:_:_ -> + 1 + end, + NumCores = try list_to_integer(NumCoresStr) of + NumCoresI -> + NumCoresI + catch + _:_:_ -> + 1 + end, + if + (Speed > 3000) -> + if + (NumProc =:= 1) -> + if + (NumCores < 2) -> + 5; + (NumCores < 4) -> + 3; + (NumCores < 6) -> + 2; + true -> + 1 + end; + true -> + if + (NumCores < 4) -> + 2; + true -> + 1 + end + end; + (Speed > 2000) -> + if + (NumProc =:= 1) -> + if + (NumCores < 2) -> + 8; + (NumCores < 4) -> + 5; + (NumCores < 6) -> + 3; + true -> + 1 + end; + true -> + if + (NumCores < 4) -> + 5; + (NumCores < 8) -> + 2; + true -> + 1 + end + end; + true -> + if + (NumProc =:= 1) -> + if + (NumCores < 2) -> + 10; + (NumCores < 4) -> + 7; + (NumCores < 6) -> + 5; + (NumCores < 8) -> + 3; + true -> + 1 + end; + true -> + if + (NumCores < 4) -> + 8; + (NumCores < 8) -> + 4; + true -> + 1 + end + end + end. + + analyze_and_print_solaris_host_info(Version) -> Release = case file:read_file_info("/etc/release") of @@ -1461,19 +1978,19 @@ analyze_and_print_solaris_host_info(Version) -> _:_:_ -> 10 end, - try erlang:system_info(schedulers) of - 1 -> - 10; - 2 -> - 5; - N when (N =< 6) -> - 2; - _ -> - 1 - catch - _:_:_ -> - 10 - end + MemFactor. + {try erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + N when (N =< 6) -> + 2; + _ -> + 1 + catch + _:_:_ -> + 10 + end + MemFactor, []}. @@ -1547,7 +2064,7 @@ analyze_and_print_win_host_info(Version) -> _ -> 2 end, - CPUFactor + MemFactor. + {CPUFactor + MemFactor, SysInfo}. win_sys_info_lookup(Key, SysInfo) -> win_sys_info_lookup(Key, SysInfo, "-"). @@ -1562,14 +2079,24 @@ win_sys_info_lookup(Key, SysInfo, Def) -> %% This function only extracts the prop we actually care about! which_win_system_info() -> - SysInfo = os:cmd("systeminfo"), - try process_win_system_info(string:tokens(SysInfo, [$\r, $\n]), []) - catch - _:_:_ -> - io:format("Failed process System info: " - "~s~n", [SysInfo]), - [] - end. + F = fun() -> + try + begin + SysInfo = os:cmd("systeminfo"), + process_win_system_info( + string:tokens(SysInfo, [$\r, $\n]), []) + end + catch + C:E:S -> + io:format("Failed get or process System info: " + " Error Class: ~p" + " Error: ~p" + " Stack: ~p" + "~n", [C, E, S]), + [] + end + end, + proxy_call(F, minutes(1), []). process_win_system_info([], Acc) -> Acc; @@ -1613,7 +2140,39 @@ str_num_schedulers() -> _:_:_ -> "-" end. - +num_schedulers_to_factor() -> + try erlang:system_info(schedulers) of + 1 -> + 10; + 2 -> + 5; + N when (N =< 6) -> + 2; + _ -> + 1 + catch + _:_:_ -> + 10 + end. + + +linux_info_lookup(Key, File) -> + try [string:trim(S) || S <- string:tokens(os:cmd("grep " ++ "\"" ++ Key ++ "\"" ++ " " ++ File), [$:,$\n])] of + Info -> + linux_info_lookup_collect(Key, Info, []) + catch + _:_:_ -> + "-" + end. + +linux_info_lookup_collect(_Key, [], Values) -> + lists:reverse(Values); +linux_info_lookup_collect(Key, [Key, Value|Rest], Values) -> + linux_info_lookup_collect(Key, Rest, [Value|Values]); +linux_info_lookup_collect(_, _, Values) -> + lists:reverse(Values). + + %% ---------------------------------------------------------------- %% Time related function @@ -1639,7 +2198,7 @@ sleep(MSecs) -> %% ---------------------------------------------------------------- %% Process utility function -%% +%% flush_mqueue() -> io:format("~p~n", [lists:reverse(flush_mqueue([]))]). @@ -1654,10 +2213,10 @@ flush_mqueue(MQ) -> trap_exit() -> - {trap_exit,Flag} = process_info(self(),trap_exit),Flag. + {trap_exit, Flag} = process_info(self(),trap_exit), Flag. trap_exit(Flag) -> - process_flag(trap_exit,Flag). + process_flag(trap_exit, Flag). diff --git a/lib/snmp/test/snmp_test_lib.hrl b/lib/snmp/test/snmp_test_lib.hrl index f4863c9a1e..a853d3cc09 100644 --- a/lib/snmp/test/snmp_test_lib.hrl +++ b/lib/snmp/test/snmp_test_lib.hrl @@ -52,10 +52,13 @@ -define(OS_BASED_SKIP(Skippable), ?LIB:os_based_skip(Skippable)). -define(NON_PC_TC_MAYBE_SKIP(Config, Condition), ?LIB:non_pc_tc_maybe_skip(Config, Condition, ?MODULE, ?LINE)). + -define(SKIP(Reason), ?LIB:skip(Reason, ?MODULE, ?LINE)). -define(FAIL(Reason), ?LIB:fail(Reason, ?MODULE, ?LINE)). -define(HAS_SUPPORT_IPV6(), ?LIB:has_support_ipv6()). +-define(PCALL(F, T, D), ?LIB:proxy_call(F, T, D)). + %% - Time macros - diff --git a/lib/snmp/test/snmp_test_manager.erl b/lib/snmp/test/snmp_test_manager.erl index 6ab5ce164c..fcbc0dff6b 100644 --- a/lib/snmp/test/snmp_test_manager.erl +++ b/lib/snmp/test/snmp_test_manager.erl @@ -193,22 +193,22 @@ handle_call(stop, _From, S) -> handle_call({sync_get, Oids}, _From, #state{agent_target_name = TargetName} = S) -> - Reply = (catch snmpm:sync_get(?USER, TargetName, Oids)), + Reply = (catch snmpm:sync_get2(?USER, TargetName, Oids)), {reply, Reply, S}; handle_call({sync_get_next, Oids}, _From, #state{agent_target_name = TargetName} = S) -> - Reply = (catch snmpm:sync_get_next(?USER, TargetName, Oids)), + Reply = (catch snmpm:sync_get_next2(?USER, TargetName, Oids)), {reply, Reply, S}; handle_call({sync_get_bulk, NR, MR, Oids}, _From, #state{agent_target_name = TargetName} = S) -> - Reply = (catch snmpm:sync_get_bulk(?USER, TargetName, NR, MR, Oids)), + Reply = (catch snmpm:sync_get_bulk2(?USER, TargetName, NR, MR, Oids)), {reply, Reply, S}; handle_call({sync_set, VarsAndVals}, _From, #state{agent_target_name = TargetName} = S) -> - Reply = (catch snmpm:sync_set(?USER, TargetName, VarsAndVals)), + Reply = (catch snmpm:sync_set2(?USER, TargetName, VarsAndVals)), {reply, Reply, S}; handle_call(Req, From, State) -> diff --git a/lib/snmp/test/snmp_to_snmpnet_SUITE.erl b/lib/snmp/test/snmp_to_snmpnet_SUITE.erl index b5c08fb697..aacdcff504 100644 --- a/lib/snmp/test/snmp_to_snmpnet_SUITE.erl +++ b/lib/snmp/test/snmp_to_snmpnet_SUITE.erl @@ -435,9 +435,9 @@ erlang_manager_netsnmp_get(Config) when is_list(Config) -> {version, v2}, {sec_model, v2c}, {sec_level, noAuthNoPriv}]) || {Domain, Addr} <- Transports], Results = - [snmp_manager_user:sync_get( + [snmp_manager_user:sync_get2( TargetName++domain_suffix(Domain), - [?sysDescr_instance]) + [?sysDescr_instance], []) || {Domain, _} <- Transports], ct:pal("sync_get -> ~p", [Results]), snmp_manager_user:stop(), diff --git a/lib/snmp/vsn.mk b/lib/snmp/vsn.mk index 84299ec250..39da86b2e1 100644 --- a/lib/snmp/vsn.mk +++ b/lib/snmp/vsn.mk @@ -19,6 +19,6 @@ # %CopyrightEnd% APPLICATION = snmp -SNMP_VSN = 5.5 +SNMP_VSN = 5.6 PRE_VSN = APP_VSN = "$(APPLICATION)-$(SNMP_VSN)$(PRE_VSN)" diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml index 4492b937a2..1cc25c00b3 100644 --- a/lib/ssh/doc/src/notes.xml +++ b/lib/ssh/doc/src/notes.xml @@ -30,6 +30,247 @@ <file>notes.xml</file> </header> +<section><title>Ssh 4.10</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix error in ssh_sftpd typespec.</p> + <p> + Own Id: OTP-16363</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The plug-in file ssh_file.erl, that is responsible for + default file handling, is re-factored, optimized and + re-written.</p> + <p> + Own Id: OTP-11688 Aux Id: OTP-12699 </p> + </item> + <item> + <p> + OpenSSH 6.5 introduced a new file representation of keys + called <url + href="https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL.key?annotate=1.1">openssh-key-v1</url>.</p> + <p> + OTP/SSH had an experimental implementation of this + format. That implementation is now improved and supported + with the exception of handling encrypted keys.</p> + <p> + Own Id: OTP-15434</p> + </item> + <item> + <p> + TCP/IP port forwarding, a.k.a tunneling a.k.a + tcp-forward/direct-tcp is implemented. In the OpenSSH + client, this corresponds to the options -L and -R.</p> + <p> + The client or server listens to a specified socket, and + when something connects to it with TCP/IP, that + connection is forwarded in an encrypted tunnel to the + peer. The peer then connects to a predefined IP/port pair + and then acts as a proxy.</p> + <p> + See the manual, <seemfa + marker="ssh:ssh#tcpip_tunnel_to_server/6"><c>ssh:tcpip_tunnel_to_server/6</c></seemfa> + and <seemfa + marker="ssh:ssh#tcpip_tunnel_from_server/6"><c>ssh:tcpip_tunnel_from_server/6</c></seemfa>.</p> + <p> + The functionality is disabled per default but can be + enabled when starting a daemon.</p> + <p> + Own Id: OTP-15998 Aux Id: PR-2376, PR-2368 </p> + </item> + <item> + <p> + The client-side of the supervisor tree (under sshc_sup) + was previously not complete; the channel handling + processes were handled with links but had no supervisors.</p> + <p> + This is now corrected with a client-side supervisor tree + under <c>sshc_sup</c>, similar to the server-side + supervisor tree under <c>sshd_sup</c>.</p> + <p> + Own Id: OTP-16026 Aux Id: PR-2368, (OTP-15998) </p> + </item> + <item> + <p> + The extension <url + href="https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL?annotate=HEAD">posix-rename@openssh.com</url> + is added to the <seemfa + marker="ssh:ssh_sftp#rename/3">ssh/sftp rename</seemfa> + operation.</p> + <p> + Own Id: OTP-16289 Aux Id: PR-2448 </p> + </item> + <item> + <p> + Calls of deprecated functions in the <seeguide + marker="crypto:new_api#the-old-api">Old Crypto + API</seeguide> are replaced by calls of their <seeguide + marker="crypto:new_api#the-new-api">substitutions</seeguide>.</p> + <p> + Own Id: OTP-16346</p> + </item> + <item> + <p> + The default known_hosts file handling is improved to + include ports.</p> + <p> + The handling of the contents in that file is updated to + support the <url + href="https://man.openbsd.org/sshd#SSH_KNOWN_HOSTS_FILE_FORMAT">full + syntax</url>, with exception of 1) the wildcard '?', 2) + wildcards in canonical names and 3) the option + '@cert-authority'</p> + <p> + Own Id: OTP-16506</p> + </item> + <item> + <p> + The MAC (Message Authorization Code) algorithms</p> + <list> <item>hmac-sha1-etm@openssh.com</item> + <item>hmac-sha2-256-etm@openssh.com</item> + <item>hmac-sha2-512-etm@openssh.com</item> </list> <p>are + implemented.</p> + <p> + Own Id: OTP-16508</p> + </item> + <item> + <p> + The key-exchange algorithms + <c>'diffie-hellman-group14-sha1'</c> and + <c>'diffie-hellman-group-exchange-sha1'</c> are disabled + per default. The reason is that SHA1 now is considered + insecure.</p> + <p> + They can be enabled if needed, see <seeapp + marker="ssh:SSH_app#algorithms">SSH (App)</seeapp>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16509</p> + </item> + <item> + <p> + The public key algorithm <c>'ssh-dss'</c> is disabled per + default. The reason is that it is now considered as + insecure.</p> + <p> + It can be enabled if needed, see <seeapp + marker="ssh:SSH_app#algorithms">SSH (App)</seeapp>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16510</p> + </item> + <item> + <p> + The public key <c>'ssh-rsa'</c> is now considered as + insecure because of its usage of SHA1.</p> + <p> + It is therefore deprecated and will no longer be enabled + per default in OTP-24.0.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16511</p> + </item> + <item> + <p> + An option <seetype + marker="ssh:ssh_file#optimize_key_lookup">optimize + (optimize_key_lookup)</seetype> is introduced for the + file interface ssh_file.erl</p> + <p> + The option enables the user to select between the default + handling which is fast but memory consuming vs memory + efficient but not as fast. The effect might be observable + only for large files.</p> + <p> + See the manual for <seemfa + marker="ssh:ssh_file#is_host_key/5">ssh_file:is_host_key/5</seemfa> + and <seemfa + marker="ssh:ssh_file#is_auth_key/3">ssh_file:is_auth_key/3</seemfa>.</p> + <p> + Own Id: OTP-16512</p> + </item> + <item> + <p> + The ssh agent is now implemented in the ssh_agent key + callback module. </p> + <p> + Enable with the the option <c> {key_cb, {ssh_agent, + []}}</c> in for example ssh:connect/3.</p> + <p> + See the <seeerl marker="ssh:ssh_agent">ssh_agent + manual</seeerl> for details.</p> + <p> + Own Id: OTP-16513</p> + </item> + <item> + <p> + Algorithm configuration could now be done in a .config + file.</p> + <p> + This is useful for example to enable an algorithm that is + disabled by default. It could now be enabled in an + .config-file without changing the code,</p> + <p> + See the SSH User's Guide chapter <seeguide + marker="ssh:configurations">"Configuration in + SSH"</seeguide>.</p> + <p> + Own Id: OTP-16540</p> + </item> + <item> + <p> + Documented which gen_tcp socket options can't be used in + calls to ssh:connect and ssh:daemon.</p> + <p> + Own Id: OTP-16589</p> + </item> + <item> + <p> + Added <seetype + marker="ssh:ssh#kb_int_fun_4">kb_int_fun_4()</seetype> to + the <seetype + marker="ssh:ssh#authentication_daemon_options">authentication_daemon_options()</seetype> + to enable generating dynamic keyboard-interactive prompts + from the user's state returned from the authentication + fun <seetype + marker="ssh:ssh#pwdfun_4">pwdfun_4()</seetype>.</p> + <p> + Own Id: OTP-16622 Aux Id: PR-2604 </p> + </item> + </list> + </section> + +</section> + +<section><title>Ssh 4.9.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Potential hazard between re-keying decision and socket + close.</p> + <p> + Own Id: OTP-16462 Aux Id: ERIERL-464 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.9</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -303,6 +544,22 @@ </section> +<section><title>Ssh 4.7.6.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Potential hazard between re-keying decision and socket + close.</p> + <p> + Own Id: OTP-16462 Aux Id: ERIERL-464 </p> + </item> + </list> + </section> + +</section> + <section><title>Ssh 4.7.6.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssh/doc/src/ssh.xml b/lib/ssh/doc/src/ssh.xml index ca6a02117b..f7818b809a 100644 --- a/lib/ssh/doc/src/ssh.xml +++ b/lib/ssh/doc/src/ssh.xml @@ -179,6 +179,17 @@ <p>Options for <seemfa marker="#connect/3">clients</seemfa>. The individual options are further explained below or by following the hyperlinks. </p> + <p>Note that not every + <seetype marker="kernel:gen_tcp#connect_option">gen_tcp:connect_option()</seetype> + is accepted. See + <seemfa marker="ssh#set_sock_opts/2">set_sock_opts/2</seemfa> + for a list of prohibited options. + </p> + <p>Also note that setting a + <seetype marker="kernel:gen_tcp#connect_option">gen_tcp:connect_option()</seetype> + could change the socket in a way that impacts the ssh client's behaviour + negatively. You use it on your own risk. + </p> </desc> </datatype> @@ -341,6 +352,17 @@ <p>Options for <seemfa marker="#daemon/1">daemons</seemfa>. The individual options are further explained below or by following the hyperlinks. </p> + <p>Note that not every + <seetype marker="kernel:gen_tcp#listen_option">gen_tcp:listen_option()</seetype> + is accepted. See + <seemfa marker="ssh#set_sock_opts/2">set_sock_opts/2</seemfa> + for a list of prohibited options. + </p> + <p>Also note that setting a + <seetype marker="kernel:gen_tcp#listen_option">gen_tcp:listen_option()</seetype> + could change the socket in a way that impacts the ssh deamon's behaviour + negatively. You use it on your own risk. + </p> </desc> </datatype> @@ -514,6 +536,7 @@ <name name="prompt_texts"/> <name name="kb_int_tuple"/> <name name="kb_int_fun_3"/> + <name name="kb_int_fun_4"/> <name name="pwdfun_2"/> <name name="pwdfun_4"/> <desc> @@ -522,7 +545,7 @@ <item> <p>Sets the text strings that the daemon sends to the client for presentation to the user when using <c>keyboard-interactive</c> authentication.</p> - <p>If the fun/3 is used, it is called when the actual authentication occurs and may therefore + <p>If the fun/3 or fun/4 is used, it is called when the actual authentication occurs and may therefore return dynamic data like time, remote ip etc.</p> <p>The parameter <c>Echo</c> guides the client about need to hide the password.</p> <p>The default value is: @@ -874,7 +897,7 @@ <datatype> <name name="disconnectfun_common_option"/> <desc> - <p>Provides a fun to implement your own logging when the peer disconnects.</p> + <p>Provides a fun to implement your own logging or other handling at disconnects.</p> </desc> </datatype> @@ -1221,8 +1244,17 @@ <p>This function calls the <seemfa marker="kernel:inet#setopts/2">inet:setopts/2</seemfa>, read that documentation and for <seetype marker="kernel:gen_tcp#option">gen_tcp:option()</seetype>. - All gen_tcp socket options except <c>active</c>, <c>deliver</c>, <c>mode</c> and <c>packet</c> - are allowed. The excluded options are reserved by the SSH application. + </p> + <p> + All gen_tcp socket options except + </p> + <list> + <item><c>active</c></item> + <item><c>deliver</c></item> + <item><c>mode</c> and</item> + <item><c>packet</c></item> + </list> + <p>are allowed. The excluded options are reserved by the SSH application. </p> <warning> <p>This is an extremly dangerous function. You use it on your own risk.</p> diff --git a/lib/ssh/src/ssh.hrl b/lib/ssh/src/ssh.hrl index ffa2ec8c06..ad90a1a841 100644 --- a/lib/ssh/src/ssh.hrl +++ b/lib/ssh/src/ssh.hrl @@ -361,9 +361,11 @@ -type prompt_texts() :: kb_int_tuple() | kb_int_fun_3() + | kb_int_fun_4() . -type kb_int_fun_3() :: fun((Peer::ip_port(), User::string(), Service::string()) -> kb_int_tuple()). +-type kb_int_fun_4() :: fun((Peer::ip_port(), User::string(), Service::string(), State::any()) -> kb_int_tuple()). -type kb_int_tuple() :: {Name::string(), Instruction::string(), Prompt::string(), Echo::boolean()}. -type pwdfun_2() :: fun((User::string(), Password::string()) -> boolean()) . diff --git a/lib/ssh/src/ssh_acceptor.erl b/lib/ssh/src/ssh_acceptor.erl index 15e59dd1fe..960058fa38 100644 --- a/lib/ssh/src/ssh_acceptor.erl +++ b/lib/ssh/src/ssh_acceptor.erl @@ -217,6 +217,11 @@ ssh_dbg_format(connections, {call, {?MODULE,acceptor_init, [_Parent, Port, Address, _Opts, _AcceptTimeout]}}) -> [io_lib:format("Starting LISTENER on ~s:~p\n", [ntoa(Address),Port]) ]; +ssh_dbg_format(connections, {return_from, {?MODULE,acceptor_init,5}, _Ret}) -> + skip; + +ssh_dbg_format(connections, {call, {?MODULE,handle_connection,[_,_,_,_,_]}}) -> + skip; ssh_dbg_format(connections, {return_from, {?MODULE,handle_connection,5}, {error,Error}}) -> ["Starting connection to server failed:\n", io_lib:format("Error = ~p", [Error]) diff --git a/lib/ssh/src/ssh_auth.erl b/lib/ssh/src/ssh_auth.erl index a42f034f1b..19df20c9f1 100644 --- a/lib/ssh/src/ssh_auth.erl +++ b/lib/ssh/src/ssh_auth.erl @@ -32,10 +32,13 @@ -export([get_public_key/2, publickey_msg/1, password_msg/1, keyboard_interactive_msg/1, service_request_msg/1, init_userauth_request_msg/1, - userauth_request_msg/1, handle_userauth_request/3, + userauth_request_msg/1, handle_userauth_request/3, ssh_msg_userauth_result/1, handle_userauth_info_request/2, handle_userauth_info_response/2 ]). +-behaviour(ssh_dbg). +-export([ssh_dbg_trace_points/0, ssh_dbg_flags/1, ssh_dbg_on/1, ssh_dbg_off/1, ssh_dbg_format/3]). + %%-------------------------------------------------------------------- %%% Internal application API %%-------------------------------------------------------------------- @@ -110,14 +113,13 @@ password_msg([#ssh{opts = Opts, not_ok -> {not_ok, Ssh}; _ -> - ssh_transport:ssh_packet( - #ssh_msg_userauth_request{user = User, - service = Service, - method = "password", - data = - <<?BOOLEAN(?FALSE), - ?STRING(unicode:characters_to_binary(Password))>>}, - Ssh) + {#ssh_msg_userauth_request{user = User, + service = Service, + method = "password", + data = + <<?BOOLEAN(?FALSE), + ?STRING(unicode:characters_to_binary(Password))>>}, + Ssh} end. %% See RFC 4256 for info on keyboard-interactive @@ -128,13 +130,12 @@ keyboard_interactive_msg([#ssh{user = User, not_ok -> {not_ok,Ssh}; % No need to use a failed pwd once more _ -> - ssh_transport:ssh_packet( - #ssh_msg_userauth_request{user = User, - service = Service, - method = "keyboard-interactive", - data = << ?STRING(<<"">>), - ?STRING(<<>>) >> }, - Ssh) + {#ssh_msg_userauth_request{user = User, + service = Service, + method = "keyboard-interactive", + data = << ?STRING(<<"">>), + ?STRING(<<>>) >> }, + Ssh} end. @@ -183,15 +184,14 @@ publickey_msg([SigAlg, #ssh{user = User, SigBlob = list_to_binary([?string(SigAlgStr), ?binary(Sig)]), - ssh_transport:ssh_packet( - #ssh_msg_userauth_request{user = User, - service = Service, - method = "publickey", - data = [?TRUE, - ?string(SigAlgStr), - ?binary(PubKeyBlob), - ?binary(SigBlob)]}, - Ssh); + {#ssh_msg_userauth_request{user = User, + service = Service, + method = "publickey", + data = [?TRUE, + ?string(SigAlgStr), + ?binary(PubKeyBlob), + ?binary(SigBlob)]}, + Ssh}; _ -> {not_ok, Ssh} @@ -199,8 +199,8 @@ publickey_msg([SigAlg, #ssh{user = User, %%%---------------------------------------------------------------- service_request_msg(Ssh) -> - ssh_transport:ssh_packet(#ssh_msg_service_request{name = "ssh-userauth"}, - Ssh#ssh{service = "ssh-userauth"}). + {#ssh_msg_service_request{name = "ssh-userauth"}, + Ssh#ssh{service = "ssh-userauth"}}. %%%---------------------------------------------------------------- init_userauth_request_msg(#ssh{opts = Opts} = Ssh) -> @@ -210,24 +210,23 @@ init_userauth_request_msg(#ssh{opts = Opts} = Ssh) -> ?DISCONNECT(?SSH_DISCONNECT_ILLEGAL_USER_NAME, "Could not determine the users name"); User -> - ssh_transport:ssh_packet( - #ssh_msg_userauth_request{user = User, - service = "ssh-connection", - method = "none", - data = <<>>}, - Ssh#ssh{user = User, - userauth_preference = method_preference(Ssh#ssh.userauth_pubkeys), - userauth_methods = none, - service = "ssh-connection"} - ) + {#ssh_msg_userauth_request{user = User, + service = "ssh-connection", + method = "none", + data = <<>>}, + Ssh#ssh{user = User, + userauth_preference = method_preference(Ssh#ssh.userauth_pubkeys), + userauth_methods = none, + service = "ssh-connection"} + } end. %%%---------------------------------------------------------------- %%% called by server handle_userauth_request(#ssh_msg_service_request{name = Name = "ssh-userauth"}, _, Ssh) -> - {ok, ssh_transport:ssh_packet(#ssh_msg_service_accept{name = Name}, - Ssh#ssh{service = "ssh-connection"})}; + {ok, {#ssh_msg_service_accept{name = Name}, + Ssh#ssh{service = "ssh-connection"}}}; handle_userauth_request(#ssh_msg_userauth_request{user = User, service = "ssh-connection", @@ -239,12 +238,13 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, case check_password(User, Password, Opts, Ssh) of {true,Ssh1} -> {authorized, User, - ssh_transport:ssh_packet(#ssh_msg_userauth_success{}, Ssh1)}; + {#ssh_msg_userauth_success{}, Ssh1} + }; {false,Ssh1} -> {not_authorized, {User, {error,"Bad user or password"}}, - ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ - authentications = Methods, - partial_success = false}, Ssh1)} + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh1} + } end; handle_userauth_request(#ssh_msg_userauth_request{user = User, @@ -264,18 +264,18 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, %% or the old password was bad. {not_authorized, {User, {error,"Password change not supported"}}, - ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ - authentications = Methods, - partial_success = false}, Ssh)}; + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh} + }; handle_userauth_request(#ssh_msg_userauth_request{user = User, service = "ssh-connection", method = "none"}, _, #ssh{userauth_supported_methods = Methods} = Ssh) -> {not_authorized, {User, undefined}, - ssh_transport:ssh_packet( - #ssh_msg_userauth_failure{authentications = Methods, - partial_success = false}, Ssh)}; + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh} + }; handle_userauth_request(#ssh_msg_userauth_request{user = User, service = "ssh-connection", @@ -293,14 +293,14 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, case pre_verify_sig(User, KeyBlob, Opts) of true -> {not_authorized, {User, undefined}, - ssh_transport:ssh_packet( - #ssh_msg_userauth_pk_ok{algorithm_name = binary_to_list(BAlg), - key_blob = KeyBlob}, Ssh)}; + {#ssh_msg_userauth_pk_ok{algorithm_name = binary_to_list(BAlg), + key_blob = KeyBlob}, Ssh} + }; false -> {not_authorized, {User, undefined}, - ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ - authentications = Methods, - partial_success = false}, Ssh)} + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh} + } end; handle_userauth_request(#ssh_msg_userauth_request{user = User, @@ -318,13 +318,13 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, BAlg, KeyBlob, SigWLen, Ssh) of true -> {authorized, User, - ssh_transport:ssh_packet( - #ssh_msg_userauth_success{}, Ssh)}; + {#ssh_msg_userauth_success{}, Ssh} + }; false -> {not_authorized, {User, undefined}, - ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ - authentications = Methods, - partial_success = false}, Ssh)} + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh} + } end; handle_userauth_request(#ssh_msg_userauth_request{user = User, @@ -337,9 +337,9 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, case KbTriesLeft of N when N<1 -> {not_authorized, {User, {authmethod, "keyboard-interactive"}}, - ssh_transport:ssh_packet( - #ssh_msg_userauth_failure{authentications = Methods, - partial_success = false}, Ssh)}; + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh} + }; _ -> %% RFC4256 @@ -364,6 +364,9 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, Default; {_,_,_,_}=V -> V; + F when is_function(F, 4) -> + {_,PeerName} = Ssh#ssh.peer, + F(PeerName, User, "ssh-connection", Ssh#ssh.pwdfun_user_state); F when is_function(F) -> {_,PeerName} = Ssh#ssh.peer, F(PeerName, User, "ssh-connection") @@ -381,8 +384,8 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, >> }, {not_authorized, {User, undefined}, - ssh_transport:ssh_packet(Msg, Ssh#ssh{user = User - })} + {Msg, Ssh#ssh{user = User}} + } end; handle_userauth_request(#ssh_msg_userauth_request{user = User, @@ -390,9 +393,9 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User, method = Other}, _, #ssh{userauth_supported_methods = Methods} = Ssh) -> {not_authorized, {User, {authmethod, Other}}, - ssh_transport:ssh_packet( - #ssh_msg_userauth_failure{authentications = Methods, - partial_success = false}, Ssh)}. + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, Ssh} + }. %%%---------------------------------------------------------------- @@ -408,9 +411,9 @@ handle_userauth_info_request(#ssh_msg_userauth_info_request{name = Name, not_ok; Responses -> {ok, - ssh_transport:ssh_packet( - #ssh_msg_userauth_info_response{num_responses = NumPrompts, - data = Responses}, Ssh)} + {#ssh_msg_userauth_info_response{num_responses = NumPrompts, + data = Responses}, + Ssh}} end. %%%---------------------------------------------------------------- @@ -428,32 +431,30 @@ handle_userauth_info_response(#ssh_msg_userauth_info_response{num_responses = 1, case check_password(User, unicode:characters_to_list(Password), Opts, Ssh) of {true,Ssh1} when SendOneEmpty==true -> - Msg = #ssh_msg_userauth_info_request{name = "", - instruction = "", - language_tag = "", - num_prompts = 0, - data = <<?BOOLEAN(?FALSE)>> - }, {authorized_but_one_more, User, - ssh_transport:ssh_packet(Msg, Ssh1)}; + {#ssh_msg_userauth_info_request{name = "", + instruction = "", + language_tag = "", + num_prompts = 0, + data = <<?BOOLEAN(?FALSE)>> + }, + Ssh1}}; {true,Ssh1} -> {authorized, User, - ssh_transport:ssh_packet(#ssh_msg_userauth_success{}, Ssh1)}; + {#ssh_msg_userauth_success{}, Ssh1}}; {false,Ssh1} -> {not_authorized, {User, {error,"Bad user or password"}}, - ssh_transport:ssh_packet(#ssh_msg_userauth_failure{ - authentications = Methods, - partial_success = false}, - Ssh1#ssh{kb_tries_left = max(KbTriesLeft-1, 0)} - )} + {#ssh_msg_userauth_failure{authentications = Methods, + partial_success = false}, + Ssh1#ssh{kb_tries_left = max(KbTriesLeft-1, 0)}}} end; handle_userauth_info_response({extra,#ssh_msg_userauth_info_response{}}, #ssh{user = User} = Ssh) -> {authorized, User, - ssh_transport:ssh_packet(#ssh_msg_userauth_success{}, Ssh)}; + {#ssh_msg_userauth_success{}, Ssh}}; handle_userauth_info_response(#ssh_msg_userauth_info_response{}, _Auth) -> @@ -619,3 +620,193 @@ write_if_nonempty(_, "") -> ok; write_if_nonempty(_, <<>>) -> ok; write_if_nonempty(IoCb, Text) -> IoCb:format("~s~n",[Text]). +%%%---------------------------------------------------------------- +%%% Called just for the tracer ssh_dbg +ssh_msg_userauth_result(_R) -> ok. + +%%%################################################################ +%%%# +%%%# Tracing +%%%# + +ssh_dbg_trace_points() -> [authentication]. + +ssh_dbg_flags(authentication) -> [c]. + +ssh_dbg_on(authentication) -> dbg:tp(?MODULE, handle_userauth_request, 3, x), + dbg:tp(?MODULE, init_userauth_request_msg, 1, x), + dbg:tp(?MODULE, ssh_msg_userauth_result, 1, x), + dbg:tp(?MODULE, userauth_request_msg, 1, x). + +ssh_dbg_off(authentication) -> dbg:ctpg(?MODULE, handle_userauth_request, 3), + dbg:ctpg(?MODULE, init_userauth_request_msg, 1), + dbg:ctpg(?MODULE, ssh_msg_userauth_result, 1), + dbg:ctpg(?MODULE, userauth_request_msg, 1). + + + +%%% Server ---------------- +ssh_dbg_format(authentication, {call, {?MODULE,handle_userauth_request, [Req,_SessionID,Ssh]}}, + Stack) -> + {skip, [{Req,Ssh}|Stack]}; + + +ssh_dbg_format(authentication, {return_from, {?MODULE,handle_userauth_request,3}, + {ok,{#ssh_msg_service_accept{name=Name},_Ssh}}}, + [{#ssh_msg_service_request{name=Name},_} | Stack]) -> + {skip, Stack}; + +ssh_dbg_format(authentication, {return_from, {?MODULE,handle_userauth_request,3}, + {authorized,User,_Repl}}, + [{#ssh_msg_userauth_request{}=Req,Ssh}|Stack]) -> + {["AUTH srvr: Peer client authorized\n", + io_lib:format("user = ~p~n", [User]), + fmt_req(Req, Ssh)], + Stack}; + +ssh_dbg_format(authentication, {return_from, {?MODULE,handle_userauth_request,3}, + {not_authorized,{User,_X},_Repl}}, + [{#ssh_msg_userauth_request{method="none"},Ssh}|Stack]) -> + Methods = Ssh#ssh.userauth_supported_methods, + {["AUTH srvr: Peer queries auth methods\n", + io_lib:format("user = ~p~nsupported methods = ~p ?", [User,Methods]) + ], + Stack}; + +ssh_dbg_format(authentication, {return_from, {?MODULE,handle_userauth_request,3}, + {not_authorized,{User,_X}, Repl} + }, + [{#ssh_msg_userauth_request{method = "publickey", + data = <<?BYTE(?FALSE), _/binary>> + }=Req,Ssh}|Stack]) -> + {case Repl of + {#ssh_msg_userauth_pk_ok{}, _} -> + ["AUTH srvr: Answer - pub key supported\n"]; + {#ssh_msg_userauth_failure{}, _} -> + ["AUTH srvr: Answer - pub key not supported\n"]; + {Other, _} -> + ["AUTH srvr: Answer - strange answer\n", + io_lib:format("strange answer = ~p~n",[Other]) + ] + end + ++ [io_lib:format("user = ~p~n", [User]), + fmt_req(Req, Ssh)], + Stack}; + + +ssh_dbg_format(authentication, {call, {?MODULE,ssh_msg_userauth_result,[success]}}, + Stack) -> + {["AUTH client: Success"],Stack}; +ssh_dbg_format(authentication, {return_from, {?MODULE,ssh_msg_userauth_result,1}, _Result}, + Stack) -> + {skip, Stack}; + +ssh_dbg_format(authentication, {return_from, {?MODULE,handle_userauth_request,3}, + {not_authorized,{User,_X},_Repl}}, + [{#ssh_msg_userauth_request{}=Req,Ssh}|Stack]) -> + {["AUTH srvr: Peer client authorization failed\n", + io_lib:format("user = ~p~n", [User]), + fmt_req(Req, Ssh)], + Stack}; + +%%% Client ---------------- +ssh_dbg_format(authentication, {call, {?MODULE,init_userauth_request_msg, [#ssh{opts = Opts}]}}, + Stack) -> + {["AUTH client: Service ssh-userauth accepted\n", + case ?GET_OPT(user, Opts) of + undefined -> + io_lib:format("user = undefined *** ERROR ***", []); + User -> + io_lib:format("user = ~p", [User]) + end + ], + Stack}; +ssh_dbg_format(authentication, {return_from, {?MODULE,init_userauth_request_msg,1}, + {Repl = #ssh_msg_userauth_request{user = User, + service = "ssh-connection", + method = "none"}, + _Ssh}}, + Stack) -> + {["AUTH client: Query for accepted methods\n", + io_lib:format("user = ~p", [User])], + [Repl|Stack]}; + +ssh_dbg_format(authentication, {call, {?MODULE,userauth_request_msg, + [#ssh{userauth_methods = Methods}]}}, + [ #ssh_msg_userauth_request{user = User, + service = "ssh-connection", + method = "none"} | Stack]) -> + {["AUTH client: Server supports\n", + io_lib:format("user = ~p~nmethods = ~p", [User,Methods])], + Stack}; + +ssh_dbg_format(authentication, {call, {?MODULE,userauth_request_msg,[_Ssh]}}, + Stack) -> + {skip,Stack}; + +ssh_dbg_format(authentication, {return_from, {?MODULE,userauth_request_msg,1}, + {send_disconnect, _Code, _Ssh}}, + Stack) -> + {skip,Stack}; +ssh_dbg_format(authentication, {return_from, {?MODULE,userauth_request_msg,1}, + {Method,{_Msg,_Ssh}}}, + Stack) -> + {["AUTH client: Try auth with\n", + io_lib:format("method = ~p", [Method])], + Stack}; + + + +ssh_dbg_format(authentication, Unhandled, Stack) -> + case Unhandled of + {call, {?MODULE,_F,_Args}} -> ok; + {return_from, {?MODULE,_F,_A}, _Resp} -> ok + end, + {["UNHANDLED AUTH FORMAT\n", + io_lib:format("Unhandled = ~p~nStack = ~p", [Unhandled,Stack])], + Stack}. + + +%%% Dbg helpers ---------------- + + +fmt_req(#ssh_msg_userauth_request{user = User, + service = "ssh-connection", + method = Method, + data = Data}, + #ssh{kb_tries_left = KbTriesLeft, + userauth_supported_methods = Methods}) -> + [io_lib:format("req user = ~p~n" + "req method = ~p~n" + "supported methods = ~p", + [User,Method,Methods]), + case Method of + "none" -> ""; + "password" -> fmt_bool(Data); + "keyboard-interactive" -> fmt_kb_tries_left(KbTriesLeft); + "publickey" -> [case Data of + <<?BYTE(_), ?UINT32(ALen), Alg:ALen/binary, _/binary>> -> + io_lib:format("~nkey-type = ~p", [Alg]); + _ -> + "" + end]; + _ -> "" + end]. + + +fmt_kb_tries_left(N) when is_integer(N)-> + io_lib:format("~ntries left = ~p", [N-1]). + + +fmt_bool(<<?BYTE(Bool),_/binary>>) -> + io_lib:format("~nBool = ~s", + [case Bool of + ?TRUE -> "true"; + ?FALSE -> "false"; + _ -> io_lib:format("? (~p)",[Bool]) + end]); +fmt_bool(<<>>) -> + "". + + + diff --git a/lib/ssh/src/ssh_cli.erl b/lib/ssh/src/ssh_cli.erl index d6c8cf84ed..729ce67968 100644 --- a/lib/ssh/src/ssh_cli.erl +++ b/lib/ssh/src/ssh_cli.erl @@ -663,6 +663,9 @@ ssh_dbg_off(terminate) -> dbg:ctpg(?MODULE, terminate, 2). ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, State]}}) -> ["Cli Terminating:\n", io_lib:format("Reason: ~p,~nState:~n~s", [Reason, wr_record(State)]) - ]. + ]; +ssh_dbg_format(terminate, {return_from, {?MODULE,terminate,2}, _Ret}) -> + skip. + ?wr_record(state). diff --git a/lib/ssh/src/ssh_client_channel.erl b/lib/ssh/src/ssh_client_channel.erl index 6de897e1b2..5ffdf49e9b 100644 --- a/lib/ssh/src/ssh_client_channel.erl +++ b/lib/ssh/src/ssh_client_channel.erl @@ -419,12 +419,16 @@ ssh_dbg_format(channels, {return_from, {?MODULE,init,1}, {stop,Reason}}) -> ["Server Channel Start FAILED!\n", io_lib:format("Reason = ~p", [Reason]) ]; + ssh_dbg_format(channels, F) -> ssh_dbg_format(terminate, F); + ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, State]}}) -> ["Server Channel Terminating:\n", io_lib:format("Reason: ~p,~nState:~n~s", [Reason, wr_record(State)]) ]; +ssh_dbg_format(terminate, {return_from, {?MODULE,terminate,2}, _Ret}) -> + skip; ssh_dbg_format(channel_events, {call, {?MODULE,handle_call, [Call,From,State]}}) -> [hdr("is called", State), @@ -434,6 +438,7 @@ ssh_dbg_format(channel_events, {return_from, {?MODULE,handle_call,3}, Ret}) -> ["Server Channel call returned:\n", io_lib:format("~p~n", [ssh_dbg:reduce_state(Ret)]) ]; + ssh_dbg_format(channel_events, {call, {?MODULE,handle_cast, [Cast,State]}}) -> [hdr("got cast", State), io_lib:format("Cast: ~p~n", [Cast]) @@ -442,6 +447,7 @@ ssh_dbg_format(channel_events, {return_from, {?MODULE,handle_cast,2}, Ret}) -> ["Server Channel cast returned:\n", io_lib:format("~p~n", [ssh_dbg:reduce_state(Ret)]) ]; + ssh_dbg_format(channel_events, {call, {?MODULE,handle_info, [Info,State]}}) -> [hdr("got info", State), io_lib:format("Info: ~p~n", [Info]) diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl index 3ad7c182d5..5e5ed9b79a 100644 --- a/lib/ssh/src/ssh_connection_handler.erl +++ b/lib/ssh/src/ssh_connection_handler.erl @@ -63,7 +63,8 @@ adjust_window/3, close/2, disconnect/4, get_print_info/1, - set_sock_opts/2, get_sock_opts/2 + set_sock_opts/2, get_sock_opts/2, + prohibited_sock_option/1 ]). -type connection_ref() :: ssh:connection_ref(). @@ -365,11 +366,11 @@ retrieve(ConnectionHandler, Key) -> %% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . set_sock_opts(ConnectionRef, SocketOptions) -> try lists:foldr(fun({Name,_Val}, Acc) -> - case lists:member(Name, [active, deliver, mode, packet]) of + case prohibited_sock_option(Name) of true -> [Name|Acc]; false -> Acc end - end, [], SocketOptions) + end, [], SocketOptions) of [] -> call(ConnectionRef, {set_sock_opts,SocketOptions}); @@ -380,6 +381,12 @@ set_sock_opts(ConnectionRef, SocketOptions) -> {error, badarg} end. +prohibited_sock_option(active) -> true; +prohibited_sock_option(deliver) -> true; +prohibited_sock_option(mode) -> true; +prohibited_sock_option(packet) -> true; +prohibited_sock_option(_) -> false. + %%-------------------------------------------------------------------- %% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . get_sock_opts(ConnectionRef, SocketGetOptions) -> @@ -818,13 +825,13 @@ handle_event(_, #ssh_msg_kex_dh_gex_reply{} = Msg, {key_exchange_dh_gex_reply,cl %%% ######## {new_keys, client|server} #### %% First key exchange round: -handle_event(_, #ssh_msg_newkeys{} = Msg, {new_keys,client,init}, D) -> - {ok, Ssh1} = ssh_transport:handle_new_keys(Msg, D#data.ssh_params), +handle_event(_, #ssh_msg_newkeys{} = Msg, {new_keys,client,init}, D0) -> + {ok, Ssh1} = ssh_transport:handle_new_keys(Msg, D0#data.ssh_params), %% {ok, ExtInfo, Ssh2} = ssh_transport:ext_info_message(Ssh1), - %% send_bytes(ExtInfo, D), + %% send_bytes(ExtInfo, D0), {MsgReq, Ssh} = ssh_auth:service_request_msg(Ssh1), - send_bytes(MsgReq, D), - {next_state, {ext_info,client,init}, D#data{ssh_params=Ssh}}; + D = send_msg(MsgReq, D0#data{ssh_params = Ssh}), + {next_state, {ext_info,client,init}, D}; handle_event(_, #ssh_msg_newkeys{} = Msg, {new_keys,server,init}, D) -> {ok, Ssh} = ssh_transport:handle_new_keys(Msg, D#data.ssh_params), @@ -870,8 +877,8 @@ handle_event(_, Msg = #ssh_msg_service_request{name=ServiceName}, StateName = {s "ssh-userauth" -> Ssh0 = #ssh{session_id=SessionId} = D0#data.ssh_params, {ok, {Reply, Ssh}} = ssh_auth:handle_userauth_request(Msg, SessionId, Ssh0), - send_bytes(Reply, D0), - {next_state, {userauth,server}, D0#data{ssh_params = Ssh}}; + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {next_state, {userauth,server}, D}; _ -> {Shutdown, D} = @@ -882,10 +889,12 @@ handle_event(_, Msg = #ssh_msg_service_request{name=ServiceName}, StateName = {s end; handle_event(_, #ssh_msg_service_accept{name = "ssh-userauth"}, {service_request,client}, - #data{ssh_params = #ssh{service="ssh-userauth"} = Ssh0} = State) -> + #data{ssh_params = #ssh{service="ssh-userauth"} = Ssh0} = D0) -> {Msg, Ssh} = ssh_auth:init_userauth_request_msg(Ssh0), - send_bytes(Msg, State), - {next_state, {userauth,client}, State#data{auth_user = Ssh#ssh.user, ssh_params = Ssh}}; + D = send_msg(Msg, D0#data{ssh_params = Ssh, + auth_user = Ssh#ssh.user + }), + {next_state, {userauth,client}, D}; %%% ######## {userauth, client|server} #### @@ -901,8 +910,8 @@ handle_event(_, %% Probably the very first userauth_request but we deny unauthorized login {not_authorized, _, {Reply,Ssh}} = ssh_auth:handle_userauth_request(Msg, Ssh0#ssh.session_id, Ssh0), - send_bytes(Reply, D0), - {keep_state, D0#data{ssh_params = Ssh}}; + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {keep_state, D}; {"ssh-connection", "ssh-connection", Method} -> %% Userauth request with a method like "password" or so @@ -910,21 +919,24 @@ handle_event(_, true -> %% Yepp! we support this method case ssh_auth:handle_userauth_request(Msg, Ssh0#ssh.session_id, Ssh0) of - {authorized, User, {Reply, Ssh}} -> - send_bytes(Reply, D0), - D0#data.starter ! ssh_connected, - connected_fun(User, Method, D0), + {authorized, User, {Reply, Ssh1}} -> + D = #data{ssh_params=Ssh} = + send_msg(Reply, D0#data{ssh_params = Ssh1}), + D#data.starter ! ssh_connected, + connected_fun(User, Method, D), {next_state, {connected,server}, - D0#data{auth_user = User, - ssh_params = Ssh#ssh{authenticated = true}}}; + D#data{auth_user=User, + %% Note: authenticated=true MUST NOT be sent + %% before send_msg! + ssh_params = Ssh#ssh{authenticated = true}}}; {not_authorized, {User, Reason}, {Reply, Ssh}} when Method == "keyboard-interactive" -> retry_fun(User, Reason, D0), - send_bytes(Reply, D0), - {next_state, {userauth_keyboard_interactive,server}, D0#data{ssh_params = Ssh}}; + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {next_state, {userauth_keyboard_interactive,server}, D}; {not_authorized, {User, Reason}, {Reply, Ssh}} -> retry_fun(User, Reason, D0), - send_bytes(Reply, D0), - {keep_state, D0#data{ssh_params = Ssh}} + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {keep_state, D} end; false -> %% No we do not support this method (=/= none) @@ -952,6 +964,7 @@ handle_event(_, #ssh_msg_ext_info{}=Msg, {userauth,client}, D0) -> {keep_state, D}; handle_event(_, #ssh_msg_userauth_success{}, {userauth,client}, D=#data{ssh_params = Ssh}) -> + ssh_auth:ssh_msg_userauth_result(success), D#data.starter ! ssh_connected, {next_state, {connected,client}, D#data{ssh_params=Ssh#ssh{authenticated = true}}}; @@ -983,11 +996,11 @@ handle_event(_, #ssh_msg_userauth_failure{authentications = Methods}, StateName= StateName, D0#data{ssh_params = Ssh}), {stop, Shutdown, D}; {"keyboard-interactive", {Msg, Ssh}} -> - send_bytes(Msg, D0), - {next_state, {userauth_keyboard_interactive,client}, D0#data{ssh_params = Ssh}}; + D = send_msg(Msg, D0#data{ssh_params = Ssh}), + {next_state, {userauth_keyboard_interactive,client}, D}; {_Method, {Msg, Ssh}} -> - send_bytes(Msg, D0), - {keep_state, D0#data{ssh_params = Ssh}} + D = send_msg(Msg, D0#data{ssh_params = Ssh}), + {keep_state, D} end; %%---- banner to client @@ -1002,39 +1015,46 @@ handle_event(_, #ssh_msg_userauth_banner{message = Msg}, {userauth,client}, D) - %%% ######## {userauth_keyboard_interactive, client|server} handle_event(_, #ssh_msg_userauth_info_request{} = Msg, {userauth_keyboard_interactive, client}, - #data{ssh_params = Ssh0} = D) -> + #data{ssh_params = Ssh0} = D0) -> case ssh_auth:handle_userauth_info_request(Msg, Ssh0) of {ok, {Reply, Ssh}} -> - send_bytes(Reply, D), - {next_state, {userauth_keyboard_interactive_info_response,client}, D#data{ssh_params = Ssh}}; + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {next_state, {userauth_keyboard_interactive_info_response,client}, D}; not_ok -> - {next_state, {userauth,client}, D, [postpone]} + {next_state, {userauth,client}, D0, [postpone]} end; -handle_event(_, #ssh_msg_userauth_info_response{} = Msg, {userauth_keyboard_interactive, server}, D) -> - case ssh_auth:handle_userauth_info_response(Msg, D#data.ssh_params) of - {authorized, User, {Reply, Ssh}} -> - send_bytes(Reply, D), +handle_event(_, #ssh_msg_userauth_info_response{} = Msg, {userauth_keyboard_interactive, server}, D0) -> + case ssh_auth:handle_userauth_info_response(Msg, D0#data.ssh_params) of + {authorized, User, {Reply, Ssh1}} -> + D = #data{ssh_params=Ssh} = + send_msg(Reply, D0#data{ssh_params = Ssh1}), D#data.starter ! ssh_connected, connected_fun(User, "keyboard-interactive", D), {next_state, {connected,server}, D#data{auth_user = User, + %% Note: authenticated=true MUST NOT be sent + %% before send_msg! ssh_params = Ssh#ssh{authenticated = true}}}; {not_authorized, {User, Reason}, {Reply, Ssh}} -> - retry_fun(User, Reason, D), - send_bytes(Reply, D), - {next_state, {userauth,server}, D#data{ssh_params = Ssh}}; + retry_fun(User, Reason, D0), + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {next_state, {userauth,server}, D}; {authorized_but_one_more, _User, {Reply, Ssh}} -> - send_bytes(Reply, D), - {next_state, {userauth_keyboard_interactive_extra,server}, D#data{ssh_params = Ssh}} + D = send_msg(Reply, D0#data{ssh_params = Ssh}), + {next_state, {userauth_keyboard_interactive_extra,server}, D} end; -handle_event(_, #ssh_msg_userauth_info_response{} = Msg, {userauth_keyboard_interactive_extra, server}, D) -> - {authorized, User, {Reply, Ssh}} = ssh_auth:handle_userauth_info_response({extra,Msg}, D#data.ssh_params), - send_bytes(Reply, D), +handle_event(_, #ssh_msg_userauth_info_response{} = Msg, {userauth_keyboard_interactive_extra, server}, D0) -> + {authorized, User, {Reply, Ssh1}} = + ssh_auth:handle_userauth_info_response({extra,Msg}, D0#data.ssh_params), + D = #data{ssh_params=Ssh} = + send_msg(Reply, D0#data{ssh_params = Ssh1}), D#data.starter ! ssh_connected, connected_fun(User, "keyboard-interactive", D), {next_state, {connected,server}, D#data{auth_user = User, + %% Note: authenticated=true MUST NOT be sent + %% before send_msg! ssh_params = Ssh#ssh{authenticated = true}}}; handle_event(_, #ssh_msg_userauth_failure{}, {userauth_keyboard_interactive, client}, @@ -1144,17 +1164,17 @@ handle_event(internal, {conn_msg,Msg}, StateName, #data{starter = User, end; -handle_event(enter, _OldState, {connected,_}=State, D) -> +handle_event(enter, OldState, {connected,_}=NewState, D) -> %% Entering the state where re-negotiation is possible - init_renegotiate_timers(State, D); + init_renegotiate_timers(OldState, NewState, D); -handle_event(enter, _OldState, {ext_info,_,renegotiate}=State, D) -> +handle_event(enter, OldState, {ext_info,_,renegotiate}=NewState, D) -> %% Could be hanging in exit_info state if nothing else arrives - init_renegotiate_timers(State, D); + init_renegotiate_timers(OldState, NewState, D); -handle_event(enter, {connected,_}, State, D) -> +handle_event(enter, {connected,_}=OldState, NewState, D) -> %% Exiting the state where re-negotiation is possible - pause_renegotiate_timers(State, D); + pause_renegotiate_timers(OldState, NewState, D); handle_event(cast, force_renegotiate, StateName, D) -> handle_event({timeout,renegotiate}, undefined, StateName, D); @@ -2112,25 +2132,32 @@ start_rekeying(Role, D0) -> {next_state, {kexinit,Role,renegotiate}, D}. -init_renegotiate_timers(State, D) -> +init_renegotiate_timers(_OldState, NewState, D) -> {RekeyTimeout,_MaxSent} = ?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), - {next_state, State, D, [{{timeout,renegotiate}, RekeyTimeout, none}, - {{timeout,check_data_size}, ?REKEY_DATA_TIMOUT, none} ]}. + {next_state, NewState, D, [{{timeout,renegotiate}, RekeyTimeout, none}, + {{timeout,check_data_size}, ?REKEY_DATA_TIMOUT, none} ]}. -pause_renegotiate_timers(State, D) -> - {next_state, State, D, [{{timeout,renegotiate}, infinity, none}, - {{timeout,check_data_size}, infinity, none} ]}. +pause_renegotiate_timers(_OldState, NewState, D) -> + {next_state, NewState, D, [{{timeout,renegotiate}, infinity, none}, + {{timeout,check_data_size}, infinity, none} ]}. check_data_rekeying(Role, D) -> - {ok, [{send_oct,SocketSentTotal}]} = inet:getstat(D#data.socket, [send_oct]), - SentSinceRekey = SocketSentTotal - D#data.last_size_rekey, - {_RekeyTimeout,MaxSent} = ?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), - case check_data_rekeying_dbg(SentSinceRekey, MaxSent) of - true -> - start_rekeying(Role, D#data{last_size_rekey = SocketSentTotal}); - _ -> - %% Not enough data sent for a re-negotiation. Restart timer. + case inet:getstat(D#data.socket, [send_oct]) of + {ok, [{send_oct,SocketSentTotal}]} -> + SentSinceRekey = SocketSentTotal - D#data.last_size_rekey, + {_RekeyTimeout,MaxSent} = ?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), + case check_data_rekeying_dbg(SentSinceRekey, MaxSent) of + true -> + start_rekeying(Role, D#data{last_size_rekey = SocketSentTotal}); + _ -> + %% Not enough data sent for a re-negotiation. Restart timer. + {keep_state, D, {{timeout,check_data_size}, ?REKEY_DATA_TIMOUT, none}} + end; + {error,_} -> + %% Socket closed, but before this module has handled that. Maybe + %% it is in the message queue. + %% Just go on like if there was not enough data transmitted to start re-keying: {keep_state, D, {{timeout,check_data_size}, ?REKEY_DATA_TIMOUT, none}} end. @@ -2489,20 +2516,22 @@ ssh_dbg_flags(disconnect) -> [c]. ssh_dbg_on(connections) -> dbg:tp(?MODULE, init_connection_handler, 3, x), ssh_dbg_on(terminate); ssh_dbg_on(connection_events) -> dbg:tp(?MODULE, handle_event, 4, x); -ssh_dbg_on(renegotiation) -> dbg:tpl(?MODULE, init_renegotiate_timers, 2, x), - dbg:tpl(?MODULE, pause_renegotiate_timers, 2, x), +ssh_dbg_on(renegotiation) -> dbg:tpl(?MODULE, init_renegotiate_timers, 3, x), + dbg:tpl(?MODULE, pause_renegotiate_timers, 3, x), dbg:tpl(?MODULE, check_data_rekeying_dbg, 2, x), - dbg:tpl(?MODULE, start_rekeying, 2, x); + dbg:tpl(?MODULE, start_rekeying, 2, x), + dbg:tp(?MODULE, renegotiate, 1, x); ssh_dbg_on(terminate) -> dbg:tp(?MODULE, terminate, 3, x); ssh_dbg_on(disconnect) -> dbg:tpl(?MODULE, send_disconnect, 7, x). ssh_dbg_off(disconnect) -> dbg:ctpl(?MODULE, send_disconnect, 7); ssh_dbg_off(terminate) -> dbg:ctpg(?MODULE, terminate, 3); -ssh_dbg_off(renegotiation) -> dbg:ctpl(?MODULE, init_renegotiate_timers, 2), - dbg:ctpl(?MODULE, pause_renegotiate_timers, 2), +ssh_dbg_off(renegotiation) -> dbg:ctpl(?MODULE, init_renegotiate_timers, 3), + dbg:ctpl(?MODULE, pause_renegotiate_timers, 3), dbg:ctpl(?MODULE, check_data_rekeying_dbg, 2), - dbg:ctpl(?MODULE, start_rekeying, 2); + dbg:ctpl(?MODULE, start_rekeying, 2), + dbg:ctpg(?MODULE, renegotiate, 1); ssh_dbg_off(connection_events) -> dbg:ctpg(?MODULE, handle_event, 4); ssh_dbg_off(connections) -> dbg:ctpg(?MODULE, init_connection_handler, 3), ssh_dbg_off(terminate). @@ -2541,22 +2570,46 @@ ssh_dbg_format(connection_events, {return_from, {?MODULE,handle_event,4}, Ret}) io_lib:format("~p~n", [event_handler_result(Ret)]) ]; -ssh_dbg_format(renegotiation, {call, {?MODULE,init_renegotiate_timers,[_State,D]}}) -> - ["Renegotiation init\n", - io_lib:format("rekey_limit: ~p ({ms,bytes})~ncheck_data_size: ~p (ms)~n", - [?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), +ssh_dbg_format(renegotiation, {call, {?MODULE,init_renegotiate_timers,[OldState,NewState,D]}}) -> + ["Renegotiation: start timer (init_renegotiate_timers)\n", + io_lib:format("State: ~p --> ~p~n" + "rekey_limit: ~p ({ms,bytes})~n" + "check_data_size: ~p (ms)~n", + [OldState, NewState, + ?GET_OPT(rekey_limit, (D#data.ssh_params)#ssh.opts), ?REKEY_DATA_TIMOUT]) ]; -ssh_dbg_format(renegotiation, {call, {?MODULE,pause_renegotiate_timers,[_State,_D]}}) -> - ["Renegotiation pause\n"]; +ssh_dbg_format(renegotiation, {return_from, {?MODULE,init_renegotiate_timers,3}, _Ret}) -> + skip; + +ssh_dbg_format(renegotiation, {call, {?MODULE,renegotiate,[ConnectionHandler]}}) -> + ["Renegotiation: renegotiation forced\n", + io_lib:format("~p:renegotiate(~p) called~n", + [?MODULE,ConnectionHandler]) + ]; +ssh_dbg_format(renegotiation, {return_from, {?MODULE,renegotiate,1}, _Ret}) -> + skip; + +ssh_dbg_format(renegotiation, {call, {?MODULE,pause_renegotiate_timers,[OldState,NewState,_D]}}) -> + ["Renegotiation: pause timers\n", + io_lib:format("State: ~p --> ~p~n", + [OldState, NewState]) + ]; +ssh_dbg_format(renegotiation, {return_from, {?MODULE,pause_renegotiate_timers,3}, _Ret}) -> + skip; + ssh_dbg_format(renegotiation, {call, {?MODULE,start_rekeying,[_Role,_D]}}) -> - ["Renegotiation start rekeying\n"]; + ["Renegotiation: start rekeying\n"]; +ssh_dbg_format(renegotiation, {return_from, {?MODULE,start_rekeying,2}, _Ret}) -> + skip; + ssh_dbg_format(renegotiation, {call, {?MODULE,check_data_rekeying_dbg,[SentSinceRekey, MaxSent]}}) -> - ["Renegotiation check data sent\n", + ["Renegotiation: check size of data sent\n", io_lib:format("TotalSentSinceRekey: ~p~nMaxBeforeRekey: ~p~nStartRekey: ~p~n", [SentSinceRekey, MaxSent, SentSinceRekey >= MaxSent]) ]; - +ssh_dbg_format(renegotiation, {return_from, {?MODULE,check_data_rekeying_dbg,2}, _Ret}) -> + skip; ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, StateName, D]}}) -> @@ -2591,6 +2644,8 @@ ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, StateName, D]}}) - [Reason, StateName, ExtraInfo, state_data2proplist(D)]) ] end; +ssh_dbg_format(renegotiation, {return_from, {?MODULE,terminate,3}, _Ret}) -> + skip; ssh_dbg_format(disconnect, {call,{?MODULE,send_disconnect, [Code, Reason, DetailedText, Module, Line, StateName, _D]}}) -> @@ -2600,7 +2655,9 @@ ssh_dbg_format(disconnect, {call,{?MODULE,send_disconnect, " DetailedText =~n" " ~p", [Module, Line, StateName, Code, Reason, lists:flatten(DetailedText)]) - ]. + ]; +ssh_dbg_format(renegotiation, {return_from, {?MODULE,send_disconnect,7}, _Ret}) -> + skip. event_handler_result({next_state, NextState, _NewData}) -> diff --git a/lib/ssh/src/ssh_dbg.erl b/lib/ssh/src/ssh_dbg.erl index 1809b14099..f8391224f8 100644 --- a/lib/ssh/src/ssh_dbg.erl +++ b/lib/ssh/src/ssh_dbg.erl @@ -54,6 +54,8 @@ start_tracer/0, start_tracer/1, on/1, on/0, off/1, off/0, + is_on/0, + is_off/0, go_on/0, %% Circular buffer cbuf_start/0, cbuf_start/1, @@ -70,6 +72,9 @@ -export([init/1, handle_call/3, handle_cast/2, handle_info/2]). +%% Internal apply_after: +-export([ets_delete/2]). + -include("ssh.hrl"). -include("ssh_transport.hrl"). -include("ssh_connect.hrl"). @@ -82,12 +87,16 @@ -type trace_point() :: atom(). -type trace_points() :: [trace_point()]. +-type stack() :: list(term()). -callback ssh_dbg_trace_points() -> trace_points(). -callback ssh_dbg_flags(trace_point()) -> [atom()]. -callback ssh_dbg_on(trace_point() | trace_points()) -> term(). -callback ssh_dbg_off(trace_point() | trace_points()) -> term(). --callback ssh_dbg_format(trace_point(), term()) -> iolist(). +-callback ssh_dbg_format(trace_point(), term()) -> iolist() | skip. +-callback ssh_dbg_format(trace_point(), term(), stack()) -> {iolist() | skip, stack()}. + +-optional_callbacks([ssh_dbg_format/2, ssh_dbg_format/3]). % At least one of them are to be used %%%================================================================ @@ -134,10 +143,13 @@ start_tracer(WriteFun, InitAcc) when is_function(WriteFun, 3) -> %%%---------------------------------------------------------------- on() -> on(?ALL_DBG_TYPES). on(Type) -> switch(on, Type). - +is_on() -> gen_server:call(?SERVER, get_on, ?CALL_TIMEOUT). + off() -> off(?ALL_DBG_TYPES). % A bit overkill... off(Type) -> switch(off, Type). +is_off() -> ?ALL_DBG_TYPES -- is_on(). + go_on() -> IsOn = gen_server:call(?SERVER, get_on, ?CALL_TIMEOUT), @@ -174,8 +186,42 @@ reduce_state(T) -> %%%---------------------------------------------------------------- init(_) -> + new_table(), {ok, #data{}}. + +new_table() -> + try + ets:new(?MODULE, [public, named_table]), + ok + catch + exit:badarg -> + ok + end. + + +get_proc_stack(Pid) when is_pid(Pid) -> + try ets:lookup_element(?MODULE, Pid, 2) + catch + error:badarg -> + %% Non-existing item + new_proc(Pid), + ets:insert(?MODULE, {Pid,[]}), + [] + end. + + +put_proc_stack(Pid, Data) when is_pid(Pid), + is_list(Data) -> + ets:insert(?MODULE, {Pid,Data}). + + +new_proc(Pid) when is_pid(Pid) -> + gen_server:cast(?SERVER, {new_proc,Pid}). + +ets_delete(Tab, Key) -> + catch ets:delete(Tab, Key). + %%%---------------------------------------------------------------- handle_call({switch,on,Types}, _From, D) -> NowOn = lists:usort(Types ++ D#data.types_on), @@ -196,10 +242,20 @@ handle_call(C, _From, D) -> {reply, {error,{unknown_call,C}}, D}. +handle_cast({new_proc,Pid}, D) -> + monitor(process, Pid), + {noreply, D}; + handle_cast(C, D) -> io:format('*** Unknown cast: ~p~n',[C]), {noreply, D}. + +handle_info({'DOWN', _MonitorRef, process, Pid, _Info}, D) -> + %% Universal real-time synchronization (there might be dbg msgs in the queue to the tracer): + timer:apply_after(20000, ?MODULE, ets_delete, [?MODULE, Pid]), + {noreply, D}; + handle_info(C, D) -> io:format('*** Unknown info: ~p~n',[C]), {noreply, D}. @@ -320,20 +376,60 @@ try_all_types_in_all_modules(TypesOn, Arg, WriteFun, Acc0) -> TS = trace_ts(Arg), PID = trace_pid(Arg), INFO = trace_info(Arg), - lists:foldl( - fun(Type, Acc1) -> - lists:foldl( - fun(SshMod,Acc) -> - try WriteFun("~n~s ~p ~s~n", - [lists:flatten(TS), - PID, - lists:flatten(SshMod:ssh_dbg_format(Type, INFO))], - Acc) - catch - _:_ -> Acc - end - end, Acc1, SshModules) - end, Acc0, TypesOn). + Acc = + lists:foldl( + fun(Type, Acc1) -> + lists:foldl( + fun(SshMod,Acc) -> + try + %% First, call without stack + SshMod:ssh_dbg_format(Type, INFO) + of + skip -> + %% Don't try to print this later + written; + Txt when is_list(Txt) -> + write_txt(WriteFun, TS, PID, Txt) + catch + error:E when E==undef ; E==function_clause ; element(1,E)==case_clause -> + try + %% then, call with stack + STACK = get_proc_stack(PID), + SshMod:ssh_dbg_format(Type, INFO, STACK) + of + {skip, NewStack} -> + %% Don't try to print this later + put_proc_stack(PID, NewStack), + written; + {Txt, NewStack} when is_list(Txt) -> + put_proc_stack(PID, NewStack), + write_txt(WriteFun, TS, PID, Txt) + catch + _:_ -> + %% and finally, signal for special formatting + %% if noone else formats it + Acc + end + end + end, Acc1, SshModules) + end, Acc0, TypesOn), + case Acc of + Acc0 -> + %% INFO :: any() + WriteFun("~n~s ~p DEBUG~n~p~n", [lists:flatten(TS),PID,INFO], Acc0); + written -> + Acc0 + end. + + + +write_txt(WriteFun, TS, PID, Txt) when is_list(Txt) -> + WriteFun("~n~s ~p ~s~n", + [lists:flatten(TS), + PID, + lists:flatten(Txt)], + written % this is returned + ). %%%---------------------------------------------------------------- wr_record(T, Fs, BL) when is_tuple(T) -> diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl index 30b0c89144..804775bd75 100644 --- a/lib/ssh/src/ssh_message.erl +++ b/lib/ssh/src/ssh_message.erl @@ -762,15 +762,26 @@ ssh_dbg_format(ssh_messages, {call,{?MODULE,encode,[Msg]}}) -> ["Going to send ",Name,":\n", wr_record(ssh_dbg:shrink_bin(Msg)) ]; +ssh_dbg_format(ssh_messages, {return_from, {?MODULE,encode,1}, _Ret}) -> + skip; + +ssh_dbg_format(ssh_messages, {call, {?MODULE,decode,[_]}}) -> + skip; ssh_dbg_format(ssh_messages, {return_from,{?MODULE,decode,1},Msg}) -> Name = string:to_upper(atom_to_list(element(1,Msg))), ["Received ",Name,":\n", wr_record(ssh_dbg:shrink_bin(Msg)) ]; + ssh_dbg_format(raw_messages, {call,{?MODULE,decode,[BytesPT]}}) -> ["Received plain text bytes (shown after decryption):\n", io_lib:format("~p",[BytesPT]) ]; +ssh_dbg_format(raw_messages, {return_from, {?MODULE,decode,1}, _Ret}) -> + skip; + +ssh_dbg_format(raw_messages, {call, {?MODULE,encode,[_]}}) -> + skip; ssh_dbg_format(raw_messages, {return_from,{?MODULE,encode,1},BytesPT}) -> ["Going to send plain text bytes (shown before encryption):\n", io_lib:format("~p",[BytesPT]) diff --git a/lib/ssh/src/ssh_options.erl b/lib/ssh/src/ssh_options.erl index cc9ef565d8..306eb86c23 100644 --- a/lib/ssh/src/ssh_options.erl +++ b/lib/ssh/src/ssh_options.erl @@ -261,22 +261,20 @@ config_val(Key, RoleCnfs, Opts) -> check_fun(Key, Defs) -> - #{chk := Fun} = maps:get(Key, Defs), - Fun. + case ssh_connection_handler:prohibited_sock_option(Key) of + false -> + #{chk := Fun} = maps:get(Key, Defs), + Fun; + true -> + fun(_,_) -> forbidden end + end. %%%================================================================ %%% %%% Check and save one option %%% - -%%% First some prohibited inet options: -save({K,V}, _, _) when K == reuseaddr ; - K == active - -> - forbidden_option(K, V); - -%%% then compatibility conversions: +%%% First compatibility conversions: save({allow_user_interaction,V}, Opts, Vals) -> save({user_interaction,V}, Opts, Vals); @@ -297,7 +295,12 @@ save({Key,Value}, Defs, OptMap) when is_map(OptMap) -> {true, ModifiedValue} -> OptMap#{Key := ModifiedValue}; false -> - error({eoptions, {Key,Value}, "Bad value"}) + error({eoptions, {Key,Value}, "Bad value"}); + forbidden -> + error({eoptions, {Key,Value}, + io_lib:format("The option '~s' is used internally. The " + "user is not allowed to specify this option.", + [Key])}) catch %% An unknown Key (= not in the definition map) is %% regarded as an inet option: @@ -424,7 +427,8 @@ default(server) -> check_string(S3) andalso is_boolean(B); (F) -> - check_function3(F) + check_function3(F) orelse + check_function4(F) end, class => user_option }, @@ -1180,10 +1184,3 @@ error_if_empty([]) -> ok. %%%---------------------------------------------------------------- -forbidden_option(K,V) -> - Txt = io_lib:format("The option '~s' is used internally. The " - "user is not allowed to specify this option.", - [K]), - error({eoptions, {K,V}, Txt}). - -%%%---------------------------------------------------------------- diff --git a/lib/ssh/src/ssh_sftp.erl b/lib/ssh/src/ssh_sftp.erl index 9ee6ed68c1..69fff09979 100644 --- a/lib/ssh/src/ssh_sftp.erl +++ b/lib/ssh/src/ssh_sftp.erl @@ -1843,7 +1843,9 @@ ssh_dbg_off(terminate) -> dbg:ctpg(?MODULE, terminate, 2). ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, State]}}) -> ["Sftp Terminating:\n", io_lib:format("Reason: ~p,~nState:~n~s", [Reason, wr_record(State)]) - ]. + ]; +ssh_dbg_format(terminate, {return_from, {?MODULE,terminate,2}, _Ret}) -> + skip. ?wr_record(state). diff --git a/lib/ssh/src/ssh_sftpd.erl b/lib/ssh/src/ssh_sftpd.erl index 78277705d8..158ef3d5ae 100644 --- a/lib/ssh/src/ssh_sftpd.erl +++ b/lib/ssh/src/ssh_sftpd.erl @@ -964,6 +964,8 @@ ssh_dbg_off(terminate) -> dbg:ctpg(?MODULE, terminate, 2). ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, State]}}) -> ["SftpD Terminating:\n", io_lib:format("Reason: ~p,~nState:~n~s", [Reason, wr_record(State)]) - ]. + ]; +ssh_dbg_format(terminate, {return_from, {?MODULE,terminate,2}, _Ret}) -> + skip. ?wr_record(state). diff --git a/lib/ssh/src/ssh_shell.erl b/lib/ssh/src/ssh_shell.erl index b5862b2395..be8a6aa8cc 100644 --- a/lib/ssh/src/ssh_shell.erl +++ b/lib/ssh/src/ssh_shell.erl @@ -200,6 +200,8 @@ ssh_dbg_off(terminate) -> dbg:ctpg(?MODULE, terminate, 2). ssh_dbg_format(terminate, {call, {?MODULE,terminate, [Reason, State]}}) -> ["Shell Terminating:\n", io_lib:format("Reason: ~p,~nState:~n~s", [Reason, wr_record(State)]) - ]. + ]; +ssh_dbg_format(terminate, {return_from, {?MODULE,terminate,2}, _Ret}) -> + skip. ?wr_record(state). diff --git a/lib/ssh/src/ssh_transport.erl b/lib/ssh/src/ssh_transport.erl index 86d4ef951c..bd901c99e0 100644 --- a/lib/ssh/src/ssh_transport.erl +++ b/lib/ssh/src/ssh_transport.erl @@ -2172,18 +2172,27 @@ ssh_dbg_off(ssh_messages) -> ssh_dbg_off(hello). +ssh_dbg_format(hello, {call,{?MODULE,hello_version_msg,[_]}}) -> + skip; ssh_dbg_format(hello, {return_from,{?MODULE,hello_version_msg,1},Hello}) -> ["Going to send hello message:\n", Hello ]; + ssh_dbg_format(hello, {call,{?MODULE,handle_hello_version,[Hello]}}) -> ["Received hello message:\n", Hello ]; +ssh_dbg_format(hello, {return_from,{?MODULE,handle_hello_version,1},_Ret}) -> + skip; + +ssh_dbg_format(alg, {call,{?MODULE,select_algorithm,[_,_,_,_]}}) -> + skip; ssh_dbg_format(alg, {return_from,{?MODULE,select_algorithm,4},{ok,Alg}}) -> ["Negotiated algorithms:\n", wr_record(Alg) ]; + ssh_dbg_format(raw_messages, X) -> ssh_dbg_format(hello, X); ssh_dbg_format(ssh_messages, X) -> ssh_dbg_format(hello, X). diff --git a/lib/ssh/test/.gitignore b/lib/ssh/test/.gitignore index c9d5f086b3..f0adbaf33f 100644 --- a/lib/ssh/test/.gitignore +++ b/lib/ssh/test/.gitignore @@ -1,4 +1,6 @@ +*COVER.html +ssh_sftp_SUITE_data/test_data* property_test/ssh_eqc_client_server_dirs/system property_test/ssh_eqc_client_server_dirs/user diff --git a/lib/ssh/test/Makefile b/lib/ssh/test/Makefile index aafec6566e..9862071b5f 100644 --- a/lib/ssh/test/Makefile +++ b/lib/ssh/test/Makefile @@ -46,6 +46,7 @@ MODULES= \ ssh_protocol_SUITE \ ssh_property_test_SUITE \ ssh_pubkey_SUITE \ + ssh_renegotiate_SUITE \ ssh_sftp_SUITE \ ssh_sftpd_SUITE \ ssh_sftpd_erlclient_SUITE \ diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server.erl b/lib/ssh/test/property_test/ssh_eqc_client_server.erl index e0b739ab53..66a79c8a17 100644 --- a/lib/ssh/test/property_test/ssh_eqc_client_server.erl +++ b/lib/ssh/test/property_test/ssh_eqc_client_server.erl @@ -480,12 +480,9 @@ setup_rsa(Dir) -> erase_dir(user_dir(Dir)), file:make_dir(system_dir(Dir)), file:make_dir(user_dir(Dir)), - - file:copy(data_dir(Dir,"id_rsa"), user_dir(Dir,"id_rsa")), - file:copy(data_dir(Dir,"ssh_host_rsa_key"), system_dir(Dir,"ssh_host_rsa_key")), - file:copy(data_dir(Dir,"ssh_host_rsa_key"), system_dir(Dir,"ssh_host_rsa_key.pub")), - ssh_test_lib:setup_rsa_known_host(data_dir(Dir), user_dir(Dir)), - ssh_test_lib:setup_rsa_auth_keys(data_dir(Dir), user_dir(Dir)). + ct:log("Dir = ~p~ndata_dir = ~p~nsystem_dir = ~p~nuser = ~p~n", + [Dir,data_dir(Dir),system_dir(Dir),user_dir(Dir)]), + ssh_test_lib:setup_all_user_host_keys( data_dir(Dir), user_dir(Dir), system_dir(Dir)). data_dir(Dir, File) -> filename:join(Dir, File). system_dir(Dir, File) -> filename:join([Dir, "system", File]). diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa index d306f8b26e..24628e071b 100644 --- a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa @@ -1,13 +1,12 @@ -----BEGIN DSA PRIVATE KEY----- -MIIBvAIBAAKBgQDfi2flSTZZofwT4yQT0NikX/LGNT7UPeB/XEWe/xovEYCElfaQ -APFixXvEgXwoojmZ5kiQRKzLM39wBP0jPERLbnZXfOOD0PDnw0haMh7dD7XKVMod -/EigVgHf/qBdM2M8yz1s/rRF7n1UpLSypziKjkzCm7JoSQ2zbWIPdmBIXwIVAMgP -kpr7Sq3O7sHdb8D601DRjoExAoGAMOQxDfB2Fd8ouz6G96f/UOzRMI/Kdv8kYYKW -JIGY+pRYrLPyYzUeJznwZreOJgrczAX+luHnKFWJ2Dnk5CyeXk67Wsr7pJ/4MBMD -OKeIS0S8qoSBN8+Krp79fgA+yS3IfqbkJLtLu4EBaCX4mKQIX4++k44d4U5lc8pt -+9hlEI8CgYEAznKxx9kyC6bVo7LUYKaGhofRFt0SYFc5PVmT2VUGRs1R6+6DPD+e -uEO6IhFct7JFSRbP9p0JD4Uk+3zlZF+XX6b2PsZkeV8f/02xlNGUSmEzCSiNg1AX -Cy/WusYhul0MncWCHMcOZB5rIvU/aP5EJJtn3xrRaz6u0SThF6AnT34CFQC63czE -ZU8w8Q+H7z0j+a+70x2iAw== +MIIBuwIBAAKBgQDIywHurUpOq6kZuMn+XlRzR4hAxF6qwSkuEqkV7iHnLQ0kIwf3 +uAmjFDhuEsQ8653SLxGVvTNp+KFFgDXiLqgM7TPUwDnpbvzEZHPAU+/zPt4sdY2D +txBfJwT2SFlK6HPOxOcxdDuD+/a59sh8hk/YVOU7ZTcBVsVG8Got4UcF5QIVAPGd +CPDQKSTlPiM9OwBB1+9p11k5AoGARLxw4l17mET9cU0uf4Ppe5nsCbODJv44ZrSs +picvypGVLrLcN5KWbm3vjRFCQ5LFunAG3FwLC2Sh0CH6TemoIfRPsRHR7wvpBGdr +c693UlMOis/mcmvNMQAzuQNW9WrxdzsvWR/r5s6NEHWqKUJGXSPi2d+Ijq/mCOmI +hzLzyiACgYEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4 +cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+C +ROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNgCFEjA7wTC +sQCY/I35vb6GUJn9tEdP -----END DSA PRIVATE KEY----- - diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa.pub new file mode 100644 index 0000000000..018ef6f537 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_dsa.pub @@ -0,0 +1 @@ +ssh-dss AAAAB3NzaC1kc3MAAACBAMjLAe6tSk6rqRm4yf5eVHNHiEDEXqrBKS4SqRXuIectDSQjB/e4CaMUOG4SxDzrndIvEZW9M2n4oUWANeIuqAztM9TAOelu/MRkc8BT7/M+3ix1jYO3EF8nBPZIWUroc87E5zF0O4P79rn2yHyGT9hU5TtlNwFWxUbwai3hRwXlAAAAFQDxnQjw0Ckk5T4jPTsAQdfvaddZOQAAAIBEvHDiXXuYRP1xTS5/g+l7mewJs4Mm/jhmtKymJy/KkZUustw3kpZube+NEUJDksW6cAbcXAsLZKHQIfpN6agh9E+xEdHvC+kEZ2tzr3dSUw6Kz+Zya80xADO5A1b1avF3Oy9ZH+vmzo0QdaopQkZdI+LZ34iOr+YI6YiHMvPKIAAAAIEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+CROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNg= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa256 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa256 new file mode 100644 index 0000000000..4b1eb12eaa --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIJfCaBKIIKhjbJl5F8BedqlXOQYDX5ba9Skypllmx/w+oAoGCCqGSM49 +AwEHoUQDQgAE49RbK2xQ/19ji3uDPM7uT4692LbwWF1TiaA9vUuebMGazoW/98br +N9xZu0L1AWwtEjs3kmJDTB7eJEGXnjUAcQ== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa256.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa256.pub new file mode 100644 index 0000000000..a0147e60fa --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPUWytsUP9fY4t7gzzO7k+Ovdi28FhdU4mgPb1LnmzBms6Fv/fG6zfcWbtC9QFsLRI7N5JiQ0we3iRBl541AHE= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa384 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa384 new file mode 100644 index 0000000000..4e8aa40959 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDCYXb6OSAZyXRfLXOtMo43za197Hdc/T0YKjgQQjwDt6rlRwqTh7v7S +PV2kXwNGdWigBwYFK4EEACKhZANiAARN2khlJUOOIiwsWHEALwDieeZR96qL4pUd +ci7aeGaczdUK5jOA9D9zmBZtSYTfO8Cr7ekVghDlcWAIJ/BXcswgQwSEQ6wyfaTF +8FYfyr4l3u9IirsnyaFzeIgeoNis8Gw= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa384.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa384.pub new file mode 100644 index 0000000000..41e722e545 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE3aSGUlQ44iLCxYcQAvAOJ55lH3qovilR1yLtp4ZpzN1QrmM4D0P3OYFm1JhN87wKvt6RWCEOVxYAgn8FdyzCBDBIRDrDJ9pMXwVh/KviXe70iKuyfJoXN4iB6g2KzwbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa521 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa521 new file mode 100644 index 0000000000..7196f46e97 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHbAgEBBEFMadoz4ckEcClfqXa2tiUuYkJdDfwq+/iFQcpt8ESuEd26IY/vm47Q +9UzbPkO4ou8xkNsQ3WvCRQBBWtn5O2kUU6AHBgUrgQQAI6GBiQOBhgAEAde5BRu5 +01/jS0jRk212xsb2DxPrxNpgp6IMCV8TA4Eps+8bSqHB091nLiBcP422HXYfuCd7 +XDjSs8ihcmhp0hCRASLqZR9EzW9W/SOt876May1Huj5X+WSO6RLe7vPn9vmf7kHf +pip6m7M7qp2qGgQ3q2vRwS2K/O6156ohiOlmuuFs +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa521.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa521.pub new file mode 100644 index 0000000000..8f059120bc --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ecdsa521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHXuQUbudNf40tI0ZNtdsbG9g8T68TaYKeiDAlfEwOBKbPvG0qhwdPdZy4gXD+Nth12H7gne1w40rPIoXJoadIQkQEi6mUfRM1vVv0jrfO+jGstR7o+V/lkjukS3u7z5/b5n+5B36YqepuzO6qdqhoEN6tr0cEtivzuteeqIYjpZrrhbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed25519 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed25519 new file mode 100644 index 0000000000..401a3e4a9a --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed25519 @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnwAAAJg3+6xpN/us +aQAAAAtzc2gtZWQyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnw +AAAEBzC/Z2WGJhZ3l3tIBnUc6DCbp+lXY2yc2RRpWQTdf8sub0/z+ALQg4qbActK+SYS1L +ZfHRFGpQOoLrBjpSANWfAAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed25519.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed25519.pub new file mode 100644 index 0000000000..a5c03b19c1 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed25519.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOb0/z+ALQg4qbActK+SYS1LZfHRFGpQOoLrBjpSANWf uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed448 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed448 new file mode 100644 index 0000000000..8ecfd710dc --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed448 @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA53OqeePNaG/NJmoMbELhskKrAHNhLZ6AQm1WjbpMoseNl/OFh +1xznExpUPqTLX36fHYsAaWRHABQAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtf +fp8diwBpZEcAFAAAAAByzSPST3FCdOdENDI3uTKQ9RH2Ql+Y5kRZ/yA+iYUIP/32 +BQBVOrwOBc0CGEvbicTM1n4YeVEmfrMo3OqeePNaG/NJmoMbELhskKrAHNhLZ6AQ +m1WjbpMoseNl/OFh1xznExpUPqTLX36fHYsAaWRHABQAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed448.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed448.pub new file mode 100644 index 0000000000..cec0765a5d --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_ed448.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtffp8diwBpZEcAFAA= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa index 9d7e0dd5fb..2202c2ead8 100644 --- a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa @@ -1,15 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIICXAIBAAKBgQD1OET+3O/Bvj/dtjxDTXmj1oiJt4sIph5kGy0RfjoPrZfaS+CU -DhakCmS6t2ivxWFgtpKWaoGMZMJqWj6F6ZsumyFl3FPBtujwY/35cgifrI9Ns4Tl -zR1uuengNBmV+WRQ5cd9F2qS6Z8aDQihzt0r8JUqLcK+VQbrmNzboCCQQwIDAQAB -AoGAPQEyqPTt8JUT7mRXuaacjFXiweAXhp9NEDpyi9eLOjtFe9lElZCrsUOkq47V -TGUeRKEm9qSodfTbKPoqc8YaBJGJPhUaTAcha+7QcDdfHBvIsgxvU7ePVnlpXRp3 -CCUEMPhlnx6xBoTYP+fRU0e3+xJIPVyVCqX1jAdUMkzfRoECQQD6ux7B1QJAIWyK -SGkbDUbBilNmzCFNgIpOP6PA+bwfi5d16diTpra5AX09keQABAo/KaP1PdV8Vg0p -z4P3A7G3AkEA+l+AKG6m0kQTTBMJDqOdVPYwe+5GxunMaqmhokpEbuGsrZBl5Dvd -WpcBjR7jmenrhKZRIuA+Fz5HPo/UQJPl1QJBAKxstDkeED8j/S2XoFhPKAJ+6t39 -sUVICVTIZQeXdmzHJXCcUSkw8+WEhakqw/3SyW0oaK2FSWQJFWJUZ+8eJj8CQEh3 -xeduB5kKnS9CvzdeghZqX6QvVosSdtlUmfUYW/BgH5PpHKTP8wTaeld3XldZTpMJ -dKiMkUw2+XYROVUrubUCQD+Na1LhULlpn4ISEtIEfqpdlUhxDgO15Wg8USmsng+x -ICliVOSQtwaZjm8kwaFt0W7XnpnDxbRs37vIEbIMWak= +MIIEpAIBAAKCAQEAztjiyj2tdfkji0fewWS0kABg0IABgG20NvL1PnHJLr98we7w +W7f3j27EGjW/ApuycsWXXKi0L82q8uDicoHHb3JI2JkT70oi0yG1Dx/zwPN+dkA7 +LBT1J3UK2hJTFPhp855CwY/ss9xpBsd1Fv3zuHifEqNGljeg1PjmQ3pNhxA/M0aZ +cLnfIUdZ5Hr+t+4es3zaWo4tLBKmZu6BkVGQKPGXeMkIAMtJlG24l7qKDRkR5TYA +ZT7P8Vn7hnuFuCNbrJSm686GawBxTQXom23dg9UcWxoHB7UiHFoR6j0bQAX+4R7b +IwculRDcvzrgCu6u06oFILwY7MlsxpX9hGTl1wIDAQABAoIBAFeP6pmQeICrYceR +OhQGLIWVE2bP+VLDnflw6i5v/qlieE6kdm1tOEgorK0nuV9CR81cJdIcvIJL/yTn +3BR7KdDcwUenrY+rg4h7CWmIrigtK4ilciccDBeS7XAZN8B11GxDv6Cu65XMJU2w +W7nK8URTE4vRQI1QqS3e26MPAAi/LVOt3ZPI6zg/GHEwnq0IVSQAOndLBr/IWZk5 +SANrkfwX8WS7/UxZgDptT9dyUQ5Pnj5mieTlIvBwyczdhZ7RDa8HdCSHW3xF83V1 +A0pkn6+TRojumYyr4RrPQj6htE64Hgx9w1Dv/UINjPXl5mGlbxQHMWGzlqD/qpyI +wg7RakECgYEA+9ARZpHfEFz+EEFi8l9J+BtJDo00WaKCOZHh5UJ8W+NreqSd8nSx +5u6wYwMJjRX2Hwv+FBEhxGbo1+ff6p++cYmiSlDtN2XRCDkBWvvGlxu55BDULrhx +f8lqaV3XGmOy2rQusp8hiHmkmPJCSVj3oJqQnbqJ2zahXAx1rTPwHqECgYEA0kln +4h+ZkZ+aldOMGF0d0txTcTqZvsSVKiFTSD9of/fiSDqb6xtLT2+ys6FZoFL9lyK8 +gtqH642CDQ+3WT6Nmn4kMF5HNVpEuCeRDeRhiquWeKaAQDyvZ5ym1+Cn3GhsO7Di +d2LJKV5hOoN77loVY5nwnUVIJ0h+WLf0T7DTCXcCgYEAiNT7X50MdTvS4splFgcp +jqRlAn9AXySrVtUqxwVlxhjCIpapLUK0GSTCvEq+OeghIaXGnujgTHUPOaNKTZgY +SGHdyjxHar7s42b2kZYWx63NSVzLr8eSBTpRlIflhvV+DtGyPmWyNxLCmkmqM2kg +xii3RL5EgtYgwIAUwdVjOYECgYBRPlsMWfkS8f7fc+PkZdVn6gey71kHAxw+MrHi +b90H09Vw4nPq2Zi3EAiSrfvanTWsdpcuVw+8See89B16NViwH5wLs+D/E+kI3QCF +xX6J/NEdu/ZA2zFJbpRnQzyXQyDNzwEv7tKZUQVvfe0boWIyIP99Q48k3jUyQZ/6 +Se6+8QKBgQCXl8H2K3CsZxoujKLb2qoEOPbxJQ2hxoMTS5XuQECReIVsNuptWrur +DF8WJi/B6AqwRX1P3l56RNwqB1yDBqv0QVLpU7vU/FmWqLWTn0r3AvM74qftvfAE +oa31wcYoCqPJoKgCG7TThLhNt2v5hL7sVgZNO0ueAiHhJbFLaf7ceg== -----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa.pub new file mode 100644 index 0000000000..b4084d320a --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO2OLKPa11+SOLR97BZLSQAGDQgAGAbbQ28vU+cckuv3zB7vBbt/ePbsQaNb8Cm7JyxZdcqLQvzary4OJygcdvckjYmRPvSiLTIbUPH/PA8352QDssFPUndQraElMU+GnznkLBj+yz3GkGx3UW/fO4eJ8So0aWN6DU+OZDek2HED8zRplwud8hR1nkev637h6zfNpaji0sEqZm7oGRUZAo8Zd4yQgAy0mUbbiXuooNGRHlNgBlPs/xWfuGe4W4I1uslKbrzoZrAHFNBeibbd2D1RxbGgcHtSIcWhHqPRtABf7hHtsjBy6VENy/OuAK7q7TqgUgvBjsyWzGlf2EZOXX uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key256 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key256 new file mode 100644 index 0000000000..2979ea88ed --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMe4MDoit0t8RzSVPwkCBemQ9fhXL+xnTSAWISw8HNCioAoGCCqGSM49 +AwEHoUQDQgAEo2q7U3P6r0W5WGOLtM78UQtofM9UalEhiZeDdiyylsR/RR17Op0s +VPGSADLmzzgcucLEKy17j2S+oz42VUJy5A== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key256.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key256.pub new file mode 100644 index 0000000000..85dc419345 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNqu1Nz+q9FuVhji7TO/FELaHzPVGpRIYmXg3YsspbEf0UdezqdLFTxkgAy5s84HLnCxCste49kvqM+NlVCcuQ= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key384 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key384 new file mode 100644 index 0000000000..fb1a862ded --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDArxbDfh3p1okrD9wQw6jJ4d4DdlBPD5GqXE8bIeRJiK41Sh40LgvPw +mkqEDSXK++CgBwYFK4EEACKhZANiAAScl43Ih2lWTDKrSox5ve5uiTXil4smsup3 +CfS1XPjKxgBAmlfBim8izbdrT0BFdQzz2joduNMtpt61wO4rGs6jm0UP7Kim9PC7 +Hneb/99fIYopdMH5NMnk60zGO1uZ2vc= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key384.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key384.pub new file mode 100644 index 0000000000..428d5fb7d7 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJyXjciHaVZMMqtKjHm97m6JNeKXiyay6ncJ9LVc+MrGAECaV8GKbyLNt2tPQEV1DPPaOh240y2m3rXA7isazqObRQ/sqKb08Lsed5v/318hiil0wfk0yeTrTMY7W5na9w== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key521 b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key521 new file mode 100644 index 0000000000..3e51ec2ecd --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHcAgEBBEIB8O1BFkl2HQjQLRLonEZ97da/h39DMa9/0/hvPZWAI8gUPEQcHxRx +U7b09p3Zh+EBbMFq8+1ae9ds+ZTxE4WFSvKgBwYFK4EEACOhgYkDgYYABAAlWVjq +Bzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/ +vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5 +ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key521.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key521.pub new file mode 100644 index 0000000000..017a29f4da --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ecdsa_key521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAAlWVjqBzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed25519_key b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed25519_key new file mode 100644 index 0000000000..13a8fcf491 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed25519_key @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQAAAJi+h4O7voeD +uwAAAAtzc2gtZWQyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQ +AAAEBaOcJfGPNemKc1wPHTCmM4Kwvh6dZ0CqY14UT361UnN0lI66JgZZo72XJ7wGBp+h3W +TDw/pxXddvaomAIHrIl9AAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed25519_key.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed25519_key.pub new file mode 100644 index 0000000000..156ef4045c --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed25519_key.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIElI66JgZZo72XJ7wGBp+h3WTDw/pxXddvaomAIHrIl9 uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed448_key b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed448_key new file mode 100644 index 0000000000..31a7e4e8c3 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed448_key @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed448_key.pub b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed448_key.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/property_test/ssh_eqc_client_server_dirs/ssh_host_ed448_key.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_agent_SUITE.erl b/lib/ssh/test/ssh_agent_SUITE.erl index 7efd1c6d88..836a61d389 100644 --- a/lib/ssh/test/ssh_agent_SUITE.erl +++ b/lib/ssh/test/ssh_agent_SUITE.erl @@ -48,6 +48,22 @@ init_per_suite(Config) -> end_per_suite(_Config) -> ok = ssh:stop(). +init_per_testcase(connect_with_ssh_agent, Config) -> + DataDir = proplists:get_value(data_dir, Config), + UserDir = proplists:get_value(priv_dir, Config), + AgentUserDir = filename:join(UserDir,"agent"), % just to separate them in the tests + % so we know that the right dir is used + file:make_dir(AgentUserDir), + %% Arrange the host keys in <priv_dir>/system + ct:log("Host keys setup for: ~p", + [ssh_test_lib:setup_all_host_keys(Config)]), + %% Copy the user's files used by the daemon + {ok,_} = file:copy(filename:join(DataDir,"authorized_keys"), + filename:join(UserDir,"authorized_keys")), + %% And copy the user's files used by the agent (and not by the user) + {ok,_} = file:copy(filename:join(DataDir,"id_rsa"), + filename:join(AgentUserDir,"id_rsa")), + Config; init_per_testcase(_TestCase, Config) -> Config. @@ -119,16 +135,18 @@ connect_with_ssh_agent() -> [{doc, "Connect with RSA key from SSH agent"}]. connect_with_ssh_agent(Config) -> - DataDir = proplists:get_value(data_dir, Config), - {ok, SocketPath} = ssh_agent_mock_server:start_link('rsa-sha2-256', DataDir), - {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, DataDir}, - {user_dir, DataDir}]), - ConnectionRef = ssh_test_lib:connect(Host, Port, [{user_dir, DataDir}, - {silently_accept_hosts, true}, - {user_interaction, false}, - {auth_methods, "publickey"}, - {key_cb, {ssh_agent, [{socket_path, SocketPath}]}} - ]), + UserDir = PrivDir = proplists:get_value(priv_dir, Config), + AgentUserDir = filename:join(UserDir,"agent"), + SystemDir = filename:join(PrivDir, "system"), + {ok, SocketPath} = ssh_agent_mock_server:start_link('rsa-sha2-256', AgentUserDir), + {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, + {user_dir, UserDir}]), + ConnectionRef = ssh_test_lib:connect(Host, Port, [{user_dir, UserDir}, + {silently_accept_hosts, true}, + {user_interaction, false}, + {auth_methods, "publickey"}, + {key_cb, {ssh_agent, [{socket_path, SocketPath}]}} + ]), ssh:close(ConnectionRef), ssh:stop_daemon(Pid), ssh_agent_mock_server:stop(SocketPath). diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_dsa_key b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_dsa_key new file mode 100644 index 0000000000..51ab6fbd88 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_dsa_key @@ -0,0 +1,13 @@ +-----BEGIN DSA PRIVATE KEY----- +MIIBuwIBAAKBgQCClaHzE2ul0gKSUxah5W0W8UiJLy4hXngKEqpaUq9SSdVdY2LK +wVfKH1gt5iuaf1FfzOhsIC9G/GLnjYttXZc92cv/Gfe3gR+s0ni2++MX+T++mE/Q +diltXv/Hp27PybS67SmiFW7I+RWnT2OKlMPtw2oUuKeztCe5UWjaj/y5FQIVAPLA +l9RpiU30Z87NRAHY3NTRaqtrAoGANMRxw8UfdtNVR0CrQj3AgPaXOGE4d+G4Gp4X +skvnCHycSVAjtYxebUkzUzt5Q6f/IabuLUdge3gXrc8BetvrcKbp+XZgM0/Vj2CF +Ymmy3in6kzGZq7Fw1sZaku6AOU8vLa5woBT2vAcHLLT1bLAzj7viL048T6MfjrOP +ef8nHvACgYBhDWFQJ1mf99sg92LalVq1dHLmVXb3PTJDfCO/Gz5NFmj9EZbAtdah +/XcF3DeRF+eEoz48wQF/ExVxSMIhLdL+o+ElpVhlM7Yii+T7dPhkQfEul6zZXu+U +ykSTXYUbtsfTNRFQGBW2/GfnEc0mnIxfn9v10NEWMzlq5z9wT9P0CgIVAN4wtL5W +Lv62jKcdskxNyz2NQoBx +-----END DSA PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_dsa_key.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_dsa_key.pub new file mode 100644 index 0000000000..4dbb1305b0 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_dsa_key.pub @@ -0,0 +1,11 @@ +---- BEGIN SSH2 PUBLIC KEY ---- +AAAAB3NzaC1kc3MAAACBAIKVofMTa6XSApJTFqHlbRbxSIkvLiFeeAoSqlpSr1JJ1V1j +YsrBV8ofWC3mK5p/UV/M6GwgL0b8YueNi21dlz3Zy/8Z97eBH6zSeLb74xf5P76YT9B2 +KW1e/8enbs/JtLrtKaIVbsj5FadPY4qUw+3DahS4p7O0J7lRaNqP/LkVAAAAFQDywJfU +aYlN9GfOzUQB2NzU0WqrawAAAIA0xHHDxR9201VHQKtCPcCA9pc4YTh34bganheyS+cI +fJxJUCO1jF5tSTNTO3lDp/8hpu4tR2B7eBetzwF62+twpun5dmAzT9WPYIViabLeKfqT +MZmrsXDWxlqS7oA5Ty8trnCgFPa8BwcstPVssDOPu+IvTjxPox+Os495/yce8AAAAIBh +DWFQJ1mf99sg92LalVq1dHLmVXb3PTJDfCO/Gz5NFmj9EZbAtdah/XcF3DeRF+eEoz48 +wQF/ExVxSMIhLdL+o+ElpVhlM7Yii+T7dPhkQfEul6zZXu+UykSTXYUbtsfTNRFQGBW2 +/GfnEc0mnIxfn9v10NEWMzlq5z9wT9P0Cg== +---- END SSH2 PUBLIC KEY ---- diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key256 b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key256 new file mode 100644 index 0000000000..2979ea88ed --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMe4MDoit0t8RzSVPwkCBemQ9fhXL+xnTSAWISw8HNCioAoGCCqGSM49 +AwEHoUQDQgAEo2q7U3P6r0W5WGOLtM78UQtofM9UalEhiZeDdiyylsR/RR17Op0s +VPGSADLmzzgcucLEKy17j2S+oz42VUJy5A== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key256.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key256.pub new file mode 100644 index 0000000000..85dc419345 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNqu1Nz+q9FuVhji7TO/FELaHzPVGpRIYmXg3YsspbEf0UdezqdLFTxkgAy5s84HLnCxCste49kvqM+NlVCcuQ= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key384 b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key384 new file mode 100644 index 0000000000..fb1a862ded --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDArxbDfh3p1okrD9wQw6jJ4d4DdlBPD5GqXE8bIeRJiK41Sh40LgvPw +mkqEDSXK++CgBwYFK4EEACKhZANiAAScl43Ih2lWTDKrSox5ve5uiTXil4smsup3 +CfS1XPjKxgBAmlfBim8izbdrT0BFdQzz2joduNMtpt61wO4rGs6jm0UP7Kim9PC7 +Hneb/99fIYopdMH5NMnk60zGO1uZ2vc= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key384.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key384.pub new file mode 100644 index 0000000000..428d5fb7d7 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJyXjciHaVZMMqtKjHm97m6JNeKXiyay6ncJ9LVc+MrGAECaV8GKbyLNt2tPQEV1DPPaOh240y2m3rXA7isazqObRQ/sqKb08Lsed5v/318hiil0wfk0yeTrTMY7W5na9w== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key521 b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key521 new file mode 100644 index 0000000000..3e51ec2ecd --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHcAgEBBEIB8O1BFkl2HQjQLRLonEZ97da/h39DMa9/0/hvPZWAI8gUPEQcHxRx +U7b09p3Zh+EBbMFq8+1ae9ds+ZTxE4WFSvKgBwYFK4EEACOhgYkDgYYABAAlWVjq +Bzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/ +vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5 +ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key521.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key521.pub new file mode 100644 index 0000000000..017a29f4da --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ecdsa_key521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAAlWVjqBzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed25519_key b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed25519_key new file mode 100644 index 0000000000..13a8fcf491 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed25519_key @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQAAAJi+h4O7voeD +uwAAAAtzc2gtZWQyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQ +AAAEBaOcJfGPNemKc1wPHTCmM4Kwvh6dZ0CqY14UT361UnN0lI66JgZZo72XJ7wGBp+h3W +TDw/pxXddvaomAIHrIl9AAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed25519_key.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed25519_key.pub new file mode 100644 index 0000000000..156ef4045c --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed25519_key.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIElI66JgZZo72XJ7wGBp+h3WTDw/pxXddvaomAIHrIl9 uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed448_key b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed448_key new file mode 100644 index 0000000000..31a7e4e8c3 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed448_key @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed448_key.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed448_key.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_ed448_key.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_rsa_key.pub b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_rsa_key.pub new file mode 100644 index 0000000000..75d2025c71 --- /dev/null +++ b/lib/ssh/test/ssh_agent_SUITE_data/ssh_host_rsa_key.pub @@ -0,0 +1,5 @@ +---- BEGIN SSH2 PUBLIC KEY ---- +AAAAB3NzaC1yc2EAAAADAQABAAAAgQDCZX+4FBDwZIh9y/Uxee1VJnEXlowpz2yDKwj8 +semM4q843337zbNfxHmladB1lpz2NqyxI175xMIJuDxogyZdsOxGnFAzAnthR4dqL/RW +RWzjaxSB6IAO9SPYVVlrpZ+1hsjLW79fwXK/yc8VdhRuWTeQiRgYY2ek8+OKbOqz4Q== +---- END SSH2 PUBLIC KEY ---- diff --git a/lib/ssh/test/ssh_algorithms_SUITE.erl b/lib/ssh/test/ssh_algorithms_SUITE.erl index 06996d7e48..4832c0ad3b 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE.erl +++ b/lib/ssh/test/ssh_algorithms_SUITE.erl @@ -35,7 +35,7 @@ suite() -> [{ct_hooks,[ts_install_cth]}, - {timetrap,{seconds,60}}]. + {timetrap,{seconds,120}}]. all() -> %% [{group,kex},{group,cipher}... etc @@ -48,9 +48,9 @@ groups() -> SshdAlgos = extract_algos(ssh_test_lib:default_algorithms(sshd)), DoubleAlgos = - [{Tag, double(Algs)} || {Tag,Algs} <- ErlAlgos, - length(Algs) > 1, - lists:member(Tag, two_way_tags())], + [{Tag, double(Tag,Algs)} || {Tag,Algs} <- ErlAlgos, + length(Algs) > 1, + lists:member(Tag, two_way_tags())], TagGroupSet = [{Tag, [], group_members_for_tag(Tag,Algs,DoubleAlgos)} || {Tag,Algs} <- ErlAlgos, @@ -60,14 +60,14 @@ groups() -> TypeSSH = ssh_test_lib:ssh_type(), AlgoTcSet = - [{Alg, [parallel], specific_test_cases(Tag,Alg,SshcAlgos,SshdAlgos,TypeSSH)} + [{Alg, [], specific_test_cases(Tag,Alg,SshcAlgos,SshdAlgos,TypeSSH)} || {Tag,Algs} <- ErlAlgos ++ DoubleAlgos, Alg <- Algs], TagGroupSet ++ AlgoTcSet. tags() -> [kex,cipher,mac,compression,public_key]. -two_way_tags() -> [cipher,mac,compression]. +two_way_tags() -> [cipher,mac,compression, public_key]. %%-------------------------------------------------------------------- init_per_suite(Config) -> @@ -91,7 +91,10 @@ init_per_suite(Config) -> ssh_test_lib:installed_ssh_version("TIMEOUT"), ssh:default_algorithms(), crypto:info_lib(), - ssh_test_lib:default_algorithms(sshc), + ssh_test_lib:default_algorithms(sshc, + %% Use a fake system_dir to enable the test + %% daemon to start: + [{system_dir,proplists:get_value(data_dir,Config)}]), ssh_test_lib:default_algorithms(sshd), {?DEFAULT_DH_GROUP_MIN,?DEFAULT_DH_GROUP_NBITS,?DEFAULT_DH_GROUP_MAX}, public_key:dh_gex_group_sizes(), @@ -126,22 +129,31 @@ init_per_group(Group, Config) -> init_per_group(public_key=Tag, Alg, Config) -> + PA = + case split(Tag, Alg) of + [_] -> + [Alg]; + [A1,A2] -> + [A1,A2] + end, OtherAlgs = [{T,L} || {T,L} <- ssh_transport:supported_algorithms(), T=/=Tag], - ct:log("Init tests for public_key ~p~nOtherAlgs=~p",[Alg,OtherAlgs]), - PrefAlgs = {preferred_algorithms,[{Tag,[Alg]}|OtherAlgs]}, + ct:log("Init tests for public_key ~p~nOtherAlgs=~p",[PA,OtherAlgs]), + PrefAlgs = {preferred_algorithms,[{Tag,PA}|OtherAlgs]}, %% Daemon started later in init_per_testcase try - setup_pubkey(Alg, + setup_pubkey(PA, [{pref_algs,PrefAlgs}, - {tag_alg,{Tag,Alg}} + {tag_alg,{Tag,PA}} | Config]) catch - _:_ -> {skip, io_lib:format("Unsupported: ~p",[Alg])} + _C:_E:_S -> + ct:log("Exception ~p:~p~n~p",[_C,_E,_S]), + {skip, io_lib:format("Unsupported: ~p",[Alg])} end; init_per_group(Tag, Alg, Config) -> PA = - case split(Alg) of + case split(Tag, Alg) of [_] -> [Alg]; [A1,A2] -> @@ -153,7 +165,7 @@ init_per_group(Tag, Alg, Config) -> PrefAlgs = {preferred_algorithms,[{Tag,PA}|OtherAlgs]}, start_std_daemon([PrefAlgs], [{pref_algs,PrefAlgs}, - {tag_alg,{Tag,Alg}} + {tag_alg,{Tag,[Alg]}} | Config]). @@ -173,6 +185,7 @@ init_per_testcase(TC, Config) -> init_per_testcase(TC, {public_key,Alg}, Config) -> + ct:log("init_per_testcase TC=~p, Alg=~p",[TC,Alg]), ExtraOpts = case TC of simple_connect -> [{user_dir, proplists:get_value(priv_dir,Config)}]; @@ -180,21 +193,26 @@ init_per_testcase(TC, {public_key,Alg}, Config) -> [] end, Opts = pubkey_opts(Config) ++ ExtraOpts, - case {ssh_file:user_key(Alg,Opts), ssh_file:host_key(Alg,Opts)} of + {UserAlg,SrvrAlg} = + case Alg of + [A1,A2] -> {A1,A2}; + [A0] -> {A0,A0} + end, + case {ssh_file:user_key(UserAlg,Opts), ssh_file:host_key(SrvrAlg,Opts)} of {{ok,_}, {ok,_}} -> start_pubkey_daemon([proplists:get_value(pref_algs,Config) | ExtraOpts], [{extra_daemon,true}|Config]); {{ok,_}, {error,Err}} -> - ct:log("Alg = ~p~nOpts = ~p",[Alg,Opts]), + ct:log("SrvrAlg = ~p~nOpts = ~p",[SrvrAlg,Opts]), {skip, io_lib:format("No host key: ~p",[Err])}; {{error,Err}, {ok,_}} -> - ct:log("Alg = ~p~nOpts = ~p",[Alg,Opts]), + ct:log("UserAlg = ~p~nOpts = ~p",[UserAlg,Opts]), {skip, io_lib:format("No user key: ~p",[Err])}; _ -> - ct:log("Alg = ~p~nOpts = ~p",[Alg,Opts]), + ct:log("UserAlg = ~p SrvrAlg = ~p~nOpts = ~p",[UserAlg,SrvrAlg,Opts]), {skip, "Neither host nor user key"} end; @@ -244,17 +262,22 @@ simple_exec(Config) -> %%-------------------------------------------------------------------- %% A simple exec call simple_connect(Config) -> + ct:log("PrivDir ~p:~n~p~n~nPrivDir/system: ~p",[proplists:get_value(priv_dir,Config), + file:list_dir(proplists:get_value(priv_dir,Config)), + catch file:list_dir( + filename:join(proplists:get_value(priv_dir,Config), + system))]), {Host,Port} = proplists:get_value(srvr_addr, Config), {preferred_algorithms,AlgEntries} = proplists:get_value(pref_algs, Config), Opts = case proplists:get_value(tag_alg, Config) of - {public_key,Alg} -> [{pref_public_key_algs,[Alg]}, + {public_key,Alg} -> [{pref_public_key_algs,Alg}, {preferred_algorithms,AlgEntries}]; _ -> [{modify_algorithms,[{append,AlgEntries}]}] end, ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{silently_accept_hosts, true}, - {user_interaction, false} | + {user_interaction, false} | Opts]), ct:log("~p:~p connected! ~p",[?MODULE,?LINE,ConnectionRef]), ssh:close(ConnectionRef). @@ -280,7 +303,7 @@ try_exec_simple_group(Group, Config) -> %% Testing all default groups simple_exec_groups() -> - [{timetrap,{seconds,180}}]. + [{timetrap,{seconds,240}}]. simple_exec_groups(Config) -> Sizes = interpolate( public_key:dh_gex_group_sizes() ), @@ -317,7 +340,10 @@ sshc_simple_exec_os_cmd(Config) -> Result = ssh_test_lib:open_sshc(Host, Port, [" -C" " -o UserKnownHostsFile=",KnownHosts, + " -o CheckHostIP=no" " -o StrictHostKeyChecking=no" + " -q" + " -x" ], " 1+1."), Parent ! {result, self(), Result, "2"} @@ -342,7 +368,7 @@ sshc_simple_exec_os_cmd(Config) -> sshd_simple_exec(Config) -> ClientPubKeyOpts = case proplists:get_value(tag_alg,Config) of - {public_key,Alg} -> [{pref_public_key_algs,[Alg]}]; + {public_key,Alg} -> [{pref_public_key_algs,Alg}]; _ -> [] end, ConnectionRef = ssh_test_lib:connect(22, [{silently_accept_hosts, true}, @@ -395,13 +421,20 @@ sshd_simple_exec(Config) -> group_members_for_tag(Tag, Algos, DoubleAlgos) -> [{group,Alg} || Alg <- Algos++proplists:get_value(Tag,DoubleAlgos,[])]. -double(Algs) -> [concat(A1,A2) || A1 <- Algs, - A2 <- Algs, - A1 =/= A2]. +double(Tag, Algs) -> [concat(Tag,A1,A2) || A1 <- Algs, + A2 <- Algs, + A1 =/= A2]. -concat(A1, A2) -> list_to_atom(lists:concat([A1," + ",A2])). +concat(Tag, A1, A2) -> + list_to_atom(lists:concat(["D: ",Tag," ",A1," + ",A2])). -split(Alg) -> ssh_test_lib:to_atoms(string:tokens(atom_to_list(Alg), " + ")). +split(TagA, Alg) -> + Tag = atom_to_list(TagA), + ssh_test_lib:to_atoms( + case string:tokens(atom_to_list(Alg), " ") of + ["D:",Tag,A1,"+",A2] ->[A1,A2]; + Other -> Other + end). specific_test_cases(Tag, Alg, SshcAlgos, SshdAlgos, TypeSSH) -> case Tag of @@ -435,7 +468,7 @@ supports(Tag, Alg, Algos) -> lists:all(fun(A) -> lists:member(A, proplists:get_value(Tag, Algos,[])) end, - split(Alg)). + split(Tag, Alg)). extract_algos(Spec) -> @@ -475,21 +508,33 @@ pubkey_opts(Config) -> {system_dir, SystemDir}]. -setup_pubkey(Alg, Config) -> +setup_pubkey([AlgClient, AlgServer], Config) -> DataDir = proplists:get_value(data_dir, Config), UserDir = proplists:get_value(priv_dir, Config), + ssh_test_lib:del_dir_contents(UserDir), + ok = ssh_test_lib:setup_user_key(AlgClient, DataDir, UserDir), + _SysDir = ssh_test_lib:setup_host_key_create_dir(AlgServer, DataDir, UserDir), +try ct:log("~p:~p AlgClient=~p, AlgServer=~p~nPrivDir ~p:~n~p~n~nSYsDir=~p~nPrivDir/system: ~p", + [?MODULE,?LINE, + AlgClient, AlgServer, + proplists:get_value(priv_dir,Config), + file:list_dir(proplists:get_value(priv_dir,Config)), + _SysDir, + catch file:list_dir( + filename:join(proplists:get_value(priv_dir,Config), + system)) + ]) +catch _C:_E:_S -> + ct:log("~p:~p ~p:~p~n~p",[?MODULE,?LINE,_C,_E,_S]) +end, + Config; + +setup_pubkey([Alg], Config) -> + DataDir = proplists:get_value(data_dir, Config), + PrivDir = proplists:get_value(priv_dir, Config), ct:log("Setup keys for ~p",[Alg]), - case Alg of - 'ssh-dss' -> ssh_test_lib:setup_dsa(DataDir, UserDir); - 'ssh-rsa' -> ssh_test_lib:setup_rsa(DataDir, UserDir); - 'rsa-sha2-256' -> ssh_test_lib:setup_rsa(DataDir, UserDir); - 'rsa-sha2-512' -> ssh_test_lib:setup_rsa(DataDir, UserDir); - 'ecdsa-sha2-nistp256' -> ssh_test_lib:setup_ecdsa("256", DataDir, UserDir); - 'ecdsa-sha2-nistp384' -> ssh_test_lib:setup_ecdsa("384", DataDir, UserDir); - 'ecdsa-sha2-nistp521' -> ssh_test_lib:setup_ecdsa("521", DataDir, UserDir); - 'ssh-ed25519' -> ssh_test_lib:setup_eddsa(ed25519, DataDir, UserDir); - 'ssh-ed448' -> ssh_test_lib:setup_eddsa(ed448, DataDir, UserDir) - end, + ssh_test_lib:setup_user_key(Alg, DataDir, PrivDir), + ssh_test_lib:setup_host_key_create_dir(Alg, DataDir, PrivDir), Config. diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa b/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa index 273866b2a6..24628e071b 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa @@ -1,12 +1,12 @@ -----BEGIN DSA PRIVATE KEY----- -MIIBvAIBAAKBgQDIaRzrxz7bSJXh6z+w1lfTW7sfYNMDNXsfN/nvt6Rqi95K4Q+8 -Xpa+LJ6oZCxfMto8w2fZ6uzikRCgBbvzos6L+GPKWhuaBTT5gYsEFphowWN9uPQy -BHwRApOkpZNObN+SnMRuO7lYpwMLnYJdLJTCOxam3mfAzTqdCnpgAWAidwIVANFI -VPGuBxMc84proP+r8X+8hT8RAoGBAI9A+KYH0LLFFIaxgDBA7xnnHIECQrhgiNIN -QfvFPH1Drf5He50OeXFEvR4gt7f3qScWxLUKrpBysS1huJHiJCOq+iREyfoqjtfc -zZT01XejDvX5ck+rz/kiWQk9cZwW6lqLtn3GxIcQvOnrzskT7YyFJ7fx3U98PxmY -qZrXhJEjAoGABFZL6Ztdx9A+7qd5s2A4T7W9fztsm85CxM4MuRuzR2U6zU/Z4D6U -OXiS9UZsz7yIQQfqrc5IgPmSa5og9oflQnQSQcRrY1XvpYLft1DOkH6+wdFf0OlG -hIKihHN+jvgbcEmJTHJwsiW2HJCH6qXK33s4lmthEeIqXuZPduaBnGICFQDKLb+y -afjRTP7FVHq+SBQUy8iqSQ== +MIIBuwIBAAKBgQDIywHurUpOq6kZuMn+XlRzR4hAxF6qwSkuEqkV7iHnLQ0kIwf3 +uAmjFDhuEsQ8653SLxGVvTNp+KFFgDXiLqgM7TPUwDnpbvzEZHPAU+/zPt4sdY2D +txBfJwT2SFlK6HPOxOcxdDuD+/a59sh8hk/YVOU7ZTcBVsVG8Got4UcF5QIVAPGd +CPDQKSTlPiM9OwBB1+9p11k5AoGARLxw4l17mET9cU0uf4Ppe5nsCbODJv44ZrSs +picvypGVLrLcN5KWbm3vjRFCQ5LFunAG3FwLC2Sh0CH6TemoIfRPsRHR7wvpBGdr +c693UlMOis/mcmvNMQAzuQNW9WrxdzsvWR/r5s6NEHWqKUJGXSPi2d+Ijq/mCOmI +hzLzyiACgYEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4 +cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+C +ROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNgCFEjA7wTC +sQCY/I35vb6GUJn9tEdP -----END DSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa.pub b/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa.pub new file mode 100644 index 0000000000..018ef6f537 --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_dsa.pub @@ -0,0 +1 @@ +ssh-dss AAAAB3NzaC1kc3MAAACBAMjLAe6tSk6rqRm4yf5eVHNHiEDEXqrBKS4SqRXuIectDSQjB/e4CaMUOG4SxDzrndIvEZW9M2n4oUWANeIuqAztM9TAOelu/MRkc8BT7/M+3ix1jYO3EF8nBPZIWUroc87E5zF0O4P79rn2yHyGT9hU5TtlNwFWxUbwai3hRwXlAAAAFQDxnQjw0Ckk5T4jPTsAQdfvaddZOQAAAIBEvHDiXXuYRP1xTS5/g+l7mewJs4Mm/jhmtKymJy/KkZUustw3kpZube+NEUJDksW6cAbcXAsLZKHQIfpN6agh9E+xEdHvC+kEZ2tzr3dSUw6Kz+Zya80xADO5A1b1avF3Oy9ZH+vmzo0QdaopQkZdI+LZ34iOr+YI6YiHMvPKIAAAAIEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+CROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNg= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa b/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa index 1f8577e866..2202c2ead8 100644 --- a/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa @@ -1,27 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIIEpAIBAAKCAQEAwT3nTd9tkKLVaqVcSpK04ID1VVCSCEOW0KXLienX4LD/q7lo -aOyuc044hDnv3P+nz2SNSR05N5dAFaKjCKw7DWz30HgrSbAFxW+KUHAG55brZMOF -oevJOlag8ih1AXjgpEcsJtGILshXyJKXA02IkeH62xeb/VUFgw1iRbbLblhcf2Mb -lyiTcWQZRei57cAA+IjjrxjGJxvG2eLyH1zy+BJa/1Lo4Kfk9tN0rfH1RBzJ5hjK -teQQa4dfc0WNRTdPM2Lzpab8DJw0C2EqO3lUu1JqOHkbDoLwEyX6mB3jRz9Kkh3Q -oSQG89+G7eePu+jl63qIhdLEUbC+JZlDlvlqEQIDAQABAoIBADmtaWGT45b9Eyge -rRpRCY3Mz+0j/EJdMiGaqtLCKj4VdmpiD6jpo/Qkj7fftxlGcWb0gzskbtSJ34XV -okXPalzKfnkJtRnsYPyaGzWBCn6LTD0qIrO+tbQk8Sr2Kl5DHwHJgIMhnT0hbRof -rtU8ihvI0GAeft+xRdDk6MUYF0YasqDDoiNTKFLTFu3bz7Jbh42zlmmu+u4PRO5b -AgxsBxvo9DdXC2lKTB/SKv1kWL2bysr6La3WBKmUwIuOEfLDuT3Vboy9orIDFog8 -QOmEgAZeR8PVw9MhkaHdosWJqL4G0kqIvlhf6rT5ZDUeKGBo2qmO48jHBCGJBFGw -FKJNoNUCgYEA6tjUiKCILKmVtkD7tJtQqEMs/NRDDX6/5uCfROqIMM6ABM3RqgRv -bMJffyGjXG+M86DxDgX0i25U8I0wmNpOU9P6qCgvhaOH+h+TNC37B6IZ+VwGxgDZ -LbdTTxA5WaKzDmwzlUHgAlf31jQ5j3Bf9Js7Tyz/X5fn3fd8i6Mwu6MCgYEA0qW6 -oIkRtzO0jkX5FHmQGuhHmhguzF3WXy/TcrAFQL5EsDQofbAUv70DxxFfufsfgPUP -wKpj/VhED2JFnqVV/68nArWl2eeLSX2gKrgHMy/9AoVYC1w/O2JPilbZXt1W0liT -SB4ZzCONom1m/wtaGjStjup2pSxcm05DKixD3rsCgYEAsKFygGwU31qRAmmvpm/m -YxdbH7FZ2S2KkdBBmei3k9XMTVCrr670Sx2KC6k2H9C6d4aFpuFtwuyxr9bRRTV0 -EfJuJMlMrLuJCuNyqJ0on94YoQbJBWUf8xVd8CooqDUJbQCOb2UDYV/eRFo1LJ/9 -W5DhM7SJQdGTj8uS/cc4YPcCgYEAkQ5DKA17v5bBfT++OFVF4OGXfQuuHll4J/A9 -Qbrowx7DGjuwrmy0vRyiH1FdhCrkFN+sy1YKqQlBRP69RnRAdmPdD0abQSTri94Q -j5pOivc+2Z+Nc7VAbdpTP8ZyxZrSEOOh+IWR6juJaxK/XF4q2+Tup33Z2gBkfSY1 -pjL5QcUCgYBZ1FQwPD1Z3EPXhqs7yVcOGp286BffEnd3xJyvqx13Jsd/LzmG8YzR -FLQ9XLRehs+exdt6yPEMcIkwWKiGPkJ4OXce9arbFp9ILb0Ppo5y41oBsrFV2BCv -NgrB5gTKtN/ipRWwND5zm+yHE3FJR8bgniTVtFF8z2XSWlWduE5gRA== +MIIEpAIBAAKCAQEAztjiyj2tdfkji0fewWS0kABg0IABgG20NvL1PnHJLr98we7w +W7f3j27EGjW/ApuycsWXXKi0L82q8uDicoHHb3JI2JkT70oi0yG1Dx/zwPN+dkA7 +LBT1J3UK2hJTFPhp855CwY/ss9xpBsd1Fv3zuHifEqNGljeg1PjmQ3pNhxA/M0aZ +cLnfIUdZ5Hr+t+4es3zaWo4tLBKmZu6BkVGQKPGXeMkIAMtJlG24l7qKDRkR5TYA +ZT7P8Vn7hnuFuCNbrJSm686GawBxTQXom23dg9UcWxoHB7UiHFoR6j0bQAX+4R7b +IwculRDcvzrgCu6u06oFILwY7MlsxpX9hGTl1wIDAQABAoIBAFeP6pmQeICrYceR +OhQGLIWVE2bP+VLDnflw6i5v/qlieE6kdm1tOEgorK0nuV9CR81cJdIcvIJL/yTn +3BR7KdDcwUenrY+rg4h7CWmIrigtK4ilciccDBeS7XAZN8B11GxDv6Cu65XMJU2w +W7nK8URTE4vRQI1QqS3e26MPAAi/LVOt3ZPI6zg/GHEwnq0IVSQAOndLBr/IWZk5 +SANrkfwX8WS7/UxZgDptT9dyUQ5Pnj5mieTlIvBwyczdhZ7RDa8HdCSHW3xF83V1 +A0pkn6+TRojumYyr4RrPQj6htE64Hgx9w1Dv/UINjPXl5mGlbxQHMWGzlqD/qpyI +wg7RakECgYEA+9ARZpHfEFz+EEFi8l9J+BtJDo00WaKCOZHh5UJ8W+NreqSd8nSx +5u6wYwMJjRX2Hwv+FBEhxGbo1+ff6p++cYmiSlDtN2XRCDkBWvvGlxu55BDULrhx +f8lqaV3XGmOy2rQusp8hiHmkmPJCSVj3oJqQnbqJ2zahXAx1rTPwHqECgYEA0kln +4h+ZkZ+aldOMGF0d0txTcTqZvsSVKiFTSD9of/fiSDqb6xtLT2+ys6FZoFL9lyK8 +gtqH642CDQ+3WT6Nmn4kMF5HNVpEuCeRDeRhiquWeKaAQDyvZ5ym1+Cn3GhsO7Di +d2LJKV5hOoN77loVY5nwnUVIJ0h+WLf0T7DTCXcCgYEAiNT7X50MdTvS4splFgcp +jqRlAn9AXySrVtUqxwVlxhjCIpapLUK0GSTCvEq+OeghIaXGnujgTHUPOaNKTZgY +SGHdyjxHar7s42b2kZYWx63NSVzLr8eSBTpRlIflhvV+DtGyPmWyNxLCmkmqM2kg +xii3RL5EgtYgwIAUwdVjOYECgYBRPlsMWfkS8f7fc+PkZdVn6gey71kHAxw+MrHi +b90H09Vw4nPq2Zi3EAiSrfvanTWsdpcuVw+8See89B16NViwH5wLs+D/E+kI3QCF +xX6J/NEdu/ZA2zFJbpRnQzyXQyDNzwEv7tKZUQVvfe0boWIyIP99Q48k3jUyQZ/6 +Se6+8QKBgQCXl8H2K3CsZxoujKLb2qoEOPbxJQ2hxoMTS5XuQECReIVsNuptWrur +DF8WJi/B6AqwRX1P3l56RNwqB1yDBqv0QVLpU7vU/FmWqLWTn0r3AvM74qftvfAE +oa31wcYoCqPJoKgCG7TThLhNt2v5hL7sVgZNO0ueAiHhJbFLaf7ceg== -----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa.pub b/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa.pub new file mode 100644 index 0000000000..b4084d320a --- /dev/null +++ b/lib/ssh/test/ssh_algorithms_SUITE_data/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO2OLKPa11+SOLR97BZLSQAGDQgAGAbbQ28vU+cckuv3zB7vBbt/ePbsQaNb8Cm7JyxZdcqLQvzary4OJygcdvckjYmRPvSiLTIbUPH/PA8352QDssFPUndQraElMU+GnznkLBj+yz3GkGx3UW/fO4eJ8So0aWN6DU+OZDek2HED8zRplwud8hR1nkev637h6zfNpaji0sEqZm7oGRUZAo8Zd4yQgAy0mUbbiXuooNGRHlNgBlPs/xWfuGe4W4I1uslKbrzoZrAHFNBeibbd2D1RxbGgcHtSIcWhHqPRtABf7hHtsjBy6VENy/OuAK7q7TqgUgvBjsyWzGlf2EZOXX uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_basic_SUITE.erl b/lib/ssh/test/ssh_basic_SUITE.erl index ecec805696..1269ab393e 100644 --- a/lib/ssh/test/ssh_basic_SUITE.erl +++ b/lib/ssh/test/ssh_basic_SUITE.erl @@ -40,103 +40,64 @@ suite() -> [{ct_hooks,[ts_install_cth]}, - {timetrap,{seconds,40}}]. + {timetrap,{seconds,90}}]. all() -> - [{group, all_tests}, - daemon_already_started + [{group, all_tests} ]. %%%-define(PARALLEL, ). -define(PARALLEL, parallel). groups() -> - [{all_tests, [?PARALLEL], [{group, ssh_renegotiate_SUITE}, - {group, ssh_basic_SUITE}, - ssh_file_is_host_key, - ssh_file_is_host_key_misc, - ssh_file_is_auth_key - ]}, - {ssh_basic_SUITE, [], [app_test, - appup_test, - {group, dsa_key}, - {group, rsa_key}, - {group, ecdsa_sha2_nistp256_key}, - {group, ecdsa_sha2_nistp384_key}, - {group, ecdsa_sha2_nistp521_key}, - {group, ed25519_key}, - {group, ed448_key}, - {group, dsa_pass_key}, - {group, rsa_pass_key}, - {group, ecdsa_sha2_nistp256_pass_key}, - {group, ecdsa_sha2_nistp384_pass_key}, - {group, ecdsa_sha2_nistp521_pass_key}, - {group, host_user_key_differs}, - {group, key_cb}, - {group, internal_error}, - {group, rsa_host_key_is_actualy_ecdsa}, - daemon_already_started, - double_close, - daemon_opt_fd, - multi_daemon_opt_fd, - packet_size, - ssh_info_print, - {group, login_bad_pwd_no_retry}, - shell_exit_status, - setopts_getopts - ]}, - - {ssh_renegotiate_SUITE, [?PARALLEL], [rekey0, - rekey1, - rekey2, - rekey3, - rekey4, - rekey_limit_client, - rekey_limit_daemon, - rekey_time_limit_client, - rekey_time_limit_daemon, - norekey_limit_client, - norekey_limit_daemon, - renegotiate1, - renegotiate2]}, - - {dsa_key, [], [{group, basic}]}, - {rsa_key, [], [{group, basic}]}, - {ecdsa_sha2_nistp256_key, [], [{group, basic}]}, - {ecdsa_sha2_nistp384_key, [], [{group, basic}]}, - {ecdsa_sha2_nistp521_key, [], [{group, basic}]}, - {ed25519_key, [], [{group, basic}]}, - {ed448_key, [], [{group, basic}]}, - {rsa_host_key_is_actualy_ecdsa, [], [fail_daemon_start]}, - {host_user_key_differs, [?PARALLEL], [exec_key_differs1, - exec_key_differs2, - exec_key_differs3, - exec_key_differs_fail]}, - {dsa_pass_key, [], [pass_phrase]}, - {rsa_pass_key, [], [pass_phrase]}, - {ecdsa_sha2_nistp256_pass_key, [], [pass_phrase]}, - {ecdsa_sha2_nistp384_pass_key, [], [pass_phrase]}, - {ecdsa_sha2_nistp521_pass_key, [], [pass_phrase]}, + [{all_tests, [?PARALLEL], [{group, sequential}, + {group, p_basic}, + {group, internal_error}, + {group, login_bad_pwd_no_retry}, + {group, key_cb} + ]}, + + {sequential, [], [app_test, + appup_test, + daemon_already_started, + daemon_error_closes_port, % Should be re-written.. + double_close, + daemon_opt_fd, + multi_daemon_opt_fd, + packet_size, + ssh_info_print, + shell_exit_status, + setopts_getopts, + known_hosts, + ssh_file_is_host_key, + ssh_file_is_host_key_misc, + ssh_file_is_auth_key + ]}, + {key_cb, [?PARALLEL], [key_callback, key_callback_options]}, - {internal_error, [], [internal_error]}, + + {internal_error, [?PARALLEL], [internal_error]}, + {login_bad_pwd_no_retry, [?PARALLEL], [login_bad_pwd_no_retry1, - login_bad_pwd_no_retry2, - login_bad_pwd_no_retry3, - login_bad_pwd_no_retry4, - login_bad_pwd_no_retry5 - ]}, + login_bad_pwd_no_retry2, + login_bad_pwd_no_retry3, + login_bad_pwd_no_retry4, + login_bad_pwd_no_retry5 + ]}, - {basic, [], [{group,p_basic}, - shell, shell_no_unicode, shell_unicode_string, - close, - known_hosts - ]}, {p_basic, [?PARALLEL], [send, peername_sockname, - exec, exec_compressed, - exec_with_io_out, exec_with_io_in, - cli, - idle_time_client, idle_time_server, openssh_zlib_basic_test, - misc_ssh_options, inet_option, inet6_option]} + exec, exec_compressed, + exec_with_io_out, exec_with_io_in, + cli, + idle_time_client, idle_time_server, openssh_zlib_basic_test, + misc_ssh_options, inet_option, inet6_option + + ,shell, + shell_no_unicode, + shell_unicode_string, + close + + ]} ]. @@ -147,6 +108,8 @@ groups() -> init_per_suite(Config) -> ?CHECK_CRYPTO(begin ssh:start(), + ct:log("Pub keys setup for: ~p", + [ssh_test_lib:setup_all_user_host_keys(Config)]), Config end). @@ -154,282 +117,22 @@ end_per_suite(_Config) -> ssh:stop(). %%-------------------------------------------------------------------- -init_per_group(ssh_renegotiate_SUITE, Config) -> - [{preferred_algorithms, ssh:default_algorithms()} | Config]; -init_per_group(dsa_key, Config) -> - case lists:member('ssh-dss', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_dsa(DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(rsa_key, Config) -> - case lists:member('ssh-rsa', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_rsa(DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(rsa_host_key_is_actualy_ecdsa, Config) -> - case - lists:member('ssh-rsa', - ssh_transport:default_algorithms(public_key)) and - lists:member('ecdsa-sha2-nistp256', - ssh_transport:default_algorithms(public_key)) - of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_ecdsa("256", DataDir, PrivDir), - %% The following sets up bad rsa keys: - begin - UserDir = PrivDir, - System = filename:join(UserDir, "system"), - file:copy(filename:join(DataDir, "id_rsa"), filename:join(UserDir, "id_rsa")), - file:rename(filename:join(System, "ssh_host_ecdsa_key"), filename:join(System, "ssh_host_rsa_key")), - file:rename(filename:join(System, "ssh_host_ecdsa_key.pub"), filename:join(System, "ssh_host_rsa_key.pub")), - ssh_test_lib:setup_rsa_known_host(DataDir, UserDir), - ssh_test_lib:setup_rsa_auth_keys(DataDir, UserDir) - end, - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ecdsa_sha2_nistp256_key, Config) -> - case lists:member('ecdsa-sha2-nistp256', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_ecdsa("256", DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ecdsa_sha2_nistp384_key, Config) -> - case lists:member('ecdsa-sha2-nistp384', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_ecdsa("384", DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ecdsa_sha2_nistp521_key, Config) -> - case lists:member('ecdsa-sha2-nistp521', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_ecdsa("521", DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ed25519_key, Config) -> - case lists:member('ssh-ed25519', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_eddsa(ed25519, DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ed448_key, Config) -> - case lists:member('ssh-ed448', - ssh_transport:default_algorithms(public_key)) of - true -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_eddsa(ed448, DataDir, PrivDir), - Config; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(rsa_pass_key, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - case lists:member('ssh-rsa', - ssh_transport:default_algorithms(public_key)) - andalso - ssh_test_lib:setup_rsa_pass_phrase(DataDir, PrivDir, "Password") - of - true -> - [{pass_phrase, {rsa_pass_phrase, "Password"}}| Config]; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(dsa_pass_key, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - case lists:member('ssh-dss', - ssh_transport:default_algorithms(public_key)) - andalso - ssh_test_lib:setup_dsa_pass_phrase(DataDir, PrivDir, "Password") - of - true -> - [{pass_phrase, {dsa_pass_phrase, "Password"}}| Config]; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ecdsa_sha2_nistp256_pass_key, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - case lists:member('ecdsa-sha2-nistp256', - ssh_transport:default_algorithms(public_key)) - andalso - ssh_test_lib:setup_ecdsa_pass_phrase("256", DataDir, PrivDir, "Password") - of - true -> - [{pass_phrase, {ecdsa_pass_phrase, "Password"}}| Config]; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ecdsa_sha2_nistp384_pass_key, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - case lists:member('ecdsa-sha2-nistp384', - ssh_transport:default_algorithms(public_key)) - andalso - ssh_test_lib:setup_ecdsa_pass_phrase("384", DataDir, PrivDir, "Password") - of - true -> - [{pass_phrase, {ecdsa_pass_phrase, "Password"}}| Config]; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(ecdsa_sha2_nistp521_pass_key, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - case lists:member('ecdsa-sha2-nistp521', - ssh_transport:default_algorithms(public_key)) - andalso - ssh_test_lib:setup_ecdsa_pass_phrase("521", DataDir, PrivDir, "Password") - of - true -> - [{pass_phrase, {ecdsa_pass_phrase, "Password"}}| Config]; - false -> - {skip, unsupported_pub_key} - end; -init_per_group(host_user_key_differs, Config) -> - Data = proplists:get_value(data_dir, Config), - Sys = filename:join(proplists:get_value(priv_dir, Config), system_rsa), - SysUsr = filename:join(Sys, user), - Usr = filename:join(proplists:get_value(priv_dir, Config), user_ecdsa_256), - file:make_dir(Sys), - file:make_dir(SysUsr), - file:make_dir(Usr), - file:copy(filename:join(Data, "ssh_host_rsa_key"), filename:join(Sys, "ssh_host_rsa_key")), - file:copy(filename:join(Data, "ssh_host_rsa_key.pub"), filename:join(Sys, "ssh_host_rsa_key.pub")), - file:copy(filename:join(Data, "id_ecdsa256"), filename:join(Usr, "id_ecdsa")), - file:copy(filename:join(Data, "id_ecdsa256.pub"), filename:join(Usr, "id_ecdsa.pub")), - ssh_test_lib:setup_ecdsa_auth_keys("256", Data, SysUsr), - ssh_test_lib:setup_rsa_known_host(Sys, Usr), - Config; init_per_group(key_cb, Config) -> case lists:member('ssh-rsa', - ssh_transport:default_algorithms(public_key)) of + ssh_transport:supported_algorithms(public_key)) of true -> DataDir = proplists:get_value(data_dir, Config), PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_rsa(DataDir, PrivDir), + ssh_test_lib:setup_user_key('ssh-rsa', DataDir, PrivDir), + ssh_test_lib:setup_host_key_create_dir('ssh-rsa', DataDir, PrivDir), Config; false -> {skip, unsupported_pub_key} end; -init_per_group(internal_error, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_rsa(DataDir, PrivDir), - ssh_test_lib:setup_dsa(DataDir, PrivDir), - ssh_test_lib:setup_ecdsa("256", DataDir, PrivDir), - %% In the test case the key will be deleted after the daemon start: - %% ... file:delete(filename:join(PrivDir, "system/ssh_host_dsa_key")), - Config; -init_per_group(dir_options, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - %% Make unreadable dir: - Dir_unreadable = filename:join(PrivDir, "unread"), - ok = file:make_dir(Dir_unreadable), - {ok,F1} = file:read_file_info(Dir_unreadable), - ok = file:write_file_info(Dir_unreadable, - F1#file_info{mode = F1#file_info.mode band (bnot 8#00444)}), - %% Make readable file: - File_readable = filename:join(PrivDir, "file"), - ok = file:write_file(File_readable, <<>>), - - %% Check: - case {file:read_file_info(Dir_unreadable), - file:read_file_info(File_readable)} of - {{ok, Id=#file_info{type=directory, access=Md}}, - {ok, If=#file_info{type=regular, access=Mf}}} -> - AccessOK = - case {Md, Mf} of - {read, _} -> false; - {read_write, _} -> false; - {_, read} -> true; - {_, read_write} -> true; - _ -> false - end, - - case AccessOK of - true -> - %% Save: - [{unreadable_dir, Dir_unreadable}, - {readable_file, File_readable} - | Config]; - false -> - ct:log("File#file_info : ~p~n" - "Dir#file_info : ~p",[If,Id]), - {skip, "File or dir mode settings failed"} - end; - - NotDirFile -> - ct:log("{Dir,File} -> ~p",[NotDirFile]), - {skip, "File/Dir creation failed"} - end; init_per_group(_, Config) -> Config. -end_per_group(dsa_key, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_dsa(PrivDir), - Config; -end_per_group(rsa_key, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_rsa(PrivDir), - Config; -end_per_group(dsa_pass_key, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_dsa(PrivDir), - Config; -end_per_group(rsa_pass_key, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_rsa(PrivDir), - Config; -end_per_group(key_cb, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_rsa(PrivDir), - Config; -end_per_group(internal_error, Config) -> - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_rsa(PrivDir), - ssh_test_lib:clean_dsa(PrivDir), - Config; - end_per_group(_, Config) -> Config. %%-------------------------------------------------------------------- @@ -462,11 +165,6 @@ init_per_testcase(inet6_option, Config) -> init_per_testcase(_TestCase, Config) -> Config. -end_per_testcase(TestCase, Config) when TestCase == server_password_option; - TestCase == server_userpassword_option -> - UserDir = filename:join(proplists:get_value(priv_dir, Config), nopubkey), - ssh_test_lib:del_dirs(UserDir), - end_per_testcase(Config); end_per_testcase(TC, Config) when TC==shell_no_unicode ; TC==shell_unicode_string -> case proplists:get_value(sftpd, Config) of @@ -527,6 +225,7 @@ inet_option(Config) when is_list(Config) -> %%-------------------------------------------------------------------- %%% Test configuring IPv6 +inet6_option() -> [{timetrap,{seconds,30}}]. inet6_option(Config) when is_list(Config) -> SystemDir = filename:join(proplists:get_value(priv_dir, Config), system), UserDir = proplists:get_value(priv_dir, Config), @@ -603,13 +302,16 @@ exec_with_io_out(Config) when is_list(Config) -> "io:write(hej).", infinity), case ssh_test_lib:receive_exec_result( [{ssh_cm, ConnectionRef, {data, ChannelId0, 0, <<"hej">>}}, - {ssh_cm, ConnectionRef, {data, ChannelId0, 0, <<"ok">>}}]) of + {ssh_cm, ConnectionRef, {data, ChannelId0, 0, <<"ok">>}}, + {ssh_cm, ConnectionRef, {eof, ChannelId0}}, + {ssh_cm, ConnectionRef, {exit_status, ChannelId0, 0}}, + {ssh_cm, ConnectionRef, {closed, ChannelId0}} + ]) of expected -> ok; Other0 -> ct:fail(Other0) end, - ssh_test_lib:receive_exec_end(ConnectionRef, ChannelId0), ssh:close(ConnectionRef), ssh:stop_daemon(Pid). @@ -738,81 +440,6 @@ shell(Config) when is_list(Config) -> end. %%-------------------------------------------------------------------- -%%% Test that we could user different types of host pubkey and user pubkey -exec_key_differs1(Config) -> exec_key_differs(Config, ['ecdsa-sha2-nistp256']). - -exec_key_differs2(Config) -> exec_key_differs(Config, ['ssh-dss','ecdsa-sha2-nistp256']). - -exec_key_differs3(Config) -> exec_key_differs(Config, ['ecdsa-sha2-nistp384','ecdsa-sha2-nistp256']). - - - -exec_key_differs(Config, UserPKAlgs) -> - case lists:usort(['ssh-rsa'|UserPKAlgs]) - -- ssh_transport:supported_algorithms(public_key) - of - [] -> - process_flag(trap_exit, true), - SystemDir = filename:join(proplists:get_value(priv_dir, Config), system_rsa), - SystemUserDir = filename:join(SystemDir, user), - UserDir = filename:join(proplists:get_value(priv_dir, Config), user_ecdsa_256), - - {_Pid, _Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, - {user_dir, SystemUserDir}, - {preferred_algorithms, - [{public_key,['ssh-rsa'|UserPKAlgs]}]}]), - ct:sleep(500), - - IO = ssh_test_lib:start_io_server(), - Shell = ssh_test_lib:start_shell(Port, IO, [{user_dir,UserDir}, - {preferred_algorithms,[{public_key,['ssh-rsa']}]}, - {pref_public_key_algs,UserPKAlgs} - ]), - - - receive - {'EXIT', _, _} -> - ct:fail(no_ssh_connection); - ErlShellStart -> - ct:log("Erlang shell start: ~p~n", [ErlShellStart]), - do_shell(IO, Shell) - after - 30000 -> ct:fail("timeout ~p:~p",[?MODULE,?LINE]) - end; - - UnsupportedPubKeys -> - {skip, io_lib:format("~p unsupported",[UnsupportedPubKeys])} - end. - -%%-------------------------------------------------------------------- -exec_key_differs_fail(Config) when is_list(Config) -> - process_flag(trap_exit, true), - SystemDir = filename:join(proplists:get_value(priv_dir, Config), system_rsa), - SystemUserDir = filename:join(SystemDir, user), - UserDir = filename:join(proplists:get_value(priv_dir, Config), user_ecdsa_256), - - {_Pid, _Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, - {user_dir, SystemUserDir}, - {preferred_algorithms, - [{public_key,['ssh-rsa']}]}]), - ct:sleep(500), - - IO = ssh_test_lib:start_io_server(), - ssh_test_lib:start_shell(Port, IO, [{user_dir,UserDir}, - {recv_ext_info, false}, - {preferred_algorithms,[{public_key,['ssh-rsa']}]}, - {pref_public_key_algs,['ssh-dss']}]), - receive - {'EXIT', _, _} -> - ok; - ErlShellStart -> - ct:log("Erlang shell start: ~p~n", [ErlShellStart]), - ct:fail(connection_not_rejected) - after - 30000 -> ct:fail("timeout ~p:~p",[?MODULE,?LINE]) - end. - -%%-------------------------------------------------------------------- cli(Config) when is_list(Config) -> process_flag(trap_exit, true), SystemDir = filename:join(proplists:get_value(priv_dir, Config), system), @@ -871,6 +498,11 @@ daemon_already_started(Config) when is_list(Config) -> %%-------------------------------------------------------------------- %%% Test that a failed daemon start does not leave the port open + +%%%%%%%%%%%%%%%%%%%%%% REWRITE! %%%%%%%%%%%%%%%%%%%% +%%% 1) check that {error,_} is not {error,eaddrinuse} +%%% 2) instead of ssh_test_lib:daemon second time, use gen_tcp:listen + daemon_error_closes_port(Config) -> GoodSystemDir = proplists:get_value(data_dir, Config), Port = inet_port(), @@ -886,6 +518,11 @@ daemon_error_closes_port(Config) -> ssh:stop_daemon(Pid) end. +inet_port() -> + {ok, Socket} = gen_tcp:listen(0, [{reuseaddr, true}]), + {ok, Port} = inet:port(Socket), + gen_tcp:close(Socket), + Port. %%-------------------------------------------------------------------- %%% check that known_hosts is updated correctly @@ -957,7 +594,7 @@ known_hosts(Config) when is_list(Config) -> ssh:stop_daemon(Pid). %%-------------------------------------------------------------------- -ssh_file_is_host_key() -> [{timetrap,{seconds,120}}]. % Some machines are S L O W ! +ssh_file_is_host_key() -> [{timetrap,{seconds,240}}]. % Some machines are S L O W ! ssh_file_is_host_key(Config) -> Dir = ssh_test_lib:create_random_dir(Config), ct:log("Dir = ~p", [Dir]), @@ -1176,12 +813,13 @@ send(Config) when is_list(Config) -> process_flag(trap_exit, true), SystemDir = filename:join(proplists:get_value(priv_dir, Config), system), UserDir = proplists:get_value(priv_dir, Config), - {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, + {preferred_algorithms, ssh_transport:supported_algorithms()}, {user_dir, UserDir}, {failfun, fun ssh_test_lib:failfun/2}]), ConnectionRef = - ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true}, + ssh_test_lib:connect(Host, Port, [{preferred_algorithms, ssh_transport:supported_algorithms()}, + {silently_accept_hosts, true}, {user_dir, UserDir}, {user_interaction, false}]), {ok, ChannelId} = ssh_connection:session_channel(ConnectionRef, infinity), @@ -1191,17 +829,6 @@ send(Config) when is_list(Config) -> %%-------------------------------------------------------------------- -%%% -fail_daemon_start(Config) when is_list(Config) -> - process_flag(trap_exit, true), - SystemDir = filename:join(proplists:get_value(priv_dir, Config), system), - UserDir = proplists:get_value(priv_dir, Config), - - {error,_} = ssh_test_lib:daemon([{system_dir, SystemDir}, - {user_dir, UserDir}, - {failfun, fun ssh_test_lib:failfun/2}]). - -%%-------------------------------------------------------------------- %%% Test ssh:connection_info([peername, sockname]) peername_sockname(Config) when is_list(Config) -> process_flag(trap_exit, true), @@ -1646,315 +1273,6 @@ setopts_getopts(Config) -> ssh:stop_daemon(Pid). -%%---------------------------------------------------------------------------- -%%% Idle timeout test -rekey0() -> [{timetrap,{seconds,90}}]. -rekey1() -> [{timetrap,{seconds,90}}]. -rekey2() -> [{timetrap,{seconds,90}}]. -rekey3() -> [{timetrap,{seconds,90}}]. -rekey4() -> [{timetrap,{seconds,90}}]. - -rekey0(Config) -> rekey_chk(Config, 0, 0). -rekey1(Config) -> rekey_chk(Config, infinity, 0). -rekey2(Config) -> rekey_chk(Config, {infinity,infinity}, 0). -rekey3(Config) -> rekey_chk(Config, 0, infinity). -rekey4(Config) -> rekey_chk(Config, 0, {infinity,infinity}). - -rekey_chk(Config, RLdaemon, RLclient) -> - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config, [{rekey_limit, RLdaemon}]), - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, RLclient}]), - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - - %% Make both sides send something: - {ok, _SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - %% Check rekeying - timer:sleep(?REKEY_DATA_TMO), - ?wait_match(false, Kex1==ssh_test_lib:get_kex_init(ConnectionRef), [], 2000, 10), - - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - -%%-------------------------------------------------------------------- -%%% Test rekeying by data volume - -rekey_limit_client() -> [{timetrap,{seconds,400}}]. -rekey_limit_client(Config) -> - Limit = 6000, - UserDir = proplists:get_value(priv_dir, Config), - DataFile = filename:join(UserDir, "rekey.data"), - Data = lists:duplicate(Limit+10,1), - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, Limit}, - {max_random_length_padding,0}]), - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - %% Check that it doesn't rekey without data transfer - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), - - %% Check that datatransfer triggers rekeying - ok = ssh_sftp:write_file(SftpPid, DataFile, Data), - timer:sleep(?REKEY_DATA_TMO), - ?wait_match(false, Kex1==(Kex2=ssh_test_lib:get_kex_init(ConnectionRef)), Kex2, 2000, 10), - - %% Check that datatransfer continues to trigger rekeying - ok = ssh_sftp:write_file(SftpPid, DataFile, Data), - timer:sleep(?REKEY_DATA_TMO), - ?wait_match(false, Kex2==(Kex3=ssh_test_lib:get_kex_init(ConnectionRef)), Kex3, 2000, 10), - - %% Check that it doesn't rekey without data transfer - timer:sleep(?REKEY_DATA_TMO), - true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), - - %% Check that it doesn't rekey on a small datatransfer - ok = ssh_sftp:write_file(SftpPid, DataFile, "hi\n"), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), - - %% Check that it doesn't rekey without data transfer - timer:sleep(?REKEY_DATA_TMO), - true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), - - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - - - -rekey_limit_daemon() -> [{timetrap,{seconds,400}}]. -rekey_limit_daemon(Config) -> - Limit = 6000, - UserDir = proplists:get_value(priv_dir, Config), - DataFile1 = filename:join(UserDir, "rekey1.data"), - DataFile2 = filename:join(UserDir, "rekey2.data"), - file:write_file(DataFile1, lists:duplicate(Limit+10,1)), - file:write_file(DataFile2, "hi\n"), - - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{rekey_limit, Limit}, - {max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{max_random_length_padding,0}]), - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - %% Check that it doesn't rekey without data transfer - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - timer:sleep(?REKEY_DATA_TMO), - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - - %% Check that datatransfer triggers rekeying - {ok,_} = ssh_sftp:read_file(SftpPid, DataFile1), - timer:sleep(?REKEY_DATA_TMO), - ?wait_match(false, Kex1==(Kex2=ssh_test_lib:get_kex_init(ConnectionRef)), Kex2, 2000, 10), - - %% Check that datatransfer continues to trigger rekeying - {ok,_} = ssh_sftp:read_file(SftpPid, DataFile1), - timer:sleep(?REKEY_DATA_TMO), - ?wait_match(false, Kex2==(Kex3=ssh_test_lib:get_kex_init(ConnectionRef)), Kex3, 2000, 10), - - %% Check that it doesn't rekey without data transfer - timer:sleep(?REKEY_DATA_TMO), - true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), - - %% Check that it doesn't rekey on a small datatransfer - {ok,_} = ssh_sftp:read_file(SftpPid, DataFile2), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), - - %% Check that it doesn't rekey without data transfer - timer:sleep(?REKEY_DATA_TMO), - true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), - - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - - -%%-------------------------------------------------------------------- -%% Check that datatransfer in the other direction does not trigger re-keying -norekey_limit_client() -> [{timetrap,{seconds,400}}]. -norekey_limit_client(Config) -> - Limit = 6000, - UserDir = proplists:get_value(priv_dir, Config), - DataFile = filename:join(UserDir, "rekey3.data"), - file:write_file(DataFile, lists:duplicate(Limit+10,1)), - - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, Limit}, - {max_random_length_padding,0}]), - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), - - {ok,_} = ssh_sftp:read_file(SftpPid, DataFile), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), - - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - -%% Check that datatransfer in the other direction does not trigger re-keying -norekey_limit_daemon() -> [{timetrap,{seconds,400}}]. -norekey_limit_daemon(Config) -> - Limit = 6000, - UserDir = proplists:get_value(priv_dir, Config), - DataFile = filename:join(UserDir, "rekey4.data"), - - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{rekey_limit, Limit}, - {max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{max_random_length_padding,0}]), - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), - - ok = ssh_sftp:write_file(SftpPid, DataFile, lists:duplicate(Limit+10,1)), - timer:sleep(?REKEY_DATA_TMO), - true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), - - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - -%%-------------------------------------------------------------------- -%%% Test rekeying by time - -rekey_time_limit_client() -> [{timetrap,{seconds,400}}]. -rekey_time_limit_client(Config) -> - Minutes = ?REKEY_DATA_TMO div 60000, - GB = 1024*1000*1000, - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, {Minutes, GB}}, - {max_random_length_padding,0}]), - rekey_time_limit(Pid, ConnectionRef). - -rekey_time_limit_daemon() -> [{timetrap,{seconds,400}}]. -rekey_time_limit_daemon(Config) -> - Minutes = ?REKEY_DATA_TMO div 60000, - GB = 1024*1000*1000, - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{rekey_limit, {Minutes, GB}}, - {max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{max_random_length_padding,0}]), - rekey_time_limit(Pid, ConnectionRef). - - -rekey_time_limit(Pid, ConnectionRef) -> - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - - timer:sleep(5000), - true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), - - %% Check that it rekeys when the max time + 30s has passed - timer:sleep(?REKEY_DATA_TMO + 30*1000), - ?wait_match(false, Kex1==(Kex2=ssh_test_lib:get_kex_init(ConnectionRef)), Kex2, 2000, 10), - - %% Check that it does not rekey when nothing is transferred - timer:sleep(?REKEY_DATA_TMO + 30*1000), - ?wait_match(false, Kex2==ssh_test_lib:get_kex_init(ConnectionRef), [], 2000, 10), - - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - -%%-------------------------------------------------------------------- - -%%% Test rekeying with simultaneous send request - -renegotiate1(Config) -> - UserDir = proplists:get_value(priv_dir, Config), - DataFile = filename:join(UserDir, "renegotiate1.data"), - - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, DPort} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - - {ok,RelayPid,_,RPort} = ssh_relay:start_link({0,0,0,0}, 0, Host, DPort), - - ConnectionRef = ssh_test_lib:std_connect(Config, Host, RPort, [{max_random_length_padding,0}]), - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - - {ok, Handle} = ssh_sftp:open(SftpPid, DataFile, [write]), - - ok = ssh_sftp:write(SftpPid, Handle, "hi\n"), - - ssh_relay:hold(RelayPid, rx, 20, 1000), - ssh_connection_handler:renegotiate(ConnectionRef), - spawn(fun() -> ok=ssh_sftp:write(SftpPid, Handle, "another hi\n") end), - - timer:sleep(2000), - - Kex2 = ssh_test_lib:get_kex_init(ConnectionRef), - - false = (Kex2 == Kex1), - - ssh_relay:stop(RelayPid), - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - -%%-------------------------------------------------------------------- - -%%% Test rekeying with inflight messages from peer - -renegotiate2(Config) -> - UserDir = proplists:get_value(priv_dir, Config), - DataFile = filename:join(UserDir, "renegotiate2.data"), - - Algs = proplists:get_value(preferred_algorithms, Config), - {Pid, Host, DPort} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, - {preferred_algorithms,Algs}]), - - {ok,RelayPid,_,RPort} = ssh_relay:start_link({0,0,0,0}, 0, Host, DPort), - - ConnectionRef = ssh_test_lib:std_connect(Config, Host, RPort, [{max_random_length_padding,0}]), - {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), - - Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), - - {ok, Handle} = ssh_sftp:open(SftpPid, DataFile, [write]), - - ok = ssh_sftp:write(SftpPid, Handle, "hi\n"), - - ssh_relay:hold(RelayPid, rx, 20, infinity), - spawn(fun() -> ok=ssh_sftp:write(SftpPid, Handle, "another hi\n") end), - %% need a small pause here to ensure ssh_sftp:write is executed - ct:sleep(10), - ssh_connection_handler:renegotiate(ConnectionRef), - ssh_relay:release(RelayPid, rx), - - timer:sleep(2000), - - Kex2 = ssh_test_lib:get_kex_init(ConnectionRef), - - false = (Kex2 == Kex1), - - ssh_relay:stop(RelayPid), - ssh_sftp:stop_channel(SftpPid), - ssh:close(ConnectionRef), - ssh:stop_daemon(Pid). - %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ %%-------------------------------------------------------------------- @@ -2134,9 +1452,3 @@ new_do_shell_prompt(IO, N, Op, Str, More) -> ct:log("Matched prompt ~p",[N]), new_do_shell(IO, N, [{Op,Str}|More]). -%%-------------------------------------------------------------------- -inet_port() -> - {ok, Socket} = gen_tcp:listen(0, [{reuseaddr, true}]), - {ok, Port} = inet:port(Socket), - gen_tcp:close(Socket), - Port. diff --git a/lib/ssh/test/ssh_basic_SUITE_data/id_dsa b/lib/ssh/test/ssh_basic_SUITE_data/id_dsa index d306f8b26e..24628e071b 100644 --- a/lib/ssh/test/ssh_basic_SUITE_data/id_dsa +++ b/lib/ssh/test/ssh_basic_SUITE_data/id_dsa @@ -1,13 +1,12 @@ -----BEGIN DSA PRIVATE KEY----- -MIIBvAIBAAKBgQDfi2flSTZZofwT4yQT0NikX/LGNT7UPeB/XEWe/xovEYCElfaQ -APFixXvEgXwoojmZ5kiQRKzLM39wBP0jPERLbnZXfOOD0PDnw0haMh7dD7XKVMod -/EigVgHf/qBdM2M8yz1s/rRF7n1UpLSypziKjkzCm7JoSQ2zbWIPdmBIXwIVAMgP -kpr7Sq3O7sHdb8D601DRjoExAoGAMOQxDfB2Fd8ouz6G96f/UOzRMI/Kdv8kYYKW -JIGY+pRYrLPyYzUeJznwZreOJgrczAX+luHnKFWJ2Dnk5CyeXk67Wsr7pJ/4MBMD -OKeIS0S8qoSBN8+Krp79fgA+yS3IfqbkJLtLu4EBaCX4mKQIX4++k44d4U5lc8pt -+9hlEI8CgYEAznKxx9kyC6bVo7LUYKaGhofRFt0SYFc5PVmT2VUGRs1R6+6DPD+e -uEO6IhFct7JFSRbP9p0JD4Uk+3zlZF+XX6b2PsZkeV8f/02xlNGUSmEzCSiNg1AX -Cy/WusYhul0MncWCHMcOZB5rIvU/aP5EJJtn3xrRaz6u0SThF6AnT34CFQC63czE -ZU8w8Q+H7z0j+a+70x2iAw== +MIIBuwIBAAKBgQDIywHurUpOq6kZuMn+XlRzR4hAxF6qwSkuEqkV7iHnLQ0kIwf3 +uAmjFDhuEsQ8653SLxGVvTNp+KFFgDXiLqgM7TPUwDnpbvzEZHPAU+/zPt4sdY2D +txBfJwT2SFlK6HPOxOcxdDuD+/a59sh8hk/YVOU7ZTcBVsVG8Got4UcF5QIVAPGd +CPDQKSTlPiM9OwBB1+9p11k5AoGARLxw4l17mET9cU0uf4Ppe5nsCbODJv44ZrSs +picvypGVLrLcN5KWbm3vjRFCQ5LFunAG3FwLC2Sh0CH6TemoIfRPsRHR7wvpBGdr +c693UlMOis/mcmvNMQAzuQNW9WrxdzsvWR/r5s6NEHWqKUJGXSPi2d+Ijq/mCOmI +hzLzyiACgYEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4 +cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+C +ROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNgCFEjA7wTC +sQCY/I35vb6GUJn9tEdP -----END DSA PRIVATE KEY----- - diff --git a/lib/ssh/test/ssh_basic_SUITE_data/id_dsa.pub b/lib/ssh/test/ssh_basic_SUITE_data/id_dsa.pub new file mode 100644 index 0000000000..018ef6f537 --- /dev/null +++ b/lib/ssh/test/ssh_basic_SUITE_data/id_dsa.pub @@ -0,0 +1 @@ +ssh-dss AAAAB3NzaC1kc3MAAACBAMjLAe6tSk6rqRm4yf5eVHNHiEDEXqrBKS4SqRXuIectDSQjB/e4CaMUOG4SxDzrndIvEZW9M2n4oUWANeIuqAztM9TAOelu/MRkc8BT7/M+3ix1jYO3EF8nBPZIWUroc87E5zF0O4P79rn2yHyGT9hU5TtlNwFWxUbwai3hRwXlAAAAFQDxnQjw0Ckk5T4jPTsAQdfvaddZOQAAAIBEvHDiXXuYRP1xTS5/g+l7mewJs4Mm/jhmtKymJy/KkZUustw3kpZube+NEUJDksW6cAbcXAsLZKHQIfpN6agh9E+xEdHvC+kEZ2tzr3dSUw6Kz+Zya80xADO5A1b1avF3Oy9ZH+vmzo0QdaopQkZdI+LZ34iOr+YI6YiHMvPKIAAAAIEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+CROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNg= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_basic_SUITE_data/id_rsa b/lib/ssh/test/ssh_basic_SUITE_data/id_rsa index 9d7e0dd5fb..2202c2ead8 100644 --- a/lib/ssh/test/ssh_basic_SUITE_data/id_rsa +++ b/lib/ssh/test/ssh_basic_SUITE_data/id_rsa @@ -1,15 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIICXAIBAAKBgQD1OET+3O/Bvj/dtjxDTXmj1oiJt4sIph5kGy0RfjoPrZfaS+CU -DhakCmS6t2ivxWFgtpKWaoGMZMJqWj6F6ZsumyFl3FPBtujwY/35cgifrI9Ns4Tl -zR1uuengNBmV+WRQ5cd9F2qS6Z8aDQihzt0r8JUqLcK+VQbrmNzboCCQQwIDAQAB -AoGAPQEyqPTt8JUT7mRXuaacjFXiweAXhp9NEDpyi9eLOjtFe9lElZCrsUOkq47V -TGUeRKEm9qSodfTbKPoqc8YaBJGJPhUaTAcha+7QcDdfHBvIsgxvU7ePVnlpXRp3 -CCUEMPhlnx6xBoTYP+fRU0e3+xJIPVyVCqX1jAdUMkzfRoECQQD6ux7B1QJAIWyK -SGkbDUbBilNmzCFNgIpOP6PA+bwfi5d16diTpra5AX09keQABAo/KaP1PdV8Vg0p -z4P3A7G3AkEA+l+AKG6m0kQTTBMJDqOdVPYwe+5GxunMaqmhokpEbuGsrZBl5Dvd -WpcBjR7jmenrhKZRIuA+Fz5HPo/UQJPl1QJBAKxstDkeED8j/S2XoFhPKAJ+6t39 -sUVICVTIZQeXdmzHJXCcUSkw8+WEhakqw/3SyW0oaK2FSWQJFWJUZ+8eJj8CQEh3 -xeduB5kKnS9CvzdeghZqX6QvVosSdtlUmfUYW/BgH5PpHKTP8wTaeld3XldZTpMJ -dKiMkUw2+XYROVUrubUCQD+Na1LhULlpn4ISEtIEfqpdlUhxDgO15Wg8USmsng+x -ICliVOSQtwaZjm8kwaFt0W7XnpnDxbRs37vIEbIMWak= +MIIEpAIBAAKCAQEAztjiyj2tdfkji0fewWS0kABg0IABgG20NvL1PnHJLr98we7w +W7f3j27EGjW/ApuycsWXXKi0L82q8uDicoHHb3JI2JkT70oi0yG1Dx/zwPN+dkA7 +LBT1J3UK2hJTFPhp855CwY/ss9xpBsd1Fv3zuHifEqNGljeg1PjmQ3pNhxA/M0aZ +cLnfIUdZ5Hr+t+4es3zaWo4tLBKmZu6BkVGQKPGXeMkIAMtJlG24l7qKDRkR5TYA +ZT7P8Vn7hnuFuCNbrJSm686GawBxTQXom23dg9UcWxoHB7UiHFoR6j0bQAX+4R7b +IwculRDcvzrgCu6u06oFILwY7MlsxpX9hGTl1wIDAQABAoIBAFeP6pmQeICrYceR +OhQGLIWVE2bP+VLDnflw6i5v/qlieE6kdm1tOEgorK0nuV9CR81cJdIcvIJL/yTn +3BR7KdDcwUenrY+rg4h7CWmIrigtK4ilciccDBeS7XAZN8B11GxDv6Cu65XMJU2w +W7nK8URTE4vRQI1QqS3e26MPAAi/LVOt3ZPI6zg/GHEwnq0IVSQAOndLBr/IWZk5 +SANrkfwX8WS7/UxZgDptT9dyUQ5Pnj5mieTlIvBwyczdhZ7RDa8HdCSHW3xF83V1 +A0pkn6+TRojumYyr4RrPQj6htE64Hgx9w1Dv/UINjPXl5mGlbxQHMWGzlqD/qpyI +wg7RakECgYEA+9ARZpHfEFz+EEFi8l9J+BtJDo00WaKCOZHh5UJ8W+NreqSd8nSx +5u6wYwMJjRX2Hwv+FBEhxGbo1+ff6p++cYmiSlDtN2XRCDkBWvvGlxu55BDULrhx +f8lqaV3XGmOy2rQusp8hiHmkmPJCSVj3oJqQnbqJ2zahXAx1rTPwHqECgYEA0kln +4h+ZkZ+aldOMGF0d0txTcTqZvsSVKiFTSD9of/fiSDqb6xtLT2+ys6FZoFL9lyK8 +gtqH642CDQ+3WT6Nmn4kMF5HNVpEuCeRDeRhiquWeKaAQDyvZ5ym1+Cn3GhsO7Di +d2LJKV5hOoN77loVY5nwnUVIJ0h+WLf0T7DTCXcCgYEAiNT7X50MdTvS4splFgcp +jqRlAn9AXySrVtUqxwVlxhjCIpapLUK0GSTCvEq+OeghIaXGnujgTHUPOaNKTZgY +SGHdyjxHar7s42b2kZYWx63NSVzLr8eSBTpRlIflhvV+DtGyPmWyNxLCmkmqM2kg +xii3RL5EgtYgwIAUwdVjOYECgYBRPlsMWfkS8f7fc+PkZdVn6gey71kHAxw+MrHi +b90H09Vw4nPq2Zi3EAiSrfvanTWsdpcuVw+8See89B16NViwH5wLs+D/E+kI3QCF +xX6J/NEdu/ZA2zFJbpRnQzyXQyDNzwEv7tKZUQVvfe0boWIyIP99Q48k3jUyQZ/6 +Se6+8QKBgQCXl8H2K3CsZxoujKLb2qoEOPbxJQ2hxoMTS5XuQECReIVsNuptWrur +DF8WJi/B6AqwRX1P3l56RNwqB1yDBqv0QVLpU7vU/FmWqLWTn0r3AvM74qftvfAE +oa31wcYoCqPJoKgCG7TThLhNt2v5hL7sVgZNO0ueAiHhJbFLaf7ceg== -----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_basic_SUITE_data/id_rsa.pub b/lib/ssh/test/ssh_basic_SUITE_data/id_rsa.pub new file mode 100644 index 0000000000..b4084d320a --- /dev/null +++ b/lib/ssh/test/ssh_basic_SUITE_data/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO2OLKPa11+SOLR97BZLSQAGDQgAGAbbQ28vU+cckuv3zB7vBbt/ePbsQaNb8Cm7JyxZdcqLQvzary4OJygcdvckjYmRPvSiLTIbUPH/PA8352QDssFPUndQraElMU+GnznkLBj+yz3GkGx3UW/fO4eJ8So0aWN6DU+OZDek2HED8zRplwud8hR1nkev637h6zfNpaji0sEqZm7oGRUZAo8Zd4yQgAy0mUbbiXuooNGRHlNgBlPs/xWfuGe4W4I1uslKbrzoZrAHFNBeibbd2D1RxbGgcHtSIcWhHqPRtABf7hHtsjBy6VENy/OuAK7q7TqgUgvBjsyWzGlf2EZOXX uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_compat_SUITE.erl b/lib/ssh/test/ssh_compat_SUITE.erl index a8b3508583..f5f0a35bac 100644 --- a/lib/ssh/test/ssh_compat_SUITE.erl +++ b/lib/ssh/test/ssh_compat_SUITE.erl @@ -41,7 +41,7 @@ %%-------------------------------------------------------------------- suite() -> - [{timetrap,{seconds,60}}]. + [{timetrap,{seconds,90}}]. all() -> %% [check_docker_present] ++ @@ -534,21 +534,22 @@ result_of_exec(C, Ch, ExitStatus, Acc) -> chk_all_algos(FunctionName, CommonAlgs, Config, DoTestFun) when is_function(DoTestFun,2) -> ct:comment("~p algorithms",[length(CommonAlgs)]), %% Check each algorithm - Failed = + Nmax = length(CommonAlgs), + {_N,Failed} = lists:foldl( - fun({Tag,Alg}, FailedAlgos) -> - %% ct:log("Try ~p",[Alg]), + fun({Tag,Alg}, {N,FailedAlgos}) -> + ct:log("Try ~p ~p/~p",[{Tag,Alg},N,Nmax]), case DoTestFun(Tag,Alg) of {ok,C} -> ssh:close(C), - FailedAlgos; + {N+1,FailedAlgos}; ok -> - FailedAlgos; + {N+1,FailedAlgos}; Other -> ct:log("FAILED! ~p ~p: ~p",[Tag,Alg,Other]), - [{Alg,Other}|FailedAlgos] + {N+1, [{Alg,Other}|FailedAlgos]} end - end, [], CommonAlgs), + end, {1,[]}, CommonAlgs), ct:pal("~s", [format_result_table_use_all_algos(FunctionName, Config, CommonAlgs, Failed)]), case Failed of [] -> diff --git a/lib/ssh/test/ssh_connection_SUITE.erl b/lib/ssh/test/ssh_connection_SUITE.erl index d428daaf59..c4621723f6 100644 --- a/lib/ssh/test/ssh_connection_SUITE.erl +++ b/lib/ssh/test/ssh_connection_SUITE.erl @@ -1270,37 +1270,6 @@ test_exec_is_enabled(ConnectionRef, Exec, Expect) -> {fail,"Exec Timeout"} end. - -%%%---------------------------------------------------------------- -get_channel_close_sequence(ConnectionRef, ChannelId) -> - Set = [eof, exit_status, closed], - get_channel_close_sequence(ConnectionRef, ChannelId, Set). - -get_channel_close_sequence(_ConnectionRef, _ChannelId, []) -> - ok; -get_channel_close_sequence(ConnectionRef, ChannelId, Set) -> - receive - {ssh_cm, ConnectionRef, Event} when element(2,Event) == ChannelId -> - try lists:member(element(1,Event), Set) - of - true -> - ct:log("get_channel_close_sequence: ~p received", [Event]), - get_channel_close_sequence(ConnectionRef, Event, Set--[element(1,Event)]); - false -> - ct:log("~p:~p Got unexpected ~p~nExpecting ~p",[?MODULE,?LINE,Event,Set]), - ct:fail("Strange response 1") - catch - _ :_ -> - ct:log("~p:~p Got unexpected ~p~nExpecting ~p",[?MODULE,?LINE,Event,Set]), - ct:fail("Strange response 2") - end; - Msg -> - ct:log("~p:~p Got unexpected ~p~nExpecting event from the set ~p",[?MODULE,?LINE,Msg,Set]), - ct:fail("Strange response 3") - after - 10000 -> ct:fail("timeout ~p:~p",[?MODULE,?LINE]) - end. - %%%---------------------------------------------------------------- big_cat_rx(ConnectionRef, ChannelId) -> big_cat_rx(ConnectionRef, ChannelId, []). diff --git a/lib/ssh/test/ssh_dbg_SUITE.erl b/lib/ssh/test/ssh_dbg_SUITE.erl index ab7918fa90..8df9c8093a 100644 --- a/lib/ssh/test/ssh_dbg_SUITE.erl +++ b/lib/ssh/test/ssh_dbg_SUITE.erl @@ -82,6 +82,8 @@ end_per_testcase(_TC, Config) -> ok after 5000 -> + ct:log("~p:~p Messages:~n~p", + [?MODULE,?LINE, process_info(self(),messages)]), ssh_dbg:stop(), ssh:stop_daemon(Pid), ct:fail("No '~s' debug message",[ExpectPfx]) @@ -448,10 +450,15 @@ dbg_SKIP(Ref, Prefixes) -> dbg_SKIP(Ref, Prefixes, UnexpectedAcc) -> receive + {Ref, [_, _C, Msg]} when is_tuple(Msg) -> + %% filter non ssh_dbg messages, for example from dbg:tp(..) etc + dbg_SKIP(Ref, Prefixes, UnexpectedAcc); {Ref, [_, _C, Msg]=M} -> case lists:any( fun(Pfx) -> - lists:prefix(Pfx, Msg) + try lists:prefix(Pfx, Msg) + catch _:_ -> false + end end, Prefixes) of true -> ct:log("Skip:~n~p", [M]), diff --git a/lib/ssh/test/ssh_engine_SUITE.erl b/lib/ssh/test/ssh_engine_SUITE.erl index 3adb23acdb..241fb255cc 100644 --- a/lib/ssh/test/ssh_engine_SUITE.erl +++ b/lib/ssh/test/ssh_engine_SUITE.erl @@ -81,7 +81,7 @@ end_per_suite(Config) -> %%-------------------------------------------------------------------- init_per_group(dsa_key, Config) -> case lists:member('ssh-dss', - ssh_transport:default_algorithms(public_key)) of + ssh_transport:supported_algorithms(public_key)) of true -> start_daemon(Config, 'ssh-dss', "dsa_private_key.pem"); false -> @@ -89,7 +89,7 @@ init_per_group(dsa_key, Config) -> end; init_per_group(rsa_key, Config) -> case lists:member('ssh-rsa', - ssh_transport:default_algorithms(public_key)) of + ssh_transport:supported_algorithms(public_key)) of true -> start_daemon(Config, 'ssh-rsa', "rsa_private_key.pem"); false -> @@ -102,9 +102,11 @@ start_daemon(Config, KeyType, KeyId) -> KeyCBOpts = [{engine, proplists:get_value(engine,Config)}, {KeyType, FullKeyId} ], - Opts = [{key_cb, {ssh_key_cb_engine_keys, KeyCBOpts}}], + Opts = [{key_cb, {ssh_key_cb_engine_keys, KeyCBOpts}}, + {modify_algorithms, [{append, [{public_key,[KeyType]}]}]} + ], {Pid, Host, Port} = ssh_test_lib:std_daemon(Config, Opts), - [{host_port,{Host,Port}}, {daemon_pid,Pid}| Config]. + [{host_port,{Host,Port}}, {daemon_pid,Pid}, {key_type,KeyType}| Config]. end_per_group(_, Config) -> @@ -118,7 +120,11 @@ end_per_group(_, Config) -> %% A simple exec call simple_connect(Config) -> {Host,Port} = proplists:get_value(host_port, Config), - CRef = ssh_test_lib:std_connect(Config, Host, Port, []), + KeyType = proplists:get_value(key_type, Config), + Opts = [ + {modify_algorithms, [{append, [{public_key,[KeyType]}]}]} + ], + CRef = ssh_test_lib:std_connect(Config, Host, Port, Opts), ssh:close(CRef). %%-------------------------------------------------------------------- @@ -146,8 +152,3 @@ load_engine() -> {error, Error} end. -start_std_daemon(Opts, Config) -> - ct:log("starting std_daemon",[]), - {Pid, Host, Port} = ssh_test_lib:std_daemon(Config, Opts), - ct:log("started ~p:~p ~p",[Host,Port,Opts]), - [{srvr_pid,Pid},{srvr_addr,{Host,Port}} | Config]. diff --git a/lib/ssh/test/ssh_options_SUITE.erl b/lib/ssh/test/ssh_options_SUITE.erl index 93c4d7b7a7..4368761d02 100644 --- a/lib/ssh/test/ssh_options_SUITE.erl +++ b/lib/ssh/test/ssh_options_SUITE.erl @@ -92,7 +92,7 @@ suite() -> [{ct_hooks,[ts_install_cth]}, - {timetrap,{seconds,30}}]. + {timetrap,{seconds,60}}]. all() -> [connectfun_disconnectfun_server, @@ -128,9 +128,9 @@ all() -> id_string_own_string_server_trail_space, id_string_random_server, save_accepted_host_option, - {group, hardening_tests}, config_file, - config_file_modify_algorithms_order + config_file_modify_algorithms_order, + {group, hardening_tests} ]. groups() -> @@ -157,10 +157,8 @@ end_per_suite(_Config) -> %%-------------------------------------------------------------------- init_per_group(hardening_tests, Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_dsa(DataDir, PrivDir), - ssh_test_lib:setup_rsa(DataDir, PrivDir), + ct:log("Pub keys setup for: ~p", + [ssh_test_lib:setup_all_user_host_keys(Config)]), Config; init_per_group(dir_options, Config) -> PrivDir = proplists:get_value(priv_dir, Config), @@ -937,7 +935,7 @@ ssh_connect_arg4_timeout(_Config) -> Msp = ms_passed(T0), exit(Server,hasta_la_vista___baby), Low = 0.9*Timeout, - High = 2.5*Timeout, + High = 4.0*Timeout, ct:log("Timeout limits: ~.4f - ~.4f ms, timeout " "was ~.4f ms, expected ~p ms",[Low,High,Msp,Timeout]), if diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_dsa b/lib/ssh/test/ssh_options_SUITE_data/id_dsa index d306f8b26e..24628e071b 100644 --- a/lib/ssh/test/ssh_options_SUITE_data/id_dsa +++ b/lib/ssh/test/ssh_options_SUITE_data/id_dsa @@ -1,13 +1,12 @@ -----BEGIN DSA PRIVATE KEY----- -MIIBvAIBAAKBgQDfi2flSTZZofwT4yQT0NikX/LGNT7UPeB/XEWe/xovEYCElfaQ -APFixXvEgXwoojmZ5kiQRKzLM39wBP0jPERLbnZXfOOD0PDnw0haMh7dD7XKVMod -/EigVgHf/qBdM2M8yz1s/rRF7n1UpLSypziKjkzCm7JoSQ2zbWIPdmBIXwIVAMgP -kpr7Sq3O7sHdb8D601DRjoExAoGAMOQxDfB2Fd8ouz6G96f/UOzRMI/Kdv8kYYKW -JIGY+pRYrLPyYzUeJznwZreOJgrczAX+luHnKFWJ2Dnk5CyeXk67Wsr7pJ/4MBMD -OKeIS0S8qoSBN8+Krp79fgA+yS3IfqbkJLtLu4EBaCX4mKQIX4++k44d4U5lc8pt -+9hlEI8CgYEAznKxx9kyC6bVo7LUYKaGhofRFt0SYFc5PVmT2VUGRs1R6+6DPD+e -uEO6IhFct7JFSRbP9p0JD4Uk+3zlZF+XX6b2PsZkeV8f/02xlNGUSmEzCSiNg1AX -Cy/WusYhul0MncWCHMcOZB5rIvU/aP5EJJtn3xrRaz6u0SThF6AnT34CFQC63czE -ZU8w8Q+H7z0j+a+70x2iAw== +MIIBuwIBAAKBgQDIywHurUpOq6kZuMn+XlRzR4hAxF6qwSkuEqkV7iHnLQ0kIwf3 +uAmjFDhuEsQ8653SLxGVvTNp+KFFgDXiLqgM7TPUwDnpbvzEZHPAU+/zPt4sdY2D +txBfJwT2SFlK6HPOxOcxdDuD+/a59sh8hk/YVOU7ZTcBVsVG8Got4UcF5QIVAPGd +CPDQKSTlPiM9OwBB1+9p11k5AoGARLxw4l17mET9cU0uf4Ppe5nsCbODJv44ZrSs +picvypGVLrLcN5KWbm3vjRFCQ5LFunAG3FwLC2Sh0CH6TemoIfRPsRHR7wvpBGdr +c693UlMOis/mcmvNMQAzuQNW9WrxdzsvWR/r5s6NEHWqKUJGXSPi2d+Ijq/mCOmI +hzLzyiACgYEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4 +cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+C +ROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNgCFEjA7wTC +sQCY/I35vb6GUJn9tEdP -----END DSA PRIVATE KEY----- - diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_dsa.pub b/lib/ssh/test/ssh_options_SUITE_data/id_dsa.pub new file mode 100644 index 0000000000..018ef6f537 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_dsa.pub @@ -0,0 +1 @@ +ssh-dss AAAAB3NzaC1kc3MAAACBAMjLAe6tSk6rqRm4yf5eVHNHiEDEXqrBKS4SqRXuIectDSQjB/e4CaMUOG4SxDzrndIvEZW9M2n4oUWANeIuqAztM9TAOelu/MRkc8BT7/M+3ix1jYO3EF8nBPZIWUroc87E5zF0O4P79rn2yHyGT9hU5TtlNwFWxUbwai3hRwXlAAAAFQDxnQjw0Ckk5T4jPTsAQdfvaddZOQAAAIBEvHDiXXuYRP1xTS5/g+l7mewJs4Mm/jhmtKymJy/KkZUustw3kpZube+NEUJDksW6cAbcXAsLZKHQIfpN6agh9E+xEdHvC+kEZ2tzr3dSUw6Kz+Zya80xADO5A1b1avF3Oy9ZH+vmzo0QdaopQkZdI+LZ34iOr+YI6YiHMvPKIAAAAIEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+CROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNg= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa256 b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa256 new file mode 100644 index 0000000000..4b1eb12eaa --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIJfCaBKIIKhjbJl5F8BedqlXOQYDX5ba9Skypllmx/w+oAoGCCqGSM49 +AwEHoUQDQgAE49RbK2xQ/19ji3uDPM7uT4692LbwWF1TiaA9vUuebMGazoW/98br +N9xZu0L1AWwtEjs3kmJDTB7eJEGXnjUAcQ== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa256.pub b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa256.pub new file mode 100644 index 0000000000..a0147e60fa --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPUWytsUP9fY4t7gzzO7k+Ovdi28FhdU4mgPb1LnmzBms6Fv/fG6zfcWbtC9QFsLRI7N5JiQ0we3iRBl541AHE= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa384 b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa384 new file mode 100644 index 0000000000..4e8aa40959 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDCYXb6OSAZyXRfLXOtMo43za197Hdc/T0YKjgQQjwDt6rlRwqTh7v7S +PV2kXwNGdWigBwYFK4EEACKhZANiAARN2khlJUOOIiwsWHEALwDieeZR96qL4pUd +ci7aeGaczdUK5jOA9D9zmBZtSYTfO8Cr7ekVghDlcWAIJ/BXcswgQwSEQ6wyfaTF +8FYfyr4l3u9IirsnyaFzeIgeoNis8Gw= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa384.pub b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa384.pub new file mode 100644 index 0000000000..41e722e545 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE3aSGUlQ44iLCxYcQAvAOJ55lH3qovilR1yLtp4ZpzN1QrmM4D0P3OYFm1JhN87wKvt6RWCEOVxYAgn8FdyzCBDBIRDrDJ9pMXwVh/KviXe70iKuyfJoXN4iB6g2KzwbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa521 b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa521 new file mode 100644 index 0000000000..7196f46e97 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHbAgEBBEFMadoz4ckEcClfqXa2tiUuYkJdDfwq+/iFQcpt8ESuEd26IY/vm47Q +9UzbPkO4ou8xkNsQ3WvCRQBBWtn5O2kUU6AHBgUrgQQAI6GBiQOBhgAEAde5BRu5 +01/jS0jRk212xsb2DxPrxNpgp6IMCV8TA4Eps+8bSqHB091nLiBcP422HXYfuCd7 +XDjSs8ihcmhp0hCRASLqZR9EzW9W/SOt876May1Huj5X+WSO6RLe7vPn9vmf7kHf +pip6m7M7qp2qGgQ3q2vRwS2K/O6156ohiOlmuuFs +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa521.pub b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa521.pub new file mode 100644 index 0000000000..8f059120bc --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ecdsa521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHXuQUbudNf40tI0ZNtdsbG9g8T68TaYKeiDAlfEwOBKbPvG0qhwdPdZy4gXD+Nth12H7gne1w40rPIoXJoadIQkQEi6mUfRM1vVv0jrfO+jGstR7o+V/lkjukS3u7z5/b5n+5B36YqepuzO6qdqhoEN6tr0cEtivzuteeqIYjpZrrhbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ed25519 b/lib/ssh/test/ssh_options_SUITE_data/id_ed25519 new file mode 100644 index 0000000000..401a3e4a9a --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ed25519 @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnwAAAJg3+6xpN/us +aQAAAAtzc2gtZWQyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnw +AAAEBzC/Z2WGJhZ3l3tIBnUc6DCbp+lXY2yc2RRpWQTdf8sub0/z+ALQg4qbActK+SYS1L +ZfHRFGpQOoLrBjpSANWfAAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ed25519.pub b/lib/ssh/test/ssh_options_SUITE_data/id_ed25519.pub new file mode 100644 index 0000000000..a5c03b19c1 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ed25519.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOb0/z+ALQg4qbActK+SYS1LZfHRFGpQOoLrBjpSANWf uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ed448 b/lib/ssh/test/ssh_options_SUITE_data/id_ed448 new file mode 100644 index 0000000000..8ecfd710dc --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ed448 @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA53OqeePNaG/NJmoMbELhskKrAHNhLZ6AQm1WjbpMoseNl/OFh +1xznExpUPqTLX36fHYsAaWRHABQAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtf +fp8diwBpZEcAFAAAAAByzSPST3FCdOdENDI3uTKQ9RH2Ql+Y5kRZ/yA+iYUIP/32 +BQBVOrwOBc0CGEvbicTM1n4YeVEmfrMo3OqeePNaG/NJmoMbELhskKrAHNhLZ6AQ +m1WjbpMoseNl/OFh1xznExpUPqTLX36fHYsAaWRHABQAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_ed448.pub b/lib/ssh/test/ssh_options_SUITE_data/id_ed448.pub new file mode 100644 index 0000000000..cec0765a5d --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_ed448.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtffp8diwBpZEcAFAA= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_rsa b/lib/ssh/test/ssh_options_SUITE_data/id_rsa index 9d7e0dd5fb..2202c2ead8 100644 --- a/lib/ssh/test/ssh_options_SUITE_data/id_rsa +++ b/lib/ssh/test/ssh_options_SUITE_data/id_rsa @@ -1,15 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIICXAIBAAKBgQD1OET+3O/Bvj/dtjxDTXmj1oiJt4sIph5kGy0RfjoPrZfaS+CU -DhakCmS6t2ivxWFgtpKWaoGMZMJqWj6F6ZsumyFl3FPBtujwY/35cgifrI9Ns4Tl -zR1uuengNBmV+WRQ5cd9F2qS6Z8aDQihzt0r8JUqLcK+VQbrmNzboCCQQwIDAQAB -AoGAPQEyqPTt8JUT7mRXuaacjFXiweAXhp9NEDpyi9eLOjtFe9lElZCrsUOkq47V -TGUeRKEm9qSodfTbKPoqc8YaBJGJPhUaTAcha+7QcDdfHBvIsgxvU7ePVnlpXRp3 -CCUEMPhlnx6xBoTYP+fRU0e3+xJIPVyVCqX1jAdUMkzfRoECQQD6ux7B1QJAIWyK -SGkbDUbBilNmzCFNgIpOP6PA+bwfi5d16diTpra5AX09keQABAo/KaP1PdV8Vg0p -z4P3A7G3AkEA+l+AKG6m0kQTTBMJDqOdVPYwe+5GxunMaqmhokpEbuGsrZBl5Dvd -WpcBjR7jmenrhKZRIuA+Fz5HPo/UQJPl1QJBAKxstDkeED8j/S2XoFhPKAJ+6t39 -sUVICVTIZQeXdmzHJXCcUSkw8+WEhakqw/3SyW0oaK2FSWQJFWJUZ+8eJj8CQEh3 -xeduB5kKnS9CvzdeghZqX6QvVosSdtlUmfUYW/BgH5PpHKTP8wTaeld3XldZTpMJ -dKiMkUw2+XYROVUrubUCQD+Na1LhULlpn4ISEtIEfqpdlUhxDgO15Wg8USmsng+x -ICliVOSQtwaZjm8kwaFt0W7XnpnDxbRs37vIEbIMWak= +MIIEpAIBAAKCAQEAztjiyj2tdfkji0fewWS0kABg0IABgG20NvL1PnHJLr98we7w +W7f3j27EGjW/ApuycsWXXKi0L82q8uDicoHHb3JI2JkT70oi0yG1Dx/zwPN+dkA7 +LBT1J3UK2hJTFPhp855CwY/ss9xpBsd1Fv3zuHifEqNGljeg1PjmQ3pNhxA/M0aZ +cLnfIUdZ5Hr+t+4es3zaWo4tLBKmZu6BkVGQKPGXeMkIAMtJlG24l7qKDRkR5TYA +ZT7P8Vn7hnuFuCNbrJSm686GawBxTQXom23dg9UcWxoHB7UiHFoR6j0bQAX+4R7b +IwculRDcvzrgCu6u06oFILwY7MlsxpX9hGTl1wIDAQABAoIBAFeP6pmQeICrYceR +OhQGLIWVE2bP+VLDnflw6i5v/qlieE6kdm1tOEgorK0nuV9CR81cJdIcvIJL/yTn +3BR7KdDcwUenrY+rg4h7CWmIrigtK4ilciccDBeS7XAZN8B11GxDv6Cu65XMJU2w +W7nK8URTE4vRQI1QqS3e26MPAAi/LVOt3ZPI6zg/GHEwnq0IVSQAOndLBr/IWZk5 +SANrkfwX8WS7/UxZgDptT9dyUQ5Pnj5mieTlIvBwyczdhZ7RDa8HdCSHW3xF83V1 +A0pkn6+TRojumYyr4RrPQj6htE64Hgx9w1Dv/UINjPXl5mGlbxQHMWGzlqD/qpyI +wg7RakECgYEA+9ARZpHfEFz+EEFi8l9J+BtJDo00WaKCOZHh5UJ8W+NreqSd8nSx +5u6wYwMJjRX2Hwv+FBEhxGbo1+ff6p++cYmiSlDtN2XRCDkBWvvGlxu55BDULrhx +f8lqaV3XGmOy2rQusp8hiHmkmPJCSVj3oJqQnbqJ2zahXAx1rTPwHqECgYEA0kln +4h+ZkZ+aldOMGF0d0txTcTqZvsSVKiFTSD9of/fiSDqb6xtLT2+ys6FZoFL9lyK8 +gtqH642CDQ+3WT6Nmn4kMF5HNVpEuCeRDeRhiquWeKaAQDyvZ5ym1+Cn3GhsO7Di +d2LJKV5hOoN77loVY5nwnUVIJ0h+WLf0T7DTCXcCgYEAiNT7X50MdTvS4splFgcp +jqRlAn9AXySrVtUqxwVlxhjCIpapLUK0GSTCvEq+OeghIaXGnujgTHUPOaNKTZgY +SGHdyjxHar7s42b2kZYWx63NSVzLr8eSBTpRlIflhvV+DtGyPmWyNxLCmkmqM2kg +xii3RL5EgtYgwIAUwdVjOYECgYBRPlsMWfkS8f7fc+PkZdVn6gey71kHAxw+MrHi +b90H09Vw4nPq2Zi3EAiSrfvanTWsdpcuVw+8See89B16NViwH5wLs+D/E+kI3QCF +xX6J/NEdu/ZA2zFJbpRnQzyXQyDNzwEv7tKZUQVvfe0boWIyIP99Q48k3jUyQZ/6 +Se6+8QKBgQCXl8H2K3CsZxoujKLb2qoEOPbxJQ2hxoMTS5XuQECReIVsNuptWrur +DF8WJi/B6AqwRX1P3l56RNwqB1yDBqv0QVLpU7vU/FmWqLWTn0r3AvM74qftvfAE +oa31wcYoCqPJoKgCG7TThLhNt2v5hL7sVgZNO0ueAiHhJbFLaf7ceg== -----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/id_rsa.pub b/lib/ssh/test/ssh_options_SUITE_data/id_rsa.pub new file mode 100644 index 0000000000..b4084d320a --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO2OLKPa11+SOLR97BZLSQAGDQgAGAbbQ28vU+cckuv3zB7vBbt/ePbsQaNb8Cm7JyxZdcqLQvzary4OJygcdvckjYmRPvSiLTIbUPH/PA8352QDssFPUndQraElMU+GnznkLBj+yz3GkGx3UW/fO4eJ8So0aWN6DU+OZDek2HED8zRplwud8hR1nkev637h6zfNpaji0sEqZm7oGRUZAo8Zd4yQgAy0mUbbiXuooNGRHlNgBlPs/xWfuGe4W4I1uslKbrzoZrAHFNBeibbd2D1RxbGgcHtSIcWhHqPRtABf7hHtsjBy6VENy/OuAK7q7TqgUgvBjsyWzGlf2EZOXX uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key256 b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key256 new file mode 100644 index 0000000000..2979ea88ed --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMe4MDoit0t8RzSVPwkCBemQ9fhXL+xnTSAWISw8HNCioAoGCCqGSM49 +AwEHoUQDQgAEo2q7U3P6r0W5WGOLtM78UQtofM9UalEhiZeDdiyylsR/RR17Op0s +VPGSADLmzzgcucLEKy17j2S+oz42VUJy5A== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key256.pub b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key256.pub new file mode 100644 index 0000000000..85dc419345 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNqu1Nz+q9FuVhji7TO/FELaHzPVGpRIYmXg3YsspbEf0UdezqdLFTxkgAy5s84HLnCxCste49kvqM+NlVCcuQ= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key384 b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key384 new file mode 100644 index 0000000000..fb1a862ded --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDArxbDfh3p1okrD9wQw6jJ4d4DdlBPD5GqXE8bIeRJiK41Sh40LgvPw +mkqEDSXK++CgBwYFK4EEACKhZANiAAScl43Ih2lWTDKrSox5ve5uiTXil4smsup3 +CfS1XPjKxgBAmlfBim8izbdrT0BFdQzz2joduNMtpt61wO4rGs6jm0UP7Kim9PC7 +Hneb/99fIYopdMH5NMnk60zGO1uZ2vc= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key384.pub b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key384.pub new file mode 100644 index 0000000000..428d5fb7d7 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJyXjciHaVZMMqtKjHm97m6JNeKXiyay6ncJ9LVc+MrGAECaV8GKbyLNt2tPQEV1DPPaOh240y2m3rXA7isazqObRQ/sqKb08Lsed5v/318hiil0wfk0yeTrTMY7W5na9w== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key521 b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key521 new file mode 100644 index 0000000000..3e51ec2ecd --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHcAgEBBEIB8O1BFkl2HQjQLRLonEZ97da/h39DMa9/0/hvPZWAI8gUPEQcHxRx +U7b09p3Zh+EBbMFq8+1ae9ds+ZTxE4WFSvKgBwYFK4EEACOhgYkDgYYABAAlWVjq +Bzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/ +vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5 +ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key521.pub b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key521.pub new file mode 100644 index 0000000000..017a29f4da --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ecdsa_key521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAAlWVjqBzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed25519_key b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed25519_key new file mode 100644 index 0000000000..13a8fcf491 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed25519_key @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQAAAJi+h4O7voeD +uwAAAAtzc2gtZWQyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQ +AAAEBaOcJfGPNemKc1wPHTCmM4Kwvh6dZ0CqY14UT361UnN0lI66JgZZo72XJ7wGBp+h3W +TDw/pxXddvaomAIHrIl9AAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed25519_key.pub b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed25519_key.pub new file mode 100644 index 0000000000..156ef4045c --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed25519_key.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIElI66JgZZo72XJ7wGBp+h3WTDw/pxXddvaomAIHrIl9 uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed448_key b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed448_key new file mode 100644 index 0000000000..31a7e4e8c3 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed448_key @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed448_key.pub b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed448_key.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/ssh_options_SUITE_data/ssh_host_ed448_key.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_protocol_SUITE.erl b/lib/ssh/test/ssh_protocol_SUITE.erl index 228e0dcbd8..3a08523039 100644 --- a/lib/ssh/test/ssh_protocol_SUITE.erl +++ b/lib/ssh/test/ssh_protocol_SUITE.erl @@ -907,10 +907,8 @@ stop_apps(_Config) -> setup_dirs(Config) -> - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_dsa(DataDir, PrivDir), - ssh_test_lib:setup_rsa(DataDir, PrivDir), + ct:log("Pub keys setup for: ~p", + [ssh_test_lib:setup_all_user_host_keys(Config)]), Config. system_dir(Config) -> filename:join(proplists:get_value(priv_dir, Config), system). diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa b/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa index d306f8b26e..24628e071b 100644 --- a/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa @@ -1,13 +1,12 @@ -----BEGIN DSA PRIVATE KEY----- -MIIBvAIBAAKBgQDfi2flSTZZofwT4yQT0NikX/LGNT7UPeB/XEWe/xovEYCElfaQ -APFixXvEgXwoojmZ5kiQRKzLM39wBP0jPERLbnZXfOOD0PDnw0haMh7dD7XKVMod -/EigVgHf/qBdM2M8yz1s/rRF7n1UpLSypziKjkzCm7JoSQ2zbWIPdmBIXwIVAMgP -kpr7Sq3O7sHdb8D601DRjoExAoGAMOQxDfB2Fd8ouz6G96f/UOzRMI/Kdv8kYYKW -JIGY+pRYrLPyYzUeJznwZreOJgrczAX+luHnKFWJ2Dnk5CyeXk67Wsr7pJ/4MBMD -OKeIS0S8qoSBN8+Krp79fgA+yS3IfqbkJLtLu4EBaCX4mKQIX4++k44d4U5lc8pt -+9hlEI8CgYEAznKxx9kyC6bVo7LUYKaGhofRFt0SYFc5PVmT2VUGRs1R6+6DPD+e -uEO6IhFct7JFSRbP9p0JD4Uk+3zlZF+XX6b2PsZkeV8f/02xlNGUSmEzCSiNg1AX -Cy/WusYhul0MncWCHMcOZB5rIvU/aP5EJJtn3xrRaz6u0SThF6AnT34CFQC63czE -ZU8w8Q+H7z0j+a+70x2iAw== +MIIBuwIBAAKBgQDIywHurUpOq6kZuMn+XlRzR4hAxF6qwSkuEqkV7iHnLQ0kIwf3 +uAmjFDhuEsQ8653SLxGVvTNp+KFFgDXiLqgM7TPUwDnpbvzEZHPAU+/zPt4sdY2D +txBfJwT2SFlK6HPOxOcxdDuD+/a59sh8hk/YVOU7ZTcBVsVG8Got4UcF5QIVAPGd +CPDQKSTlPiM9OwBB1+9p11k5AoGARLxw4l17mET9cU0uf4Ppe5nsCbODJv44ZrSs +picvypGVLrLcN5KWbm3vjRFCQ5LFunAG3FwLC2Sh0CH6TemoIfRPsRHR7wvpBGdr +c693UlMOis/mcmvNMQAzuQNW9WrxdzsvWR/r5s6NEHWqKUJGXSPi2d+Ijq/mCOmI +hzLzyiACgYEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4 +cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+C +ROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNgCFEjA7wTC +sQCY/I35vb6GUJn9tEdP -----END DSA PRIVATE KEY----- - diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa.pub new file mode 100644 index 0000000000..018ef6f537 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_dsa.pub @@ -0,0 +1 @@ +ssh-dss AAAAB3NzaC1kc3MAAACBAMjLAe6tSk6rqRm4yf5eVHNHiEDEXqrBKS4SqRXuIectDSQjB/e4CaMUOG4SxDzrndIvEZW9M2n4oUWANeIuqAztM9TAOelu/MRkc8BT7/M+3ix1jYO3EF8nBPZIWUroc87E5zF0O4P79rn2yHyGT9hU5TtlNwFWxUbwai3hRwXlAAAAFQDxnQjw0Ckk5T4jPTsAQdfvaddZOQAAAIBEvHDiXXuYRP1xTS5/g+l7mewJs4Mm/jhmtKymJy/KkZUustw3kpZube+NEUJDksW6cAbcXAsLZKHQIfpN6agh9E+xEdHvC+kEZ2tzr3dSUw6Kz+Zya80xADO5A1b1avF3Oy9ZH+vmzo0QdaopQkZdI+LZ34iOr+YI6YiHMvPKIAAAAIEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+CROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNg= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa256 b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa256 new file mode 100644 index 0000000000..4b1eb12eaa --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIJfCaBKIIKhjbJl5F8BedqlXOQYDX5ba9Skypllmx/w+oAoGCCqGSM49 +AwEHoUQDQgAE49RbK2xQ/19ji3uDPM7uT4692LbwWF1TiaA9vUuebMGazoW/98br +N9xZu0L1AWwtEjs3kmJDTB7eJEGXnjUAcQ== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa256.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa256.pub new file mode 100644 index 0000000000..a0147e60fa --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPUWytsUP9fY4t7gzzO7k+Ovdi28FhdU4mgPb1LnmzBms6Fv/fG6zfcWbtC9QFsLRI7N5JiQ0we3iRBl541AHE= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa384 b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa384 new file mode 100644 index 0000000000..4e8aa40959 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDCYXb6OSAZyXRfLXOtMo43za197Hdc/T0YKjgQQjwDt6rlRwqTh7v7S +PV2kXwNGdWigBwYFK4EEACKhZANiAARN2khlJUOOIiwsWHEALwDieeZR96qL4pUd +ci7aeGaczdUK5jOA9D9zmBZtSYTfO8Cr7ekVghDlcWAIJ/BXcswgQwSEQ6wyfaTF +8FYfyr4l3u9IirsnyaFzeIgeoNis8Gw= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa384.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa384.pub new file mode 100644 index 0000000000..41e722e545 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE3aSGUlQ44iLCxYcQAvAOJ55lH3qovilR1yLtp4ZpzN1QrmM4D0P3OYFm1JhN87wKvt6RWCEOVxYAgn8FdyzCBDBIRDrDJ9pMXwVh/KviXe70iKuyfJoXN4iB6g2KzwbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa521 b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa521 new file mode 100644 index 0000000000..7196f46e97 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHbAgEBBEFMadoz4ckEcClfqXa2tiUuYkJdDfwq+/iFQcpt8ESuEd26IY/vm47Q +9UzbPkO4ou8xkNsQ3WvCRQBBWtn5O2kUU6AHBgUrgQQAI6GBiQOBhgAEAde5BRu5 +01/jS0jRk212xsb2DxPrxNpgp6IMCV8TA4Eps+8bSqHB091nLiBcP422HXYfuCd7 +XDjSs8ihcmhp0hCRASLqZR9EzW9W/SOt876May1Huj5X+WSO6RLe7vPn9vmf7kHf +pip6m7M7qp2qGgQ3q2vRwS2K/O6156ohiOlmuuFs +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa521.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa521.pub new file mode 100644 index 0000000000..8f059120bc --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ecdsa521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHXuQUbudNf40tI0ZNtdsbG9g8T68TaYKeiDAlfEwOBKbPvG0qhwdPdZy4gXD+Nth12H7gne1w40rPIoXJoadIQkQEi6mUfRM1vVv0jrfO+jGstR7o+V/lkjukS3u7z5/b5n+5B36YqepuzO6qdqhoEN6tr0cEtivzuteeqIYjpZrrhbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ed25519 b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed25519 new file mode 100644 index 0000000000..401a3e4a9a --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed25519 @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnwAAAJg3+6xpN/us +aQAAAAtzc2gtZWQyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnw +AAAEBzC/Z2WGJhZ3l3tIBnUc6DCbp+lXY2yc2RRpWQTdf8sub0/z+ALQg4qbActK+SYS1L +ZfHRFGpQOoLrBjpSANWfAAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ed25519.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed25519.pub new file mode 100644 index 0000000000..a5c03b19c1 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed25519.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOb0/z+ALQg4qbActK+SYS1LZfHRFGpQOoLrBjpSANWf uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ed448 b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed448 new file mode 100644 index 0000000000..8ecfd710dc --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed448 @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA53OqeePNaG/NJmoMbELhskKrAHNhLZ6AQm1WjbpMoseNl/OFh +1xznExpUPqTLX36fHYsAaWRHABQAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtf +fp8diwBpZEcAFAAAAAByzSPST3FCdOdENDI3uTKQ9RH2Ql+Y5kRZ/yA+iYUIP/32 +BQBVOrwOBc0CGEvbicTM1n4YeVEmfrMo3OqeePNaG/NJmoMbELhskKrAHNhLZ6AQ +m1WjbpMoseNl/OFh1xznExpUPqTLX36fHYsAaWRHABQAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_ed448.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed448.pub new file mode 100644 index 0000000000..cec0765a5d --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_ed448.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtffp8diwBpZEcAFAA= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa b/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa index 9d7e0dd5fb..2202c2ead8 100644 --- a/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa @@ -1,15 +1,27 @@ -----BEGIN RSA PRIVATE KEY----- -MIICXAIBAAKBgQD1OET+3O/Bvj/dtjxDTXmj1oiJt4sIph5kGy0RfjoPrZfaS+CU -DhakCmS6t2ivxWFgtpKWaoGMZMJqWj6F6ZsumyFl3FPBtujwY/35cgifrI9Ns4Tl -zR1uuengNBmV+WRQ5cd9F2qS6Z8aDQihzt0r8JUqLcK+VQbrmNzboCCQQwIDAQAB -AoGAPQEyqPTt8JUT7mRXuaacjFXiweAXhp9NEDpyi9eLOjtFe9lElZCrsUOkq47V -TGUeRKEm9qSodfTbKPoqc8YaBJGJPhUaTAcha+7QcDdfHBvIsgxvU7ePVnlpXRp3 -CCUEMPhlnx6xBoTYP+fRU0e3+xJIPVyVCqX1jAdUMkzfRoECQQD6ux7B1QJAIWyK -SGkbDUbBilNmzCFNgIpOP6PA+bwfi5d16diTpra5AX09keQABAo/KaP1PdV8Vg0p -z4P3A7G3AkEA+l+AKG6m0kQTTBMJDqOdVPYwe+5GxunMaqmhokpEbuGsrZBl5Dvd -WpcBjR7jmenrhKZRIuA+Fz5HPo/UQJPl1QJBAKxstDkeED8j/S2XoFhPKAJ+6t39 -sUVICVTIZQeXdmzHJXCcUSkw8+WEhakqw/3SyW0oaK2FSWQJFWJUZ+8eJj8CQEh3 -xeduB5kKnS9CvzdeghZqX6QvVosSdtlUmfUYW/BgH5PpHKTP8wTaeld3XldZTpMJ -dKiMkUw2+XYROVUrubUCQD+Na1LhULlpn4ISEtIEfqpdlUhxDgO15Wg8USmsng+x -ICliVOSQtwaZjm8kwaFt0W7XnpnDxbRs37vIEbIMWak= +MIIEpAIBAAKCAQEAztjiyj2tdfkji0fewWS0kABg0IABgG20NvL1PnHJLr98we7w +W7f3j27EGjW/ApuycsWXXKi0L82q8uDicoHHb3JI2JkT70oi0yG1Dx/zwPN+dkA7 +LBT1J3UK2hJTFPhp855CwY/ss9xpBsd1Fv3zuHifEqNGljeg1PjmQ3pNhxA/M0aZ +cLnfIUdZ5Hr+t+4es3zaWo4tLBKmZu6BkVGQKPGXeMkIAMtJlG24l7qKDRkR5TYA +ZT7P8Vn7hnuFuCNbrJSm686GawBxTQXom23dg9UcWxoHB7UiHFoR6j0bQAX+4R7b +IwculRDcvzrgCu6u06oFILwY7MlsxpX9hGTl1wIDAQABAoIBAFeP6pmQeICrYceR +OhQGLIWVE2bP+VLDnflw6i5v/qlieE6kdm1tOEgorK0nuV9CR81cJdIcvIJL/yTn +3BR7KdDcwUenrY+rg4h7CWmIrigtK4ilciccDBeS7XAZN8B11GxDv6Cu65XMJU2w +W7nK8URTE4vRQI1QqS3e26MPAAi/LVOt3ZPI6zg/GHEwnq0IVSQAOndLBr/IWZk5 +SANrkfwX8WS7/UxZgDptT9dyUQ5Pnj5mieTlIvBwyczdhZ7RDa8HdCSHW3xF83V1 +A0pkn6+TRojumYyr4RrPQj6htE64Hgx9w1Dv/UINjPXl5mGlbxQHMWGzlqD/qpyI +wg7RakECgYEA+9ARZpHfEFz+EEFi8l9J+BtJDo00WaKCOZHh5UJ8W+NreqSd8nSx +5u6wYwMJjRX2Hwv+FBEhxGbo1+ff6p++cYmiSlDtN2XRCDkBWvvGlxu55BDULrhx +f8lqaV3XGmOy2rQusp8hiHmkmPJCSVj3oJqQnbqJ2zahXAx1rTPwHqECgYEA0kln +4h+ZkZ+aldOMGF0d0txTcTqZvsSVKiFTSD9of/fiSDqb6xtLT2+ys6FZoFL9lyK8 +gtqH642CDQ+3WT6Nmn4kMF5HNVpEuCeRDeRhiquWeKaAQDyvZ5ym1+Cn3GhsO7Di +d2LJKV5hOoN77loVY5nwnUVIJ0h+WLf0T7DTCXcCgYEAiNT7X50MdTvS4splFgcp +jqRlAn9AXySrVtUqxwVlxhjCIpapLUK0GSTCvEq+OeghIaXGnujgTHUPOaNKTZgY +SGHdyjxHar7s42b2kZYWx63NSVzLr8eSBTpRlIflhvV+DtGyPmWyNxLCmkmqM2kg +xii3RL5EgtYgwIAUwdVjOYECgYBRPlsMWfkS8f7fc+PkZdVn6gey71kHAxw+MrHi +b90H09Vw4nPq2Zi3EAiSrfvanTWsdpcuVw+8See89B16NViwH5wLs+D/E+kI3QCF +xX6J/NEdu/ZA2zFJbpRnQzyXQyDNzwEv7tKZUQVvfe0boWIyIP99Q48k3jUyQZ/6 +Se6+8QKBgQCXl8H2K3CsZxoujKLb2qoEOPbxJQ2hxoMTS5XuQECReIVsNuptWrur +DF8WJi/B6AqwRX1P3l56RNwqB1yDBqv0QVLpU7vU/FmWqLWTn0r3AvM74qftvfAE +oa31wcYoCqPJoKgCG7TThLhNt2v5hL7sVgZNO0ueAiHhJbFLaf7ceg== -----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa.pub b/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa.pub new file mode 100644 index 0000000000..b4084d320a --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO2OLKPa11+SOLR97BZLSQAGDQgAGAbbQ28vU+cckuv3zB7vBbt/ePbsQaNb8Cm7JyxZdcqLQvzary4OJygcdvckjYmRPvSiLTIbUPH/PA8352QDssFPUndQraElMU+GnznkLBj+yz3GkGx3UW/fO4eJ8So0aWN6DU+OZDek2HED8zRplwud8hR1nkev637h6zfNpaji0sEqZm7oGRUZAo8Zd4yQgAy0mUbbiXuooNGRHlNgBlPs/xWfuGe4W4I1uslKbrzoZrAHFNBeibbd2D1RxbGgcHtSIcWhHqPRtABf7hHtsjBy6VENy/OuAK7q7TqgUgvBjsyWzGlf2EZOXX uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key256 b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key256 new file mode 100644 index 0000000000..2979ea88ed --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMe4MDoit0t8RzSVPwkCBemQ9fhXL+xnTSAWISw8HNCioAoGCCqGSM49 +AwEHoUQDQgAEo2q7U3P6r0W5WGOLtM78UQtofM9UalEhiZeDdiyylsR/RR17Op0s +VPGSADLmzzgcucLEKy17j2S+oz42VUJy5A== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key256.pub b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key256.pub new file mode 100644 index 0000000000..85dc419345 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNqu1Nz+q9FuVhji7TO/FELaHzPVGpRIYmXg3YsspbEf0UdezqdLFTxkgAy5s84HLnCxCste49kvqM+NlVCcuQ= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key384 b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key384 new file mode 100644 index 0000000000..fb1a862ded --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDArxbDfh3p1okrD9wQw6jJ4d4DdlBPD5GqXE8bIeRJiK41Sh40LgvPw +mkqEDSXK++CgBwYFK4EEACKhZANiAAScl43Ih2lWTDKrSox5ve5uiTXil4smsup3 +CfS1XPjKxgBAmlfBim8izbdrT0BFdQzz2joduNMtpt61wO4rGs6jm0UP7Kim9PC7 +Hneb/99fIYopdMH5NMnk60zGO1uZ2vc= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key384.pub b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key384.pub new file mode 100644 index 0000000000..428d5fb7d7 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJyXjciHaVZMMqtKjHm97m6JNeKXiyay6ncJ9LVc+MrGAECaV8GKbyLNt2tPQEV1DPPaOh240y2m3rXA7isazqObRQ/sqKb08Lsed5v/318hiil0wfk0yeTrTMY7W5na9w== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key521 b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key521 new file mode 100644 index 0000000000..3e51ec2ecd --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHcAgEBBEIB8O1BFkl2HQjQLRLonEZ97da/h39DMa9/0/hvPZWAI8gUPEQcHxRx +U7b09p3Zh+EBbMFq8+1ae9ds+ZTxE4WFSvKgBwYFK4EEACOhgYkDgYYABAAlWVjq +Bzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/ +vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5 +ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key521.pub b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key521.pub new file mode 100644 index 0000000000..017a29f4da --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ecdsa_key521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAAlWVjqBzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed25519_key b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed25519_key new file mode 100644 index 0000000000..13a8fcf491 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed25519_key @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQAAAJi+h4O7voeD +uwAAAAtzc2gtZWQyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQ +AAAEBaOcJfGPNemKc1wPHTCmM4Kwvh6dZ0CqY14UT361UnN0lI66JgZZo72XJ7wGBp+h3W +TDw/pxXddvaomAIHrIl9AAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed25519_key.pub b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed25519_key.pub new file mode 100644 index 0000000000..156ef4045c --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed25519_key.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIElI66JgZZo72XJ7wGBp+h3WTDw/pxXddvaomAIHrIl9 uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed448_key b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed448_key new file mode 100644 index 0000000000..31a7e4e8c3 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed448_key @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed448_key.pub b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed448_key.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/ssh_protocol_SUITE_data/ssh_host_ed448_key.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_pubkey_SUITE.erl b/lib/ssh/test/ssh_pubkey_SUITE.erl index fbc17e5028..f1afb496c1 100644 --- a/lib/ssh/test/ssh_pubkey_SUITE.erl +++ b/lib/ssh/test/ssh_pubkey_SUITE.erl @@ -36,7 +36,8 @@ suite() -> all() -> [{group, old_format}, - {group, new_format} + {group, new_format}, + {group, option_space} ]. @@ -51,19 +52,31 @@ all() -> connect_ecdsa_to_ecdsa ]). --define(tests_new, [connect_ecdsa_to_ed25519, - connect_rsa_to_ed25519, +-define(tests_new, [ connect_dsa_to_ed25519, - connect_ed25519_to_rsa, + connect_dsa_to_ed448, + connect_ecdsa_to_ed25519, + connect_ecdsa_to_ed448, connect_ed25519_to_dsa, connect_ed25519_to_ecdsa, - connect_ed25519_to_ed25519 - | ?tests_old]). + connect_ed25519_to_ed448, + connect_ed25519_to_ed25519, + connect_ed25519_to_rsa, + connect_ed448_to_dsa, + connect_ed448_to_ecdsa, + connect_ed448_to_ed25519, + connect_ed448_to_ed448, + connect_ed448_to_rsa, + connect_rsa_to_ed25519, + connect_rsa_to_ed448 + | ?tests_old % but taken from the new format directory + ]). groups() -> - [{new_format, [], ?tests_new}, - {old_format, [], ?tests_old++[{group,passphrase}]}, - {passphrase, [], ?tests_old} + [{new_format, [], ?tests_new}, + {old_format, [], ?tests_old++[{group,passphrase}]}, + {passphrase, [], ?tests_old}, + {option_space,[], [{group,new_format}]} ]. %%%---------------------------------------------------------------- @@ -71,7 +84,8 @@ init_per_suite(Config) -> ?CHECK_CRYPTO( begin ssh:start(), - [{client_opts,[]} + [{client_opts,[]}, + {daemon_opts,[]} | Config] end). @@ -89,6 +103,11 @@ init_per_group(old_format, Config) -> [{fmt,old_format}, {key_src_dir,Dir} | Config]; +init_per_group(option_space, Config) -> + extend_optsL([client_opts,daemon_opts], + [{key_cb, {ssh_file, [{optimize, space}]}}], + Config); + init_per_group(passphrase, Config0) -> case supported(hashs, md5) of true -> @@ -110,7 +129,11 @@ extend_opts(OptName, Value, Config) -> Opts = proplists:get_value(OptName, Config), replace_opt(OptName, [Value|Opts], Config). -extend_optsL(OptName, Values, Config) -> +extend_optsL(OptNames, Values, Config) when is_list(OptNames) -> + lists:foldl(fun(N, Cnf) -> + extend_optsL(N, Values, Cnf) + end, Config, OptNames); +extend_optsL(OptName, Values, Config) when is_atom(OptName) -> Opts = proplists:get_value(OptName, Config), replace_opt(OptName, Values ++ Opts, Config). @@ -131,6 +154,8 @@ init_per_testcase(connect_rsa_to_ecdsa, Config0) -> setup_user_system_dir(rsa, ecdsa, Config0); init_per_testcase(connect_rsa_to_ed25519, Config0) -> setup_user_system_dir(rsa, ed25519, Config0); +init_per_testcase(connect_rsa_to_ed448, Config0) -> + setup_user_system_dir(rsa, ed448, Config0); init_per_testcase(connect_dsa_to_rsa, Config0) -> setup_user_system_dir(dsa, rsa, Config0); init_per_testcase(connect_dsa_to_dsa, Config0) -> @@ -139,6 +164,8 @@ init_per_testcase(connect_dsa_to_ecdsa, Config0) -> setup_user_system_dir(dsa, ecdsa, Config0); init_per_testcase(connect_dsa_to_ed25519, Config0) -> setup_user_system_dir(dsa, ed25519, Config0); +init_per_testcase(connect_dsa_to_ed448, Config0) -> + setup_user_system_dir(dsa, ed448, Config0); init_per_testcase(connect_ecdsa_to_rsa, Config0) -> setup_user_system_dir(ecdsa, rsa, Config0); init_per_testcase(connect_ecdsa_to_dsa, Config0) -> @@ -147,6 +174,8 @@ init_per_testcase(connect_ecdsa_to_ecdsa, Config0) -> setup_user_system_dir(ecdsa, ecdsa, Config0); init_per_testcase(connect_ecdsa_to_ed25519, Config0) -> setup_user_system_dir(ecdsa, ed25519, Config0); +init_per_testcase(connect_ecdsa_to_ed448, Config0) -> + setup_user_system_dir(ecdsa, ed448, Config0); init_per_testcase(connect_ed25519_to_rsa, Config0) -> setup_user_system_dir(ed25519, rsa, Config0); init_per_testcase(connect_ed25519_to_dsa, Config0) -> @@ -155,6 +184,18 @@ init_per_testcase(connect_ed25519_to_ecdsa, Config0) -> setup_user_system_dir(ed25519, ecdsa, Config0); init_per_testcase(connect_ed25519_to_ed25519, Config0) -> setup_user_system_dir(ed25519, ed25519, Config0); +init_per_testcase(connect_ed25519_to_ed448, Config0) -> + setup_user_system_dir(ed25519, ed448, Config0); +init_per_testcase(connect_ed448_to_rsa, Config0) -> + setup_user_system_dir(ed448, rsa, Config0); +init_per_testcase(connect_ed448_to_dsa, Config0) -> + setup_user_system_dir(ed448, dsa, Config0); +init_per_testcase(connect_ed448_to_ecdsa, Config0) -> + setup_user_system_dir(ed448, ecdsa, Config0); +init_per_testcase(connect_ed448_to_ed25519, Config0) -> + setup_user_system_dir(ed448, ed25519, Config0); +init_per_testcase(connect_ed448_to_ed448, Config0) -> + setup_user_system_dir(ed448, ed448, Config0); init_per_testcase(_, Config) -> Config. @@ -176,6 +217,9 @@ connect_rsa_to_ecdsa(Config) -> connect_rsa_to_ed25519(Config) -> try_connect(Config). +connect_rsa_to_ed448(Config) -> + try_connect(Config). + connect_dsa_to_rsa(Config) -> try_connect(Config). @@ -188,6 +232,9 @@ connect_dsa_to_ecdsa(Config) -> connect_dsa_to_ed25519(Config) -> try_connect(Config). +connect_dsa_to_ed448(Config) -> + try_connect(Config). + connect_ecdsa_to_rsa(Config) -> try_connect(Config). @@ -200,6 +247,9 @@ connect_ecdsa_to_ecdsa(Config) -> connect_ecdsa_to_ed25519(Config) -> try_connect(Config). +connect_ecdsa_to_ed448(Config) -> + try_connect(Config). + connect_ed25519_to_rsa(Config) -> try_connect(Config). @@ -212,6 +262,24 @@ connect_ed25519_to_ecdsa(Config) -> connect_ed25519_to_ed25519(Config) -> try_connect(Config). +connect_ed25519_to_ed448(Config) -> + try_connect(Config). + +connect_ed448_to_rsa(Config) -> + try_connect(Config). + +connect_ed448_to_dsa(Config) -> + try_connect(Config). + +connect_ed448_to_ecdsa(Config) -> + try_connect(Config). + +connect_ed448_to_ed25519(Config) -> + try_connect(Config). + +connect_ed448_to_ed448(Config) -> + try_connect(Config). + %%%---------------------------------------------------------------- try_connect({skip,Reson}) -> @@ -220,9 +288,12 @@ try_connect(Config) -> SystemDir = proplists:get_value(system_dir, Config), UserDir = proplists:get_value(user_dir, Config), ClientOpts = proplists:get_value(client_opts, Config, []), + DaemonOpts = proplists:get_value(daemon_opts, Config, []), {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, - {user_dir, UserDir}]), + {user_dir, UserDir} + | DaemonOpts]), + C = ssh_test_lib:connect(Host, Port, [{user_dir, UserDir}, {silently_accept_hosts, true}, {user_interaction, false} @@ -250,18 +321,32 @@ setup_user_system_dir(ClientAlg, ServerAlg, Config) -> HostSrcFile = filename:join(KeySrcDir, file(host,ServerAlg)), HostDstFile = filename:join(SystemDir, file(host,ServerAlg)), - {ok,_} = file:copy(HostSrcFile, HostDstFile), UserSrcFile = filename:join(KeySrcDir, file(user,ClientAlg)), UserDstFile = filename:join(UserDir, file(user,ClientAlg)), - {ok,_} = file:copy(UserSrcFile, UserDstFile), UserPubSrcFile = filename:join(KeySrcDir, file(user,ClientAlg)++".pub"), AuthorizedKeys = filename:join(UserDir, "authorized_keys"), - {ok,_} = file:copy(UserPubSrcFile, AuthorizedKeys), - [{system_dir,SystemDir}, - {user_dir,UserDir} | Config]; + try + {ok,_} = file:copy(UserSrcFile, UserDstFile), + {ok,_} = file:copy(UserPubSrcFile, AuthorizedKeys), + {ok,_} = file:copy(HostSrcFile, HostDstFile) + of + _ -> + ModAlgs = [{modify_algorithms, + [{append,[{public_key, + lists:usort([alg(ClientAlg), + alg(ServerAlg)])}]}]} + ], + [{system_dir,SystemDir}, + {user_dir,UserDir} + | extend_optsL([daemon_opts,client_opts], ModAlgs, Config)] + catch + error:{badmatch,{error,enoent}}:S -> + ct:log("~p:~p Stack:~n~p", [?MODULE,?LINE,S]), + {skip, no_key_file_found} + end; false -> {skip, unsupported_algorithm} @@ -271,12 +356,20 @@ setup_user_system_dir(ClientAlg, ServerAlg, Config) -> file(host, dsa) -> "ssh_host_dsa_key"; file(host, ecdsa) -> "ssh_host_ecdsa_key"; file(host, ed25519) -> "ssh_host_ed25519_key"; +file(host, ed448) -> "ssh_host_ed448_key"; file(host, rsa) -> "ssh_host_rsa_key"; file(user, dsa) -> "id_dsa"; file(user, ecdsa) -> "id_ecdsa"; file(user, ed25519) -> "id_ed25519"; +file(user, ed448) -> "id_ed448"; file(user, rsa) -> "id_rsa". +alg(dsa) -> 'ssh-dss'; +alg(ecdsa) -> 'ecdsa-sha2-nistp256'; +alg(ed25519) -> 'ssh-ed25519'; +alg(ed448) -> 'ssh-ed448'; +alg(rsa) -> 'ssh-rsa'. + supported(public_keys, rsa) -> supported(public_key, 'ssh-rsa') orelse supported(public_key, 'rsa-sha2-256') orelse @@ -288,7 +381,7 @@ supported(public_keys, ecdsa) -> supported(public_key, 'ecdsa-sha2-nistp256') supported(public_keys, ed448) -> supported(public_key, 'ssh-ed448'); supported(public_keys, ed25519) -> supported(public_key, 'ssh-ed25519'); supported(Type, Alg) -> - case proplists:get_value(Type,ssh:default_algorithms()) of + case proplists:get_value(Type,ssh_transport:supported_algorithms()) of undefined -> lists:member(Alg, crypto:supports(Type)); L -> diff --git a/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/id_ed448 b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/id_ed448 new file mode 100644 index 0000000000..fc8e79e8f0 --- /dev/null +++ b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/id_ed448 @@ -0,0 +1,15 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +A: Key +SomeKey: Very long \ +line \ +spanning more lines \ + +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/id_ed448.pub b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/id_ed448.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/id_ed448.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/ssh_host_ed448_key b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/ssh_host_ed448_key new file mode 100644 index 0000000000..fc8e79e8f0 --- /dev/null +++ b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/ssh_host_ed448_key @@ -0,0 +1,15 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +A: Key +SomeKey: Very long \ +line \ +spanning more lines \ + +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/ssh_host_ed448_key.pub b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/ssh_host_ed448_key.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/ssh_pubkey_SUITE_data/new_format/ssh_host_ed448_key.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_renegotiate_SUITE.erl b/lib/ssh/test/ssh_renegotiate_SUITE.erl new file mode 100644 index 0000000000..b08a5ab1eb --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE.erl @@ -0,0 +1,393 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2020. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% + +%% + +-module(ssh_renegotiate_SUITE). + +-include_lib("common_test/include/ct.hrl"). +-include_lib("kernel/include/inet.hrl"). +-include_lib("kernel/include/file.hrl"). +-include("ssh_test_lib.hrl"). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-define(NEWLINE, <<"\r\n">>). + +-define(REKEY_DATA_TMO, 1 * 60000). % Should be multiples of 60000 + +%%-------------------------------------------------------------------- +%% Common Test interface functions ----------------------------------- +%%-------------------------------------------------------------------- + +suite() -> + [{ct_hooks,[ts_install_cth]}, + {timetrap,{seconds,90}}]. + +all() -> + [{group, renegotiate} + ]. + +groups() -> + [{renegotiate, [parallel], [rekey0, + rekey1, + rekey2, + rekey3, + rekey4, + rekey_limit_client, + rekey_limit_daemon, + rekey_time_limit_client, + rekey_time_limit_daemon, + norekey_limit_client, + norekey_limit_daemon, + renegotiate1, + renegotiate2]} + ]. + +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + ?CHECK_CRYPTO(begin + ssh:start(), + ct:log("Pub keys setup for: ~p", + [ssh_test_lib:setup_all_user_host_keys(Config)]), + [{preferred_algorithms,ssh_transport:supported_algorithms()} + | Config] + end). + +end_per_suite(_Config) -> + ssh:stop(). + + +init_per_group(_, Config) -> Config. + +end_per_group(_, Config) -> Config. +%%---------------------------------------------------------------------------- +%%% Idle timeout test +rekey0() -> [{timetrap,{seconds,120}}]. +rekey1() -> [{timetrap,{seconds,120}}]. +rekey2() -> [{timetrap,{seconds,120}}]. +rekey3() -> [{timetrap,{seconds,120}}]. +rekey4() -> [{timetrap,{seconds,120}}]. + +rekey0(Config) -> rekey_chk(Config, 0, 0). +rekey1(Config) -> rekey_chk(Config, infinity, 0). +rekey2(Config) -> rekey_chk(Config, {infinity,infinity}, 0). +rekey3(Config) -> rekey_chk(Config, 0, infinity). +rekey4(Config) -> rekey_chk(Config, 0, {infinity,infinity}). + +rekey_chk(Config, RLdaemon, RLclient) -> + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config, [{rekey_limit, RLdaemon}]), + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, RLclient}]), + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + + %% Make both sides send something: + {ok, _SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + %% Check rekeying + timer:sleep(?REKEY_DATA_TMO), + ?wait_match(false, Kex1==ssh_test_lib:get_kex_init(ConnectionRef), [], 2000, 10), + + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- +%%% Test rekeying by data volume + +rekey_limit_client() -> [{timetrap,{seconds,500}}]. +rekey_limit_client(Config) -> + Limit = 6000, + UserDir = proplists:get_value(priv_dir, Config), + DataFile = filename:join(UserDir, "rekey.data"), + Data = lists:duplicate(Limit+10,1), + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, Limit}, + {max_random_length_padding,0}]), + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + %% Check that it doesn't rekey without data transfer + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), + + %% Check that datatransfer triggers rekeying + ok = ssh_sftp:write_file(SftpPid, DataFile, Data), + timer:sleep(?REKEY_DATA_TMO), + ?wait_match(false, Kex1==(Kex2=ssh_test_lib:get_kex_init(ConnectionRef)), Kex2, 2000, 10), + + %% Check that datatransfer continues to trigger rekeying + ok = ssh_sftp:write_file(SftpPid, DataFile, Data), + timer:sleep(?REKEY_DATA_TMO), + ?wait_match(false, Kex2==(Kex3=ssh_test_lib:get_kex_init(ConnectionRef)), Kex3, 2000, 10), + + %% Check that it doesn't rekey without data transfer + timer:sleep(?REKEY_DATA_TMO), + true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), + + %% Check that it doesn't rekey on a small datatransfer + ok = ssh_sftp:write_file(SftpPid, DataFile, "hi\n"), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), + + %% Check that it doesn't rekey without data transfer + timer:sleep(?REKEY_DATA_TMO), + true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), + + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + + + +rekey_limit_daemon() -> [{timetrap,{seconds,500}}]. +rekey_limit_daemon(Config) -> + Limit = 6000, + UserDir = proplists:get_value(priv_dir, Config), + DataFile1 = filename:join(UserDir, "rekey1.data"), + DataFile2 = filename:join(UserDir, "rekey2.data"), + file:write_file(DataFile1, lists:duplicate(Limit+10,1)), + file:write_file(DataFile2, "hi\n"), + + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{rekey_limit, Limit}, + {max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{max_random_length_padding,0}]), + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + %% Check that it doesn't rekey without data transfer + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + timer:sleep(?REKEY_DATA_TMO), + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + + %% Check that datatransfer triggers rekeying + {ok,_} = ssh_sftp:read_file(SftpPid, DataFile1), + timer:sleep(?REKEY_DATA_TMO), + ?wait_match(false, Kex1==(Kex2=ssh_test_lib:get_kex_init(ConnectionRef)), Kex2, 2000, 10), + + %% Check that datatransfer continues to trigger rekeying + {ok,_} = ssh_sftp:read_file(SftpPid, DataFile1), + timer:sleep(?REKEY_DATA_TMO), + ?wait_match(false, Kex2==(Kex3=ssh_test_lib:get_kex_init(ConnectionRef)), Kex3, 2000, 10), + + %% Check that it doesn't rekey without data transfer + timer:sleep(?REKEY_DATA_TMO), + true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), + + %% Check that it doesn't rekey on a small datatransfer + {ok,_} = ssh_sftp:read_file(SftpPid, DataFile2), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), + + %% Check that it doesn't rekey without data transfer + timer:sleep(?REKEY_DATA_TMO), + true = (Kex3 == ssh_test_lib:get_kex_init(ConnectionRef)), + + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + + +%%-------------------------------------------------------------------- +%% Check that datatransfer in the other direction does not trigger re-keying +norekey_limit_client() -> [{timetrap,{seconds,500}}]. +norekey_limit_client(Config) -> + Limit = 6000, + UserDir = proplists:get_value(priv_dir, Config), + DataFile = filename:join(UserDir, "rekey3.data"), + file:write_file(DataFile, lists:duplicate(Limit+10,1)), + + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, Limit}, + {max_random_length_padding,0}]), + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), + + {ok,_} = ssh_sftp:read_file(SftpPid, DataFile), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), + + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%% Check that datatransfer in the other direction does not trigger re-keying +norekey_limit_daemon() -> [{timetrap,{seconds,500}}]. +norekey_limit_daemon(Config) -> + Limit = 6000, + UserDir = proplists:get_value(priv_dir, Config), + DataFile = filename:join(UserDir, "rekey4.data"), + + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{rekey_limit, Limit}, + {max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{max_random_length_padding,0}]), + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), + + ok = ssh_sftp:write_file(SftpPid, DataFile, lists:duplicate(Limit+10,1)), + timer:sleep(?REKEY_DATA_TMO), + true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), + + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- +%%% Test rekeying by time + +rekey_time_limit_client() -> [{timetrap,{seconds,500}}]. +rekey_time_limit_client(Config) -> + Minutes = ?REKEY_DATA_TMO div 60000, + GB = 1024*1000*1000, + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{rekey_limit, {Minutes, GB}}, + {max_random_length_padding,0}]), + rekey_time_limit(Pid, ConnectionRef). + +rekey_time_limit_daemon() -> [{timetrap,{seconds,500}}]. +rekey_time_limit_daemon(Config) -> + Minutes = ?REKEY_DATA_TMO div 60000, + GB = 1024*1000*1000, + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, Port} = ssh_test_lib:std_daemon(Config,[{rekey_limit, {Minutes, GB}}, + {max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + ConnectionRef = ssh_test_lib:std_connect(Config, Host, Port, [{max_random_length_padding,0}]), + rekey_time_limit(Pid, ConnectionRef). + + +rekey_time_limit(Pid, ConnectionRef) -> + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + + timer:sleep(5000), + true = (Kex1 == ssh_test_lib:get_kex_init(ConnectionRef)), + + %% Check that it rekeys when the max time + 30s has passed + timer:sleep(?REKEY_DATA_TMO + 30*1000), + ?wait_match(false, Kex1==(Kex2=ssh_test_lib:get_kex_init(ConnectionRef)), Kex2, 2000, 10), + + %% Check that it does not rekey when nothing is transferred + timer:sleep(?REKEY_DATA_TMO + 30*1000), + ?wait_match(false, Kex2==ssh_test_lib:get_kex_init(ConnectionRef), [], 2000, 10), + + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- + +%%% Test rekeying with simultaneous send request + +renegotiate1(Config) -> + UserDir = proplists:get_value(priv_dir, Config), + DataFile = filename:join(UserDir, "renegotiate1.data"), + + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, DPort} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + + {ok,RelayPid,_,RPort} = ssh_relay:start_link({0,0,0,0}, 0, Host, DPort), + + ConnectionRef = ssh_test_lib:std_connect(Config, Host, RPort, [{max_random_length_padding,0}]), + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + + {ok, Handle} = ssh_sftp:open(SftpPid, DataFile, [write]), + + ok = ssh_sftp:write(SftpPid, Handle, "hi\n"), + + ssh_relay:hold(RelayPid, rx, 20, 1000), + ssh_connection_handler:renegotiate(ConnectionRef), + spawn(fun() -> ok=ssh_sftp:write(SftpPid, Handle, "another hi\n") end), + + timer:sleep(2000), + + Kex2 = ssh_test_lib:get_kex_init(ConnectionRef), + + false = (Kex2 == Kex1), + + ssh_relay:stop(RelayPid), + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- + +%%% Test rekeying with inflight messages from peer + +renegotiate2(Config) -> + UserDir = proplists:get_value(priv_dir, Config), + DataFile = filename:join(UserDir, "renegotiate2.data"), + + Algs = proplists:get_value(preferred_algorithms, Config), + {Pid, Host, DPort} = ssh_test_lib:std_daemon(Config,[{max_random_length_padding,0}, + {preferred_algorithms,Algs}]), + + {ok,RelayPid,_,RPort} = ssh_relay:start_link({0,0,0,0}, 0, Host, DPort), + + ConnectionRef = ssh_test_lib:std_connect(Config, Host, RPort, [{max_random_length_padding,0}]), + {ok, SftpPid} = ssh_sftp:start_channel(ConnectionRef), + + Kex1 = ssh_test_lib:get_kex_init(ConnectionRef), + + {ok, Handle} = ssh_sftp:open(SftpPid, DataFile, [write]), + + ok = ssh_sftp:write(SftpPid, Handle, "hi\n"), + + ssh_relay:hold(RelayPid, rx, 20, infinity), + spawn(fun() -> ok=ssh_sftp:write(SftpPid, Handle, "another hi\n") end), + %% need a small pause here to ensure ssh_sftp:write is executed + ct:sleep(10), + ssh_connection_handler:renegotiate(ConnectionRef), + ssh_relay:release(RelayPid, rx), + + timer:sleep(2000), + + Kex2 = ssh_test_lib:get_kex_init(ConnectionRef), + + false = (Kex2 == Kex1), + + ssh_relay:stop(RelayPid), + ssh_sftp:stop_channel(SftpPid), + ssh:close(ConnectionRef), + ssh:stop_daemon(Pid). + +%%-------------------------------------------------------------------- +%% Internal functions ------------------------------------------------ +%%-------------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_dsa b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_dsa new file mode 100644 index 0000000000..24628e071b --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_dsa @@ -0,0 +1,12 @@ +-----BEGIN DSA PRIVATE KEY----- +MIIBuwIBAAKBgQDIywHurUpOq6kZuMn+XlRzR4hAxF6qwSkuEqkV7iHnLQ0kIwf3 +uAmjFDhuEsQ8653SLxGVvTNp+KFFgDXiLqgM7TPUwDnpbvzEZHPAU+/zPt4sdY2D +txBfJwT2SFlK6HPOxOcxdDuD+/a59sh8hk/YVOU7ZTcBVsVG8Got4UcF5QIVAPGd +CPDQKSTlPiM9OwBB1+9p11k5AoGARLxw4l17mET9cU0uf4Ppe5nsCbODJv44ZrSs +picvypGVLrLcN5KWbm3vjRFCQ5LFunAG3FwLC2Sh0CH6TemoIfRPsRHR7wvpBGdr +c693UlMOis/mcmvNMQAzuQNW9WrxdzsvWR/r5s6NEHWqKUJGXSPi2d+Ijq/mCOmI +hzLzyiACgYEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4 +cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+C +ROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNgCFEjA7wTC +sQCY/I35vb6GUJn9tEdP +-----END DSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_dsa.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_dsa.pub new file mode 100644 index 0000000000..018ef6f537 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_dsa.pub @@ -0,0 +1 @@ +ssh-dss AAAAB3NzaC1kc3MAAACBAMjLAe6tSk6rqRm4yf5eVHNHiEDEXqrBKS4SqRXuIectDSQjB/e4CaMUOG4SxDzrndIvEZW9M2n4oUWANeIuqAztM9TAOelu/MRkc8BT7/M+3ix1jYO3EF8nBPZIWUroc87E5zF0O4P79rn2yHyGT9hU5TtlNwFWxUbwai3hRwXlAAAAFQDxnQjw0Ckk5T4jPTsAQdfvaddZOQAAAIBEvHDiXXuYRP1xTS5/g+l7mewJs4Mm/jhmtKymJy/KkZUustw3kpZube+NEUJDksW6cAbcXAsLZKHQIfpN6agh9E+xEdHvC+kEZ2tzr3dSUw6Kz+Zya80xADO5A1b1avF3Oy9ZH+vmzo0QdaopQkZdI+LZ34iOr+YI6YiHMvPKIAAAAIEAsTRcHZqZlamr0PM7jKt2edCpcd8rEFGtWuescebc6Ga5JGSv7Ue4cdYKpAjT10Mns1WYaU9t6ZR+6ARP7DkzzDmS1elwkRu21T+b81PmeZwaEJxgqr+CROQVHgzpqMqEx8ic3c/juxZpRrCAlRCjCWSJLDMobBQvtfyG0qsleNg= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa256 b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa256 new file mode 100644 index 0000000000..4b1eb12eaa --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIJfCaBKIIKhjbJl5F8BedqlXOQYDX5ba9Skypllmx/w+oAoGCCqGSM49 +AwEHoUQDQgAE49RbK2xQ/19ji3uDPM7uT4692LbwWF1TiaA9vUuebMGazoW/98br +N9xZu0L1AWwtEjs3kmJDTB7eJEGXnjUAcQ== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa256.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa256.pub new file mode 100644 index 0000000000..a0147e60fa --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOPUWytsUP9fY4t7gzzO7k+Ovdi28FhdU4mgPb1LnmzBms6Fv/fG6zfcWbtC9QFsLRI7N5JiQ0we3iRBl541AHE= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa384 b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa384 new file mode 100644 index 0000000000..4e8aa40959 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDCYXb6OSAZyXRfLXOtMo43za197Hdc/T0YKjgQQjwDt6rlRwqTh7v7S +PV2kXwNGdWigBwYFK4EEACKhZANiAARN2khlJUOOIiwsWHEALwDieeZR96qL4pUd +ci7aeGaczdUK5jOA9D9zmBZtSYTfO8Cr7ekVghDlcWAIJ/BXcswgQwSEQ6wyfaTF +8FYfyr4l3u9IirsnyaFzeIgeoNis8Gw= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa384.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa384.pub new file mode 100644 index 0000000000..41e722e545 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE3aSGUlQ44iLCxYcQAvAOJ55lH3qovilR1yLtp4ZpzN1QrmM4D0P3OYFm1JhN87wKvt6RWCEOVxYAgn8FdyzCBDBIRDrDJ9pMXwVh/KviXe70iKuyfJoXN4iB6g2KzwbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa521 b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa521 new file mode 100644 index 0000000000..7196f46e97 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHbAgEBBEFMadoz4ckEcClfqXa2tiUuYkJdDfwq+/iFQcpt8ESuEd26IY/vm47Q +9UzbPkO4ou8xkNsQ3WvCRQBBWtn5O2kUU6AHBgUrgQQAI6GBiQOBhgAEAde5BRu5 +01/jS0jRk212xsb2DxPrxNpgp6IMCV8TA4Eps+8bSqHB091nLiBcP422HXYfuCd7 +XDjSs8ihcmhp0hCRASLqZR9EzW9W/SOt876May1Huj5X+WSO6RLe7vPn9vmf7kHf +pip6m7M7qp2qGgQ3q2vRwS2K/O6156ohiOlmuuFs +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa521.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa521.pub new file mode 100644 index 0000000000..8f059120bc --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ecdsa521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHXuQUbudNf40tI0ZNtdsbG9g8T68TaYKeiDAlfEwOBKbPvG0qhwdPdZy4gXD+Nth12H7gne1w40rPIoXJoadIQkQEi6mUfRM1vVv0jrfO+jGstR7o+V/lkjukS3u7z5/b5n+5B36YqepuzO6qdqhoEN6tr0cEtivzuteeqIYjpZrrhbA== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed25519 b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed25519 new file mode 100644 index 0000000000..401a3e4a9a --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed25519 @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnwAAAJg3+6xpN/us +aQAAAAtzc2gtZWQyNTUxOQAAACDm9P8/gC0IOKmwHLSvkmEtS2Xx0RRqUDqC6wY6UgDVnw +AAAEBzC/Z2WGJhZ3l3tIBnUc6DCbp+lXY2yc2RRpWQTdf8sub0/z+ALQg4qbActK+SYS1L +ZfHRFGpQOoLrBjpSANWfAAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed25519.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed25519.pub new file mode 100644 index 0000000000..a5c03b19c1 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed25519.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOb0/z+ALQg4qbActK+SYS1LZfHRFGpQOoLrBjpSANWf uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed448 b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed448 new file mode 100644 index 0000000000..8ecfd710dc --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed448 @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA53OqeePNaG/NJmoMbELhskKrAHNhLZ6AQm1WjbpMoseNl/OFh +1xznExpUPqTLX36fHYsAaWRHABQAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtf +fp8diwBpZEcAFAAAAAByzSPST3FCdOdENDI3uTKQ9RH2Ql+Y5kRZ/yA+iYUIP/32 +BQBVOrwOBc0CGEvbicTM1n4YeVEmfrMo3OqeePNaG/NJmoMbELhskKrAHNhLZ6AQ +m1WjbpMoseNl/OFh1xznExpUPqTLX36fHYsAaWRHABQAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed448.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed448.pub new file mode 100644 index 0000000000..cec0765a5d --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_ed448.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADnc6p5481ob80magxsQuGyQqsAc2EtnoBCbVaNukyix42X84WHXHOcTGlQ+pMtffp8diwBpZEcAFAA= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_rsa b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_rsa new file mode 100644 index 0000000000..2202c2ead8 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_rsa @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEAztjiyj2tdfkji0fewWS0kABg0IABgG20NvL1PnHJLr98we7w +W7f3j27EGjW/ApuycsWXXKi0L82q8uDicoHHb3JI2JkT70oi0yG1Dx/zwPN+dkA7 +LBT1J3UK2hJTFPhp855CwY/ss9xpBsd1Fv3zuHifEqNGljeg1PjmQ3pNhxA/M0aZ +cLnfIUdZ5Hr+t+4es3zaWo4tLBKmZu6BkVGQKPGXeMkIAMtJlG24l7qKDRkR5TYA +ZT7P8Vn7hnuFuCNbrJSm686GawBxTQXom23dg9UcWxoHB7UiHFoR6j0bQAX+4R7b +IwculRDcvzrgCu6u06oFILwY7MlsxpX9hGTl1wIDAQABAoIBAFeP6pmQeICrYceR +OhQGLIWVE2bP+VLDnflw6i5v/qlieE6kdm1tOEgorK0nuV9CR81cJdIcvIJL/yTn +3BR7KdDcwUenrY+rg4h7CWmIrigtK4ilciccDBeS7XAZN8B11GxDv6Cu65XMJU2w +W7nK8URTE4vRQI1QqS3e26MPAAi/LVOt3ZPI6zg/GHEwnq0IVSQAOndLBr/IWZk5 +SANrkfwX8WS7/UxZgDptT9dyUQ5Pnj5mieTlIvBwyczdhZ7RDa8HdCSHW3xF83V1 +A0pkn6+TRojumYyr4RrPQj6htE64Hgx9w1Dv/UINjPXl5mGlbxQHMWGzlqD/qpyI +wg7RakECgYEA+9ARZpHfEFz+EEFi8l9J+BtJDo00WaKCOZHh5UJ8W+NreqSd8nSx +5u6wYwMJjRX2Hwv+FBEhxGbo1+ff6p++cYmiSlDtN2XRCDkBWvvGlxu55BDULrhx +f8lqaV3XGmOy2rQusp8hiHmkmPJCSVj3oJqQnbqJ2zahXAx1rTPwHqECgYEA0kln +4h+ZkZ+aldOMGF0d0txTcTqZvsSVKiFTSD9of/fiSDqb6xtLT2+ys6FZoFL9lyK8 +gtqH642CDQ+3WT6Nmn4kMF5HNVpEuCeRDeRhiquWeKaAQDyvZ5ym1+Cn3GhsO7Di +d2LJKV5hOoN77loVY5nwnUVIJ0h+WLf0T7DTCXcCgYEAiNT7X50MdTvS4splFgcp +jqRlAn9AXySrVtUqxwVlxhjCIpapLUK0GSTCvEq+OeghIaXGnujgTHUPOaNKTZgY +SGHdyjxHar7s42b2kZYWx63NSVzLr8eSBTpRlIflhvV+DtGyPmWyNxLCmkmqM2kg +xii3RL5EgtYgwIAUwdVjOYECgYBRPlsMWfkS8f7fc+PkZdVn6gey71kHAxw+MrHi +b90H09Vw4nPq2Zi3EAiSrfvanTWsdpcuVw+8See89B16NViwH5wLs+D/E+kI3QCF +xX6J/NEdu/ZA2zFJbpRnQzyXQyDNzwEv7tKZUQVvfe0boWIyIP99Q48k3jUyQZ/6 +Se6+8QKBgQCXl8H2K3CsZxoujKLb2qoEOPbxJQ2hxoMTS5XuQECReIVsNuptWrur +DF8WJi/B6AqwRX1P3l56RNwqB1yDBqv0QVLpU7vU/FmWqLWTn0r3AvM74qftvfAE +oa31wcYoCqPJoKgCG7TThLhNt2v5hL7sVgZNO0ueAiHhJbFLaf7ceg== +-----END RSA PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/id_rsa.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_rsa.pub new file mode 100644 index 0000000000..b4084d320a --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/id_rsa.pub @@ -0,0 +1 @@ +ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO2OLKPa11+SOLR97BZLSQAGDQgAGAbbQ28vU+cckuv3zB7vBbt/ePbsQaNb8Cm7JyxZdcqLQvzary4OJygcdvckjYmRPvSiLTIbUPH/PA8352QDssFPUndQraElMU+GnznkLBj+yz3GkGx3UW/fO4eJ8So0aWN6DU+OZDek2HED8zRplwud8hR1nkev637h6zfNpaji0sEqZm7oGRUZAo8Zd4yQgAy0mUbbiXuooNGRHlNgBlPs/xWfuGe4W4I1uslKbrzoZrAHFNBeibbd2D1RxbGgcHtSIcWhHqPRtABf7hHtsjBy6VENy/OuAK7q7TqgUgvBjsyWzGlf2EZOXX uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_dsa_key b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_dsa_key new file mode 100644 index 0000000000..51ab6fbd88 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_dsa_key @@ -0,0 +1,13 @@ +-----BEGIN DSA PRIVATE KEY----- +MIIBuwIBAAKBgQCClaHzE2ul0gKSUxah5W0W8UiJLy4hXngKEqpaUq9SSdVdY2LK +wVfKH1gt5iuaf1FfzOhsIC9G/GLnjYttXZc92cv/Gfe3gR+s0ni2++MX+T++mE/Q +diltXv/Hp27PybS67SmiFW7I+RWnT2OKlMPtw2oUuKeztCe5UWjaj/y5FQIVAPLA +l9RpiU30Z87NRAHY3NTRaqtrAoGANMRxw8UfdtNVR0CrQj3AgPaXOGE4d+G4Gp4X +skvnCHycSVAjtYxebUkzUzt5Q6f/IabuLUdge3gXrc8BetvrcKbp+XZgM0/Vj2CF +Ymmy3in6kzGZq7Fw1sZaku6AOU8vLa5woBT2vAcHLLT1bLAzj7viL048T6MfjrOP +ef8nHvACgYBhDWFQJ1mf99sg92LalVq1dHLmVXb3PTJDfCO/Gz5NFmj9EZbAtdah +/XcF3DeRF+eEoz48wQF/ExVxSMIhLdL+o+ElpVhlM7Yii+T7dPhkQfEul6zZXu+U +ykSTXYUbtsfTNRFQGBW2/GfnEc0mnIxfn9v10NEWMzlq5z9wT9P0CgIVAN4wtL5W +Lv62jKcdskxNyz2NQoBx +-----END DSA PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_dsa_key.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_dsa_key.pub new file mode 100644 index 0000000000..4dbb1305b0 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_dsa_key.pub @@ -0,0 +1,11 @@ +---- BEGIN SSH2 PUBLIC KEY ---- +AAAAB3NzaC1kc3MAAACBAIKVofMTa6XSApJTFqHlbRbxSIkvLiFeeAoSqlpSr1JJ1V1j +YsrBV8ofWC3mK5p/UV/M6GwgL0b8YueNi21dlz3Zy/8Z97eBH6zSeLb74xf5P76YT9B2 +KW1e/8enbs/JtLrtKaIVbsj5FadPY4qUw+3DahS4p7O0J7lRaNqP/LkVAAAAFQDywJfU +aYlN9GfOzUQB2NzU0WqrawAAAIA0xHHDxR9201VHQKtCPcCA9pc4YTh34bganheyS+cI +fJxJUCO1jF5tSTNTO3lDp/8hpu4tR2B7eBetzwF62+twpun5dmAzT9WPYIViabLeKfqT +MZmrsXDWxlqS7oA5Ty8trnCgFPa8BwcstPVssDOPu+IvTjxPox+Os495/yce8AAAAIBh +DWFQJ1mf99sg92LalVq1dHLmVXb3PTJDfCO/Gz5NFmj9EZbAtdah/XcF3DeRF+eEoz48 +wQF/ExVxSMIhLdL+o+ElpVhlM7Yii+T7dPhkQfEul6zZXu+UykSTXYUbtsfTNRFQGBW2 +/GfnEc0mnIxfn9v10NEWMzlq5z9wT9P0Cg== +---- END SSH2 PUBLIC KEY ---- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key256 b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key256 new file mode 100644 index 0000000000..2979ea88ed --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key256 @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMe4MDoit0t8RzSVPwkCBemQ9fhXL+xnTSAWISw8HNCioAoGCCqGSM49 +AwEHoUQDQgAEo2q7U3P6r0W5WGOLtM78UQtofM9UalEhiZeDdiyylsR/RR17Op0s +VPGSADLmzzgcucLEKy17j2S+oz42VUJy5A== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key256.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key256.pub new file mode 100644 index 0000000000..85dc419345 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key256.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNqu1Nz+q9FuVhji7TO/FELaHzPVGpRIYmXg3YsspbEf0UdezqdLFTxkgAy5s84HLnCxCste49kvqM+NlVCcuQ= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key384 b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key384 new file mode 100644 index 0000000000..fb1a862ded --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key384 @@ -0,0 +1,6 @@ +-----BEGIN EC PRIVATE KEY----- +MIGkAgEBBDArxbDfh3p1okrD9wQw6jJ4d4DdlBPD5GqXE8bIeRJiK41Sh40LgvPw +mkqEDSXK++CgBwYFK4EEACKhZANiAAScl43Ih2lWTDKrSox5ve5uiTXil4smsup3 +CfS1XPjKxgBAmlfBim8izbdrT0BFdQzz2joduNMtpt61wO4rGs6jm0UP7Kim9PC7 +Hneb/99fIYopdMH5NMnk60zGO1uZ2vc= +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key384.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key384.pub new file mode 100644 index 0000000000..428d5fb7d7 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key384.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJyXjciHaVZMMqtKjHm97m6JNeKXiyay6ncJ9LVc+MrGAECaV8GKbyLNt2tPQEV1DPPaOh240y2m3rXA7isazqObRQ/sqKb08Lsed5v/318hiil0wfk0yeTrTMY7W5na9w== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key521 b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key521 new file mode 100644 index 0000000000..3e51ec2ecd --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key521 @@ -0,0 +1,7 @@ +-----BEGIN EC PRIVATE KEY----- +MIHcAgEBBEIB8O1BFkl2HQjQLRLonEZ97da/h39DMa9/0/hvPZWAI8gUPEQcHxRx +U7b09p3Zh+EBbMFq8+1ae9ds+ZTxE4WFSvKgBwYFK4EEACOhgYkDgYYABAAlWVjq +Bzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/ +vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5 +ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key521.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key521.pub new file mode 100644 index 0000000000..017a29f4da --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ecdsa_key521.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAAlWVjqBzg7Wt4gE6UNb1lRE2cnlmH2L/A5uo6qZRx5lPnSKOxEhxSb/Oay1+9d6KRdrh6/vlhd9SHDBhLcAPDvWgBnJIEj92Q3pXX4JtoitL0yl+SvvU+vUh966mzHShHzj8p5ccOgPkPNoA70yrpGzkIhPezpZOQdCaOXj/jFqNCTDg== uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed25519_key b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed25519_key new file mode 100644 index 0000000000..13a8fcf491 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed25519_key @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQAAAJi+h4O7voeD +uwAAAAtzc2gtZWQyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQ +AAAEBaOcJfGPNemKc1wPHTCmM4Kwvh6dZ0CqY14UT361UnN0lI66JgZZo72XJ7wGBp+h3W +TDw/pxXddvaomAIHrIl9AAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed25519_key.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed25519_key.pub new file mode 100644 index 0000000000..156ef4045c --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed25519_key.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIElI66JgZZo72XJ7wGBp+h3WTDw/pxXddvaomAIHrIl9 uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed448_key b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed448_key new file mode 100644 index 0000000000..31a7e4e8c3 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed448_key @@ -0,0 +1,10 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAASgAAAAlz +c2gtZWQ0NDgAAAA5X9dEm1m0Yf0s54fsYWrUah2hNCSFpw4fig6nXYDpZ3jt8SR2 +m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAA0AAAEREAABERAAAACXNzaC1lZDQ0OAAA +ADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHl +D2zR+hq+r+glYYAAAABybIKlYsuAjRDWMr6JyFE+v2ySnzTd+oyfY8mWDvbjSKNS +jIo/zC8ETjmj/FuUSS+PAy51SaIAmPlbX9dEm1m0Yf0s54fsYWrUah2hNCSFpw4f +ig6nXYDpZ3jt8SR2m0bHBhvWeD3x5Q9s0foavq/oJWGAAAAAAAECAwQ= +-----END OPENSSH PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed448_key.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed448_key.pub new file mode 100644 index 0000000000..8c390dcb58 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_ed448_key.pub @@ -0,0 +1 @@ +ssh-ed448 AAAACXNzaC1lZDQ0OAAAADlf10SbWbRh/Sznh+xhatRqHaE0JIWnDh+KDqddgOlneO3xJHabRscGG9Z4PfHlD2zR+hq+r+glYYA= diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_rsa_key b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_rsa_key new file mode 100644 index 0000000000..79968bdd7d --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_rsa_key @@ -0,0 +1,16 @@ +-----BEGIN RSA PRIVATE KEY----- +MIICXQIBAAKBgQDCZX+4FBDwZIh9y/Uxee1VJnEXlowpz2yDKwj8semM4q843337 +zbNfxHmladB1lpz2NqyxI175xMIJuDxogyZdsOxGnFAzAnthR4dqL/RWRWzjaxSB +6IAO9SPYVVlrpZ+1hsjLW79fwXK/yc8VdhRuWTeQiRgYY2ek8+OKbOqz4QIDAQAB +AoGANmvJzJO5hkLuvyDZHKfAnGTtpifcR1wtSa9DjdKUyn8vhKF0mIimnbnYQEmW +NUUb3gXCZLi9PvkpRSVRrASDOZwcjoU/Kvww163vBUVb2cOZfFhyn6o2Sk88Tt++ +udH3hdjpf9i7jTtUkUe+QYPsia+wgvvrmn4QrahLAH86+kECQQDx5gFeXTME3cnW +WMpFz3PPumduzjqgqMMWEccX4FtQkMX/gyGa5UC7OHFyh0N/gSWvPbRHa8A6YgIt +n8DO+fh5AkEAzbqX4DOn8NY6xJIi42q7l/2jIA0RkB6P7YugW5NblhqBZ0XDnpA5 +sMt+rz+K07u9XZtxgh1xi7mNfwY6lEAMqQJBAJBEauCKmRj35Z6OyeQku59SPsnY ++SJEREVvSNw2lH9SOKQQ4wPsYlTGbvKtNVZgAcen91L5MmYfeckYE/fdIZECQQCt +64zxsTnM1I8iFxj/gP/OYlJBikrKt8udWmjaghzvLMEw+T2DExJyb9ZNeT53+UMB +m6O+B/4xzU/djvp+0hbhAkAemIt+rA5kTmYlFndhpvzkSSM8a2EXsO4XIPgGWCTT +tQKS/tTly0ADMjN/TVy11+9d6zcqadNVuHXHGtR4W0GR +-----END RSA PRIVATE KEY----- + diff --git a/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_rsa_key.pub b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_rsa_key.pub new file mode 100644 index 0000000000..75d2025c71 --- /dev/null +++ b/lib/ssh/test/ssh_renegotiate_SUITE_data/ssh_host_rsa_key.pub @@ -0,0 +1,5 @@ +---- BEGIN SSH2 PUBLIC KEY ---- +AAAAB3NzaC1yc2EAAAADAQABAAAAgQDCZX+4FBDwZIh9y/Uxee1VJnEXlowpz2yDKwj8 +semM4q843337zbNfxHmladB1lpz2NqyxI175xMIJuDxogyZdsOxGnFAzAnthR4dqL/RW +RWzjaxSB6IAO9SPYVVlrpZ+1hsjLW79fwXK/yc8VdhRuWTeQiRgYY2ek8+OKbOqz4Q== +---- END SSH2 PUBLIC KEY ---- diff --git a/lib/ssh/test/ssh_sftpd_SUITE.erl b/lib/ssh/test/ssh_sftpd_SUITE.erl index 852801d013..4edc28aa96 100644 --- a/lib/ssh/test/ssh_sftpd_SUITE.erl +++ b/lib/ssh/test/ssh_sftpd_SUITE.erl @@ -80,9 +80,9 @@ groups() -> init_per_suite(Config) -> ?CHECK_CRYPTO( begin - DataDir = proplists:get_value(data_dir, Config), - PrivDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_dsa(DataDir, PrivDir), + ssh:start(), + ct:log("Pub keys setup for: ~p", + [ssh_test_lib:setup_all_user_host_keys(Config)]), %% to make sure we don't use public-key-auth %% this should be tested by other test suites UserDir = filename:join(proplists:get_value(priv_dir, Config), nopubkey), @@ -91,8 +91,6 @@ init_per_suite(Config) -> end). end_per_suite(Config) -> - SysDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_dsa(SysDir), UserDir = filename:join(proplists:get_value(priv_dir, Config), nopubkey), file:del_dir(UserDir), ssh:stop(). diff --git a/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl b/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl index dbf79e3537..c4fb21f483 100644 --- a/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl +++ b/lib/ssh/test/ssh_sftpd_erlclient_SUITE.erl @@ -64,15 +64,12 @@ init_per_suite(Config) -> {ok, FileInfo} = file:read_file_info(FileName), ok = file:write_file_info(FileName, FileInfo#file_info{mode = 8#400}), + ct:log("Pub keys setup for: ~p", + [ssh_test_lib:setup_all_user_host_keys(Config)]), Config end). -end_per_suite(Config) -> - UserDir = filename:join(proplists:get_value(priv_dir, Config), nopubkey), - file:del_dir(UserDir), - SysDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_rsa(SysDir), - ssh_test_lib:clean_dsa(SysDir), +end_per_suite(_Config) -> ok. %%-------------------------------------------------------------------- @@ -83,10 +80,10 @@ init_per_group(_GroupName, Config) -> end_per_group(_GroupName, Config) -> Config. %%-------------------------------------------------------------------- - init_per_testcase(TestCase, Config) -> ssh:start(), - DataDir = proplists:get_value(data_dir, Config), + UserDir = PrivDir = proplists:get_value(priv_dir, Config), + SysDir = filename:join(PrivDir,"system"), Options = case atom_to_list(TestCase) of @@ -110,9 +107,10 @@ init_per_testcase(TestCase, Config) -> [] end, - {Sftpd, Host, Port} = ssh_test_lib:daemon([{preferred_algorithms, ssh_transport:supported_algorithms()}, - {system_dir, DataDir}, - {user_dir, DataDir}, + {Sftpd, Host, Port} = ssh_test_lib:daemon([{preferred_algorithms, + ssh_transport:supported_algorithms()}, + {system_dir, SysDir}, + {user_dir, UserDir}, {user_passwords, [{?USER,?PASSWD}]} | Options]), @@ -120,7 +118,7 @@ init_per_testcase(TestCase, Config) -> ssh_sftp:start_channel(Host, Port, [{silently_accept_hosts, true}, {preferred_algorithms, ssh_transport:supported_algorithms()}, - {user_dir, DataDir}, + {user_dir, UserDir}, {timeout, 30000}]), TmpConfig = lists:keydelete(sftp, 1, Config), NewConfig = lists:keydelete(sftpd, 1, TmpConfig), @@ -178,7 +176,8 @@ quit(Config) when is_list(Config) -> timer:sleep(5000), {ok, NewSftp, _Conn} = ssh_sftp:start_channel(Host, Port, [{silently_accept_hosts, true}, - {preferred_algorithms, ssh_transport:supported_algorithms()}, + {preferred_algorithms, + ssh_transport:supported_algorithms()}, {user_dir, UserDir}, {user, ?USER}, {password, ?PASSWD}]), diff --git a/lib/ssh/test/ssh_test_lib.erl b/lib/ssh/test/ssh_test_lib.erl index ebf60bc1d0..f16c0d6278 100644 --- a/lib/ssh/test/ssh_test_lib.erl +++ b/lib/ssh/test/ssh_test_lib.erl @@ -294,25 +294,37 @@ rcv_lingering(Timeout) -> receive_exec_result([]) -> expected; receive_exec_result(Msgs) when is_list(Msgs) -> - ct:log("Expect data! ~p", [Msgs]), + ct:log("~p:~p Expect data! ~p", [?MODULE,?FUNCTION_NAME,Msgs]), receive Msg -> - case lists:member(Msg, Msgs) of + case lists:member(Msg, Msgs) + orelse lists:member({optional,Msg}, Msgs) + of true -> - ct:log("Collected data ~p", [Msg]), - receive_exec_result(Msgs--[Msg]); + ct:log("~p:~p Collected data ~p", [?MODULE,?FUNCTION_NAME,Msg]), + receive_exec_result(Msgs--[Msg,{optional,Msg}]); false -> case Msg of {ssh_cm,_,{data,_,1, Data}} -> - ct:log("StdErr: ~p~n", [Data]), + ct:log("~p:~p StdErr: ~p~n", [?MODULE,?FUNCTION_NAME,Data]), receive_exec_result(Msgs); Other -> - ct:log("Other ~p", [Other]), + ct:log("~p:~p Other ~p", [?MODULE,?FUNCTION_NAME,Other]), {unexpected_msg, Other} end end after - 30000 -> ct:fail("timeout ~p:~p",[?MODULE,?LINE]) + 30000 -> + case lists:all(fun(M) -> + is_tuple(M) andalso (element(1,M) == optional) + end, Msgs) + of + false -> + ct:fail("timeout ~p:~p",[?MODULE,?FUNCTION_NAME]); + true -> + ct:log("~p:~p Only optional messages expected!~n ~p", [?MODULE,?FUNCTION_NAME,Msgs]), + expected + end end; receive_exec_result(Msg) -> receive_exec_result([Msg]). @@ -325,26 +337,11 @@ receive_exec_result_or_fail(Msg) -> end. receive_exec_end(ConnectionRef, ChannelId) -> - Eof = {ssh_cm, ConnectionRef, {eof, ChannelId}}, - ExitStatus = {ssh_cm, ConnectionRef, {exit_status, ChannelId, 0}}, - Closed = {ssh_cm, ConnectionRef,{closed, ChannelId}}, - case receive_exec_result(ExitStatus) of - {unexpected_msg, Eof} -> %% Open ssh seems to not allways send these messages - %% in the same order! - ct:log("2: Collected data ~p", [Eof]), - case receive_exec_result(ExitStatus) of - expected -> - expected = receive_exec_result(Closed); - {unexpected_msg, Closed} -> - ct:log("3: Collected data ~p", [Closed]) - end; - expected -> - ct:log("4: Collected data ~p", [ExitStatus]), - expected = receive_exec_result(Eof), - expected = receive_exec_result(Closed); - Other -> - ct:fail({unexpected_msg, Other}) - end. + receive_exec_result( + [{ssh_cm, ConnectionRef, {eof, ChannelId}}, + {optional, {ssh_cm, ConnectionRef, {exit_status, ChannelId, 0}}}, + {ssh_cm, ConnectionRef, {closed, ChannelId}} + ]). receive_exec_result(Data, ConnectionRef, ChannelId) -> Eof = {ssh_cm, ConnectionRef, {eof, ChannelId}}, @@ -354,21 +351,6 @@ receive_exec_result(Data, ConnectionRef, ChannelId) -> expected = receive_exec_result(Closed). -setup_ssh_auth_keys(RSAFile, DSAFile, Dir) -> - Entries = ssh_file_entry(RSAFile) ++ ssh_file_entry(DSAFile), - AuthKeys = public_key:ssh_encode(Entries , auth_keys), - AuthKeysFile = filename:join(Dir, "authorized_keys"), - file:write_file(AuthKeysFile, AuthKeys). - -ssh_file_entry(PubFile) -> - case file:read_file(PubFile) of - {ok, Ssh} -> - [{Key, _}] = public_key:ssh_decode(Ssh, public_key), - [{Key, [{comment, "Test"}]}]; - _ -> - [] - end. - failfun(_User, {authmethod,none}) -> ok; failfun(User, Reason) -> @@ -378,236 +360,31 @@ hostname() -> {ok,Host} = inet:gethostname(), Host. -known_hosts(BR) -> - KnownHosts = ssh_file:file_name(user, "known_hosts", []), - B = KnownHosts ++ "xxx", - case BR of - backup -> - file:rename(KnownHosts, B); - restore -> - file:delete(KnownHosts), - file:rename(B, KnownHosts) - end. - -setup_dsa(DataDir, UserDir) -> - file:copy(filename:join(DataDir, "id_dsa"), filename:join(UserDir, "id_dsa")), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, "ssh_host_dsa_key"), filename:join(System, "ssh_host_dsa_key")), - file:copy(filename:join(DataDir, "ssh_host_dsa_key.pub"), filename:join(System, "ssh_host_dsa_key.pub")), -ct:log("DataDir ~p:~n ~p~n~nSystDir ~p:~n ~p~n~nUserDir ~p:~n ~p",[DataDir, file:list_dir(DataDir), System, file:list_dir(System), UserDir, file:list_dir(UserDir)]), - setup_dsa_known_host(DataDir, UserDir), - setup_dsa_auth_keys(DataDir, UserDir). - -setup_rsa(DataDir, UserDir) -> - file:copy(filename:join(DataDir, "id_rsa"), filename:join(UserDir, "id_rsa")), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, "ssh_host_rsa_key"), filename:join(System, "ssh_host_rsa_key")), - file:copy(filename:join(DataDir, "ssh_host_rsa_key.pub"), filename:join(System, "ssh_host_rsa_key.pub")), -ct:log("DataDir ~p:~n ~p~n~nSystDir ~p:~n ~p~n~nUserDir ~p:~n ~p",[DataDir, file:list_dir(DataDir), System, file:list_dir(System), UserDir, file:list_dir(UserDir)]), - setup_rsa_known_host(DataDir, UserDir), - setup_rsa_auth_keys(DataDir, UserDir). - -setup_ecdsa(Size, DataDir, UserDir) -> - file:copy(filename:join(DataDir, "id_ecdsa"++Size), filename:join(UserDir, "id_ecdsa")), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, "ssh_host_ecdsa_key"++Size), filename:join(System, "ssh_host_ecdsa_key")), - file:copy(filename:join(DataDir, "ssh_host_ecdsa_key"++Size++".pub"), filename:join(System, "ssh_host_ecdsa_key.pub")), -ct:log("DataDir ~p:~n ~p~n~nSystDir ~p:~n ~p~n~nUserDir ~p:~n ~p",[DataDir, file:list_dir(DataDir), System, file:list_dir(System), UserDir, file:list_dir(UserDir)]), - setup_ecdsa_known_host(Size, System, UserDir), - setup_ecdsa_auth_keys(Size, DataDir, UserDir). - -setup_eddsa(Alg, DataDir, UserDir) -> - {IdPriv, _IdPub, HostPriv, HostPub} = - case Alg of - ed25519 -> {"id_ed25519", "id_ed25519.pub", "ssh_host_ed25519_key", "ssh_host_ed25519_key.pub"}; - ed448 -> {"id_ed448", "id_ed448.pub", "ssh_host_ed448_key", "ssh_host_ed448_key.pub"} - end, - file:copy(filename:join(DataDir, IdPriv), filename:join(UserDir, IdPriv)), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, HostPriv), filename:join(System, HostPriv)), - file:copy(filename:join(DataDir, HostPub), filename:join(System, HostPub)), -ct:log("DataDir ~p:~n ~p~n~nSystDir ~p:~n ~p~n~nUserDir ~p:~n ~p",[DataDir, file:list_dir(DataDir), System, file:list_dir(System), UserDir, file:list_dir(UserDir)]), - setup_eddsa_known_host(HostPub, DataDir, UserDir), - setup_eddsa_auth_keys(Alg, DataDir, UserDir). - -clean_dsa(UserDir) -> - del_dirs(filename:join(UserDir, "system")), - file:delete(filename:join(UserDir,"id_dsa")), - file:delete(filename:join(UserDir,"known_hosts")), - file:delete(filename:join(UserDir,"authorized_keys")). - -clean_rsa(UserDir) -> - del_dirs(filename:join(UserDir, "system")), - file:delete(filename:join(UserDir,"id_rsa")), - file:delete(filename:join(UserDir,"known_hosts")), - file:delete(filename:join(UserDir,"authorized_keys")). - -setup_dsa_pass_phrase(DataDir, UserDir, Phrase) -> - try - {ok, KeyBin} = file:read_file(filename:join(DataDir, "id_dsa")), - setup_pass_phrase(KeyBin, filename:join(UserDir, "id_dsa"), Phrase), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, "ssh_host_dsa_key"), filename:join(System, "ssh_host_dsa_key")), - file:copy(filename:join(DataDir, "ssh_host_dsa_key.pub"), filename:join(System, "ssh_host_dsa_key.pub")), - setup_dsa_known_host(DataDir, UserDir), - setup_dsa_auth_keys(DataDir, UserDir) - of - _ -> true - catch - _:_ -> false - end. +del_dirs(Dir) -> + del_dir_contents(Dir), + file:del_dir(Dir), + ok. -setup_rsa_pass_phrase(DataDir, UserDir, Phrase) -> - try - {ok, KeyBin} = file:read_file(filename:join(DataDir, "id_rsa")), - setup_pass_phrase(KeyBin, filename:join(UserDir, "id_rsa"), Phrase), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, "ssh_host_rsa_key"), filename:join(System, "ssh_host_rsa_key")), - file:copy(filename:join(DataDir, "ssh_host_rsa_key.pub"), filename:join(System, "ssh_host_rsa_key.pub")), - setup_rsa_known_host(DataDir, UserDir), - setup_rsa_auth_keys(DataDir, UserDir) - of - _ -> true - catch - _:_ -> false - end. -setup_ecdsa_pass_phrase(Size, DataDir, UserDir, Phrase) -> - try - {ok, KeyBin} = - case file:read_file(F=filename:join(DataDir, "id_ecdsa"++Size)) of - {error,E} -> - ct:log("Failed (~p) to read ~p~nFiles: ~p", [E,F,file:list_dir(DataDir)]), - file:read_file(filename:join(DataDir, "id_ecdsa")); - Other -> - Other - end, - setup_pass_phrase(KeyBin, filename:join(UserDir, "id_ecdsa"), Phrase), - System = filename:join(UserDir, "system"), - file:make_dir(System), - file:copy(filename:join(DataDir, "ssh_host_ecdsa_key"++Size), filename:join(System, "ssh_host_ecdsa_key")), - file:copy(filename:join(DataDir, "ssh_host_ecdsa_key"++Size++".pub"), filename:join(System, "ssh_host_ecdsa_key.pub")), - setup_ecdsa_known_host(Size, System, UserDir), - setup_ecdsa_auth_keys(Size, DataDir, UserDir) - of - _ -> true - catch - _:_ -> false +del_dir_contents(Dir) -> + case file:list_dir(Dir) of + {ok, Files} -> + do_del_files(Dir, Files); + _ -> + ok end. -setup_pass_phrase(KeyBin, OutFile, Phrase) -> - [{KeyType, _,_} = Entry0] = public_key:pem_decode(KeyBin), - Key = public_key:pem_entry_decode(Entry0), - Salt = crypto:strong_rand_bytes(8), - Entry = public_key:pem_entry_encode(KeyType, Key, - {{"DES-CBC", Salt}, Phrase}), - Pem = public_key:pem_encode([Entry]), - file:write_file(OutFile, Pem). - -setup_dsa_known_host(SystemDir, UserDir) -> - {ok, SshBin} = file:read_file(filename:join(SystemDir, "ssh_host_dsa_key.pub")), - [{Key, _}] = public_key:ssh_decode(SshBin, public_key), - setup_known_hosts(Key, UserDir). - -setup_rsa_known_host(SystemDir, UserDir) -> - {ok, SshBin} = file:read_file(filename:join(SystemDir, "ssh_host_rsa_key.pub")), - [{Key, _}] = public_key:ssh_decode(SshBin, public_key), - setup_known_hosts(Key, UserDir). - -setup_ecdsa_known_host(_Size, SystemDir, UserDir) -> - {ok, SshBin} = file:read_file(filename:join(SystemDir, "ssh_host_ecdsa_key.pub")), - [{Key, _}] = public_key:ssh_decode(SshBin, public_key), - setup_known_hosts(Key, UserDir). - -setup_eddsa_known_host(HostPub, SystemDir, UserDir) -> - {ok, SshBin} = file:read_file(filename:join(SystemDir, HostPub)), - [{Key, _}] = public_key:ssh_decode(SshBin, public_key), - setup_known_hosts(Key, UserDir). - -setup_known_hosts(Key, UserDir) -> - {ok, Hostname} = inet:gethostname(), - {ok, {A, B, C, D}} = inet:getaddr(Hostname, inet), - IP = lists:concat([A, ".", B, ".", C, ".", D]), - HostNames = [{hostnames,[Hostname, IP]}], - KnownHosts = [{Key, HostNames}], - KnownHostsEnc = public_key:ssh_encode(KnownHosts, known_hosts), - KHFile = filename:join(UserDir, "known_hosts"), - file:write_file(KHFile, KnownHostsEnc). - -setup_dsa_auth_keys(Dir, UserDir) -> - {ok, Pem} = file:read_file(filename:join(Dir, "id_dsa")), - DSA = public_key:pem_entry_decode(hd(public_key:pem_decode(Pem))), - PKey = DSA#'DSAPrivateKey'.y, - P = DSA#'DSAPrivateKey'.p, - Q = DSA#'DSAPrivateKey'.q, - G = DSA#'DSAPrivateKey'.g, - Dss = #'Dss-Parms'{p=P, q=Q, g=G}, - setup_auth_keys([{{PKey, Dss}, [{comment, "Test"}]}], UserDir). - -setup_rsa_auth_keys(Dir, UserDir) -> - {ok, Pem} = file:read_file(filename:join(Dir, "id_rsa")), - RSA = public_key:pem_entry_decode(hd(public_key:pem_decode(Pem))), - #'RSAPrivateKey'{publicExponent = E, modulus = N} = RSA, - PKey = #'RSAPublicKey'{publicExponent = E, modulus = N}, - setup_auth_keys([{ PKey, [{comment, "Test"}]}], UserDir). - -setup_ecdsa_auth_keys(Size, Dir, UserDir) -> - {ok, Pem} = - case file:read_file(F=filename:join(Dir, "id_ecdsa"++Size)) of - {error,E} -> - ct:log("Failed (~p) to read ~p~nFiles: ~p", [E,F,file:list_dir(Dir)]), - file:read_file(filename:join(Dir, "id_ecdsa")); - Other -> - Other - end, - ECDSA = public_key:pem_entry_decode(hd(public_key:pem_decode(Pem))), - #'ECPrivateKey'{publicKey = Q, - parameters = Param = {namedCurve,_Id0}} = ECDSA, - PKey = #'ECPoint'{point = Q}, - setup_auth_keys([{ {PKey,Param}, [{comment, "Test"}]}], UserDir). - -setup_eddsa_auth_keys(Alg, Dir, UserDir) -> - SshAlg = case Alg of - ed25519 -> 'ssh-ed25519'; - ed448 -> 'ssh-ed448' - end, - {ok, {ed_pri,Alg,Pub,_}} = ssh_file:user_key(SshAlg, [{user_dir,Dir}]), - setup_auth_keys([{{ed_pub,Alg,Pub}, [{comment, "Test"}]}], UserDir). - -setup_auth_keys(Keys, Dir) -> - AuthKeys = public_key:ssh_encode(Keys, auth_keys), - AuthKeysFile = filename:join(Dir, "authorized_keys"), - ok = file:write_file(AuthKeysFile, AuthKeys), - AuthKeys. - -write_auth_keys(Keys, Dir) -> - AuthKeysFile = filename:join(Dir, "authorized_keys"), - file:write_file(AuthKeysFile, Keys). +do_del_files(Dir, Files) -> + lists:foreach(fun(File) -> + FullPath = filename:join(Dir,File), + case filelib:is_dir(FullPath) of + true -> + del_dirs(FullPath); + false -> + file:delete(FullPath) + end + end, Files). -del_dirs(Dir) -> - case file:list_dir(Dir) of - {ok, []} -> - file:del_dir(Dir); - {ok, Files} -> - lists:foreach(fun(File) -> - FullPath = filename:join(Dir,File), - case filelib:is_dir(FullPath) of - true -> - del_dirs(FullPath), - file:del_dir(FullPath); - false -> - file:delete(FullPath) - end - end, Files); - _ -> - ok - end. openssh_sanity_check(Config) -> ssh:start(), @@ -625,34 +402,6 @@ openssh_sanity_check(Config) -> {skip, Str} end. -openssh_supports(ClientOrServer, Tag, Alg) when ClientOrServer == sshc ; - ClientOrServer == sshd -> - SSH_algos = ssh_test_lib:default_algorithms(ClientOrServer), - L = proplists:get_value(Tag, SSH_algos, []), - lists:member(Alg, L) orelse - lists:member(Alg, proplists:get_value(client2server, L, [])) orelse - lists:member(Alg, proplists:get_value(server2client, L, [])). - -%%-------------------------------------------------------------------- -%% Check if we have a "newer" ssh client that supports these test cases - -ssh_client_supports_Q() -> - 0 == check_ssh_client_support2( - ?MODULE:open_port({spawn, "ssh -Q cipher"}) - ). - -check_ssh_client_support2(P) -> - receive - {P, {data, _A}} -> - check_ssh_client_support2(P); - {P, {exit_status, E}} -> - ct:log("~p:~p exit_status:~n~p",[?MODULE,?LINE,E]), - E - after 5000 -> - ct:log("Openssh command timed out ~n"), - -1 - end. - %%%-------------------------------------------------------------------- %%% Probe a server or a client about algorithm support @@ -1189,3 +938,132 @@ mk_dir_path(DirPath) -> %%ct:log("~p:~p return Other ~p ~ts", [?MODULE,?LINE,Other,DirPath]), Other end. + +%%%---------------------------------------------------------------- +%%% New + +setup_all_user_host_keys(Config) -> + DataDir = proplists:get_value(data_dir, Config), + PrivDir = proplists:get_value(priv_dir, Config), + setup_all_user_host_keys(DataDir, PrivDir). + +setup_all_user_host_keys(DataDir, PrivDir) -> + setup_all_user_host_keys(DataDir, PrivDir, filename:join(PrivDir,"system")). + +setup_all_user_host_keys(DataDir, UserDir, SysDir) -> + lists:foldl(fun(Alg, OkAlgs) -> + try + ok = ssh_test_lib:setup_user_key(Alg, DataDir, UserDir), + ok = ssh_test_lib:setup_host_key(Alg, DataDir, SysDir) + of + ok -> [Alg|OkAlgs] + catch + error:{badmatch, {error,enoent}} -> + OkAlgs; + C:E:S -> + ct:log("Exception in ~p:~p for alg ~p: ~p:~p~n~p", + [?MODULE,?FUNCTION_NAME,Alg,C,E,S]), + OkAlgs + end + end, [], ssh_transport:supported_algorithms(public_key)). + + +setup_all_host_keys(Config) -> + DataDir = proplists:get_value(data_dir, Config), + PrivDir = proplists:get_value(priv_dir, Config), + setup_all_host_keys(DataDir, filename:join(PrivDir,"system")). + +setup_all_host_keys(DataDir, SysDir) -> + lists:foldl(fun(Alg, OkAlgs) -> + try + ok = ssh_test_lib:setup_host_key(Alg, DataDir, SysDir) + of + ok -> [Alg|OkAlgs] + catch + error:{badmatch, {error,enoent}} -> + OkAlgs; + C:E:S -> + ct:log("Exception in ~p:~p for alg ~p: ~p:~p~n~p", + [?MODULE,?FUNCTION_NAME,Alg,C,E,S]), + OkAlgs + end + end, [], ssh_transport:supported_algorithms(public_key)). + +setup_user_key(SshAlg, DataDir, UserDir) -> + file:make_dir(UserDir), + %% Copy private user key to user's dir + {ok,_} = file:copy(filename:join(DataDir, file_base_name(user_src,SshAlg)), + filename:join(UserDir, file_base_name(user,SshAlg))), + %% Setup authorized_keys in user's dir + {ok,Pub} = file:read_file(filename:join(DataDir, file_base_name(user_src,SshAlg)++".pub")), + ok = file:write_file(filename:join(UserDir, "authorized_keys"), + io_lib:format("~n~s~n",[Pub]), + [append]), + ?ct_log_show_file( filename:join(DataDir, file_base_name(user_src,SshAlg)++".pub") ), + ?ct_log_show_file( filename:join(UserDir, "authorized_keys") ), + ok. + +setup_host_key_create_dir(SshAlg, DataDir, BaseDir) -> + SysDir = filename:join(BaseDir,"system"), + ct:log("~p:~p SshAlg=~p~nDataDir = ~p~nBaseDir = ~p~nSysDir = ~p",[?MODULE,?LINE,SshAlg, DataDir, BaseDir,SysDir]), + file:make_dir(SysDir), + setup_host_key(SshAlg, DataDir, SysDir), + SysDir. + +setup_host_key(SshAlg, DataDir, SysDir) -> + mk_dir_path(SysDir), + %% Copy private host key to system's dir + {ok,_} = file:copy(filename:join(DataDir, file_base_name(system_src,SshAlg)), + filename:join(SysDir, file_base_name(system,SshAlg))), + ?ct_log_show_file( filename:join(SysDir, file_base_name(system,SshAlg)) ), + ok. + +setup_known_host(SshAlg, DataDir, UserDir) -> + {ok,Pub} = file:read_file(filename:join(DataDir, file_base_name(system_src,SshAlg)++".pub")), + S = lists:join(" ", lists:reverse(tl(lists:reverse(string:tokens(binary_to_list(Pub), " "))))), + ok = file:write_file(filename:join(UserDir, "known_hosts"), + io_lib:format("~p~n",[S])), + ?ct_log_show_file( filename:join(UserDir, "known_hosts") ), + ok. + + +get_addr_str() -> + {ok, Hostname} = inet:gethostname(), + {ok, {A, B, C, D}} = inet:getaddr(Hostname, inet), + IP = lists:concat([A, ".", B, ".", C, ".", D]), + lists:concat([Hostname,",",IP]). + + +file_base_name(user, 'ecdsa-sha2-nistp256') -> "id_ecdsa"; +file_base_name(user, 'ecdsa-sha2-nistp384') -> "id_ecdsa"; +file_base_name(user, 'ecdsa-sha2-nistp521') -> "id_ecdsa"; +file_base_name(user, 'rsa-sha2-256' ) -> "id_rsa"; +file_base_name(user, 'rsa-sha2-384' ) -> "id_rsa"; +file_base_name(user, 'rsa-sha2-512' ) -> "id_rsa"; +file_base_name(user, 'ssh-dss' ) -> "id_dsa"; +file_base_name(user, 'ssh-ed25519' ) -> "id_ed25519"; +file_base_name(user, 'ssh-ed448' ) -> "id_ed448"; +file_base_name(user, 'ssh-rsa' ) -> "id_rsa"; + +file_base_name(user_src, 'ecdsa-sha2-nistp256') -> "id_ecdsa256"; +file_base_name(user_src, 'ecdsa-sha2-nistp384') -> "id_ecdsa384"; +file_base_name(user_src, 'ecdsa-sha2-nistp521') -> "id_ecdsa521"; +file_base_name(user_src, Alg) -> file_base_name(user, Alg); + +file_base_name(system, 'ecdsa-sha2-nistp256') -> "ssh_host_ecdsa_key"; +file_base_name(system, 'ecdsa-sha2-nistp384') -> "ssh_host_ecdsa_key"; +file_base_name(system, 'ecdsa-sha2-nistp521') -> "ssh_host_ecdsa_key"; +file_base_name(system, 'rsa-sha2-256' ) -> "ssh_host_rsa_key"; +file_base_name(system, 'rsa-sha2-384' ) -> "ssh_host_rsa_key"; +file_base_name(system, 'rsa-sha2-512' ) -> "ssh_host_rsa_key"; +file_base_name(system, 'ssh-dss' ) -> "ssh_host_dsa_key"; +file_base_name(system, 'ssh-ed25519' ) -> "ssh_host_ed25519_key"; +file_base_name(system, 'ssh-ed448' ) -> "ssh_host_ed448_key"; +file_base_name(system, 'ssh-rsa' ) -> "ssh_host_rsa_key"; + +file_base_name(system_src, 'ecdsa-sha2-nistp256') -> "ssh_host_ecdsa_key256"; +file_base_name(system_src, 'ecdsa-sha2-nistp384') -> "ssh_host_ecdsa_key384"; +file_base_name(system_src, 'ecdsa-sha2-nistp521') -> "ssh_host_ecdsa_key521"; +file_base_name(system_src, Alg) -> file_base_name(system, Alg). + +%%%---------------------------------------------------------------- diff --git a/lib/ssh/test/ssh_test_lib.hrl b/lib/ssh/test/ssh_test_lib.hrl index b9af2ecb5d..098ee79044 100644 --- a/lib/ssh/test/ssh_test_lib.hrl +++ b/lib/ssh/test/ssh_test_lib.hrl @@ -49,3 +49,13 @@ -define(wait_match(Pattern, FunctionCall), ?wait_match(Pattern, FunctionCall, ok) ). +%%------------------------------------------------------------------------- +%% Write file into log +%%------------------------------------------------------------------------- + +-define(ct_log_show_file(File), + (fun(File__) -> + {ok,Contents__} = file:read_file(File__), + ct:log("~p:~p Show file~n~s =~n~s~n", + [?MODULE,?LINE,File__, Contents__]) + end)(File)). diff --git a/lib/ssh/test/ssh_to_openssh_SUITE.erl b/lib/ssh/test/ssh_to_openssh_SUITE.erl index 5aa3824702..426efbb5e3 100644 --- a/lib/ssh/test/ssh_to_openssh_SUITE.erl +++ b/lib/ssh/test/ssh_to_openssh_SUITE.erl @@ -81,10 +81,6 @@ end_per_suite(_Config) -> ok. init_per_group(erlang_server, Config) -> - DataDir = proplists:get_value(data_dir, Config), - UserDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:setup_dsa_known_host(DataDir, UserDir), - ssh_test_lib:setup_rsa_known_host(DataDir, UserDir), Config; init_per_group(G, Config) when G==tunnel_distro_server ; G==tunnel_distro_client -> @@ -102,11 +98,6 @@ init_per_group(erlang_client, Config) -> init_per_group(_, Config) -> Config. -end_per_group(erlang_server, Config) -> - UserDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_dsa(UserDir), - ssh_test_lib:clean_rsa(UserDir), - Config; end_per_group(_, Config) -> Config. @@ -166,10 +157,16 @@ exec_with_io_in_sshc(Config) when is_list(Config) -> {failfun, fun ssh_test_lib:failfun/2}]), ct:sleep(500), + PrivDir = proplists:get_value(priv_dir, Config), + KnownHosts = filename:join(PrivDir, "known_hosts"), ExecStr = "\"io:read('% ').\"", Cmd = "echo howdy. | " ++ ssh_test_lib:open_sshc_cmd(Host, Port, - " -o StrictHostKeyChecking=no" - " -x", % Disable X forwarding + [" -o UserKnownHostsFile=", KnownHosts, + " -o CheckHostIP=no" + " -o StrictHostKeyChecking=no" + " -q" + " -x" % Disable X forwarding + ], ExecStr), ct:pal("Cmd = ~p~n",[Cmd]), case os:cmd(Cmd) of @@ -194,9 +191,15 @@ exec_direct_with_io_in_sshc(Config) when is_list(Config) -> ]), ct:sleep(500), + PrivDir = proplists:get_value(priv_dir, Config), + KnownHosts = filename:join(PrivDir, "known_hosts"), Cmd = "echo ciao. | " ++ ssh_test_lib:open_sshc_cmd(Host, Port, - " -o StrictHostKeyChecking=no" - " -x", % Disable X forwarding + [" -o UserKnownHostsFile=", KnownHosts, + " -o CheckHostIP=no" + " -o StrictHostKeyChecking=no" + " -q" + " -x" % Disable X forwarding + ], "'? '"), ct:pal("Cmd = ~p~n",[Cmd]), case os:cmd(Cmd) of @@ -214,7 +217,6 @@ erlang_server_openssh_client_renegotiate(Config) -> _PubKeyAlg = ssh_rsa, SystemDir = proplists:get_value(data_dir, Config), PrivDir = proplists:get_value(priv_dir, Config), - KnownHosts = filename:join(PrivDir, "known_hosts"), {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SystemDir}, {failfun, fun ssh_test_lib:failfun/2}]), @@ -225,9 +227,13 @@ erlang_server_openssh_client_renegotiate(Config) -> Data = lists:duplicate(trunc(1.1*RenegLimitK*1024), $a), ok = file:write_file(DataFile, Data), + KnownHosts = filename:join(PrivDir, "known_hosts"), Cmd = ssh_test_lib:open_sshc_cmd(Host, Port, [" -o UserKnownHostsFile=", KnownHosts, - " -o StrictHostKeyChecking=no", + " -o CheckHostIP=no" + " -o StrictHostKeyChecking=no" + " -q" + " -x", " -o RekeyLimit=",integer_to_list(RenegLimitK),"K"]), @@ -268,7 +274,6 @@ erlang_server_openssh_client_renegotiate(Config) -> tunnel_out_non_erlclient_erlserver(Config) -> SystemDir = proplists:get_value(data_dir, Config), PrivDir = proplists:get_value(priv_dir, Config), - KnownHosts = filename:join(PrivDir, "known_hosts"), {_Pid, Host, Port} = ssh_test_lib:daemon([{tcpip_tunnel_out, true}, {system_dir, SystemDir}, @@ -278,9 +283,13 @@ tunnel_out_non_erlclient_erlserver(Config) -> ListenHost = {127,0,0,1}, ListenPort = 2345, + KnownHosts = filename:join(PrivDir, "known_hosts"), Cmd = ssh_test_lib:open_sshc_cmd(Host, Port, [" -o UserKnownHostsFile=", KnownHosts, - " -o StrictHostKeyChecking=no", + " -o CheckHostIP=no" + " -o StrictHostKeyChecking=no" + " -q" + " -x", " -R ",integer_to_list(ListenPort),":127.0.0.1:",integer_to_list(ToPort)]), spawn(fun() -> ct:log(["ssh command:\r\n ",Cmd],[]), @@ -295,7 +304,6 @@ tunnel_out_non_erlclient_erlserver(Config) -> tunnel_in_non_erlclient_erlserver(Config) -> SystemDir = proplists:get_value(data_dir, Config), UserDir = proplists:get_value(priv_dir, Config), - KnownHosts = filename:join(UserDir, "known_hosts"), {_Pid, Host, Port} = ssh_test_lib:daemon([{tcpip_tunnel_in, true}, {system_dir, SystemDir}, {failfun, fun ssh_test_lib:failfun/2}]), @@ -304,10 +312,14 @@ tunnel_in_non_erlclient_erlserver(Config) -> ListenHost = {127,0,0,1}, ListenPort = 2345, + KnownHosts = filename:join(UserDir, "known_hosts"), Cmd = ssh_test_lib:open_sshc_cmd(Host, Port, [" -o UserKnownHostsFile=", KnownHosts, - " -o StrictHostKeyChecking=no", + " -o CheckHostIP=no" + " -o StrictHostKeyChecking=no" + " -q" + " -x", " -L ",integer_to_list(ListenPort),":127.0.0.1:",integer_to_list(ToPort)]), spawn(fun() -> ct:log(["ssh command:\r\n ",Cmd],[]), @@ -519,22 +531,6 @@ extra_logout() -> ok end. -%%-------------------------------------------------------------------- -%% Check if we have a "newer" ssh client that supports these test cases -check_ssh_client_support(Config) -> - case ssh_test_lib:ssh_client_supports_Q() of - true -> - ssh:start(), - Config; - _ -> - {skip, "test case not supported by ssh client"} - end. - -comment(AtomList) -> - ct:comment( - string:join(lists:map(fun erlang:atom_to_list/1, AtomList), - ", ")). - %%%---------------------------------------------------------------- no_forwarding() -> %%% Check if the ssh of the OS has tunneling enabled diff --git a/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ecdsa_key b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ecdsa_key new file mode 100644 index 0000000000..2979ea88ed --- /dev/null +++ b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ecdsa_key @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMe4MDoit0t8RzSVPwkCBemQ9fhXL+xnTSAWISw8HNCioAoGCCqGSM49 +AwEHoUQDQgAEo2q7U3P6r0W5WGOLtM78UQtofM9UalEhiZeDdiyylsR/RR17Op0s +VPGSADLmzzgcucLEKy17j2S+oz42VUJy5A== +-----END EC PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ecdsa_key.pub b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ecdsa_key.pub new file mode 100644 index 0000000000..85dc419345 --- /dev/null +++ b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ecdsa_key.pub @@ -0,0 +1 @@ +ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKNqu1Nz+q9FuVhji7TO/FELaHzPVGpRIYmXg3YsspbEf0UdezqdLFTxkgAy5s84HLnCxCste49kvqM+NlVCcuQ= uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ed25519_key b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ed25519_key new file mode 100644 index 0000000000..13a8fcf491 --- /dev/null +++ b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ed25519_key @@ -0,0 +1,7 @@ +-----BEGIN OPENSSH PRIVATE KEY----- +b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW +QyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQAAAJi+h4O7voeD +uwAAAAtzc2gtZWQyNTUxOQAAACBJSOuiYGWaO9lye8Bgafod1kw8P6cV3Xb2qJgCB6yJfQ +AAAEBaOcJfGPNemKc1wPHTCmM4Kwvh6dZ0CqY14UT361UnN0lI66JgZZo72XJ7wGBp+h3W +TDw/pxXddvaomAIHrIl9AAAAE3VhYmhuaWxAZWx4YWRsajNxMzIBAg== +-----END OPENSSH PRIVATE KEY----- diff --git a/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ed25519_key.pub b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ed25519_key.pub new file mode 100644 index 0000000000..156ef4045c --- /dev/null +++ b/lib/ssh/test/ssh_to_openssh_SUITE_data/ssh_host_ed25519_key.pub @@ -0,0 +1 @@ +ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIElI66JgZZo72XJ7wGBp+h3WTDw/pxXddvaomAIHrIl9 uabhnil@elxadlj3q32 diff --git a/lib/ssh/test/ssh_upgrade_SUITE.erl b/lib/ssh/test/ssh_upgrade_SUITE.erl index 4417962d26..78657b2014 100644 --- a/lib/ssh/test/ssh_upgrade_SUITE.erl +++ b/lib/ssh/test/ssh_upgrade_SUITE.erl @@ -61,9 +61,7 @@ init_per_suite(Config0) -> end_per_suite(Config) -> ct_release_test:cleanup(Config), - ssh:stop(), - UserDir = proplists:get_value(priv_dir, Config), - ssh_test_lib:clean_rsa(UserDir). + ssh:stop(). init_per_testcase(_TestCase, Config) -> Config. diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk index 70307d6039..732c3f8766 100644 --- a/lib/ssh/vsn.mk +++ b/lib/ssh/vsn.mk @@ -1,4 +1,4 @@ #-*-makefile-*- ; force emacs to enter makefile-mode -SSH_VSN = 4.9 +SSH_VSN = 4.10 APP_VSN = "ssh-$(SSH_VSN)" diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 93d58939f2..8eed6a9d1a 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -27,6 +27,177 @@ </header> <p>This document describes the changes made to the SSL application.</p> +<section><title>SSL 10.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix a bug that causes cross-build failure.</p> + <p> + This change excludes the ssl.d dependency file from the + source tar balls.</p> + <p> + Own Id: OTP-16562 Aux Id: ERL-1168 </p> + </item> + <item> + <p> + Correct translation of OpenSSL legacy names for two + legacy cipher suites</p> + <p> + Own Id: OTP-16573 Aux Id: ERIERL-477 </p> + </item> + <item> + <p> + Correct documentation for PSK identity and SRP username.</p> + <p> + Own Id: OTP-16585</p> + </item> + <item> + <p> + Make sure client hostname check is run when client uses + its own verify_fun</p> + <p> + Own Id: OTP-16626 Aux Id: ERL-1232 </p> + </item> + <item> + <p> + Improved signature selection mechanism in TLS 1.3 for + increased interoperability.</p> + <p> + Own Id: OTP-16638 Aux Id: ERL-1206 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Drop support for SSL-3.0. Support for this legacy TLS + version has not been enabled by default since OTP 19. Now + all code to support it has been removed, that is SSL-3.0 + protocol version can not be used and is considered + invalid.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-14790</p> + </item> + <item> + <p> + Added support for RSA-PSS signature schemes</p> + <p> + Own Id: OTP-15247</p> + </item> + <item> + <p> + Improve interoperability by implementing the middlebox + compatiblity mode.</p> + <p> + The middlebox compatibility mode makes the TLS 1.3 + handshake look more like a TLS 1.2 handshake and + increases the chance of successfully establishing TLS 1.3 + connections through legacy middleboxes.</p> + <p> + Own Id: OTP-15589</p> + </item> + <item> + <p> + Utilize new properties of <seemfa + marker="erts:erlang#dist_ctrl_get_data/1"><c>erlang:dist_ctrl_get_data()</c></seemfa> + for performance improvement of Erlang distribution over + TLS.</p> + <p> + Own Id: OTP-16127 Aux Id: OTP-15618 </p> + </item> + <item> + <p> + Calls of deprecated functions in the <seeguide + marker="crypto:new_api#the-old-api">Old Crypto + API</seeguide> are replaced by calls of their <seeguide + marker="crypto:new_api#the-new-api">substitutions</seeguide>.</p> + <p> + Own Id: OTP-16346</p> + </item> + <item> + <p> + Implement cipher suite TLS_AES_128_CCM_8_SHA256.</p> + <p> + Own Id: OTP-16391</p> + </item> + <item> + <p> + This change adds TLS-1.3 to the list of default supported + versions. That is, TLS-1.3 and TLS-1.2 are configured + when ssl option 'versions' is not explicitly set.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-16400</p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p> + Extended ssl:versions so that it lists supported, + available and implemented TLS/DTLS versions.</p> + <p> + Own Id: OTP-16519</p> + </item> + <item> + <p> + Added new option exclusive for ssl:cipher_suites/2,3</p> + <p> + Own Id: OTP-16532</p> + </item> + <item> + <p> + Avoid DoS attack against stateful session_tickets by + making session ticket ids unpredictable.</p> + <p> + Own Id: OTP-16533</p> + </item> + <item> + <p> + Add support for the max_fragment_length extension (RFC + 6066).</p> + <p> + Own Id: OTP-16547 Aux Id: PR-2547 </p> + </item> + <item> + <p> + Add srp_username in ssl:connection_info, update the + document with types of this function.</p> + <p> + Own Id: OTP-16584</p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 9.6.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix timing bug that could cause ssl sockets to become + unresponsive after an ssl:recv/3 call timed out</p> + <p> + Own Id: OTP-16619 Aux Id: ERL-1213 </p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.6.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -557,6 +728,22 @@ </section> +<section><title>SSL 9.2.3.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix timing bug that could cause ssl sockets to become + unresponsive after an ssl:recv/3 call timed out</p> + <p> + Own Id: OTP-16619 Aux Id: ERL-1213 </p> + </item> + </list> + </section> + +</section> + <section><title>SSL 9.2.3.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 566b5e7eed..1424c795a6 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -449,8 +449,8 @@ <code> fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom() | {revoked, atom()}} | - {extension, #'Extension'{}}, InitialUserState :: term()) -> - {valid, UserState :: term()} | {valid_peer, UserState :: term()} | + {extension, #'Extension'{}} | valid | valid_peer, InitialUserState :: term()) -> + {valid, UserState :: term()} | {fail, Reason :: term()} | {unknown, UserState :: term()}. </code> @@ -658,9 +658,9 @@ fun(Chain::[public_key:der_encoded()]) -> <desc><p>The lookup fun is to defined as follows:</p> <code> -fun(psk, PSKIdentity ::string(), UserState :: term()) -> +fun(psk, PSKIdentity :: binary(), UserState :: term()) -> {ok, SharedSecret :: binary()} | error; -fun(srp, Username :: string(), UserState :: term()) -> +fun(srp, Username :: binary(), UserState :: term()) -> {ok, {SRPParams :: srp_param_type(), Salt :: binary(), DerivedKey :: binary()}} | error. </code> @@ -879,6 +879,15 @@ fun(srp, Username :: string(), UserState :: term()) -> </datatype> <datatype> + <name name="max_fragment_length"/> + <desc> + <p>Specifies the maximum fragment length the client + is prepared to accept from the server. + See <url href="http://www.ietf.org/rfc/rfc6066.txt">RFC 6066</url></p> + </desc> + </datatype> + + <datatype> <name name="client_psk_identity"/> <desc> <p>Specifies the identity the client presents to the server. @@ -1006,8 +1015,7 @@ fun(srp, Username :: string(), UserState :: term()) -> </desc> </datatype> - <datatype_title>TLS/DTLS OPTION DESCRIPTIONS - SERVER </datatype_title> - + <datatype_title>TLS/DTLS OPTION DESCRIPTIONS - SERVER</datatype_title> <datatype> <name name="server_option"/> @@ -1260,6 +1268,39 @@ fun(srp, Username :: string(), UserState :: term()) -> </p></note> </desc> </datatype> + + <datatype> + <name name="connection_info"/> + </datatype> + + <datatype> + <name name="common_info"/> + </datatype> + + <datatype> + <name name="curve_info"/> + </datatype> + + <datatype> + <name name="ssl_options_info"/> + </datatype> + + <datatype> + <name name="security_info"/> + </datatype> + + <datatype> + <name name="connection_info_items"/> + </datatype> + + <datatype> + <name name="connection_info_item"/> + </datatype> + + <datatype> + <name name="tls_options_name"/> + </datatype> + </datatypes> <!-- @@ -1293,21 +1334,31 @@ fun(srp, Username :: string(), UserState :: term()) -> <func> <name name="cipher_suites" arity="2" since="OTP 20.3"/> - <fsummary>Returns a list of all default or - all supported cipher suites.</fsummary> - <desc><p>Returns all default or all supported (except anonymous), - or all anonymous cipher suites for a - TLS version</p> - - <note><p>The cipher suites returned by this function are the - cipher suites that the OTP ssl application can support provided - that they are supported by the cryptolib linked with the OTP - crypto application. Use <seemfa - marker="#filter_cipher_suites/2"> ssl:filter_cipher_suites(Suites, - []).</seemfa> to filter the list for the current - cryptolib. Note that cipher suites may be filtered out because - they are too old or too new depending on the - cryptolib</p></note> + <fsummary>Returns a list of cipher suites.</fsummary> + <desc><p>Lists all possible cipher suites corresponding to + <c>Description</c> that are available. The + <c>exclusive</c> option will exclusively list cipher suites + introduced in <c>Version</c> whereas the the other options + are inclusive from the lowest possible version to + <c>Version</c>. The <c>all</c> options includes all suites + except the anonymous. + </p> + + <note><p>TLS-1.3 has no overlapping cipher suites with previous + TLS versions, that is the result of + <c>cipher_suites(all, 'tlsv1.3').</c> contains a separate set of + suites that can be used with TLS-1.3 an other set that can be used + if a lower version is negotiated. No anonymous suites are + supported by TLS-1.3.</p> + + <p>Also note that the cipher suites returned + by this function are the cipher suites that the OTP ssl + application can support provided that they are supported by the + cryptolib linked with the OTP crypto application. Use <seemfa + marker="#filter_cipher_suites/2"> ssl:filter_cipher_suites(Suites, + []).</seemfa> to filter the list for the current cryptolib. Note + that cipher suites may be filtered out because they are too old or + too new depending on the cryptolib</p></note> </desc> </func> @@ -1836,18 +1887,19 @@ fun(srp, Username :: string(), UserState :: term()) -> <func> <name since="OTP R14B" name="versions" arity="0" /> - <fsummary>Returns version information relevant for the - SSL application.</fsummary> + <fsummary>Lists information, mainly concerning TLS/DTLS versions, + in runtime for debugging and testing purposes.</fsummary> <desc> - <p>Returns version information relevant for the SSL - application.</p> + <p>Lists information, mainly concerning TLS/DTLS versions, + in runtime for debugging and testing purposes. + </p> <taglist> <tag><c>app_vsn</c></tag> <item>The application version of the SSL application.</item> - <tag><c>default_supported</c></tag> - <item>TLS versions supported by default if crypto lib - support is sufficent· + <tag><c>supported</c></tag> + <item>TLS versions supported with current application environment + and crypto library configuration. Overridden by a version option on <seemfa marker="#connect/2"> connect/[2,3,4]</seemfa>, <seemfa marker="#listen/2"> listen/2</seemfa>, and <seemfa @@ -1856,9 +1908,9 @@ fun(srp, Username :: string(), UserState :: term()) -> marker="#connection_information/1">connection_information/1 </seemfa>.</item> - <tag><c>default_supported_dtls</c></tag> - <item>DTLS versions supported by default if crypto lib - support is sufficent· + <tag><c>supported_dtls</c></tag> + <item>DTLS versions supported with current application environment + and crypto library configuration. Overridden by a version option on <seemfa marker="#connect/2"> connect/[2,3,4]</seemfa>, <seemfa marker="#listen/2"> listen/2</seemfa>, and <seemfa @@ -1868,19 +1920,23 @@ fun(srp, Username :: string(), UserState :: term()) -> </seemfa>.</item> <tag><c>available</c></tag> - <item>All TLS versions than can be supported by the SSL application. + <item>All TLS versions supported with the + linked crypto library. </item> <tag><c>available_dtls</c></tag> - <item>All DTLS versions than can be supported by the SSL application.</item> + <item>All DTLS versions supported with the + linked crypto library.</item> - <tag><c>crypto_support</c></tag> - <item>TLS versions than has sufficient crypto lib - support through the Crypto applications currently linked crypto lib.</item> - - <tag><c>crypto_support_dtls</c></tag> - <item>All DTLS versions than has sufficient crypto lib - support through the Crypto applications currently linked crypto lib.</item> + <tag><c>implemented</c></tag> + <item>All TLS versions supported by the SSL + application if linked with a crypto library with the + necessary support.</item> + + <tag><c>implemented_dtls</c></tag> + <item>All DTLS versions supported by the SSL + application if linked with a crypto library with the + necessary support.</item> </taglist> </desc> diff --git a/lib/ssl/doc/src/standards_compliance.xml b/lib/ssl/doc/src/standards_compliance.xml index abbcfc46ce..7a803a6195 100644 --- a/lib/ssl/doc/src/standards_compliance.xml +++ b/lib/ssl/doc/src/standards_compliance.xml @@ -133,10 +133,8 @@ <item>Key Exchange: ECDHE</item> <item>Groups: all standard groups supported for the Diffie-Hellman key exchange</item> <item>Ciphers: all cipher suites are supported</item> - <item>Signature Algorithms: rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, - ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, - rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pkcs1_sha1 and ecdsa_sha1</item> - <item>Certificates: RSA (it MUST use the rsaEncryption OID) and ECDSA keys</item> + <item>Signature Algorithms: All algorithms form RFC 8456</item> + <item>Certificates: RSA and ECDSA keys</item> </list> <p>Other notable features:</p> <list type="bulleted"> @@ -180,7 +178,7 @@ <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">RSASSA-PSS signature schemes</cell> <cell align="left" valign="middle"><em>PC</em></cell> - <cell align="left" valign="middle"><em>22</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -305,8 +303,8 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">max_fragment_length (RFC6066)</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>ssl-@OTP-16547@</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -444,8 +442,8 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">max_fragment_length (RFC6066)</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>ssl-@OTP-16547@</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -704,8 +702,8 @@ </url> </cell> <cell align="left" valign="middle"><em>Client</em></cell> - <cell align="left" valign="middle"><em>PC</em></cell> - <cell align="left" valign="middle"><em>22.1</em></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -776,20 +774,20 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">rsa_pss_pss_sha256</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">rsa_pss_pss_sha384</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">rsa_pss_pss_sha512</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -879,20 +877,20 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">rsa_pss_pss_sha256</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">rsa_pss_pss_sha384</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">rsa_pss_pss_sha512</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>23</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -1234,8 +1232,8 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">max_fragment_length (RFC6066)</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>ssl-@OTP-16547@</em></cell> </row> <row> <cell align="left" valign="middle"></cell> @@ -1301,8 +1299,8 @@ <row> <cell align="left" valign="middle"></cell> <cell align="left" valign="middle">max_fragment_length (RFC6066)</cell> - <cell align="left" valign="middle"><em>NC</em></cell> - <cell align="left" valign="middle"></cell> + <cell align="left" valign="middle"><em>C</em></cell> + <cell align="left" valign="middle"><em>ssl-@OTP-16547@</em></cell> </row> <row> <cell align="left" valign="middle"></cell> diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index 0b99cdde52..d53b73a747 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -155,7 +155,7 @@ EXTRA_ERLC_FLAGS = +warn_unused_vars ERL_COMPILE_FLAGS += -I$(ERL_TOP)/lib/kernel/src \ -pz $(EBIN) \ -pz $(ERL_TOP)/lib/public_key/ebin \ - $(EXTRA_ERLC_FLAGS) + $(EXTRA_ERLC_FLAGS) -DVSN=\"$(VSN)\" # ---------------------------------------------------- # Targets diff --git a/lib/ssl/src/dtls_connection.erl b/lib/ssl/src/dtls_connection.erl index 975dc5fc4e..5d2f3a4253 100644 --- a/lib/ssl/src/dtls_connection.erl +++ b/lib/ssl/src/dtls_connection.erl @@ -448,9 +448,12 @@ init({call, From}, {start, Timeout}, Hello = dtls_handshake:client_hello(Host, Port, ConnectionStates0, SslOpts, Session#session.session_id, Renegotiation, Cert), + MaxFragEnum = maps:get(max_frag_enum, Hello#client_hello.extensions, undefined), + ConnectionStates1 = ssl_record:set_max_fragment_length(MaxFragEnum, ConnectionStates0), Version = Hello#client_hello.client_version, HelloVersion = dtls_record:hello_version(Version, Versions), - State1 = prepare_flight(State0#state{connection_env = CEnv#connection_env{negotiated_version = Version}}), + State1 = prepare_flight(State0#state{connection_env = CEnv#connection_env{negotiated_version = Version}, + connection_states = ConnectionStates1}), {State2, Actions} = send_handshake(Hello, State1#state{connection_env = CEnv#connection_env{negotiated_version = HelloVersion}}), State = State2#state{connection_env = CEnv#connection_env{negotiated_version = Version}, %% RequestedVersion session = Session, @@ -580,8 +583,9 @@ hello(internal, #server_hello{} = Hello, handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, connection_env = #connection_env{negotiated_version = ReqVersion}, connection_states = ConnectionStates0, + session = #session{session_id = OldId}, ssl_options = SslOptions} = State) -> - case dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of + case dtls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation, OldId) of #alert{} = Alert -> handle_own_alert(Alert, ReqVersion, ?FUNCTION_NAME, State); {Version, NewId, ConnectionStates, ProtoExt, Protocol} -> @@ -1107,9 +1111,11 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> - %% TODO remove hardcoded Max size + PMTUEstimate = 1400, %% TODO make configurable + #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, + MaxSize = min(MaxFragmentLength, PMTUEstimate), {Encoded, ConnectionStates} = - encode_handshake_flight(lists:reverse(Flight), Version, 1400, Epoch, ConnectionStates0), + encode_handshake_flight(lists:reverse(Flight), Version, MaxSize, Epoch, ConnectionStates0), send(Transport, Socket, Encoded), ssl_logger:debug(LogLevel, outbound, 'record', Encoded), {State0#state{connection_states = ConnectionStates}, []}; @@ -1123,8 +1129,11 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> + PMTUEstimate = 1400, %% TODO make configurable + #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, + MaxSize = min(MaxFragmentLength, PMTUEstimate), {HsBefore, ConnectionStates1} = - encode_handshake_flight(lists:reverse(Flight0), Version, 1400, Epoch, ConnectionStates0), + encode_handshake_flight(lists:reverse(Flight0), Version, MaxSize, Epoch, ConnectionStates0), {EncChangeCipher, ConnectionStates} = encode_change_cipher(ChangeCipher, Version, Epoch, ConnectionStates1), send(Transport, Socket, [HsBefore, EncChangeCipher]), @@ -1141,12 +1150,15 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> + PMTUEstimate = 1400, %% TODO make configurable + #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, + MaxSize = min(MaxFragmentLength, PMTUEstimate), {HsBefore, ConnectionStates1} = - encode_handshake_flight(lists:reverse(Flight0), Version, 1400, Epoch-1, ConnectionStates0), + encode_handshake_flight(lists:reverse(Flight0), Version, MaxSize, Epoch-1, ConnectionStates0), {EncChangeCipher, ConnectionStates2} = encode_change_cipher(ChangeCipher, Version, Epoch-1, ConnectionStates1), {HsAfter, ConnectionStates} = - encode_handshake_flight(lists:reverse(Flight1), Version, 1400, Epoch, ConnectionStates2), + encode_handshake_flight(lists:reverse(Flight1), Version, MaxSize, Epoch, ConnectionStates2), send(Transport, Socket, [HsBefore, EncChangeCipher, HsAfter]), ssl_logger:debug(LogLevel, outbound, 'record', [HsBefore]), ssl_logger:debug(LogLevel, outbound, 'record', [EncChangeCipher]), @@ -1162,10 +1174,13 @@ send_handshake_flight(#state{static_env = #static_env{socket = Socket, connection_states = ConnectionStates0, ssl_options = #{log_level := LogLevel}} = State0, Epoch) -> + PMTUEstimate = 1400, %% TODO make configurable + #{current_write := #{max_fragment_length := MaxFragmentLength}} = ConnectionStates0, + MaxSize = min(MaxFragmentLength, PMTUEstimate), {EncChangeCipher, ConnectionStates1} = encode_change_cipher(ChangeCipher, Version, Epoch-1, ConnectionStates0), {HsAfter, ConnectionStates} = - encode_handshake_flight(lists:reverse(Flight1), Version, 1400, Epoch, ConnectionStates1), + encode_handshake_flight(lists:reverse(Flight1), Version, MaxSize, Epoch, ConnectionStates1), send(Transport, Socket, [EncChangeCipher, HsAfter]), ssl_logger:debug(LogLevel, outbound, 'record', [EncChangeCipher]), ssl_logger:debug(LogLevel, outbound, 'record', [HsAfter]), diff --git a/lib/ssl/src/dtls_handshake.erl b/lib/ssl/src/dtls_handshake.erl index 861b1aacb4..157e3814c2 100644 --- a/lib/ssl/src/dtls_handshake.erl +++ b/lib/ssl/src/dtls_handshake.erl @@ -30,7 +30,7 @@ -include("ssl_alert.hrl"). %% Handshake handling --export([client_hello/7, client_hello/8, cookie/4, hello/4, +-export([client_hello/7, client_hello/8, cookie/4, hello/5, hello/4, hello_verify_request/2]). %% Handshake encoding @@ -97,15 +97,16 @@ hello(#server_hello{server_version = Version, random = Random, compression_method = Compression, session_id = SessionId, extensions = HelloExt}, #{versions := SupportedVersions} = SslOpt, - ConnectionStates0, Renegotiation) -> + ConnectionStates0, Renegotiation, OldId) -> + IsNew = ssl_session:is_new(OldId, SessionId), case dtls_record:is_acceptable_version(Version, SupportedVersions) of true -> handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, Compression, HelloExt, SslOpt, - ConnectionStates0, Renegotiation); + ConnectionStates0, Renegotiation, IsNew); false -> ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) - end; + end. hello(#client_hello{client_version = ClientVersion} = Hello, #{versions := Versions} = SslOpts, Info, Renegotiation) -> @@ -212,7 +213,8 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, try ssl_handshake:handle_client_hello_extensions(dtls_record, Random, CipherSuites, HelloExt, dtls_v1:corresponding_tls_version(Version), SslOpts, Session0, - ConnectionStates0, Renegotiation) of + ConnectionStates0, Renegotiation, + Session0#session.is_resumable) of {Session, ConnectionStates, Protocol, ServerHelloExt} -> {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign} catch throw:Alert -> @@ -220,11 +222,11 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, end. handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, - Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation) -> + Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew) -> try ssl_handshake:handle_server_hello_extensions(dtls_record, Random, CipherSuite, Compression, HelloExt, dtls_v1:corresponding_tls_version(Version), - SslOpt, ConnectionStates0, Renegotiation) of + SslOpt, ConnectionStates0, Renegotiation, IsNew) of {ConnectionStates, ProtoExt, Protocol} -> {Version, SessionId, ConnectionStates, ProtoExt, Protocol} catch throw:Alert -> diff --git a/lib/ssl/src/dtls_record.erl b/lib/ssl/src/dtls_record.erl index ee0ce2d22a..16542a8eb3 100644 --- a/lib/ssl/src/dtls_record.erl +++ b/lib/ssl/src/dtls_record.erl @@ -215,8 +215,26 @@ encode_change_cipher_spec(Version, Epoch, ConnectionStates) -> %% Description: Encodes data to send on the ssl-socket. %%-------------------------------------------------------------------- encode_data(Data, Version, ConnectionStates) -> - #{epoch := Epoch} = ssl_record:current_connection_state(ConnectionStates, write), - encode_plain_text(?APPLICATION_DATA, Version, Epoch, Data, ConnectionStates). + #{epoch := Epoch, max_fragment_length := MaxFragmentLength} + = ssl_record:current_connection_state(ConnectionStates, write), + MaxLength = if is_integer(MaxFragmentLength) -> + MaxFragmentLength; + true -> + ?MAX_PLAIN_TEXT_LENGTH + end, + case iolist_size(Data) of + N when N > MaxLength -> + Frags = tls_record:split_iovec(erlang:iolist_to_iovec(Data), MaxLength), + {RevCipherText, ConnectionStates1} = + lists:foldl(fun(Frag, {Acc, CS0}) -> + {CipherText, CS1} = + encode_plain_text(?APPLICATION_DATA, Version, Epoch, Frag, CS0), + {[CipherText|Acc], CS1} + end, {[], ConnectionStates}, Frags), + {lists:reverse(RevCipherText), ConnectionStates1}; + _ -> + encode_plain_text(?APPLICATION_DATA, Version, Epoch, Data, ConnectionStates) + end. encode_plain_text(Type, Version, Epoch, Data, ConnectionStates) -> Write0 = get_connection_state_by_epoch(Epoch, ConnectionStates, write), @@ -393,7 +411,8 @@ initial_connection_state(ConnectionEnd, BeastMitigation) -> mac_secret => undefined, secure_renegotiation => undefined, client_verify_data => undefined, - server_verify_data => undefined + server_verify_data => undefined, + max_fragment_length => undefined }. get_dtls_records_aux({DataTag, StateName, _, Versions} = Vinfo, <<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), diff --git a/lib/ssl/src/ssl.app.src b/lib/ssl/src/ssl.app.src index bebf35a20f..f08cb96d17 100644 --- a/lib/ssl/src/ssl.app.src +++ b/lib/ssl/src/ssl.app.src @@ -73,5 +73,5 @@ {applications, [crypto, public_key, kernel, stdlib]}, {env, []}, {mod, {ssl_app, []}}, - {runtime_dependencies, ["stdlib-3.5","public_key-1.7.2","kernel-6.0", + {runtime_dependencies, ["stdlib-3.5","public_key-1.8","kernel-6.0", "erts-10.0","crypto-4.2", "inets-5.10.7"]}]}. diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 5ba731dd54..7e52cf573d 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -34,6 +34,11 @@ -include("ssl_handshake.hrl"). -include("ssl_srp.hrl"). +%% Needed to make documentation rendering happy +-ifndef(VSN). +-define(VSN,"unknown"). +-endif. + %% Application handling -export([start/0, start/1, @@ -102,6 +107,9 @@ -deprecated({ssl_accept, '_', "use ssl_handshake/1,2,3 instead"}). +-deprecated({cipher_suites, 0, "use cipher_suites/2,3 instead"}). +-deprecated({cipher_suites, 1, "use cipher_suites/2,3 instead"}). + -removed([{negotiated_next_protocol,1, "use ssl:negotiated_protocol/1 instead"}]). -removed([{connection_info,1, @@ -344,7 +352,9 @@ -type custom_verify() :: {Verifyfun :: fun(), InitialUserState :: any()}. -type crl_check() :: boolean() | peer | best_effort. --type crl_cache_opts() :: [any()]. +-type crl_cache_opts() :: {Module :: atom(), + {DbHandle :: internal | term(), + Args :: list()}}. -type handshake_size() :: integer(). -type hibernate_after() :: timeout(). -type root_fun() :: fun(). @@ -385,6 +395,7 @@ {psk_identity, client_psk_identity()} | {srp_identity, client_srp_identity()} | {server_name_indication, sni()} | + {max_fragment_length, max_fragment_length()} | {customize_hostname_check, customize_hostname_check()} | {signature_algs, client_signature_algs()} | {fallback, fallback()} | @@ -407,6 +418,7 @@ -type client_srp_identity() :: srp_identity(). -type customize_hostname_check() :: list(). -type sni() :: HostName :: hostname() | disable. +-type max_fragment_length() :: undefined | 512 | 1024 | 2048 | 4096. -type client_signature_algs() :: signature_algs(). -type fallback() :: boolean(). -type ssl_imp() :: new | old. @@ -458,10 +470,37 @@ alpn => app_level_protocol(), srp => binary(), next_protocol => app_level_protocol(), + max_frag_enum => 1..4, ec_point_formats => [0..2], elliptic_curves => [public_key:oid()], sni => hostname()}. % exported %% ------------------------------------------------------------------------------------------------------- +-type connection_info() :: [common_info() | curve_info() | ssl_options_info() | security_info()]. +-type common_info() :: {protocol, protocol_version()} | + {session_id, session_id()} | + {session_resumption, boolean()} | + {selected_cipher_suite, erl_cipher_suite()} | + {sni_hostname, term()} | + {srp_username, term()}. +-type curve_info() :: {ecc, {named_curve, term()}}. +-type ssl_options_info() :: tls_option(). +-type security_info() :: {client_random, binary()} | + {server_random, binary()} | + {master_secret, binary()}. +-type connection_info_items() :: [connection_info_item()]. +-type connection_info_item() :: protocol | + session_id | + session_resumption | + selected_cipher_suite | + sni_hostname | + srp_username | + ecc | + client_random | + server_random | + master_secret | + tls_options_name(). +-type tls_options_name() :: atom(). +%% ------------------------------------------------------------------------------------------------------- %%%-------------------------------------------------------------------- %%% API @@ -900,9 +939,7 @@ controlling_process(#sslsocket{pid = {Listen, %%-------------------------------------------------------------------- -spec connection_information(SslSocket) -> {ok, Result} | {error, reason()} when SslSocket :: sslsocket(), - Result :: [{OptionName, OptionValue}], - OptionName :: atom(), - OptionValue :: any(). + Result :: connection_info(). %% %% Description: Return SSL information for the connection %%-------------------------------------------------------------------- @@ -921,10 +958,8 @@ connection_information(#sslsocket{pid = {dtls,_}}) -> %%-------------------------------------------------------------------- -spec connection_information(SslSocket, Items) -> {ok, Result} | {error, reason()} when SslSocket :: sslsocket(), - Items :: [OptionName], - Result :: [{OptionName, OptionValue}], - OptionName :: atom(), - OptionValue :: any(). + Items :: connection_info_items(), + Result :: connection_info(). %% %% Description: Return SSL information for the connection %%-------------------------------------------------------------------- @@ -1010,45 +1045,46 @@ cipher_suites(all) -> [ssl_cipher_format:suite_legacy(Suite) || Suite <- available_suites(all)]. %%-------------------------------------------------------------------- --spec cipher_suites(Supported, Version) -> ciphers() when - Supported :: default | all | anonymous, +-spec cipher_suites(Description, Version) -> ciphers() when + Description :: default | all | exclusive | anonymous, Version :: protocol_version(). %% Description: Returns all default and all supported cipher suites for a %% TLS/DTLS version %%-------------------------------------------------------------------- -cipher_suites(Base, Version) when Version == 'tlsv1.3'; +cipher_suites(Description, Version) when Version == 'tlsv1.3'; Version == 'tlsv1.2'; Version == 'tlsv1.1'; Version == tlsv1 -> - cipher_suites(Base, tls_record:protocol_version(Version)); -cipher_suites(Base, Version) when Version == 'dtlsv1.2'; + cipher_suites(Description, tls_record:protocol_version(Version)); +cipher_suites(Description, Version) when Version == 'dtlsv1.2'; Version == 'dtlsv1'-> - cipher_suites(Base, dtls_record:protocol_version(Version)); -cipher_suites(Base, Version) -> - [ssl_cipher_format:suite_bin_to_map(Suite) || Suite <- supported_suites(Base, Version)]. + cipher_suites(Description, dtls_record:protocol_version(Version)); +cipher_suites(Description, Version) -> + [ssl_cipher_format:suite_bin_to_map(Suite) || Suite <- supported_suites(Description, Version)]. %%-------------------------------------------------------------------- --spec cipher_suites(Supported, Version, rfc | openssl) -> [string()] when - Supported :: default | all | anonymous, +-spec cipher_suites(Description, Version, rfc | openssl) -> [string()] when + Description :: default | all | exclusive | anonymous, Version :: protocol_version(). %% Description: Returns all default and all supported cipher suites for a %% TLS/DTLS version %%-------------------------------------------------------------------- -cipher_suites(Base, Version, StringType) when Version == 'tlsv1.2'; - Version == 'tlsv1.1'; - Version == tlsv1 -> - cipher_suites(Base, tls_record:protocol_version(Version), StringType); -cipher_suites(Base, Version, StringType) when Version == 'dtlsv1.2'; +cipher_suites(Description, Version, StringType) when Version == 'tlsv1.3'; + Version == 'tlsv1.2'; + Version == 'tlsv1.1'; + Version == tlsv1 -> + cipher_suites(Description, tls_record:protocol_version(Version), StringType); +cipher_suites(Description, Version, StringType) when Version == 'dtlsv1.2'; Version == 'dtlsv1'-> - cipher_suites(Base, dtls_record:protocol_version(Version), StringType); -cipher_suites(Base, Version, rfc) -> - [ssl_cipher_format:suite_map_to_str(ssl_cipher_format:suite_bin_to_map(Suite)) - || Suite <- supported_suites(Base, Version)]; -cipher_suites(Base, Version, openssl) -> - [ssl_cipher_format:suite_map_to_openssl_str(ssl_cipher_format:suite_bin_to_map(Suite)) - || Suite <- supported_suites(Base, Version)]. + cipher_suites(Description, dtls_record:protocol_version(Version), StringType); +cipher_suites(Description, Version, rfc) -> + [ssl_cipher_format:suite_map_to_str(ssl_cipher_format:suite_bin_to_map(Suite)) + || Suite <- supported_suites(Description, Version)]; +cipher_suites(Description, Version, openssl) -> + [ssl_cipher_format:suite_map_to_openssl_str(ssl_cipher_format:suite_bin_to_map(Suite)) + || Suite <- supported_suites(Description, Version)]. %%-------------------------------------------------------------------- -spec filter_cipher_suites(Suites, Filters) -> Ciphers when @@ -1316,31 +1352,36 @@ sockname(#sslsocket{pid = [Pid| _], fd = {Transport, Socket,_,_}}) when is_pid(P %%--------------------------------------------------------------- -spec versions() -> [VersionInfo] when VersionInfo :: {ssl_app, string()} | - {supported | available, [tls_version()]} | - {supported_dtls | available_dtls, [dtls_version()]}. + {supported | available | implemented, [tls_version()]} | + {supported_dtls | available_dtls | implemented_dtls, [dtls_version()]}. %% %% Description: Returns a list of relevant versions. %%-------------------------------------------------------------------- versions() -> - TLSVsns = tls_record:supported_protocol_versions(), - DTLSVsns = dtls_record:supported_protocol_versions(), - SupportedTLSVsns = [tls_record:protocol_version(Vsn) || Vsn <- TLSVsns], - SupportedDTLSVsns = [dtls_record:protocol_version(Vsn) || Vsn <- DTLSVsns], - AvailableTLSVsns = ?ALL_AVAILABLE_VERSIONS, - AvailableDTLSVsns = ?ALL_AVAILABLE_DATAGRAM_VERSIONS, - CryptoSupVersionsTLS = [Vsn || Vsn <- AvailableTLSVsns, - tls_record:sufficient_crypto_support(Vsn)], - CryptoSupVersionsDTLS = [Vsn || Vsn <- AvailableDTLSVsns, - tls_record:sufficient_crypto_support(tls_record:protocol_version( - dtls_v1:corresponding_tls_version( - dtls_record:protocol_version(Vsn))))], - - [{ssl_app, "9.2"}, {default_supported, SupportedTLSVsns}, - {default_supported_dtls, SupportedDTLSVsns}, + ConfTLSVsns = tls_record:supported_protocol_versions(), + ConfDTLSVsns = dtls_record:supported_protocol_versions(), + ImplementedTLSVsns = ?ALL_AVAILABLE_VERSIONS, + ImplementedDTLSVsns = ?ALL_AVAILABLE_DATAGRAM_VERSIONS, + + TLSCryptoSupported = fun(Vsn) -> + tls_record:sufficient_crypto_support(Vsn) + end, + DTLSCryptoSupported = fun(Vsn) -> + tls_record:sufficient_crypto_support(dtls_v1:corresponding_tls_version(Vsn)) + end, + SupportedTLSVsns = [tls_record:protocol_version(Vsn) || Vsn <- ConfTLSVsns, TLSCryptoSupported(Vsn)], + SupportedDTLSVsns = [dtls_record:protocol_version(Vsn) || Vsn <- ConfDTLSVsns, DTLSCryptoSupported(Vsn)], + + AvailableTLSVsns = [Vsn || Vsn <- ImplementedTLSVsns, TLSCryptoSupported(tls_record:protocol_version(Vsn))], + AvailableDTLSVsns = [Vsn || Vsn <- ImplementedDTLSVsns, DTLSCryptoSupported(dtls_record:protocol_version(Vsn))], + + [{ssl_app, ?VSN}, + {supported, SupportedTLSVsns}, + {supported_dtls, SupportedDTLSVsns}, {available, AvailableTLSVsns}, {available_dtls, AvailableDTLSVsns}, - {crypto_support, CryptoSupVersionsTLS}, - {crypto_support_dtls, CryptoSupVersionsDTLS} + {implemented, ImplementedTLSVsns}, + {implemented_dtls, ImplementedDTLSVsns} ]. %%--------------------------------------------------------------- @@ -1509,6 +1550,8 @@ available_suites(all) -> Version = tls_record:highest_protocol_version([]), ssl_cipher:filter_suites(ssl_cipher:all_suites(Version)). +supported_suites(exclusive, {3,Minor}) -> + tls_v1:exclusive_suites(Minor); supported_suites(default, Version) -> ssl_cipher:suites(Version); supported_suites(all, Version) -> @@ -1600,7 +1643,9 @@ handle_option(anti_replay = Option, unbound, OptionsMap, #{rules := Rules}) -> Value = validate_option(Option, default_value(Option, Rules)), OptionsMap#{Option => Value}; handle_option(anti_replay = Option, Value0, - #{session_tickets := SessionTickets} = OptionsMap, #{rules := Rules}) -> + #{session_tickets := SessionTickets, + versions := Versions} = OptionsMap, #{rules := Rules}) -> + assert_option_dependency(Option, versions, Versions, ['tlsv1.3']), assert_option_dependency(Option, session_tickets, [SessionTickets], [stateless]), case SessionTickets of stateless -> @@ -1609,6 +1654,13 @@ handle_option(anti_replay = Option, Value0, _ -> OptionsMap#{Option => default_value(Option, Rules)} end; +handle_option(beast_mitigation = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(beast_mitigation = Option, Value0, #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(Option, versions, Versions, ['tlsv1']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; handle_option(cacertfile = Option, unbound, #{cacerts := CaCerts, verify := Verify, verify_fun := VerifyFun} = OptionsMap, _Env) @@ -1627,17 +1679,20 @@ handle_option(cacertfile = Option, unbound, #{cacerts := CaCerts, handle_option(cacertfile = Option, Value0, OptionsMap, _Env) -> Value = validate_option(Option, Value0), OptionsMap#{Option => Value}; -handle_option(ciphers = Option, unbound, #{versions := [HighestVersion|_]} = OptionsMap, #{rules := Rules}) -> - Value = handle_cipher_option(default_value(Option, Rules), HighestVersion), +handle_option(ciphers = Option, unbound, #{versions := Versions} = OptionsMap, #{rules := Rules}) -> + Value = handle_cipher_option(default_value(Option, Rules), Versions), OptionsMap#{Option => Value}; -handle_option(ciphers = Option, Value0, #{versions := [HighestVersion|_]} = OptionsMap, _Env) -> - Value = handle_cipher_option(Value0, HighestVersion), +handle_option(ciphers = Option, Value0, #{versions := Versions} = OptionsMap, _Env) -> + Value = handle_cipher_option(Value0, Versions), OptionsMap#{Option => Value}; handle_option(client_renegotiation = Option, unbound, OptionsMap, #{role := Role}) -> Value = default_option_role(server, true, Role), OptionsMap#{Option => Value}; -handle_option(client_renegotiation = Option, Value0, OptionsMap, #{role := Role}) -> +handle_option(client_renegotiation = Option, Value0, + #{versions := Versions} = OptionsMap, #{role := Role}) -> assert_role(server_only, Role, Option, Value0), + assert_option_dependency(Option, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), Value = validate_option(Option, Value0), OptionsMap#{Option => Value}; handle_option(eccs = Option, unbound, #{versions := [HighestVersion|_]} = OptionsMap, #{rules := _Rules}) -> @@ -1677,13 +1732,50 @@ handle_option(key_update_at = Option, Value0, #{versions := Versions} = OptionsM assert_option_dependency(Option, versions, Versions, ['tlsv1.3']), Value = validate_option(Option, Value0), OptionsMap#{Option => Value}; +handle_option(next_protocols_advertised = Option, unbound, OptionsMap, + #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(next_protocols_advertised = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(next_protocols_advertised, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; handle_option(next_protocol_selector = Option, unbound, OptionsMap, #{rules := Rules}) -> Value = default_value(Option, Rules), OptionsMap#{Option => Value}; -handle_option(next_protocol_selector = Option, Value0, OptionsMap, _Env) -> +handle_option(next_protocol_selector = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(client_preferred_next_protocols, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), Value = make_next_protocol_selector( validate_option(client_preferred_next_protocols, Value0)), OptionsMap#{Option => Value}; +handle_option(padding_check = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(padding_check = Option, Value0, #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(Option, versions, Versions, ['tlsv1']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; +handle_option(psk_identity = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(psk_identity = Option, Value0, #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(Option, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; +handle_option(secure_renegotiate = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(secure_renegotiate= Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(secure_renegotiate, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; handle_option(reuse_session = Option, unbound, OptionsMap, #{role := Role}) -> Value = case Role of @@ -1693,14 +1785,20 @@ handle_option(reuse_session = Option, unbound, OptionsMap, #{role := Role}) -> fun(_, _, _, _) -> true end end, OptionsMap#{Option => Value}; -handle_option(reuse_session = Option, Value0, OptionsMap, _Env) -> +handle_option(reuse_session = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(reuse_session, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), Value = validate_option(Option, Value0), OptionsMap#{Option => Value}; %% TODO: validate based on role handle_option(reuse_sessions = Option, unbound, OptionsMap, #{rules := Rules}) -> Value = validate_option(Option, default_value(Option, Rules)), OptionsMap#{Option => Value}; -handle_option(reuse_sessions = Option, Value0, OptionsMap, _Env) -> +handle_option(reuse_sessions = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(reuse_sessions, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), Value = validate_option(Option, Value0), OptionsMap#{Option => Value}; handle_option(server_name_indication = Option, unbound, OptionsMap, #{host := Host, @@ -1754,25 +1852,48 @@ handle_option(sni_fun = Option, Value0, OptionsMap, _Env) -> throw({error, {conflict_options, [sni_fun, sni_hosts]}}) end, OptionsMap#{Option => Value}; +handle_option(srp_identity = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(srp_identity = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(srp_identity, versions, Versions, + ['tlsv1','tlsv1.1','tlsv1.2']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; handle_option(supported_groups = Option, unbound, #{versions := [HighestVersion|_]} = OptionsMap, #{rules := _Rules}) -> Value = handle_supported_groups_option(groups(default), HighestVersion), OptionsMap#{Option => Value}; -handle_option(supported_groups = Option, Value0, #{versions := [HighestVersion|_]} = OptionsMap, _Env) -> +handle_option(supported_groups = Option, Value0, + #{versions := [HighestVersion|_] = Versions} = OptionsMap, _Env) -> + assert_option_dependency(Option, versions, Versions, ['tlsv1.3']), Value = handle_supported_groups_option(Value0, HighestVersion), OptionsMap#{Option => Value}; +handle_option(use_ticket = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(use_ticket = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(Option, versions, Versions, ['tlsv1.3']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; +handle_option(user_lookup_fun = Option, unbound, OptionsMap, #{rules := Rules}) -> + Value = validate_option(Option, default_value(Option, Rules)), + OptionsMap#{Option => Value}; +handle_option(user_lookup_fun = Option, Value0, + #{versions := Versions} = OptionsMap, _Env) -> + assert_option_dependency(Option, versions, Versions, ['tlsv1','tlsv1.1','tlsv1.2']), + Value = validate_option(Option, Value0), + OptionsMap#{Option => Value}; handle_option(verify = Option, unbound, OptionsMap, #{rules := Rules}) -> handle_verify_option(default_value(Option, Rules), OptionsMap); handle_option(verify = _Option, Value, OptionsMap, _Env) -> handle_verify_option(Value, OptionsMap); - handle_option(verify_fun = Option, unbound, #{verify := Verify} = OptionsMap, #{rules := Rules}) - when Verify =:= verify_none orelse - Verify =:= 0 -> + when Verify =:= verify_none -> OptionsMap#{Option => default_value(Option, Rules)}; handle_option(verify_fun = Option, unbound, #{verify := Verify} = OptionsMap, _Env) - when Verify =:= verify_peer orelse - Verify =:= 1 orelse - Verify =:= 2 -> + when Verify =:= verify_peer -> OptionsMap#{Option => undefined}; handle_option(verify_fun = Option, Value0, OptionsMap, _Env) -> Value = validate_option(Option, Value0), @@ -1923,27 +2044,39 @@ assert_role_value(server, Option, Value, ServerValues, _) -> throw({error, {options, role, {Option, {Value, {server, ServerValues}}}}}) end. - assert_option_dependency(Option, OptionDep, Values0, AllowedValues) -> - %% special handling for version - Values = - case OptionDep of - versions -> - lists:map(fun tls_record:protocol_version/1, Values0); - _ -> - Values0 - end, - Set1 = sets:from_list(Values), - Set2 = sets:from_list(AllowedValues), - case sets:size(sets:intersection(Set1, Set2)) > 0 of + case is_dtls_configured(Values0) of true -> + %% TODO: Check option dependency for DTLS ok; false -> - %% Message = build_error_message(Option, OptionDep, AllowedValues), - %% throw({error, {options, Message}}) - throw({error, {options, dependency, {Option, {OptionDep, AllowedValues}}}}) + %% special handling for version + Values = + case OptionDep of + versions -> + lists:map(fun tls_record:protocol_version/1, Values0); + _ -> + Values0 + end, + Set1 = sets:from_list(Values), + Set2 = sets:from_list(AllowedValues), + case sets:size(sets:intersection(Set1, Set2)) > 0 of + true -> + ok; + false -> + throw({error, {options, dependency, + {Option, {OptionDep, AllowedValues}}}}) + end end. +is_dtls_configured(Versions) -> + Fun = fun (Version) when Version =:= {254, 253} orelse + Version =:= {254, 255} -> + true; + (_) -> + false + end, + lists:any(Fun, Versions). validate_option(versions, Versions) -> validate_versions(Versions, Versions); @@ -1974,8 +2107,6 @@ validate_option(partial_chain, Value) when is_function(Value) -> Value; validate_option(fail_if_no_peer_cert, Value) when is_boolean(Value) -> Value; -validate_option(verify_client_once, Value) when is_boolean(Value) -> - Value; validate_option(depth, Value) when is_integer(Value), Value >= 0, Value =< 255-> Value; @@ -2136,6 +2267,13 @@ validate_option(server_name_indication, undefined) -> validate_option(server_name_indication, disable) -> disable; +%% RFC 6066, Section 4 +validate_option(max_fragment_length, I) when I == ?MAX_FRAGMENT_LENGTH_BYTES_1; I == ?MAX_FRAGMENT_LENGTH_BYTES_2; + I == ?MAX_FRAGMENT_LENGTH_BYTES_3; I == ?MAX_FRAGMENT_LENGTH_BYTES_4 -> + I; +validate_option(max_fragment_length, undefined) -> + undefined; + validate_option(sni_hosts, []) -> []; validate_option(sni_hosts, [{Hostname, SSLOptions} | Tail]) when is_list(Hostname) -> @@ -2365,8 +2503,8 @@ emulated_options(Protocol, Opts) -> dtls_socket:emulated_options(Opts) end. -handle_cipher_option(Value, Version) when is_list(Value) -> - try binary_cipher_suites(Version, Value) of +handle_cipher_option(Value, Versions) when is_list(Value) -> + try binary_cipher_suites(Versions, Value) of Suites -> Suites catch @@ -2376,37 +2514,44 @@ handle_cipher_option(Value, Version) when is_list(Value) -> throw({error, {options, {ciphers, Value}}}) end. -binary_cipher_suites(Version, []) -> +binary_cipher_suites([{3,4} = Version], []) -> + %% Defaults to all supported suites that does + %% not require explicit configuration TLS-1.3 + %% only mode. + default_binary_suites(exclusive, Version); +binary_cipher_suites([Version| _], []) -> %% Defaults to all supported suites that does %% not require explicit configuration - default_binary_suites(Version); -binary_cipher_suites(Version, [Map|_] = Ciphers0) when is_map(Map) -> + default_binary_suites(default, Version); +binary_cipher_suites(Versions, [Map|_] = Ciphers0) when is_map(Map) -> Ciphers = [ssl_cipher_format:suite_map_to_bin(C) || C <- Ciphers0], - binary_cipher_suites(Version, Ciphers); -binary_cipher_suites(Version, [Tuple|_] = Ciphers0) when is_tuple(Tuple) -> + binary_cipher_suites(Versions, Ciphers); +binary_cipher_suites(Versions, [Tuple|_] = Ciphers0) when is_tuple(Tuple) -> Ciphers = [ssl_cipher_format:suite_map_to_bin(tuple_to_map(C)) || C <- Ciphers0], - binary_cipher_suites(Version, Ciphers); -binary_cipher_suites(Version, [Cipher0 | _] = Ciphers0) when is_binary(Cipher0) -> + binary_cipher_suites(Versions, Ciphers); +binary_cipher_suites([Version |_] = Versions, [Cipher0 | _] = Ciphers0) when is_binary(Cipher0) -> All = ssl_cipher:all_suites(Version) ++ ssl_cipher:anonymous_suites(Version), case [Cipher || Cipher <- Ciphers0, lists:member(Cipher, All)] of [] -> %% Defaults to all supported suites that does %% not require explicit configuration - default_binary_suites(Version); + binary_cipher_suites(Versions, []); Ciphers -> Ciphers end; -binary_cipher_suites(Version, [Head | _] = Ciphers0) when is_list(Head) -> +binary_cipher_suites(Versions, [Head | _] = Ciphers0) when is_list(Head) -> %% Format: ["RC4-SHA","RC4-MD5"] Ciphers = [ssl_cipher_format:suite_openssl_str_to_map(C) || C <- Ciphers0], - binary_cipher_suites(Version, Ciphers); -binary_cipher_suites(Version, Ciphers0) -> + binary_cipher_suites(Versions, Ciphers); +binary_cipher_suites(Versions, Ciphers0) -> %% Format: "RC4-SHA:RC4-MD5" Ciphers = [ssl_cipher_format:suite_openssl_str_to_map(C) || C <- string:lexemes(Ciphers0, ":")], - binary_cipher_suites(Version, Ciphers). + binary_cipher_suites(Versions, Ciphers). -default_binary_suites(Version) -> +default_binary_suites(exclusive, {_, Minor}) -> + ssl_cipher:filter_suites(tls_v1:exclusive_suites(Minor)); +default_binary_suites(default, Version) -> ssl_cipher:filter_suites(ssl_cipher:suites(Version)). tuple_to_map({Kex, Cipher, Mac}) -> @@ -2541,19 +2686,14 @@ assert_proplist([Value | _]) -> throw({option_not_a_key_value_tuple, Value}). -handle_verify_option(verify_none, #{fail_if_no_peer_cert := _FailIfNoPeerCert} = OptionsMap) -> - OptionsMap#{verify => verify_none, - fail_if_no_peer_cert => false}; -handle_verify_option(verify_peer, #{fail_if_no_peer_cert := FailIfNoPeerCert} = OptionsMap) -> - OptionsMap#{verify => verify_peer, - fail_if_no_peer_cert => FailIfNoPeerCert}; -%% Handle 0, 1, 2 for backwards compatibility -handle_verify_option(0, OptionsMap) -> - handle_verify_option(verify_none, OptionsMap); -handle_verify_option(1, OptionsMap) -> - handle_verify_option(verify_peer, OptionsMap#{fail_if_no_peer_cert => false}); -handle_verify_option(2, OptionsMap) -> - handle_verify_option(verify_peer, OptionsMap#{fail_if_no_peer_cert => true}); +handle_verify_option(verify_none, #{fail_if_no_peer_cert := false} = OptionsMap) -> + OptionsMap#{verify => verify_none}; +handle_verify_option(verify_none, #{fail_if_no_peer_cert := true}) -> + throw({error, {options, incompatible, + {verify, verify_none}, + {fail_if_no_peer_cert, true}}}); +handle_verify_option(verify_peer, OptionsMap) -> + OptionsMap#{verify => verify_peer}; handle_verify_option(Value, _) -> throw({error, {options, {verify, Value}}}). diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index 6d718dfef9..ade1d396cd 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -133,7 +133,7 @@ file_to_crls(File, DbHandle) -> [Bin || {'CertificateList', Bin, not_encrypted} <- List]. %%-------------------------------------------------------------------- --spec validate(term(), {extension, #'Extension'{}} | {bad_cert, atom()} | valid, +-spec validate(term(), {extension, #'Extension'{}} | {bad_cert, atom()} | valid | valid_peer, term()) -> {valid, term()} | {fail, tuple()} | {unknown, term()}. @@ -321,6 +321,10 @@ public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorith public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorithm = ?'rsaEncryption'}, subjectPublicKey = Key}) -> Key; +public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = Params}, + subjectPublicKey = Key}) -> + {Key, Params}; public_key(#'OTPSubjectPublicKeyInfo'{algorithm = #'PublicKeyAlgorithm'{algorithm = ?'id-dsa', parameters = {params, Params}}, subjectPublicKey = Key}) -> diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index e9eb78203c..f08a7e6b00 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -34,18 +34,43 @@ -include("tls_handshake_1_3.hrl"). -include_lib("public_key/include/public_key.hrl"). --export([security_parameters/2, security_parameters/3, security_parameters_1_3/2, - cipher_init/3, nonce_seed/2, decipher/6, cipher/5, aead_encrypt/6, aead_decrypt/6, - suites/1, all_suites/1, crypto_support_filters/0, - chacha_suites/1, anonymous_suites/1, psk_suites/1, psk_suites_anon/1, - srp_suites/1, srp_suites_anon/1, - rc4_suites/1, des_suites/1, rsa_suites/1, - filter/3, filter_suites/1, filter_suites/2, - hash_algorithm/1, sign_algorithm/1, is_acceptable_hash/2, is_fallback/1, - random_bytes/1, calc_mac_hash/4, calc_mac_hash/6, - is_stream_ciphersuite/1, signature_scheme/1, - scheme_to_components/1, hash_size/1, effective_key_bits/1, - key_material/1, signature_algorithm_to_scheme/1]). +-export([security_parameters/2, + security_parameters/3, + security_parameters_1_3/2, + cipher_init/3, + nonce_seed/2, + decipher/6, + cipher/5, + aead_encrypt/6, + aead_decrypt/6, + suites/1, + all_suites/1, + crypto_support_filters/0, + anonymous_suites/1, + psk_suites/1, + psk_suites_anon/1, + srp_suites/1, + srp_suites_anon/1, + rc4_suites/1, + des_suites/1, + rsa_suites/1, + filter/3, + filter_suites/1, + filter_suites/2, + hash_algorithm/1, + sign_algorithm/1, + is_acceptable_hash/2, + is_fallback/1, + random_bytes/1, + calc_mac_hash/4, + calc_mac_hash/6, + is_stream_ciphersuite/1, + signature_scheme/1, + scheme_to_components/1, + hash_size/1, + effective_key_bits/1, + key_material/1, + signature_algorithm_to_scheme/1]). %% RFC 8446 TLS 1.3 -export([generate_client_shares/1, @@ -303,7 +328,6 @@ suites({_, Minor}) -> all_suites({3, _} = Version) -> suites(Version) - ++ chacha_suites(Version) ++ psk_suites(Version) ++ srp_suites(Version) ++ rsa_suites(Version) @@ -312,20 +336,6 @@ all_suites({3, _} = Version) -> all_suites(Version) -> dtls_v1:all_suites(Version). -%%-------------------------------------------------------------------- --spec chacha_suites(ssl_record:ssl_version() | integer()) -> - [ssl_cipher_format:cipher_suite()]. -%% -%% Description: Returns list of the chacha cipher suites, only supported -%% if explicitly set by user for now due to interop problems, proably need -%% to be fixed in crypto. -%%-------------------------------------------------------------------- -chacha_suites({3, _}) -> - [?TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, - ?TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, - ?TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256]; -chacha_suites(_) -> - []. %%-------------------------------------------------------------------- -spec anonymous_suites(ssl_record:ssl_version() | integer()) -> @@ -982,8 +992,20 @@ scheme_to_components(ecdsa_sha1) -> {sha1, ecdsa, undefined}; %% Handling legacy signature algorithms scheme_to_components({Hash,Sign}) -> {Hash, Sign, undefined}. - -%% TODO: Add support for ed25519, ed448, rsa_pss* +signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS', + parameters = #'RSASSA-PSS-params'{ + maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = HashAlgo}}}) -> + #'HashAlgorithm'{algorithm = HashOid} = HashAlgo, + case public_key:pkix_hash_type(HashOid) of + sha256 -> + rsa_pss_pss_sha256; + sha384 -> + rsa_pss_pss_sha384; + sha512 -> + rsa_pss_pss_sha512 + end; signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?sha256WithRSAEncryption}) -> rsa_pkcs1_sha256; signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?sha384WithRSAEncryption}) -> @@ -1001,8 +1023,18 @@ signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'sha-1WithRSAEn signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?sha1WithRSAEncryption}) -> rsa_pkcs1_sha1; signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'ecdsa-with-SHA1'}) -> - ecdsa_sha1. - + ecdsa_sha1; +signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'id-Ed25519'}) -> + eddsa_ed25519; +signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'id-Ed448'}) -> + eddsa_ed448; +signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'rsaEncryption', + parameters = ?NULL}) -> + rsa_pkcs1_sha1; +signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'rsaEncryption'}) -> + rsa_pss_rsae; +signature_algorithm_to_scheme(#'SignatureAlgorithm'{algorithm = ?'id-RSASSA-PSS'}) -> + rsa_pss_pss. %% RFC 5246: 6.2.3.2. CBC Block Cipher %% diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index 8e2e794280..8ee9261c38 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -1652,7 +1652,10 @@ connection_info(#state{static_env = #static_env{protocol_cb = Connection}, handshake_env = #handshake_env{sni_hostname = SNIHostname, resumption = Resumption}, session = #session{session_id = SessionId, - cipher_suite = CipherSuite, ecc = ECCCurve}, + cipher_suite = CipherSuite, + srp_username = SrpUsername, + ecc = ECCCurve}, + connection_states = #{current_write := CurrentWrite}, connection_env = #connection_env{negotiated_version = {_,_} = Version}, ssl_options = Opts}) -> RecordCB = record_cb(Connection), @@ -1665,12 +1668,19 @@ connection_info(#state{static_env = #static_env{protocol_cb = Connection}, _ -> [] end, + + MFLInfo = case maps:get(max_fragment_length, CurrentWrite, undefined) of + MaxFragmentLength when is_integer(MaxFragmentLength) -> + [{max_fragment_length, MaxFragmentLength}]; + _ -> + [] + end, [{protocol, RecordCB:protocol_version(Version)}, {session_id, SessionId}, {session_resumption, Resumption}, - {cipher_suite, ssl_cipher_format:suite_legacy(CipherSuiteDef)}, {selected_cipher_suite, CipherSuiteDef}, - {sni_hostname, SNIHostname} | CurveInfo] ++ ssl_options_list(Opts). + {sni_hostname, SNIHostname}, + {srp_username, SrpUsername} | CurveInfo] ++ MFLInfo ++ ssl_options_list(Opts). security_info(#state{connection_states = ConnectionStates}) -> #{security_parameters := @@ -2807,6 +2817,9 @@ ssl_options_list([{erl_dist, _}|T], Acc) -> ssl_options_list(T, Acc); ssl_options_list([{renegotiate_at, _}|T], Acc) -> ssl_options_list(T, Acc); +ssl_options_list([{max_fragment_length, _}|T], Acc) -> + %% skip max_fragment_length from options since it is taken above from connection_states + ssl_options_list(T, Acc); ssl_options_list([{ciphers = Key, Value}|T], Acc) -> ssl_options_list(T, [{Key, lists:map( diff --git a/lib/ssl/src/ssl_connection.hrl b/lib/ssl/src/ssl_connection.hrl index 89bbdd0f54..b7e9e769ea 100644 --- a/lib/ssl/src/ssl_connection.hrl +++ b/lib/ssl/src/ssl_connection.hrl @@ -67,6 +67,7 @@ %% Ext handling hello, %%:: #client_hello{} | #server_hello{} sni_hostname = undefined, + max_frag_enum :: undefined | {max_frag_enum, integer()}, expecting_next_protocol_negotiation = false ::boolean(), next_protocol = undefined :: undefined | binary(), alpn = undefined, %% Used in TLS 1.3 diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index e1d629a3e3..c17062d888 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -48,7 +48,7 @@ %% Create handshake messages -export([hello_request/0, server_hello/4, server_hello_done/0, certificate/4, client_certificate_verify/6, certificate_request/5, key_exchange/3, - finished/5, next_protocol/1]). + finished/5, next_protocol/1, digitally_signed/5]). %% Handle handshake messages -export([certify/7, certificate_verify/6, verify_signature/5, @@ -73,11 +73,11 @@ %% Extensions handling -export([client_hello_extensions/7, - handle_client_hello_extensions/9, %% Returns server hello extensions - handle_server_hello_extensions/9, select_curve/2, select_curve/3, + handle_client_hello_extensions/10, %% Returns server hello extensions + handle_server_hello_extensions/10, select_curve/2, select_curve/3, select_hashsign/4, select_hashsign/5, select_hashsign_algs/3, empty_extensions/2, add_server_share/3, - add_alpn/2, add_selected_version/1, decode_alpn/1 + add_alpn/2, add_selected_version/1, decode_alpn/1, max_frag_enum/1 ]). -export([get_cert_params/1, @@ -171,7 +171,7 @@ client_certificate_verify(OwnCert, MasterSecret, Version, false -> Hashes = calc_certificate_verify(Version, HashAlgo, MasterSecret, Handshake), - Signed = digitally_signed(Version, Hashes, HashAlgo, PrivateKey), + Signed = digitally_signed(Version, Hashes, HashAlgo, PrivateKey, SignAlgo), #certificate_verify{signature = Signed, hashsign_algorithm = {HashAlgo, SignAlgo}} end. @@ -400,27 +400,34 @@ certificate_verify(Signature, PublicKeyInfo, Version, %% %% Description: Checks that a public_key signature is valid. %%-------------------------------------------------------------------- -verify_signature(_Version, _Hash, {_HashAlgo, anon}, _Signature, _) -> - true; -verify_signature({3, Minor}, Hash, {HashAlgo, rsa_pss_rsae}, Signature, {?rsaEncryption, PubKey, _PubKeyParams}) - when Minor >= 3 -> - public_key:verify({digest, Hash}, HashAlgo, Signature, PubKey, - [{rsa_padding, rsa_pkcs1_pss_padding}, - {rsa_pss_saltlen, -1}, - {rsa_mgf1_md, HashAlgo}]); -verify_signature({3, Minor}, Hash, {HashAlgo, rsa}, Signature, {?rsaEncryption, PubKey, _PubKeyParams}) +verify_signature({3, 4}, Hash, {HashAlgo, SignAlgo}, Signature, + {_, PubKey, PubKeyParams}) when SignAlgo == rsa_pss_rsae; + SignAlgo == rsa_pss_pss -> + Options = verify_options(SignAlgo, HashAlgo, PubKeyParams), + public_key:verify(Hash, HashAlgo, Signature, PubKey, Options); +verify_signature({3, 3}, Hash, {HashAlgo, SignAlgo}, Signature, + {_, PubKey, PubKeyParams}) when SignAlgo == rsa_pss_rsae; + SignAlgo == rsa_pss_pss -> + Options = verify_options(SignAlgo, HashAlgo, PubKeyParams), + public_key:verify({digest, Hash}, HashAlgo, Signature, PubKey, Options); +verify_signature({3, Minor}, Hash, {HashAlgo, SignAlgo}, Signature, {?rsaEncryption, PubKey, PubKeyParams}) when Minor >= 3 -> - public_key:verify({digest, Hash}, HashAlgo, Signature, PubKey); -verify_signature(_Version, Hash, _HashAlgo, Signature, {?rsaEncryption, PubKey, _PubKeyParams}) -> + Options = verify_options(SignAlgo, HashAlgo, PubKeyParams), + public_key:verify({digest, Hash}, HashAlgo, Signature, PubKey, Options); +verify_signature({3, Minor}, Hash, _HashAlgo, Signature, {?rsaEncryption, PubKey, _PubKeyParams}) when Minor =< 2 -> case public_key:decrypt_public(Signature, PubKey, [{rsa_pad, rsa_pkcs1_padding}]) of Hash -> true; - _ -> false + _ -> false end; -verify_signature(_Version, Hash, {HashAlgo, dsa}, Signature, {?'id-dsa', PublicKey, PublicKeyParams}) -> - public_key:verify({digest, Hash}, HashAlgo, Signature, {PublicKey, PublicKeyParams}); +verify_signature({3, 4}, Hash, {HashAlgo, _SignAlgo}, Signature, {?'id-ecPublicKey', PubKey, PubKeyParams}) -> + public_key:verify(Hash, HashAlgo, Signature, {PubKey, PubKeyParams}); verify_signature(_, Hash, {HashAlgo, _SignAlg}, Signature, {?'id-ecPublicKey', PublicKey, PublicKeyParams}) -> + public_key:verify({digest, Hash}, HashAlgo, Signature, {PublicKey, PublicKeyParams}); +verify_signature({3, Minor}, _Hash, {_HashAlgo, anon}, _Signature, _) when Minor =< 3 -> + true; +verify_signature({3, Minor}, Hash, {HashAlgo, dsa}, Signature, {?'id-dsa', PublicKey, PublicKeyParams}) when Minor =< 3-> public_key:verify({digest, Hash}, HashAlgo, Signature, {PublicKey, PublicKeyParams}). %%-------------------------------------------------------------------- @@ -695,6 +702,10 @@ encode_extensions([#sni{hostname = Hostname} | Rest], Acc) -> ?BYTE(?SNI_NAMETYPE_HOST_NAME), ?UINT16(HostLen), HostnameBin/binary, Acc/binary>>); +encode_extensions([#max_frag_enum{enum = MaxFragEnum} | Rest], Acc) -> + ExtLength = 1, + encode_extensions(Rest, <<?UINT16(?MAX_FRAGMENT_LENGTH_EXT), ?UINT16(ExtLength), ?BYTE(MaxFragEnum), + Acc/binary>>); encode_extensions([#client_hello_versions{versions = Versions0} | Rest], Acc) -> Versions = encode_versions(Versions0), VerLen = byte_size(Versions), @@ -1088,7 +1099,8 @@ client_hello_extensions(Version, CipherSuites, SslOpts, ConnectionStates, Renego add_tls12_extensions(_Version, #{alpn_advertised_protocols := AlpnAdvertisedProtocols, next_protocol_selector := NextProtocolSelector, - server_name_indication := ServerNameIndication} = SslOpts, + server_name_indication := ServerNameIndication, + max_fragment_length := MaxFragmentLength} = SslOpts, ConnectionStates, Renegotiation) -> SRP = srp_user(SslOpts), @@ -1099,7 +1111,8 @@ add_tls12_extensions(_Version, next_protocol_negotiation => encode_client_protocol_negotiation(NextProtocolSelector, Renegotiation), - sni => sni(ServerNameIndication) + sni => sni(ServerNameIndication), + max_frag_enum => max_frag_enum(MaxFragmentLength) }. @@ -1284,18 +1297,27 @@ handle_client_hello_extensions(RecordCB, Random, ClientCipherSuites, alpn_preferred_protocols := ALPNPreferredProtocols} = Opts, #session{cipher_suite = NegotiatedCipherSuite, compression_method = Compression} = Session0, - ConnectionStates0, Renegotiation) -> + ConnectionStates0, Renegotiation, IsResumed) -> Session = handle_srp_extension(maps:get(srp, Exts, undefined), Session0), + MaxFragEnum = handle_mfl_extension(maps:get(max_frag_enum, Exts, undefined)), + ConnectionStates1 = ssl_record:set_max_fragment_length(MaxFragEnum, ConnectionStates0), ConnectionStates = handle_renegotiation_extension(server, RecordCB, Version, maps:get(renegotiation_info, Exts, undefined), Random, NegotiatedCipherSuite, ClientCipherSuites, Compression, - ConnectionStates0, Renegotiation, SecureRenegotation), + ConnectionStates1, Renegotiation, SecureRenegotation), Empty = empty_extensions(Version, server_hello), + %% RFC 6066 - server doesn't include max_fragment_length for resumed sessions + ServerMaxFragEnum = if IsResumed -> + undefined; + true -> + MaxFragEnum + end, ServerHelloExtensions = Empty#{renegotiation_info => renegotiation_info(RecordCB, server, ConnectionStates, Renegotiation), ec_point_formats => server_ecc_extension(Version, - maps:get(ec_point_formats, Exts, undefined)) + maps:get(ec_point_formats, Exts, undefined)), + max_frag_enum => ServerMaxFragEnum }, %% If we receive an ALPN extension and have ALPN configured for this connection, @@ -1318,13 +1340,28 @@ handle_server_hello_extensions(RecordCB, Random, CipherSuite, Compression, Exts, Version, #{secure_renegotiate := SecureRenegotation, next_protocol_selector := NextProtoSelector}, - ConnectionStates0, Renegotiation) -> + ConnectionStates0, Renegotiation, IsNew) -> ConnectionStates = handle_renegotiation_extension(client, RecordCB, Version, maps:get(renegotiation_info, Exts, undefined), Random, CipherSuite, undefined, Compression, ConnectionStates0, Renegotiation, SecureRenegotation), + %% RFC 6066: handle received/expected maximum fragment length + if IsNew -> + ServerMaxFragEnum = maps:get(max_frag_enum, Exts, undefined), + #{current_write := #{max_fragment_length := ConnMaxFragLen}} = ConnectionStates, + ClientMaxFragEnum = max_frag_enum(ConnMaxFragLen), + + if ServerMaxFragEnum == ClientMaxFragEnum -> + ok; + true -> + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)) + end; + true -> + ok + end, + %% If we receive an ALPN extension then this is the protocol selected, %% otherwise handle the NPN extension. ALPN = maps:get(alpn, Exts, undefined), @@ -1391,7 +1428,7 @@ select_hashsign({#hash_sign_algos{hash_sign_algos = ClientHashSigns}, Cert, KeyExAlgo, SupportedHashSigns, {Major, Minor}) when Major >= 3 andalso Minor >= 3 -> ClientSignatureSchemes = get_signature_scheme(ClientSignatureSchemes0), - {SignAlgo0, Param, PublicKeyAlgo0} = get_cert_params(Cert), + {SignAlgo0, Param, PublicKeyAlgo0, _} = get_cert_params(Cert), SignAlgo = sign_algo(SignAlgo0), PublicKeyAlgo = public_key_algo(PublicKeyAlgo0), @@ -1445,7 +1482,7 @@ select_hashsign(#certificate_request{ Cert, SupportedHashSigns, {Major, Minor}) when Major >= 3 andalso Minor >= 3-> - {SignAlgo0, Param, PublicKeyAlgo0} = get_cert_params(Cert), + {SignAlgo0, Param, PublicKeyAlgo0, _} = get_cert_params(Cert), SignAlgo = sign_algo(SignAlgo0), PublicKeyAlgo = public_key_algo(PublicKeyAlgo0), @@ -1468,7 +1505,7 @@ select_hashsign(#certificate_request{ ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_signature_algorithm) end; select_hashsign(#certificate_request{certificate_types = Types}, Cert, _, Version) -> - {_, _, PublicKeyAlgo0} = get_cert_params(Cert), + {_, _, PublicKeyAlgo0, _} = get_cert_params(Cert), PublicKeyAlgo = public_key_algo(PublicKeyAlgo0), %% Check cert even for TLS 1.0/1.1 @@ -1484,14 +1521,23 @@ select_hashsign(#certificate_request{certificate_types = Types}, Cert, _, Versio %% - signature algorithm %% - parameters of the signature algorithm %% - public key algorithm (key type) +%% - RSA key size in bytes get_cert_params(Cert) -> #'OTPCertificate'{tbsCertificate = TBSCert, signatureAlgorithm = {_,SignAlgo, Param}} = public_key:pkix_decode_cert(Cert, otp), - #'OTPSubjectPublicKeyInfo'{algorithm = {_, PublicKeyAlgo, _}} = + #'OTPSubjectPublicKeyInfo'{algorithm = {_, PublicKeyAlgo, _}, + subjectPublicKey = PublicKey} = TBSCert#'OTPTBSCertificate'.subjectPublicKeyInfo, - {SignAlgo, Param, PublicKeyAlgo}. - + RSAKeySize = + case PublicKey of + #'RSAPublicKey'{modulus = Modulus} -> + %% Get RSA key size in bytes + byte_size(binary:encode_unsigned(Modulus)); + _ -> + undefined + end, + {SignAlgo, Param, PublicKeyAlgo, RSAKeySize}. get_signature_scheme(undefined) -> undefined; @@ -1553,6 +1599,8 @@ extension_value(#hash_sign_algos{hash_sign_algos = Algos}) -> Algos; extension_value(#alpn{extension_data = Data}) -> Data; +extension_value(#max_frag_enum{enum = Enum}) -> + Enum; extension_value(#next_protocol_negotiation{extension_data = Data}) -> Data; extension_value(#srp{username = Name}) -> @@ -1680,9 +1728,10 @@ validation_fun_and_state(undefined, Role, CertDbHandle, CertDbRef, SslState) end, {Role, CertDbHandle, CertDbRef, {ServerNameIndication, CustomizeHostCheck}, CRLCheck, CRLDbHandle}}. -apply_user_fun(Fun, OtpCert, VerifyResult, UserState0, +apply_user_fun(Fun, OtpCert, VerifyResult0, UserState0, {_, CertDbHandle, CertDbRef, _, CRLCheck, CRLDbHandle} = SslState, CertPath, LogLevel) when - (VerifyResult == valid) or (VerifyResult == valid_peer) -> + (VerifyResult0 == valid) or (VerifyResult0 == valid_peer) -> + VerifyResult = maybe_check_hostname(OtpCert, VerifyResult0, SslState), case Fun(OtpCert, VerifyResult, UserState0) of {Valid, UserState} when (Valid == valid) or (Valid == valid_peer) -> case crl_check(OtpCert, CRLCheck, CertDbHandle, CertDbRef, @@ -1705,6 +1754,16 @@ apply_user_fun(Fun, OtpCert, ExtensionOrError, UserState0, SslState, _CertPath, {unknown, {SslState, UserState}} end. +maybe_check_hostname(OtpCert, valid_peer, SslState) -> + case ssl_certificate:validate(OtpCert, valid_peer, SslState) of + {valid, _} -> + valid_peer; + {fail, Reason} -> + Reason + end; +maybe_check_hostname(_, valid, _) -> + valid. + handle_path_validation_error({bad_cert, unknown_ca} = Reason, PeerCert, Chain, Opts, Options, CertDbHandle, CertsDbRef) -> handle_incomplete_chain(PeerCert, Chain, Opts, Options, CertDbHandle, CertsDbRef, Reason); @@ -1780,30 +1839,64 @@ path_validation_alert({bad_cert, unknown_ca}) -> path_validation_alert(Reason) -> ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, Reason). -digitally_signed(Version, Hashes, HashAlgo, PrivateKey) -> - try do_digitally_signed(Version, Hashes, HashAlgo, PrivateKey) of +digitally_signed(Version, Hashes, HashAlgo, PrivateKey, SignAlgo) -> + try do_digitally_signed(Version, Hashes, HashAlgo, PrivateKey, SignAlgo) of Signature -> Signature catch error:badkey-> throw(?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, bad_key(PrivateKey))) end. -do_digitally_signed({3, Minor}, Hash, HashAlgo, #{algorithm := Alg} = Engine) - when Minor >= 3 -> - crypto:sign(Alg, HashAlgo, {digest, Hash}, maps:remove(algorithm, Engine)); -do_digitally_signed({3, Minor}, Hash, HashAlgo, Key) when Minor >= 3 -> - public_key:sign({digest, Hash}, HashAlgo, Key); -do_digitally_signed(_Version, Hash, _HashAlgo, #'RSAPrivateKey'{} = Key) -> - public_key:encrypt_private(Hash, Key, - [{rsa_pad, rsa_pkcs1_padding}]); -do_digitally_signed({3, _}, Hash, _, - #{algorithm := rsa} = Engine) -> + +do_digitally_signed({3, Minor}, Hash, _, + #{algorithm := rsa} = Engine, rsa) when Minor =< 2-> crypto:private_encrypt(rsa, Hash, maps:remove(algorithm, Engine), rsa_pkcs1_padding); -do_digitally_signed({3, _}, Hash, HashAlgo, #{algorithm := Alg} = Engine) -> - crypto:sign(Alg, HashAlgo, {digest, Hash}, maps:remove(algorithm, Engine)); -do_digitally_signed(_Version, Hash, HashAlgo, Key) -> +do_digitally_signed({3, Minor}, Hash, HashAlgo, #{algorithm := Alg} = Engine, SignAlgo) + when Minor > 3 -> + Options = signature_options(SignAlgo, HashAlgo), + crypto:sign(Alg, HashAlgo, Hash, maps:remove(algorithm, Engine), Options); +do_digitally_signed({3, Minor}, Hash, HashAlgo, #{algorithm := Alg} = Engine, SignAlgo) + when Minor > 3 -> + Options = signature_options(SignAlgo, HashAlgo), + crypto:sign(Alg, HashAlgo, Hash, maps:remove(algorithm, Engine), Options); +do_digitally_signed({3, 3}, Hash, HashAlgo, #{algorithm := Alg} = Engine, SignAlgo) -> + Options = signature_options(SignAlgo, HashAlgo), + crypto:sign(Alg, HashAlgo, {digest, Hash}, maps:remove(algorithm, Engine), Options); +do_digitally_signed({3, 4}, Hash, HashAlgo, {#'RSAPrivateKey'{} = Key, + #'RSASSA-PSS-params'{}}, SignAlgo) -> + Options = signature_options(SignAlgo, HashAlgo), + public_key:sign(Hash, HashAlgo, Key, Options); +do_digitally_signed({3, 4}, Hash, HashAlgo, Key, SignAlgo) -> + Options = signature_options(SignAlgo, HashAlgo), + public_key:sign(Hash, HashAlgo, Key, Options); +do_digitally_signed({3, Minor}, Hash, HashAlgo, Key, SignAlgo) when Minor >= 3 -> + Options = signature_options(HashAlgo, SignAlgo), + public_key:sign({digest,Hash}, HashAlgo, Key, Options); +do_digitally_signed({3, Minor}, Hash, _HashAlgo, #'RSAPrivateKey'{} = Key, rsa) when Minor =< 2 -> + public_key:encrypt_private(Hash, Key, + [{rsa_pad, rsa_pkcs1_padding}]); +do_digitally_signed(_Version, Hash, HashAlgo, Key, _SignAlgo) -> public_key:sign({digest, Hash}, HashAlgo, Key). + +signature_options(SignAlgo, HashAlgo) when SignAlgo =:= rsa_pss_rsae orelse + SignAlgo =:= rsa_pss_pss -> + pss_options(HashAlgo); +signature_options(_, _) -> + []. + +verify_options(SignAlgo, HashAlgo, _KeyParams) + when SignAlgo =:= rsa_pss_rsae orelse + SignAlgo =:= rsa_pss_pss -> + pss_options(HashAlgo); +verify_options(_, _, _) -> + []. + +pss_options(HashAlgo) -> + %% of the digest algorithm: rsa_pss_saltlen = -1 + [{rsa_padding, rsa_pkcs1_pss_padding}, + {rsa_pss_saltlen, -1}, + {rsa_mgf1_md, HashAlgo}]. bad_key(#'DSAPrivateKey'{}) -> unacceptable_dsa_key; @@ -2160,7 +2253,7 @@ enc_server_key_exchange(Version, Params, {HashAlgo, SignAlgo}, server_key_exchange_hash(HashAlgo, <<ClientRandom/binary, ServerRandom/binary, EncParams/binary>>), - Signature = digitally_signed(Version, Hash, HashAlgo, PrivateKey), + Signature = digitally_signed(Version, Hash, HashAlgo, PrivateKey, SignAlgo), #server_key_params{params = Params, params_bin = EncParams, hashsign = {HashAlgo, SignAlgo}, @@ -2574,6 +2667,10 @@ decode_extensions(<<?UINT16(?SNI_EXT), ?UINT16(Len), decode_extensions(Rest, Version, MessageType, Acc#{sni => dec_sni(NameList)}); +decode_extensions(<<?UINT16(?MAX_FRAGMENT_LENGTH_EXT), ?UINT16(1), ?BYTE(MaxFragEnum), Rest/binary>>, + Version, MessageType, Acc) -> + %% RFC 6066 Section 4 + decode_extensions(Rest, Version, MessageType, Acc#{max_frag_enum => #max_frag_enum{enum = MaxFragEnum}}); decode_extensions(<<?UINT16(?SUPPORTED_VERSIONS_EXT), ?UINT16(Len), ExtData:Len/binary, Rest/binary>>, Version, MessageType, Acc) when Len > 2 -> <<?BYTE(_),Versions/binary>> = ExtData, @@ -2935,6 +3032,13 @@ handle_alpn_extension([ServerProtocol|Tail], ClientProtocols) -> false -> handle_alpn_extension(Tail, ClientProtocols) end. +handle_mfl_extension(#max_frag_enum{enum = Enum}=MaxFragEnum) when Enum >= 1, Enum =< 4 -> + MaxFragEnum; +handle_mfl_extension(#max_frag_enum{}) -> + throw(?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)); +handle_mfl_extension(_) -> + undefined. + handle_next_protocol(undefined, _NextProtocolSelector, _Renegotiating) -> undefined; @@ -3153,6 +3257,18 @@ sni(disable) -> sni(Hostname) -> #sni{hostname = Hostname}. +%% convert max_fragment_length (in bytes) to the RFC 6066 ENUM +max_frag_enum(?MAX_FRAGMENT_LENGTH_BYTES_1) -> + #max_frag_enum{enum = 1}; +max_frag_enum(?MAX_FRAGMENT_LENGTH_BYTES_2) -> + #max_frag_enum{enum = 2}; +max_frag_enum(?MAX_FRAGMENT_LENGTH_BYTES_3) -> + #max_frag_enum{enum = 3}; +max_frag_enum(?MAX_FRAGMENT_LENGTH_BYTES_4) -> + #max_frag_enum{enum = 4}; +max_frag_enum(undefined) -> + undefined. + renegotiation_info(_, client, _, false) -> #renegotiation_info{renegotiated_connection = undefined}; renegotiation_info(_RecordCB, server, ConnectionStates, false) -> @@ -3270,7 +3386,7 @@ empty_extensions() -> empty_extensions({3,4}, client_hello) -> #{ sni => undefined, - %% max_fragment_length => undefined, + %% max_frag_enum => undefined, %% status_request => undefined, elliptic_curves => undefined, signature_algs => undefined, diff --git a/lib/ssl/src/ssl_handshake.hrl b/lib/ssl/src/ssl_handshake.hrl index a772567846..ac397a8d88 100644 --- a/lib/ssl/src/ssl_handshake.hrl +++ b/lib/ssl/src/ssl_handshake.hrl @@ -389,8 +389,18 @@ hostname = undefined }). +%% enum{ 2^9(1), 2^10(2), 2^11(3), 2^12(4), (255) } MaxFragmentLength; +-define(MAX_FRAGMENT_LENGTH_EXT, 1). +-define(MAX_FRAGMENT_LENGTH_BYTES_1, 512). +-define(MAX_FRAGMENT_LENGTH_BYTES_2, 1024). +-define(MAX_FRAGMENT_LENGTH_BYTES_3, 2048). +-define(MAX_FRAGMENT_LENGTH_BYTES_4, 4096). + +-record(max_frag_enum, { + enum = undefined %% contains the enum value 1..4 + }). + %% Other possible values from RFC 6066, not supported --define(MAX_FRAGMENT_LENGTH, 1). -define(CLIENT_CERTIFICATE_URL, 2). -define(TRUSTED_CA_KEYS, 3). -define(TRUNCATED_HMAC, 4). diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index cb41742404..2da39b199f 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -25,7 +25,6 @@ -include_lib("public_key/include/public_key.hrl"). --define(VSN, "8.2.6"). -define(SECRET_PRINTOUT, "***"). -type reason() :: any(). @@ -155,6 +154,7 @@ log_level => {notice, [versions]}, max_handshake_size => {?DEFAULT_MAX_HANDSHAKE_SIZE, [versions]}, middlebox_comp_mode => {true, [versions]}, + max_fragment_length => {undefined, [versions]}, next_protocol_selector => {undefined, [versions]}, next_protocols_advertised => {undefined, [versions]}, padding_check => {true, [versions]}, @@ -177,12 +177,9 @@ supported_groups => {undefined, [versions]}, use_ticket => {undefined, [versions]}, user_lookup_fun => {undefined, [versions]}, - validate_extensions_fun => {undefined, [versions]}, verify => {verify_none, [versions, fail_if_no_peer_cert, - partial_chain, - verify_client_once]}, - verify_client_once => {false, [versions]}, + partial_chain]}, verify_fun => { {fun(_,{bad_cert, _}, UserState) -> diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index 867d2cfc5a..c19c6eeea9 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -40,6 +40,7 @@ set_renegotiation_flag/2, set_client_verify_data/3, set_server_verify_data/3, + set_max_fragment_length/2, empty_connection_state/2, initial_connection_state/2, record_protocol_role/1, step_encryption_state/1]). @@ -203,6 +204,33 @@ set_renegotiation_flag(Flag, #{current_read := CurrentRead0, pending_write => PendingWrite}. %%-------------------------------------------------------------------- +-spec set_max_fragment_length(term(), connection_states()) -> connection_states(). +%% +%% Description: Set maximum fragment length in all connection states +%%-------------------------------------------------------------------- +set_max_fragment_length(#max_frag_enum{enum = MaxFragEnum}, + #{current_read := CurrentRead0, + current_write := CurrentWrite0, + pending_read := PendingRead0, + pending_write := PendingWrite0} + = ConnectionStates) -> + MaxFragmentLength = if MaxFragEnum == 1 -> ?MAX_FRAGMENT_LENGTH_BYTES_1; + MaxFragEnum == 2 -> ?MAX_FRAGMENT_LENGTH_BYTES_2; + MaxFragEnum == 3 -> ?MAX_FRAGMENT_LENGTH_BYTES_3; + MaxFragEnum == 4 -> ?MAX_FRAGMENT_LENGTH_BYTES_4 + end, + CurrentRead = CurrentRead0#{max_fragment_length => MaxFragmentLength}, + CurrentWrite = CurrentWrite0#{max_fragment_length => MaxFragmentLength}, + PendingRead = PendingRead0#{max_fragment_length => MaxFragmentLength}, + PendingWrite = PendingWrite0#{max_fragment_length => MaxFragmentLength}, + ConnectionStates#{current_read => CurrentRead, + current_write => CurrentWrite, + pending_read => PendingRead, + pending_write => PendingWrite}; +set_max_fragment_length(_,ConnectionStates) -> + ConnectionStates. + +%%-------------------------------------------------------------------- -spec set_client_verify_data(current_read | current_write | current_both, binary(), connection_states())-> connection_states(). @@ -424,7 +452,8 @@ empty_connection_state(ConnectionEnd, BeastMitigation) -> mac_secret => undefined, secure_renegotiation => undefined, client_verify_data => undefined, - server_verify_data => undefined + server_verify_data => undefined, + max_fragment_length => undefined }. empty_security_params(ConnectionEnd = ?CLIENT) -> @@ -461,7 +490,8 @@ initial_connection_state(ConnectionEnd, BeastMitigation) -> mac_secret => undefined, secure_renegotiation => undefined, client_verify_data => undefined, - server_verify_data => undefined + server_verify_data => undefined, + max_fragment_length => undefined }. initial_security_params(ConnectionEnd) -> diff --git a/lib/ssl/src/ssl_record.hrl b/lib/ssl/src/ssl_record.hrl index c0555046c3..9aa598daed 100644 --- a/lib/ssl/src/ssl_record.hrl +++ b/lib/ssl/src/ssl_record.hrl @@ -154,6 +154,8 @@ -define(MAX_COMPRESSED_LENGTH, (?MAX_PLAIN_TEXT_LENGTH+1024)). -define(MAX_CIPHER_TEXT_LENGTH, (?MAX_PLAIN_TEXT_LENGTH+2048)). -define(TLS13_MAX_CIPHER_TEXT_LENGTH, (?MAX_PLAIN_TEXT_LENGTH+256)). +-define(MAX_PADDING_LENGTH,256). +-define(MAX_MAC_LENGTH,32). %% -record(protocol_version, { %% major, % unit 8 diff --git a/lib/ssl/src/tls_connection.erl b/lib/ssl/src/tls_connection.erl index 4a02d34a6b..5a41b69e54 100644 --- a/lib/ssl/src/tls_connection.erl +++ b/lib/ssl/src/tls_connection.erl @@ -604,8 +604,11 @@ init({call, From}, {start, Timeout}, %% Update pre_shared_key extension with binders (TLS 1.3) Hello1 = tls_handshake_1_3:maybe_add_binders(Hello, TicketData, HelloVersion), + MaxFragEnum = maps:get(max_frag_enum, Hello1#client_hello.extensions, undefined), + ConnectionStates1 = ssl_record:set_max_fragment_length(MaxFragEnum, ConnectionStates0), + {BinMsg, ConnectionStates, Handshake} = - encode_handshake(Hello1, HelloVersion, ConnectionStates0, Handshake0), + encode_handshake(Hello1, HelloVersion, ConnectionStates1, Handshake0), tls_socket:send(Transport, Socket, BinMsg), ssl_logger:debug(LogLevel, outbound, 'handshake', Hello1), @@ -718,8 +721,9 @@ hello(internal, #server_hello{} = Hello, connection_env = #connection_env{negotiated_version = ReqVersion} = CEnv, static_env = #static_env{role = client}, handshake_env = #handshake_env{renegotiation = {Renegotiation, _}}, + session = #session{session_id = OldId}, ssl_options = SslOptions} = State) -> - case tls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of + case tls_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation, OldId) of #alert{} = Alert -> %%TODO ssl_connection:handle_own_alert(Alert, ReqVersion, hello, State#state{connection_env = @@ -1164,7 +1168,8 @@ next_tls_record(Data, StateName, _ -> State0#state.connection_env#connection_env.negotiated_version end, - case tls_record:get_tls_records(Data, Versions, Buf0, SslOpts) of + #{current_write := #{max_fragment_length := MaxFragLen}} = State0#state.connection_states, + case tls_record:get_tls_records(Data, Versions, Buf0, MaxFragLen, SslOpts) of {Records, Buf1} -> CT1 = CT0 ++ Records, next_record(StateName, State0#state{protocol_buffers = @@ -1196,10 +1201,18 @@ handle_info({Protocol, _, Data}, StateName, handle_info({PassiveTag, Socket}, StateName, #state{static_env = #static_env{socket = Socket, passive_tag = PassiveTag}, + start_or_recv_from = From, + protocol_buffers = #protocol_buffers{tls_cipher_texts = CTs}, protocol_specific = PS - } = State) -> - next_event(StateName, no_record, - State#state{protocol_specific = PS#{active_n_toggle => true}}); + } = State0) -> + case (From =/= undefined) andalso (CTs == []) of + true -> + {Record, State} = activate_socket(State0#state{protocol_specific = PS#{active_n_toggle => true}}), + next_event(StateName, Record, State); + false -> + next_event(StateName, no_record, + State0#state{protocol_specific = PS#{active_n_toggle => true}}) + end; handle_info({CloseTag, Socket}, StateName, #state{static_env = #static_env{ role = Role, diff --git a/lib/ssl/src/tls_connection_1_3.erl b/lib/ssl/src/tls_connection_1_3.erl index b440d65706..0b5ff98474 100644 --- a/lib/ssl/src/tls_connection_1_3.erl +++ b/lib/ssl/src/tls_connection_1_3.erl @@ -218,6 +218,8 @@ wait_ee(internal, #change_cipher_spec{}, State, _Module) -> tls_connection:next_event(?FUNCTION_NAME, no_record, State); wait_ee(internal, #encrypted_extensions{} = EE, State0, _Module) -> case tls_handshake_1_3:do_wait_ee(EE, State0) of + #alert{} = Alert -> + ssl_connection:handle_own_alert(Alert, {3,4}, wait_ee, State0); {State1, NextState} -> tls_connection:next_event(NextState, no_record, State1) end; diff --git a/lib/ssl/src/tls_handshake.erl b/lib/ssl/src/tls_handshake.erl index f279e041be..897133846f 100644 --- a/lib/ssl/src/tls_handshake.erl +++ b/lib/ssl/src/tls_handshake.erl @@ -36,7 +36,7 @@ -include_lib("kernel/include/logger.hrl"). %% Handshake handling --export([client_hello/9, hello/4]). +-export([client_hello/9, hello/5, hello/4]). %% Handshake encoding -export([encode_handshake/2]). @@ -95,11 +95,11 @@ client_hello(_Host, _Port, ConnectionStates, }. %%-------------------------------------------------------------------- --spec hello(#server_hello{} | #client_hello{}, ssl_options(), +-spec hello(#server_hello{}, ssl_options(), ssl_record:connection_states() | {inet:port_number(), #session{}, db_handle(), atom(), ssl_record:connection_states(), binary() | undefined, ssl:kex_algo()}, - boolean()) -> + boolean(), #session{}) -> {tls_record:tls_version(), ssl:session_id(), ssl_record:connection_states(), alpn | npn, binary() | undefined}| {tls_record:tls_version(), {resumed | new, #session{}}, @@ -117,7 +117,7 @@ client_hello(_Host, _Port, ConnectionStates, %% values. hello(#server_hello{server_version = {Major, Minor}, random = <<_:24/binary,Down:8/binary>>}, - #{versions := [{M,N}|_]}, _, _) + #{versions := [{M,N}|_]}, _, _, _) when (M > 3 orelse M =:= 3 andalso N >= 4) andalso %% TLS 1.3 client (Major =:= 3 andalso Minor =:= 3 andalso %% Negotiating TLS 1.2 Down =:= ?RANDOM_OVERRIDE_TLS12) orelse @@ -131,7 +131,7 @@ hello(#server_hello{server_version = {Major, Minor}, %% equal to the second value if the ServerHello indicates TLS 1.1 or below. hello(#server_hello{server_version = {Major, Minor}, random = <<_:24/binary,Down:8/binary>>}, - #{versions := [{M,N}|_]}, _, _) + #{versions := [{M,N}|_]}, _, _, _) when (M =:= 3 andalso N =:= 3) andalso %% TLS 1.2 client (Major =:= 3 andalso Minor < 3 andalso %% Negotiating TLS 1.1 or prior Down =:= ?RANDOM_OVERRIDE_TLS11) -> @@ -157,7 +157,7 @@ hello(#server_hello{server_version = LegacyVersion, #server_hello_selected_version{selected_version = Version} = HelloExt} }, #{versions := SupportedVersions} = SslOpt, - ConnectionStates0, Renegotiation) -> + ConnectionStates0, Renegotiation, OldId) -> %% In TLS 1.3, the TLS server indicates its version using the "supported_versions" extension %% (Section 4.2.1), and the legacy_version field MUST be set to 0x0303, which is the version %% number for TLS 1.2. @@ -171,10 +171,11 @@ hello(#server_hello{server_version = LegacyVersion, true -> case Version of {3,3} -> + IsNew = ssl_session:is_new(OldId, SessionId), %% TLS 1.2 ServerHello with "supported_versions" (special case) handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, Compression, HelloExt, SslOpt, - ConnectionStates0, Renegotiation); + ConnectionStates0, Renegotiation, IsNew); SelectedVersion -> %% TLS 1.3 {next_state, wait_sh, SelectedVersion} @@ -191,17 +192,30 @@ hello(#server_hello{server_version = Version, session_id = SessionId, extensions = HelloExt}, #{versions := SupportedVersions} = SslOpt, - ConnectionStates0, Renegotiation) -> + ConnectionStates0, Renegotiation, OldId) -> + IsNew = ssl_session:is_new(OldId, SessionId), case tls_record:is_acceptable_version(Version, SupportedVersions) of true -> handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, Compression, HelloExt, SslOpt, - ConnectionStates0, Renegotiation); + ConnectionStates0, Renegotiation, IsNew); false -> ?ALERT_REC(?FATAL, ?PROTOCOL_VERSION) - end; + end. +%%-------------------------------------------------------------------- +-spec hello(#client_hello{}, ssl_options(), + ssl_record:connection_states() | {inet:port_number(), #session{}, db_handle(), + atom(), ssl_record:connection_states(), + binary() | undefined, ssl:kex_algo()}, + boolean()) -> + {tls_record:tls_version(), ssl:session_id(), + ssl_record:connection_states(), alpn | npn, binary() | undefined}| + {tls_record:tls_version(), {resumed | new, #session{}}, + ssl_record:connection_states(), binary() | undefined, + HelloExt::map(), {ssl:hash(), ssl:sign_algo()} | + undefined} | {atom(), atom()} | {atom(), atom(), tuple()} | #alert{}. %% TLS 1.2 Server %% - If "supported_versions" is present (ClientHello): %% - Select version from "supported_versions" (ignore ClientHello.legacy_version) @@ -338,7 +352,8 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, try ssl_handshake:handle_client_hello_extensions(tls_record, Random, CipherSuites, HelloExt, Version, SslOpts, Session0, ConnectionStates0, - Renegotiation) of + Renegotiation, + Session0#session.is_resumable) of {Session, ConnectionStates, Protocol, ServerHelloExt} -> {Version, {Type, Session}, ConnectionStates, Protocol, ServerHelloExt, HashSign} @@ -348,11 +363,11 @@ handle_client_hello_extensions(Version, Type, Random, CipherSuites, handle_server_hello_extensions(Version, SessionId, Random, CipherSuite, - Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation) -> + Compression, HelloExt, SslOpt, ConnectionStates0, Renegotiation, IsNew) -> try ssl_handshake:handle_server_hello_extensions(tls_record, Random, CipherSuite, Compression, HelloExt, Version, SslOpt, ConnectionStates0, - Renegotiation) of + Renegotiation, IsNew) of {ConnectionStates, ProtoExt, Protocol} -> {Version, SessionId, ConnectionStates, ProtoExt, Protocol} catch throw:Alert -> diff --git a/lib/ssl/src/tls_handshake_1_3.erl b/lib/ssl/src/tls_handshake_1_3.erl index 982d0fa787..3dc01fbcb0 100644 --- a/lib/ssl/src/tls_handshake_1_3.erl +++ b/lib/ssl/src/tls_handshake_1_3.erl @@ -117,14 +117,22 @@ server_hello_random(hello_retry_request, _) -> ?HELLO_RETRY_REQUEST_RANDOM. -encrypted_extensions(#state{handshake_env = #handshake_env{alpn = undefined}}) -> - #encrypted_extensions{ - extensions = #{} - }; -encrypted_extensions(#state{handshake_env = #handshake_env{alpn = ALPNProtocol}}) -> - Extensions = ssl_handshake:add_alpn(#{}, ALPNProtocol), +encrypted_extensions(#state{handshake_env = HandshakeEnv}) -> + E0 = #{}, + E1 = case HandshakeEnv#handshake_env.alpn of + undefined -> + E0; + ALPNProtocol -> + ssl_handshake:add_alpn(#{}, ALPNProtocol) + end, + E2 = case HandshakeEnv#handshake_env.max_frag_enum of + undefined -> + E1; + MaxFragEnum -> + E1#{max_frag_enum => MaxFragEnum} + end, #encrypted_extensions{ - extensions = Extensions + extensions = E2 }. @@ -214,7 +222,7 @@ certificate_verify(PrivateKey, SignatureScheme, ssl_record:pending_connection_state(ConnectionStates, write), #security_parameters{prf_algorithm = HKDFAlgo} = SecParamsR, - {HashAlgo, _, _} = + {HashAlgo, SignAlgo, _} = ssl_cipher:scheme_to_components(SignatureScheme), Context = lists:reverse(Messages), @@ -225,7 +233,7 @@ certificate_verify(PrivateKey, SignatureScheme, %% Digital signatures use the hash function defined by the selected signature %% scheme. - case sign(THash, ContextString, HashAlgo, PrivateKey) of + case sign(THash, ContextString, HashAlgo, PrivateKey, SignAlgo) of {ok, Signature} -> {ok, #certificate_verify_1_3{ algorithm = SignatureScheme, @@ -487,24 +495,9 @@ certificate_entry(DER) -> %% 79 %% 00 %% 0101010101010101010101010101010101010101010101010101010101010101 -sign(THash, Context, HashAlgo, #'ECPrivateKey'{} = PrivateKey) -> +sign(THash, Context, HashAlgo, PrivateKey, SignAlgo) -> Content = build_content(Context, THash), - try digitally_signed(Content, HashAlgo, PrivateKey) of - Signature -> - {ok, Signature} - catch - error:badarg -> - {error, ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, badarg)} - end; -sign(THash, Context, HashAlgo, PrivateKey) -> - Content = build_content(Context, THash), - - %% The length of the Salt MUST be equal to the length of the output - %% of the digest algorithm: rsa_pss_saltlen = -1 - try digitally_signed(Content, HashAlgo, PrivateKey, - [{rsa_padding, rsa_pkcs1_pss_padding}, - {rsa_pss_saltlen, -1}, - {rsa_mgf1_md, HashAlgo}]) of + try ssl_handshake:digitally_signed({3,4}, Content, HashAlgo, PrivateKey, SignAlgo) of Signature -> {ok, Signature} catch @@ -512,25 +505,9 @@ sign(THash, Context, HashAlgo, PrivateKey) -> {error, ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, badarg)} end. - -verify(THash, Context, HashAlgo, Signature, {?'id-ecPublicKey', PublicKey, PublicKeyParams}) -> +verify(THash, Context, HashAlgo, SignAlgo, Signature, PublicKeyInfo) -> Content = build_content(Context, THash), - try public_key:verify(Content, HashAlgo, Signature, {PublicKey, PublicKeyParams}) of - Result -> - {ok, Result} - catch - error:badarg -> - {error, ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, badarg)} - end; -verify(THash, Context, HashAlgo, Signature, {?rsaEncryption, PublicKey, _PubKeyParams}) -> - Content = build_content(Context, THash), - - %% The length of the Salt MUST be equal to the length of the output - %% of the digest algorithm: rsa_pss_saltlen = -1 - try public_key:verify(Content, HashAlgo, Signature, PublicKey, - [{rsa_padding, rsa_pkcs1_pss_padding}, - {rsa_pss_saltlen, -1}, - {rsa_mgf1_md, HashAlgo}]) of + try ssl_handshake:verify_signature({3, 4}, Content, {HashAlgo, SignAlgo}, Signature, PublicKeyInfo) of Result -> {ok, Result} catch @@ -538,7 +515,6 @@ verify(THash, Context, HashAlgo, Signature, {?rsaEncryption, PublicKey, _PubKeyP {error, ?ALERT_REC(?FATAL, ?INTERNAL_ERROR, badarg)} end. - build_content(Context, THash) -> Prefix = binary:copy(<<32>>, 64), <<Prefix/binary,Context/binary,?BYTE(0),THash/binary>>. @@ -553,13 +529,14 @@ build_content(Context, THash) -> do_start(#client_hello{cipher_suites = ClientCiphers, session_id = SessionId, extensions = Extensions} = _Hello, - #state{connection_states = _ConnectionStates0, + #state{connection_states = ConnectionStates0, ssl_options = #{ciphers := ServerCiphers, signature_algs := ServerSignAlgs, supported_groups := ServerGroups0, alpn_preferred_protocols := ALPNPreferredProtocols, honor_cipher_order := HonorCipherOrder}, session = #session{own_certificate = Cert}} = State0) -> + ClientGroups0 = maps:get(elliptic_curves, Extensions, undefined), ClientGroups = get_supported_groups(ClientGroups0), ServerGroups = get_supported_groups(ServerGroups0), @@ -580,6 +557,7 @@ do_start(#client_hello{cipher_suites = ClientCiphers, {Ref,Maybe} = maybe(), try + %% Handle ALPN extension if ALPN is configured ALPNProtocol = Maybe(handle_alpn(ALPNPreferredProtocols, ClientALPN)), @@ -588,17 +566,15 @@ do_start(#client_hello{cipher_suites = ClientCiphers, %% and a signature algorithm/certificate pair to authenticate itself to %% the client. Cipher = Maybe(select_cipher_suite(HonorCipherOrder, ClientCiphers, ServerCiphers)), - Groups = Maybe(select_common_groups(ServerGroups, ClientGroups)), Maybe(validate_client_key_share(ClientGroups, ClientShares)), - - {PublicKeyAlgo, SignAlgo, SignHash} = get_certificate_params(Cert), + {PublicKeyAlgo, SignAlgo, SignHash, RSAKeySize} = get_certificate_params(Cert), %% Check if client supports signature algorithm of server certificate Maybe(check_cert_sign_algo(SignAlgo, SignHash, ClientSignAlgs, ClientSignAlgsCert)), %% Select signature algorithm (used in CertificateVerify message). - SelectedSignAlg = Maybe(select_sign_algo(PublicKeyAlgo, ClientSignAlgs, ServerSignAlgs)), + SelectedSignAlg = Maybe(select_sign_algo(PublicKeyAlgo, RSAKeySize, ClientSignAlgs, ServerSignAlgs)), %% Select client public key. If no public key found in ClientShares or %% ClientShares is empty, trigger HelloRetryRequest as we were able @@ -609,7 +585,17 @@ do_start(#client_hello{cipher_suites = ClientCiphers, %% Generate server_share KeyShare = ssl_cipher:generate_server_share(Group), - State1 = update_start_state(State0, + State1 = case maps:get(max_frag_enum, Extensions, undefined) of + MaxFragEnum when is_record(MaxFragEnum, max_frag_enum) -> + ConnectionStates1 = ssl_record:set_max_fragment_length(MaxFragEnum, ConnectionStates0), + HsEnv1 = (State0#state.handshake_env)#handshake_env{max_frag_enum = MaxFragEnum}, + State0#state{handshake_env = HsEnv1, + connection_states = ConnectionStates1}; + _ -> + State0 + end, + + State2 = update_start_state(State1, #{cipher => Cipher, key_share => KeyShare, session_id => SessionId, @@ -624,12 +610,12 @@ do_start(#client_hello{cipher_suites = ClientCiphers, %% message if it is able to find an acceptable set of parameters but the %% ClientHello does not contain sufficient information to proceed with %% the handshake. - case Maybe(send_hello_retry_request(State1, ClientPubKey, KeyShare, SessionId)) of + case Maybe(send_hello_retry_request(State2, ClientPubKey, KeyShare, SessionId)) of {_, start} = NextStateTuple -> NextStateTuple; {_, negotiated} = NextStateTuple -> %% Exclude any incompatible PSKs. - PSK = Maybe(handle_pre_shared_key(State1, OfferedPSKs, Cipher)), + PSK = Maybe(handle_pre_shared_key(State2, OfferedPSKs, Cipher)), Maybe(session_resumption(NextStateTuple, PSK)) end catch @@ -933,6 +919,9 @@ do_wait_ee(#encrypted_extensions{extensions = Extensions}, State0) -> {Ref, Maybe} = maybe(), try + %% RFC 6066: handle received/expected maximum fragment length + Maybe(maybe_max_fragment_length(Extensions, State0)), + %% Go to state 'wait_finished' if using PSK. Maybe(maybe_resumption(State0)), @@ -943,7 +932,9 @@ do_wait_ee(#encrypted_extensions{extensions = Extensions}, State0) -> {State1, wait_cert_cr} catch {Ref, {State, StateName}} -> - {State, StateName} + {State, StateName}; + {Ref, #alert{} = Alert} -> + Alert end. @@ -983,6 +974,16 @@ maybe_hello_retry_request(#server_hello{random = ?HELLO_RETRY_REQUEST_RANDOM} = maybe_hello_retry_request(_, _) -> ok. +maybe_max_fragment_length(Extensions, State) -> + ServerMaxFragEnum = maps:get(max_frag_enum, Extensions, undefined), + ClientMaxFragEnum = ssl_handshake:max_frag_enum( + maps:get(max_fragment_length, State#state.ssl_options, undefined)), + if ServerMaxFragEnum == ClientMaxFragEnum -> + ok; + true -> + {error, ?ALERT_REC(?FATAL, ?ILLEGAL_PARAMETER)} + end. + maybe_resumption(#state{handshake_env = #handshake_env{resumption = true}} = State) -> {error, {State, wait_finished}}; @@ -1250,7 +1251,7 @@ process_certificate_request(#certificate_request_1_3{ ServerSignAlgsCert = get_signature_scheme_list( maps:get(signature_algs_cert, Extensions, undefined)), - {_PublicKeyAlgo, SignAlgo, SignHash} = get_certificate_params(Cert), + {_PublicKeyAlgo, SignAlgo, SignHash, _} = get_certificate_params(Cert), %% Check if server supports signature algorithm of client certificate case check_cert_sign_algo(SignAlgo, SignHash, ServerSignAlgs, ServerSignAlgsCert) of @@ -1850,7 +1851,7 @@ verify_certificate_verify(#state{ ssl_record:pending_connection_state(ConnectionStates, write), #security_parameters{prf_algorithm = HKDFAlgo} = SecParamsR, - {HashAlgo, _, _} = + {HashAlgo, SignAlg, _} = ssl_cipher:scheme_to_components(SignatureScheme), Messages = get_handshake_context_cv(HHistory), @@ -1864,7 +1865,7 @@ verify_certificate_verify(#state{ %% Digital signatures use the hash function defined by the selected signature %% scheme. - case verify(THash, ContextString, HashAlgo, Signature, PublicKeyInfo) of + case verify(THash, ContextString, HashAlgo, SignAlg, Signature, PublicKeyInfo) of {ok, true} -> {ok, {State0, wait_finished}}; {ok, false} -> @@ -2026,7 +2027,7 @@ select_cipher_suite(_, [], _) -> select_cipher_suite(true, ClientCiphers, ServerCiphers) -> select_cipher_suite(false, ServerCiphers, ClientCiphers); select_cipher_suite(false, [Cipher|ClientCiphers], ServerCiphers) -> - case lists:member(Cipher, tls_v1:suites('TLS_v1.3')) andalso + case lists:member(Cipher, tls_v1:exclusive_suites(4)) andalso lists:member(Cipher, ServerCiphers) of true -> {ok, Cipher}; @@ -2068,11 +2069,11 @@ check_cert_sign_algo(SignAlgo, SignHash, _, ClientSignAlgsCert) -> %% DSA keys are not supported by TLS 1.3 -select_sign_algo(dsa, _ClientSignAlgs, _ServerSignAlgs) -> +select_sign_algo(dsa, _RSAKeySize, _ClientSignAlgs, _ServerSignAlgs) -> {error, ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_public_key)}; -select_sign_algo(_, [], _) -> +select_sign_algo(_, _RSAKeySize, [], _) -> {error, ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_signature_algorithm)}; -select_sign_algo(PublicKeyAlgo, [C|ClientSignAlgs], ServerSignAlgs) -> +select_sign_algo(PublicKeyAlgo, RSAKeySize, [C|ClientSignAlgs], ServerSignAlgs) -> {_, S, _} = ssl_cipher:scheme_to_components(C), %% RSASSA-PKCS1-v1_5 and Legacy algorithms are not defined for use in signed %% TLS handshake messages: filter sha-1 and rsa_pkcs1. @@ -2082,16 +2083,48 @@ select_sign_algo(PublicKeyAlgo, [C|ClientSignAlgs], ServerSignAlgs) -> %% RSASSA-PSS PSS algorithms: If the public key is carried in an X.509 certificate, %% it MUST use the RSASSA-PSS OID. case ((PublicKeyAlgo =:= rsa andalso S =:= rsa_pss_rsae) - orelse (PublicKeyAlgo =:= rsa_pss andalso S =:= rsa_pss_pss) + orelse (PublicKeyAlgo =:= rsa_pss_pss andalso S =:= rsa_pss_pss) orelse (PublicKeyAlgo =:= ecdsa andalso S =:= ecdsa)) andalso lists:member(C, ServerSignAlgs) of true -> - {ok, C}; + validate_key_compatibility(PublicKeyAlgo, RSAKeySize, + [C|ClientSignAlgs], ServerSignAlgs); false -> - select_sign_algo(PublicKeyAlgo, ClientSignAlgs, ServerSignAlgs) + select_sign_algo(PublicKeyAlgo, RSAKeySize, ClientSignAlgs, ServerSignAlgs) end. +validate_key_compatibility(PublicKeyAlgo, RSAKeySize, [C|ClientSignAlgs], ServerSignAlgs) + when PublicKeyAlgo =:= rsa orelse + PublicKeyAlgo =:= rsa_pss_pss -> + case is_rsa_key_compatible(RSAKeySize, C) of + true -> + {ok, C}; + false -> + select_sign_algo(PublicKeyAlgo, RSAKeySize, ClientSignAlgs, ServerSignAlgs) + end; +validate_key_compatibility(_, _, [C|_], _) -> + {ok, C}. + +is_rsa_key_compatible(KeySize, SigAlg) -> + {Hash, _, _} = ssl_cipher:scheme_to_components(SigAlg), + HashSize = ssl_cipher:hash_size(Hash), + + %% OpenSSL crypto lib defines a limit on the size of the random salt + %% in PSS signatures based on the size of signing RSA key. + %% If the limit is unchecked, it causes handshake failures when the + %% configured certificates contain short (e.g. 1024-bit) RSA keys. + %% For more information see the OpenSSL crypto library + %% (rsa_pss:c{77,86}). + %% TODO: Move this check into crypto. Investigate if this is a bug in + %% OpenSSL crypto lib. + if (KeySize < (HashSize + 2)) -> + false; + (HashSize > (KeySize - HashSize - 2)) -> + false; + true -> + true + end. do_check_cert_sign_algo(_, _, []) -> {error, ?ALERT_REC(?FATAL, ?INSUFFICIENT_SECURITY, no_suitable_signature_algorithm)}; @@ -2106,10 +2139,8 @@ do_check_cert_sign_algo(SignAlgo, SignHash, [Scheme|T]) -> %% id-RSASSA-PSS (rsa_pss) indicates that the key may only be used for PSS signatures. -%% TODO: Uncomment when rsa_pss signatures are supported in certificates -%% compare_sign_algos(rsa_pss, Hash, Algo, Hash) -%% when Algo =:= rsa_pss_pss -> -%% true; +compare_sign_algos(rsa_pss_pss, Hash, rsa_pss_pss, Hash) -> + true; %% rsaEncryption (rsa) allows the key to be used for any of the standard encryption or %% signature schemes. compare_sign_algos(rsa, Hash, Algo, Hash) @@ -2121,23 +2152,28 @@ compare_sign_algos(Algo, Hash, Algo, Hash) -> compare_sign_algos(_, _, _, _) -> false. - get_certificate_params(Cert) -> - {SignAlgo0, _Param, PublicKeyAlgo0} = ssl_handshake:get_cert_params(Cert), - {SignHash0, SignAlgo} = public_key:pkix_sign_types(SignAlgo0), - %% Convert hash to new format - SignHash = case SignHash0 of - sha -> - sha1; - H -> H - end, - PublicKeyAlgo = public_key_algo(PublicKeyAlgo0), - {PublicKeyAlgo, SignAlgo, SignHash}. - - + {SignAlgo0, Param, SubjectPublicKeyAlgo0, RSAKeySize} = + ssl_handshake:get_cert_params(Cert), + {SignHash, SignAlgo} = oids_to_atoms(SignAlgo0, Param), + SubjectPublicKeyAlgo = public_key_algo(SubjectPublicKeyAlgo0), + {SubjectPublicKeyAlgo, SignAlgo, SignHash, RSAKeySize}. + +oids_to_atoms(?'id-RSASSA-PSS', #'RSASSA-PSS-params'{maskGenAlgorithm = + #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = HashOid}}}) -> + Hash = public_key:pkix_hash_type(HashOid), + {Hash, rsa_pss_pss}; +oids_to_atoms(SignAlgo, _) -> + case public_key:pkix_sign_types(SignAlgo) of + {sha, Sign} -> + {sha1, Sign}; + {_,_} = Algs -> + Algs + end. %% Note: copied from ssl_handshake public_key_algo(?'id-RSASSA-PSS') -> - rsa_pss; + rsa_pss_pss; public_key_algo(?rsaEncryption) -> rsa; public_key_algo(?'id-ecPublicKey') -> @@ -2411,30 +2447,3 @@ process_user_tickets([H|T], Acc, N) -> %% (see Section 4.6.1), modulo 2^32. obfuscate_ticket_age(TicketAge, AgeAdd) -> (TicketAge + AgeAdd) rem round(math:pow(2,32)). - - -digitally_signed(Msg, HashAlgo, PrivateKey) -> - digitally_signed(Msg, HashAlgo, PrivateKey, []). - -digitally_signed(Msg, HashAlgo, PrivateKey, Options) -> - try do_digitally_signed(Msg, HashAlgo, PrivateKey, Options) of - Signature -> - Signature - catch - error:_ -> - {error, ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE, bad_key(PrivateKey))} - end. - -do_digitally_signed(Msg, HashAlgo, #{algorithm := Alg} = Engine, Options) -> - crypto:sign(Alg, HashAlgo, Msg, maps:remove(algorithm, Engine), Options); -do_digitally_signed(Msg, HashAlgo, Key, Options) -> - public_key:sign(Msg, HashAlgo, Key, Options). - -bad_key(#'RSAPrivateKey'{}) -> - unacceptable_rsa_key; -bad_key(#'ECPrivateKey'{}) -> - unacceptable_ecdsa_key; -bad_key(#{algorithm := rsa}) -> - unacceptable_rsa_key; -bad_key(#{algorithm := ecdsa}) -> - unacceptable_ecdsa_key. diff --git a/lib/ssl/src/tls_record.erl b/lib/ssl/src/tls_record.erl index dfdc0bd50b..8d67be687a 100644 --- a/lib/ssl/src/tls_record.erl +++ b/lib/ssl/src/tls_record.erl @@ -33,12 +33,12 @@ -include_lib("kernel/include/logger.hrl"). %% Handling of incoming data --export([get_tls_records/4, init_connection_states/2]). +-export([get_tls_records/5, init_connection_states/2]). %% Encoding TLS records -export([encode_handshake/3, encode_alert_record/3, encode_change_cipher_spec/2, encode_data/3]). --export([encode_plain_text/4, split_iovec/1]). +-export([encode_plain_text/4, split_iovec/2]). %% Decoding -export([decode_cipher_text/4]). @@ -55,7 +55,8 @@ -export_type([tls_version/0, tls_atom_version/0]). -type tls_version() :: ssl_record:ssl_version(). --type tls_atom_version() :: sslv3 | tlsv1 | 'tlsv1.1' | 'tlsv1.2'. +-type tls_atom_version() :: sslv3 | tlsv1 | 'tlsv1.1' | 'tlsv1.2' | 'tlsv1.3'. +-type tls_max_frag_len() :: undefined | 512 | 1024 | 2048 | 4096. -compile(inline). @@ -83,6 +84,7 @@ init_connection_states(Role, BeastMitigation) -> binary(), [tls_version()] | tls_version(), Buffer0 :: binary() | {'undefined' | #ssl_tls{}, {[binary()],non_neg_integer(),[binary()]}}, + tls_max_frag_len(), ssl_options()) -> {Records :: [#ssl_tls{}], Buffer :: {'undefined' | #ssl_tls{}, {[binary()],non_neg_integer(),[binary()]}}} | @@ -92,10 +94,10 @@ init_connection_states(Role, BeastMitigation) -> %% Description: Given old buffer and new data from TCP, packs up a records %% data %%-------------------------------------------------------------------- -get_tls_records(Data, Versions, Buffer, SslOpts) when is_binary(Buffer) -> - parse_tls_records(Versions, {[Data],byte_size(Data),[]}, SslOpts, undefined); -get_tls_records(Data, Versions, {Hdr, {Front,Size,Rear}}, SslOpts) -> - parse_tls_records(Versions, {Front,Size + byte_size(Data),[Data|Rear]}, SslOpts, Hdr). +get_tls_records(Data, Versions, Buffer, MaxFragLen, SslOpts) when is_binary(Buffer) -> + parse_tls_records(Versions, {[Data],byte_size(Data),[]}, MaxFragLen, SslOpts, undefined); +get_tls_records(Data, Versions, {Hdr, {Front,Size,Rear}}, MaxFragLen, SslOpts) -> + parse_tls_records(Versions, {Front,Size + byte_size(Data),[Data|Rear]}, MaxFragLen, SslOpts, Hdr). %%==================================================================== %% Encoding @@ -112,12 +114,18 @@ encode_handshake(Frag, {3, 4}, ConnectionStates) -> encode_handshake(Frag, Version, #{current_write := #{beast_mitigation := BeastMitigation, + max_fragment_length := MaxFragmentLength, security_parameters := #security_parameters{bulk_cipher_algorithm = BCA}}} = ConnectionStates) -> + MaxLength = if is_integer(MaxFragmentLength) -> + MaxFragmentLength; + true -> + ?MAX_PLAIN_TEXT_LENGTH + end, case iolist_size(Frag) of - N when N > ?MAX_PLAIN_TEXT_LENGTH -> - Data = split_iovec(erlang:iolist_to_iovec(Frag), Version, BCA, BeastMitigation), + N when N > MaxLength -> + Data = split_iovec(erlang:iolist_to_iovec(Frag), Version, BCA, BeastMitigation, MaxLength), encode_fragments(?HANDSHAKE, Version, Data, ConnectionStates); _ -> encode_plain_text(?HANDSHAKE, Version, Frag, ConnectionStates) @@ -155,10 +163,16 @@ encode_data(Data, {3, 4}, ConnectionStates) -> tls_record_1_3:encode_data(Data, ConnectionStates); encode_data(Data, Version, #{current_write := #{beast_mitigation := BeastMitigation, + max_fragment_length := MaxFragmentLength, security_parameters := #security_parameters{bulk_cipher_algorithm = BCA}}} = ConnectionStates) -> - Fragments = split_iovec(Data, Version, BCA, BeastMitigation), + MaxLength = if is_integer(MaxFragmentLength) -> + MaxFragmentLength; + true -> + ?MAX_PLAIN_TEXT_LENGTH + end, + Fragments = split_iovec(Data, Version, BCA, BeastMitigation, MaxLength), encode_fragments(?APPLICATION_DATA, Version, Fragments, ConnectionStates). %%==================================================================== @@ -442,11 +456,11 @@ hello_version([Highest|_]) when Highest >= {3,3} -> hello_version(Versions) -> lowest_protocol_version(Versions). -split_iovec([]) -> +split_iovec([], _) -> []; -split_iovec(Data) -> - {Part,Rest} = split_iovec(Data, ?MAX_PLAIN_TEXT_LENGTH, []), - [Part|split_iovec(Rest)]. +split_iovec(Data, MaximumFragmentLength) -> + {Part,Rest} = split_iovec(Data, MaximumFragmentLength, []), + [Part|split_iovec(Rest, MaximumFragmentLength)]. %%-------------------------------------------------------------------- %%% Internal functions @@ -461,7 +475,8 @@ initial_connection_state(ConnectionEnd, BeastMitigation) -> mac_secret => undefined, secure_renegotiation => undefined, client_verify_data => undefined, - server_verify_data => undefined + server_verify_data => undefined, + max_fragment_length => undefined }. %% Used by logging to recreate the received bytes @@ -470,88 +485,92 @@ build_tls_record(#ssl_tls{type = Type, version = {MajVer, MinVer}, fragment = Fr <<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer),?UINT16(Length), Fragment/binary>>. -parse_tls_records(Versions, Q, SslOpts, undefined) -> - decode_tls_records(Versions, Q, SslOpts, [], undefined, undefined, undefined); -parse_tls_records(Versions, Q, SslOpts, #ssl_tls{type = Type, version = Version, fragment = Length}) -> - decode_tls_records(Versions, Q, SslOpts, [], Type, Version, Length). +parse_tls_records(Versions, Q, MaxFragLen, SslOpts, undefined) -> + decode_tls_records(Versions, Q, MaxFragLen, SslOpts, [], undefined, undefined, undefined); +parse_tls_records(Versions, Q, MaxFragLen, SslOpts, #ssl_tls{type = Type, version = Version, fragment = Length}) -> + decode_tls_records(Versions, Q, MaxFragLen, SslOpts, [], Type, Version, Length). %% Generic code path -decode_tls_records(Versions, {_,Size,_} = Q0, SslOpts, Acc, undefined, _Version, _Length) -> +decode_tls_records(Versions, {_,Size,_} = Q0, MaxFragLen, SslOpts, Acc, undefined, _Version, _Length) -> if 5 =< Size -> {<<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length)>>, Q} = binary_from_front(5, Q0), - validate_tls_records_type(Versions, Q, SslOpts, Acc, Type, {MajVer,MinVer}, Length); + validate_tls_records_type(Versions, Q, MaxFragLen, SslOpts, Acc, Type, {MajVer,MinVer}, Length); 3 =< Size -> {<<?BYTE(Type),?BYTE(MajVer),?BYTE(MinVer)>>, Q} = binary_from_front(3, Q0), - validate_tls_records_type(Versions, Q, SslOpts, Acc, Type, {MajVer,MinVer}, undefined); + validate_tls_records_type(Versions, Q, MaxFragLen, SslOpts, Acc, Type, {MajVer,MinVer}, undefined); 1 =< Size -> {<<?BYTE(Type)>>, Q} = binary_from_front(1, Q0), - validate_tls_records_type(Versions, Q, SslOpts, Acc, Type, undefined, undefined); + validate_tls_records_type(Versions, Q, MaxFragLen, SslOpts, Acc, Type, undefined, undefined); true -> - validate_tls_records_type(Versions, Q0, SslOpts, Acc, undefined, undefined, undefined) + validate_tls_records_type(Versions, Q0, MaxFragLen, SslOpts, Acc, undefined, undefined, undefined) end; -decode_tls_records(Versions, {_,Size,_} = Q0, SslOpts, Acc, Type, undefined, _Length) -> +decode_tls_records(Versions, {_,Size,_} = Q0, MaxFragLen, SslOpts, Acc, Type, undefined, _Length) -> if 4 =< Size -> {<<?BYTE(MajVer),?BYTE(MinVer), ?UINT16(Length)>>, Q} = binary_from_front(4, Q0), - validate_tls_record_version(Versions, Q, SslOpts, Acc, Type, {MajVer,MinVer}, Length); + validate_tls_record_version(Versions, Q, MaxFragLen, SslOpts, Acc, Type, {MajVer,MinVer}, Length); 2 =< Size -> {<<?BYTE(MajVer),?BYTE(MinVer)>>, Q} = binary_from_front(2, Q0), - validate_tls_record_version(Versions, Q, SslOpts, Acc, Type, {MajVer,MinVer}, undefined); + validate_tls_record_version(Versions, Q, MaxFragLen, SslOpts, Acc, Type, {MajVer,MinVer}, undefined); true -> - validate_tls_record_version(Versions, Q0, SslOpts, Acc, Type, undefined, undefined) + validate_tls_record_version(Versions, Q0, MaxFragLen, SslOpts, Acc, Type, undefined, undefined) end; -decode_tls_records(Versions, {_,Size,_} = Q0, SslOpts, Acc, Type, Version, undefined) -> +decode_tls_records(Versions, {_,Size,_} = Q0, MaxFragLen, SslOpts, Acc, Type, Version, undefined) -> if 2 =< Size -> {<<?UINT16(Length)>>, Q} = binary_from_front(2, Q0), - validate_tls_record_length(Versions, Q, SslOpts, Acc, Type, Version, Length); + validate_tls_record_length(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length); true -> - validate_tls_record_length(Versions, Q0, SslOpts, Acc, Type, Version, undefined) + validate_tls_record_length(Versions, Q0, MaxFragLen, SslOpts, Acc, Type, Version, undefined) end; -decode_tls_records(Versions, Q, SslOpts, Acc, Type, Version, Length) -> - validate_tls_record_length(Versions, Q, SslOpts, Acc, Type, Version, Length). +decode_tls_records(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length) -> + validate_tls_record_length(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length). -validate_tls_records_type(_Versions, Q, _SslOpts, Acc, undefined, _Version, _Length) -> +validate_tls_records_type(_Versions, Q, _MaxFragLen, _SslOpts, Acc, undefined, _Version, _Length) -> {lists:reverse(Acc), {undefined, Q}}; -validate_tls_records_type(Versions, Q, SslOpts, Acc, Type, Version, Length) -> +validate_tls_records_type(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length) -> if ?KNOWN_RECORD_TYPE(Type) -> - validate_tls_record_version(Versions, Q, SslOpts, Acc, Type, Version, Length); + validate_tls_record_version(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length); true -> %% Not ?KNOWN_RECORD_TYPE(Type) ?ALERT_REC(?FATAL, ?UNEXPECTED_MESSAGE, {unsupported_record_type, Type}) end. -validate_tls_record_version(_Versions, Q, _SslOpts, Acc, Type, undefined, _Length) -> +validate_tls_record_version(_Versions, Q, _MaxFragLen, _SslOpts, Acc, Type, undefined, _Length) -> {lists:reverse(Acc), {#ssl_tls{type = Type, version = undefined, fragment = undefined}, Q}}; -validate_tls_record_version(Versions, Q, SslOpts, Acc, Type, Version, Length) -> +validate_tls_record_version(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length) -> case Versions of _ when is_list(Versions) -> case is_acceptable_version(Version, Versions) of true -> - validate_tls_record_length(Versions, Q, SslOpts, Acc, Type, Version, Length); + validate_tls_record_length(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length); false -> ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC, {unsupported_version, Version}) end; {3, 4} when Version =:= {3, 3} -> - validate_tls_record_length(Versions, Q, SslOpts, Acc, Type, Version, Length); + validate_tls_record_length(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length); Version -> %% Exact version match - validate_tls_record_length(Versions, Q, SslOpts, Acc, Type, Version, Length); + validate_tls_record_length(Versions, Q, MaxFragLen, SslOpts, Acc, Type, Version, Length); _ -> ?ALERT_REC(?FATAL, ?BAD_RECORD_MAC, {unsupported_version, Version}) end. -validate_tls_record_length(_Versions, Q, _SslOpts, Acc, Type, Version, undefined) -> +validate_tls_record_length(_Versions, Q, _MaxFragLen, _SslOpts, Acc, Type, Version, undefined) -> {lists:reverse(Acc), {#ssl_tls{type = Type, version = Version, fragment = undefined}, Q}}; -validate_tls_record_length(Versions, {_,Size0,_} = Q0, +validate_tls_record_length(Versions, {_,Size0,_} = Q0, MaxFragLen, #{log_level := LogLevel} = SslOpts, Acc, Type, Version, Length) -> - Max = max_len(Versions), + Max = if is_integer(MaxFragLen) -> + MaxFragLen + ?MAX_PADDING_LENGTH + ?MAX_MAC_LENGTH; + true -> + max_len(Versions) + end, if Length =< Max -> if @@ -560,7 +579,7 @@ validate_tls_record_length(Versions, {_,Size0,_} = Q0, {Fragment, Q} = binary_from_front(Length, Q0), Record = #ssl_tls{type = Type, version = Version, fragment = Fragment}, ssl_logger:debug(LogLevel, inbound, 'record', Record), - decode_tls_records(Versions, Q, SslOpts, [Record|Acc], undefined, undefined, undefined); + decode_tls_records(Versions, Q, MaxFragLen, SslOpts, [Record|Acc], undefined, undefined, undefined); true -> {lists:reverse(Acc), {#ssl_tls{type = Type, version = Version, fragment = Length}, Q0}} @@ -669,20 +688,20 @@ encode_fragments(_Type, _Version, _Data, CS, _CompS, _CipherS, _Seq, _CipherFrag %% 1/n-1 splitting countermeasure Rizzo/Duong-Beast, RC4 chiphers are %% not vulnerable to this attack. -split_iovec(Data, Version, BCA, one_n_minus_one) +split_iovec(Data, Version, BCA, one_n_minus_one, MaxLength) when (BCA =/= ?RC4) andalso ({3, 1} == Version orelse {3, 0} == Version) -> {Part, RestData} = split_iovec(Data, 1, []), - [Part|split_iovec(RestData)]; + [Part|split_iovec(RestData, MaxLength)]; %% 0/n splitting countermeasure for clients that are incompatible with 1/n-1 %% splitting. -split_iovec(Data, Version, BCA, zero_n) +split_iovec(Data, Version, BCA, zero_n, MaxLength) when (BCA =/= ?RC4) andalso ({3, 1} == Version orelse {3, 0} == Version) -> {Part, RestData} = split_iovec(Data, 0, []), - [Part|split_iovec(RestData)]; -split_iovec(Data, _Version, _BCA, _BeatMitigation) -> - split_iovec(Data). + [Part|split_iovec(RestData, MaxLength)]; +split_iovec(Data, _Version, _BCA, _BeatMitigation, MaxLength) -> + split_iovec(Data, MaxLength). split_iovec([Bin|Data] = Bin_Data, SplitSize, Acc) -> BinSize = byte_size(Bin), diff --git a/lib/ssl/src/tls_record_1_3.erl b/lib/ssl/src/tls_record_1_3.erl index 89f2a484ff..a9ba415099 100644 --- a/lib/ssl/src/tls_record_1_3.erl +++ b/lib/ssl/src/tls_record_1_3.erl @@ -43,11 +43,17 @@ % %% Description: Encodes a handshake message to send on the tls-1.3-socket. %%-------------------------------------------------------------------- -encode_handshake(Frag, ConnectionStates) -> +encode_handshake(Frag, #{current_write := #{max_fragment_length := MaxFragmentLength}} = + ConnectionStates) -> + MaxLength = if is_integer(MaxFragmentLength) -> + MaxFragmentLength; + true -> + %% TODO: Consider padding here + ?MAX_PLAIN_TEXT_LENGTH + end, case iolist_size(Frag) of - N when N > ?MAX_PLAIN_TEXT_LENGTH -> - %% TODO: Consider padding here - Data = tls_record:split_iovec(Frag), + N when N > MaxLength -> + Data = tls_record:split_iovec(erlang:iolist_to_iovec(Frag), MaxLength), encode_iolist(?HANDSHAKE, Data, ConnectionStates); _ -> encode_plain_text(?HANDSHAKE, Frag, ConnectionStates) @@ -69,8 +75,14 @@ encode_alert_record(#alert{level = Level, description = Description}, %% %% Description: Encodes data to send on the ssl-socket. %%-------------------------------------------------------------------- -encode_data(Frag, ConnectionStates) -> - Data = tls_record:split_iovec(Frag), +encode_data(Frag, #{current_write := #{max_fragment_length := MaxFragmentLength}} = + ConnectionStates) -> + MaxLength = if is_integer(MaxFragmentLength) -> + MaxFragmentLength; + true -> + ?MAX_PLAIN_TEXT_LENGTH + end, + Data = tls_record:split_iovec(Frag, MaxLength), encode_iolist(?APPLICATION_DATA, Data, ConnectionStates). encode_plain_text(Type, Data0, #{current_write := Write0} = ConnectionStates) -> diff --git a/lib/ssl/src/tls_server_session_ticket.erl b/lib/ssl/src/tls_server_session_ticket.erl index a804f81eaa..9517cc5afd 100644 --- a/lib/ssl/src/tls_server_session_ticket.erl +++ b/lib/ssl/src/tls_server_session_ticket.erl @@ -81,8 +81,8 @@ init(Args) -> handle_call({new_session_ticket, Prf, MasterSecret}, _From, #state{nonce = Nonce, lifetime = LifeTime, - stateful = #{}} = State0) -> - Id = stateful_psk_id(), + stateful = #{id_generator := IdGen}} = State0) -> + Id = stateful_psk_ticket_id(IdGen), PSK = tls_v1:pre_shared_key(MasterSecret, ticket_nonce(Nonce), Prf), SessionTicket = new_session_ticket(Id, Nonce, LifeTime), State = stateful_ticket_store(Id, SessionTicket, Prf, PSK, State0), @@ -166,7 +166,8 @@ inital_state([stateful, Lifetime, TicketStoreSize|_]) -> nonce = 0, stateful = #{db => stateful_store(), max => TicketStoreSize, - ref_index => #{} + ref_index => #{}, + id_generator => crypto:strong_rand_bytes(16) } }. @@ -295,8 +296,13 @@ stateful_living_ticket({TimeStamp,_}, Lived < LifeTime. -stateful_psk_id() -> - term_to_binary(make_ref()). +stateful_psk_ticket_id(Key) -> + Unique = erlang:unique_integer(), + %% Obfuscate to avoid DoS attack possiblities + %% that could invalidate tickets and render them + %% unusable. This id should be unpredictable + %% and unique but have no other cryptographic requirements. + crypto:crypto_one_time(aes_128_ecb, Key, <<Unique:128>>, true). %%%=================================================================== %%% Stateless ticket diff --git a/lib/ssl/src/tls_v1.erl b/lib/ssl/src/tls_v1.erl index 8237886f27..8e6807d0ab 100644 --- a/lib/ssl/src/tls_v1.erl +++ b/lib/ssl/src/tls_v1.erl @@ -29,22 +29,52 @@ -include("ssl_internal.hrl"). -include("ssl_record.hrl"). --export([master_secret/4, finished/5, certificate_verify/3, mac_hash/7, hmac_hash/3, - setup_keys/8, suites/1, prf/5, - ecc_curves/1, ecc_curves/2, oid_to_enum/1, enum_to_oid/1, - default_signature_algs/1, signature_algs/2, - default_signature_schemes/1, signature_schemes/2, - groups/1, groups/2, group_to_enum/1, enum_to_group/1, default_groups/1]). - --export([derive_secret/4, hkdf_expand_label/5, hkdf_extract/3, hkdf_expand/4, - key_schedule/3, key_schedule/4, create_info/3, - external_binder_key/2, resumption_binder_key/2, - client_early_traffic_secret/3, early_exporter_master_secret/3, - client_handshake_traffic_secret/3, server_handshake_traffic_secret/3, - client_application_traffic_secret_0/3, server_application_traffic_secret_0/3, - exporter_master_secret/3, resumption_master_secret/3, - update_traffic_secret/2, calculate_traffic_keys/3, - transcript_hash/2, finished_key/2, finished_verify_data/3, pre_shared_key/3]). +-export([master_secret/4, + finished/5, + certificate_verify/3, + mac_hash/7, + hmac_hash/3, + setup_keys/8, + suites/1, + exclusive_suites/1, + prf/5, + ecc_curves/1, + ecc_curves/2, + oid_to_enum/1, + enum_to_oid/1, + default_signature_algs/1, + signature_algs/2, + default_signature_schemes/1, + signature_schemes/2, + groups/1, + groups/2, + group_to_enum/1, + enum_to_group/1, + default_groups/1]). + +-export([derive_secret/4, + hkdf_expand_label/5, + hkdf_extract/3, + hkdf_expand/4, + key_schedule/3, + key_schedule/4, + create_info/3, + external_binder_key/2, + resumption_binder_key/2, + client_early_traffic_secret/3, + early_exporter_master_secret/3, + client_handshake_traffic_secret/3, + server_handshake_traffic_secret/3, + client_application_traffic_secret_0/3, + server_application_traffic_secret_0/3, + exporter_master_secret/3, + resumption_master_secret/3, + update_traffic_secret/2, + calculate_traffic_keys/3, + transcript_hash/2, + finished_key/2, + finished_verify_data/3, + pre_shared_key/3]). -type named_curve() :: sect571r1 | sect571k1 | secp521r1 | brainpoolP512r1 | sect409k1 | sect409r1 | brainpoolP384r1 | secp384r1 | @@ -453,7 +483,7 @@ mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, %% TODO 1.3 same as above? --spec suites(1|2|3|4|'TLS_v1.3') -> [ssl_cipher_format:cipher_suite()]. +-spec suites(1|2|3|4) -> [ssl_cipher_format:cipher_suite()]. suites(Minor) when Minor == 1; Minor == 2 -> [ @@ -472,31 +502,57 @@ suites(Minor) when Minor == 1; Minor == 2 -> ?TLS_ECDH_RSA_WITH_AES_128_CBC_SHA ]; suites(3) -> + exclusive_suites(3) ++ suites(2); + +suites(4) -> + exclusive_suites(4) ++ suites(3). + +exclusive_suites(4) -> + [?TLS_AES_256_GCM_SHA384, + ?TLS_AES_128_GCM_SHA256, + ?TLS_CHACHA20_POLY1305_SHA256, + ?TLS_AES_128_CCM_SHA256, + ?TLS_AES_128_CCM_8_SHA256 + ]; +exclusive_suites(3) -> [?TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, ?TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, + ?TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, ?TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, + ?TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, ?TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, + ?TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, ?TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, ?TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, ?TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, + ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, ?TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, ?TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, ?TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, + + ?TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, + ?TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, + ?TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, ?TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, + ?TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, ?TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, + ?TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, ?TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, ?TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, ?TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, + + ?TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256, + ?TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, ?TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 @@ -505,25 +561,24 @@ suites(3) -> %% ?TLS_DH_DSS_WITH_AES_256_GCM_SHA384, %% ?TLS_DH_RSA_WITH_AES_128_GCM_SHA256, %% ?TLS_DH_DSS_WITH_AES_128_GCM_SHA256 - ] ++ suites(2); - -suites(4) -> - [?TLS_AES_256_GCM_SHA384, - ?TLS_AES_128_GCM_SHA256, - ?TLS_CHACHA20_POLY1305_SHA256, - ?TLS_AES_128_CCM_SHA256, - ?TLS_AES_128_CCM_8_SHA256 - ] ++ suites(3); + ]; +exclusive_suites(Minor) when Minor == 1; Minor == 2 -> + [ + ?TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, + ?TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, + ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA, + ?TLS_DHE_DSS_WITH_AES_256_CBC_SHA, + ?TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, + ?TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, -suites('TLS_v1.3') -> - [?TLS_AES_256_GCM_SHA384, - ?TLS_AES_128_GCM_SHA256, - ?TLS_CHACHA20_POLY1305_SHA256, - ?TLS_AES_128_CCM_SHA256, - ?TLS_AES_128_CCM_8_SHA256 + ?TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, + ?TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, + ?TLS_DHE_RSA_WITH_AES_128_CBC_SHA, + ?TLS_DHE_DSS_WITH_AES_128_CBC_SHA, + ?TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, + ?TLS_ECDH_RSA_WITH_AES_128_CBC_SHA ]. - signature_algs({3, 4}, HashSigns) -> signature_algs({3, 3}, HashSigns); signature_algs({3, 3}, HashSigns) -> diff --git a/lib/ssl/test/Makefile b/lib/ssl/test/Makefile index f4324f2f13..27b7598991 100644 --- a/lib/ssl/test/Makefile +++ b/lib/ssl/test/Makefile @@ -54,6 +54,7 @@ MODULES = \ ssl_npn_SUITE \ openssl_npn_SUITE\ openssl_sni_SUITE\ + ssl_mfl_SUITE \ ssl_renegotiate_SUITE\ openssl_renegotiate_SUITE\ openssl_reject_SUITE\ diff --git a/lib/ssl/test/openssl_alpn_SUITE.erl b/lib/ssl/test/openssl_alpn_SUITE.erl index fc18d053aa..409c90b0a8 100644 --- a/lib/ssl/test/openssl_alpn_SUITE.erl +++ b/lib/ssl/test/openssl_alpn_SUITE.erl @@ -129,7 +129,7 @@ init_per_testcase(TestCase, Config) -> special_init(erlang_client_alpn_openssl_server_alpn_renegotiate, Config) -> {ok, Version} = application:get_env(ssl, protocol_version), - case ssl_test_lib:check_sane_openssl_renegotaite(Config, Version) of + case ssl_test_lib:check_sane_openssl_renegotiate(Config, Version) of {skip, _} = Skip -> Skip; Config -> @@ -137,7 +137,7 @@ special_init(erlang_client_alpn_openssl_server_alpn_renegotiate, Config) -> end; special_init(erlang_server_alpn_openssl_client_alpn_renegotiate, Config) -> {ok, Version} = application:get_env(ssl, protocol_version), - case ssl_test_lib:check_sane_openssl_renegotaite(Config, Version) of + case ssl_test_lib:check_sane_openssl_renegotiate(Config, Version) of {skip, _} = Skip -> Skip; Config -> @@ -299,7 +299,7 @@ start_erlang_client_and_openssl_server_for_alpn_negotiation(Config, Data, Callba {from, self()}, {mfa, {ssl_test_lib, erlang_ssl_receive_and_assert_negotiated_protocol, [<<"spdy/2">>, Data]}}, - {options, [{reuse_sessions, false} | ClientOpts]}]), + {options, ClientOpts}]), Callback(Client, OpensslPort), @@ -368,7 +368,7 @@ start_erlang_client_and_openssl_server_for_alpn_npn_negotiation(Config, Data, Ca {from, self()}, {mfa, {ssl_test_lib, erlang_ssl_receive_and_assert_negotiated_protocol, [<<"spdy/2">>, Data]}}, - {options, [{reuse_sessions, false} | ClientOpts]}]), + {options, ClientOpts}]), Callback(Client, OpensslPort), diff --git a/lib/ssl/test/openssl_cipher_suite_SUITE.erl b/lib/ssl/test/openssl_cipher_suite_SUITE.erl index 048fbb1e92..5246cc028e 100644 --- a/lib/ssl/test/openssl_cipher_suite_SUITE.erl +++ b/lib/ssl/test/openssl_cipher_suite_SUITE.erl @@ -27,7 +27,7 @@ -include_lib("common_test/include/ct.hrl"). --define(DEFAULT_TIMEOUT, {seconds, 30}). +-define(DEFAULT_TIMEOUT, {seconds, 6}). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- diff --git a/lib/ssl/test/openssl_client_cert_SUITE.erl b/lib/ssl/test/openssl_client_cert_SUITE.erl index d04ba601cf..3c97822915 100644 --- a/lib/ssl/test/openssl_client_cert_SUITE.erl +++ b/lib/ssl/test/openssl_client_cert_SUITE.erl @@ -48,6 +48,8 @@ groups() -> {dsa, [], all_version_tests()}, {rsa_1_3, [], all_version_tests() ++ tls_1_3_tests() ++ [unsupported_sign_algo_client_auth, unsupported_sign_algo_cert_client_auth]}, + {rsa_pss_rsae, [], all_version_tests() ++ tls_1_3_tests()}, + {rsa_pss_pss, [], all_version_tests() ++ tls_1_3_tests()}, {ecdsa_1_3, [], all_version_tests() ++ tls_1_3_tests()} ]. @@ -71,6 +73,8 @@ pre_tls_1_3_protocol_groups() -> tls_1_3_protocol_groups() -> [{group, rsa_1_3}, + {group, rsa_pss_rsae}, + {group, rsa_pss_pss}, {group, ecdsa_1_3}]. tls_1_3_tests() -> @@ -139,6 +143,28 @@ init_per_group(Group, Config0) when Group == rsa; [] -> {skip, {no_sup, Group, Version}} end; +init_per_group(Alg, Config) when Alg == rsa_pss_rsae; + Alg == rsa_pss_pss -> + Supports = crypto:supports(), + RSAOpts = proplists:get_value(rsa_opts, Supports), + + case lists:member(rsa_pkcs1_pss_padding, RSAOpts) + andalso lists:member(rsa_pss_saltlen, RSAOpts) + andalso lists:member(rsa_mgf1_md, RSAOpts) + andalso ssl_test_lib:is_sane_oppenssl_pss(Alg) + of + true -> + #{client_config := COpts, + server_config := SOpts} = ssl_test_lib:make_rsa_pss_pem(Alg, [], Config, ""), + [{cert_key_alg, Alg} | + lists:delete(cert_key_alg, + [{client_cert_opts, COpts}, + {server_cert_opts, SOpts} | + lists:delete(server_cert_opts, + lists:delete(client_cert_opts, Config))])]; + false -> + {skip, "Missing crypto or OpenSSL support"} + end; init_per_group(Group, Config0) when Group == ecdsa; Group == ecdsa_1_3 -> PKAlg = crypto:supports(public_keys), diff --git a/lib/ssl/test/openssl_npn_SUITE.erl b/lib/ssl/test/openssl_npn_SUITE.erl index 11b2e46358..a37a4bf1f6 100644 --- a/lib/ssl/test/openssl_npn_SUITE.erl +++ b/lib/ssl/test/openssl_npn_SUITE.erl @@ -88,30 +88,10 @@ end_per_suite(_Config) -> ssl_test_lib:kill_openssl(). init_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - case ssl_test_lib:supports_ssl_tls_version(GroupName) of - true -> - case ssl_test_lib:check_sane_openssl_version(GroupName) of - true -> - ssl_test_lib:init_tls_version(GroupName, Config); - false -> - {skip, openssl_does_not_support_version} - end; - false -> - {skip, openssl_does_not_support_version} - end; - _ -> - Config - end. + ssl_test_lib:init_per_group_openssl(GroupName, Config). end_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - ssl_test_lib:clean_tls_version(Config); - false -> - Config - end. + ssl_test_lib:end_per_group(GroupName, Config). init_per_testcase(TestCase, Config) -> ct:timetrap({seconds, 10}), @@ -119,10 +99,10 @@ init_per_testcase(TestCase, Config) -> special_init(erlang_client_openssl_server_npn_renegotiate, Config) -> {ok, Version} = application:get_env(ssl, protocol_version), - ssl_test_lib:check_sane_openssl_renegotaite(Config, Version); + ssl_test_lib:check_sane_openssl_renegotiate(Config, Version); special_init(erlang_server_openssl_client_npn_renegotiate, Config) -> {ok, Version} = application:get_env(ssl, protocol_version), - case ssl_test_lib:check_sane_openssl_renegotaite(Config, Version) of + case ssl_test_lib:check_sane_openssl_renegotiate(Config, Version) of Config -> ssl_test_lib:openssl_allows_client_renegotiate(Config); Skip -> diff --git a/lib/ssl/test/openssl_reject_SUITE.erl b/lib/ssl/test/openssl_reject_SUITE.erl index a637035f3c..d451f8437e 100644 --- a/lib/ssl/test/openssl_reject_SUITE.erl +++ b/lib/ssl/test/openssl_reject_SUITE.erl @@ -74,30 +74,10 @@ end_per_suite(_Config) -> ssl_test_lib:kill_openssl(). init_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - case ssl_test_lib:supports_ssl_tls_version(GroupName) of - true -> - case ssl_test_lib:check_sane_openssl_version(GroupName) of - true -> - ssl_test_lib:init_tls_version(GroupName, Config); - false -> - {skip, openssl_does_not_support_version} - end; - false -> - {skip, openssl_does_not_support_version} - end; - _ -> - Config - end. + ssl_test_lib:init_per_group_openssl(GroupName, Config). end_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - ssl_test_lib:clean_tls_version(Config); - false -> - Config - end. + ssl_test_lib:end_per_group(GroupName, Config). init_per_testcase(TestCase, Config) -> ct:timetrap({seconds, 10}), diff --git a/lib/ssl/test/openssl_renegotiate_SUITE.erl b/lib/ssl/test/openssl_renegotiate_SUITE.erl index 78cd4446fc..66dfdc8115 100644 --- a/lib/ssl/test/openssl_renegotiate_SUITE.erl +++ b/lib/ssl/test/openssl_renegotiate_SUITE.erl @@ -96,32 +96,20 @@ end_per_suite(_Config) -> ssl_test_lib:kill_openssl(). init_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of + case ssl_test_lib:check_sane_openssl_version(GroupName) of true -> - case ssl_test_lib:supports_ssl_tls_version(GroupName) of - true -> - case ssl_test_lib:check_sane_openssl_version(GroupName) of - true -> - ssl_test_lib:check_sane_openssl_renegotiate( - ssl_test_lib:init_tls_version(GroupName, Config), - GroupName); - false -> - {skip, openssl_does_not_support_version} - end; - false -> - {skip, openssl_does_not_support_version} - end; - _ -> - Config + case ssl_test_lib:check_sane_openssl_renegotiate(Config, GroupName) of + {skip,_} = Skip -> + Skip; + _ -> + ssl_test_lib:init_per_group_openssl(GroupName, Config) + end; + false -> + {skip, {atom_to_list(GroupName) ++ " not supported by OpenSSL"}} end. - end_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - ssl_test_lib:clean_tls_version(Config); - false -> - Config - end. + ssl_test_lib:end_per_group(GroupName, Config). + init_per_testcase(erlang_client_openssl_server_nowrap_seqnum, Config) -> ct:timetrap(?DEFAULT_TIMEOUT), ssl_test_lib:openssl_allows_client_renegotiate(Config); diff --git a/lib/ssl/test/openssl_server_cert_SUITE.erl b/lib/ssl/test/openssl_server_cert_SUITE.erl index 9d8e095460..805da27510 100644 --- a/lib/ssl/test/openssl_server_cert_SUITE.erl +++ b/lib/ssl/test/openssl_server_cert_SUITE.erl @@ -49,6 +49,8 @@ groups() -> %% TODO: Create proper conf of openssl server %%++ [unsupported_sign_algo_client_auth, %% unsupported_sign_algo_cert_client_auth]}, + {rsa_pss_rsae, [], all_version_tests() ++ tls_1_3_tests()}, + {rsa_pss_pss, [], all_version_tests() ++ tls_1_3_tests()}, {ecdsa_1_3, [], all_version_tests() ++ tls_1_3_tests()} ]. @@ -72,6 +74,8 @@ pre_tls_1_3_protocol_groups() -> tls_1_3_protocol_groups() -> [{group, rsa_1_3}, + {group, rsa_pss_rsae}, + {group, rsa_pss_pss}, {group, ecdsa_1_3}]. tls_1_3_tests() -> @@ -151,6 +155,28 @@ init_per_group(rsa_1_3 = Group, Config0) -> [] -> {skip, {no_sup, Group, Version}} end; +init_per_group(Alg, Config) when Alg == rsa_pss_rsae; + Alg == rsa_pss_pss -> + Supports = crypto:supports(), + RSAOpts = proplists:get_value(rsa_opts, Supports), + + case lists:member(rsa_pkcs1_pss_padding, RSAOpts) + andalso lists:member(rsa_pss_saltlen, RSAOpts) + andalso lists:member(rsa_mgf1_md, RSAOpts) + andalso ssl_test_lib:is_sane_oppenssl_pss(Alg) + of + true -> + #{client_config := COpts, + server_config := SOpts} = ssl_test_lib:make_rsa_pss_pem(Alg, [], Config, ""), + [{cert_key_alg, Alg} | + lists:delete(cert_key_alg, + [{client_cert_opts, COpts}, + {server_cert_opts, SOpts} | + lists:delete(server_cert_opts, + lists:delete(client_cert_opts, Config))])]; + false -> + {skip, "Missing crypto or OpenSSL support"} + end; init_per_group(ecdsa = Group, Config0) -> PKAlg = crypto:supports(public_keys), case lists:member(ecdsa, PKAlg) andalso (lists:member(ecdh, PKAlg) orelse diff --git a/lib/ssl/test/openssl_session_SUITE.erl b/lib/ssl/test/openssl_session_SUITE.erl index e528936eef..ae66cdeb51 100644 --- a/lib/ssl/test/openssl_session_SUITE.erl +++ b/lib/ssl/test/openssl_session_SUITE.erl @@ -28,6 +28,7 @@ -define(SLEEP, 1000). -define(EXPIRE, 10). +-define(TIMEOUT, {seconds, 120}). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -93,31 +94,10 @@ end_per_suite(_Config) -> ssl_test_lib:kill_openssl(). init_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - case ssl_test_lib:supports_ssl_tls_version(GroupName) of - true -> - case ssl_test_lib:check_sane_openssl_version(GroupName) of - true -> - ssl_test_lib:init_tls_version(GroupName, Config); - false -> - {skip, openssl_does_not_support_version} - end; - false -> - {skip, openssl_does_not_support_version} - end; - _ -> - ssl:start(), - Config - end. + ssl_test_lib:init_per_group_openssl(GroupName, Config). end_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - ssl_test_lib:clean_tls_version(Config); - false -> - Config - end. + ssl_test_lib:end_per_group(GroupName, Config). init_per_testcase(reuse_session_erlang_client, Config) -> ct:timetrap(?EXPIRE * 1000 * 5), @@ -132,17 +112,17 @@ init_per_testcase(reuse_session_erlang_server, Config) -> true -> case ssl_test_lib:openssl_sane_dtls_session_reuse() of true -> - ct:timetrap({seconds, 10}), + ct:timetrap(?TIMEOUT), Config; false -> {skip, "Broken OpenSSL DTLS session reuse"} end; false -> - ct:timetrap({seconds, 10}), + ct:timetrap(?TIMEOUT), Config end; -init_per_testcase(TestCase, Config) -> - ct:timetrap({seconds, 10}), +init_per_testcase(_TestCase, Config) -> + ct:timetrap(?TIMEOUT), Config. end_per_testcase(reuse_session_erlang_client, Config) -> diff --git a/lib/ssl/test/openssl_sni_SUITE.erl b/lib/ssl/test/openssl_sni_SUITE.erl index 3010eabf4e..070d94245e 100644 --- a/lib/ssl/test/openssl_sni_SUITE.erl +++ b/lib/ssl/test/openssl_sni_SUITE.erl @@ -214,11 +214,13 @@ send_and_hostname(SSLSocket) -> end. openssl_client_args(Version, Hostname, Port) -> - ["s_client", "-connect", Hostname ++ ":" ++ integer_to_list(Port), ssl_test_lib:version_flag(Version)]. + ssl_test_lib:maybe_force_ipv4(["s_client", "-connect", Hostname ++ ":" ++ integer_to_list(Port), + ssl_test_lib:version_flag(Version)]). openssl_client_args(Version, Hostname, Port, ServerName) -> - ["s_client", "-connect", Hostname ++ ":" ++ - integer_to_list(Port), ssl_test_lib:version_flag(Version), "-servername", ServerName]. + ssl_test_lib:maybe_force_ipv4(["s_client", "-connect", Hostname ++ ":" ++ + integer_to_list(Port), + ssl_test_lib:version_flag(Version), "-servername", ServerName]). check_openssl_sni_support(Config) -> HelpText = os:cmd("openssl s_client --help"), diff --git a/lib/ssl/test/ssl_alpn_SUITE.erl b/lib/ssl/test/ssl_alpn_SUITE.erl index 6fb08671c1..424776293a 100644 --- a/lib/ssl/test/ssl_alpn_SUITE.erl +++ b/lib/ssl/test/ssl_alpn_SUITE.erl @@ -61,7 +61,6 @@ alpn_tests() -> client_alpn_and_server_alpn, client_alpn_and_server_no_support, client_no_support_and_server_alpn, - client_alpn_npn_and_server_alpn, client_renegotiate, session_reused ]. @@ -69,7 +68,8 @@ alpn_tests() -> alpn_npn_coexist() -> [ client_alpn_npn_and_server_alpn_npn, - client_alpn_and_server_alpn_npn + client_alpn_and_server_alpn_npn, + client_alpn_npn_and_server_alpn ]. diff --git a/lib/ssl/test/ssl_api_SUITE.erl b/lib/ssl/test/ssl_api_SUITE.erl index a80363227f..9856c5db0f 100644 --- a/lib/ssl/test/ssl_api_SUITE.erl +++ b/lib/ssl/test/ssl_api_SUITE.erl @@ -43,9 +43,14 @@ all() -> groups() -> [ - {'tlsv1.3', [], ((gen_api_tests() ++ tls13_group() ++ handshake_paus_tests()) -- - [dh_params, honor_server_cipher_order, honor_client_cipher_order, - new_options_in_handshake, handshake_continue_tls13_client]) + {'tlsv1.3', [], ((gen_api_tests() ++ tls13_group() ++ + handshake_paus_tests()) -- + [dh_params, + honor_server_cipher_order, + honor_client_cipher_order, + new_options_in_handshake, + handshake_continue_tls13_client, + invalid_options]) ++ (since_1_2() -- [conf_signature_algs])}, {'tlsv1.2', [], gen_api_tests() ++ since_1_2() ++ handshake_paus_tests() ++ pre_1_3()}, {'tlsv1.1', [], gen_api_tests() ++ handshake_paus_tests() ++ pre_1_3()}, @@ -68,7 +73,8 @@ since_1_2() -> pre_1_3() -> [ - default_reject_anonymous + default_reject_anonymous, + connection_information_with_srp ]. gen_api_tests() -> [ @@ -140,10 +146,10 @@ tls13_group() -> client_options_negative_dependency_stateless, client_options_negative_dependency_role, server_options_negative_version_gap, - server_options_negative_dependency_role + server_options_negative_dependency_role, + invalid_options_tls13 ]. - init_per_suite(Config0) -> catch crypto:stop(), try crypto:start() of @@ -197,6 +203,14 @@ init_per_testcase(handshake_continue_tls13_client, Config) -> false -> {skip, "Missing crypto support: TLS 1.3 not supported"} end; +init_per_testcase(connection_information_with_srp, Config) -> + PKAlg = proplists:get_value(public_keys, crypto:supports()), + case lists:member(srp, PKAlg) of + true -> + Config; + false -> + {skip, "Missing SRP crypto support"} + end; init_per_testcase(_TestCase, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), ct:timetrap({seconds, 10}), @@ -304,6 +318,56 @@ connection_information(Config) when is_list(Config) -> ssl_test_lib:close(Server), ssl_test_lib:close(Client). + +%%-------------------------------------------------------------------- +connection_information_with_srp() -> + [{doc,"Test the result of API function ssl:connection_information/1" + "includes srp_username."}]. +connection_information_with_srp(Config) when is_list(Config) -> + run_conn_info_srp_test(srp_anon, 'aes_128_cbc', Config). + +run_conn_info_srp_test(Kex, Cipher, Config) -> + Version = ssl_test_lib:protocol_version(Config), + TestCiphers = ssl_test_lib:test_ciphers(Kex, Cipher, Version), + + case TestCiphers of + [] -> + {skip, {not_sup, Kex, Cipher, Version}}; + [TestCipher | _T] -> + do_run_conn_info_srp_test(TestCipher, Version, Config) + end. + +do_run_conn_info_srp_test(ErlangCipherSuite, Version, Config) -> + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + SOpts = [{user_lookup_fun, {fun ssl_test_lib:user_lookup/3, undefined}}], + COpts = [{srp_identity, {"Test-User", "secret"}}], + + ServerOpts = ssl_test_lib:ssl_options(SOpts, Config), + ClientOpts = ssl_test_lib:ssl_options(COpts, Config), + + ct:log("Erlang Cipher Suite is: ~p~n", [ErlangCipherSuite]), + + Server = ssl_test_lib:start_server([ + {node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, check_srp_in_connection_information, [<<"Test-User">>, server]}}, + {options, [{versions, [Version]}, {ciphers, [ErlangCipherSuite]} | + ServerOpts]}]), + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client( + [{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, check_srp_in_connection_information, [<<"Test-User">>, client]}}, + {options, [{versions, [Version]}, {ciphers, [ErlangCipherSuite]} | + ClientOpts]}]), + + ssl_test_lib:check_result(Server, ok, Client, ok), + + ssl_test_lib:close(Server), + ssl_test_lib:close(Client). + %%-------------------------------------------------------------------- secret_connection_info() -> @@ -1554,7 +1618,6 @@ invalid_options(Config) when is_list(Config) -> {verify, 4}, {verify_fun, function}, {fail_if_no_peer_cert, 0}, - {verify_client_once, 1}, {depth, four}, {certfile, 'cert.pem'}, {keyfile,'key.pem' }, @@ -1572,6 +1635,20 @@ invalid_options(Config) when is_list(Config) -> {active, trice}, {key, 'key.pem' }], + TestOpts2 = + [{[{anti_replay, '10k'}], + %% anti_replay is a server only option but tested with client + %% for simplicity + {options,dependency,{anti_replay,{versions,['tlsv1.3']}}}}, + {[{supported_groups, []}], + {options,dependency,{supported_groups,{versions,['tlsv1.3']}}}}, + {[{use_ticket, [<<1,2,3,4>>]}], + {options,dependency,{use_ticket,{versions,['tlsv1.3']}}}}, + {[{verify, verify_none}, {fail_if_no_peer_cert, true}], + {options, incompatible, + {verify, verify_none}, + {fail_if_no_peer_cert, true}}}], + [begin Server = ssl_test_lib:start_server_error([{node, ServerNode}, {port, 0}, @@ -1585,7 +1662,13 @@ invalid_options(Config) when is_list(Config) -> Check(Client, Server, TestOpt), ok end || TestOpt <- TestOpts], + + [begin + start_client_negative(Config, TestOpt, ErrorMsg), + ok + end || {TestOpt, ErrorMsg} <- TestOpts2], ok. + %%------------------------------------------------------------------- default_reject_anonymous()-> @@ -1853,6 +1936,103 @@ getstat(Config) when is_list(Config) -> {options, ClientOpts}]), ssl_test_lib:check_result(Server, ok, Client, ok). +invalid_options_tls13() -> + [{doc, "Test invalid options with TLS 1.3"}]. +invalid_options_tls13(Config) when is_list(Config) -> + TestOpts = + [{{beast_mitigation, one_n_minus_one}, + {options, dependency, + {beast_mitigation,{versions,[tlsv1]}}}, + common}, + + {{next_protocols_advertised, [<<"http/1.1">>]}, + {options, dependency, + {next_protocols_advertised, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + server}, + + {{client_preferred_next_protocols, + {client, [<<"http/1.1">>]}}, + {options, dependency, + {client_preferred_next_protocols, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + client}, + + {{client_renegotiation, false}, + {options, dependency, + {client_renegotiation, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + server + }, + + {{padding_check, false}, + {options, dependency, + {padding_check,{versions,[tlsv1]}}}, + common}, + + {{psk_identity, "Test-User"}, + {options, dependency, + {psk_identity,{versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + common}, + + {{user_lookup_fun, + {fun ssl_test_lib:user_lookup/3, <<1,2,3>>}}, + {options, dependency, + {user_lookup_fun,{versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + common}, + + {{reuse_session, fun(_,_,_,_) -> false end}, + {options, dependency, + {reuse_session, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + server}, + + {{reuse_session, <<1,2,3,4>>}, + {options, dependency, + {reuse_session, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + client}, + + {{reuse_sessions, true}, + {options, dependency, + {reuse_sessions, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + common}, + + {{secure_renegotiate, false}, + {options, dependency, + {secure_renegotiate, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + common}, + + {{srp_identity, false}, + {options, dependency, + {srp_identity, + {versions,[tlsv1,'tlsv1.1','tlsv1.2']}}}, + client} + ], + + Fun = fun(Option, ErrorMsg, Type) -> + case Type of + server -> + start_server_negative(Config, + [Option,{versions, ['tlsv1.3']}], + ErrorMsg); + client -> + start_client_negative(Config, + [Option,{versions, ['tlsv1.3']}], + ErrorMsg); + common -> + start_server_negative(Config, + [Option,{versions, ['tlsv1.3']}], + ErrorMsg), + start_client_negative(Config, + [Option,{versions, ['tlsv1.3']}], + ErrorMsg) + end + end, + [Fun(Option, ErrorMsg, Type) || {Option, ErrorMsg, Type} <- TestOpts]. + %%-------------------------------------------------------------------- %% Internal functions ------------------------------------------------ @@ -1875,7 +2055,18 @@ secret_connection_info_result(Socket) -> {ok, [{protocol, Protocol}]} = ssl:connection_information(Socket, [protocol]), {ok, ConnInfo} = ssl:connection_information(Socket, [client_random, server_random, master_secret]), check_connection_info(Protocol, ConnInfo). - + +check_srp_in_connection_information(_Socket, _Username, client) -> + ok; +check_srp_in_connection_information(Socket, Username, server) -> + {ok, Info} = ssl:connection_information(Socket), + ct:log("Info ~p~n", [Info]), + case proplists:get_value(srp_username, Info, not_found) of + Username -> + ok; + not_found -> + ct:fail(srp_username_not_found) + end. %% In TLS 1.3 the master_secret field is used to store multiple secrets from the key schedule and it is a tuple. %% client_random and server_random are not used in the TLS 1.3 key schedule. diff --git a/lib/ssl/test/ssl_app_env_SUITE.erl b/lib/ssl/test/ssl_app_env_SUITE.erl index 498bed2573..d337dabb69 100644 --- a/lib/ssl/test/ssl_app_env_SUITE.erl +++ b/lib/ssl/test/ssl_app_env_SUITE.erl @@ -25,7 +25,7 @@ -compile(export_all). -include_lib("common_test/include/ct.hrl"). -include_lib("ssl/src/ssl_api.hrl"). - +-define(TIMEOUT, {seconds, 5}). -define(SLEEP, 500). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -94,14 +94,14 @@ init_per_testcase(internal_active_1, Config) -> application:set_env(ssl, internal_active_n, 1), ssl_test_lib:set_protocol_versions(Version), ssl:start(), - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), ssl_test_lib:ct_log_supported_protocol_versions(Config), Config; init_per_testcase(protocol_versions, Config) -> Version = ssl_test_lib:protocol_version(Config), ssl_test_lib:set_protocol_versions(Version), ssl_test_lib:ct_log_supported_protocol_versions(Config), - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; init_per_testcase(empty_protocol_versions, Config) -> ssl:stop(), @@ -111,10 +111,10 @@ init_per_testcase(empty_protocol_versions, Config) -> application:set_env(ssl, dtls_protocol_version, []), ssl:start(), ssl_test_lib:ct_log_supported_protocol_versions(Config), - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; init_per_testcase(_TestCase, Config) -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(_, Config) -> diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 18c984d76b..2ca6c7de3d 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -125,7 +125,7 @@ appup(Config) when is_list(Config) -> version_option() -> [{doc, "Use version option and do no specify ciphers list. Bug specified incorrect ciphers"}]. version_option(Config) when is_list(Config) -> - Versions = proplists:get_value(default_supported, ssl:versions()), + Versions = proplists:get_value(supported, ssl:versions()), [version_option_test(Config, Version) || Version <- Versions]. %%-------------------------------------------------------------------- @@ -174,13 +174,13 @@ connect_twice(Config) when is_list(Config) -> defaults(Config) when is_list(Config)-> Versions = ssl:versions(), false = lists:member(sslv3, proplists:get_value(available, Versions)), - false = lists:member(sslv3, proplists:get_value(default_supported, Versions)), + false = lists:member(sslv3, proplists:get_value(supported, Versions)), true = lists:member('tlsv1', proplists:get_value(available, Versions)), - false = lists:member('tlsv1', proplists:get_value(default_supported, Versions)), + false = lists:member('tlsv1', proplists:get_value(supported, Versions)), true = lists:member('tlsv1.1', proplists:get_value(available, Versions)), - false = lists:member('tlsv1.1', proplists:get_value(default_supported, Versions)), + false = lists:member('tlsv1.1', proplists:get_value(supported, Versions)), true = lists:member('tlsv1.2', proplists:get_value(available, Versions)), - true = lists:member('tlsv1.2', proplists:get_value(default_supported, Versions)), + true = lists:member('tlsv1.2', proplists:get_value(supported, Versions)), false = lists:member({rsa,rc4_128,sha}, ssl:cipher_suites()), true = lists:member({rsa,rc4_128,sha}, ssl:cipher_suites(all)), false = lists:member({rsa,des_cbc,sha}, ssl:cipher_suites()), @@ -189,8 +189,8 @@ defaults(Config) when is_list(Config)-> true = lists:member({dhe_rsa,des_cbc,sha}, ssl:cipher_suites(all)), true = lists:member('dtlsv1.2', proplists:get_value(available_dtls, Versions)), true = lists:member('dtlsv1', proplists:get_value(available_dtls, Versions)), - true = lists:member('dtlsv1.2', proplists:get_value(default_supported_dtls, Versions)), - false = lists:member('dtlsv1', proplists:get_value(default_supported_dtls, Versions)). + true = lists:member('dtlsv1.2', proplists:get_value(supported_dtls, Versions)), + false = lists:member('dtlsv1', proplists:get_value(supported_dtls, Versions)). fallback() -> @@ -421,7 +421,7 @@ tls_versions_option(Config) when is_list(Config) -> ClientOpts = ssl_test_lib:ssl_options(client_opts, Config), ServerOpts = ssl_test_lib:ssl_options(server_opts, Config), - Supported = proplists:get_value(default_supported, ssl:versions()), + Supported = proplists:get_value(supported, ssl:versions()), Available = proplists:get_value(available, ssl:versions()), {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, diff --git a/lib/ssl/test/ssl_cert_SUITE.erl b/lib/ssl/test/ssl_cert_SUITE.erl index bcf2086cca..2fe470e281 100644 --- a/lib/ssl/test/ssl_cert_SUITE.erl +++ b/lib/ssl/test/ssl_cert_SUITE.erl @@ -43,15 +43,20 @@ all() -> groups() -> [ {'tlsv1.3', [], tls_1_3_protocol_groups()}, - {'tlsv1.2', [], tls_1_2_protocol_groups()}, + {'tlsv1.2', [], tls_1_2_protocol_groups() -- [{group,rsa_pss_pss}]}, {'tlsv1.1', [], ssl_protocol_groups()}, {'tlsv1', [], ssl_protocol_groups()}, - {'dtlsv1.2', [], tls_1_2_protocol_groups()}, + {'dtlsv1.2', [], tls_1_2_protocol_groups() -- [{group,rsa_pss_rsae}, {group,rsa_pss_pss}]}, {'dtlsv1', [], ssl_protocol_groups()}, {rsa, [], all_version_tests() ++ rsa_tests() ++ pre_tls_1_3_rsa_tests()}, {ecdsa, [], all_version_tests()}, {dsa, [], all_version_tests()}, - {rsa_1_3, [], all_version_tests() ++ rsa_tests() ++ tls_1_3_tests() ++ tls_1_3_rsa_tests()}, + {rsa_1_3, [], all_version_tests() ++ rsa_tests() ++ + tls_1_3_tests() ++ tls_1_3_rsa_tests() ++ [basic_rsa_1024]}, + {rsa_pss_rsae, [], all_version_tests() ++ rsa_tests()}, + {rsa_pss_rsae_1_3, [], all_version_tests() ++ rsa_tests() ++ tls_1_3_tests() ++ tls_1_3_rsa_tests()}, + {rsa_pss_pss, [], all_version_tests() ++ rsa_tests()}, + {rsa_pss_pss_1_3, [], all_version_tests() ++ rsa_tests() ++ tls_1_3_tests() ++ tls_1_3_rsa_tests()}, {ecdsa_1_3, [], all_version_tests() ++ tls_1_3_tests()} ]. @@ -62,11 +67,17 @@ ssl_protocol_groups() -> tls_1_2_protocol_groups() -> [{group, rsa}, {group, ecdsa}, - {group, dsa}]. + {group, dsa}, + {group, rsa_pss_rsae}, + {group, rsa_pss_pss} + ]. tls_1_3_protocol_groups() -> [{group, rsa_1_3}, - {group, ecdsa_1_3}]. + {group, ecdsa_1_3}, + {group, rsa_pss_rsae_1_3}, + {group, rsa_pss_pss_1_3} + ]. tls_1_3_tests() -> [ @@ -103,6 +114,7 @@ all_version_tests() -> client_auth_allow_partial_chain, client_auth_do_not_allow_partial_chain, client_auth_partial_chain_fun_fail, + client_auth_sni, missing_root_cert_no_auth, missing_root_cert_auth, missing_root_cert_auth_user_verify_fun_accept, @@ -118,7 +130,6 @@ all_version_tests() -> extended_key_usage_auth, extended_key_usage_client_auth, cert_expired, - client_auth_once, no_auth_key_identifier_ext, no_auth_key_identifier_ext_keyEncipherment ]. @@ -140,7 +151,8 @@ end_per_suite(_Config) -> init_per_group(Group, Config0) when Group == rsa; Group == rsa_1_3 -> - Config = ssl_test_lib:make_rsa_cert(Config0), + Config1 = ssl_test_lib:make_rsa_cert(Config0), + Config = ssl_test_lib:make_rsa_1024_cert(Config1), COpts = proplists:get_value(client_rsa_opts, Config), SOpts = proplists:get_value(server_rsa_opts, Config), [{cert_key_alg, rsa} | @@ -149,6 +161,30 @@ init_per_group(Group, Config0) when Group == rsa; {server_cert_opts, SOpts} | lists:delete(server_cert_opts, lists:delete(client_cert_opts, Config))])]; + +init_per_group(Alg, Config) when Alg == rsa_pss_rsae; + Alg == rsa_pss_pss; + Alg == rsa_pss_rsae_1_3; + Alg == rsa_pss_pss_1_3 -> + + Supports = crypto:supports(), + RSAOpts = proplists:get_value(rsa_opts, Supports), + + case lists:member(rsa_pkcs1_pss_padding, RSAOpts) + andalso lists:member(rsa_pss_saltlen, RSAOpts) + andalso lists:member(rsa_mgf1_md, RSAOpts) of + true -> + #{client_config := COpts, + server_config := SOpts} = ssl_test_lib:make_rsa_pss_pem(rsa_alg(Alg), [], Config, ""), + [{cert_key_alg, rsa_alg(Alg)} | + lists:delete(cert_key_alg, + [{client_cert_opts, COpts}, + {server_cert_opts, SOpts} | + lists:delete(server_cert_opts, + lists:delete(client_cert_opts, Config))])]; + false -> + {skip, "Missing EC crypto support"} + end; init_per_group(Group, Config0) when Group == ecdsa; Group == ecdsa_1_3 -> @@ -264,6 +300,12 @@ client_auth_partial_chain_fun_fail(Config) when is_list(Config) -> ssl_cert_tests:client_auth_partial_chain_fun_fail(Config). %%-------------------------------------------------------------------- +client_auth_sni() -> + ssl_cert_tests:client_auth_sni(). +client_auth_sni(Config) when is_list(Config) -> + ssl_cert_tests:client_auth_sni(Config). + +%%-------------------------------------------------------------------- missing_root_cert_no_auth() -> ssl_cert_tests:missing_root_cert_no_auth(). missing_root_cert_no_auth(Config) when is_list(Config) -> @@ -655,41 +697,6 @@ cert_expired(Config) when is_list(Config) -> ssl_test_lib:check_client_alert(Server, Client, certificate_expired). %%-------------------------------------------------------------------- -client_auth_once() -> - [{doc,"Test server option verify_client_once"}]. - -client_auth_once(Config) when is_list(Config) -> - ClientOpts = ssl_test_lib:ssl_options(client_cert_opts, Config), - ServerOpts = ssl_test_lib:ssl_options(server_cert_opts, Config), - - {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), - Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, - {from, self()}, - {mfa, {ssl_test_lib, send_recv_result_active, []}}, - {options, [{verify, verify_peer}, - {verify_client_once, true} - | ServerOpts]}]), - Port = ssl_test_lib:inet_port(Server), - Client0 = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, - {host, Hostname}, - {from, self()}, - {mfa, {ssl_test_lib, send_recv_result_active, []}}, - {options, ClientOpts}]), - - ssl_test_lib:check_result(Server, ok, Client0, ok), - Server ! {listen, {mfa, {ssl_test_lib, send_recv_result_active, []}}}, - ssl_test_lib:close(Client0), - Client1 = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, - {host, Hostname}, - {from, self()}, - {mfa, {ssl_test_lib, send_recv_result_active, []}}, - {options, ClientOpts}]), - - ssl_test_lib:check_result(Client1, ok, Server, ok), - ssl_test_lib:close(Server), - ssl_test_lib:close(Client1). - -%%-------------------------------------------------------------------- no_auth_key_identifier_ext() -> [{doc, "Test cert that does not have authorityKeyIdentifier extension"}]. @@ -815,7 +822,7 @@ unsupported_sign_algo_cert_client_auth(Config) -> ServerOpts0 = ssl_test_lib:ssl_options(server_cert_opts, Config), ServerOpts = [{versions, ['tlsv1.2','tlsv1.3']}, {verify, verify_peer}, - {signature_algs, [rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pss_rsae_sha256]}, + {signature_algs, [rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pss_rsae_sha256, rsa_pss_pss_sha256]}, %% Skip rsa_pkcs1_sha256! {signature_algs_cert, [rsa_pkcs1_sha384, rsa_pkcs1_sha512]}, {fail_if_no_peer_cert, true}|ServerOpts0], @@ -894,6 +901,19 @@ hello_retry_client_auth_empty_cert_rejected(Config) -> ssl_test_lib:basic_alert(ClientOpts, ServerOpts, Config, certificate_required). %%-------------------------------------------------------------------- +basic_rsa_1024() -> + [{doc, "TLS 1.3 (Basic): Test if connection can be established using 1024 bits RSA keys in certificates."}]. + +basic_rsa_1024(Config) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_1024_opts, Config), + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_1024_opts, Config), + ServerOpts1 = [{versions, ['tlsv1.2','tlsv1.3']}|ServerOpts0], + ClientOpts = [{versions, ['tlsv1.2','tlsv1.3']}|ClientOpts0], + ServerOpts = [{verify, verify_peer}, + {fail_if_no_peer_cert, true} | ServerOpts1], + ssl_test_lib:basic_test(ClientOpts, ServerOpts, Config). + +%%-------------------------------------------------------------------- %% Internal functions ----------------------------------------------- %%-------------------------------------------------------------------- two_digits_str(N) when N < 10 -> @@ -918,3 +938,10 @@ n_version(Version) when Version == 'tlsv1.2'; n_version(Version) when Version == 'dtlsv1.2'; Version == 'dtlsv1' -> dtls_record:protocol_version(Version). + +rsa_alg(rsa_pss_rsae_1_3) -> + rsa_pss_rsae; +rsa_alg(rsa_pss_pss_1_3) -> + rsa_pss_pss; +rsa_alg(Atom) -> + Atom. diff --git a/lib/ssl/test/ssl_cert_tests.erl b/lib/ssl/test/ssl_cert_tests.erl index c88daa2185..657ccd2079 100644 --- a/lib/ssl/test/ssl_cert_tests.erl +++ b/lib/ssl/test/ssl_cert_tests.erl @@ -162,6 +162,39 @@ client_auth_partial_chain_fun_fail(Config) when is_list(Config) -> ssl_test_lib:basic_alert(ClientOpts, ServerOpts, Config, unknown_ca). %%-------------------------------------------------------------------- +client_auth_sni() -> + [{doc, "Check that sni check works with user verify_fun"}]. +client_auth_sni(Config) when is_list(Config) -> + ServerOpts0 = ssl_test_lib:ssl_options(server_cert_opts, Config), + + FunAndState = {fun(valid_peer, {bad_cert, unknown_ca}, UserState) -> + {valid_peer, UserState}; + (_,{bad_cert, _} = Reason, _) -> + {fail, Reason}; + (_,{extension, _}, UserState) -> + {unknown, UserState}; + (_, valid, UserState) -> + {valid, UserState}; + (_, valid_peer, UserState) -> + {valid, UserState} + end, []}, + + ClientOpts0 = ssl_test_lib:ssl_options(client_cert_opts, Config), + ClientOpts = [{verify, verify_peer}, {verify_fun, FunAndState + }, {server_name_indication, "localhost"} | ClientOpts0], + + {ok, ServerCAs} = file:read_file(proplists:get_value(cacertfile, ServerOpts0)), + [{_,_,_}, {_, IntermidiateCA, _} | _] = public_key:pem_decode(ServerCAs), + + ServerOpts = [{cacerts, [IntermidiateCA]} | + proplists:delete(cacertfile, ServerOpts0)], + %% Basic test if hostname check is not performed the connection will succeed + ssl_test_lib:basic_alert(ClientOpts, ServerOpts0, Config, handshake_failure), + %% Also test that user verify_fun is run. + %% If user verify fun is not used the ALERT will be unknown_ca + ssl_test_lib:basic_alert(ClientOpts, ServerOpts, Config, handshake_failure). + +%%-------------------------------------------------------------------- missing_root_cert_no_auth() -> [{doc,"Test that the client succeds if the ROOT CA is unknown in verify_none mode"}]. diff --git a/lib/ssl/test/ssl_cipher_suite_SUITE.erl b/lib/ssl/test/ssl_cipher_suite_SUITE.erl index 4b19314a7a..71628e9b40 100644 --- a/lib/ssl/test/ssl_cipher_suite_SUITE.erl +++ b/lib/ssl/test/ssl_cipher_suite_SUITE.erl @@ -27,6 +27,8 @@ -include_lib("common_test/include/ct.hrl"). +-define(TIMEOUT, {seconds, 5}). + %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- %%-------------------------------------------------------------------- @@ -267,7 +269,7 @@ init_per_testcase(TestCase, Config) when TestCase == psk_3des_ede_cbc; SupCiphers = proplists:get_value(ciphers, crypto:supports()), case lists:member(des_ede3, SupCiphers) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing 3DES crypto support"} @@ -283,7 +285,7 @@ init_per_testcase(TestCase, Config) when TestCase == psk_rc4_128; SupCiphers = proplists:get_value(ciphers, crypto:supports()), case lists:member(rc4, SupCiphers) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing RC4 crypto support"} @@ -296,7 +298,7 @@ init_per_testcase(TestCase, Config) when TestCase == psk_aes_128_ccm_8; SupCiphers = proplists:get_value(ciphers, crypto:supports()), case lists:member(aes_128_ccm, SupCiphers) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing AES_128_CCM crypto support"} @@ -309,7 +311,7 @@ init_per_testcase(TestCase, Config) when TestCase == psk_aes_256_ccm_8; SupCiphers = proplists:get_value(ciphers, crypto:supports()), case lists:member(aes_256_ccm, SupCiphers) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing AES_256_CCM crypto support"} @@ -321,7 +323,7 @@ init_per_testcase(aes_256_gcm_sha384, Config) -> (lists:member(sha384, SupHashs)) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing AES_256_GCM_SHA384 crypto support"} @@ -333,7 +335,7 @@ init_per_testcase(aes_128_gcm_sha256, Config) -> (lists:member(sha256, SupHashs)) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing AES_128_GCM_SHA256 crypto support"} @@ -345,7 +347,7 @@ init_per_testcase(chacha20_poly1305_sha256, Config) -> (lists:member(sha256, SupHashs)) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing chacha20_poly1305_sha256 crypto support"} @@ -356,7 +358,7 @@ init_per_testcase(aes_128_ccm_sha256, Config) -> case (lists:member(aes_128_ccm, SupCiphers)) andalso (lists:member(sha256, SupHashs)) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing AES_128_CCM_SHA256 crypto support"} @@ -367,7 +369,7 @@ init_per_testcase(aes_128_ccm_8_sha256, Config) -> case (lists:member(aes_128_ccm, SupCiphers)) andalso (lists:member(sha256, SupHashs)) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, "Missing AES_128_CCM_8_SHA256 crypto support"} @@ -377,7 +379,7 @@ init_per_testcase(TestCase, Config) -> SupCiphers = proplists:get_value(ciphers, crypto:supports()), case lists:member(Cipher, SupCiphers) of true -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; _ -> {skip, {Cipher, SupCiphers}} diff --git a/lib/ssl/test/ssl_crl_SUITE.erl b/lib/ssl/test/ssl_crl_SUITE.erl index 8678d43450..5e0da22eb0 100644 --- a/lib/ssl/test/ssl_crl_SUITE.erl +++ b/lib/ssl/test/ssl_crl_SUITE.erl @@ -27,6 +27,8 @@ -include_lib("common_test/include/ct.hrl"). -include_lib("public_key/include/public_key.hrl"). +-define(TIMEOUT, {seconds, 30}). + %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- %%-------------------------------------------------------------------- @@ -159,7 +161,7 @@ init_per_testcase(Case, Config0) -> {CertOpts, Config} = init_certs(CertDir, idp_crl, Config), case make_certs:all(DataDir, CertDir, CertOpts) of {ok, _} -> - ct:timetrap({seconds, 6}), + ct:timetrap(?TIMEOUT), [{cert_dir, CertDir} | Config]; _ -> end_per_testcase(Case, Config0), diff --git a/lib/ssl/test/ssl_mfl_SUITE.erl b/lib/ssl/test/ssl_mfl_SUITE.erl new file mode 100644 index 0000000000..bcc2b24651 --- /dev/null +++ b/lib/ssl/test/ssl_mfl_SUITE.erl @@ -0,0 +1,513 @@ +%% +-module(ssl_mfl_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). +-include_lib("common_test/include/ct.hrl"). + +-define(SLEEP, 500). +%%-------------------------------------------------------------------- +%% Common Test interface functions ----------------------------------- +%%-------------------------------------------------------------------- + +all() -> + [{group, 'tlsv1.3'}, + {group, 'tlsv1.2'}, + {group, 'tlsv1.1'}, + {group, 'tlsv1'}, + {group, 'dtlsv1.2'}, + {group, 'dtlsv1'}]. + +groups() -> + [{'tlsv1.3', [], tls_mfl_1_3()}, + {'tlsv1.2', [], tls_mfl()}, + {'tlsv1.1', [], tls_mfl()}, + {'tlsv1', [], tls_mfl()}, + {'dtlsv1.2', [], tls_mfl()}, + {'dtlsv1', [], tls_mfl()} + ]. + +tls_mfl_common() -> + [mfl_client_option, mfl_server_option, mfl_openssl_client, mfl_openssl_server, handshake_continue]. + +tls_mfl() -> + tls_mfl_common() ++ [reuse_session, reuse_session_erlang_server, reuse_session_erlang_client]. + +tls_mfl_1_3() -> + tls_mfl_common(). + +init_per_suite(Config0) -> + catch crypto:stop(), + try crypto:start() of + ok -> + ssl_test_lib:clean_start(), + ssl:clear_pem_cache(), + Config = ssl_test_lib:make_rsa_cert(Config0), + ssl_test_lib:cert_options(Config) + catch _:_ -> + {skip, "Crypto did not start"} + end. + +end_per_suite(_Config) -> + ssl:stop(), + application:stop(crypto). + +init_per_group(GroupName, Config) -> + ssl_test_lib:init_per_group_openssl(GroupName, Config). + +end_per_group(GroupName, Config) -> + ssl_test_lib:end_per_group(GroupName, Config). + +init_per_testcase(TestCase, Config) when TestCase == mfl_openssl_server; + TestCase == mfl_openssl_client; + TestCase == reuse_session_erlang_server; + TestCase == reuse_session_erlang_client -> + case os:cmd("openssl version") of + %% Max fragmentation support introduced in OpenSSL 1.1.1 + "OpenSSL 1.1.1" ++ _ = OpenSSLVersion -> + case lists:member({protocol, dtls}, Config) of + true -> + %% Fixed but not yet released https://github.com/openssl/openssl/commit/dfbaef6 + {skip, "Broken DTLS max fragmentation support in "++OpenSSLVersion}; + false -> + init_per_testcase(all, Config) + end; + Other -> + {skip, "No/unknown max fragmentation support in "++Other} + end; +init_per_testcase(_TestCase, Config) -> + ct:timetrap({seconds, 10}), + Config. + +end_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Test Cases -------------------------------------------------------- +%%-------------------------------------------------------------------- + +nyi(Config) when is_list(Config) -> + {skip, "NYI"}. + +%-------------------------------------------------------------------------------- +%% check max_fragment_length option on the client is accepted +%% and both sides can successfully send > MFL +mfl_client_option(Config) when is_list(Config) -> + ok = run_mfl_handshake(Config, 512), + ok = run_mfl_handshake(Config, 1024), + ok = run_mfl_handshake(Config, 2048), + ok = run_mfl_handshake(Config, 4096), + ok = run_mfl_handshake(Config, undefined), + ok. + +%-------------------------------------------------------------------------------- +%% check max_fragment_length option on the server is ignored +%% and both sides can successfully send > 512 bytes +mfl_server_option(Config) when is_list(Config) -> + Data = "mfl_server_options " ++ lists:duplicate(512, $x), + run_mfl_handshake(Config, undefined, Data, [], [{max_fragment_length, 512}]). + +%-------------------------------------------------------------------------------- +%% check max_fragment_length interworking with openssl server +mfl_openssl_server(Config) when is_list(Config) -> + mfl_openssl_server(512, Config), + mfl_openssl_server(2048, Config). + +%-------------------------------------------------------------------------------- +%% check max_fragment_length interworking with openssl client +mfl_openssl_client(Config) when is_list(Config) -> + mfl_openssl_client(1024, Config), + mfl_openssl_client(4096, Config). + +%-------------------------------------------------------------------------------- +%% check max_fragment_length option on the client is accepted and reused +reuse_session(Config) when is_list(Config) -> + MFL = 512, + ClientOpts = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config) ++ + [{max_fragment_length, MFL}], + ServerOpts = ssl_test_lib:ssl_options(server_rsa_verify_opts, Config), + + ssl_test_lib:reuse_session(ClientOpts, ServerOpts, Config). + +%%-------------------------------------------------------------------- + +reuse_session_erlang_server(Config) when is_list(Config) -> + process_flag(trap_exit, true), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + + {_, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + + MFL = 512, + Data = "reuse_session_erlang_server " ++ lists:duplicate(MFL, $r), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {ssl_test_lib, active_recv, [length(Data)]}}, + {reconnect_times, 5}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + Version = ssl_test_lib:protocol_version(Config), + + Exe = "openssl", + Args = ["s_client", "-connect", ssl_test_lib:hostname_format(Hostname) + ++ ":" ++ integer_to_list(Port), + "-tlsextdebug", "-4", "-maxfraglen", integer_to_list(MFL), + ssl_test_lib:version_flag(Version), + "-reconnect"], + + OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), + + run_mfl_openssl(Server, OpensslPort, MFL, Data), + + %% Clean close down! Server needs to be closed first !! + ssl_test_lib:close(Server), + ssl_test_lib:close_port(OpensslPort). + +%%-------------------------------------------------------------------- + +reuse_session_erlang_client(Config) when is_list(Config) -> + process_flag(trap_exit, true), + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_opts, Config), + ServerOpts = ssl_test_lib:ssl_options(server_rsa_opts, Config), + {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), + + MFL = 512, + Data = "reuse_session_erlang_client " ++ lists:duplicate(MFL, $r), + ClientOpts = [{max_fragment_length, 512} | ClientOpts0], + + Version = ssl_test_lib:protocol_version(Config), + Port = ssl_test_lib:inet_port(node()), + CertFile = proplists:get_value(certfile, ServerOpts), + CACertFile = proplists:get_value(cacertfile, ServerOpts), + KeyFile = proplists:get_value(keyfile, ServerOpts), + + Exe = "openssl", + Args = ["s_server", "-accept", integer_to_list(Port), ssl_test_lib:version_flag(Version), + "-tlsextdebug", "-cert", CertFile,"-key", KeyFile, "-CAfile", CACertFile], + + OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), + + OpensslProtocol = case proplists:get_value(protocol, Config) of + undefined -> + tls; + ConfigProtocol -> + ConfigProtocol + end, + + ssl_test_lib:wait_for_openssl_server(Port, OpensslProtocol), + + Client0 = + ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, + {options, [{reuse_sessions, save}, {verify, verify_peer}| ClientOpts]}]), + + SID = receive + {Client0, Id0} -> + Id0 + end, + + %% quit s_server's current session so we can interact with the next client + true = port_command(OpensslPort, "q\n"), + ssl_test_lib:close(Client0), + + flush(), + + Client1 = + ssl_test_lib:start_client([{node, ClientNode}, + {port, Port}, {host, Hostname}, + {mfa, {ssl_test_lib, session_id, []}}, + {from, self()}, {options, [{reuse_session, SID} | ClientOpts]}]), + receive + {Client1, SID} -> + ok + after ?SLEEP -> + ct:fail(session_not_reused) + end, + + ErlRecvFun = fun() -> + Data = ssl_test_lib:check_active_receive(Client1, Data) + end, + run_mfl_openssl(Client1, OpensslPort, MFL, Data, ErlRecvFun), + + %% Clean close down! Server needs to be closed first !! + ssl_test_lib:close_port(OpensslPort), + ssl_test_lib:close(Client1). + +%%-------------------------------------------------------------------- + +handshake_continue(Config) when is_list(Config) -> + ok = run_mfl_handshake_continue(Config, 1024), + ok = run_mfl_handshake_continue(Config, undefined), + ok. + +run_mfl_handshake_continue(Config, MFL) -> + Data = if is_integer(MFL) -> + "hello world" ++ lists:duplicate(MFL, $u); + true -> + "hello world" ++ lists:duplicate(999, $x) + end, + ClientExtraOpts = [{handshake, hello}, {max_fragment_length, MFL}], + ServerExtraOpts = [{handshake, hello}], + ExtraStartOpts = [{continue_options, [{want_ext, self()}]}], + MflEnum = mfl_enum(MFL), + PostF = fun(Server, Client) -> + receive {Server, {ext, ServerExt}} -> + ct:log("Server handshake Ext ~p~n", [ServerExt]), + MflEnum = maps:get(max_frag_enum, ServerExt, undefined) + end, + receive {Client, {ext, ClientExt}} -> + ct:log("Client handshake Ext ~p~n", [ClientExt]), + case maps:get(server_hello_selected_version, ClientExt, undefined) of + {3,4} -> + %% For TLS 1.3 the ssl {handshake, hello} API is inconsistent: + %% the server gets all the extensions CH+EE, but the client only CH + ignore; + _ -> + MflEnum = maps:get(max_frag_enum, ClientExt, undefined) + end + end, + ok + end, + + run_mfl_handshake(Config, MFL, Data, ClientExtraOpts, ServerExtraOpts, ExtraStartOpts, ExtraStartOpts, PostF). + +%%-------------------------------------------------------------------- +%% Internal functions ------------------------------------------------ +%%-------------------------------------------------------------------- +run_mfl_handshake(Config, MFL) when is_integer(MFL) -> + Data = "hello world" ++ lists:duplicate(MFL, $0), + ClientExtraOpts = [{max_fragment_length, MFL}], + run_mfl_handshake(Config, MFL, Data, ClientExtraOpts, []); +run_mfl_handshake(Config, MFL) -> + Data = "hello world" ++ lists:duplicate(512, $9), + ClientExtraOpts = [], + run_mfl_handshake(Config, MFL, Data, ClientExtraOpts, []). + +run_mfl_handshake(Config, MFL, Data, ClientExtraOpts, ServerExtraOpts) -> + run_mfl_handshake(Config, MFL, Data, ClientExtraOpts, ServerExtraOpts, [], [], fun(_,_) -> ok end). + +run_mfl_handshake(Config, MFL, Data, ClientExtraOpts, ServerExtraOpts, ClientExtraStartOpts, ServerExtraStartOpts, + PostFun) -> + ClientOpts0 = ssl_test_lib:ssl_options(client_rsa_verify_opts, Config), + ClientOpts = ClientExtraOpts ++ ClientOpts0, + ServerOpts0 = ssl_test_lib:ssl_options(server_rsa_opts, Config), + ServerOpts = ServerExtraOpts ++ ServerOpts0, + + {ClientNode, ServerNode, Hostname} = ssl_test_lib:run_where(Config), + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, assert_mfl_and_send_first, [MFL, Data]}}, + {options, ServerOpts} | ServerExtraStartOpts]), + + Port = ssl_test_lib:inet_port(Server), + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, assert_mfl_and_recv_first, [MFL, Data]}}, + {options, ClientOpts} | ClientExtraStartOpts]), + + ok = PostFun(Server, Client), + + ssl_test_lib:check_result(Server, ok, Client, ok). + +assert_mfl(Socket, undefined) -> + InfoMFL = ssl:connection_information(Socket, [max_fragment_length]), + ct:log("Connection MFL ~p, Expecting: [] ~n", [InfoMFL]), + {ok, []} = InfoMFL; +assert_mfl(Socket, MFL) -> + InfoMFL = ssl:connection_information(Socket, [max_fragment_length]), + ct:log("Connection MFL ~p, Expecting: ~p ~n", [InfoMFL, MFL]), + {ok, [{max_fragment_length, ConnMFL}]} = InfoMFL, + ConnMFL = MFL. + +assert_mfl_and_send_first(Socket, MFL, Data) -> + assert_mfl(Socket, MFL), + ssl_send(Socket, Data), + ssl_receive(Socket, "Got it"++lists:reverse(Data)). + +assert_mfl_and_recv_first(Socket, MFL, Data) -> + assert_mfl(Socket, MFL), + ssl_receive(Socket, Data), + ssl_send(Socket, lists:reverse(Data)). + +ssl_send(Socket, Data) -> + ssl:send(Socket, Data). + +ssl_receive(Socket, Data) -> + ssl_receive(Socket, Data, []). + +ssl_receive(Socket, Data, Buffer) -> + receive + {ssl, Socket, MoreData} -> + ct:log("Received ~p~n",[MoreData]), + NewBuffer = Buffer ++ MoreData, + case NewBuffer of + Data -> + ssl:send(Socket, "Got it"), + ok; + _ -> + ssl_receive(Socket, Data, NewBuffer) + end; + Other -> + ct:fail({unexpected_message, Other}) + after 4000 -> + ct:fail({did_not_get, Data}) + end. + +%% ------------------------------------------------------------ +mfl_openssl_server(MFL, Config) -> + Data = "mfl_openssl_server " ++ lists:duplicate(MFL, $s), + Fun = fun(C,S) -> run_mfl_openssl(C, S, MFL, Data) end, + ssl_test_lib:start_erlang_client_and_openssl_server_with_opts(Config, + [{max_fragment_length, MFL}], + ["-tlsextdebug", "-tlsextdebug"], + Data, Fun). + +%% ------------------------------------------------------------ +mfl_openssl_client(MFL, Config) -> + Data = "mfl_openssl_client " ++ lists:duplicate(MFL, $c), + Fun = fun(S,C) -> run_mfl_openssl(S, C, MFL, Data) end, + ClientArgs = ["-tlsextdebug", "-4", "-maxfraglen", integer_to_list(MFL)], + ssl_test_lib:start_erlang_server_and_openssl_client_with_opts(Config, + [], + ClientArgs, + Data, Fun). + +%% ------------------------------------------------------------ +run_mfl_openssl(ErlProc, OpenSSL, MFL, Data) -> + ErlRecvFun = fun() -> + receive + {ErlProc, Data} -> + ok + after 1000 -> + flush(true), + error(timeout) + end + end, + run_mfl_openssl(ErlProc, OpenSSL, MFL, Data, ErlRecvFun). + +run_mfl_openssl(ErlProc, OpenSSL, MFL, Data, ErlRecvFun) -> + MFL = get_openssl_max_fragment_length(OpenSSL), + + true = port_command(OpenSSL, Data), + ErlRecvFun(), + + ErlProc ! get_socket, + ErlSocket = receive + {ErlProc, {socket, ErlSocket0}} -> + ErlSocket0 + end, + assert_mfl(ErlSocket, MFL), + + RData = lists:reverse(Data), + flush(), + ssl:send(ErlSocket, RData), + RData = ssl_test_lib:active_recv(OpenSSL, length(RData)), + ok. + +%% ------------------------------------------------------------ +flush() -> + flush(false). +flush(Noisy) -> + receive Rx -> + if Noisy -> + io:format("~p:~p: ~999p~n", [self(), ?FUNCTION_NAME, Rx]); + true -> + ignore + end, + flush(Noisy) + after 100 -> + ok + end. + +%% ------------------------------------------------------------ +get_openssl_max_fragment_length(Port) -> + get_openssl_max_fragment_length(Port, []). + +get_openssl_max_fragment_length(Port, Acc) -> + receive + {Port, {data, Data}} -> + get_openssl_max_fragment_length_line(Port, Acc++Data) + after 1000 -> + error(timeout) + end. + +get_openssl_max_fragment_length_line(Port, Acc) -> + case get_line(Acc) of + more -> + get_openssl_max_fragment_length(Port, Acc); + {"TLS "++TlsInfo, Acc2} -> + get_openssl_max_fragment_length_tlsinfo(TlsInfo, Port, Acc2); + {_Discard, Acc2} -> + get_openssl_max_fragment_length_line(Port, Acc2) + end. + +get_openssl_max_fragment_length_tlsinfo("client extension "++ExtInfo, Port, Acc) -> + get_openssl_max_fragment_length_ext(ExtInfo, Port, Acc); +get_openssl_max_fragment_length_tlsinfo("server extension "++ExtInfo, Port, Acc) -> + get_openssl_max_fragment_length_ext(ExtInfo, Port, Acc); +get_openssl_max_fragment_length_tlsinfo(_Acc, Port, Acc) -> + get_openssl_max_fragment_length_line(Port, Acc). + +get_openssl_max_fragment_length_ext("\"max fragment length\" (id=1), len=1"=Ext, Port, Acc) -> + case get_line(Acc) of + more -> + receive + {Port, {data, Data}} -> + Acc1 = Acc++Data, + get_openssl_max_fragment_length_ext(Ext, Port, Acc1) + after 1000 -> + error(timeout) + end; + {"0000 - 01 "++_, _} -> + 512; + {"0000 - 02 "++_, _} -> + 1024; + {"0000 - 03 "++_, _} -> + 2048; + {"0000 - 04 "++_, _} -> + 4096 + end; +get_openssl_max_fragment_length_ext(_Acc, Port, Acc2) -> + get_openssl_max_fragment_length_line(Port, Acc2). + + +get_line(Data) -> + get_line(Data, []). + +get_line([$\r|T], A) -> + get_line(T, A); +get_line([$\n|T], A) -> + {lists:reverse(A), T}; +get_line([], _) -> + more; +get_line([H|T], A) -> + get_line(T, [H|A]). + + +get_openssl_data(Port, Exp) -> + get_openssl_data(Port, Exp, []). + +get_openssl_data(_Port, Exp, Exp) -> + ok; +get_openssl_data(Port, Exp, Acc) -> + case lists:prefix(Acc, Exp) of + true -> + receive + {Port, {data, Data}} -> + get_openssl_data(Port, Exp, Acc++Data) + after 1000 -> + error(timeout) + end; + false -> + ct:fail({get_openssl_data, {{expected, Exp}, {got, Acc}}}) + end. + +%% RFC 6066 +mfl_enum(512) -> 1; +mfl_enum(1024) -> 2; +mfl_enum(2048) -> 3; +mfl_enum(4096) -> 4; +mfl_enum(undefined) -> undefined. diff --git a/lib/ssl/test/ssl_npn_SUITE.erl b/lib/ssl/test/ssl_npn_SUITE.erl index 94bd21418c..b3c93c19fb 100644 --- a/lib/ssl/test/ssl_npn_SUITE.erl +++ b/lib/ssl/test/ssl_npn_SUITE.erl @@ -24,7 +24,7 @@ %% Note: This directive should only be used in test suites. -compile(export_all). -include_lib("common_test/include/ct.hrl"). - +-define(TIMEOUT, {seconds, 5}). -define(SLEEP, 500). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -100,7 +100,7 @@ end_per_group(GroupName, Config) -> init_per_testcase(_TestCase, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), ct:log("Ciphers: ~p~n ", [ ssl:cipher_suites()]), - ct:timetrap({seconds, 10}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(_TestCase, Config) -> diff --git a/lib/ssl/test/ssl_packet_SUITE.erl b/lib/ssl/test/ssl_packet_SUITE.erl index cc265e5849..2c6f169fd0 100644 --- a/lib/ssl/test/ssl_packet_SUITE.erl +++ b/lib/ssl/test/ssl_packet_SUITE.erl @@ -41,7 +41,7 @@ -define(MANY, 1000). -define(SOME, 50). --define(BASE_TIMEOUT_SECONDS, 5). +-define(BASE_TIMEOUT_SECONDS, 20). -define(SOME_SCALE, 2). -define(MANY_SCALE, 3). @@ -2148,7 +2148,7 @@ server_packet_decode(Socket, Packet) -> {ssl, Socket, Packet} -> ok; Other1 -> exit({?LINE, Other1}) end, - ok = ssl:send(Socket, Packet), + spawn(fun() -> ssl:send(Socket, Packet) end), receive {ssl, Socket, Packet} -> ok; Other2 -> exit({?LINE, Other2}) diff --git a/lib/ssl/test/ssl_payload_SUITE.erl b/lib/ssl/test/ssl_payload_SUITE.erl index f771ec8cf9..6b3df7ec3e 100644 --- a/lib/ssl/test/ssl_payload_SUITE.erl +++ b/lib/ssl/test/ssl_payload_SUITE.erl @@ -24,9 +24,8 @@ -compile(export_all). -include_lib("common_test/include/ct.hrl"). - --define(TIMEOUT, 600000). - +-define(TIMEOUT, {seconds, 20}). +-define(TIMEOUT_LONG, {seconds, 80}). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -127,7 +126,7 @@ init_per_testcase(TestCase, Config) "sparc-sun-solaris2.10" -> {skip,"Will take to long time on an old Sparc"}; _ -> - ct:timetrap({seconds, 90}), + ct:timetrap(?TIMEOUT_LONG), Config end; @@ -140,11 +139,11 @@ init_per_testcase(TestCase, Config) TestCase == client_echos_passive_chunk_big; TestCase == client_echos_active_once_big; TestCase == client_echos_active_big -> - ct:timetrap({seconds, 60}), + ct:timetrap(?TIMEOUT_LONG), Config; init_per_testcase(_TestCase, Config) -> - ct:timetrap({seconds, 15}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(_TestCase, Config) -> @@ -726,7 +725,7 @@ client_active_once_server_close( send(_Socket, _Data, 0, _) -> ok; send(Socket, Data, Count, RecvEcho) -> - ok = ssl:send(Socket, Data), + spawn(fun() -> ssl:send(Socket, Data) end), RecvEcho(), send(Socket, Data, Count - 1, RecvEcho). diff --git a/lib/ssl/test/ssl_session_SUITE.erl b/lib/ssl/test/ssl_session_SUITE.erl index db51b5a0a0..60e71501fa 100644 --- a/lib/ssl/test/ssl_session_SUITE.erl +++ b/lib/ssl/test/ssl_session_SUITE.erl @@ -552,4 +552,4 @@ connection_states(Random) -> client_random = Random, server_random = undefined, exportable = undefined}, - sequence_number => 0,server_verify_data => undefined}}. + sequence_number => 0,server_verify_data => undefined,max_fragment_length => undefined}}. diff --git a/lib/ssl/test/ssl_session_cache_SUITE.erl b/lib/ssl/test/ssl_session_cache_SUITE.erl index 1bafb5f4d0..f6b527aaf9 100644 --- a/lib/ssl/test/ssl_session_cache_SUITE.erl +++ b/lib/ssl/test/ssl_session_cache_SUITE.erl @@ -28,8 +28,7 @@ -include_lib("common_test/include/ct.hrl"). -define(SLEEP, 1000). --define(TIMEOUT, 60000). --define(LONG_TIMEOUT, 600000). +-define(TIMEOUT, {seconds, 20}). -define(MAX_TABLE_SIZE, 5). -behaviour(ssl_session_cache_api). @@ -123,18 +122,18 @@ init_per_testcase(session_cleanup, Config) -> application:set_env(ssl, session_lifetime, 5), ssl:start(), ssl_test_lib:ct_log_supported_protocol_versions(Config), - ct:timetrap({seconds, 20}), + ct:timetrap(?TIMEOUT), Config; init_per_testcase(client_unique_session, Config) -> - ct:timetrap({seconds, 40}), + ct:timetrap(?TIMEOUT), ssl_test_lib:ct_log_supported_protocol_versions(Config), Config; init_per_testcase(save_specific_session, Config) -> Versions = ssl_test_lib:protocol_version(Config), ssl_test_lib:clean_start(), ssl_test_lib:set_protocol_versions(Versions), - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; init_per_testcase(max_table_size, Config) -> Versions = ssl_test_lib:protocol_version(Config), @@ -161,7 +160,7 @@ init_customized_session_cache(Type, Config) -> Config)), ets:new(ssl_test, [named_table, public, set]), ets:insert(ssl_test, {type, Type}), - ct:timetrap({seconds, 20}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(session_cache_process_list, Config) -> @@ -240,16 +239,13 @@ session_cleanup(Config) when is_list(Config) -> ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, {host, Hostname}, {mfa, {ssl_test_lib, no_result, []}}, - {from, self()}, {options, ClientOpts}]), + {from, self()}, {options, [{reuse_sessions, save} | ClientOpts]}]), SessionInfo = receive {Server, Info} -> Info end, - %% Make sure session is registered - ct:sleep(?SLEEP*2), - {status, _, _, StatusInfo} = sys:get_status(whereis(ssl_manager)), [_, _,_, _, Prop] = StatusInfo, State = ssl_test_lib:state(Prop), diff --git a/lib/ssl/test/ssl_sni_SUITE.erl b/lib/ssl/test/ssl_sni_SUITE.erl index f14ea5fae3..4660a98ca5 100644 --- a/lib/ssl/test/ssl_sni_SUITE.erl +++ b/lib/ssl/test/ssl_sni_SUITE.erl @@ -27,13 +27,15 @@ -include_lib("public_key/include/public_key.hrl"). -include_lib("kernel/include/inet.hrl"). +-define(TIMEOUT, {seconds, 6}). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- %%-------------------------------------------------------------------- all() -> - [{group, 'tlsv1.2'}, + [{group, 'tlsv1.3'}, + {group, 'tlsv1.2'}, {group, 'tlsv1.1'}, {group, 'tlsv1'}, {group, 'dtlsv1.2'}, @@ -42,6 +44,7 @@ all() -> groups() -> [ + {'tlsv1.3', [], sni_tests()}, {'tlsv1.2', [], sni_tests()}, {'tlsv1.1', [], sni_tests()}, {'tlsv1', [], sni_tests()}, @@ -85,26 +88,10 @@ init_per_suite(Config0) -> {skip, "Crypto did not start"} end. init_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - case ssl_test_lib:sufficient_crypto_support(GroupName) of - true -> - ssl_test_lib:init_tls_version(GroupName, Config); - false -> - {skip, "Missing crypto support"} - end; - _ -> - ssl:start(), - Config - end. + ssl_test_lib:init_per_group(GroupName, Config). end_per_group(GroupName, Config) -> - case ssl_test_lib:is_tls_version(GroupName) of - true -> - ssl_test_lib:clean_tls_version(Config); - false -> - Config - end. + ssl_test_lib:end_per_group(GroupName, Config). end_per_suite(_) -> ssl:stop(), @@ -113,12 +100,12 @@ end_per_suite(_) -> init_per_testcase(customize_hostname_check, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), ssl_test_lib:clean_start(), - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config; init_per_testcase(_TestCase, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), ct:log("Ciphers: ~p~n ", [ ssl:cipher_suites()]), - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(_TestCase, Config) -> diff --git a/lib/ssl/test/ssl_socket_SUITE.erl b/lib/ssl/test/ssl_socket_SUITE.erl index d648f2f9e1..47d3a72988 100644 --- a/lib/ssl/test/ssl_socket_SUITE.erl +++ b/lib/ssl/test/ssl_socket_SUITE.erl @@ -25,7 +25,7 @@ -include_lib("common_test/include/ct.hrl"). -include_lib("public_key/include/public_key.hrl"). - +-define(TIMEOUT, {seconds, 5}). -define(SLEEP, 500). %%-------------------------------------------------------------------- %% Common Test interface functions ----------------------------------- @@ -87,7 +87,7 @@ end_per_group(_GroupName, Config) -> Config. init_per_testcase(raw_inet_option, Config) -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), case os:type() of {unix,linux} -> Config; @@ -95,7 +95,7 @@ init_per_testcase(raw_inet_option, Config) -> {skip, "Raw options are platform-specific"} end; init_per_testcase(_TestCase, Config) -> - ct:timetrap({seconds, 5}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(_TestCase, Config) -> diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 4cacc91a04..d93338be15 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -95,7 +95,9 @@ init_per_group(GroupName, Config) -> init_per_group_openssl(GroupName, Config) -> case is_tls_version(GroupName) andalso sufficient_crypto_support(GroupName) of true -> - case check_sane_openssl_version(GroupName) of + case check_sane_openssl_version(GroupName) + andalso maybe_legacy_tls_version_support(GroupName, Config) + of true -> [{version, GroupName}|init_tls_version(GroupName, Config)]; false -> @@ -264,6 +266,9 @@ do_run_server_core(ListenSocket, AcceptSocket, Opts, Transport, Pid) -> Pid ! {self(), Reason} end, do_run_server_core(ListenSocket, AcceptSocket, Opts, Transport, Pid); + get_socket -> + Pid ! {self(), {socket, AcceptSocket}}, + do_run_server_core(ListenSocket, AcceptSocket, Opts, Transport, Pid); listen -> run_server(ListenSocket, Opts); {listen, MFA} -> @@ -311,7 +316,7 @@ connect(ListenSocket, Node, _N, _, Timeout, SslOpts, cancel) -> ct:log("~p:~p~nssl:handshake@~p ret ~p",[?MODULE,?LINE, Node,Result]), Result end; -connect(ListenSocket, Node, N, _, Timeout, SslOpts, [_|_] =ContOpts) -> +connect(ListenSocket, Node, N, _, Timeout, SslOpts, [_|_] =ContOpts0) -> ct:log("ssl:transport_accept(~p)~n", [ListenSocket]), {ok, AcceptSocket} = ssl:transport_accept(ListenSocket), ct:log("~p:~p~nssl:handshake(~p,~p,~p)~n", [?MODULE,?LINE, AcceptSocket, SslOpts,Timeout]), @@ -320,10 +325,21 @@ connect(ListenSocket, Node, N, _, Timeout, SslOpts, [_|_] =ContOpts) -> {ok, Socket0, Ext} -> [_|_] = maps:get(sni, Ext), ct:log("Ext ~p:~n", [Ext]), - ct:log("~p:~p~nssl:handshake_continue(~p,~p,~p)~n", [?MODULE,?LINE, Socket0, ContOpts,Timeout]), + ContOpts = case lists:keytake(want_ext, 1, ContOpts0) of + {value, {_, WantExt}, ContOpts1} -> + if is_pid(WantExt) -> + WantExt ! {self(), {ext, Ext}}; + true -> + ignore + end, + ContOpts1; + _ -> + ContOpts0 + end, + ct:log("~p:~p~nssl:handshake_continue(~p,~p,~p)~n", [?MODULE,?LINE, Socket0, ContOpts,Timeout]), case ssl:handshake_continue(Socket0, ContOpts, Timeout) of {ok, Socket} -> - connect(ListenSocket, Node, N-1, Socket, Timeout, SslOpts, ContOpts); + connect(ListenSocket, Node, N-1, Socket, Timeout, SslOpts, ContOpts0); Error -> ct:log("~p:~p~nssl:handshake_continue@~p ret ~p",[?MODULE,?LINE, Node,Error]), Error @@ -376,8 +392,6 @@ start_erlang_client_and_openssl_server_with_opts(Config, ErlangClientOpts, Opens {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), - Data = "From openssl to erlang", - Port = ssl_test_lib:inet_port(node()), CaCertFile = proplists:get_value(cacertfile, ServerOpts), CertFile = proplists:get_value(certfile, ServerOpts), @@ -815,6 +829,9 @@ client_loop_core(Socket, Pid, Transport) -> Pid ! {self(), Reason} end, client_loop_core(Socket, Pid, Transport); + get_socket -> + Pid ! {self(), {socket, Socket}}, + client_loop_core(Socket, Pid, Transport); close -> ct:log("~p:~p~nClient closing~n", [?MODULE,?LINE]), Transport:close(Socket); @@ -835,9 +852,20 @@ client_cont_loop(_Node, Host, Port, Pid, Transport, Options, cancel, _Opts) -> Pid ! {connect_failed, Reason} end; -client_cont_loop(_Node, Host, Port, Pid, Transport, Options, ContOpts, Opts) -> +client_cont_loop(_Node, Host, Port, Pid, Transport, Options, ContOpts0, Opts) -> case Transport:connect(Host, Port, Options) of - {ok, Socket0, _} -> + {ok, Socket0, Ext} -> + ContOpts = case lists:keytake(want_ext, 1, ContOpts0) of + {value, {_, WantExt}, ContOpts1} -> + if is_pid(WantExt) -> + WantExt ! {self(), {ext, Ext}}; + true -> + ignore + end, + ContOpts1; + _ -> + ContOpts0 + end, ct:log("~p:~p~nClient: handshake_continue(~p, ~p, infinity) ~n", [?MODULE, ?LINE, Socket0, ContOpts]), case Transport:handshake_continue(Socket0, ContOpts) of {ok, Socket} -> @@ -1211,6 +1239,18 @@ default_cert_chain_conf() -> %% Use only default options [[],[],[]]. +make_rsa_pss_pem(Alg, _UserConf, Config, Suffix) -> + DefClientConf = chain_spec(client, Alg, []), + DefServerConf = chain_spec(server, Alg, []), + CertChainConf = new_format([{client_chain, DefClientConf}, {server_chain, DefServerConf}]), + ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(Alg) ++ Suffix]), + ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), atom_to_list(Alg) ++ Suffix]), + GenCertData = public_key:pkix_test_data(CertChainConf), + Conf = x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), + CConf = proplists:get_value(client_config, Conf), + SConf = proplists:get_value(server_config, Conf), + #{server_config => SConf, + client_config => CConf}. gen_conf(ClientChainType, ServerChainType, UserClient, UserServer) -> gen_conf(ClientChainType, ServerChainType, UserClient, UserServer, ?DEFAULT_CURVE). @@ -1287,8 +1327,33 @@ chain_spec(_Role, ecdsa, Curve) -> chain_spec(_Role, rsa, _) -> Digest = {digest, appropriate_sha(crypto:supports())}, [[Digest, {key, hardcode_rsa_key(1)}], - [Digest, {key, hardcode_rsa_key(2)}], - [Digest, {key, hardcode_rsa_key(3)}]]; + [Digest, {key, hardcode_rsa_key(2)}], + [Digest, {key, hardcode_rsa_key(3)}]]; +chain_spec(_Role, 'rsa-1024', _) -> + Digest = {digest, appropriate_sha(crypto:supports())}, + [[Digest, {key, hardcode_rsa_1024_key(1)}], + [Digest, {key, hardcode_rsa_1024_key(2)}], + [Digest, {key, hardcode_rsa_1024_key(3)}]]; +chain_spec(client, rsa_pss_rsae, _) -> + Digest = {digest, sha256}, + [[Digest, {rsa_padding, rsa_pss_rsae}, {key, hardcode_rsa_key(1)}], + [Digest, {rsa_padding, rsa_pss_rsae}, {key, hardcode_rsa_key(2)}], + [Digest, {rsa_padding, rsa_pss_rsae}, {key, hardcode_rsa_key(3)}]]; +chain_spec(server, rsa_pss_rsae, _) -> + Digest = {digest, sha256}, + [[Digest, {rsa_padding, rsa_pss_rsae}, {key, hardcode_rsa_key(4)}], + [Digest, {rsa_padding, rsa_pss_rsae}, {key, hardcode_rsa_key(5)}], + [Digest, {rsa_padding, rsa_pss_rsae}, {key, hardcode_rsa_key(6)}]]; +chain_spec(client, rsa_pss_pss, _) -> + Digest = {digest, sha256}, + [[Digest, {rsa_padding, rsa_pss_pss}, {key, {hardcode_rsa_key(1), pss_params(sha256)}}], + [Digest, {rsa_padding, rsa_pss_pss}, {key, {hardcode_rsa_key(2), pss_params(sha256)}}], + [Digest, {rsa_padding, rsa_pss_pss}, {key, {hardcode_rsa_key(3), pss_params(sha256)}}]]; +chain_spec(server, rsa_pss_pss, _) -> + Digest = {digest, sha256}, + [[Digest, {rsa_padding, rsa_pss_pss}, {key, {hardcode_rsa_key(4), pss_params(sha256)}}], + [Digest, {rsa_padding, rsa_pss_pss}, {key, {hardcode_rsa_key(5), pss_params(sha256)}}], + [Digest, {rsa_padding, rsa_pss_pss}, {key, {hardcode_rsa_key(6), pss_params(sha256)}}]]; chain_spec(_Role, dsa, _) -> Digest = {digest, appropriate_sha(crypto:supports())}, [[Digest, {key, hardcode_dsa_key(1)}], @@ -1404,6 +1469,31 @@ make_rsa_cert(Config) -> false -> Config end. + +make_rsa_1024_cert(Config) -> + CryptoSupport = crypto:supports(), + case proplists:get_bool(rsa, proplists:get_value(public_keys, CryptoSupport)) of + true -> + ClientFileBase = filename:join([proplists:get_value(priv_dir, Config), "rsa-1024"]), + ServerFileBase = filename:join([proplists:get_value(priv_dir, Config), "rsa-1024"]), + ClientChain = proplists:get_value(client_chain, Config, default_cert_chain_conf()), + ServerChain = proplists:get_value(server_chain, Config, default_cert_chain_conf()), + CertChainConf = gen_conf('rsa-1024', 'rsa-1024', ClientChain, ServerChain), + GenCertData = public_key:pkix_test_data(CertChainConf), + [{server_config, ServerConf}, + {client_config, ClientConf}] = + x509_test:gen_pem_config_files(GenCertData, ClientFileBase, ServerFileBase), + [{server_rsa_1024_opts, [{ssl_imp, new},{reuseaddr, true} | ServerConf]}, + + {server_rsa_1024_verify_opts, [{ssl_imp, new}, {reuseaddr, true}, + {verify, verify_peer} | ServerConf]}, + {client_rsa_1024_opts, ClientConf}, + {client_rsa_1024_verify_opts, [{verify, verify_peer} |ClientConf]} + | Config]; + false -> + Config + end. + appropriate_sha(CryptoSupport) -> Hashes = proplists:get_value(hashs, CryptoSupport), case lists:member(sha256, Hashes) of @@ -1696,51 +1786,31 @@ ecc_test_error(COpts, SOpts, CECCOpts, SECCOpts, Config) -> Client = start_client_ecc_error(erlang, Port, COpts, CECCOpts, Config), check_server_alert(Server, Client, insufficient_security). -start_basic_client(openssl, Version, Port, ClientOpts) -> - Cert = proplists:get_value(certfile, ClientOpts), - Key = proplists:get_value(keyfile, ClientOpts), - CA = proplists:get_value(cacertfile, ClientOpts), - Groups0 = proplists:get_value(groups, ClientOpts), - Exe = "openssl", - Args0 = ["s_client", "-verify", "2", "-port", integer_to_list(Port), - ssl_test_lib:version_flag(Version), - "-CAfile", CA, "-host", "localhost", "-msg", "-debug"], - Args1 = - case Groups0 of - undefined -> - Args0; - G -> - Args0 ++ ["-groups", G] - end, - Args = - case {Cert, Key} of - {C, K} when C =:= undefined orelse - K =:= undefined -> - Args1; - {C, K} -> - Args1 ++ ["-cert", C, "-key", K] - end, - - OpenSslPort = ssl_test_lib:portable_open_port(Exe, Args), - true = port_command(OpenSslPort, "Hello world"), - OpenSslPort. - start_client(openssl, Port, ClientOpts, Config) -> Version = ssl_test_lib:protocol_version(Config), Exe = "openssl", Ciphers = proplists:get_value(ciphers, ClientOpts, ssl:cipher_suites(default,Version)), Groups0 = proplists:get_value(groups, ClientOpts), CertArgs = openssl_cert_options(ClientOpts, client), + Exe = "openssl", Args0 = case Groups0 of undefined -> - ["s_client", "-verify", "2", "-port", integer_to_list(Port), cipher_flag(Version), + ["s_client", + "-verify", "2", + "-port", integer_to_list(Port), cipher_flag(Version), ciphers(Ciphers, Version), - ssl_test_lib:version_flag(Version)] ++ CertArgs ++ ["-msg", "-debug"]; + ssl_test_lib:version_flag(Version)] + ++ CertArgs + ++ ["-msg", "-debug"]; Group -> - ["s_client", "-verify", "2", "-port", integer_to_list(Port), cipher_flag(Version), + ["s_client", + "-verify", "2", + "-port", integer_to_list(Port), cipher_flag(Version), ciphers(Ciphers, Version), "-groups", Group, - ssl_test_lib:version_flag(Version)] ++ CertArgs ++ ["-msg", "-debug"] + ssl_test_lib:version_flag(Version)] + ++CertArgs + ++ ["-msg", "-debug"] end, Args = maybe_force_ipv4(Args0), OpenSslPort = ssl_test_lib:portable_open_port(Exe, Args), @@ -1796,15 +1866,16 @@ start_server(openssl, ClientOpts, ServerOpts, Config) -> CertArgs = openssl_cert_options(ServerOpts, server), Ciphers = proplists:get_value(ciphers, ClientOpts, ssl:cipher_suites(default,Version)), Groups0 = proplists:get_value(groups, ServerOpts), + SigAlgs = proplists:get_value(openssl_sigalgs, Config, undefined), Args = case Groups0 of undefined -> ["s_server", "-accept", integer_to_list(Port), cipher_flag(Version), ciphers(Ciphers, Version), - ssl_test_lib:version_flag(Version)] ++ CertArgs ++ ["-msg", "-debug"]; + ssl_test_lib:version_flag(Version)] ++ sig_algs(SigAlgs) ++ CertArgs ++ ["-msg", "-debug"]; Group -> ["s_server", "-accept", integer_to_list(Port), cipher_flag(Version), ciphers(Ciphers, Version), "-groups", Group, - ssl_test_lib:version_flag(Version)] ++ CertArgs ++ ["-msg", "-debug"] + ssl_test_lib:version_flag(Version)] ++ sig_algs(SigAlgs) ++ CertArgs ++ ["-msg", "-debug"] end, OpenSslPort = portable_open_port(Exe, Args), true = port_command(OpenSslPort, "Hello world"), @@ -1821,6 +1892,11 @@ start_server(erlang, _, ServerOpts, Config) -> {options, [{verify, verify_peer}, {versions, Versions} | ServerOpts]}]), {Server, inet_port(Server)}. +sig_algs(undefined) -> + []; +sig_algs(SigAlgs) -> + ["-sigalgs " ++ SigAlgs]. + cipher_flag('tlsv1.3') -> "-ciphersuites"; cipher_flag(_) -> @@ -2266,6 +2342,39 @@ is_dtls_version('dtlsv1') -> is_dtls_version(_) -> false. +maybe_legacy_tls_version_support(Version, Config0) when + Version == 'tlsv1'; + Version == 'tlsv1.1' -> + %% Check if legacy version is supported + Config = ssl_test_lib:make_rsa_cert(Config0), + ServerOpts = proplists:get_value(server_rsa_opts, Config), + Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), + CertFile = proplists:get_value(certfile, ServerOpts), + KeyFile = proplists:get_value(keyfile, ServerOpts), + Exe = "openssl", + Args = ["s_server", "-accept", + integer_to_list(Port), "-CAfile", CaCertFile, + "-cert", CertFile,"-key", KeyFile], + + OpensslPort = ssl_test_lib:portable_open_port(Exe, Args), + ssl_test_lib:wait_for_openssl_server(Port, tls), + + case ssl:connect("localhost", Port, [{versions, [Version]}]) of + {ok, Socket} -> + ssl:close(Socket), + close_port(OpensslPort), + true; + {error, {tls_alert, {protocol_version, _}}} -> + close_port(OpensslPort), + false + end; +maybe_legacy_tls_version_support('dtlsv1', Config) -> + maybe_legacy_tls_version_support('tlsv1.1', Config); +maybe_legacy_tls_version_support(_, _) -> + %% Not a legacy version + true. + init_tls_version(Version, Config) when Version == 'dtlsv1.2'; Version == 'dtlsv1' -> ssl:stop(), @@ -2470,7 +2579,7 @@ active_recv(Socket, N, Acc) -> filter_openssl_debug_data(Bytes) -> re:replace(Bytes, "(read.*\n|write to.*\n|[\\dabcdefABCDEF]{4,4} -.*\n|>>> .*\n|<<< .*\n| \\d\\d.*\n|KEYUPDATE\n|.*Read BLOCK\n)*", - "", [{return, list}]). + "", [global,{return, list}]). active_once_recv(_Socket, 0) -> ok; @@ -2561,6 +2670,21 @@ is_sane_oppenssl_client() -> true end. +is_sane_oppenssl_pss(rsa_pss_pss) -> + case portable_cmd("openssl",["version"]) of + "OpenSSL 1.1.1" ++ Rest -> + hd(Rest) >= $c; + _ -> + false + end; +is_sane_oppenssl_pss(rsa_pss_rsae) -> + case portable_cmd("openssl",["version"]) of + "OpenSSL 1.1.1" ++ _ -> + true; + _ -> + false + end. + is_fips(openssl) -> VersionStr = portable_cmd("openssl",["version"]), case re:split(VersionStr, "fips") of @@ -2910,10 +3034,10 @@ supports_ssl_tls_version(Version) when Version == sslv2; case ubuntu_legacy_support() of true -> case portable_cmd("openssl", ["version"]) of - "OpenSSL 1.1" ++ _ -> - false; - "OpenSSL 1" ++ _ -> + "OpenSSL 1.0.1" ++ _ -> Version =/= sslv2; + "OpenSSL 1" ++ _ -> + false; %% Appears to be broken "OpenSSL 0.9.8.o" ++ _ -> false; @@ -3149,6 +3273,41 @@ hardcode_rsa_key(6) -> coefficient = 81173034184183681160439870161505779100040258708276674532866007896310418779840630960490793104541748007902477778658270784073595697910785917474138815202903114440800310078464142273778315781957021015333260021813037604142367434117205299831740956310682461174553260184078272196958146289378701001596552915990080834227, otherPrimeInfos = asn1_NOVALUE}. +hardcode_rsa_1024_key(1) -> + #'RSAPrivateKey'{version = 'two-prime', + modulus = 152618920709346576506952607098028299458615405194120516804067302859774798720862572082626851690572130284910454988859007980367926204341637028795420927026111160369130942718840998292351565050537705794496742217762844103737634290634532232714374862322877076125650783658974242305324207239909234718160759907957502819181, + publicExponent = 17, + privateExponent = 89775835711380339121736827704722529093303179525953245178863119329279293365213277695662853935630664873476738228740592929628191884906845311056129957074183020957315463095429495020547731127789657232144654051871515007759243605000909583210051114028049068215595185959728886310943042856399988846590947179831354428913, + prime1 = 13018105310773689694711101111767176661493882304979760063552973933059514785910240943852845097923711145970844208861778343060919395218474310542285865516544653, + prime2 = 11723589344682921162046319310366118627005968525349821205037572987102618200031016344115630095736447992996683226273798377973464634035204645607416378683745377, + exponent1 = 7657709006337464526300647712804221565584636649988094155031161137093832227006024084619320645837477144688731887565751966506423173657926065024874038539143913, + exponent2 = 11033966442054514034867124056815170472476205670917478781211833399625993600029191853285298913634303993408643036492986708680907890856663195865803650525878001, + coefficient = 7357357483264399363785138527396251818499941660605442417644885251395376792981387533016821796011101917057636813775613592220898054882923958484000235934554630, + otherPrimeInfos = asn1_NOVALUE}; +hardcode_rsa_1024_key(2) -> + #'RSAPrivateKey'{version = 'two-prime', + modulus = 132417984874904377472239436563253515498607309816574784497785056464473431603604973287322746340055062696030016903830406088140534281534301418467490242269156926775506208514027213826501153438861284871625076651352798208559277520683414148048437439635357639033850360133068980157555507518934285770383924814915583919331, + publicExponent = 17, + privateExponent = 116839398419033274240211267555811925439947626308742456909810343939241263179651447018225952652989761202379426679850358313065177307236148310412491390237491385620149263549211570156731410125598364338974865883306073709062002620705336269289633237348474049621806833904576124689232282666798030505410189805859996211233, + prime1 = 12354286715326546399270830019697416039683060665495490476376955068446562229853736822465010796530936501225722243114286822522048306078247961653481711526701259, + prime2 = 10718383661165041035044708868932433765604392896488115438294667272770655136522450030638962957185192634722652306257889603065114923772949624056219896061512009, + exponent1 = 5087059235722695576170341772816583075163613215204025490272863851713290329939773985720886798571562088740003276576471044567902243679278572445551292981582871, + exponent2 = 6304931565391200608849828746430843391531995821463597316643921925159208903836735312140566445403054491013324886034052707685361719866440955327188174153830593, + coefficient = 6764088858264512915296172980190092445938774052616013205418164952211827027745702759906572599388571087295432259160097016323193144471211837074613329649320009, + otherPrimeInfos = asn1_NOVALUE}; +hardcode_rsa_1024_key(3) -> + #'RSAPrivateKey'{version = 'two-prime', + modulus = 132603582566987335579015215397416921308461253540735107996254563101087328483405996961761145905021132317760270172654141110354018131670337351296871719192630978670273323069438897632586026697023844069174787494970866246368200405578784055149230641370998125414763230872277095376893138420738507940599560410343688278361, + publicExponent = 17, + privateExponent = 124803371827752786427308438021098278878551768038338925172945471153964544454970350081657549087078712769656724868380368103862605300395611624749996912181299722918452562565562892031863847812293655197586374503957768862684015202213024002730410420619423541154205461118764880018581745374583581669240937327152309672753, + prime1 = 12202483472094988277172439292742673588688995751099198683383744575043357099902468606144011463115716181768777309695574163698153032647393450605174909802187971, + prime2 = 10866934003248540047676291395653788246732743513485317053446021859209870346149779563425451397497222238159656279714782986335807210805023580459325334557063091, + exponent1 = 3588965727086761257991893909630198114320292867970352553936395463248046205853667237101179842092857700520228620498698283440633244896292191354463208765349403, + exponent2 = 4474619883690575313749061162916265748654659093788071727889538412615828966061673937881068222498856215712799644588440053197097086802068533130310431876437743, + coefficient = 6440880395775803235356940314241907933534073137546236980469653455119937607298142560546736915150573386382326185901566797818281064505978928392351326571984856, + otherPrimeInfos = asn1_NOVALUE}. + + hardcode_dsa_key(1) -> {'DSAPrivateKey',0, 99438313664986922963487511141216248076486724382260996073922424025828494981416579966171753999204426907349400798052572573634137057487829150578821328280864500098312146772602202702021153757550650696224643730869835650674962433068943942837519621267815961566259265204876799778977478160416743037274938277357237615491, @@ -3432,3 +3591,31 @@ set_protocol_versions(_, undefined) -> ok; set_protocol_versions(AppVar, Value) -> application:set_env(ssl, AppVar, Value). + +openssl_sigalgs(rsa_pss_pss, Config) -> + [{openssl_sigalgs, "rsa_pss_rsae_sha256:rsa_pss_pss_sha256"} | + proplists:delete(openssl_sigalgs, Config)]; +openssl_sigalgs(_, Config) -> + proplists:delete(openssl_sigalgs, Config). + +pss_params(sha256) -> + #'RSASSA-PSS-params'{ + hashAlgorithm = #'HashAlgorithm'{algorithm = ?'id-sha256'}, + maskGenAlgorithm = #'MaskGenAlgorithm'{algorithm = ?'id-mgf1', + parameters = #'HashAlgorithm'{algorithm = ?'id-sha256'} + }, + saltLength = 32, + trailerField = 1}. + +test_ciphers(Kex, Cipher, Version) -> + ssl:filter_cipher_suites( + ssl:cipher_suites(all, Version) ++ ssl:cipher_suites(anonymous, Version), + [{key_exchange, + fun(K) when K == Kex -> true; + (_) -> false + end}, + {cipher, + fun(C) when C == Cipher -> true; + (_) -> false + end}]). + diff --git a/lib/ssl/test/tls_1_3_record_SUITE.erl b/lib/ssl/test/tls_1_3_record_SUITE.erl index fc7572cdfa..8bf1097e55 100644 --- a/lib/ssl/test/tls_1_3_record_SUITE.erl +++ b/lib/ssl/test/tls_1_3_record_SUITE.erl @@ -112,7 +112,7 @@ encode_decode(_Config) -> <<92,24,205,75,244,60,136,212,250,32,214,20,37,3,213,87,61,207, 147,61,168,145,177,118,160,153,33,53,48,108,191,174>>, undefined}, - sequence_number => 0,server_verify_data => undefined}}, + sequence_number => 0,server_verify_data => undefined},max_fragment_length => undefined}, PlainText = [11, <<0,2,175>>, @@ -168,7 +168,8 @@ encode_decode(_Config) -> #{current_write => #{security_parameters => #security_parameters{cipher_suite = ?TLS_NULL_WITH_NULL_NULL}, - sequence_number => 0 + sequence_number => 0, + max_fragment_length => undefined } }, diff --git a/lib/ssl/test/tls_api_SUITE.erl b/lib/ssl/test/tls_api_SUITE.erl index 01de312a81..6804b09687 100644 --- a/lib/ssl/test/tls_api_SUITE.erl +++ b/lib/ssl/test/tls_api_SUITE.erl @@ -29,6 +29,7 @@ -include_lib("ssl/src/ssl_api.hrl"). -include_lib("ssl/src/tls_handshake.hrl"). +-define(TIMEOUT, {seconds, 10}). -define(SLEEP, 500). %%-------------------------------------------------------------------- @@ -115,7 +116,7 @@ end_per_group(GroupName, Config) -> init_per_testcase(_TestCase, Config) -> ssl_test_lib:ct_log_supported_protocol_versions(Config), - ct:timetrap({seconds, 10}), + ct:timetrap(?TIMEOUT), Config. end_per_testcase(_TestCase, Config) -> diff --git a/lib/ssl/test/x509_test.erl b/lib/ssl/test/x509_test.erl index dd774d4c7c..e99c35e249 100644 --- a/lib/ssl/test/x509_test.erl +++ b/lib/ssl/test/x509_test.erl @@ -64,6 +64,8 @@ do_gen_pem_config_files(Config, CertFile, KeyFile, CAFile) -> cert_entry(Cert) -> {'Certificate', Cert, not_encrypted}. +key_entry({'PrivateKeyInfo', DERKeyInfo}) -> + {'PrivateKeyInfo', DERKeyInfo, not_encrypted}; key_entry({'RSAPrivateKey', DERKey}) -> {'RSAPrivateKey', DERKey, not_encrypted}; key_entry({'DSAPrivateKey', DERKey}) -> diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index c85d1fa173..14ba8ed31c 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1 +1 @@ -SSL_VSN = 9.6.1 +SSL_VSN = 10.0 diff --git a/lib/stdlib/doc/src/c.xml b/lib/stdlib/doc/src/c.xml index 50e1dea52b..b481596379 100644 --- a/lib/stdlib/doc/src/c.xml +++ b/lib/stdlib/doc/src/c.xml @@ -120,7 +120,7 @@ </func> <func> - <name name="h" arity="1" since="OTP @OTP-16222@"/> + <name name="h" arity="1" since="OTP 23.0"/> <fsummary>Module help information</fsummary> <type name="h_return"/> <desc> @@ -129,7 +129,7 @@ </func> <func> - <name name="h" arity="2" since="OTP @OTP-16222@"/> + <name name="h" arity="2" since="OTP 23.0"/> <fsummary>Function help information</fsummary> <type name="h_return"/> <type name="hf_return"/> @@ -139,7 +139,7 @@ </func> <func> - <name name="h" arity="3" since="OTP @OTP-16222@"/> + <name name="h" arity="3" since="OTP 23.0"/> <fsummary>Function help information</fsummary> <type name="h_return"/> <type name="hf_return"/> @@ -149,7 +149,7 @@ </func> <func> - <name name="ht" arity="1" since="OTP @OTP-16222@"/> + <name name="ht" arity="1" since="OTP 23.0"/> <fsummary>Type help information</fsummary> <type name="h_return"/> <desc> @@ -158,7 +158,7 @@ </func> <func> - <name name="ht" arity="2" since="OTP @OTP-16222@"/> + <name name="ht" arity="2" since="OTP 23.0"/> <fsummary>Type help information</fsummary> <type name="h_return"/> <type name="ht_return"/> @@ -168,7 +168,7 @@ </func> <func> - <name name="ht" arity="3" since="OTP @OTP-16222@"/> + <name name="ht" arity="3" since="OTP 23.0"/> <fsummary>Type help information</fsummary> <type name="h_return"/> <type name="ht_return"/> diff --git a/lib/stdlib/doc/src/gen_event.xml b/lib/stdlib/doc/src/gen_event.xml index 3a5e4cb619..2ddff240d2 100644 --- a/lib/stdlib/doc/src/gen_event.xml +++ b/lib/stdlib/doc/src/gen_event.xml @@ -524,9 +524,9 @@ gen_event:stop -----> Module:terminate/2 </func> <func> - <name since="OTP @OTP-16120@">start_monitor() -> Result</name> - <name since="OTP @OTP-16120@">start_monitor(EventMgrName | Options) -> Result</name> - <name since="OTP @OTP-16120@">start_monitor(EventMgrName, Options) -> Result</name> + <name since="OTP 23.0">start_monitor() -> Result</name> + <name since="OTP 23.0">start_monitor(EventMgrName | Options) -> Result</name> + <name since="OTP 23.0">start_monitor(EventMgrName, Options) -> Result</name> <fsummary>Create a stand-alone event manager process.</fsummary> <type> <v>EventMgrName = {local,Name} | {global,GlobalName} | {via,Module,ViaName}</v> diff --git a/lib/stdlib/doc/src/gen_server.xml b/lib/stdlib/doc/src/gen_server.xml index 4ab8360fd1..4abb91439e 100644 --- a/lib/stdlib/doc/src/gen_server.xml +++ b/lib/stdlib/doc/src/gen_server.xml @@ -551,8 +551,8 @@ gen_server:abcast -----> Module:handle_cast/2 </func> <func> - <name since="OTP @OTP-16120@">start_monitor(Module, Args, Options) -> Result</name> - <name since="OTP @OTP-16120@">start_monitor(ServerName, Module, Args, Options) -> Result</name> + <name since="OTP 23.0">start_monitor(Module, Args, Options) -> Result</name> + <name since="OTP 23.0">start_monitor(ServerName, Module, Args, Options) -> Result</name> <fsummary>Create a standalone <c>gen_server</c> process.</fsummary> <type> <v>ServerName = {local,Name} | {global,GlobalName}</v> diff --git a/lib/stdlib/doc/src/gen_statem.xml b/lib/stdlib/doc/src/gen_statem.xml index 1674773c64..fa3f20535d 100644 --- a/lib/stdlib/doc/src/gen_statem.xml +++ b/lib/stdlib/doc/src/gen_statem.xml @@ -1250,7 +1250,7 @@ handle_event(_, _, State, Data) -> </p> </item> <tag> - <c>push_callback_module</c><br /> + <c>push_callback_module</c> </tag> <item> <p> @@ -2049,8 +2049,8 @@ handle_event(_, _, State, Data) -> </func> <func> - <name name="start_monitor" arity="3" since="OTP @OTP-16120@"/> - <name name="start_monitor" arity="4" since="OTP @OTP-16120@"/> + <name name="start_monitor" arity="3" since="OTP 23.0"/> + <name name="start_monitor" arity="4" since="OTP 23.0"/> <fsummary>Create a standalone <c>gen_statem</c> process.</fsummary> <desc> <p> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index f1737f071b..24a8d4c089 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -31,6 +31,209 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 3.13</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Compiling a match specification with excessive nesting + caused the runtime system to crash due to scheduler stack + exhaustion. Instead of crashing the runtime system, + effected functions will now raise a <c>system_limit</c> + error exception in this situation.</p> + <p> + Own Id: OTP-16431 Aux Id: ERL-592 </p> + </item> + <item> + <p> Initialization of record fields using <c>_</c> is no + longer allowed if the number of affected fields is zero. + </p> + <p> + Own Id: OTP-16516</p> + </item> + <item> + <p> Fix bugs in <c>eval_bits</c>. </p> + <p> + Own Id: OTP-16545</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved the printout of single line logger events for + most of the OTP behaviours in STDLIB and Kernel. This + includes <c>proc_lib</c>, <c>gen_server</c>, + <c>gen_event</c>, <c>gen_statem</c>, <c>gen_fsm</c>, + <c>supervisor</c>, <c>supervisor_bridge</c> and + <c>application</c>.</p> + <p> + Improved the <seeerl + marker="kernel:logger_formatter#chars_limit"><c>chars_limit</c></seeerl> + and <seeerl + marker="kernel:logger_formatter#depth"><c>depth</c></seeerl> + handling in <c>proc_lib</c> and when formatting of + exceptions.</p> + <p> + Own Id: OTP-15299</p> + </item> + <item> + <p> + Remove usage and documentation of old requests of the + I/O-protocol.</p> + <p> + Own Id: OTP-15695</p> + </item> + <item> + <p>Improved ETS scalability of concurrent calls that + change the size of a table, like <c>ets:insert/2</c> and + <c>ets:delete/2</c>.</p> <p>This performance feature was + implemented for <c>ordered_set</c> in OTP 22.0 and does + now apply for all ETS table types.</p> <p>The improved + scalability may come at the cost of longer latency of + <c>ets:info(T,size)</c> and <c>ets:info(T,memory)</c>. A + new table option <c>decentralized_counters</c> has + therefore been added. It is default <c>true</c> for + <c>ordered_set</c> with <c>write_concurrency</c> enabled + and default <c>false</c> for all other table types.</p> + <p> + Own Id: OTP-15744 Aux Id: OTP-15623, PR-2229 </p> + </item> + <item> + <p> Handle Unicode filenames in the <c>zip</c> module. + </p> + <p> + Own Id: OTP-16005 Aux Id: ERL-1003, ERL-1150 </p> + </item> + <item> + <p> + Unicode support was updated to the Unicode 12.1 standard.</p> + <p> + Own Id: OTP-16073 Aux Id: PR-2339 </p> + </item> + <item> + <p> + All of the modules <seemfa + marker="stdlib:proc_lib#start_monitor/3"><c>proc_lib</c></seemfa>, + <seemfa + marker="stdlib:gen_server#start_monitor/3"><c>gen_server</c></seemfa>, + <seemfa + marker="stdlib:gen_statem#start_monitor/3"><c>gen_statem</c></seemfa>, + and <seemfa + marker="stdlib:gen_event#start_monitor/0"><c>gen_event</c></seemfa> + have been extended with a <c>start_monitor()</c> + function. For more information, see the documentation of + <c>start_monitor()</c> for these modules.</p> + <p> + Own Id: OTP-16120 Aux Id: ERIERL-402, PR-2427 </p> + </item> + <item> + <p> + Updates for new <c>erlang:term_to_iovec()</c> BIF.</p> + <p> + Own Id: OTP-16128 Aux Id: OTP-15618 </p> + </item> + <item> + <p>Documented a quirk regarding extraction from file + descriptors in <c>erl_tar</c>.</p> + <p> + Own Id: OTP-16171 Aux Id: ERL-1057 </p> + </item> + <item> + <p> + Added <c>ok</c> as return value to + <c>gen_server:reply/2</c></p> + <p> + Own Id: OTP-16210 Aux Id: PR-2411 </p> + </item> + <item> + <p>New functions have been added to <seeerl + marker="c"><c>c(3)</c></seeerl> for printing embedded + documentation for Erlang modules. The functions are:</p> + <taglist> <tag>h/1,2,3</tag> <item>Print the + documentation for a Module:Function/Arity.</item> + <tag>ht/1,2,3</tag> <item>Print the type documentation + for a Module:Type/Arity.</item> </taglist> <p>The + embedded documentation is created when building the + Erlang/OTP documentation.</p> + <p> + Own Id: OTP-16222</p> + </item> + <item> + <p> Add <c>indent</c> and <c>linewidth</c> to the options + of the <c>erl_pp</c> module's functions. </p> + <p> + Own Id: OTP-16276 Aux Id: PR-2443 </p> + </item> + <item> + <p> + Minor updates due to the new spawn improvements made.</p> + <p> + Own Id: OTP-16368 Aux Id: OTP-15251 </p> + </item> + <item> + <p>The compiler will now raise a warning when inlining is + used in modules that load NIFs.</p> + <p> + Own Id: OTP-16429 Aux Id: ERL-303 </p> + </item> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + <item> + <p> Extend <c>erl_parse:abstract/1,2</c> to handle + external fun expressions (<c>fun M:F/A</c>). </p> + <p> + Own Id: OTP-16480</p> + </item> + <item> + <p>Added <c>filelib:safe_relative_path/2</c> to replace + <c>filename:safe_relative_path/1</c>, which did not + safely handle symbolic links.</p> + <p><c>filename:safe_relative_path/1</c> has been + deprecated.</p> + <p> + Own Id: OTP-16483 Aux Id: PR-2542 </p> + </item> + <item> + <p> + The module <c>shell_docs</c> has been added. The module + contains functions for rendering, validating and + normalizing embedded documentation.</p> + <p> + Own Id: OTP-16500</p> + </item> + <item> + <p> + Module and function auto-completion in the shell now + looks at all available modules instead of only those + loaded. A module is considered available if it either is + loaded already or would be loaded if called.</p> + <p> + The auto-completion has also been expanded to work in the + new <c>h/1,2,3</c> function in <c>c(3)</c>.</p> + <p> + Own Id: OTP-16501 Aux Id: OTP-16494, OTP-16222, + OTP-16406, OTP-16499, OTP-16500, PR-2545, ERL-708 </p> + </item> + <item> + <p>Updated the internal <c>pcre</c> library to + <c>8.44</c>.</p> + <p> + Own Id: OTP-16557</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.12.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -653,6 +856,27 @@ </section> +<section><title>STDLIB 3.8.2.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + <seemfa marker="stdlib:re#run/3">re:run(Subject, RE, + [unicode])</seemfa> returned <c>nomatch</c> instead of + failing with a <c>badarg</c> error exception when + <c>Subject</c> contained illegal utf8 and <c>RE</c> was + passed as a binary. This has been corrected along with + corrections of reduction counting in <c>re:run()</c> + error cases.</p> + <p> + Own Id: OTP-16553</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 3.8.2.3</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/doc/src/proc_lib.xml b/lib/stdlib/doc/src/proc_lib.xml index 709f157b14..aa649a280a 100644 --- a/lib/stdlib/doc/src/proc_lib.xml +++ b/lib/stdlib/doc/src/proc_lib.xml @@ -360,9 +360,9 @@ init(Parent) -> </func> <func> - <name name="start_monitor" arity="3" since="OTP @OTP-16120@"/> - <name name="start_monitor" arity="4" since="OTP @OTP-16120@"/> - <name name="start_monitor" arity="5" since="OTP @OTP-16120@"/> + <name name="start_monitor" arity="3" since="OTP 23.0"/> + <name name="start_monitor" arity="4" since="OTP 23.0"/> + <name name="start_monitor" arity="5" since="OTP 23.0"/> <fsummary>Start a new process synchronously.</fsummary> <desc> <p> diff --git a/lib/stdlib/doc/src/shell.xml b/lib/stdlib/doc/src/shell.xml index b601c2397d..62ce63ec33 100644 --- a/lib/stdlib/doc/src/shell.xml +++ b/lib/stdlib/doc/src/shell.xml @@ -119,6 +119,9 @@ <section> <title>Shell Commands</title> + <p>The commands below are the built-in shell commands that are always + available. In most system the commands listed in the <seeerl marker="c">c(3)</seeerl> + module are also available in the shell.</p> <taglist> <tag><c>b()</c></tag> <item> @@ -740,8 +743,8 @@ q - quit erlang <p>If you want an Erlang node to have a remote job active from the start (rather than the default local job), start Erlang with flag - <c>-remsh</c>, for example, - <c>erl -sname this_node -remsh other_node@other_host</c></p> + <seecom marker="erts:erl#remsh"><c>-remsh</c></seecom>, for example, + <c>erl -remsh other_node@other_host</c></p> </section> <section> diff --git a/lib/stdlib/doc/src/shell_docs.xml b/lib/stdlib/doc/src/shell_docs.xml index 4a39ef9eef..c8fba1b43e 100644 --- a/lib/stdlib/doc/src/shell_docs.xml +++ b/lib/stdlib/doc/src/shell_docs.xml @@ -32,11 +32,11 @@ <rev>A</rev> <file>shell_docs.xml</file> </header> - <module since="OTP @OTP-16222@">shell_docs</module> - <modulesummary>Functions used to render documentation for a shell.</modulesummary> + <module since="OTP 23.0">shell_docs</module> + <modulesummary>Functions used to render EEP-48 style documentation for a shell.</modulesummary> <description> <p>This module can be used to render function and type documentation - to be printed in a shell. It can only render documentation of the format + to be printed in a shell. It can only render EEP-48 documentation of the format <c>application/erlang+html</c>. For more information about this format see <seeguide marker="erl_docgen:doc_storage">Documentation Storage</seeguide> in Erl_Docgen's User's Guide. @@ -48,6 +48,8 @@ <name name="docs_v1"/> </datatype> <datatype> + <name name="chunk_element_block_type"/> + <name name="chunk_element_inline_type"/> <name name="chunk_element_type"/> <desc> <p> @@ -68,38 +70,53 @@ <funcs> <func> - <name name="render" arity="2" since="OTP @OTP-16222@"/> + <name name="render" arity="2" since="OTP 23.0"/> <fsummary>Render the documentation for a module.</fsummary> <desc> <p>Render the documentation for a module.</p> </desc> </func> <func> - <name name="render" arity="3" since="OTP @OTP-16222@"/> - <name name="render" arity="4" since="OTP @OTP-16222@"/> + <name name="render" arity="3" since="OTP 23.0"/> + <name name="render" arity="4" since="OTP 23.0"/> <fsummary>Render the documentation for a function.</fsummary> <desc> <p>Render the documentation for a function.</p> </desc> </func> <func> - <name name="render_type" arity="2" since="OTP @OTP-16222@"/> + <name name="render_type" arity="2" since="OTP 23.0"/> <fsummary>Render a list of all available types in a module.</fsummary> <desc> <p>Render a list of all available types in a module.</p> </desc> </func> <func> - <name name="render_type" arity="3" since="OTP @OTP-16222@"/> - <name name="render_type" arity="4" since="OTP @OTP-16222@"/> + <name name="render_type" arity="3" since="OTP 23.0"/> + <name name="render_type" arity="4" since="OTP 23.0"/> <fsummary>Render the documentation of a type in a module.</fsummary> <desc> <p>Render the documentation of a type in a module.</p> </desc> </func> + <func> + <name name="render_callback" arity="2" since="OTP 23.0"/> + <fsummary>Render a list of all available callbacks in a module.</fsummary> + <desc> + <p>Render a list of all available callbacks in a module.</p> + </desc> + </func> + <func> + <name name="render_callback" arity="3" since="OTP 23.0"/> + <name name="render_callback" arity="4" since="OTP 23.0"/> + <fsummary>Render the documentation of a callback in a module.</fsummary> + <desc> + <p>Render the documentation of a callback in a module.</p> + </desc> + </func> <func> - <name name="validate" arity="1" since="OTP @OTP-16222@"/> + <name name="validate" arity="1" since="OTP 23.0"/> <fsummary>Validate the documentation</fsummary> <desc> <p>This function can be used to do a basic validation of @@ -108,7 +125,7 @@ </func> <func> - <name name="normalize" arity="1" since="OTP @OTP-16222@"/> + <name name="normalize" arity="1" since="OTP 23.0"/> <fsummary>Normalize the documentation</fsummary> <desc> <p>This function can be used to do whitespace normalization diff --git a/lib/stdlib/examples/erl_id_trans.erl b/lib/stdlib/examples/erl_id_trans.erl index eab2ec4164..a707c45eb9 100644 --- a/lib/stdlib/examples/erl_id_trans.erl +++ b/lib/stdlib/examples/erl_id_trans.erl @@ -193,9 +193,6 @@ pattern({map_field_exact,Line,K,V}) -> Ke = expr(K), Ve = pattern(V), {map_field_exact,Line,Ke,Ve}; -%%pattern({struct,Line,Tag,Ps0}) -> -%% Ps1 = pattern_list(Ps0), -%% {struct,Line,Tag,Ps1}; pattern({record,Line,Name,Pfs0}) -> Pfs1 = pattern_fields(Pfs0), {record,Line,Name,Pfs1}; @@ -433,9 +430,6 @@ expr({map_field_exact,Line,K,V}) -> Ke = expr(K), Ve = expr(V), {map_field_exact,Line,Ke,Ve}; -%%expr({struct,Line,Tag,Es0}) -> -%% Es1 = pattern_list(Es0), -%% {struct,Line,Tag,Es1}; expr({record_index,Line,Name,Field0}) -> Field1 = expr(Field0), {record_index,Line,Name,Field1}; diff --git a/lib/stdlib/src/c.erl b/lib/stdlib/src/c.erl index 6af3951604..fa7bf8bfbc 100644 --- a/lib/stdlib/src/c.erl +++ b/lib/stdlib/src/c.erl @@ -30,7 +30,7 @@ lc_batch/0, lc_batch/1, i/3,pid/3,m/0,m/1,mm/0,lm/0, bt/1, q/0, - h/1,h/2,h/3,ht/1,ht/2,ht/3, + h/1,h/2,h/3,ht/1,ht/2,ht/3,hcb/1,hcb/2,hcb/3, erlangrc/0,erlangrc/1,bi/1, flush/0, regs/0, uptime/0, nregs/0,pwd/0,ls/0,ls/1,cd/1,memory/1,memory/0, xm/1]). @@ -154,8 +154,9 @@ c(SrcFile, NewOpts, Filter, BeamFile, Info) -> safe_recompile(SrcFile, Options, BeamFile). -type h_return() :: ok | {error, missing | {unknown_format, unicode:chardata()}}. --type ht_return() :: h_return() | {error, type_missing}. -type hf_return() :: h_return() | {error, function_missing}. +-type ht_return() :: h_return() | {error, type_missing}. +-type hcb_return() :: h_return() | {error, callback_missing}. -spec h(module()) -> h_return(). h(Module) -> @@ -224,10 +225,49 @@ ht(Module,Type,Arity) -> Error end. +-spec hcb(module()) -> h_return(). +hcb(Module) -> + case code:get_doc(Module) of + {ok, #docs_v1{ format = ?NATIVE_FORMAT } = Docs} -> + format_docs(shell_docs:render_callback(Module, Docs)); + {ok, #docs_v1{ format = Enc }} -> + {error, {unknown_format, Enc}}; + Error -> + Error + end. + +-spec hcb(module(),Callback :: atom()) -> hcb_return(). +hcb(Module,Callback) -> + case code:get_doc(Module) of + {ok, #docs_v1{ format = ?NATIVE_FORMAT } = Docs} -> + format_docs(shell_docs:render_callback(Module, Callback, Docs)); + {ok, #docs_v1{ format = Enc }} -> + {error, {unknown_format, Enc}}; + Error -> + Error + end. + +-spec hcb(module(),Callback :: atom(),arity()) -> + hcb_return(). +hcb(Module,Callback,Arity) -> + case code:get_doc(Module) of + {ok, #docs_v1{ format = ?NATIVE_FORMAT } = Docs} -> + format_docs(shell_docs:render_callback(Module, Callback, Arity, Docs)); + {ok, #docs_v1{ format = Enc }} -> + {error, {unknown_format, Enc}}; + Error -> + Error + end. + format_docs({error,_} = E) -> E; format_docs(Docs) -> - format("~ts",[Docs]). + {match, Lines} = re:run(Docs,"(.+\n|\n)",[unicode,global,{capture,all_but_first,binary}]), + _ = paged_output(fun(Line,_) -> + format("~ts",Line), + {1,undefined} + end, undefined, Lines), + ok. old_options(Info) -> case lists:keyfind(options, 1, Info) of @@ -557,58 +597,54 @@ ni() -> i(all_procs()). -spec i([pid()]) -> 'ok'. i(Ps) -> - i(Ps, length(Ps)). - --spec i([pid()], non_neg_integer()) -> 'ok'. - -i(Ps, N) when N =< 100 -> - iformat("Pid", "Initial Call", "Heap", "Reds", - "Msgs"), - iformat("Registered", "Current Function", "Stack", "", - ""), - {R,M,H,S} = foldl(fun(Pid, {R0,M0,H0,S0}) -> - {A,B,C,D} = display_info(Pid), - {R0+A,M0+B,H0+C,S0+D} - end, {0,0,0,0}, Ps), - iformat("Total", "", w(H), w(R), w(M)), - iformat("", "", w(S), "", ""); -i(Ps, N) -> - iformat("Pid", "Initial Call", "Heap", "Reds", - "Msgs"), - iformat("Registered", "Current Function", "Stack", "", - ""), - paged_i(Ps, {0,0,0,0}, N, 50). - -paged_i([], {R,M,H,S}, _, _) -> - iformat("Total", "", w(H), w(R), w(M)), - iformat("", "", w(S), "", ""); -paged_i(Ps, Acc, N, Page) -> - {Pids, Rest, N1} = - if N > Page -> - {L1,L2} = lists:split(Page, Ps), - {L1,L2,N-Page}; - true -> - {Ps, [], 0} - end, - NewAcc = foldl(fun(Pid, {R,M,H,S}) -> - {A,B,C,D} = display_info(Pid), - {R+A,M+B,H+C,S+D} - end, Acc, Pids), - case Rest of - [_|_] -> - choice(fun() -> paged_i(Rest, NewAcc, N1, Page) end); - [] -> - paged_i([], NewAcc, 0, Page) + iformat("Pid", "Initial Call", "Heap", "Reds", "Msgs"), + iformat("Registered", "Current Function", "Stack", "", ""), + case paged_output(fun(Pid, {R,M,H,S}) -> + {A,B,C,D} = display_info(Pid), + {2,{R+A,M+B,H+C,S+D}} + end, 2, {0,0,0,0}, Ps) of + {R,M,H,S} -> + iformat("Total", "", w(H), w(R), w(M)), + iformat("", "", w(S), "", ""); + less -> + ok end. -choice(F) -> - case get_line('(c)ontinue (q)uit -->', "c\n") of +paged_output(Fun, Acc, Items) -> + paged_output(Fun, 0, Acc, Items). +paged_output(Fun, CurrLine, Acc, Items) -> + Limit = + case io:rows() of + {ok, Rows} -> Rows-2; + _ -> 100 + end, + paged_output(Fun, CurrLine, Limit, Acc, Items). + +paged_output(PrintFun, CurrLine, Limit, Acc, Items) when CurrLine >= Limit -> + case more() of + more -> + paged_output(PrintFun, 0, Limit, Acc, Items); + less -> + less + end; +paged_output(PrintFun, CurrLine, Limit, Acc, [H|T]) -> + {Lines, NewAcc} = PrintFun(H, Acc), + paged_output(PrintFun, CurrLine+Lines, Limit, NewAcc, T); +paged_output(_, _, _, Acc, []) -> + Acc. + +more() -> + case get_line('more (y/n)? (y) ', "y\n") of "c\n" -> - F(); + more; + "y\n" -> + more; "q\n" -> - quit; + less; + "n\n" -> + less; _ -> - choice(F) + more() end. get_line(P, Default) -> diff --git a/lib/stdlib/src/digraph.erl b/lib/stdlib/src/digraph.erl index 8a4df95027..58d493cf54 100644 --- a/lib/stdlib/src/digraph.erl +++ b/lib/stdlib/src/digraph.erl @@ -230,7 +230,7 @@ in_neighbours(G, V) -> Edges :: [edge()]. in_edges(G, V) -> - ets:select(G#digraph.ntab, [{{{in, V}, '$1'}, [], ['$1']}]). + [E || {{in, _}, E} <- ets:lookup(G#digraph.ntab, {in, V})]. -spec out_degree(G, V) -> non_neg_integer() when G :: graph(), @@ -255,7 +255,7 @@ out_neighbours(G, V) -> Edges :: [edge()]. out_edges(G, V) -> - ets:select(G#digraph.ntab, [{{{out, V}, '$1'}, [], ['$1']}]). + [E || {{out, _}, E} <- ets:lookup(G#digraph.ntab, {out, V})]. -spec add_edge(G, V1, V2) -> edge() | {'error', add_edge_err_rsn()} when G :: graph(), diff --git a/lib/stdlib/src/erl_expand_records.erl b/lib/stdlib/src/erl_expand_records.erl index 5351490b1a..6200978d4d 100644 --- a/lib/stdlib/src/erl_expand_records.erl +++ b/lib/stdlib/src/erl_expand_records.erl @@ -20,10 +20,6 @@ %% Purpose: Expand records into tuples. Also add explicit module %% names to calls to imported functions and BIFs. -%% N.B. Although structs (tagged tuples) are not yet allowed in the -%% language there is code included in pattern/2 and expr/3 (commented out) -%% that handles them. - -module(erl_expand_records). -export([module/2]). @@ -125,9 +121,6 @@ pattern({map_field_exact,Line,K0,V0}, St0) -> {K,St1} = expr(K0, St0), {V,St2} = pattern(V0, St1), {{map_field_exact,Line,K,V},St2}; -%%pattern({struct,Line,Tag,Ps}, St0) -> -%% {TPs,TPsvs,St1} = pattern_list(Ps, St0), -%% {{struct,Line,Tag,TPs},TPsvs,St1}; pattern({record_index,Line,Name,Field}, St) -> {index_expr(Line, Field, Name, record_fields(Name, St)),St}; pattern({record,Line0,Name,Pfs}, St0) -> @@ -310,9 +303,6 @@ expr({map_field_exact,Line,K0,V0}, St0) -> {K,St1} = expr(K0, St0), {V,St2} = expr(V0, St1), {{map_field_exact,Line,K,V},St2}; -%%expr({struct,Line,Tag,Es0}, Vs, St0) -> -%% {Es1,Esvs,Esus,St1} = expr_list(Es0, Vs, St0), -%% {{struct,Line,Tag,Es1},Esvs,Esus,St1}; expr({record_index,Line,Name,F}, St) -> I = index_expr(Line, F, Name, record_fields(Name, St)), expr(I, St); diff --git a/lib/stdlib/src/erl_internal.erl b/lib/stdlib/src/erl_internal.erl index f5059ac710..6ff5e23ee3 100644 --- a/lib/stdlib/src/erl_internal.erl +++ b/lib/stdlib/src/erl_internal.erl @@ -311,7 +311,6 @@ bif(is_process_alive, 1) -> true; bif(is_atom, 1) -> true; bif(is_boolean, 1) -> true; bif(is_binary, 1) -> true; -bif(is_bitstr, 1) -> true; bif(is_bitstring, 1) -> true; bif(is_float, 1) -> true; bif(is_function, 1) -> true; @@ -348,7 +347,6 @@ bif(max,2) -> true; bif(min,2) -> true; bif(module_loaded, 1) -> true; bif(monitor, 2) -> true; -bif(monitor, 3) -> true; bif(monitor_node, 2) -> true; bif(node, 0) -> true; bif(node, 1) -> true; @@ -465,7 +463,6 @@ old_bif(is_process_alive, 1) -> true; old_bif(is_atom, 1) -> true; old_bif(is_boolean, 1) -> true; old_bif(is_binary, 1) -> true; -old_bif(is_bitstr, 1) -> true; old_bif(is_bitstring, 1) -> true; old_bif(is_float, 1) -> true; old_bif(is_function, 1) -> true; diff --git a/lib/stdlib/src/erl_lint.erl b/lib/stdlib/src/erl_lint.erl index 4892dd5131..7c717e47d1 100644 --- a/lib/stdlib/src/erl_lint.erl +++ b/lib/stdlib/src/erl_lint.erl @@ -20,9 +20,6 @@ %% %% Do necessary checking of Erlang code. -%% N.B. All the code necessary for checking structs (tagged tuples) is -%% here. Just comment out the lines in pattern/2, gexpr/3 and expr/3. - -module(erl_lint). -export([module/1,module/2,module/3,format_error/1]). @@ -1646,8 +1643,6 @@ pattern({tuple,_Line,Ps}, Vt, Old, Bvt, St) -> pattern_list(Ps, Vt, Old, Bvt, St); pattern({map,_Line,Ps}, Vt, Old, Bvt, St) -> pattern_map(Ps, Vt, Old, Bvt, St); -%%pattern({struct,_Line,_Tag,Ps}, Vt, Old, Bvt, St) -> -%% pattern_list(Ps, Vt, Old, Bvt, St); pattern({record_index,Line,Name,Field}, _Vt, _Old, _Bvt, St) -> {Vt1,St1} = check_record(Line, Name, St, @@ -2266,8 +2261,6 @@ is_gexpr({string,_L,_S}, _Info) -> true; is_gexpr({nil,_L}, _Info) -> true; is_gexpr({cons,_L,H,T}, Info) -> is_gexpr_list([H,T], Info); is_gexpr({tuple,_L,Es}, Info) -> is_gexpr_list(Es, Info); -%%is_gexpr({struct,_L,_Tag,Es}, Info) -> -%% is_gexpr_list(Es, Info); is_gexpr({map,_L,Es}, Info) -> is_map_fields(Es, Info); is_gexpr({map,_L,Src,Es}, Info) -> diff --git a/lib/stdlib/src/erl_pp.erl b/lib/stdlib/src/erl_pp.erl index 387fa395ce..651c601bb0 100644 --- a/lib/stdlib/src/erl_pp.erl +++ b/lib/stdlib/src/erl_pp.erl @@ -560,8 +560,6 @@ lexpr({bc,_,E,Qs}, _Prec, Opts) -> %% {list,[{step,'<<',Lcl},'>>']}; lexpr({tuple,_,Elts}, _, Opts) -> tuple(Elts, Opts); -%%lexpr({struct,_,Tag,Elts}, _, Opts) -> -%% {first,format("~w", [Tag]),tuple(Elts, Opts)}; lexpr({record_index, _, Name, F}, Prec, Opts) -> {P,R} = preop_prec('#'), Nl = record_name(Name), diff --git a/lib/stdlib/src/otp_internal.erl b/lib/stdlib/src/otp_internal.erl index 11594483e0..b6531d9b5c 100644 --- a/lib/stdlib/src/otp_internal.erl +++ b/lib/stdlib/src/otp_internal.erl @@ -245,6 +245,10 @@ obsolete(snmpm, sync_set, 5) -> {deprecated, "use snmpm:sync_set2/4 instead.", "OTP 25"}; obsolete(snmpm, sync_set, 6) -> {deprecated, "use snmpm:sync_set2/4 instead.", "OTP 25"}; +obsolete(ssl, cipher_suites, 0) -> + {deprecated, "use cipher_suites/2,3 instead", "OTP 24"}; +obsolete(ssl, cipher_suites, 1) -> + {deprecated, "use cipher_suites/2,3 instead", "OTP 24"}; obsolete(sys, get_debug, 3) -> {deprecated, "incorrectly documented and only for internal use. Can often be replaced with sys:get_log/1"}; obsolete(wxCalendarCtrl, enableYearChange, 1) -> @@ -466,7 +470,7 @@ obsolete(crypto, stream_init, _) -> obsolete(filename, find_src, _) -> {deprecated, "use filelib:find_source/1,3 instead", "OTP 24"}; obsolete(ssl, ssl_accept, _) -> - {deprecated, "use ssl_handshake/1,2,3 instead"}; + {deprecated, "use ssl_handshake/1,2,3 instead", "OTP 24"}; obsolete(asn1ct, decode, _) -> {removed, "use Mod:decode/2 instead"}; obsolete(asn1ct, encode, _) -> @@ -494,6 +498,14 @@ obsolete(os_mon_mib, _, _) -> obsolete(_,_,_) -> no. -dialyzer({no_match, obsolete_type/3}). +obsolete_type(crypto, retired_cbc_cipher_aliases, 0) -> + {deprecated, "Use aes_*_cbc or des_ede3_cbc"}; +obsolete_type(crypto, retired_cfb_cipher_aliases, 0) -> + {deprecated, "Use aes_*_cfb8, aes_*_cfb128 or des_ede3_cfb"}; +obsolete_type(crypto, retired_ctr_cipher_aliases, 0) -> + {deprecated, "Use aes_*_ctr"}; +obsolete_type(crypto, retired_ecb_cipher_aliases, 0) -> + {deprecated, "Use aes_*_ecb"}; obsolete_type(erl_scan, column, 0) -> {removed, "use erl_anno:column() instead"}; obsolete_type(erl_scan, line, 0) -> diff --git a/lib/stdlib/src/otp_internal.hrl b/lib/stdlib/src/otp_internal.hrl index ace1fa5cc1..17e15da68f 100644 --- a/lib/stdlib/src/otp_internal.hrl +++ b/lib/stdlib/src/otp_internal.hrl @@ -26,7 +26,7 @@ -export([obsolete/3, obsolete_type/3]). -type tag() :: 'deprecated' | 'removed'. %% | 'experimental'. --type mfas() :: mfa() | {atom(), atom(), [byte()]}. +-type mfas() :: mfa() | {atom(), atom(), [byte()]} | string(). -type release() :: string(). -spec obsolete(module(), atom(), arity()) -> diff --git a/lib/stdlib/src/shell_default.erl b/lib/stdlib/src/shell_default.erl index e65340e663..4819beb62b 100644 --- a/lib/stdlib/src/shell_default.erl +++ b/lib/stdlib/src/shell_default.erl @@ -28,7 +28,7 @@ erlangrc/1,bi/1, regs/0, flush/0,pwd/0,ls/0,ls/1,cd/1, y/1, y/2, xm/1, bt/1, q/0, - h/1, h/2, h/3, ht/1, ht/2, ht/3, + h/1, h/2, h/3, ht/1, ht/2, ht/3, hcb/1, hcb/2, hcb/3, ni/0, nregs/0]). -export([ih/0,iv/0,im/0,ii/1,ii/2,iq/1,ini/1,ini/2,inq/1,ib/2,ib/3, @@ -49,8 +49,11 @@ help() -> format("h(Mod,Func)-- help about function in module\n"), format("h(Mod,Func,Arity) -- help about function with arity in module\n"), format("ht(Mod) -- help about a module's types\n"), - format("ht(Mod,Func) -- help about type in module\n"), - format("ht(Mod,Func,Arity) -- help about type with arity in module\n"), + format("ht(Mod,Type) -- help about type in module\n"), + format("ht(Mod,Type,Arity) -- help about type with arity in module\n"), + format("hcb(Mod) -- help about a module's callbacks\n"), + format("hcb(Mod,CB) -- help about callback in module\n"), + format("hcb(Mod,CB,Arity) -- help about callback with arity in module\n"), format("history(N) -- set how many previous commands to keep\n"), format("results(N) -- set how many previous command results to keep\n"), format("catch_exception(B) -- how exceptions are handled\n"), @@ -89,6 +92,9 @@ h(M,F,A) -> c:h(M,F,A). ht(M) -> c:ht(M). ht(M,F) -> c:ht(M,F). ht(M,F,A) -> c:ht(M,F,A). +hcb(M) -> c:hcb(M). +hcb(M,F) -> c:hcb(M,F). +hcb(M,F,A) -> c:hcb(M,F,A). i() -> c:i(). i(X,Y,Z) -> c:i(X,Y,Z). l(Mod) -> c:l(Mod). diff --git a/lib/stdlib/src/shell_docs.erl b/lib/stdlib/src/shell_docs.erl index ad93268fb2..03e7c47309 100644 --- a/lib/stdlib/src/shell_docs.erl +++ b/lib/stdlib/src/shell_docs.erl @@ -23,38 +23,41 @@ -export([render/2, render/3, render/4]). -export([render_type/2, render_type/3, render_type/4]). +-export([render_callback/2, render_callback/3, render_callback/4]). %% Used by chunks.escript in erl_docgen -export([validate/1, normalize/1]). %% Convinience functions --export([get_doc/1, get_doc/3, get_type_doc/3]). +-export([get_doc/1, get_doc/3, get_type_doc/3, get_callback_doc/3]). -record(config, { docs, io_opts = io:getopts(), io_columns = element(2,io:columns()) }). --define(ALL_ELEMENTS,[a,p,'div',h1,h2,h3,i,br,em,pre,code,ul,ol,li,dl,dt,dd]). +-define(ALL_ELEMENTS,[a,p,'div',br,h1,h2,h3,i,em,pre,code,ul,ol,li,dl,dt,dd]). %% inline elements are: --define(INLINE,[i,br,em,code,a]). +-define(INLINE,[i,em,code,a]). -define(IS_INLINE(ELEM),(((ELEM) =:= a) orelse ((ELEM) =:= code) - orelse ((ELEM) =:= i) orelse ((ELEM) =:= br) - orelse ((ELEM) =:= em))). + orelse ((ELEM) =:= i) orelse ((ELEM) =:= em))). %% non-inline elements are: --define(BLOCK,[p,'div',pre,ul,ol,li,dl,dt,dd,h1,h2,h3]). +-define(BLOCK,[p,'div',pre,br,ul,ol,li,dl,dt,dd,h1,h2,h3]). -define(IS_BLOCK(ELEM),not ?IS_INLINE(ELEM)). -define(IS_PRE(ELEM),(((ELEM) =:= pre))). --type chunk_element_type() :: a | p | 'div' | i | br | em | pre | code | ul | - ol | li | dl | dt | dd. --type chunk_element_attr() :: {atom(),unicode:chardata()}. --type chunk_element_attrs() :: [chunk_element_attr()]. +%% If you update the below types, make sure to update the documentation in +%% erl_docgen/doc/src/doc_storage.xml as well!!! +-type docs_v1() :: #docs_v1{}. +-type chunk_elements() :: [chunk_element()]. -type chunk_element() :: {chunk_element_type(),chunk_element_attrs(), chunk_elements()} | binary(). --type chunk_elements() :: [chunk_element()]. --type docs_v1() :: #docs_v1{}. - +-type chunk_element_attrs() :: [chunk_element_attr()]. +-type chunk_element_attr() :: {atom(),unicode:chardata()}. +-type chunk_element_type() :: chunk_element_inline_type() | chunk_element_block_type(). +-type chunk_element_inline_type() :: a | code | em | i. +-type chunk_element_block_type() :: p | 'div' | br | pre | ul | + ol | li | dl | dt | dd | h1 | h2 | h3. -spec validate(Module) -> ok when Module :: module() | docs_v1(). @@ -71,28 +74,80 @@ validate(#docs_v1{ module_doc = MDocs, docs = AllDocs }) -> true = lists:all(fun(Elem) -> ?IS_INLINE(Elem) end, ?INLINE), true = lists:all(fun(Elem) -> ?IS_BLOCK(Elem) end, ?BLOCK), - _ = maps:map(fun(_Key,MDoc) -> validate(MDoc,[]) end, MDocs), + _ = validate_docs(MDocs), lists:foreach(fun({_,_Anno, Sig, Docs, _Meta}) -> - case lists:all(fun erlang:is_binary/1, Sig) of - false -> throw({invalid_signature,Sig}); - true -> ok - end, - maps:map(fun(_Key,Doc) -> validate(Doc,[]) end, Docs) - end, AllDocs), + case lists:all(fun erlang:is_binary/1, Sig) of + false -> throw({invalid_signature,Sig}); + true -> ok + end, + validate_docs(Docs) + end, AllDocs), + ok. + +validate_docs(hidden) -> + ok; +validate_docs(none) -> + ok; +validate_docs(#{} = MDocs) -> + _ = maps:map(fun(_Key,MDoc) -> validate_docs(MDoc,[]) end, MDocs), ok. -validate([H|T],Path) when is_tuple(H) -> - _ = validate(H,Path), - validate(T,Path); -validate({Tag,Attr,Content},Path) -> +validate_docs([H|T],Path) when is_tuple(H) -> + _ = validate_docs(H,Path), + validate_docs(T,Path); +validate_docs({br,Attr,Content} = Br,Path) -> + if Attr =:= [], Content =:= [] -> + ok; + true -> + throw({content_to_allowed_in_br,Br,Path}) + end; +validate_docs({Tag,Attr,Content},Path) -> + + %% Test that we only have li's within ul and ol + case (Tag =/= li) andalso (length(Path) > 0) andalso ((hd(Path) =:= ul) orelse (hd(Path) =:= ol)) of + true -> + throw({only_li_allowed_within_ul_or_ol,Tag,Path}); + _ -> + ok + end, + + %% Test that we only have dd's and dt's within dl + case (Tag =/= dd) andalso (Tag =/= dt) andalso (length(Path) > 0) andalso (hd(Path) =:= dl) of + true -> + throw({only_dd_or_dt_allowed_within_dl,Tag,Path}); + _ -> + ok + end, + + %% Test that we do not have p's within p's case Tag =:= p andalso lists:member(p, Path) of true -> - throw({nested,p,not_allowed}); + throw({nested_p_not_allowed,Tag,Path}); + false -> + ok + end, + %% Test that there are no block tags within a pre, h1, h2 or h3 + case lists:member(pre,Path) or lists:member(h1,Path) or + lists:member(h2,Path) or lists:member(h3,Path) of + true when ?IS_BLOCK(Tag) -> + throw({cannot_put_block_tag_within_pre,Tag,Path}); + _ -> + ok + end, + %% Test that a block tag is not within an inline tag + case lists:member(Tag,?BLOCK) of + true -> + case lists:any(fun(P) -> ?IS_INLINE(P) end, Path) of + true -> + throw({cannot_put_inline_tag_outside_block, Tag, Path}); + false -> + ok + end; false -> ok end, case lists:member(Tag,?ALL_ELEMENTS) of false -> - throw({invalid_tag,Tag}); + throw({invalid_tag,Tag,Path}); true -> ok end, @@ -100,10 +155,10 @@ validate({Tag,Attr,Content},Path) -> true -> ok; false -> throw({invalid_attribute,{Tag,Attr}}) end, - validate(Content,[Tag | Path]); -validate([Chars | T], Path) when is_binary(Chars) -> - validate(T, Path); -validate([],_) -> + validate_docs(Content,[Tag | Path]); +validate_docs([Chars | T], Path) when is_binary(Chars) -> + validate_docs(T, Path); +validate_docs([],_) -> ok. %% Follows algorithm described here: @@ -119,13 +174,13 @@ normalize(Docs) -> normalize_trim(Bin,true) when is_binary(Bin) -> %% Remove any whitespace (except \n) before or after a newline - NoSpace = re:replace(Bin,"[^\\S\n]*\n+[^\\S\n]*","\n",[global]), + NoSpace = re:replace(Bin,"[^\\S\n]*\n+[^\\S\n]*","\n",[unicode,global]), %% Replace any tabs with space - NoTab = re:replace(NoSpace,"\t"," ",[global]), + NoTab = re:replace(NoSpace,"\t"," ",[unicode,global]), %% Replace any newlines with space - NoNewLine = re:replace(NoTab,"\\v"," ",[global]), + NoNewLine = re:replace(NoTab,"\\v"," ",[unicode,global]), %% Replace any sequences of \s with a single " " - re:replace(NoNewLine,"\\s+"," ",[global,{return,binary}]); + re:replace(NoNewLine,"\\s+"," ",[unicode,global,{return,binary}]); normalize_trim(Bin,false) when is_binary(Bin) -> Bin; normalize_trim([{pre,Attr,Content}|T],Trim) -> @@ -148,38 +203,41 @@ normalize_trim([],_Trim) -> normalize_space([{Pre,Attr,Content}|T]) when ?IS_PRE(Pre) -> [{Pre,Attr,trim_first_and_last(Content,$\n)} | normalize_space(T)]; normalize_space([{Block,Attr,Content}|T]) when ?IS_BLOCK(Block) -> - [{Block,Attr,trim_first_and_last(trim_inline(Content),$ )} | normalize_space(T)]; -normalize_space([B]) when is_binary(B) -> - trim_first_and_last([B],$ ); -normalize_space([E|T]) -> - [E|normalize_space(T)]; + [{Block,Attr,normalize_space(Content)} | normalize_space(T)]; normalize_space([]) -> - []. + []; +normalize_space(Elems) -> + {InlineElems, T} = + lists:splitwith(fun(E) -> + is_binary(E) orelse (is_tuple(E) andalso ?IS_INLINE(element(1,E))) + end, Elems), + trim_inline(InlineElems) ++ normalize_space(T). trim_inline(Content) -> {NewContent,_} = trim_inline(Content,false), - NewContent. + trim_first_and_last(NewContent,$ ). trim_inline([Bin|T],false) when is_binary(Bin) -> LastElem = binary:at(Bin,byte_size(Bin)-1), - {NewT, NewState} = trim_inline(T,LastElem =:= $ ), - {[Bin | NewT],NewState}; + case trim_inline(T,LastElem =:= $ ) of + {[B2 | NewT],NewState} when is_binary(B2) -> + {[<<Bin/binary,B2/binary>>|NewT],NewState}; + {NewT, NewState} -> + {[Bin|NewT],NewState} + end; trim_inline([<<" ">>|T],true) -> - trim_inline(T,false); + trim_inline(T,true); trim_inline([<<" ",Bin/binary>>|T],true) when is_binary(Bin) -> - trim_inline([Bin | T],false); + trim_inline([Bin | T],true); trim_inline([Bin|T],true) when is_binary(Bin) -> trim_inline([Bin|T],false); -trim_inline([{Elem,Attr,Content}|T],TrimSpace) when ?IS_INLINE(Elem) -> +trim_inline([{Elem,Attr,Content}|T],TrimSpace) -> {NewContent,ContentTrimSpace} = trim_inline(Content,TrimSpace), {NewT,TTrimSpace} = trim_inline(T,ContentTrimSpace), - {[{Elem,Attr,NewContent} | NewT], TTrimSpace}; -trim_inline([{Elem1,_A1,_C1} = B1,<<" ">>,{Elem2,_A2,_C2} = B2|T],TrimSpace) - when ?IS_BLOCK(Elem1),?IS_BLOCK(Elem2) -> - trim_inline([B1,B2|T],TrimSpace); -trim_inline([{Elem,_Attr,_Content} = Block|T],_TrimSpace) when ?IS_BLOCK(Elem) -> - [NewBlock] = normalize_space([Block]), - {NewT,TTrimSpace} = trim_inline(T,false), - {[NewBlock | NewT], TTrimSpace}; + if NewContent == [] -> + {NewT, TTrimSpace}; + true -> + {[{Elem,Attr,NewContent} | NewT], TTrimSpace} + end; trim_inline([],TrimSpace) -> {[],TrimSpace}. @@ -205,6 +263,8 @@ trim_first([Bin|T],false,What) when is_binary(Bin) -> end; trim_first([{Elem,Attr,Content} = Tag|T],false,What) -> case trim_first(Content,false,What) of + {[],true} -> + {T,true}; {NewContent,true} -> {[{Elem,Attr,NewContent}|T],true}; {Content,false} -> @@ -233,8 +293,12 @@ trim_last([{Elem,Attr,Content} = Tag|T],What) -> {NewT,true} -> {[Tag | NewT],true}; {T,false} -> - {NewContent,NewState} = trim_last(Content,What), - {[{Elem,Attr,NewContent}|T],NewState} + case trim_last(Content,What) of + {[],NewState} -> + {T,NewState}; + {NewContent,NewState} -> + {[{Elem,Attr,NewContent}|T],NewState} + end end; trim_last([],_What) -> {[],false}. @@ -265,9 +329,9 @@ get_doc(Module, Function, Arity) -> [{F,A,S,get_local_doc({F,A},D),M} || {F,A,S,D,M} <- FnFunctions]. -spec render(Module :: module(), Docs :: docs_v1()) -> unicode:chardata(). -render(Module, #docs_v1{ module_doc = ModuleDoc, metadata = MD } = D) -> - render_docs([["\t",atom_to_binary(Module)]], - get_local_doc(Module, ModuleDoc), MD, D). +render(Module, #docs_v1{ module_doc = ModuleDoc } = D) -> + render_headers_and_docs([[{h2,[],[<<"\t",(atom_to_binary(Module))/binary>>]}]], + get_local_doc(Module, ModuleDoc), D). -spec render(Module :: module(), Function :: function(), Docs :: docs_v1()) -> unicode:chardata() | {error,function_missing}. @@ -309,18 +373,13 @@ get_type_doc(Module, Type, Arity) -> [{F,A,S,get_local_doc(F, D),M} || {F,A,S,D,M} <- FnFunctions]. -spec render_type(Module :: module(), Docs :: docs_v1()) -> unicode:chardata(). -render_type(Module, #docs_v1{ docs = Docs } = D) -> - render_type_signatures(Module, - lists:filter(fun({{type, _, _},_Anno,_Sig,_Doc,_Meta}) -> - true; - (_) -> - false - end, Docs), D). +render_type(Module, D) -> + render_signature_listing(Module, type, D). -spec render_type(Module :: module(), Type :: atom(), Docs :: docs_v1()) -> unicode:chardata() | {error,type_missing}. render_type(_Module, Type, #docs_v1{ docs = Docs } = D) -> - render_type_docs( + render_typecb_docs( lists:filter(fun({{type, T, _},_Anno,_Sig,_Doc,_Meta}) -> T =:= Type; (_) -> @@ -330,13 +389,56 @@ render_type(_Module, Type, #docs_v1{ docs = Docs } = D) -> -spec render_type(Module :: module(), Type :: atom(), Arity :: arity(), Docs :: docs_v1()) -> unicode:chardata() | {error,type_missing}. render_type(_Module, Type, Arity, #docs_v1{ docs = Docs } = D) -> - render_type_docs( + render_typecb_docs( lists:filter(fun({{type, T, A},_Anno,_Sig,_Doc,_Meta}) -> T =:= Type andalso A =:= Arity; (_) -> false end, Docs), D). +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% API function for dealing with the callback documentation +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +-spec get_callback_doc(Module :: module(), Callback :: atom(), Arity :: arity()) -> + [{{Callback,Arity}, Anno, Signature, chunk_elements(), Metadata}] when + Callback :: atom(), + Arity :: arity(), + Anno :: erl_anno:anno(), + Signature :: [binary()], + Metadata :: #{}. +get_callback_doc(Module, Callback, Arity) -> + {ok, #docs_v1{ docs = Docs } } = code:get_doc(Module), + FnFunctions = + lists:filter(fun({{callback, T, A},_Anno,_Sig,_Doc,_Meta}) -> + T =:= Callback andalso A =:= Arity; + (_) -> + false + end, Docs), + [{F,A,S,get_local_doc(F, D),M} || {F,A,S,D,M} <- FnFunctions]. + +-spec render_callback(Module :: module(), Docs :: docs_v1()) -> unicode:chardata(). +render_callback(Module, D) -> + render_signature_listing(Module, callback, D). + +-spec render_callback(Module :: module(), Callback :: atom(), Docs :: docs_v1()) -> + unicode:chardata() | {error,callback_missing}. +render_callback(_Module, Callback, #docs_v1{ docs = Docs } = D) -> + render_typecb_docs( + lists:filter(fun({{callback, T, _},_Anno,_Sig,_Doc,_Meta}) -> + T =:= Callback; + (_) -> + false + end, Docs), D). + +-spec render_callback(Module :: module(), Callback :: atom(), Arity :: arity(), + Docs :: docs_v1()) -> unicode:chardata() | {error,callback_missing}. +render_callback(_Module, Callback, Arity, #docs_v1{ docs = Docs } = D) -> + render_typecb_docs( + lists:filter(fun({{callback, T, A},_Anno,_Sig,_Doc,_Meta}) -> + T =:= Callback andalso A =:= Arity; + (_) -> + false + end, Docs), D). %% Get the docs in the correct locale if it exists. get_local_doc(MissingMod, Docs) when is_atom(MissingMod) -> @@ -359,58 +461,127 @@ get_local_doc(Missing, None) when None =:= none; None =:= #{} -> %%% Functions for rendering reference documentation render_function([], _D) -> {error,function_missing}; -render_function(FDocs, D) -> - [render_docs(render_signature(Func), get_local_doc({F,A},Doc), Meta, D) - || {{_,F,A},_Anno,_Sig,Doc,Meta} = Func <- lists:sort(FDocs)]. - -%% Render the signature of either function or a type, or anything else really. -render_signature({{_Type,_F,_A},_Anno,_Sig,_Docs,#{ signature := Specs }}) -> - [erl_pp:attribute(Spec,[{encoding,utf8}]) || Spec <- Specs]; -render_signature({{_Type,_F,_A},_Anno,Sigs,_Docs,_Meta}) -> - [Sig || Sig <- Sigs]. - -render_since(#{ since := Vsn }) -> - ["\n\nSince: ",Vsn]; -render_since(_) -> +render_function(FDocs, #docs_v1{ docs = Docs } = D) -> + Grouping = + lists:foldl( + fun({_Group,_Anno,_Sig,_Doc,#{ equiv := Group }} = Func,Acc) -> + Members = maps:get(Group, Acc, []), + Acc#{ Group => [Func|Members] }; + ({Group, _Anno, _Sig, _Doc, _Meta} = Func, Acc) -> + Members = maps:get(Group, Acc, []), + Acc#{ Group => [Func|Members] } + end, #{}, lists:sort(FDocs)), + lists:map( + fun({{_,F,A} = Group,Members}) -> + Signatures = lists:flatmap(fun render_signature/1,lists:reverse(Members)), + case lists:search(fun({_,_,_,Doc,_}) -> + Doc =/= #{} + end, Members) of + {value, {_,_,_,Doc,_Meta}} -> + render_headers_and_docs(Signatures, get_local_doc({F,A},Doc), D); + false -> + case lists:keyfind(Group, 1, Docs) of + false -> + render_headers_and_docs(Signatures, get_local_doc({F,A},none), D); + {_,_,_,Doc,_} -> + render_headers_and_docs(Signatures, get_local_doc({F,A},Doc), D) + end + end + end, maps:to_list(Grouping)). + +%% Render the signature of either function, type, or anything else really. +render_signature({{_Type,_F,_A},_Anno,_Sigs,_Docs,#{ signature := Specs } = Meta}) -> + lists:flatmap( + fun(ASTSpec) -> + PPSpec = erl_pp:attribute(ASTSpec,[{encoding,utf8}]), + Spec = + case ASTSpec of + {_Attribute, _Line, opaque, _} -> + %% We do not want show the internals of the opaque type + hd(string:split(PPSpec,"::")); + _ -> + PPSpec + end, + BinSpec = + unicode:characters_to_binary( + string:trim(Spec, trailing, "\n")), + [{pre,[],[{em,[],BinSpec}]}|render_meta(Meta)] + end, Specs); +render_signature({{_Type,_F,_A},_Anno,Sigs,_Docs,Meta}) -> + lists:flatmap( + fun(Sig) -> + [{h2,[],[<<"  "/utf8,Sig/binary>>]}|render_meta(Meta)] + end, Sigs). + +render_meta(M) -> + case render_meta_(M) of + [] -> []; + Meta -> + [[{dl,[],Meta}]] + end. +render_meta_(#{ since := Vsn } = M) -> + [{dt,[],<<"Since">>},{dd,[],[Vsn]} + | render_meta_(maps:remove(since, M))]; +render_meta_(#{ deprecated := Depr } = M) -> + [{dt,[],<<"Deprecated">>},{dd,[],[Depr]} + | render_meta_(maps:remove(deprecated, M))]; +render_meta_(_) -> []. -render_docs(Headers, DocContents, MD, D = #config{}) -> - init_ansi(D), - try - {Doc,_} = trimnl(render_docs(DocContents,[],0,2,D)), - [sansi(bold), - [io_lib:format("~n~ts",[Header]) || Header <- Headers], - ransi(bold), - render_since(MD), - io_lib:format("~n~n~ts",[Doc])] - after - clean_ansi() - end; -render_docs(Headers, DocContents, MD, D) -> - render_docs(Headers, DocContents, MD, #config{ docs = D }). +render_headers_and_docs(Headers, DocContents, D) -> + ["\n",render_docs( + lists:flatmap( + fun(Header) -> + [{br,[],[]},Header] + end,Headers), 0, D), + "\n", + render_docs(DocContents,2,D)]. + +%%% Functions for rendering type/callback documentation +render_signature_listing(Module, Type, #docs_v1{ docs = Docs } = D) -> + Slogan = [{h2,[],[<<"\t",(atom_to_binary(Module))/binary>>]},{br,[],[]}], + case lists:filter(fun({{T, _, _},_Anno,_Sig,_Doc,_Meta}) -> + Type =:= T + end, Docs) of + [] -> + render_docs( + Slogan ++ [<<"There are no ",(atom_to_binary(Type))/binary,"s " + "in this module">>], D); + Headers -> + Hdr = lists:flatmap( + fun(Header) -> + [{br,[],[]},render_signature(Header)] + end,Headers), + render_docs( + Slogan ++ + [{p,[],[<<"These ",(atom_to_binary(Type))/binary,"s " + "are documented in this module:">>]}, + {br,[],[]}, Hdr], D) + end. + +render_typecb_docs([], _D) -> + {error,type_missing}; +render_typecb_docs(TypeCBs, #config{} = D) when is_list(TypeCBs) -> + [render_typecb_docs(TypeCB, D) || TypeCB <- TypeCBs]; +render_typecb_docs({{_,F,A},_,_Sig,Docs,_Meta} = TypeCB, #config{} = D) -> + render_headers_and_docs(render_signature(TypeCB), get_local_doc({F,A},Docs), D); +render_typecb_docs(Docs, D) -> + render_typecb_docs(Docs, #config{ docs = D }). -%%% Functions for rendering type documentation -render_type_signatures(Module, Types, D = #config{}) -> +%%% General rendering functions +render_docs(DocContents, D) -> + render_docs(DocContents, 0, D). +render_docs(DocContents, Ind, D = #config{}) -> init_ansi(D), try - [sansi(bold),"\t",atom_to_list(Module),ransi(bold),"\n\n", - [render_signature(Type) || Type <- Types ]] + {Doc,_} = trimnl(render_docs(DocContents, [], 0, Ind, D)), + Doc after clean_ansi() end; -render_type_signatures(Module, Types, D) -> - render_type_signatures(Module, Types, #config{ docs = D }). - -render_type_docs([], _D) -> - {error,type_missing}; -render_type_docs(Types, #config{} = D) when is_list(Types) -> - [render_type_docs(Type, D) || Type <- Types]; -render_type_docs({{_,F,A},_,_Sig,Docs,Meta} = Type, #config{} = D) -> - render_docs(render_signature(Type), get_local_doc({F,A},Docs), Meta, D); -render_type_docs(Docs, D) -> - render_type_docs(Docs, #config{ docs = D }). +render_docs(DocContents, Ind, D) -> + render_docs(DocContents, Ind, #config{ docs = D }). -%%% General rendering functions render_docs(Elems,State,Pos,Ind,D) when is_list(Elems) -> lists:mapfoldl(fun(Elem,P) -> % io:format("Elem: ~p (~p) (~p,~p)~n",[Elem,State,P,Ind]), @@ -443,10 +614,10 @@ render_docs(Elem,State,Pos,Ind,D) -> {unicode:chardata(), Pos :: non_neg_integer()}. render_element({IgnoreMe,_,Content}, State, Pos, Ind,D) - when IgnoreMe =:= a; IgnoreMe =:= anno -> + when IgnoreMe =:= a -> render_docs(Content, State, Pos, Ind,D); -%% Catch h1, h2 and h3 before the padding is done as there reset padding +%% Catch h1, h2 and h3 before the padding is done as they reset padding render_element({h1,_,Content},State,0 = Pos,_Ind,D) -> trimnlnl(render_element({code,[],[{em,[],Content}]}, State, Pos, 0, D)); render_element({h2,_,Content},State,0 = Pos,_Ind,D) -> @@ -461,7 +632,7 @@ render_element({'div',[{class,What}],Content},State,Pos,Ind,D) -> {Docs,_} = render_docs(Content, ['div'|State], 0, Ind+2, D), trimnlnl([pad(Ind - Pos),string:titlecase(What),":\n",Docs]); render_element({Tag,_,Content},State,Pos,Ind,D) when Tag =:= p; Tag =:= 'div' -> - trimnlnl(render_docs(Content, [Tag|State], Pos, Ind,D)); + trimnlnl(render_docs(Content, [Tag|State], Pos, Ind, D)); render_element(Elem,State,Pos,Ind,D) when Pos < Ind -> % io:format("Pad: ~p~n",[Ind - Pos]), @@ -481,8 +652,8 @@ render_element({i,_,Content},State,Pos,Ind,D) -> %% Just ignore i as ansi does not have cursive style render_docs(Content, State, Pos, Ind,D); -render_element({br,[],[]},_State,_Pos,_Ind,_D) -> - nl(""); +render_element({br,[],[]},_State,Pos,_Ind,_D) -> + {"",Pos}; render_element({em,_,Content},State,Pos,Ind,D) -> Bold = sansi(bold), @@ -550,7 +721,13 @@ render_element(B, State, Pos, Ind,#config{ io_columns = Cols }) when is_binary(B end; render_element({Tag,Attr,Content}, State, Pos, Ind,D) -> - throw({unhandled,{Tag,Attr,Content,Pos,Ind}}), + case lists:member(Tag,?ALL_ELEMENTS) of + true -> + throw({unhandled_element,Tag,Attr,Content}); + false -> + %% We ignore tags that we do not care about + ok + end, render_docs(Content, State, Pos, Ind,D). render_words(Words,[_,types|State],Pos,Ind,Acc,Cols) -> @@ -560,8 +737,10 @@ render_words(Words,[_,types|State],Pos,Ind,Acc,Cols) -> render_words([Word|T],State,Pos,Ind,Acc,Cols) when is_binary(Word) -> WordLength = string:length(Word), NewPos = WordLength + Pos, + %% We do not want to add a newline if this word is only a punctuation + IsPunct = is_tuple(re:run(Word,"^\\W$",[unicode])), if - NewPos > (Cols - 10 - Ind) -> + NewPos > (Cols - 10 - Ind), Word =/= <<>>, not IsPunct -> %% Word does not fit, time to add a newline and also pad to Indent level render_words(T,State,WordLength+Ind+1,Ind,[[[pad(Ind), Word]]|Acc],Cols); true -> @@ -601,9 +780,14 @@ get_bullet(_State,latin1) -> <<" * ">>; get_bullet(State,unicode) -> %% Fancy bullet point logic! - lists:nth(length([l || l <- State]), - [<<" • "/utf8>>,<<" ￮ "/utf8>>, - <<" ◼ "/utf8>>,<<" ◻ "/utf8>>]). + case length([l || l <- State]) of + Level when Level > 4 -> + get_bullet(State, latin1); + Level -> + lists:nth(Level, + [<<" • "/utf8>>,<<" ￮ "/utf8>>, + <<" ◼ "/utf8>>,<<" ◻ "/utf8>>]) + end. % Look for the length of the last line of a string lastline(Str) -> diff --git a/lib/stdlib/src/stdlib.app.src b/lib/stdlib/src/stdlib.app.src index 774f7eaa9c..b59e3b28c0 100644 --- a/lib/stdlib/src/stdlib.app.src +++ b/lib/stdlib/src/stdlib.app.src @@ -109,6 +109,6 @@ dets]}, {applications, [kernel]}, {env, []}, - {runtime_dependencies, ["sasl-3.0","kernel-@OTP-15251@","erts-@OTP-15251:OTP-16431@","crypto-3.3", + {runtime_dependencies, ["sasl-3.0","kernel-7.0","erts-11.0","crypto-3.3", "compiler-5.0"]} ]}. diff --git a/lib/stdlib/src/stdlib.appup.src b/lib/stdlib/src/stdlib.appup.src index 6d6ee14d29..85fba9ebbd 100644 --- a/lib/stdlib/src/stdlib.appup.src +++ b/lib/stdlib/src/stdlib.appup.src @@ -21,6 +21,7 @@ %% versions from the following OTP releases: %% - OTP 21 %% - OTP 22 +%% - OTP 23 %% %% We also allow upgrade from, and downgrade to all %% versions that have branched off from the above @@ -35,6 +36,7 @@ {<<"^3\\.11\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.12$">>,[restart_new_emulator]}, {<<"^3\\.12\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.12\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.5$">>,[restart_new_emulator]}, {<<"^3\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, @@ -59,6 +61,7 @@ {<<"^3\\.11\\.2(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.12$">>,[restart_new_emulator]}, {<<"^3\\.12\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, + {<<"^3\\.12\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, {<<"^3\\.5$">>,[restart_new_emulator]}, {<<"^3\\.5\\.0(?:\\.[0-9]+)+$">>,[restart_new_emulator]}, {<<"^3\\.5\\.1(?:\\.[0-9]+)*$">>,[restart_new_emulator]}, diff --git a/lib/stdlib/test/digraph_SUITE.erl b/lib/stdlib/test/digraph_SUITE.erl index b5d3452616..ce0bc90f1c 100644 --- a/lib/stdlib/test/digraph_SUITE.erl +++ b/lib/stdlib/test/digraph_SUITE.erl @@ -31,7 +31,7 @@ init_per_group/2,end_per_group/2]). -export([opts/1, degree/1, path/1, cycle/1, vertices/1, - edges/1, data/1, otp_3522/1, otp_3630/1, otp_8066/1]). + edges/1, data/1, otp_3522/1, otp_3630/1, otp_8066/1, vertex_names/1]). -export([spawn_graph/2]). @@ -41,10 +41,10 @@ suite() -> [{ct_hooks,[ts_install_cth]}]. all() -> [opts, degree, path, cycle, {group, misc}, - {group, tickets}]. + {group, tickets}, vertex_names]. groups() -> - [{misc, [], [vertices, edges, data]}, + [{misc, [], [vertices, edges, data, vertex_names]}, {tickets, [], [otp_3522, otp_3630, otp_8066]}]. init_per_suite(Config) -> @@ -337,6 +337,51 @@ otp_8066(Config) when is_list(Config) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +vertex_names(Config) when is_list(Config) -> + %% Check that a node named '_' does not lead to wildcard matches + %% in ets. + + G = digraph:new([acyclic]), + A = digraph:add_vertex(G, 'A'), + B = digraph:add_vertex(G, '_'), + AB = digraph:add_edge(G, A, B), + + %% Link A -> B + 1 = digraph:out_degree(G, A), + 1 = digraph:in_degree(G, B), + 0 = digraph:out_degree(G, B), + 0 = digraph:in_degree(G, A), + [B] = digraph:out_neighbours(G, A), + [A] = digraph:in_neighbours(G, B), + [] = digraph:out_neighbours(G, B), + [] = digraph:in_neighbours(G, A), + [AB] = digraph:out_edges(G, A), + [AB] = digraph:in_edges(G, B), + [] = digraph:out_edges(G, B), + [] = digraph:in_edges(G, A), + + %% Reverse the edge + digraph:del_edge(G, AB), + BA = digraph:add_edge(G, B, A), + + 1 = digraph:out_degree(G, B), + 1 = digraph:in_degree(G, A), + 0 = digraph:out_degree(G, A), + 0 = digraph:in_degree(G, B), + [A] = digraph:out_neighbours(G, B), + [B] = digraph:in_neighbours(G, A), + [] = digraph:out_neighbours(G, A), + [] = digraph:in_neighbours(G, B), + [BA] = digraph:out_edges(G, B), + [BA] = digraph:in_edges(G, A), + [] = digraph:out_edges(G, A), + [] = digraph:in_edges(G, B), + + digraph:delete(G), + ok. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + sane(G) -> sane1(G), erase(sane) =:= undefined. diff --git a/lib/stdlib/test/gen_statem_SUITE.erl b/lib/stdlib/test/gen_statem_SUITE.erl index 1bde1b3d74..76dee868e9 100644 --- a/lib/stdlib/test/gen_statem_SUITE.erl +++ b/lib/stdlib/test/gen_statem_SUITE.erl @@ -453,19 +453,24 @@ stop7(Config) -> stop8(Config) -> Node = gen_statem_stop8, {ok,NodeName} = ct_slave:start(Node), - Dir = filename:dirname(code:which(?MODULE)), - rpc:call(NodeName, code, add_path, [Dir]), - {ok,Pid} = - rpc:call( - NodeName, gen_statem,start, - [?MODULE,start_arg(Config, []),[]]), - ok = gen_statem:stop(Pid), - false = rpc:call(NodeName, erlang, is_process_alive, [Pid]), - noproc = - ?EXPECT_FAILURE(gen_statem:stop(Pid), Reason1), - {ok,NodeName} = ct_slave:stop(Node), + Statem = + try + Dir = filename:dirname(code:which(?MODULE)), + rpc:block_call(NodeName, code, add_path, [Dir]), + {ok,Pid} = + rpc:block_call( + NodeName, gen_statem,start, + [?MODULE,start_arg(Config, []),[]]), + ok = gen_statem:stop(Pid), + false = rpc:block_call(NodeName, erlang, is_process_alive, [Pid]), + noproc = + ?EXPECT_FAILURE(gen_statem:stop(Pid), Reason1), + Pid + after + {ok,NodeName} = ct_slave:stop(Node) + end, {{nodedown,NodeName},{sys,terminate,_}} = - ?EXPECT_FAILURE(gen_statem:stop(Pid), Reason2), + ?EXPECT_FAILURE(gen_statem:stop(Statem), Reason2), ok. %% Registered name on remote node @@ -474,21 +479,26 @@ stop9(Config) -> LocalSTM = {local,Name}, Node = gen_statem__stop9, {ok,NodeName} = ct_slave:start(Node), - STM = {Name,NodeName}, - Dir = filename:dirname(code:which(?MODULE)), - rpc:call(NodeName, code, add_path, [Dir]), - {ok,Pid} = - rpc:call( - NodeName, gen_statem, start, - [LocalSTM,?MODULE,start_arg(Config, []),[]]), - ok = gen_statem:stop(STM), - undefined = rpc:call(NodeName,erlang,whereis,[Name]), - false = rpc:call(NodeName,erlang,is_process_alive,[Pid]), - noproc = - ?EXPECT_FAILURE(gen_statem:stop(STM), Reason1), - {ok,NodeName} = ct_slave:stop(Node), + Statem = + try + STM = {Name,NodeName}, + Dir = filename:dirname(code:which(?MODULE)), + rpc:block_call(NodeName, code, add_path, [Dir]), + {ok,Pid} = + rpc:block_call( + NodeName, gen_statem, start, + [LocalSTM,?MODULE,start_arg(Config, []),[]]), + ok = gen_statem:stop(STM), + undefined = rpc:block_call(NodeName,erlang,whereis,[Name]), + false = rpc:block_call(NodeName,erlang,is_process_alive,[Pid]), + noproc = + ?EXPECT_FAILURE(gen_statem:stop(STM), Reason1), + STM + after + {ok,NodeName} = ct_slave:stop(Node) + end, {{nodedown,NodeName},{sys,terminate,_}} = - ?EXPECT_FAILURE(gen_statem:stop(STM), Reason2), + ?EXPECT_FAILURE(gen_statem:stop(Statem), Reason2), ok. %% Globally registered name on remote node @@ -496,18 +506,21 @@ stop10(Config) -> Node = gen_statem_stop10, STM = {global,to_stop}, {ok,NodeName} = ct_slave:start(Node), - Dir = filename:dirname(code:which(?MODULE)), - rpc:call(NodeName,code,add_path,[Dir]), - {ok,Pid} = - rpc:call( - NodeName, gen_statem, start, - [STM,?MODULE,start_arg(Config, []),[]]), - global:sync(), - ok = gen_statem:stop(STM), - false = rpc:call(NodeName, erlang, is_process_alive, [Pid]), - noproc = - ?EXPECT_FAILURE(gen_statem:stop(STM), Reason1), - {ok,NodeName} = ct_slave:stop(Node), + try + Dir = filename:dirname(code:which(?MODULE)), + rpc:block_call(NodeName,code,add_path,[Dir]), + {ok,Pid} = + rpc:block_call( + NodeName, gen_statem, start, + [STM,?MODULE,start_arg(Config, []),[]]), + global:sync(), + ok = gen_statem:stop(STM), + false = rpc:block_call(NodeName, erlang, is_process_alive, [Pid]), + noproc = + ?EXPECT_FAILURE(gen_statem:stop(STM), Reason1) + after + {ok,NodeName} = ct_slave:stop(Node) + end, noproc = ?EXPECT_FAILURE(gen_statem:stop(STM), Reason2), ok. @@ -521,10 +534,10 @@ abnormal1(Config) -> gen_statem:start(LocalSTM, ?MODULE, start_arg(Config, []), []), %% timeout call. - delayed = gen_statem:call(Name, {delayed_answer,1}, 100), + delayed = gen_statem:call(Name, {delayed_answer,100}, 2000), {timeout,_} = ?EXPECT_FAILURE( - gen_statem:call(Name, {delayed_answer,1000}, 10), + gen_statem:call(Name, {delayed_answer,2000}, 100), Reason), ok = gen_statem:stop(Name), ?t:sleep(1100), @@ -1424,7 +1437,7 @@ hibernate(Config) -> {ok,Pid0} = gen_statem:start_link( ?MODULE, start_arg(Config, hiber_now), []), - is_in_erlang_hibernate(Pid0), + wait_erlang_hibernate(Pid0), stop_it(Pid0), receive {'EXIT',Pid0,normal} -> ok @@ -1437,23 +1450,23 @@ hibernate(Config) -> true = ({current_function,{erlang,hibernate,3}} =/= erlang:process_info(Pid,current_function)), hibernating = gen_statem:call(Pid, hibernate_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), good_morning = gen_statem:call(Pid, wakeup_sync), is_not_in_erlang_hibernate(Pid), hibernating = gen_statem:call(Pid, hibernate_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), please_just_five_more = gen_statem:call(Pid, snooze_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), good_morning = gen_statem:call(Pid, wakeup_sync), is_not_in_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, hibernate_async), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, wakeup_async), is_not_in_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, hibernate_async), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, snooze_async), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, wakeup_async), is_not_in_erlang_hibernate(Pid), @@ -1461,14 +1474,14 @@ hibernate(Config) -> true = ({current_function,{erlang,hibernate,3}} =/= erlang:process_info(Pid, current_function)), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), 'alive!' = gen_statem:call(Pid, 'alive?'), true = ({current_function,{erlang,hibernate,3}} =/= erlang:process_info(Pid, current_function)), Pid ! hibernate_now, - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), 'alive!' = gen_statem:call(Pid, 'alive?'), true = @@ -1476,34 +1489,34 @@ hibernate(Config) -> erlang:process_info(Pid, current_function)), hibernating = gen_statem:call(Pid, hibernate_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), good_morning = gen_statem:call(Pid, wakeup_sync), is_not_in_erlang_hibernate(Pid), hibernating = gen_statem:call(Pid, hibernate_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), please_just_five_more = gen_statem:call(Pid, snooze_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), good_morning = gen_statem:call(Pid, wakeup_sync), is_not_in_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, hibernate_async), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, wakeup_async), is_not_in_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, hibernate_async), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, snooze_async), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), ok = gen_statem:cast(Pid, wakeup_async), is_not_in_erlang_hibernate(Pid), hibernating = gen_statem:call(Pid, hibernate_sync), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), sys:suspend(Pid), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), sys:resume(Pid), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), receive after 1000 -> ok end, - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), good_morning = gen_statem:call(Pid, wakeup_sync), is_not_in_erlang_hibernate(Pid), @@ -1519,15 +1532,16 @@ hibernate(Config) -> %% Auto-hibernation timeout auto_hibernate(Config) -> OldFl = process_flag(trap_exit, true), - HibernateAfterTimeout = 100, + HibernateAfterTimeout = 1000, {ok,Pid} = gen_statem:start_link( - ?MODULE, start_arg(Config, []), [{hibernate_after, HibernateAfterTimeout}]), + ?MODULE, start_arg(Config, []), + [{hibernate_after, HibernateAfterTimeout}]), %% After init test is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), %% After info test Pid ! {hping, self()}, receive @@ -1538,7 +1552,7 @@ auto_hibernate(Config) -> end, is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), %% After cast test ok = gen_statem:cast(Pid, {hping, self()}), receive @@ -1549,42 +1563,42 @@ auto_hibernate(Config) -> end, is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), %% After call test hpong = gen_statem:call(Pid, hping), is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), %% Timer test 1 - TimerTimeout1 = 50, - ok = gen_statem:call(Pid, {arm_htimer, self(), TimerTimeout1}), + TimerTimeout1 = HibernateAfterTimeout div 2, + ok = gen_statem:call(Pid, {start_htimer, self(), TimerTimeout1}), is_not_in_erlang_hibernate(Pid), timer:sleep(TimerTimeout1), is_not_in_erlang_hibernate(Pid), receive - {Pid, htimer_armed} -> + {Pid, htimer_timeout} -> ok after 1000 -> ct:fail(timer1) end, is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), %% Timer test 2 - TimerTimeout2 = 150, - ok = gen_statem:call(Pid, {arm_htimer, self(), TimerTimeout2}), + TimerTimeout2 = HibernateAfterTimeout * 2, + ok = gen_statem:call(Pid, {start_htimer, self(), TimerTimeout2}), is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), receive - {Pid, htimer_armed} -> + {Pid, htimer_timeout} -> ok - after 1000 -> + after TimerTimeout2 -> ct:fail(timer2) end, is_not_in_erlang_hibernate(Pid), timer:sleep(HibernateAfterTimeout), - is_in_erlang_hibernate(Pid), + wait_erlang_hibernate(Pid), stop_it(Pid), process_flag(trap_exit, OldFl), receive @@ -1594,38 +1608,38 @@ auto_hibernate(Config) -> end, ok = verify_empty_msgq(). -is_in_erlang_hibernate(Pid) -> + +wait_erlang_hibernate(Pid) -> receive after 1 -> ok end, - is_in_erlang_hibernate_1(200, Pid). + wait_erlang_hibernate_1(200, Pid). -is_in_erlang_hibernate_1(0, Pid) -> +wait_erlang_hibernate_1(0, Pid) -> ct:log("~p\n", [erlang:process_info(Pid, current_function)]), - ct:fail(not_in_erlang_hibernate_3); -is_in_erlang_hibernate_1(N, Pid) -> + ct:fail(should_be_in_erlang_hibernate_3); +wait_erlang_hibernate_1(N, Pid) -> {current_function,MFA} = erlang:process_info(Pid, current_function), case MFA of {erlang,hibernate,3} -> ok; _ -> receive after 10 -> ok end, - is_in_erlang_hibernate_1(N-1, Pid) + wait_erlang_hibernate_1(N-1, Pid) end. is_not_in_erlang_hibernate(Pid) -> receive after 1 -> ok end, is_not_in_erlang_hibernate_1(200, Pid). -is_not_in_erlang_hibernate_1(0, Pid) -> - ct:log("~p\n", [erlang:process_info(Pid, current_function)]), - ct:fail(not_in_erlang_hibernate_3); +is_not_in_erlang_hibernate_1(0, _Pid) -> + ct:fail(should_not_be_in_erlang_hibernate_3); is_not_in_erlang_hibernate_1(N, Pid) -> {current_function,MFA} = erlang:process_info(Pid, current_function), case MFA of - {erlang,hibernate,3} -> + {erlang,hibernate,3} -> receive after 10 -> ok end, is_not_in_erlang_hibernate_1(N-1, Pid); - _ -> - ok + _ -> + ok end. @@ -2340,13 +2354,13 @@ init(stop_shutdown) -> {stop,shutdown}; init(sleep) -> ?t:sleep(1000), - {ok,idle,data}; + init_sup({ok,idle,data}); init(hiber) -> - {ok,hiber_idle,[]}; + init_sup({ok,hiber_idle,[]}); init(hiber_now) -> - {ok,hiber_idle,[],[hibernate]}; + init_sup({ok,hiber_idle,[],[hibernate]}); init({data, Data}) -> - {ok,idle,Data}; + init_sup({ok,idle,Data}); init({callback_mode,CallbackMode,Arg}) -> ets:new(?MODULE, [named_table,private]), ets:insert(?MODULE, {callback_mode,CallbackMode}), @@ -2356,14 +2370,35 @@ init({map_statem,#{init := Init}=Machine,Modes}) -> ets:insert(?MODULE, {callback_mode,[handle_event_function|Modes]}), case Init() of {ok,State,Data,Ops} -> - {ok,State,[Data|Machine],Ops}; + init_sup({ok,State,[Data|Machine],Ops}); {ok,State,Data} -> - {ok,State,[Data|Machine]}; + init_sup({ok,State,[Data|Machine]}); Other -> - Other + init_sup(Other) end; init([]) -> - {ok,idle,data}. + init_sup({ok,idle,data}). + +%% Supervise state machine parent i.e the test case, and if it dies +%% (fails due to some reason), kill the state machine, +%% just to not leak resources (process, name, ETS table, etc...) +%% +init_sup(Result) -> + Parent = gen:get_parent(), + Statem = self(), + _Supervisor = + spawn( + fun () -> + StatemRef = monitor(process, Statem), + ParentRef = monitor(process, Parent), + receive + {'DOWN', StatemRef, _, _, Reason} -> + exit(Reason); + {'DOWN', ParentRef, _, _, _} -> + exit(Statem, kill) + end + end), + Result. callback_mode() -> try ets:lookup(?MODULE, callback_mode) of @@ -2396,10 +2431,10 @@ idle(cast, {hping,Pid}, Data) -> {keep_state, Data}; idle({call, From}, hping, _Data) -> {keep_state_and_data, [{reply, From, hpong}]}; -idle({call, From}, {arm_htimer, Pid, Timeout}, _Data) -> - {keep_state_and_data, [{reply, From, ok}, {timeout, Timeout, {arm_htimer, Pid}}]}; -idle(timeout, {arm_htimer, Pid}, _Data) -> - Pid ! {self(), htimer_armed}, +idle({call, From}, {start_htimer, Pid, Timeout}, _Data) -> + {keep_state_and_data, [{reply, From, ok}, {timeout, Timeout, {htimer, Pid}}]}; +idle(timeout, {htimer, Pid}, _Data) -> + Pid ! {self(), htimer_timeout}, keep_state_and_data; idle(cast, {connect,Pid}, Data) -> Pid ! accept, diff --git a/lib/stdlib/test/gen_statem_SUITE_data/oc_statem.erl b/lib/stdlib/test/gen_statem_SUITE_data/oc_statem.erl index 1bcd08867f..1de7de527b 100644 --- a/lib/stdlib/test/gen_statem_SUITE_data/oc_statem.erl +++ b/lib/stdlib/test/gen_statem_SUITE_data/oc_statem.erl @@ -31,6 +31,24 @@ start(Opts) -> gen_statem:start({local, ?MODULE}, ?MODULE, [], Opts). init([]) -> + %% Supervise state machine parent i.e the test case, and if it dies + %% (fails due to some reason), kill the state machine, + %% just to not leak resources (process, name, ETS table, etc...) + %% + Parent = gen:get_parent(), + Statem = self(), + _Supervisor = + spawn( + fun () -> + StatemRef = monitor(process, Statem), + ParentRef = monitor(process, Parent), + receive + {'DOWN', StatemRef, _, _, Reason} -> + exit(Reason); + {'DOWN', ParentRef, _, _, _} -> + exit(Statem, kill) + end + end), {ok, start, #{}}. callback_mode() -> diff --git a/lib/stdlib/test/property_test/shell_docs_prop.erl b/lib/stdlib/test/property_test/shell_docs_prop.erl new file mode 100644 index 0000000000..83b62e8837 --- /dev/null +++ b/lib/stdlib/test/property_test/shell_docs_prop.erl @@ -0,0 +1,135 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2020. All Rights Reserved. +%% +%% Licensed under the Apache License, Version 2.0 (the "License"); +%% you may not use this file except in compliance with the License. +%% You may obtain a copy of the License at +%% +%% http://www.apache.org/licenses/LICENSE-2.0 +%% +%% Unless required by applicable law or agreed to in writing, software +%% distributed under the License is distributed on an "AS IS" BASIS, +%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +%% See the License for the specific language governing permissions and +%% limitations under the License. +%% +%% %CopyrightEnd% +%% +-module(shell_docs_prop). +-export([prop_render/0]). + +%%% This will include the .hrl file for the installed testing tool: +-include_lib("common_test/include/ct_property_test.hrl"). +-include_lib("kernel/include/eep48.hrl"). +-compile([export_all]). + +prop_render() -> + numtests(10000, + ?FORALL(Doc, + ?LET(Blocks,blocks(), + #docs_v1{ module_doc = #{ <<"en">> => Blocks }, docs = [] } + ), + begin + try + shell_docs:render(test, Doc), + shell_docs:validate(Doc), + N1 = shell_docs:normalize(maps:get(<<"en">>,Doc#docs_v1.module_doc)), + N1 = shell_docs:normalize(N1), + true + catch C:E:ST -> + ct:pal("~p ~p:~p ~p~n",[Doc,C,E,ST]), + false + end + end)). + +-define(LIMIT,5). + +blocks() -> + blocks([]). + +blocks([l|_] = S) -> + ?LAZY( + frequency( + [{2,[]}, + {2,[{li,[],blocks([li | S])}]}]) + ); +blocks([dl|_] = S) -> + ?LAZY( + frequency( + [{2,[]}, + {2,[{dt,[],blocks([dt | S])}]}, + {2,[{dd,[],blocks([dd | S])}]}]) + ); +blocks(S) -> + ?LAZY( + frequency( + [{?LIMIT div 2,[]}, + {max(1,?LIMIT - length(S)), + ?LET(Lst,[block(S)|blocks(S)],lists:flatten(Lst))}, + {max(1,?LIMIT - length(S)), + ?LET(Lst,[inlines()|blocks(S)],lists:flatten(Lst))} + ] + )). + +inlines() -> + inlines([]). +inlines(S) -> + ?LAZY( + frequency( + [{?LIMIT,[]}, + {max(1,?LIMIT - length(S)),[inline(S)|inlines(S)]} + ] + )). + +block(S) -> + frequency( + fmax('div',3,S,{'div',oneof([[],[{class,<<"Warning">>}]]), blocks(['div'|S])}) ++ + fmax(p,1,S,{p,[],blocks([p|S])}) ++ + fmax(l,3,S,{ul,[],blocks([l|S])}) ++ + fmax(l,3,S,{ol,[],blocks([l|S])}) ++ + fmax(dl,3,S,{dl,[],blocks([dl|S])}) ++ + fmax(['div',l,dl],0,S,{h1,[],inlines(['div'|S])}) ++ + fmax(['div',l,dl],0,S,{h2,[],inlines(['div'|S])}) ++ + fmax(['div',l,dl],1,S,{h3,[],inlines(['div'|S])}) ++ + [{5,{br,[],[]}}] + ). + +inline(S) -> + frequency( + fmax(i,1,S,{i,[],?LAZY(inlines([i|S]))}) ++ + fmax(code,1,S,{code,[],?LAZY(inlines([code|S]))}) ++ + fmax(a,1,S,{a,[],?LAZY(inlines([a|S]))}) ++ + fmax(em,1,S,{em,[],?LAZY(inlines([em|S]))}) ++ + [{10,characters()}]). + +characters() -> + ?LET(Str,list(frequency([{10,printable_character()},{1,char()}])), + unicode:characters_to_binary(Str)). + +printable_character() -> + oneof([integer($\040,$\176), + integer(16#A0, 16#D800-1), + integer(16#DFFF+1,16#FFFE-1), + integer(16#FFFF+1,16#10FFFF), + $\n,$\r,$\t,$\v,$\b,$\f,$\e]). + +fmax(What,Depth,S,E) when not is_list(What) -> + fmax([What],Depth,S,E); +fmax(What,Depth,S,E) -> + Cnt = + lists:foldl( + fun(E,Cnt) -> + case lists:member(E,What) of + true -> + Cnt+1; + false -> + Cnt + end + end, 0, S), + if Depth-Cnt =< 0 -> + []; + true -> + [{10 - (Depth-Cnt),E}] + end. diff --git a/lib/stdlib/test/re_SUITE_data/testoutput2 b/lib/stdlib/test/re_SUITE_data/testoutput2 index 4ccda27201..f5d32d6ad0 100644 --- a/lib/stdlib/test/re_SUITE_data/testoutput2 +++ b/lib/stdlib/test/re_SUITE_data/testoutput2 @@ -5614,9 +5614,8 @@ No match 123456\P No match -//KF>testsavedregex +//S-KF>testsavedregex Compiled pattern written to testsavedregex -Study data written to testsavedregex /abc/IS>testsavedregex Capturing subpattern count = 0 diff --git a/lib/stdlib/test/shell_docs_SUITE.erl b/lib/stdlib/test/shell_docs_SUITE.erl index d8241f7530..11282db3e7 100644 --- a/lib/stdlib/test/shell_docs_SUITE.erl +++ b/lib/stdlib/test/shell_docs_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2004-2019. All Rights Reserved. +%% Copyright Ericsson AB 2020. All Rights Reserved. %% %% Licensed under the Apache License, Version 2.0 (the "License"); %% you may not use this file except in compliance with the License. @@ -18,35 +18,39 @@ %% %CopyrightEnd% %% -module(shell_docs_SUITE). --export([all/0, suite/0,groups/0,init_per_suite/1, end_per_suite/1, - init_per_group/2,end_per_group/2]). --export([init_per_testcase/2, end_per_testcase/2]). +-export([all/0, suite/0, groups/0, init_per_suite/1, end_per_suite/1, + init_per_group/2, end_per_group/2]). --export([render/1, links/1]). +-export([render/1, links/1, normalize/1, render_prop/1]). --include_lib("kernel/include/eep48.hrl"). - -init_per_testcase(_Case, Config) -> - Config. +-export([render_all/1]). -end_per_testcase(_Case, _Config) -> - ok. +-include_lib("kernel/include/eep48.hrl"). +-include_lib("stdlib/include/assert.hrl"). suite() -> [{timetrap,{minutes,10}}]. all() -> - [render, links]. + [render, links, normalize, {group, prop}]. groups() -> - []. + [{prop,[],[render_prop]}]. +%% Include a spec here in order to test that specs of undocumented functions +%% is rendered correctly. +-spec init_per_suite(Config1) -> Config2 when + Config1 :: list({atom(),term()}), + Config2 :: list({atom(),term()}). init_per_suite(Config) -> + {ok, ?MODULE} = c:c(?MODULE,[debug_info]), Config. end_per_suite(_Config) -> ok. +init_per_group(prop, Config) -> + ct_property_test:init_per_suite(Config); init_per_group(_GroupName, Config) -> Config. @@ -59,6 +63,7 @@ render(_Config) -> try shell_docs:render(Mod, D), shell_docs:render_type(Mod, D), + shell_docs:render_callback(Mod, D), [try shell_docs:render(Mod, F, A, D) catch _E:R:ST -> @@ -72,12 +77,25 @@ render(_Config) -> io:format("Failed to render type ~p:~p/~p~n~p:~p~n~p~n", [Mod,T,A,R,ST,shell_docs:get_type_doc(Mod,T,A)]), erlang:raise(error,R,ST) - end || {{type,T,A},_,_,_,_} <- Docs] + end || {{type,T,A},_,_,_,_} <- Docs], + [try + shell_docs:render_callback(Mod, T, A, D) + catch _E:R:ST -> + io:format("Failed to render callback ~p:~p/~p~n~p:~p~n~p~n", + [Mod,T,A,R,ST,shell_docs:get_callback_doc(Mod,T,A)]), + erlang:raise(error,R,ST) + end || {{callback,T,A},_,_,_,_} <- Docs] catch throw:R:ST -> io:format("Failed to render ~p~n~p:~p~n",[Mod,R,ST]), exit(R) end - end). + end), + ok. + +render_prop(Config) -> +% dbg:tracer(),dbg:p(all,c),dbg:tpl(shell_docs_prop,[]), + ct_property_test:quickcheck( + shell_docs_prop:prop_render(),Config). links(_Config) -> docsmap( @@ -132,6 +150,17 @@ check_links(Mod, [C|T]) when is_binary(C) -> check_links(_, []) -> ok. +normalize(_Config) -> + ?assertMatch( + [{p,[],[{em,[],[<<"a ">>,{code,[],[<<"b ">>]},<<"c">>]}]}], + shell_docs:normalize([{p,[],[{em,[],[<<" a ">>,{code,[],[<<" b ">>]},<<" c">>]}]}]) + ), + ?assertMatch( + [{'div',[],[<<"!">>]}], + shell_docs:normalize([{'div',[],[{code,[],[<<" ">>,{i,[],[<<" ">>]}]},<<" !">>]}]) + ), + ok. + %% Special binary_to_atom that deals with <<"'and'">> b2a(Bin) -> case erl_scan:string(binary_to_list(Bin)) of @@ -139,21 +168,61 @@ b2a(Bin) -> {ok,[{A,_}],_} -> A end. +%% Testing functions +render_all(Dir) -> + file:make_dir(Dir), + docsmap( + fun(Mod, #docs_v1{ docs = Docs } = D) -> + SMod = atom_to_list(Mod), + file:write_file(filename:join(Dir,SMod ++ ".txt"), + unicode:characters_to_binary(shell_docs:render(Mod, D))), + file:write_file(filename:join(Dir,SMod ++ "_type.txt"), + unicode:characters_to_binary(shell_docs:render_type(Mod, D))), + file:write_file(filename:join(Dir,SMod ++ "_cb.txt"), + unicode:characters_to_binary(shell_docs:render_callback(Mod, D))), + lists:foreach( + fun({{function,Name,Arity},_Anno,_Sig,_Doc,_Meta}) -> + FName = SMod ++ "_"++atom_to_list(Name)++"_"++integer_to_list(Arity)++"_func.txt", + ok = file:write_file(filename:join(Dir,re:replace(FName,"[/:]","_", + [global,{return,list}])), + unicode:characters_to_binary(shell_docs:render(Mod, Name, Arity, D))); + ({{type,Name,Arity},_Anno,_Sig,_Doc,_Meta}) -> + FName = SMod ++ "_"++atom_to_list(Name)++"_"++integer_to_list(Arity)++"_type.txt", + ok = file:write_file(filename:join(Dir,re:replace(FName,"[/:]","_", + [global,{return,list}])), + unicode:characters_to_binary(shell_docs:render_type(Mod, Name, Arity, D))); + ({{callback,Name,Arity},_Anno,_Sig,_Doc,_Meta}) -> + FName = SMod ++ "_"++atom_to_list(Name)++"_"++integer_to_list(Arity)++"_cb.txt", + file:write_file(filename:join(Dir,re:replace(FName,"[/:]","_", + [global,{return,list}])), + unicode:characters_to_binary(shell_docs:render_callback(Mod, Name, Arity, D))) + end, Docs) + end). + docsmap(Fun) -> - lists:map(fun F({Mod,_,_}) -> - F(Mod); - F(Mod) when is_list(Mod) -> - F(list_to_atom(Mod)); - F(Mod) -> - case code:get_doc(Mod) of - {error, missing} -> - ok; - {error, cover_compiled} -> - ok; - {error, eacces} -> - %% This can happen in BSD's for some reason... - ok; - {ok, Docs} -> - Fun(Mod, Docs) + lists:map( + fun F({Mod,_,_}) -> + F(Mod); + F(Mod) when is_list(Mod) -> + F(list_to_atom(Mod)); + F(Mod) -> + case code:get_doc(Mod) of + {error, missing} -> + ok; + {error, cover_compiled} -> + ok; + {error, E} when E =:= eperm; E =:= eacces -> + %% This can happen in BSD's for some reason... + ok; + {error, eisdir} -> + %% Uhm? + ok; + {ok, Docs} -> + try + Fun(Mod, Docs) + catch E:R:ST -> + io:format("Failed to render ~p~n~p:~p:~p~n",[Mod,E,R,ST]), + erlang:raise(E,R,ST) end - end, code:all_available()). + end + end, code:all_available()). diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index 02eee400bf..3aa3690d12 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 3.12.1 +STDLIB_VSN = 3.13 diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index 9963ac41ae..52f085bf0c 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -32,6 +32,28 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 2.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Remove incomplete support for <c>cond</c> + expressions. </p> + <p> + Own Id: OTP-15925 Aux Id: PR-2304 </p> + </item> + <item> + <p> + Improved indentation for code generated with + <c>erl_prettypr</c> and <c>tidier</c>.</p> + <p> + Own Id: OTP-16386 Aux Id: PR-2451 </p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 2.2.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index 9e6967d45d..87167529c3 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 2.2.1 +SYNTAX_TOOLS_VSN = 2.3 diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index 728343a86f..f4d2f0772f 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -31,6 +31,33 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 3.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Updates for new <c>erlang:term_to_iovec()</c> BIF.</p> + <p> + Own Id: OTP-16128 Aux Id: OTP-15618 </p> + </item> + <item> + <p>Improved the presentation of allocations and carriers + in the <c>instrument</c> module.</p> + <p> + Own Id: OTP-16327</p> + </item> + <item> + <p> + Minor updates due to the new spawn improvements made.</p> + <p> + Own Id: OTP-16368 Aux Id: OTP-15251 </p> + </item> + </list> + </section> + +</section> + <section><title>Tools 3.3.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/tools/src/tools.app.src b/lib/tools/src/tools.app.src index beb5b98e15..f0c7ec1ead 100644 --- a/lib/tools/src/tools.app.src +++ b/lib/tools/src/tools.app.src @@ -43,6 +43,6 @@ ] }, {runtime_dependencies, ["stdlib-3.4","runtime_tools-1.8.14", - "kernel-5.4","erts-9.1","compiler-5.0", "erts-@OTP-16327@"]} + "kernel-5.4","erts-9.1","compiler-5.0", "erts-11.0"]} ] }. diff --git a/lib/tools/test/fprof_SUITE.erl b/lib/tools/test/fprof_SUITE.erl index 898d20f560..f5d68c20f0 100644 --- a/lib/tools/test/fprof_SUITE.erl +++ b/lib/tools/test/fprof_SUITE.erl @@ -501,7 +501,11 @@ cpu_create_file_slow(Config) when is_list(Config) -> %% {ok, [T, P]} = parse(AnalysisFile), io:format("~p~n~n~p~n", [P, ets:tab2list(T)]), - ok = verify(T, P), + try + ok = verify(T, P) + catch throw:{negative,_Weird} -> + io:format("Aborted as counter is negative because of bad cpu_time") + end, Proc = pid_to_list(self()), case P of [{analysis_options, _}, @@ -520,13 +524,7 @@ cpu_create_file_slow(Config) when is_list(Config) -> io:format("cpu_ts:~w, fprof:~w~n", [Acc, Acc1]), {comment, io_lib:format("~p% cpu utilization", [100*divide(Acc,Acc1)])}; {'EXIT', not_supported} -> - case {os:type(), os:version()} of - {{unix, sunos}, {Major, Minor, _}} - when Major >= 5, Minor >= 7 -> - ct:fail(Result); - _ -> - {skipped, "not_supported"} - end; + {skipped, "not_supported"}; _ -> ct:fail(Result) end, @@ -633,6 +631,8 @@ verify(Tab, [{analysis_options, _}, {Proc, Cnt_P, Acc_P, Own_P} = Clocks when Acc_P >= Own_P -> Clocks; + {_, _, Acc_p, _} = Weird when Acc_p < 0 -> + throw({negative, Weird}); Weird -> throw({error, [?MODULE, ?LINE, Weird]}) end diff --git a/lib/tools/test/prof_bench_SUITE.erl b/lib/tools/test/prof_bench_SUITE.erl index e8c4642d37..c0b9a7913c 100644 --- a/lib/tools/test/prof_bench_SUITE.erl +++ b/lib/tools/test/prof_bench_SUITE.erl @@ -32,7 +32,7 @@ suite() -> - [{timetrap,{minutes,10}}]. + [{timetrap,{minutes,15}}]. all() -> [overhead]. diff --git a/lib/tools/vsn.mk b/lib/tools/vsn.mk index 66f2c03149..b9f4811392 100644 --- a/lib/tools/vsn.mk +++ b/lib/tools/vsn.mk @@ -1 +1 @@ -TOOLS_VSN = 3.3.1 +TOOLS_VSN = 3.4 diff --git a/lib/wx/c_src/egl_impl.cpp b/lib/wx/c_src/egl_impl.cpp index 61e05ee6f1..06c1de9cb6 100644 --- a/lib/wx/c_src/egl_impl.cpp +++ b/lib/wx/c_src/egl_impl.cpp @@ -48,6 +48,7 @@ int egl_initiated = 0; #define OPENGLU_LIB L"glu32.dll" typedef HMODULE DL_LIB_P; typedef WCHAR DL_CHAR; +#define DL_STR_FMT "%S" void * dlsym(HMODULE Lib, const char *func) { void * funcp; if((funcp = (void *) GetProcAddress(Lib, func))) @@ -67,6 +68,7 @@ void dlclose(HMODULE Lib) { #else typedef void * DL_LIB_P; typedef char DL_CHAR; +# define DL_STR_FMT "%s" # ifdef _MACOSX # define OPENGL_LIB "/System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib" # define OPENGLU_LIB "/System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLU.dylib" @@ -125,7 +127,7 @@ int load_gl_functions() { // dlclose(LIBhandle); // fprintf(stderr, "OPENGL library is loaded\r\n"); } else { - fprintf(stderr, "Could NOT load OpenGL library: %s\r\n", DLName); + fprintf(stderr, "Could NOT load OpenGL library: " DL_STR_FMT "\r\n", DLName); }; DLName = (DL_CHAR *) OPENGLU_LIB; @@ -154,7 +156,7 @@ int load_gl_functions() { // dlclose(LIBhandle); // fprintf(stderr, "GLU library is loaded\r\n"); } else { - fprintf(stderr, "Could NOT load OpenGL GLU library: %s\r\n", DLName); + fprintf(stderr, "Could NOT load OpenGL GLU library: " DL_STR_FMT "\r\n", DLName); }; return 1; diff --git a/lib/wx/doc/src/notes.xml b/lib/wx/doc/src/notes.xml index 7dcfbb1588..b9012054a8 100644 --- a/lib/wx/doc/src/notes.xml +++ b/lib/wx/doc/src/notes.xml @@ -32,6 +32,33 @@ <p>This document describes the changes made to the wxErlang application.</p> +<section><title>Wx 1.9.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Fix various compiler warnings on 64-bit Windows.</p> + <p> + Own Id: OTP-15800</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Refactored the internal handling of deprecated and + removed functions.</p> + <p> + Own Id: OTP-16469</p> + </item> + </list> + </section> + +</section> + <section><title>Wx 1.9</title> <section><title>Improvements and New Features</title> diff --git a/lib/wx/vsn.mk b/lib/wx/vsn.mk index b498d21f3f..552e09ee2a 100644 --- a/lib/wx/vsn.mk +++ b/lib/wx/vsn.mk @@ -1 +1 @@ -WX_VSN = 1.9 +WX_VSN = 1.9.1 diff --git a/lib/xmerl/doc/src/notes.xml b/lib/xmerl/doc/src/notes.xml index 997af9d037..d8b2852097 100644 --- a/lib/xmerl/doc/src/notes.xml +++ b/lib/xmerl/doc/src/notes.xml @@ -32,6 +32,22 @@ <p>This document describes the changes made to the Xmerl application.</p> +<section><title>Xmerl 1.3.25</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix a bug that the function name didn't get + normalized in some case which left white spaces in links. + </p> + <p> + Own Id: OTP-16617</p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.3.24</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/xmerl/vsn.mk b/lib/xmerl/vsn.mk index 79be4c8a95..8711ed946f 100644 --- a/lib/xmerl/vsn.mk +++ b/lib/xmerl/vsn.mk @@ -1 +1 @@ -XMERL_VSN = 1.3.24 +XMERL_VSN = 1.3.25 diff --git a/make/otp_patch_solve_forward_merge_version b/make/otp_patch_solve_forward_merge_version index 8351c19397..60d3b2f4a4 100644 --- a/make/otp_patch_solve_forward_merge_version +++ b/make/otp_patch_solve_forward_merge_version @@ -1 +1 @@ -14 +15 diff --git a/make/otp_version_tickets_in_merge b/make/otp_version_tickets_in_merge index e69de29bb2..1c9a3e3bb9 100644 --- a/make/otp_version_tickets_in_merge +++ b/make/otp_version_tickets_in_merge @@ -0,0 +1,6 @@ +OTP-16582 +OTP-16607 +OTP-16639 +OTP-16652 +OTP-16654 +OTP-16657 diff --git a/otp_versions.table b/otp_versions.table index 212aa72141..82506466b6 100644 --- a/otp_versions.table +++ b/otp_versions.table @@ -1,3 +1,8 @@ +OTP-23.0.1 : compiler-7.6.1 erts-11.0.1 # asn1-5.0.13 common_test-1.19 crypto-4.7 debugger-5.0 dialyzer-4.2 diameter-2.2.3 edoc-0.12 eldap-1.2.8 erl_docgen-1.0 erl_interface-4.0 et-1.6.4 eunit-2.5 ftp-1.0.4 hipe-4.0 inets-7.2 jinterface-1.11 kernel-7.0 megaco-3.19 mnesia-4.17 observer-2.9.4 odbc-2.13 os_mon-2.5.2 parsetools-2.2 public_key-1.8 reltool-0.8 runtime_tools-1.15 sasl-4.0 snmp-5.6 ssh-4.10 ssl-10.0 stdlib-3.13 syntax_tools-2.3 tftp-1.0.2 tools-3.4 wx-1.9.1 xmerl-1.3.25 : +OTP-23.0 : asn1-5.0.13 common_test-1.19 compiler-7.6 crypto-4.7 debugger-5.0 dialyzer-4.2 edoc-0.12 erl_docgen-1.0 erl_interface-4.0 erts-11.0 eunit-2.5 hipe-4.0 inets-7.2 jinterface-1.11 kernel-7.0 megaco-3.19 mnesia-4.17 observer-2.9.4 odbc-2.13 os_mon-2.5.2 parsetools-2.2 public_key-1.8 runtime_tools-1.15 sasl-4.0 snmp-5.6 ssh-4.10 ssl-10.0 stdlib-3.13 syntax_tools-2.3 tools-3.4 wx-1.9.1 xmerl-1.3.25 # diameter-2.2.3 eldap-1.2.8 et-1.6.4 ftp-1.0.4 reltool-0.8 tftp-1.0.2 : +OTP-22.3.4.1 : erts-10.7.2.1 kernel-6.5.2.1 megaco-3.18.8.1 # asn1-5.0.12 common_test-1.18.2 compiler-7.5.4 crypto-4.6.5 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4 hipe-3.19.3 inets-7.1.3 jinterface-1.10.1 mnesia-4.16.3 observer-2.9.3 odbc-2.12.4 os_mon-2.5.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14 sasl-3.4.2 snmp-5.5 ssh-4.9.1 ssl-9.6.2 stdlib-3.12.1 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1 wx-1.9 xmerl-1.3.24 : +OTP-22.3.4 : asn1-5.0.12 erts-10.7.2 # common_test-1.18.2 compiler-7.5.4 crypto-4.6.5 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4 hipe-3.19.3 inets-7.1.3 jinterface-1.10.1 kernel-6.5.2 megaco-3.18.8 mnesia-4.16.3 observer-2.9.3 odbc-2.12.4 os_mon-2.5.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14 sasl-3.4.2 snmp-5.5 ssh-4.9.1 ssl-9.6.2 stdlib-3.12.1 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1 wx-1.9 xmerl-1.3.24 : +OTP-22.3.3 : ssh-4.9.1 ssl-9.6.2 # asn1-5.0.11 common_test-1.18.2 compiler-7.5.4 crypto-4.6.5 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2 erts-10.7.1 et-1.6.4 eunit-2.4.1 ftp-1.0.4 hipe-3.19.3 inets-7.1.3 jinterface-1.10.1 kernel-6.5.2 megaco-3.18.8 mnesia-4.16.3 observer-2.9.3 odbc-2.12.4 os_mon-2.5.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14 sasl-3.4.2 snmp-5.5 stdlib-3.12.1 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1 wx-1.9 xmerl-1.3.24 : OTP-22.3.2 : asn1-5.0.11 # common_test-1.18.2 compiler-7.5.4 crypto-4.6.5 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2 erts-10.7.1 et-1.6.4 eunit-2.4.1 ftp-1.0.4 hipe-3.19.3 inets-7.1.3 jinterface-1.10.1 kernel-6.5.2 megaco-3.18.8 mnesia-4.16.3 observer-2.9.3 odbc-2.12.4 os_mon-2.5.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14 sasl-3.4.2 snmp-5.5 ssh-4.9 ssl-9.6.1 stdlib-3.12.1 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1 wx-1.9 xmerl-1.3.24 : OTP-22.3.1 : compiler-7.5.4 erts-10.7.1 inets-7.1.3 ssl-9.6.1 stdlib-3.12.1 xmerl-1.3.24 # asn1-5.0.10 common_test-1.18.2 crypto-4.6.5 debugger-4.2.8 dialyzer-4.1.1 diameter-2.2.3 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 erl_interface-3.13.2 et-1.6.4 eunit-2.4.1 ftp-1.0.4 hipe-3.19.3 jinterface-1.10.1 kernel-6.5.2 megaco-3.18.8 mnesia-4.16.3 observer-2.9.3 odbc-2.12.4 os_mon-2.5.1 parsetools-2.1.8 public_key-1.7.2 reltool-0.8 runtime_tools-1.14 sasl-3.4.2 snmp-5.5 ssh-4.9 syntax_tools-2.2.1 tftp-1.0.2 tools-3.3.1 wx-1.9 : OTP-22.3 : asn1-5.0.10 common_test-1.18.2 compiler-7.5.3 crypto-4.6.5 diameter-2.2.3 erl_interface-3.13.2 erts-10.7 eunit-2.4.1 hipe-3.19.3 kernel-6.5.2 megaco-3.18.8 mnesia-4.16.3 public_key-1.7.2 sasl-3.4.2 snmp-5.5 ssh-4.9 ssl-9.6 stdlib-3.12 tools-3.3.1 # debugger-4.2.8 dialyzer-4.1.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.11 et-1.6.4 ftp-1.0.4 inets-7.1.2 jinterface-1.10.1 observer-2.9.3 odbc-2.12.4 os_mon-2.5.1 parsetools-2.1.8 reltool-0.8 runtime_tools-1.14 syntax_tools-2.2.1 tftp-1.0.2 wx-1.9 xmerl-1.3.23 : @@ -28,6 +33,8 @@ OTP-22.0.3 : compiler-7.4.2 dialyzer-4.0.1 erts-10.4.2 ssl-9.3.2 stdlib-3.9.2 # OTP-22.0.2 : compiler-7.4.1 crypto-4.5.1 erts-10.4.1 stdlib-3.9.1 # asn1-5.0.9 common_test-1.17.3 debugger-4.2.7 dialyzer-4.0 diameter-2.2.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 parsetools-2.1.8 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 ssl-9.3.1 syntax_tools-2.2 tftp-1.0.1 tools-3.2 wx-1.8.8 xmerl-1.3.21 : OTP-22.0.1 : ssl-9.3.1 # asn1-5.0.9 common_test-1.17.3 compiler-7.4 crypto-4.5 debugger-4.2.7 dialyzer-4.0 diameter-2.2.1 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 erts-10.4 et-1.6.4 eunit-2.3.7 ftp-1.0.2 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 parsetools-2.1.8 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 stdlib-3.9 syntax_tools-2.2 tftp-1.0.1 tools-3.2 wx-1.8.8 xmerl-1.3.21 : OTP-22.0 : asn1-5.0.9 common_test-1.17.3 compiler-7.4 crypto-4.5 debugger-4.2.7 dialyzer-4.0 edoc-0.11 eldap-1.2.8 erl_docgen-0.9.1 erl_interface-3.12 erts-10.4 hipe-3.19 inets-7.0.8 jinterface-1.10 kernel-6.4 megaco-3.18.5 mnesia-4.16 observer-2.9.1 odbc-2.12.4 os_mon-2.5 public_key-1.6.7 reltool-0.8 runtime_tools-1.13.3 sasl-3.4 snmp-5.3 ssh-4.7.7 ssl-9.3 stdlib-3.9 syntax_tools-2.2 tools-3.2 wx-1.8.8 xmerl-1.3.21 # diameter-2.2.1 et-1.6.4 eunit-2.3.7 ftp-1.0.2 parsetools-2.1.8 tftp-1.0.1 : +OTP-21.3.8.16 : erts-10.3.5.12 # asn1-5.0.8 common_test-1.17.2.1 compiler-7.3.2 crypto-4.4.2.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2.2 hipe-3.18.3 inets-7.0.7.2 jinterface-1.9.1 kernel-6.3.1.3 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6.1 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6.4 ssl-9.2.3.6 stdlib-3.8.2.4 syntax_tools-2.1.7.1 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : +OTP-21.3.8.15 : erts-10.3.5.11 ssh-4.7.6.4 ssl-9.2.3.6 stdlib-3.8.2.4 # asn1-5.0.8 common_test-1.17.2.1 compiler-7.3.2 crypto-4.4.2.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2.2 hipe-3.18.3 inets-7.0.7.2 jinterface-1.9.1 kernel-6.3.1.3 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6.1 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 syntax_tools-2.1.7.1 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : OTP-21.3.8.14 : erts-10.3.5.10 # asn1-5.0.8 common_test-1.17.2.1 compiler-7.3.2 crypto-4.4.2.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2.2 hipe-3.18.3 inets-7.0.7.2 jinterface-1.9.1 kernel-6.3.1.3 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6.1 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6.3 ssl-9.2.3.5 stdlib-3.8.2.3 syntax_tools-2.1.7.1 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : OTP-21.3.8.13 : erts-10.3.5.9 stdlib-3.8.2.3 # asn1-5.0.8 common_test-1.17.2.1 compiler-7.3.2 crypto-4.4.2.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2.2 hipe-3.18.3 inets-7.0.7.2 jinterface-1.9.1 kernel-6.3.1.3 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6.1 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssh-4.7.6.3 ssl-9.2.3.5 syntax_tools-2.1.7.1 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : OTP-21.3.8.12 : crypto-4.4.2.2 erts-10.3.5.8 ssh-4.7.6.3 # asn1-5.0.8 common_test-1.17.2.1 compiler-7.3.2 debugger-4.2.6 dialyzer-3.3.2 diameter-2.2.1 edoc-0.10 eldap-1.2.7 erl_docgen-0.9 erl_interface-3.11.3 et-1.6.4 eunit-2.3.7 ftp-1.0.2.2 hipe-3.18.3 inets-7.0.7.2 jinterface-1.9.1 kernel-6.3.1.3 megaco-3.18.4 mnesia-4.15.6 observer-2.9 odbc-2.12.3 os_mon-2.4.7 otp_mibs-1.2.1 parsetools-2.1.8 public_key-1.6.6.1 reltool-0.7.8 runtime_tools-1.13.2 sasl-3.3 snmp-5.2.12 ssl-9.2.3.5 stdlib-3.8.2.2 syntax_tools-2.1.7.1 tftp-1.0.1 tools-3.1.0.1 wx-1.8.7 xmerl-1.3.20.1 : diff --git a/scripts/build-otp b/scripts/build-otp index 582c209e4d..f3f7f8ee86 100755 --- a/scripts/build-otp +++ b/scripts/build-otp @@ -58,6 +58,7 @@ if [ "$1" = "docs" ]; then DOC_TARGET=${TRAVIS_TAG:-$DOC_TARGET} TESTROOT=$PWD/$DOC_TARGET do_and_log "Building documentation" make release_docs do_and_log "Linting documentation" make xmllint + do_and_log "Checking html links" scripts/otp_html_check $PWD/$DOC_TARGET doc/index.html do_and_log "Test chunks" erlc lib/stdlib/test/shell_docs_SUITE.erl && ct_run -no_auto_compile -suite shell_docs_SUITE -case render # The code below prepares this build to be used as a deploy to diff --git a/scripts/otp_html_check b/scripts/otp_html_check new file mode 100755 index 0000000000..62d5b47edd --- /dev/null +++ b/scripts/otp_html_check @@ -0,0 +1,536 @@ +#!/usr/bin/perl -w + +########################################################################### +# +# Find broken links and files not referenced. +# +# Author: Kent Boortz <kent@erix.ericsson.se> +# +########################################################################### + +use File::Find; +use strict; + +undef $/; # No record separator reading files + +########################################################################### +# +# When we talk about "a page" we mean the actual page/file +# When we talk about "a link" we mean a referense to a page/file. +# All links/URL's start with an slash except the top link that is +# the empty string. +# +# So basically we have a set of links and a set of URL's to pages and +# check if this is a valid combination. +# +########################################################################### + +my $debug = 1; +my $expand_url = 0; # If we are to expand an URL with default + # names like "index.html" +my @indexes = # The order to try URL expansion + ( + "index.shtml", + "index.html", + "index.htm", + ); + +my $html_ext = 'shtml|html|htm'; # HTML pages ends in these + +my @links; # Set of [page,link] we want to check +my @exclude; # Pages/dir/prefix to exclude +my %pages; # Set of all files found in the file system + # limited by the script arguments. + # After the spider is done all members in the + # set thas has the value 1 was visited. + +my %missing; # Pages not found "$page$;$link" +my %invalid; # After expansion it is invalid +my %access; # Can't access but exists + +my %anchor_refs; # Absolute links including anchor part +my %anchor_defs; # <a name="..."> in the form "$page#$anchor" + +########################################################################### +# +# Argument processing, see usage() function below +# +########################################################################### + +@ARGV or usage("No base directory given"); +my $base = shift @ARGV; +-d $base or usage("Not a directory: $base"); +$base =~ m&^/& or usage("Has to be absolute path: $base"); +$base =~ s&/+$&&; # Remove ending slash if any + +my $link; +while ($link = shift @ARGV) { + last if $link eq '--'; + $link =~ s&/+$&&; # Remove ending slash if any + $link =~ s&$base&&; # Make absolute URL + $link =~ m&^/& and usage("Invalid start point of HTML tree \"$_\""); + $link = "/$link"; + push(@links,["",$link]); +} + +while ($link = shift @ARGV) { + $link =~ s&/+$&&; # Remove ending slash if any + $link =~ s&$base&&; # Make absolute URL + $link =~ m&^/& and usage("Invalid exclude URL \"$_\""); + $link = "/$link"; + push(@exclude,$link); +} + +# OTP specific + +push(@links,["","/doc/index.html"]) unless @links; + +########################################################################### +# +# Traverse all files and directories and put all possible URL's into +# the set %pages. When we later find a referense to a page that URL +# is removed from the set. When we have followed all links the set +# contains the pages never visited. +# +# We skip files and directories in @exclude. +# +########################################################################### + +find(\&wanted,$base); + +sub wanted { + return unless -f; + return if /^\.info\./; + return if /~$/; + + my $url = $File::Find::name; + $url =~ s&$base&&; + $pages{$url} = 0 unless map {$url =~ m&^$_&} @exclude; +} + + +########################################################################### +# +# Spider that follow all links adding links to the @links set. +# +# @links is expanded, normalized links +# +# We check if there is an valid URL for this link. +# @links may contain links that look bad, this is cleaned up here +# before checking it. +# +########################################################################### + +while (@links) { + my $page_and_link = shift @links; + my ($page,$link) = @$page_and_link; + + # We skip some links directly + + next if $link =~ /^\w{3,10}:/i; + next if $link =~ /cgi-bin|cgiwrap|user-cgi/; + next if $link =~ /^and|or$/; +# next if $link eq ""; + +# print STDERR "1 link: $link\n"; + + $link = expand_link($link,\%pages) if $expand_url; + + unless (exists $pages{$link}) { + # No page for link, mark as invalid + $missing{"$page$;$link"} = 1; + next; + } + +# print STDERR "2 link: $link\n"; + + next if $pages{$link}; # If == 1 it is visited + $pages{$link} = 1; # Mark as visited + +# print STDERR "3 link: $link\n"; + +# next unless $link =~ /\.(shtml|html|htm)$/oi; + next unless $link =~ /\.($html_ext)$/oi; + + push(@links,get_page_links($base,$link)); +} + + +########################################################################### +# +# Read the page and get all the links. We know that the URL for the page +# is absolute and that a page/file exists. +# +########################################################################### + +sub get_page_links { + my $base = shift; + my $page = shift; # Absolute URL + +# print STDERR "open: $page\n"; + + my $path = "$base$page"; + + open(HTML,$path) + or print STDERR "INTERNAL ERROR: Can't open page $page: $!\n"; + + my $html = <HTML>; + close HTML; + +# my $url_base = $page; +# $url_base =~ s&/[^/]+$&&; + + # Remove comments + $html =~ s/\<\!\-\-\s*(.*?)\s*\-\-\>//gs; + +# # Remove comments and expand SSI +# $html =~ s/\<\!\-\-\s*(.*?)\s*\-\-\>/ +# expand_ssi($url_base,$page,$1)/gsie; + + my @links; # Links in this document +# push(@links,$html =~ /\/\*URL\*\/\s*\'([^\']+\.[^\']+)\'/gsi); +# push(@links,$html =~ /=\s*\'([^\']+\.(?:gif|jpg|jpeg))\'/gsi); +# push(@links,$html =~ /option value=\s*\"(\/[^\"]+)\"/gsi); +# push(@links,$html =~ /option value=\s*\"([^\"]+\.[^\"]+)\"/gsi); +# FIXME: This is not working.... +# push(@links,$html =~ /url\s*=\s*([\w-\.\/]+)/gsi); +# push(@links,$html =~ /\"([^\"]+\.html)\"/gsi); + + # Find real HTML links + push(@links,$html =~ /\<\s*\w[^\>]*\sHREF=\s*\"([^\"]*)\"[^\>]*\>/gsi); + push(@links,$html =~ /\<\s*\w[^\>]*\sSRC=\s*\"([^\"]*)\"[^\>]*\>/gsi); + push(@links,$html =~ /\<\s*\w[^\>]*\sLOWSRC=\s*\"([^\"]*)\"[^\>]*\>/gsi); + push(@links,$html =~ /\<\s*\w[^\>]*\sBACKGROUND=\s*\"([^\"]*)\"[^\>]*\>/gsi); + + # FIXME: Now we have the raw links, if we want to complain about + # spaces etc this is the time. + + # Remove referenses to the same page FIXME??? Was removed , why... +# @links = grep {$_ and $_ !~ /^\#/} @links; + + # Find the URL to the current directory + my $rpath = $page; + $rpath =~ s&/[^/]+$&&; # Remove name + + # Links pointing to the same page + # should look the same + map {$_ = normalize_link($page,$rpath,$_)} @links; + +# print "XXX $page\n" if grep {m&lib/asn1-1.3.2/doc/index\.html&} @links; + + map {$_ = [$page,$_]} @links; # Add what page was referensing it + + # Find the anchors + + my @anchors = + ($html =~ m/ + < + \s* + A + [^>]* + \s (?: NAME|ID) \s* = \s* + (?: \"([^\"]*)\" | \'([^\']*)\' | ([^>\s]+) ) + [^>]* + > + /gsix); + + foreach my $anchor (@anchors) { + # FIXME if already there, duplicate + next unless defined $anchor; + $anchor =~ s/%([\da-fA-F]{2})/chr(hex($1))/eg; # Translate hex to char + $anchor =~ s/</</g; # + $anchor =~ s/>/>/g; # + $anchor_defs{"$page#$anchor"} = 1; + } + + return @links; +} + + +# ------------------------------------------------------------------------- +# ------------------------------------------------------------------------- + +sub normalize_link { + my $page = shift; # Page where we found this link + my $rpath = shift; # URL to directory where we found this link + my $link = shift; # The link to normalize + +# print STDERR "\n"; +# print STDERR "1 normalize_link: $link\n"; + + # Handle javascript:erlhref() specially to be able to check those links. + if ($link =~ /^javascript:erlhref\(([^\)]*)\);$/) { + my($up,$part,$mod) = split(/,\s*/, $1); + $up =~ tr/\'//d; + $part =~ tr/\'//d; + $mod =~ tr/\'//d; + my $dir; + if ($part =~ m&^[a-z]+/&) { + $dir = "$base$rpath/${up}/$part"; + } else { + my $path = "$base$rpath/${up}lib/$part/doc/html"; + ($dir) = <$path-*>; + return $link unless defined $dir; + } + $dir =~ s&^$base&&o; + $link = "$dir/$mod"; + } + + return $link if $link =~ /^\w{3,10}:/i; # mailto: http: ..... + + $link =~ s/%([\da-fA-F]{2})/chr(hex($1))/eg; # Translate hex to char + + if ($link eq "") { + # The empty link is a reference to URL directory + return $rpath; + } elsif ($link =~ /^#(.*)$/s) { + # Lokal reference to anchor + my $anchor = $1; + $anchor =~ s/%([\da-fA-F]{2})/chr(hex($1))/eg; # Translate hex to char + $anchor =~ s/</</g; # + $anchor =~ s/>/>/g; # + $anchor =~ s&^\s+&&; # Remove leading any whitespaces + $anchor =~ s&\s+$&&; # Remove trailing any whitespaces + push(@{$anchor_refs{"$page#$anchor"}}, $page); + return $page; + } + + my $anchor = ""; + + if ($link =~ s&#(.*)$&&s) { + # Removed page ref (anchor) + $anchor = $1; + $anchor =~ s/%([\da-fA-F]{2})/chr(hex($1))/eg; # Translate hex to char + $anchor =~ s/</</g; # + $anchor =~ s/>/>/g; # + $anchor =~ s&^\s+&&; # Remove leading any whitespaces + $anchor =~ s&\s+$&&; # Remove trailing any whitespaces + } + + $link = "" if $link eq "/"; + + # Make the link absolute + # FIXME: maybe move down..... + + if ($link !~ m&^/&) { + if ($link) { + $link = "$rpath/$link"; + } else { + $link = $rpath; + } + } + + my $xlink = $link; + + $link =~ s&//+&/&g; # Replace multiple slashes with one slash +# $link =~ s&^(\./)+&&g; # Remove starting dot slash "./" (can't be if absolute) + $link =~ s&(/\.)+$&&; # Remove ending slash dot "/." + $link =~ s&(/\.)+/&/&g; # Remove all slash dot slash "/./" + $link =~ s&/+$&&; # Remove ending slashes + $link =~ s&\?.*$&&; # Remove any query parameters + + # Remove a real directory part followed by ".." + + while ($link =~ s&/[^/]+/\.\.&&) {} + +# print STDERR "4 normalize_link: $link\n"; + + $link = "" if $link eq "/"; # We do this again + + # print STDERR "5 normalize_link: $link\n"; + + push(@{$anchor_refs{"$link#$anchor"}}, $page) if $anchor; + + return $link; +} + + +# ------------------------------------------------------------------------- +# We know the link is normalized +# ------------------------------------------------------------------------- + +sub expand_link { + my $link = shift; + my $pages = shift; + + return $link if exists $pages{$link}; + + my $newlink; + + foreach my $index (@indexes) { + $newlink = "$link/$index"; + return $newlink if exists $pages{$newlink}; + } + + return $link; +} + +########################################################################### +# +# Report the result +# +########################################################################### + +if (keys %missing) { + print "\n\n\n**** Broken links\n\n"; + foreach (sort keys %missing) { + my ($page,$link) = split($;); + print qq(Broken Link: $page -> "$link"\n); + } +} + + +# Entrys in %pages that has the value 0 is not visited +if (keys %pages) { + print "\n\n\n**** Files not used (that I can see)\n\n"; + foreach my $page (sort keys %pages) { + next if $pages{$page}; # If == 1 it is visited + + # OTP specific + + next if $page =~ m&^/(man|pdf|logs|COPYRIGHT|PR.template|README)&; + next if $page =~ m&^/.*\.tar.gz$&; + next if $page =~ m&(/info|\.kwc)$&; + + print qq("$page"\n); + } +} + + +# Remove all references that has a matching NAME=.... +map {delete $anchor_refs{$_}} keys %anchor_defs; + +if (keys %anchor_refs) { + print "\n\n\n**** References to missing anchors\n\n"; + foreach my $ref (sort keys %anchor_refs) { + foreach my $anchor (sort @{$anchor_refs{$ref}}) { + print qq(Missing Anchor: "$ref" from ${anchor}\n); + } + } +} + + +########################################################################### + +sub usage { + print STDERR "ERROR: ",join("\n",@_),"\n" if @_; + print <<HERE; +Usage: $0 BaseDirectory URL [ URLs... ] [ -- ExcludeURLs... ] + +This script try to find out what files are used and not of your +HTML documents, graphic files etc. It doesn't use HTTP, i.e. you +work off-line, so this script may fail to find a link. Javascripts +and other extensions also makes it very hard. But for many sites +it work very well. + +The base directory has to given has to start with a slash. + +For URLs and ExcludeURLs absolute paths or relative the base +directory can be used. + +ExcludeURLs is used as prefixes of directories or files that +should be excluded from the search. + +You call it something like + + % $0 /test/r7a /test/r7a/doc/index.html /test/r7a/lib/*/doc/index.html + +or using relative start points + + % $0 /test/r7a doc/index.html + +HERE + exit 1; +} + + +__END__ + +# FIXME: The order below is important + +if (%access) { + print "\n**** Link exists but can't open\n\n"; + + my $file; + + foreach $file (sort keys %access) { + print "$file\n"; + } +} + + +if (%invalid) { + print "\n**** Invalid links (goes up above top directory)\n\n"; + + foreach (sort keys %invalid) { + my ($page,$link) = split($;,$_); + delete $done{$link}; # FIXME: xxxx + print "$page\n\t-> $link\n"; + } +} + +if (%done) { + print "\n**** Internal error, should be no files here\n\n"; + + foreach (sort keys %done) { + print "$_\n"; + } +} + + +__END__ +########################################################################### + + +sub expand_ssi { + my $url_base = shift; + my $page = shift; + my $comment = shift; # Text between <!-- and --> + + return "" unless $comment =~ s/^\#//; + + # This is an SSI + unless ($comment =~ /([\w-]+)=\"([^\"]+)\"/) { +# print STDERR "WARNING: Unknown SSI $comment\n\ton $page\n"; + return ""; + } + + my $op = lc($1); # Operator + my $inc = $2; # Absolute or relative URL anding in anything + + if ($debug) { + print STDERR "X: url_base = $url_base\n"; + print STDERR "X: page = $page\n"; + print STDERR "X: op = $op\n"; + print STDERR "X: inc = $inc\n"; + print STDERR "X: base = $base\n"; + } + + unless ($op eq 'virtual') { +# print STDERR "WARNING: Unknown SSI $comment\n\ton $page\n"; + return ""; + } + + $inc = make_url_absolute($url_base,$page,$inc); + + my $path = "$base$inc"; + + if ($debug) { + print STDERR "X: inc = $inc\n"; + print STDERR "X: path = $path\n\n"; + } + + unless (open(HTML,$path)) { +# print STDERR "ERROR: Can't open page $inc: $!\n"; + $access{$inc} = 1; + return ""; + } + + my $html = <HTML>; + close HTML; + + $done{$inc} = 1; # Mark done + + return $html; +} + diff --git a/scripts/pre-push b/scripts/pre-push index 7da1f575db..6689daf03a 100755 --- a/scripts/pre-push +++ b/scripts/pre-push @@ -23,14 +23,14 @@ # # Bump this version to give users an update notification. -PRE_PUSH_SCRIPT_VERSION=1 +PRE_PUSH_SCRIPT_VERSION=2 -NEW_RELEASES="22 21 20 19 18 17" +NEW_RELEASES="23 22 21 20 19 18 17" OLD_RELEASES="r16 r15 r14 r13" RELEASES="$NEW_RELEASES $OLD_RELEASES" # First commit on master, not allowed in other branches -MASTER_ONLY=f633fe962ea7078c32f8c81d34950c0ebce0f472 +MASTER_ONLY=740b29ecc21c73a4bf4ebfc494490865d3c31978 # Number of commits and files allowed in one push by this script NCOMMITS_MAX=100 diff --git a/system/COPYRIGHT b/system/COPYRIGHT index 3156886f7c..0368961f85 100644 --- a/system/COPYRIGHT +++ b/system/COPYRIGHT @@ -62,7 +62,7 @@ Email domain: cam.ac.uk University of Cambridge Computing Service, Cambridge, England. -Copyright (c) 1997-2019 University of Cambridge +Copyright (c) 1997-2020 University of Cambridge All rights reserved. @@ -73,7 +73,7 @@ Written by: Zoltan Herczeg Email local part: hzmester Email domain: freemail.hu -Copyright(c) 2010-2019 Zoltan Herczeg +Copyright(c) 2010-2020 Zoltan Herczeg All rights reserved. @@ -84,7 +84,7 @@ Written by: Zoltan Herczeg Email local part: hzmester Email domain: freemail.hu -Copyright(c) 2009-2019 Zoltan Herczeg +Copyright(c) 2009-2020 Zoltan Herczeg All rights reserved. diff --git a/system/doc/general_info/DEPRECATIONS b/system/doc/general_info/DEPRECATIONS index 847d34cbdd..9983df9421 100644 --- a/system/doc/general_info/DEPRECATIONS +++ b/system/doc/general_info/DEPRECATIONS @@ -21,6 +21,8 @@ # # Added in OTP 23. # +ssl:cipher_suites/1 since=21 remove=24 +ssl:cipher_suites/0 since=21 remove=24 http_uri:parse/1 since=23 remove=25 http_uri:parse/2 since=23 remove=25 @@ -99,7 +101,7 @@ sys:get_debug/3 since=22 # Added in OTP 21. # -ssl:ssl_accept/_ since=21 +ssl:ssl_accept/_ since=21 remove=24 # # Added in OTP 20. diff --git a/system/doc/general_info/deprecations_23.inc b/system/doc/general_info/deprecations_23.inc index 63d6d4c8ed..f7c5dc0790 100644 --- a/system/doc/general_info/deprecations_23.inc +++ b/system/doc/general_info/deprecations_23.inc @@ -55,3 +55,16 @@ in OTP 24. </p> </section> + + <section> + <title>erl_interface registry</title> + <p> + As of OTP 23, the + <seecref marker="erl_interface:registry"><c>registry</c></seecref> + functionality part of + <seeapp marker="erl_interface:index"><c>erl_interface</c></seeapp> + has been deprecated and it has also been + <seeguide marker="scheduled_for_removal#otp-24">scheduled for removal</seeguide> + in OTP 24. + </p> + </section> diff --git a/system/doc/general_info/scheduled_for_removal_24.inc b/system/doc/general_info/scheduled_for_removal_24.inc index 05db29ded2..72aa9ba648 100644 --- a/system/doc/general_info/scheduled_for_removal_24.inc +++ b/system/doc/general_info/scheduled_for_removal_24.inc @@ -32,3 +32,21 @@ will still be supported, even though its no longer necessary to specify it this way. </p> </section> + <section> + <title>Compilation of Latin-1 Encoded Erlang Files</title> + <p>As of OTP 24, the Erlang compiler will refuse to compile source files + encoded in Latin-1 but without a <c>%% coding: latin-1</c> comment at the + beginning of the file.</p> + </section> + <section> + <title>erl_interface registry</title> + <p> + The + <seecref marker="erl_interface:registry"><c>registry</c></seecref> + functionality part of + <seeapp marker="erl_interface:index"><c>erl_interface</c></seeapp> + is as of + <seeguide marker="deprecations#otp-23">OTP 23 deprecated</seeguide> + and will be removed in OTP 24. + </p> + </section> diff --git a/system/doc/reference_manual/typespec.xml b/system/doc/reference_manual/typespec.xml index fa21733bd2..a450f85742 100644 --- a/system/doc/reference_manual/typespec.xml +++ b/system/doc/reference_manual/typespec.xml @@ -207,6 +207,7 @@ They can be thought as predefined aliases for the type unions also shown in the table. </p> + <marker id="builtin_types"/> <table> <row> <cell><em>Built-in type</em></cell><cell><em>Defined as</em></cell> |