build: fix various shared library build issues

Node.js unofficially supports a shared library variant where the
main node executable is a thin wrapper around node.dll/libnode.so.
The key benefit of this is to support embedding Node.js in other
applications.

Since Node.js 12 there have been a number of issues preventing the
shared library build from working correctly, primarily on Windows:

* A number of functions used executables such as `mksnapshot` are
    not exported from `libnode.dll` using a `NODE_EXTERN` attribute
* A dependency on the `Winmm` system library is missing
* Incorrect defines on executable targets leads to `node.exe`
    claiming to export a number of functions that are actually in
    `libnode.dll`
* Because `node.exe` attempts to export symbols, `node.lib` gets
    generated causing native extensions to try to link against
    `node.exe` not `libnode.dll`.
* Similarly, because `node.dll` was renamed to `libnode.dll`,
    native extensions don't know to look for `libnode.lib` rather
    than `node.lib`.
* On macOS an RPATH is added to find `libnode.dylib` relative to
    `node` in the same folder. This works fine from the
    `out/Release` folder but not from an installed prefix, where
    `node` will be in `bin/` and `libnode.dylib` will be in `lib/`.
* Similarly on Linux, no RPATH is added so LD_LIBRARY_PATH needs
    setting correctly for `bin/node` to find `lib/libnode.so`.

For the `libnode.lib` vs `node.lib` issue there are two possible
options:

1. Ensure `node.lib` from `node.exe` does not get generated, and
    instead copy `libnode.lib` to `node.lib`. This means addons
    compiled when referencing the correct `node.lib` file will
    correctly depend on `libnode.dll`. The down side is that
    native addons compiled with stock Node.js will still try to
    resolve symbols against node.exe rather than libnode.dll.
2. After building `libnode.dll`, dump the exports using `dumpbin`,
    and process this to generate a `node.def` file to be linked into
    `node.exe` with the `/DEF:node.def` flag. The export entries
    in `node.def` will all read
    ```
    my_symbol=libnode.my_symbol
    ```
    so that `node.exe` will redirect all exported symbols back to
    `libnode.dll`. This has the benefit that addons compiled with
    stock Node.js will load correctly into `node.exe` from a shared
    library build, but means that every embedding executable also
    needs to perform this same trick.

I went with the first option as it is the cleaner of the two
solutions in my opinion. Projects wishing to generate a shared
library variant of Node.js can now, for example,
```
.\vcbuild dll package vs
```
to generate a full node installation including `libnode.dll`,
`Release\node.lib`, and all the necessary headers. Native addons
can then be built against the shared library build easily by
specifying the correct `nodedir` option.

For example
```
>npx node-gyp configure --nodedir
   C:\Users\User\node\Release\node-v18.0.0-win-x64
...
>npx node-gyp build
...
>dumpbin /dependents build\Release\binding.node
Microsoft (R) COFF/PE Dumper Version 14.29.30136.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file build\Release\binding.node

File Type: DLL

  Image has the following dependencies:

    KERNEL32.dll
    libnode.dll
    VCRUNTIME140.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-stdio-l1-1-0.dll
    api-ms-win-crt-runtime-l1-1-0.dll
...
```

PR-URL: https://github.com/nodejs/node/pull/41850
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>

13 files changed, 316 insertions, 30 deletions

diff --git a/node.gyp b/node.gyp
index 512e341b5b..b4ce6188da 100644
--- a/node.gyp
+++ b/node.gyp
@@ -196,6 +196,16 @@
           'dependencies': [ 'node_aix_shared' ],
         }, {
           'dependencies': [ '<(node_lib_target_name)' ],
+          'conditions': [
+            ['OS=="win" and node_shared=="true"', {
+              'dependencies': ['generate_node_def'],
+              'msvs_settings': {
+                'VCLinkerTool': {
+                  'ModuleDefinitionFile': '<(PRODUCT_DIR)/<(node_core_target_name).def',
+                },
+              },
+            }],
+          ],
         }],
         [ 'node_intermediate_lib_type=="static_library" and node_shared=="false"', {
           'xcode_settings': {
@@ -235,8 +245,15 @@
         }],
         [ 'node_shared=="true"', {
           'xcode_settings': {
-            'OTHER_LDFLAGS': [ '-Wl,-rpath,@loader_path', ],
+            'OTHER_LDFLAGS': [ '-Wl,-rpath,@loader_path', '-Wl,-rpath,@loader_path/../lib'],
           },
+          'conditions': [
+            ['OS=="linux"', {
+               'ldflags': [
+                 '-Wl,-rpath,\\$$ORIGIN/../lib'
+               ],
+            }],
+          ],
         }],
         [ 'enable_lto=="true"', {
           'xcode_settings': {
@@ -751,6 +768,7 @@
           'libraries': [
             'Dbghelp',
             'Psapi',
+            'Winmm',
             'Ws2_32',
           ],
         }],
@@ -1504,5 +1522,40 @@
         },
       ]
     }], # end aix section
+    ['OS=="win" and node_shared=="true"', {
+     'targets': [
+       {
+         'target_name': 'gen_node_def',
+         'type': 'executable',
+         'sources': [
+           'tools/gen_node_def.cc'
+         ],
+       },
+       {
+         'target_name': 'generate_node_def',
+         'dependencies': [
+           'gen_node_def',
+           '<(node_lib_target_name)',
+         ],
+         'type': 'none',
+         'actions': [
+           {
+             'action_name': 'generate_node_def_action',
+             'inputs': [
+               '<(PRODUCT_DIR)/<(node_lib_target_name).dll'
+             ],
+             'outputs': [
+               '<(PRODUCT_DIR)/<(node_core_target_name).def',
+             ],
+             'action': [
+               '<(PRODUCT_DIR)/gen_node_def.exe',
+               '<@(_inputs)',
+               '<@(_outputs)',
+             ],
+           },
+         ],
+       },
+     ],
+   }], # end win section
   ], # end conditions block
 }
diff --git a/node.gypi b/node.gypi
index e8aa664468..d24928df8b 100644
--- a/node.gypi
+++ b/node.gypi
@@ -29,7 +29,7 @@
     [ 'clang==1', {
       'cflags': [ '-Werror=undefined-inline', ]
     }],
-    [ 'node_shared=="false" and "<(_type)"=="executable"', {
+    [ '"<(_type)"=="executable"', {
       'msvs_settings': {
         'VCManifestTool': {
           'EmbedManifest': 'true',
@@ -41,6 +41,19 @@
       'defines': [
         'NODE_SHARED_MODE',
       ],
+      'conditions': [
+        ['"<(_type)"=="executable"', {
+          'defines': [
+            'USING_UV_SHARED',
+            'USING_V8_SHARED',
+            'BUILDING_NODE_EXTENSION'
+          ],
+          'defines!': [
+            'BUILDING_V8_SHARED=1',
+            'BUILDING_UV_SHARED=1'
+          ]
+        }],
+      ],
     }],
     [ 'OS=="win"', {
       'defines!': [
diff --git a/src/debug_utils.h b/src/debug_utils.h
index 377493359e..bd1fa5207f 100644
--- a/src/debug_utils.h
+++ b/src/debug_utils.h
@@ -35,7 +35,7 @@ template <typename... Args>
 inline std::string SPrintF(const char* format, Args&&... args);
 template <typename... Args>
 inline void FPrintF(FILE* file, const char* format, Args&&... args);
-void FWrite(FILE* file, const std::string& str);
+void NODE_EXTERN_PRIVATE FWrite(FILE* file, const std::string& str);
 
 // Listing the AsyncWrap provider types first enables us to cast directly
 // from a provider type to a debug category.
@@ -57,7 +57,7 @@ enum class DebugCategory {
       CATEGORY_COUNT
 };
 
-class EnabledDebugList {
+class NODE_EXTERN_PRIVATE EnabledDebugList {
  public:
   bool enabled(DebugCategory category) const {
     DCHECK_GE(static_cast<int>(category), 0);
@@ -168,7 +168,7 @@ void CheckedUvLoopClose(uv_loop_t* loop);
 void PrintLibuvHandleInformation(uv_loop_t* loop, FILE* stream);
 
 namespace per_process {
-extern EnabledDebugList enabled_debug_list;
+extern NODE_EXTERN_PRIVATE EnabledDebugList enabled_debug_list;
 
 template <typename... Args>
 inline void FORCE_INLINE Debug(DebugCategory cat,
diff --git a/src/env.h b/src/env.h
index 7e35833e45..475ca4d8b7 100644
--- a/src/env.h
+++ b/src/env.h
@@ -558,7 +558,7 @@ class Environment;
 struct AllocatedBuffer;
 
 typedef size_t SnapshotIndex;
-class IsolateData : public MemoryRetainer {
+class NODE_EXTERN_PRIVATE IsolateData : public MemoryRetainer {
  public:
   IsolateData(v8::Isolate* isolate,
               uv_loop_t* event_loop,
diff --git a/src/node.h b/src/node.h
index dec8dd6805..e866fcd936 100644
--- a/src/node.h
+++ b/src/node.h
@@ -32,6 +32,16 @@
 # define NODE_EXTERN __attribute__((visibility("default")))
 #endif
 
+// Declarations annotated with NODE_EXTERN_PRIVATE do not form part of
+// the public API. They are implementation details that can and will
+// change between releases, even in semver patch releases. Do not use
+// any such symbol in external code.
+#ifdef NODE_SHARED_MODE
+#define NODE_EXTERN_PRIVATE NODE_EXTERN
+#else
+#define NODE_EXTERN_PRIVATE
+#endif
+
 #ifdef BUILDING_NODE_EXTENSION
 # undef BUILDING_V8_SHARED
 # undef BUILDING_UV_SHARED
diff --git a/src/node_internals.h b/src/node_internals.h
index bf760f4d99..c56fccb984 100644
--- a/src/node_internals.h
+++ b/src/node_internals.h
@@ -320,13 +320,14 @@ enum InitializationSettingsFlags : uint64_t {
 };
 
 // TODO(codebytere): eventually document and expose to embedders.
-InitializationResult InitializeOncePerProcess(int argc, char** argv);
-InitializationResult InitializeOncePerProcess(
-  int argc,
-  char** argv,
-  InitializationSettingsFlags flags,
-  ProcessFlags::Flags process_flags = ProcessFlags::kNoFlags);
-void TearDownOncePerProcess();
+InitializationResult NODE_EXTERN_PRIVATE InitializeOncePerProcess(int argc,
+                                                                  char** argv);
+InitializationResult NODE_EXTERN_PRIVATE InitializeOncePerProcess(
+    int argc,
+    char** argv,
+    InitializationSettingsFlags flags,
+    ProcessFlags::Flags process_flags = ProcessFlags::kNoFlags);
+void NODE_EXTERN_PRIVATE TearDownOncePerProcess();
 void SetIsolateErrorHandlers(v8::Isolate* isolate, const IsolateSettings& s);
 void SetIsolateMiscHandlers(v8::Isolate* isolate, const IsolateSettings& s);
 void SetIsolateCreateParamsForNode(v8::Isolate::CreateParams* params);
diff --git a/src/node_native_module.h b/src/node_native_module.h
index 3be3f2364d..7acd154d41 100644
--- a/src/node_native_module.h
+++ b/src/node_native_module.h
@@ -29,7 +29,7 @@ using NativeModuleCacheMap =
 // This class should not depend on any Environment, or depend on access to
 // the its own singleton - that should be encapsulated in NativeModuleEnv
 // instead.
-class NativeModuleLoader {
+class NODE_EXTERN_PRIVATE NativeModuleLoader {
  public:
   NativeModuleLoader(const NativeModuleLoader&) = delete;
   NativeModuleLoader& operator=(const NativeModuleLoader&) = delete;
diff --git a/src/node_options.h b/src/node_options.h
index 71c937aca0..f38ba330e4 100644
--- a/src/node_options.h
+++ b/src/node_options.h
@@ -492,7 +492,7 @@ void Parse(
 namespace per_process {
 
 extern Mutex cli_options_mutex;
-extern std::shared_ptr<PerProcessOptions> cli_options;
+extern NODE_EXTERN_PRIVATE std::shared_ptr<PerProcessOptions> cli_options;
 
 }  // namespace per_process
 
diff --git a/src/node_snapshot_builder.h b/src/node_snapshot_builder.h
index 61450a51df..2714293fbc 100644
--- a/src/node_snapshot_builder.h
+++ b/src/node_snapshot_builder.h
@@ -13,7 +13,7 @@ namespace node {
 class ExternalReferenceRegistry;
 struct SnapshotData;
 
-class SnapshotBuilder {
+class NODE_EXTERN_PRIVATE SnapshotBuilder {
  public:
   static std::string Generate(const std::vector<std::string> args,
                               const std::vector<std::string> exec_args);
diff --git a/src/util.h b/src/util.h
index 8c21c53a13..14c8758c84 100644
--- a/src/util.h
+++ b/src/util.h
@@ -26,6 +26,8 @@
 
 #include "v8.h"
 
+#include "node.h"
+
 #include <climits>
 #include <cstddef>
 #include <cstdio>
@@ -110,8 +112,8 @@ struct AssertionInfo {
   const char* message;
   const char* function;
 };
-[[noreturn]] void Assert(const AssertionInfo& info);
-[[noreturn]] void Abort();
+[[noreturn]] void NODE_EXTERN_PRIVATE Assert(const AssertionInfo& info);
+[[noreturn]] void NODE_EXTERN_PRIVATE Abort();
 void DumpBacktrace(FILE* fp);
 
 // Windows 8+ does not like abort() in Release mode
diff --git a/tools/gen_node_def.cc b/tools/gen_node_def.cc
new file mode 100644
index 0000000000..f4cabbd84c
--- /dev/null
+++ b/tools/gen_node_def.cc
@@ -0,0 +1,197 @@
+#include <Windows.h>
+#include <algorithm>
+#include <cstdint>
+#include <fstream>
+#include <iostream>
+#include <memory>
+#include <vector>
+
+// This executable takes a Windows DLL and uses it to generate
+// a module-definition file [1] which forwards all the exported
+// symbols from the DLL and redirects them back to the DLL.
+// This allows node.exe to export the same symbols as libnode.dll
+// when building Node.js as a shared library. This is conceptually
+// similary to the create_expfile.sh script used on AIX.
+//
+// Generating this .def file requires parsing data out of the 
+// PE32/PE32+ file format. Helper structs are defined in <Windows.h>
+// hence why this is an executable and not a script. See [2] for 
+// details on the PE format.
+//
+// [1]: https://docs.microsoft.com/en-us/cpp/build/reference/module-definition-dot-def-files
+// [2]: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format
+
+// The PE32 format encodes pointers as Relative Virtual Addresses
+// which are 32 bit offsets from the start of the image. This helper
+// class hides the mess of the pointer arithmetic
+struct RelativeAddress {
+  uintptr_t root;
+  uintptr_t offset = 0;
+
+  RelativeAddress(HMODULE handle) noexcept
+      : root(reinterpret_cast<uintptr_t>(handle)) {}
+
+  RelativeAddress(HMODULE handle, uintptr_t offset) noexcept
+      : root(reinterpret_cast<uintptr_t>(handle)), offset(offset) {}
+
+  RelativeAddress(uintptr_t root, uintptr_t offset) noexcept
+      : root(root), offset(offset) {}
+
+  template <typename T>
+  const T* AsPtrTo() const noexcept {
+    return reinterpret_cast<const T*>(root + offset);
+  }
+
+  template <typename T>
+  T Read() const noexcept {
+    return *AsPtrTo<T>();
+  }
+
+  RelativeAddress AtOffset(uintptr_t amount) const noexcept {
+    return {root, offset + amount};
+  }
+
+  RelativeAddress operator+(uintptr_t amount) const noexcept {
+    return {root, offset + amount};
+  }
+
+  RelativeAddress ReadRelativeAddress() const noexcept {
+    return {root, Read<uint32_t>()};
+  }
+};
+
+// A wrapper around a dynamically loaded Windows DLL. This steps through the 
+// PE file structure to find the export directory and pulls out a list of
+// all the exported symbol names.
+struct Library {
+  HMODULE library;
+  std::string libraryName;
+  std::vector<std::string> exportedSymbols;
+
+  Library(HMODULE library) : library(library) {
+    auto libnode = RelativeAddress(library);
+
+    // At relative offset 0x3C is a 32 bit offset to the COFF signature, 4 bytes
+    // after that is the start of the COFF header.
+    auto coffHeaderPtr =
+        libnode.AtOffset(0x3C).ReadRelativeAddress().AtOffset(4);
+    auto coffHeader = coffHeaderPtr.AsPtrTo<IMAGE_FILE_HEADER>();
+
+    // After the coff header is the Optional Header (which is not optional). We
+    // don't know what type of optional header we have without examining the
+    // magic number
+    auto optionalHeaderPtr = coffHeaderPtr.AtOffset(sizeof(IMAGE_FILE_HEADER));
+    auto optionalHeader = optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER>();
+
+    auto exportDirectory =
+        (optionalHeader->Magic == 0x20b) ? optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER64>()
+                               ->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT]
+                         : optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER32>()
+                               ->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT];
+
+    auto exportTable = libnode.AtOffset(exportDirectory.VirtualAddress)
+            .AsPtrTo<IMAGE_EXPORT_DIRECTORY>();
+
+    // This is the name of the library without the suffix, this is more robust
+    // than parsing the filename as this is what the linker uses.
+    libraryName = libnode.AtOffset(exportTable->Name).AsPtrTo<char>();
+    libraryName = libraryName.substr(0, libraryName.size() - 4);
+
+    const uint32_t* functionNameTable =
+        libnode.AtOffset(exportTable->AddressOfNames).AsPtrTo<uint32_t>();
+
+    // Given an RVA, parse it as a std::string. The resulting string is empty
+    // if the symbol does not have a name (i.e. it is ordinal only).
+    auto nameRvaToName = [&](uint32_t rva) -> std::string {
+      auto namePtr = libnode.AtOffset(rva).AsPtrTo<char>();
+      if (namePtr == nullptr) return {};
+      return {namePtr};
+    };
+    std::transform(functionNameTable,
+                   functionNameTable + exportTable->NumberOfNames,
+                   std::back_inserter(exportedSymbols),
+                   nameRvaToName);
+  }
+
+  ~Library() { FreeLibrary(library); }
+};
+
+bool IsPageExecutable(void* address) {
+  MEMORY_BASIC_INFORMATION memoryInformation;
+  size_t rc = VirtualQuery(
+      address, &memoryInformation, sizeof(MEMORY_BASIC_INFORMATION));
+
+  if (rc != 0 && memoryInformation.Protect != 0) {
+    return memoryInformation.Protect == PAGE_EXECUTE ||
+           memoryInformation.Protect == PAGE_EXECUTE_READ ||
+           memoryInformation.Protect == PAGE_EXECUTE_READWRITE ||
+           memoryInformation.Protect == PAGE_EXECUTE_WRITECOPY;
+  }
+  return false;
+}
+
+Library LoadLibraryOrExit(const char* dllPath) {
+  auto library = LoadLibrary(dllPath);
+  if (library != nullptr) return library;
+
+  auto error = GetLastError();
+  std::cerr << "ERROR: Failed to load " << dllPath << std::endl;
+  LPCSTR buffer = nullptr;
+  auto rc = FormatMessageA(
+      FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM,
+      nullptr,
+      error,
+      LANG_USER_DEFAULT,
+      (LPSTR)&buffer,
+      0,
+      nullptr);
+  if (rc != 0) {
+    std::cerr << buffer << std::endl;
+    LocalFree((HLOCAL)buffer);
+  }
+  exit(1);
+}
+
+int main(int argc, char** argv) {
+  if (argc != 3) {
+    std::cerr << "Usage: " << argv[0]
+              << " path\\to\\libnode.dll path\\to\\node.def" << std::endl;
+    return 1;
+  }
+
+  auto libnode = LoadLibraryOrExit(argv[1]);
+  auto defFile = std::ofstream(argv[2]);
+  defFile << "EXPORTS" << std::endl;
+
+  for (const std::string& functionName : libnode.exportedSymbols) {
+    // If a symbol doesn't have a name then it has been exported as an
+    // ordinal only. We assume that only named symbols are exported.
+    if (functionName.empty()) continue;
+
+    // Every name in the exported symbols table should be resolvable
+    // to an address because we have actually loaded the library into 
+    // our address space.
+    auto address = GetProcAddress(libnode.library, functionName.c_str());
+    if (address == nullptr) {
+      std::cerr << "WARNING: " << functionName
+                << " appears in export table but is not a valid symbol"
+                << std::endl;
+      continue;
+    }
+
+    defFile << "    " << functionName << " = " << libnode.libraryName << "."
+            << functionName;
+    
+    // Nothing distinguishes exported global data from exported functions
+    // with C linkage. If we do not specify the DATA keyword for such symbols
+    // then consumers of the .def file will get a linker error. This manifests
+    // as nodedbg_ symbols not being found. We assert that if the symbol is in
+    // an executable page in this process then it is a function, not data.
+    if (!IsPageExecutable(address)) {
+      defFile << " DATA";
+    }
+    defFile << std::endl;
+  }
+
+  return 0;
+}
diff --git a/tools/install.py b/tools/install.py
index e2440a082f..47e9d8bd7a 100755
--- a/tools/install.py
+++ b/tools/install.py
@@ -133,20 +133,17 @@ def files(action):
   output_file = 'node'
   output_prefix = 'out/Release/'
 
-  if 'false' == variables.get('node_shared'):
-    if is_windows:
-      output_file += '.exe'
-  else:
+  if is_windows:
+    output_file += '.exe'
+  action([output_prefix + output_file], 'bin/' + output_file)
+
+  if 'true' == variables.get('node_shared'):
     if is_windows:
-      output_file += '.dll'
+      action([output_prefix + 'libnode.dll'], 'bin/libnode.dll')
+      action([output_prefix + 'libnode.lib'], 'lib/libnode.lib')
     else:
-      output_file = 'lib' + output_file + '.' + variables.get('shlib_suffix')
-
-  if 'false' == variables.get('node_shared'):
-    action([output_prefix + output_file], 'bin/' + output_file)
-  else:
-    action([output_prefix + output_file], 'lib/' + output_file)
-
+      output_lib = 'libnode.' + variables.get('shlib_suffix')
+      action([output_prefix + output_lib], 'lib/' + output_lib)
   if 'true' == variables.get('node_use_dtrace'):
     action(['out/Release/node.d'], 'lib/dtrace/node.d')
 
diff --git a/vcbuild.bat b/vcbuild.bat
index e486b83b6e..96306e407e 100644
--- a/vcbuild.bat
+++ b/vcbuild.bat
@@ -468,6 +468,19 @@ if not defined noetw (
     copy /Y ..\src\res\node_etw_provider.man %TARGET_NAME%\ > nul
     if errorlevel 1 echo Cannot copy node_etw_provider.man && goto package_error
 )
+if defined dll (
+  copy /Y libnode.dll %TARGET_NAME%\ > nul
+  if errorlevel 1 echo Cannot copy libnode.dll && goto package_error
+
+  mkdir %TARGET_NAME%\Release > nul
+  copy /Y node.def %TARGET_NAME%\Release\ > nul
+  if errorlevel 1 echo Cannot copy node.def && goto package_error
+
+  set HEADERS_ONLY=1
+  python ..\tools\install.py install %CD%\%TARGET_NAME% \ > nul
+  if errorlevel 1 echo Cannot install headers && goto package_error
+  set HEADERS_ONLY=
+)
 cd ..
 
 :package