Reference manual: Improve documentation of signals

2 files changed, 509 insertions, 15 deletions

diff --git a/system/doc/reference_manual/expressions.xml b/system/doc/reference_manual/expressions.xml
index 0dbbba6795..382f9a8a8c 100644
--- a/system/doc/reference_manual/expressions.xml
+++ b/system/doc/reference_manual/expressions.xml
@@ -433,9 +433,11 @@ end</pre>
     <p>
       Fetches a received message present in the message queue of
       the process. The patterns <c>Pattern</c> are sequentially matched
-      against the first message in time order in the message queue, then
-      the second, and so on. If a match succeeds and the optional
-      guard sequence <c>GuardSeq</c> is true, the corresponding
+      against the first message in the message queue, then the second,
+      and so on. Messages in the message queue are
+      <seeguide marker="processes#message-queue-order">ordered in the
+      order they were received</seeguide>. If a match succeeds and the
+      optional guard sequence <c>GuardSeq</c> is true, the corresponding
       <c>Body</c> is evaluated. The matching message is consumed, that
       is, removed from the mail queue, while any other messages in
       the mail queue remain unchanged.</p>
diff --git a/system/doc/reference_manual/processes.xml b/system/doc/reference_manual/processes.xml
index 3808ca74d9..204cbf36fc 100644
--- a/system/doc/reference_manual/processes.xml
+++ b/system/doc/reference_manual/processes.xml
@@ -185,15 +185,507 @@ spawn(Module, Name, Args) -> pid()
   </section>
 
   <section>
-    <title>Message Sending</title>
-    <p>Processes communicate by sending and receiving messages.
-      Messages are sent by using
-      the <seeguide marker="expressions#send">send operator !</seeguide>
-      and received by calling
-      <seeguide marker="expressions#receive">receive</seeguide>.</p>
-    <p>Message sending is asynchronous and safe, the message is
-      guaranteed to eventually reach the recipient, provided that
-      the recipient exists.</p>
+    <title>Signals</title>
+    <p>
+      <marker id="message-sending"/>
+      All communication between Erlang processes and Erlang ports is done by
+      sending and receiving asynchronous signals. The most common signals are
+      Erlang message signals. A message signal can be sent using the
+      <seeguide marker="expressions#send">send operator <c>!</c></seeguide>.
+      A received message can be fetched from the message queue by the
+      receiving process using the
+      <seeguide marker="expressions#receive"><c>receive</c></seeguide>
+      expression.
+    </p>
+
+    <marker id="sync-comm"/>
+    <p>
+      Synchronous communication can be broken down into multiple asynchronous
+      signals. An example of such a synchronous communication is a call to the
+      <seemfa marker="erts:erlang#process_info/2"><c>erlang:process_info/2</c></seemfa>
+      BIF when the first argument does not equal the process identifier of
+      the calling process. The caller sends an asynchronous signal requesting
+      information, and then blocks waiting for the reply signal containing
+      the requested information. When the request signal reaches its
+      destination, the destination process replies with the requested
+      information.
+    </p>
+
+    <section>
+      <title>Sending Signals</title>
+
+      <p>
+	There are many signals that processes and ports use to communicate. The list below
+	contains the most important signals. In all the
+	cases of request/reply signal pairs, the request signal is sent
+	by the process calling the specific BIF, and the reply signal is sent
+	back to it when the requested operation has been performed.
+      </p>
+      <taglist>
+	<tag><c>message</c></tag>
+	<item>
+	  Sent when using the
+	  <seeguide marker="expressions#send">send operator <c>!</c></seeguide>,
+	  or when calling one of the
+	  <seemfa marker="erts:erlang#send/2"><c>erlang:send/2,3</c></seemfa>
+	  or
+	  <seemfa marker="erts:erlang#send_nosuspend/2"><c>erlang:send_nosuspend/2,3</c></seemfa>
+	  BIFs.
+	</item>
+
+	<tag><c>link</c></tag>
+	<item>
+	  Sent when calling the
+	  <seemfa marker="erts:erlang#link/1">link/1</seemfa> BIF.
+	</item>
+
+	<tag><c>unlink</c></tag>
+	<item>
+	  Sent when calling the
+	  <seemfa marker="erts:erlang#unlink/1">unlink/1</seemfa> BIF.
+	</item>
+
+	<tag><c>exit</c></tag>
+	<item>
+	  Sent either when explicitly sending an <c>exit</c> signal by
+	  calling the
+	  <seemfa marker="erts:erlang#exit/2">exit/2</seemfa> BIF, or when
+	  a <seeguide marker="#sending_exit_signals">linked process
+	  terminates</seeguide>. If the signal is sent due to a link, the
+	  signal is sent after all <seeguide marker="#visible-resources">
+	  <i>directly visible Erlang resources</i></seeguide> used by the
+	  process have been released.
+	</item>
+
+	<tag><c>monitor</c></tag>
+	<item>
+	  Sent when calling one of the
+	  <seemfa marker="erts:erlang#monitor/3">monitor/2,3</seemfa> BIFs.
+	</item>
+
+	<tag><c>demonitor</c></tag>
+	<item>
+	  Sent when calling one of the
+	  <seemfa marker="erts:erlang#demonitor/1">demonitor/1,2</seemfa>
+	  BIFs, or when a process monitoring another process terminates.
+	</item>
+
+	<tag><c>down</c></tag>
+	<item>
+	  Sent by a <seeguide marker="#monitors">monitored process or
+	  port that terminates</seeguide>. The signal is sent after
+	  all <seeguide marker="#visible-resources"><i>directly visible
+	  Erlang resources</i></seeguide> used by the process or the port
+	  have been released.
+	</item>
+
+	<tag><c>change</c></tag>
+	<item>
+	  Sent by the <seeguide marker="#runtime-service">clock
+	  service</seeguide>  on the local runtime system, when the
+	  <seemfa marker="erts:erlang#time_offset/0">time offset</seemfa>
+	  changes, to processes which have
+	  <seemfa marker="erts:erlang#monitor/2">monitored the
+	  <c>time_offset</c></seemfa>.
+	</item>
+
+	<tag><c>group_leader</c></tag>
+	<item>
+	  Sent when calling the
+	  <seemfa marker="erts:erlang#group_leader/2">group_leader/2</seemfa>
+	  BIF.
+	</item>
+
+	<tag>
+	  <c>spawn_request</c>/<c>spawn_reply</c>,
+	  <c>open_port_request</c>/<c>open_port_reply</c>
+	</tag>
+	<item>
+	  Sent due to a call to one of the
+    	  <seemfa marker="erts:erlang#spawn/4"><c>spawn/1,2,3,4</c></seemfa>,
+	  <seemfa marker="erts:erlang#spawn_link/4"><c>spawn_link/1,2,3,4</c></seemfa>,
+	  <seemfa marker="erts:erlang#spawn_monitor/4"><c>spawn_monitor/1,2,3,4</c></seemfa>,
+	  <seemfa marker="erts:erlang#spawn_opt/5"><c>spawn_opt/2,3,4,5</c></seemfa>,
+	  <seemfa marker="erts:erlang#spawn_request/5"><c>spawn_request/1,2,3,4,5</c></seemfa>,
+	  or <seemfa marker="erts:erlang#open_port/2"><c>erlang:open_port/2</c></seemfa>
+	  BIFs. The request signal is sent to the
+	  <seeguide marker="#runtime-service">spawn service</seeguide> which
+	  responds with the reply signal.
+	</item>
+
+	<tag><c>alive_request</c>/<c>alive_reply</c></tag>
+	<item>
+	  Sent due to a call to the
+	  <seemfa marker="erts:erlang#is_process_alive/1">is_process_alive/1</seemfa>
+	  BIF.
+	</item>
+
+	<tag>
+	  <c>garbage_collect_request</c>/<c>garbage_collect_reply</c>,
+	  <c>check_process_code_request</c>/<c>check_process_code_reply</c>,
+	  <c>process_info_request</c>/<c>process_info_reply</c>
+	</tag>
+	<item>
+	  Sent due to a call to one of the
+	  <seemfa marker="erts:erlang#garbage_collect/1">garbage_collect/1,2</seemfa>,
+	  <seemfa marker="erts:erlang#check_process_code/2">erlang:check_process_code/2,3</seemfa>,
+	  or <seemfa marker="erts:erlang#process_info/2">process_info/1,2</seemfa>
+	  BIFs. Note that if the request is directed towards the caller itself
+	  and it is a synchronous request, no signaling will be performed
+	  and the caller will instead synchronously perform the request before
+	  returning from the BIF.
+	</item>
+
+	<tag><c>port_command</c>, <c>port_connect</c>, <c>port_close</c></tag>
+	<item>
+	  Sent by a process to a port on the local node using the
+	  <seeguide marker="expressions#send">send operator <c>!</c></seeguide>,
+	  or by calling one of the
+	  <seemfa marker="erts:erlang#send/2"><c>send()</c></seemfa>
+	  BIFs. The signal is sent by passing a term on the format
+	  <c>{Owner, {command, Data}}</c>, <c>{Owner, {connect, Pid}}</c>,
+	  or <c>{Owner, close}</c> as message.
+	</item>
+
+	<tag>
+	  <c>port_command_request</c>/<c>port_command_reply</c>,
+	  <c>port_connect_request</c>/<c>port_connect_reply</c>,
+	  <c>port_close_request</c>/<c>port_close_reply</c>,
+	  <c>port_control_request</c>/<c>port_control_reply</c>,
+	  <c>port_call_request</c>/<c>port_call_reply</c>,
+	  <c>port_info_request</c>/<c>port_info_reply</c>
+	</tag>
+	<item>
+	  Sent due to a call to one of the
+ 	  <seemfa marker="erts:erlang#port_command/2"><c>erlang:port_command/2,3</c></seemfa>,
+	  <seemfa marker="erts:erlang#port_connect/2"><c>erlang:port_connect/2</c></seemfa>,
+	  <seemfa marker="erts:erlang#port_close/1"><c>erlang:port_close/1</c></seemfa>,
+	  <seemfa marker="erts:erlang#port_control/3"><c>erlang:port_control/3</c></seemfa>,
+	  <seemfa marker="erts:erlang#port_call/3"><c>erlang:port_call/3</c></seemfa>,
+	  <seemfa marker="erts:erlang#port_info/1"><c>erlang:port_info/1,2</c></seemfa>
+	  BIFs. The request signal is sent to a port on the local node which
+	  responds with the reply signal.
+	</item>
+
+	<tag>
+	  <c>timer_start_request</c>/<c>timer_start_reply</c>,
+	  <c>timer_cancel_request</c>/<c>timer_cancel_reply</c>
+	</tag>
+	<item>
+	  Sent due to a call to one of the
+          <seemfa marker="erts:erlang#send_after/3"><c>erlang:send_after/3,4</c></seemfa>,
+          <seemfa marker="erts:erlang#start_timer/3"><c>erlang:start_timer/3,4</c></seemfa>,
+	  or
+	  <seemfa marker="erts:erlang#cancel_timer/1"><c>erlang:cancel_timer/1,2</c></seemfa>
+	  BIFs. The request signal is sent to the
+	  <seeguide marker="#runtime-service">timer service</seeguide> which
+	  responds with the reply signal.
+	</item>
+	
+      </taglist>
+
+      <p><marker id="runtime-service"/>
+	The clock service, the timer service, and the spawn service mentioned
+	above are services provided by the runtime system. Each of these
+	services consists of multiple independently executing entities. Such a
+	service can be viewed as a group of processes, and could actually be
+	implemented like that. Since each service consists of multiple
+	independently executing entities, the order between multiple signals
+	sent from one service to one process is <em>not</em> preserved. Note
+	that this does <em>not</em> violate the
+	<seeguide marker="#signal-delivery">signal ordering
+	guarantee</seeguide> of the language.
+      </p>
+
+      <p>
+	The realization of the signals described above may change both at
+	runtime and due to changes in implementation. You may be able to
+	detect such changes using <c>receive</c> tracing or by inspecting
+	message queues. However, these are internal implementation details of
+	the runtime system that you should <em>not</em> rely on. As an example,
+	many of the reply signals above are ordinary message signals.
+	When the operation is synchronous, the reply signals do not have to be
+	message signals. The current implementation takes advantage of this
+	and, depending on the state of the system, use alternative ways of
+	delivering the reply signals. The implementation of these reply signals
+	may also, at any time, be changed to not use message signals where it
+	previously did.
+      </p>
+
+    </section>
+
+    <section>
+      <title>Receiving Signals</title>
+      <p>
+	When a signal is received by a process, some kind of action is
+	taken. The specific action taken depends on the signal type,
+	contents of the signal, and the state of the receiving process.
+	Actions taken for the most common signals:
+      </p>
+      <taglist>
+	<tag><c>message</c></tag>
+	<item>
+	  If the message signal was sent using a
+	  <seeguide marker="#process-aliases">process alias</seeguide>
+	  that is no longer active, the message signal will be dropped;
+	  otherwise, if the alias is still active or the message signal
+	  was sent by other means, the message is added to the end of the
+	  message queue. When the message has been added to the message
+	  queue, the receiving process can fetch the message from the
+	  message queue using the
+	  <seeguide marker="expressions#receive"><c>receive</c></seeguide>
+	  expression.
+	</item>
+
+	<tag><c>link</c>, <c>unlink</c></tag>
+	<item>
+	  Very simplified it can be viewed as updating process local
+	  information about the link. A detailed description of the
+	  <seeguide marker="erts:erl_dist_protocol#link_protocol">link
+	  protocol</seeguide> can be found in the <i>Distribution Protocol</i>
+	  chapter of the <i>ERTS User's Guide</i>.
+	</item>
+
+	<tag><c>exit</c></tag>
+	<item>
+	  Set the receiver in an exiting state, drop the signal, or convert
+	  the signal into a message and add it to the end of the message
+	  queue. If the receiver is set in an exiting state, no more Erlang
+	  code will be executed and the process is scheduled for termination.
+	  The section <seeguide marker="#receiving_exit_signals"><i>Receiving
+	  Exit Signals</i></seeguide> below gives more details on the
+	  action taken when an <c>exit</c> signal is received.
+	</item>
+
+	<tag><c>monitor</c>, <c>demonitor</c></tag>
+	<item>
+	  Update process local information about the monitor.
+	</item>
+
+	<tag><c>down</c>, <c>change</c></tag>
+	<item>
+	  Convert into a message if the corresponding monitor is still
+	  active; otherwise, drop the signal. If the signal is converted
+	  into a message, it is also added to the end of the message
+	  queue.
+	</item>
+
+	<tag><c>group_leader</c></tag>
+	<item>
+	  Change the group leader of the process.
+	</item>
+
+	<tag><c>spawn_reply</c></tag>
+	<item>
+	  Convert into a message, or drop the signal depending on the reply
+	  and how the <c>spawn_request</c> signal was configured. If the
+	  signal is converted into a message it is also added to the end
+	  of the message queue. For more information see the
+	  <seemfa marker="erts:erlang#spawn_request/5"><c>spawn_request()</c></seemfa>
+	  BIF.
+	</item>
+
+	<tag><c>alive_request</c></tag>
+	<item>
+	  Schedule execution of the <i>is alive</i> test. If the process
+	  is in an exiting state, the <i>is alive</i> test will not be
+	  executed until after all
+	  <seeguide marker="#visible-resources"><i>directly visible Erlang
+	  resources</i></seeguide> used by the process have been released.
+	  The <c>alive_reply</c> will be sent after the <i>is alive</i>
+	  test has executed.
+	</item>
+	
+	<tag>
+	  <c>process_info_request</c>,
+	  <c>garbage_collect_request</c>,
+	  <c>check_process_code_request</c>
+	</tag>
+	<item>
+	  Schedule execution of the requested operation. The reply signal
+	  will be sent when the operation has been executed.
+	</item>
+      </taglist>
+      <p>
+	Note that some actions taken when a signal is received involves
+	<em>scheduling</em> further actions which will result in a reply
+	signal when these scheduled actions have completed. This implies that
+	the reply signals may be sent in a different order than the order of
+	the incoming signals that triggered these operations. This does,
+	however, <em>not</em> violate the
+	<seeguide marker="#signal-delivery">signal ordering
+	guarantee</seeguide> of the language.
+      </p>
+
+      <marker id="message-queue-order"/>
+      <p>
+	The order of messages in the message queue of a process reflects the
+	order in which the signals corresponding to the messages has been
+	received since <seeguide marker="processes#receiving-signals">all
+	signals	that add messages to the message queue add them at the end of
+	the message queue</seeguide>. Messages corresponding to signals from
+	the same sender are also ordered in the same order as the signals were
+	sent due to the <seeguide marker="processes#signal-delivery">signal
+	ordering guarantee</seeguide> of the language.
+      </p>
+
+    </section>
+    
+    <section>
+      <marker id="visible-resources"/>
+      <title>Directly Visible Erlang Resources</title>
+      <p>
+	As described above, <c>exit</c> signals due to links, <c>down</c>
+	signals, and reply signals from an exiting process due to
+	<c>alive_request</c>s are not sent until all <i>directly visible
+	Erlang resources</i> held by the terminating process have been
+	released. With <i>directly visible Erlang resources</i> we here mean
+	all resources made available by the language excluding resources held
+	by heap data, dirty native code execution and the process identifier of
+	the terminating process. Examples of <i>directly visible Erlang
+	resources</i> are <seeguide marker="#registered-processes">registered
+	name</seeguide> and <seeerl marker="stdlib:ets">ETS</seeerl> tables.
+      </p>
+      <section>
+	<title>The Excluded Resources</title>
+	<p>
+	  The process identifier of the process cannot be released for
+	  reuse until everything regarding the process has been released.
+	</p>
+	<p>
+	  A process executing dirty native code in a NIF when it receives
+	  an exit signal will be set into an exiting state even if it is
+	  still executing dirty native code. <i>Directly visible Erlang
+	  resources</i> will be released, but the runtime system cannot
+	  force the native code to stop executing. The runtime system tries
+	  to prevent the execution of the dirty native code from effecting
+	  other processes by, for example, disabling functionality such as
+	  <seecref marker="erts:erl_nif#enif_send"><c>enif_send()</c></seecref>
+	  when used from a terminated process, but if the NIF is not well
+	  behaved it can still effect other processes. A well behaved dirty
+	  NIF should test if
+	  <seecref marker="erts:erl_nif#enif_is_current_process_alive">the
+	  process it is executing in has exited</seecref>, and if so stop
+	  executing.
+	</p>
+	<p>
+	  In the general case, the heap of a process cannot be removed before
+	  all signals that it needs to send have been sent. Resources held
+	  by heap data are the memory blocks containing the heap, but also
+	  include things referred to from the heap such as off heap binaries,
+	  and resources held via NIF
+	  <seecref marker="erts:erl_nif#resource_objects">resource
+	  objects</seecref> on the heap.
+	</p>
+      </section>
+    </section>
+
+    <section>
+      <marker id="signal-delivery"/>
+      <title>Delivery of Signals</title>
+      <p>
+	The amount of time that passes between the time a signal is sent and
+	the arrival of the signal at the destination is unspecified but
+	positive. If the receiver has terminated, the signal does not arrive,
+	but it can trigger another signal. For example, a <c>link</c> signal
+	sent to a non-existing process triggers an <c>exit</c> signal, which
+	is sent back to where the <c>link</c> signal originated from. When
+	communicating over the distribution, signals can be lost if the
+	distribution channel goes down.
+      </p>
+
+      <p>
+	The only signal ordering guarantee given is the following: if an
+	entity sends multiple signals to the same destination entity, the
+	order is preserved; that is, if <c>A</c> sends a signal <c>S1</c> to
+	<c>B</c>, and later sends signal <c>S2</c> to <c>B</c>, <c>S1</c> is
+	guaranteed not to arrive after <c>S2</c>. Note that <c>S1</c> may,
+	or may not have been lost.
+      </p>
+    </section>
+
+    <section>
+      <marker id="signal-irregularities"/>
+      <title>Irregularities</title>
+      <taglist>
+	<tag>Synchronous Error Checking</tag>
+	<item>
+	  <p>
+	    Some functionality that send signals have synchronous error
+	    checking when sending locally on a node and fail if the receiver
+	    is not present at the time when the signal is sent:
+	  </p>
+	  <list>
+	    <item>
+	      The <seeguide marker="expressions#send">send operator
+	      <c>!</c></seeguide>,
+	      <seemfa marker="erts:erlang#send/2"><c>erlang:send/2,3</c></seemfa>,
+	      BIFs and
+	      <seemfa marker="erts:erlang#send_nosuspend/2"><c>erlang:send_nosuspend/2,3</c></seemfa>
+	      BIFs when the receiver is identified by a name that is
+	      expected to be registered locally.
+	    </item>
+	    <item>
+	      <seemfa marker="erts:erlang#link/1"><c>erlang:link/1</c></seemfa>
+	    </item>
+	    <item>
+	      <seemfa marker="erts:erlang#group_leader/2"><c>erlang:group_leader/2</c></seemfa>
+	    </item>
+	  </list>
+	  <p></p>
+	</item>
+	<tag>Unexpected Behaviours of Exit Signals</tag>
+	<item>
+	  <p>
+	    When a process sends an exit signal with exit reason <c>normal</c>
+	    to itself by calling <seemfa marker="erts:erlang#exit/2"><c>erlang:exit(self(),
+	    normal)</c></seemfa> it will be terminated
+	    <seeguide marker="#receiving_exit_signals">when the <c>exit</c> signal
+	    is received</seeguide>. In all other cases when an exit signal with
+	    exit reason <c>normal</c> is received, it is dropped.
+	  </p>
+	  <p>
+	    When an <seeguide marker="#receiving_exit_signals"><c>exit</c> signal
+	    with exit reason <c>kill</c> is received</seeguide>,
+	    the action taken is different depending on whether the signal was
+	    sent due to a linked process terminating, or the signal was
+	    explicitly sent using the
+	    <seemfa marker="erts:erlang#exit/2"><c>exit/2</c></seemfa> BIF. When
+	    sent using the <c>exit/2</c> BIF, the signal cannot be
+	    <seeerl marker="erts:erlang#process_flag_trap_exit">trapped</seeerl>,
+	    while it can be trapped if the signal was sent due to a link.
+	  </p>
+	</item>
+	<tag>Blocking Signaling Over Distribution</tag>
+	<item>
+	  <p>
+	    When sending a signal over a distribution channel, the sending
+	    process may be suspended even though the signal is supposed to be
+	    sent asynchronously. This is due to the built in flow control over
+	    the channel that has been present more or less for ever. When the
+	    size of the output buffer for the channel reach the <i>distribution
+	    buffer busy limit</i>, processes sending on the channel will be
+	    suspended until the size of the buffer shrinks below the limit.
+	    The size of the limit can be inspected by calling
+	    <seeerl marker="erts:erlang#system_info_dist_buf_busy_limit">
+	      <c>erlang:system_info(dist_buf_busy_limit)</c></seeerl>.
+	    Since this functionality has been present for so long, it is not
+	    possible to remove it, but it is possible to increase the limit
+	    to a point where it more or less never is reached using the
+	    <c>erl</c> command line argument
+	    <seecom marker="erts:erl#+zdbbl"><c>+zdbbl</c></seecom>. Note
+	    that if you do raise the limit like this, you need to take care
+	    of flow control yourself to ensure that you do not get into a
+	    situation with excessive memory usage.
+	  </p>
+	</item>
+      </taglist>
+      <p>
+	The irregularities mentioned above cannot be fixed as they have been
+	part of Erlang too long and it would break a lot of existing code.
+      </p>
+    </section>
+
   </section>
 
   <section>
@@ -201,7 +693,7 @@ spawn(Module, Name, Args) -> pid()
     <p>
       Two processes can be <em>linked</em> to each other. Also a
       process and a port that reside on the same node can be linked
-      to each other. A link beteen two processes can be created
+      to each other. A link between two processes can be created
       if one of them calls the
       <seemfa marker="erts:erlang#link/1"><c>link/1</c></seemfa> BIF
       with the process identifier of the other process as argument.
@@ -231,7 +723,7 @@ spawn(Module, Name, Args) -> pid()
       Either one of the involved processes may create or remove a
       link.
     </p>
-    <p>Links are used to monitor the behaviour of other processes, see
+    <p>Links are used to monitor the behavior of other processes, see
       <seeguide marker="#errors">Error Handling</seeguide>.</p>
   </section>
 
@@ -416,7 +908,7 @@ spawn(Module, Name, Args) -> pid()
 	<item>
 	  <p>
 	    The exit signal is converted to a message signal and
-	    moved into the message queue of the receiver, if the
+	    added to the end of the message queue of the receiver, if the
 	    receiver is trapping exits, the <c>link</c> flag
 	    of the exit signal is:
 	  </p>