From 68d53c01b0b8e9a007a6a30158c19e34b2d2a34e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Gustavsson?= Date: Wed, 18 May 2016 15:53:35 +0200 Subject: Update STDLIB documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Language cleaned up by the technical writers xsipewe and tmanevik from Combitech. Proofreading and corrections by Björn Gustavsson and Hans Bolinder. --- lib/stdlib/doc/src/ms_transform.xml | 958 +++++++++++++++++++++--------------- 1 file changed, 549 insertions(+), 409 deletions(-) (limited to 'lib/stdlib/doc/src/ms_transform.xml') diff --git a/lib/stdlib/doc/src/ms_transform.xml b/lib/stdlib/doc/src/ms_transform.xml index 84712486ea..0a05fa37c5 100644 --- a/lib/stdlib/doc/src/ms_transform.xml +++ b/lib/stdlib/doc/src/ms_transform.xml @@ -28,65 +28,81 @@ 1 Bjarne Däcker - 99-02-09 + 1999-02-09 C - ms_transform.sgml + ms_transform.xml ms_transform - Parse_transform that translates fun syntax into match specifications. + A parse transformation that translates fun syntax into match + specifications. -

This module implements the parse_transform that makes calls to - ets and dbg:fun2ms/1 translate into literal - match specifications. It also implements the back end for the same - functions when called from the Erlang shell.

-

The translations from fun's to match_specs - is accessed through the two "pseudo - functions" ets:fun2ms/1 and dbg:fun2ms/1.

-

Actually this introduction is more or less an introduction to the - whole concept of match specifications. Since everyone trying to use - ets:select or dbg seems to end up reading - this page, it seems in good place to explain a little more than - just what this module does.

-

There are some caveats one should be aware of, please read through - the whole manual page if it's the first time you're using the - transformations.

-

Match specifications are used more or less as filters. - They resemble usual Erlang matching in a list comprehension or in - a fun used in conjunction with lists:foldl etc. The - syntax of pure match specifications is somewhat awkward though, as - they are made up purely by Erlang terms and there is no syntax in the - language to make the match specifications more readable.

-

As the match specifications execution and structure is quite like - that of a fun, it would for most programmers be more straight forward - to simply write it using the familiar fun syntax and having that - translated into a match specification automatically. Of course a real - fun is more powerful than the match specifications allow, but bearing - the match specifications in mind, and what they can do, it's still +

This module provides the parse transformation that makes calls to + ets and + dbg:fun2ms/1 + translate into literal match specifications. It also provides the back end + for the same functions when called from the Erlang shell.

+ +

The translation from funs to match specifications + is accessed through the two "pseudo functions" + ets:fun2ms/1 and + dbg:fun2ms/1.

+ +

As everyone trying to use + ets:select/2 or + dbg seems to end up + reading this manual page, this description is an introduction to the + concept of match specifications.

+ +

Read the whole manual page if it is the first time you are using + the transformations.

+ +

Match specifications are used more or less as filters. They resemble + usual Erlang matching in a list comprehension or in a fun used with + lists:foldl/3, and so on. + However, the syntax of pure match specifications is awkward, as + they are made up purely by Erlang terms, and the language has no + syntax to make the match specifications more readable.

+ +

As the execution and structure of the match specifications are like + that of a fun, it is more straightforward + to write it using the familiar fun syntax and to have that + translated into a match specification automatically. A real fun is + clearly more powerful than the match specifications allow, but bearing + the match specifications in mind, and what they can do, it is still more convenient to write it all as a fun. This module contains the - code that simply translates the fun syntax into match_spec terms.

-

Let's start with an ets example. Using ets:select and - a match specification, one can filter out rows of a table and construct - a list of tuples containing relevant parts of the data in these - rows. Of course one could use ets:foldl instead, but the - select call is far more efficient. Without the translation, one has to - struggle with writing match specifications terms to accommodate this, - or one has to resort to the less powerful - ets:match(_object) calls, or simply give up and use - the more inefficient method of ets:foldl. Using the - ets:fun2ms transformation, a ets:select call - is at least as easy to write as any of the alternatives.

-

As an example, consider a simple table of employees:

+ code that translates the fun syntax into match specification + terms.

+ + +
+ Example 1 +

Using ets:select/2 + and a match specification, one can filter out rows of + a table and construct a list of tuples containing relevant parts + of the data in these rows. + One can use ets:foldl/3 + instead, but the ets:select/2 call is far more efficient. + Without the translation provided by ms_transform, + one must struggle with writing match specifications terms + to accommodate this.

+ +

Consider a simple table of employees:

+ -record(emp, {empno, %Employee number as a string, the key surname, %Surname of the employee givenname, %Given name of employee - dept, %Department one of {dev,sales,prod,adm} - empyear}). %Year the employee was employed + dept, %Department, one of {dev,sales,prod,adm} + empyear}). %Year the employee was employed +

We create the table using:

+ -ets:new(emp_tab,[{keypos,#emp.empno},named_table,ordered_set]). -

Let's also fill it with some randomly chosen data for the examples:

+ets:new(emp_tab, [{keypos,#emp.empno},named_table,ordered_set]). + +

We fill the table with randomly chosen data:

+ [{emp,"011103","Black","Alfred",sales,2000}, {emp,"041231","Doe","John",prod,2001}, @@ -96,167 +112,204 @@ ets:new(emp_tab,[{keypos,#emp.empno},named_table,ordered_set]). {emp,"535216","Chalker","Samuel",adm,1998}, {emp,"789789","Harrysson","Joe",adm,1996}, {emp,"963721","Scott","Juliana",dev,2003}, - {emp,"989891","Brown","Gabriel",prod,1999}] -

Now, the amount of data in the table is of course to small to justify - complicated ets searches, but on real tables, using select to get - exactly the data you want will increase efficiency remarkably.

-

Lets say for example that we'd want the employee numbers of - everyone in the sales department. One might use ets:match - in such a situation:

+ {emp,"989891","Brown","Gabriel",prod,1999}] + +

Assuming that we want the employee numbers of everyone in the sales + department, there are several ways.

+ +

ets:match/2 can be used:

+ 
 1> ets:match(emp_tab, {'_', '$1', '_', '_', sales, '_'}).
-[["011103"],["076324"]]
-

Even though ets:match does not require a full match - specification, but a simpler type, it's still somewhat unreadable, and - one has little control over the returned result, it's always a list of - lists. OK, one might use ets:foldl or - ets:foldr instead:

+[["011103"],["076324"]] + +

ets:match/2 uses a simpler type of match specification, + but it is still unreadable, and one has little control over the + returned result. It is always a list of lists.

+ +

ets:foldl/3 or + ets:foldr/3 can be used to avoid the nested lists:

+ ets:foldr(fun(#emp{empno = E, dept = sales},Acc) -> [E | Acc]; (_,Acc) -> Acc end, [], - emp_tab). -

Running that would result in ["011103","076324"] - , which at least gets rid of the extra lists. The fun is also quite + emp_tab). + +

The result is ["011103","076324"]. The fun is straightforward, so the only problem is that all the data from the - table has to be transferred from the table to the calling process for - filtering. That's inefficient compared to the ets:match + table must be transferred from the table to the calling process for + filtering. That is inefficient compared to the ets:match/2 call where the filtering can be done "inside" the emulator and only - the result is transferred to the process. Remember that ets tables are - all about efficiency, if it wasn't for efficiency all of ets could be - implemented in Erlang, as a process receiving requests and sending - answers back. One uses ets because one wants performance, and - therefore one wouldn't want all of the table transferred to the - process for filtering. OK, let's look at a pure - ets:select call that does what the ets:foldr - does:

+ the result is transferred to the process.

+ +

Consider a "pure" ets:select/2 call that does what + ets:foldr does:

+ -ets:select(emp_tab,[{#emp{empno = '$1', dept = sales, _='_'},[],['$1']}]). -

Even though the record syntax is used, it's still somewhat hard to +ets:select(emp_tab, [{#emp{empno = '$1', dept = sales, _='_'},[],['$1']}]). + +

Although the record syntax is used, it is still hard to read and even harder to write. The first element of the tuple, - #emp{empno = '$1', dept = sales, _='_'} tells what to - match, elements not matching this will not be returned at all, as in - the ets:match example. The second element, the empty list - is a list of guard expressions, which we need none, and the third + #emp{empno = '$1', dept = sales, _='_'}, tells what to + match. Elements not matching this are not returned, as in + the ets:match/2 example. The second element, the empty list, + is a list of guard expressions, which we do not need. The third element is the list of expressions constructing the return value (in - ets this almost always is a list containing one single term). In our - case '$1' is bound to the employee number in the head - (first element of tuple), and hence it is the employee number that is - returned. The result is ["011103","076324"], just as in - the ets:foldr example, but the result is retrieved much - more efficiently in terms of execution speed and memory consumption.

-

We have one efficient but hardly readable way of doing it and one - inefficient but fairly readable (at least to the skilled Erlang - programmer) way of doing it. With the use of ets:fun2ms, - one could have something that is as efficient as possible but still is - written as a filter using the fun syntax:

+ ETS this is almost always a list containing one single term). + In our case '$1' is bound to the employee number in the head + (first element of the tuple), and hence the employee number is + returned. The result is ["011103","076324"], as in + the ets:foldr/3 example, but the result is retrieved much + more efficiently in terms of execution speed and + memory consumption.

+ +

Using ets:fun2ms/1, we can combine the ease of use of + the ets:foldr/3 and the efficiency of the pure + ets:select/2 example:

+ -include_lib("stdlib/include/ms_transform.hrl"). -% ... - ets:select(emp_tab, ets:fun2ms( fun(#emp{empno = E, dept = sales}) -> E - end)). -

This may not be the shortest of the expressions, but it requires no - special knowledge of match specifications to read. The fun's head - should simply match what you want to filter out and the body returns - what you want returned. As long as the fun can be kept within the - limits of the match specifications, there is no need to transfer all - data of the table to the process for filtering as in the - ets:foldr example. In fact it's even easier to read then - the ets:foldr example, as the select call in itself - discards anything that doesn't match, while the fun of the - foldr call needs to handle both the elements matching and - the ones not matching.

-

It's worth noting in the above ets:fun2ms example that one - needs to include ms_transform.hrl in the source code, as this is - what triggers the parse transformation of the ets:fun2ms call - to a valid match specification. This also implies that the - transformation is done at compile time (except when called from the - shell of course) and therefore will take no resources at all in - runtime. So although you use the more intuitive fun syntax, it gets as - efficient in runtime as writing match specifications by hand.

-

Let's look at some more ets examples. Let's say one - wants to get all the employee numbers of any employee hired before the - year 2000. Using ets:match isn't an alternative here as - relational operators cannot be expressed there. Once again, an - ets:foldr could do it (slowly, but correct):

+ end)). + +

This example requires no special knowledge of match + specifications to understand. The head of the fun matches what + you want to filter out and the body returns what you want + returned. As long as the fun can be kept within the limits of the + match specifications, there is no need to transfer all table data + to the process for filtering as in the ets:foldr/3 + example. It is easier to read than the ets:foldr/3 example, + as the select call in itself discards anything that does not + match, while the fun of the ets:foldr/3 call needs to + handle both the elements matching and the ones not matching.

+ +

In the ets:fun2ms/1 example above, it is needed to + include ms_transform.hrl in the source code, as this is + what triggers the parse transformation of the ets:fun2ms/1 + call to a valid match specification. This also implies that the + transformation is done at compile time (except when called from + the shell) and therefore takes no resources in runtime. That is, + although you use the more intuitive fun syntax, it gets as + efficient in runtime as writing match specifications by hand.

+
+ +
+ Example 2 +

Assume that we want to get all the employee numbers of employees + hired before year 2000. Using ets:match/2 is not + an alternative here, as relational operators cannot be + expressed there. + Once again, ets:foldr/3 can do it (slowly, but correct):

+ [E | Acc]; (_,Acc) -> Acc end, [], emp_tab). ]]> -

The result will be - ["052341","076324","535216","789789","989891"], as - expected. Now the equivalent expression using a handwritten match - specification would look something like this:

+ +

The result is ["052341","076324","535216","789789","989891"], + as expected. The equivalent expression using a handwritten match + specification would look like this:

+ -

This gives the same result, the is in - the guard part and therefore discards anything that does not have a - empyear (bound to '$2' in the head) less than 2000, just as the guard - in the foldl example. Lets jump on to writing it using - ets:fun2ms

+ +

This gives the same result. is in + the guard part and therefore discards anything that does not have an + empyear (bound to '$2' in the head) less than 2000, as + the guard in the foldr/3 example.

+ +

We write it using ets:fun2ms/1:

+ - E + E end)). ]]> -

Obviously readability is gained by using the parse transformation.

-

I'll show some more examples without the tiresome - comparing-to-alternatives stuff. Let's say we'd want the whole object - matching instead of only one element. We could of course assign a - variable to every part of the record and build it up once again in the - body of the fun, but it's easier to do like this:

+
+ +
+ Example 3 +

Assume that we want the whole object matching instead of only one + element. One alternative is to assign a variable to every part + of the record and build it up once again in the body of the fun, but + the following is easier:

+ Obj - end)). ]]> -

Just as in ordinary Erlang matching, you can bind a variable to the - whole matched object using a "match in then match", i.e. a - =. Unfortunately this is not general in fun's translated - to match specifications, only on the "top level", i.e. matching the - whole object arriving to be matched into a separate variable, - is it allowed. For the one's used to writing match specifications by - hand, I'll have to mention that the variable A will simply be - translated into '$_'. It's not general, but it has very common usage, - why it is handled as a special, but useful, case. If this bothers you, - the pseudo function object also returns the whole matched - object, see the part about caveats and limitations below.

-

Let's do something in the fun's body too: Let's say - that someone realizes that there are a few people having an employee - number beginning with a zero (0), which shouldn't be - allowed. All those should have their numbers changed to begin with a - one (1) instead and one wants the - list ,}]]]> created:

+ end)).]]> + +

As in ordinary Erlang matching, you can bind a variable to the + whole matched object using a "match inside the match", that is, a + =. Unfortunately in funs translated to match specifications, + it is allowed only at the "top-level", that is, + matching the whole object arriving to be matched + into a separate variable. + If you are used to writing match specifications by hand, we + mention that variable A is simply translated into '$_'. + Alternatively, pseudo function object/0 + also returns the whole matched object, see section + + Warnings and Restrictions.

+
+ +
+ Example 4 +

This example concerns the body of the fun. Assume that all employee + numbers beginning with zero (0) must be changed to begin with + one (1) instead, and that we want to create the list + ,}]]]>:

+ ets:select(emp_tab, ets:fun2ms( fun(#emp{empno = [$0 | Rest] }) -> {[$0|Rest],[$1|Rest]} - end)). -

As a matter of fact, this query hits the feature of partially bound - keys in the table type ordered_set, so that not the whole - table need be searched, only the part of the table containing keys - beginning with 0 is in fact looked into.

-

The fun of course can have several clauses, so that if one could do - the following: For each employee, if he or she is hired prior to 1997, - return the tuple }]]>, for each hired 1997 - or later, but before 2001, return }]]>, for all others return }]]>. All except for the ones named Smith as - they would be affronted by anything other than the tag - guru and that is also what's returned for their numbers; - }]]>:

+ end)). + +

This query hits the feature of partially bound + keys in table type ordered_set, so that not the whole + table needs to be searched, only the part containing keys + beginning with 0 is looked into.

+
+ +
+ Example 5 +

The fun can have many clauses. Assume that we want to do + the following:

+ + + +

If an employee started before 1997, return the tuple + }]]>.

+ + +

If an employee started 1997 or later, but before 2001, return + }]]>.

+ + +

For all other employees, return + }]]>, except for those + named Smith as they would be affronted by anything other + than the tag guru and that is also what is returned for their + numbers: }]]>.

+ + + +

This is accomplished as follows:

+ @@ -268,7 +321,9 @@ ets:select(emp_tab, ets:fun2ms( (#emp{empno = E, empyear = Y}) -> % 1997 -- 2001 {rookie, E} end)). ]]> -

The result will be:

+ +

The result is as follows:

+ [{rookie,"011103"}, {rookie,"041231"}, @@ -278,162 +333,207 @@ ets:select(emp_tab, ets:fun2ms( {rookie,"535216"}, {inventory,"789789"}, {newbie,"963721"}, - {rookie,"989891"}] -

and so the Smith's will be happy...

-

So, what more can you do? Well, the simple answer would be; look - in the documentation of match specifications in ERTS users - guide. However let's briefly go through the most useful "built in - functions" that you can use when the fun is to be - translated into a match specification by ets:fun2ms (it's - worth mentioning, although it might be obvious to some, that calling - other functions than the one's allowed in match specifications cannot - be done. No "usual" Erlang code can be executed by the fun being - translated by fun2ms, the fun is after all limited + {rookie,"989891"}] +

+ +
+ Useful BIFs +

What more can you do? A simple answer is: see the documentation of + match specifications + in ERTS User's Guide. + However, the following is a brief overview of the most useful "built-in + functions" that you can use when the fun is to be translated into a match + specification by + ets:fun2ms/1. It is not + possible to call other functions than those allowed in match + specifications. No "usual" Erlang code can be executed by the fun that + is translated by ets:fun2ms/1. The fun is limited exactly to the power of the match specifications, which is - unfortunate, but the price one has to pay for the execution speed of - an ets:select compared to ets:foldl/foldr).

-

The head of the fun is obviously a head matching (or mismatching) - one parameter, one object of the table we select + unfortunate, but the price one must pay for the execution speed of + ets:select/2 compared to ets:foldl/foldr.

+ +

The head of the fun is a head matching (or mismatching) + one parameter, one object of the table we select from. The object is always a single variable (can be _) or - a tuple, as that's what's in ets, dets and - mnesia tables (the match specification returned by - ets:fun2ms can of course be used with - dets:select and mnesia:select as well as - with ets:select). The use of = in the head - is allowed (and encouraged) on the top level.

+ a tuple, as ETS, Dets, and Mnesia tables include + that. The match specification returned by ets:fun2ms/1 can + be used with dets:select/2 and mnesia:select/2, and + with ets:select/2. The use of = in the head + is allowed (and encouraged) at the top-level.

+

The guard section can contain any guard expression of Erlang. - Even the "old" type test are allowed on the toplevel of the guard - (integer(X) instead of is_integer(X)). As the new type tests (the - is_ tests) are in practice just guard bif's they can also - be called from within the body of the fun, but so they can in ordinary - Erlang code. Also arithmetics is allowed, as well as ordinary guard - bif's. Here's a list of bif's and expressions:

+ The following is a list of BIFs and expressions:

+ - The type tests: is_atom, is_float, is_integer, - is_list, is_number, is_pid, is_port, is_reference, is_tuple, - is_binary, is_function, is_record - The boolean operators: not, and, or, andalso, orelse - The relational operators: >, >=, <, =<, =:=, ==, =/=, /= - Arithmetics: +, -, *, div, rem - Bitwise operators: band, bor, bxor, bnot, bsl, bsr - The guard bif's: abs, element, hd, length, node, round, size, tl, - trunc, self - The obsolete type test (only in guards): - atom, float, integer, - list, number, pid, port, reference, tuple, - binary, function, record + +

Type tests: is_atom, is_float, is_integer, + is_list, is_number, is_pid, is_port, + is_reference, is_tuple, is_binary, + is_function, is_record

+ + +

Boolean operators: not, and, or, + andalso, orelse

+ + +

Relational operators: >, >=, <, =<, =:=, ==, =/=, /=

+ + +

Arithmetics: +, -, *, + div, rem

+ + +

Bitwise operators: band, bor, bxor, bnot, + bsl, bsr

+ + +

The guard BIFs: abs, element, + hd, length, + node, round, size, tl, trunc, + self

+ +

Contrary to the fact with "handwritten" match specifications, the is_record guard works as in ordinary Erlang code.

-

Semicolons (;) in guards are allowed, the result will be (as - expected) one "match_spec-clause" for each semicolon-separated - part of the guard. The semantics being identical to the Erlang + +

Semicolons (;) in guards are allowed, the result is (as + expected) one "match specification clause" for each semicolon-separated + part of the guard. The semantics is identical to the Erlang semantics.

-

The body of the fun is used to construct the - resulting value. When selecting from tables one usually just construct + +

The body of the fun is used to construct the + resulting value. When selecting from tables, one usually construct a suiting term here, using ordinary Erlang term construction, like - tuple parentheses, list brackets and variables matched out in the - head, possibly in conjunction with the occasional constant. Whatever - expressions are allowed in guards are also allowed here, but there are - no special functions except object and + tuple parentheses, list brackets, and variables matched out in the + head, possibly with the occasional constant. Whatever + expressions are allowed in guards are also allowed here, but no special + functions exist except object and bindings (see further down), which returns the whole - matched object and all known variable bindings respectively.

+ matched object and all known variable bindings, respectively.

+

The dbg variants of match specifications have an - imperative approach to the match specification body, the ets dialect - hasn't. The fun body for ets:fun2ms returns the result - without side effects, and as matching (=) in the body of + imperative approach to the match specification body, the ETS + dialect has not. The fun body for ets:fun2ms/1 returns the result + without side effects. As matching (=) in the body of the match specifications is not allowed (for performance reasons) the - only thing left, more or less, is term construction...

-

Let's move on to the dbg dialect, the slightly - different match specifications translated by dbg:fun2ms.

-

The same reasons for using the parse transformation applies to - dbg, maybe even more so as filtering using Erlang code is - simply not a good idea when tracing (except afterwards, if you trace - to file). The concept is similar to that of ets:fun2ms - except that you usually use it directly from the shell (which can also - be done with ets:fun2ms).

-

Let's manufacture a toy module to trace on

+ only thing left, more or less, is term construction.

+
+ +
+ Example with dbg +

This section describes the slightly different match specifications + translated by + dbg:fun2ms/1.

+ +

The same reasons for using the parse transformation apply to + dbg, maybe even more, as filtering using Erlang code is + not a good idea when tracing (except afterwards, if you trace + to file). The concept is similar to that of ets:fun2ms/1 + except that you usually use it directly from the shell + (which can also be done with ets:fun2ms/1).

+ +

The following is an example module to trace on:

+ -module(toy). -export([start/1, store/2, retrieve/1]). start(Args) -> - toy_table = ets:new(toy_table,Args). + toy_table = ets:new(toy_table, Args). store(Key, Value) -> - ets:insert(toy_table,{Key,Value}). + ets:insert(toy_table, {Key,Value}). retrieve(Key) -> - [{Key, Value}] = ets:lookup(toy_table,Key), - Value. -

During model testing, the first test bails out with a + [{Key, Value}] = ets:lookup(toy_table, Key), + Value. + +

During model testing, the first test results in {badmatch,16} in {toy,start,1}, why?

-

We suspect the ets call, as we match hard on the return value, but - want only the particular new call with - toy_table as first parameter. - So we start a default tracer on the node:

+ +

We suspect the ets:new/2 call, as we match hard on the + return value, but want only the particular new/2 call with + toy_table as first parameter. So we start a default tracer + on the node:

+ 
 1> dbg:tracer().
 {ok,<0.88.0>}
-

And so we turn on call tracing for all processes, we are going to - make a pretty restrictive trace pattern, so there's no need to call - trace only a few processes (it usually isn't):

+ +

We turn on call tracing for all processes, we want to + make a pretty restrictive trace pattern, so there is no need to call + trace only a few processes (usually it is not):

+ 
 2> dbg:p(all,call).
-{ok,[{matched,nonode@nohost,25}]}
-

It's time to specify the filter. We want to view calls that resemble - )]]>:

+{ok,[{matched,nonode@nohost,25}]} + +

We specify the filter, we want to view calls that resemble + )]]>:

+ 
 3> dbg:tp(ets,new,dbg:fun2ms(fun([toy_table,_]) -> true end)).
-{ok,[{matched,nonode@nohost,1},{saved,1}]}
-

As can be seen, the fun's used with - dbg:fun2ms takes a single list as parameter instead of a +{ok,[{matched,nonode@nohost,1},{saved,1}]} + +

As can be seen, the fun used with + dbg:fun2ms/1 takes a single list as parameter instead of a single tuple. The list matches a list of the parameters to the traced - function. A single variable may also be used of course. The body - of the fun expresses in a more imperative way actions to be taken if - the fun head (and the guards) matches. I return true here, but it's - only because the body of a fun cannot be empty, the return value will - be discarded.

-

When we run the test of our module now, we get the following trace - output:

+ function. A single variable can also be used. The body + of the fun expresses, in a more imperative way, actions to be taken if + the fun head (and the guards) matches. true is returned here, + only because the body of a fun cannot be empty. The return value + is discarded.

+ +

The following trace output is received during test:

+ ) call ets:new(toy_table,[ordered_set]) ]]> -

Let's play we haven't spotted the problem yet, and want to see what - ets:new returns. We do a slightly different trace - pattern:

+(<0.86.0>) call ets:new(toy_table, [ordered_set]) ]]> + +

Assume that we have not found the problem yet, and want to see what + ets:new/2 returns. We use a slightly different trace pattern:

+ 
 4> dbg:tp(ets,new,dbg:fun2ms(fun([toy_table,_]) -> return_trace() end)).
-

Resulting in the following trace output when we run the test:

+ +

The following trace output is received during test:

+ ) call ets:new(toy_table,[ordered_set]) (<0.86.0>) returned from ets:new/2 -> 24 ]]> -

The call to return_trace, makes a trace message appear + +

The call to return_trace results in a trace message when the function returns. It applies only to the specific function call triggering the match specification (and matching the head/guards of - the match specification). This is the by far the most common call in the + the match specification). This is by far the most common call in the body of a dbg match specification.

-

As the test now fails with {badmatch,24}, it's obvious - that the badmatch is because the atom toy_table does not - match the number returned for an unnamed table. So we spotted the - problem, the table should be named and the arguments supplied by our - test program does not include named_table. We rewrite the - start function to:

+ +

The test now fails with {badmatch,24} because the atom + toy_table does not match the number returned for an unnamed table. + So, the problem is found, the table is to be named, and the arguments + supplied by the test program do not include named_table. We + rewrite the start function:

+ start(Args) -> - toy_table = ets:new(toy_table,[named_table |Args]). -

And with the same tracing turned on, we get the following trace - output:

+ toy_table = ets:new(toy_table, [named_table|Args]). + +

With the same tracing turned on, the following trace output is + received:

+ ) call ets:new(toy_table,[named_table,ordered_set]) (<0.86.0>) returned from ets:new/2 -> toy_table ]]> -

Very well. Let's say the module now passes all testing and goes into - the system. After a while someone realizes that the table - toy_table grows while the system is running and that for some - reason there are a lot of elements with atom's as keys. You had - expected only integer keys and so does the rest of the system. Well, - obviously not all of the system. You turn on call tracing and try to - see calls to your module with an atom as the key:

+ +

Assume that the module now passes all testing and goes into + the system. After a while, it is found that table + toy_table grows while the system is running and that + there are many elements with atoms as keys. We expected + only integer keys and so does the rest of the system, but + clearly not the entire system. We turn on call tracing and try to + see calls to the module with an atom as the key:

+ 
 1> dbg:tracer().
 {ok,<0.88.0>}
@@ -441,80 +541,101 @@ start(Args) ->
 {ok,[{matched,nonode@nohost,25}]}
 3> dbg:tpl(toy,store,dbg:fun2ms(fun([A,_]) when is_atom(A) -> true end)).
 {ok,[{matched,nonode@nohost,1},{saved,1}]}
-

We use dbg:tpl here to make sure to catch local calls - (let's say the module has grown since the smaller version and we're - not sure this inserting of atoms is not done locally...). When in - doubt always use local call tracing.

-

Let's say nothing happens when we trace in this way. Our function - is never called with these parameters. We make the conclusion that - someone else (some other module) is doing it and we realize that we - must trace on ets:insert and want to see the calling function. The - calling function may be retrieved using the match specification - function caller and to get it into the trace message, one - has to use the match spec function message. The filter - call looks like this (looking for calls to ets:insert):

+ +

We use dbg:tpl/3 to ensure to catch local calls + (assume that the module has grown since the smaller version and we are + unsure if this inserting of atoms is not done locally). When in + doubt, always use local call tracing.

+ +

Assume that nothing happens when tracing in this way. The function + is never called with these parameters. We conclude that + someone else (some other module) is doing it and realize that we + must trace on ets:insert/2 and want to see the calling function. + The calling function can be retrieved using the match specification + function caller. To get it into the trace message, the match + specification function message must be used. The filter + call looks like this (looking for calls to ets:insert/2):

+ 
 4> dbg:tpl(ets,insert,dbg:fun2ms(fun([toy_table,{A,_}]) when is_atom(A) -> 
                                     message(caller()) 
                                   end)). 
-{ok,[{matched,nonode@nohost,1},{saved,2}]}
-

The caller will now appear in the "additional message" part of the - trace output, and so after a while, the following output comes:

+{ok,[{matched,nonode@nohost,1},{saved,2}]} + +

The caller is now displayed in the "additional message" part of the + trace output, and the following is displayed after a while:

+ ) call ets:insert(toy_table,{garbage,can}) ({evil_mod,evil_fun,2}) ]]> -

You have found out that the function evil_fun of the - module evil_mod, with arity 2, is the one - causing all this trouble.

-

This was just a toy example, but it illustrated the most used - calls in match specifications for dbg The other, more - esotheric calls are listed and explained in the Users guide of the ERTS application, they really are beyond the scope of this - document.

-

To end this chatty introduction with something more precise, here - follows some parts about caveats and restrictions concerning the fun's - used in conjunction with ets:fun2ms and - dbg:fun2ms:

+ +

You have realized that function evil_fun of the + evil_mod module, with arity 2, is causing all this trouble. +

+ +

This example illustrates the most used calls in match specifications for + dbg. The other, more esoteric, calls are listed and explained in + Match specifications in Erlang + in ERTS User's Guide, as they are beyond + the scope of this description.

+
+ +
+ Warnings and Restrictions + +

The following warnings and restrictions apply to the funs used in + with ets:fun2ms/1 and dbg:fun2ms/1.

+ -

To use the pseudo functions triggering the translation, one - has to include the header file ms_transform.hrl - in the source code. Failure to do so will possibly result in - runtime errors rather than compile time, as the expression may +

To use the pseudo functions triggering the translation, + ensure to include the header file ms_transform.hrl + in the source code. Failure to do so possibly results in + runtime errors rather than compile time, as the expression can be valid as a plain Erlang program without translation.

 + -

The fun has to be literally constructed inside the - parameter list to the pseudo functions. The fun cannot +

The fun must be literally constructed inside the + parameter list to the pseudo functions. The fun cannot be bound to a variable first and then passed to - ets:fun2ms or dbg:fun2ms, i.e this - will work: ets:fun2ms(fun(A) -> A end) but not this: - F = fun(A) -> A end, ets:fun2ms(F). The later will result - in a compile time error if the header is included, otherwise a - runtime error. Even if the later construction would ever - appear to work, it really doesn't, so don't ever use it.

+ ets:fun2ms/1 or dbg:fun2ms/1. For example, + ets:fun2ms(fun(A) -> A end) works, but not + F = fun(A) -> A end, ets:fun2ms(F). The latter results + in a compile-time error if the header is included, otherwise a + runtime error.

 -

Several restrictions apply to the fun that is being translated - into a match_spec. To put it simple you cannot use anything in - the fun that you cannot use in a match_spec. This means that, + +

Many restrictions apply to the fun that is translated into a match + specification. To put it simple: you cannot use anything in the fun + that you cannot use in a match specification. This means that, among others, the following restrictions apply to the fun itself:

+ - Functions written in Erlang cannot be called, neither - local functions, global functions or real fun's - Everything that is written as a function call will be - translated into a match_spec call to a builtin function, so that - the call is_list(X) will be translated to {'is_list', '$1'} ('$1' is just an example, the numbering may - vary). If one tries to call a function that is not a match_spec - builtin, it will cause an error. - Variables occurring in the head of the fun will be - replaced by match_spec variables in the order of occurrence, so - that the fragment fun({A,B,C}) will be replaced by - {'$1', '$2', '$3'} etc. Every occurrence of such a - variable later in the match_spec will be replaced by a - match_spec variable in the same way, so that the fun - fun({A,B}) when is_atom(A) -> B end will be translated into - [{{'$1','$2'},[{is_atom,'$1'}],['$2']}]. -

Variables that are not appearing in the head are imported - from the environment and made into - match_spec const expressions. Example from the shell:

+

Functions written in Erlang cannot be called, neither can + local functions, global functions, or real funs.

+ + +

Everything that is written as a function call is translated + into a match specification call to a built-in function, so that + the call is_list(X) is translated to {'is_list', '$1'} + ('$1' is only an example, the numbering can vary). + If one tries to call a function that is not a match specification + built-in, it causes an error.

+ + +

Variables occurring in the head of the fun are replaced by + match specification variables in the order of occurrence, so + that fragment fun({A,B,C}) is replaced by + {'$1', '$2', '$3'}, and so on. Every occurrence of such a + variable in the match specification is replaced by a match + specification variable in the same way, so that the fun + fun({A,B}) when is_atom(A) -> B end is translated into + [{{'$1','$2'},[{is_atom,'$1'}],['$2']}].

+ + +

Variables that are not included in the head are imported + from the environment and made into match specification + const expressions. Example from the shell:

 1> X = 25.
 25
@@ -523,7 +644,7 @@ start(Args) ->
       
       
         
Matching with = cannot be used in the body. It can only
-          be used on the top level in the head of the fun. 
+          be used on the top-level in the head of the fun.
           Example from the shell again:

         
 1> ets:fun2ms(fun({A,[B|C]} = D) when A > B -> D end).
@@ -534,106 +655,125 @@ match_spec
 {error,transform_error}
 3> ets:fun2ms(fun({A,[B|C]}) when A > B -> D = [B|C], D end).
 Error: fun with body matching ('=' in body) is illegal as match_spec
-{error,transform_error}        

-        
All variables are bound in the head of a match_spec, so the 
-          translator can not allow multiple bindings. The special case
-          when matching is done on the top level makes the variable bind
-          to '$_' in the resulting match_spec, it is to allow a more
-          natural access to the whole matched object. The pseudo
-          function object() could be used instead, see below. 
-          The following expressions are translated equally: 

+{error,transform_error}
+

All variables are bound in the head of a match specification, so + the translator cannot allow multiple bindings. The special case + when matching is done on the top-level makes the variable bind + to '$_' in the resulting match specification. It is to allow + a more natural access to the whole matched object. Pseudo + function object() can be used instead, see below.

+

The following expressions are translated equally:

ets:fun2ms(fun({a,_} = A) -> A end). ets:fun2ms(fun({a,_}) -> object() end). -

The special match_spec variables '$_' and '$*' +

The special match specification variables '$_' and '$*' can be accessed through the pseudo functions object() (for '$_') and bindings() (for '$*'). - as an example, one could translate the following - ets:match_object/2 call to a ets:select call:

+ As an example, one can translate the following + ets:match_object/2 call to a ets:select/2 call:

ets:match_object(Table, {'$1',test,'$2'}). -

...is the same as...

+

This is the same as:

ets:select(Table, ets:fun2ms(fun({A,test,B}) -> object() end)). -

(This was just an example, in this simple case the former - expression is probably preferable in terms of readability). - The ets:select/2 call will conceptually look like this +

In this simple case, the former + expression is probably preferable in terms of readability.

+

The ets:select/2 call conceptually looks like this in the resulting code:

ets:select(Table, [{{'$1',test,'$2'},[],['$_']}]). -

Matching on the top level of the fun head might feel like a +

Matching on the top-level of the fun head can be a more natural way to access '$_', see above.

 - Term constructions/literals are translated as much as is - needed to get them into valid match_specs, so that tuples are - made into match_spec tuple constructions (a one element tuple - containing the tuple) and constant expressions are used when - importing variables from the environment. Records are also - translated into plain tuple constructions, calls to element - etc. The guard test is_record/2 is translated into - match_spec code using the three parameter version that's built - into match_specs, so that is_record(A,t) is translated - into {is_record,'$1',t,5} given that the record size of - record type t is 5. - Language constructions like case, if, - catch etc that are not present in match_specs are not - allowed. - If the header file ms_transform.hrl is not included, - the fun won't be translated, which may result in a - runtime error (depending on if the fun is valid in a - pure Erlang context). Be absolutely sure that the header is - included when using ets and dbg:fun2ms/1 in - compiled code. - If the pseudo function triggering the translation is - ets:fun2ms/1, the fun's head must contain a single - variable or a single tuple. If the pseudo function is - dbg:fun2ms/1 the fun's head must contain a single - variable or a single list. + +

Term constructions/literals are translated as much as is needed to + get them into valid match specification. This way tuples are made + into match specification tuple constructions (a one element tuple + containing the tuple) and constant expressions are used when + importing variables from the environment. Records are also + translated into plain tuple constructions, calls to element, + and so on. The guard test is_record/2 is translated into + match specification code using the three parameter version that is + built into match specification, so that is_record(A,t) is + translated into {is_record,'$1',t,5} if the record + size of record type t is 5.

+ + +

Language constructions such as case, if, and + catch that are not present in match specifications are not + allowed.

+ + +

If header file ms_transform.hrl is not included, + the fun is not translated, which can result in a + runtime error (depending on whether the fun is + valid in a pure Erlang context).

+

Ensure that the header is included when using ets and + dbg:fun2ms/1 in compiled code.

+ + +

If pseudo function triggering the translation is + ets:fun2ms/1, the head of the fun must contain a single + variable or a single tuple. If the pseudo function is + dbg:fun2ms/1, the head of the fun must contain a single + variable or a single list.

+ -

The translation from fun's to match_specs is done at compile +

The translation from funs to match specifications is done at compile time, so runtime performance is not affected by using these pseudo - functions. The compile time might be somewhat longer though.

-

For more information about match_specs, please read about them - in ERTS users guide.

- + functions.

+

For more information about match specifications, see the + Match specifications in Erlang + in ERTS User's Guide.

+
+ - - Transforms Erlang abstract format containing calls to ets/dbg:fun2ms into literal match specifications. - Option list, required but not used. + + Error formatting function as required by the parse transformation interface. -

Implements the actual transformation at compile time. This - function is called by the compiler to do the source code - transformation if and when the ms_transform.hrl header - file is included in your source code. See the ets and - dbg:fun2ms/1 function manual pages for - documentation on how to use this parse_transform, see the - match_spec chapter in ERTS users guide for a - description of match specifications.

+

Takes an error code returned by one of the other functions + in the module and creates a textual description of the + error.

 + - - Used when transforming fun's created in the shell into match_specifications. - List of variable bindings in the shell environment. + + Transforms Erlang abstract format containing calls to + ets/dbg:fun2ms/1 into literal match specifications. + Option list, required but not used. + -

Implements the actual transformation when the fun2ms - functions are called from the shell. In this case the abstract - form is for one single fun (parsed by the Erlang shell), and - all imported variables should be in the key-value list passed - as BoundEnvironment. The result is a term, normalized, - i.e. not in abstract format.

+

Implements the transformation at compile time. This + function is called by the compiler to do the source code + transformation if and when header file ms_transform.hrl + is included in the source code.

+

For information about how to use this parse transformation, see + ets and + + dbg:fun2ms/1.

+

For a description of match specifications, see section + + Match Specification in Erlang in ERTS User's Guide.

 + - - Error formatting function as required by the parse_transform interface. + + Used when transforming funs created in the shell into + match_specifications. + List of variable bindings in the + shell environment. -

Takes an error code returned by one of the other functions - in the module and creates a textual description of the - error. Fairly uninteresting function actually.

+

Implements the transformation when the fun2ms/1 + functions are called from the shell. In this case, the abstract + form is for one single fun (parsed by the Erlang shell). + All imported variables are to be in the key-value list passed + as BoundEnvironment. The result is a term, + normalized, that is, not in abstract format.

 -- cgit v1.2.1