1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
|
# Packetized console mode
[TOC]
## Overview
Some EC board images are getting very close to their flash space size limits.
A significant part of the image is often taken by the text strings printed out
as debug messages or console command output.
Removing text strings from the image would allow to free up a lot of space
(approximately 10% in case of Cr50), on top of that moving console command
support code from the image would allow to free quite a bit more.
One requirement for such a facility would be that removing text strings from
the image is possible without touching the source code, so that all existing
boards could be retrofitted with the new approach without actual code changes.
## Post processing the source code
The vast majority of printouts generated by the EC images come from
invocations to console output generating functions (aka `cprintf()`,
`cprints()` and `cputs()`). In many cases these functions are invoked through
macros.
The proposed approach is to introduce additional processing steps where the
source code could be modified pre-compilation during build time.
In the standard processing case `.o` files are generated directly from `.c`
files by the compiler. To support the packet mode console the source files'
processing has to be more involved.
First the source code is taken through C preprocessor. This extends all macro
invocations, and each input `.c` file gets a corresponding generated `.E`
file.
Then invocations of the console output generating functions are substituted
such that the format strings can be replaced by their indices (see below).
This produces an additional set of intermediate files, each `.E` gets a
corresponding `.Ep` file. Then the set of `.Ep` files are taken through the
compilation step, resulting an a set of `.o` files generated.
In more details, the following steps are taken when generating the set of `.o`
files necessary to generate the executable image:
- run `.c` sources though C preprocessor, generating `.E` files instead of
`.o` files (.i.e. use -E compiler command line option instead of -c)
- once all sources are preoprocessed, invoke the `util_precompile.py` script
to scan all `.E` files together. For each instance of `cprintf()`,
`cprints()`, and `cputs()` found in `.E` files do the following:
- save the format string in a dictionary, unless the string is already
there. The keys in the dictionary are the format strings, the values - are
the string indices, for each new string the value is the length of the
dictionary before the string was added.
- replace the console output generating function names with `cmsgX`, where
`X` is the number of arguments, values from 0 to 8 are supported.
- scan the format string for format characters (%), and based on number of
format characters pick the `X` value for `cmsgX` function name.
- based on the format attributes analyze the parameters and make sure that
they can be typecasted to `uintptr_t`. This is the case for pretty much
any parameter other than 64 bit values, timestamps in particular. In case
there are 64 bit parameters, augment the code by adding a block around the
invocation, declaring a 64 bit variable, assigning it the parameter value
and using the variable address as the appropriate `cmsgX` parameter.
- Wile scanning the parameters, also prepare a 32 bit descriptor, with each
parameter's description packed into 4 bits, thus supporting up to 8
parameters.
- complete `cmsgX` call declaration giving it as the parameters the console
channel number (as is, retrieved from the source code), the index the text
string got when it was added to the dictionary, the 32 bit parameter
descriptor and the original parameters typecasted to `uintptr_t`, unless
processing of 64 bit value(s) took place, in which case pointer(s) to the
value(s) are used.
- for each `.E` file generate the `.Ep` file with all console output
generating functions replaced with `cmsgX()` invocations.
- once all .E files have been processed, the `util_precompile.py` script
converts the format string dictionary into a list of strings, each string
placed in the list at its index, so knowing the index the terminal program
can retrieve the format string. The list is serialized, compressed and saved
in a file.
## Sending packets to the terminal
Some code has to be added to process `cmsgX()` invocations.
Each of these functions prepares an array of parameters and calls the common
function, which processes the format descriptor and the parameters, and
generates a packet sent over the console channel to the terminal. The packet
has the following structure, very similar to the one used in the Acropora
project:
```c
struct console_packet {
/* Magic number = CONSOLE_PACKET_MAGIC */
uint8_t magic;
/*
* Packet sequence number, incremented each time sender sends a packet.
* So receiver can detect small numbers of corrupt/dropped packets.
*/
uint8_t sequence : 4;
/* Set if the sender had to discard packets due to buffer overflow. */
uint8_t overflow : 1;
uint8_t placeholder : 3;
/* Channel; values from enum console_channel */
uint8_t channel;
/* Bottom 48 bits of system time; enough for 8 years @ 1 us */
uint8_t timestamp[6];
/*
* Length of variable-length section in bytes, not including the
* packet end trailer.
*/
uint8_t data_len;
/* Index of the format string in the string database. */
uint16_t str_index;
/* Header checksum */
uint8_t crc;
/* Fixed length header to here.
*
* Followed by variable-length data, if data_len > 0.
*
* params: 1-8 of objects matching the format of the string indexed by
* 'str_index' above.
*
* CONSOLE_PACKET_END, as a validity check that we haven't dropped
* anything. A checksum or CRC would be kinda expensive for debug
* data. Note that it is not present if data_len == 0.
*/
}
```
The data part of the packet is the concatenated values of the parameters
processed in accordance with the format: all integer values less than 64 bits
in size and pointers are transferred as 4 byte entities. In case the format
calls for a 64 bit value, the parameter is interpreted as an address or an 8
byte value, which is retrieved from memory and added to the packet. In case
the format calls for a string, the string is included in the packet, along
with the trailing zero.
Inclusion of `__func__` in the parameter list is a special case. The name of
the function is added to the string dictionary, and 3 bytes are sent in the
packet, 0xff and then the number `__func__` was assigned in the string
database. This allows the terminal to tell that not the actual string but the
index is sent as the parameter.
## Terminal program
A Python script (`acroterm.py`) was copied from the Acropora project and
modified to support the changed packet format.
The script receives the name of the file containing the format strings blob as
one of command lie parameters. When starting, the scrip attaches to the TTY
device (be it UART or USB channel) and searches for the packet header
characters in the stream. Symbols received before packet header is encountered
or between packets are sent to the console directly.
When packets are received, header integrity is verified and the `str_index`
field from the header is used to retrieve the format string from the blob. The
string is scanned for the format characters, and the data section of the
packet is interpreted according to the format specification, recreated strings
are sent to the console.
Data received out of packets is sent to the console directly and is displayed
using a different color. This, among other things, allows to display text
generated by early boot stages and in general support builds which do not yet
deploy packet mode.
|
