Source range tracking in libcpp and C FE, with bit-packing optimization

This patch combines:
  [PATCH 05/10] Add ranges to libcpp tokens (via ad-hoc data, unoptimized)
  [PATCH 06/10] Track expression ranges in C frontend
  [PATCH 07/10] Add plugin to recursively dump the source-ranges in a tree (v2)
  [PATCH 08/10] Wire things up so that libcpp users get token underlines
  [PATCH 09/10] Delay some resolution of ad-hoc locations, preserving ranges
  [PATCH 10/10] Compress short ranges into source_location
  [PATCH] libcpp: add examples to source_location description
along with fixes for the nits identified during review.

gcc/ChangeLog:
	* Makefile.in (OBJS): Add gcc-rich-location.o.
	* diagnostic.c (diagnostic_append_note): Pass line_table to
	rich_location ctor.
	(emit_diagnostic): Likewise.
	(inform): Likewise.
	(inform_n): Likewise.
	(warning): Likewise.
	(warning_at): Likewise.
	(warning_n): Likewise.
	(pedwarn): Likewise.
	(permerror): Likewise.
	(error): Likewise.
	(error_n): Likewise.
	(error_at): Likewise.
	(sorry): Likewise.
	(fatal_error): Likewise.
	(internal_error): Likewise.
	(internal_error_no_backtrace): Likewise.
	(source_range::debug): Likewise.
	* gcc-rich-location.c: New file.
	* gcc-rich-location.h: New file.
	* genmatch.c (fatal_at): Pass line_table to rich_location ctor.
	(warning_at): Likewise.
	* gimple.h (gimple_set_block): Use set_block function.
	* input.c (dump_line_table_statistics): Dump stats on how many
	ranges were optimized vs how many needed ad-hoc table.
	(write_digit_row): Add "map" param; use its range_bits
	to calculate the per-character offset.
	(dump_location_info): Print the range and column bits for each
	ordinary map.  Use the range bits to calculate the per-character
	offset.  Pass the map as a new param to the various calls to
	write_digit_row.  Eliminate uses of
	ORDINARY_MAP_NUMBER_OF_COLUMN_BITS.
	* print-tree.c (print_node): Print any source range information.
	* rtl-error.c (diagnostic_for_asm): Likewise.
	* toplev.c (general_init): Initialize line_table's
	default_range_bits.
	* tree-cfg.c (move_block_to_fn): Likewise.
	(move_block_to_fn): Likewise.
	* tree-inline.c (copy_phis_for_bb): Likewise.
	* tree.c (tree_set_block): Likewise.
	(get_pure_location): New function.
	(set_source_range): New functions.
	(set_block): New function.
	(set_source_range): New functions.
	* tree.h (CAN_HAVE_RANGE_P): New.
	(EXPR_LOCATION_RANGE): New.
	(EXPR_HAS_RANGE): New.
	(get_expr_source_range): New inline function.
	(DECL_LOCATION_RANGE): New.
	(set_source_range): New decls.
	(get_decl_source_range): New inline function.

gcc/ada/ChangeLog:
	* gcc-interface/trans.c (Sloc_to_locus): Add line_table param when
	calling linemap_position_for_line_and_column.

gcc/c-family/ChangeLog:
	* c-common.c (c_fully_fold_internal): Capture existing souce_range,
	and store it on the result.
	* c-opts.c (c_common_init_options): Set
	global_dc->colorize_source_p.

gcc/c/ChangeLog:
	* c-decl.c (warn_defaults_to): Pass line_table to
	rich_location ctor.
	* c-errors.c (pedwarn_c99): Likewise.
	(pedwarn_c90): Likewise.
	* c-parser.c (set_c_expr_source_range): New functions.
	(c_token::get_range): New method.
	(c_token::get_finish): New method.
	(c_parser_expr_no_commas): Call set_c_expr_source_range on the ret
	based on the range from the start of the LHS to the end of the
	RHS.
	(c_parser_conditional_expression): Likewise, based on the range
	from the start of the cond.value to the end of exp2.value.
	(c_parser_binary_expression): Call set_c_expr_source_range on
	the stack values for TRUTH_ANDIF_EXPR and TRUTH_ORIF_EXPR.
	(c_parser_cast_expression): Call set_c_expr_source_range on ret
	based on the cast_loc through to the end of the expr.
	(c_parser_unary_expression): Likewise, based on the
	op_loc through to the end of op.
	(c_parser_sizeof_expression) Likewise, based on the start of the
	sizeof token through to either the closing paren or the end of
	expr.
	(c_parser_postfix_expression): Likewise, using the token range,
	or from the open paren through to the close paren for
	parenthesized expressions.
	(c_parser_postfix_expression_after_primary): Likewise, for
	various kinds of expression.
	* c-tree.h (struct c_expr): Add field "src_range".
	(c_expr::get_start): New method.
	(c_expr::get_finish): New method.
	(set_c_expr_source_range): New decls.
	* c-typeck.c (parser_build_unary_op): Call set_c_expr_source_range
	on ret for prefix unary ops.
	(parser_build_binary_op): Likewise, running from the start of
	arg1.value through to the end of arg2.value.

gcc/cp/ChangeLog:
	* error.c (pedwarn_cxx98): Pass line_table to rich_location ctor.

gcc/fortran/ChangeLog:
	* error.c (gfc_warning): Pass line_table to rich_location ctor.
	(gfc_warning_now_at): Likewise.
	(gfc_warning_now): Likewise.
	(gfc_error_now): Likewise.
	(gfc_fatal_error): Likewise.
	(gfc_error): Likewise.
	(gfc_internal_error): Likewise.

gcc/testsuite/ChangeLog:
	* gcc.dg/diagnostic-token-ranges.c: New file.
	* gcc.dg/diagnostic-tree-expr-ranges-2.c: New file.
	* gcc.dg/plugin/diagnostic-test-expressions-1.c: New file.
	* gcc.dg/plugin/diagnostic-test-show-trees-1.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_show_trees.c: New file.
	* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c (get_loc): Add
	line_table param when calling
	linemap_position_for_line_and_column.
	(test_show_locus): Pass line_table to rich_location ctors.
	(plugin_init): Remove setting of global_dc->colorize_source_p.
	* gcc.dg/plugin/diagnostic_plugin_test_tree_expression_range.c:
	New file.
	* gcc.dg/plugin/plugin.exp (plugin_test_list): Add
	diagnostic_plugin_test_tree_expression_range.c,
	diagnostic-test-expressions-1.c, diagnostic_plugin_show_trees.c,
	and diagnostic-test-show-trees-1.c.

libcpp/ChangeLog:
	* errors.c (cpp_diagnostic): Pass pfile->line_table to
	rich_location ctor.
	(cpp_diagnostic_with_line): Likewise.
	* include/cpplib.h (struct cpp_token): Update comment for src_loc
	to indicate that the range of the token is "baked into" the
	source_location.
	* include/line-map.h (source_location): Update the descriptive
	comment to reflect the packing scheme for short ranges, adding
	worked examples of location encoding.
	(struct line_map_ordinary): Drop field "column_bits" in favor
	of field "m_column_and_range_bits"; add field "m_range_bits".
	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS): Delete.
	(location_adhoc_data): Add source_range field.
	(struct line_maps): Add fields "default_range_bits",
	"num_optimized_ranges" and "num_unoptimized_ranges".
	(get_combined_adhoc_loc): Add source_range param.
	(get_range_from_loc): New declaration.
	(pure_location_p): New prototype.
	(COMBINE_LOCATION_DATA):  Add source_range param.
	(SOURCE_LINE): Update for renaming of column_bits.
	(SOURCE_COLUMN): Likewise.  Shift the column right by the map's
	range_bits.
	(LAST_SOURCE_LINE_LOCATION): Update for renaming of column_bits.
	(linemap_position_for_line_and_column): Add line_maps * params.
	(rich_location::rich_location): Likewise.
	* lex.c (_cpp_lex_direct): Capture the range of the token, baking
	it into token->src_loc via a call to COMBINE_LOCATION_DATA.
	* line-map.c (LINE_MAP_MAX_COLUMN_NUMBER): Reduce from 1U << 17 to
	1U << 12.
	(location_adhoc_data_hash): Add the src_range into
	the hash value.
	(location_adhoc_data_eq): Require equality of the src_range
	values.
	(can_be_stored_compactly_p): New function.
	(get_combined_adhoc_loc): Add src_range param, and store it,
	via a bit-packing scheme for short ranges, otherwise within the
	lookaside table.  Remove the requirement that data is non-NULL.
	(get_range_from_adhoc_loc): New function.
	(get_range_from_loc): New function.
	(pure_location_p): New function.
	(linemap_add): Ensure that start_location has zero for the
	range_bits, unless we're past LINE_MAP_MAX_LOCATION_WITH_COLS.
	Initialize range_bits to zero.  Assert that the start_location
	is "pure".
	(linemap_line_start): Assert that the
	column_and_range_bits >= range_bits.
	Update determinination of whether we need to start a new map
	using the effective column bits, without the range bits.
	Use the set's default_range_bits in new maps, apart from
	those with column_bits == 0, which should also have 0 range_bits.
	Increase the column bits for new maps by the range bits.
	When adding lines to an existing map, use set->highest_line
	directly rather than offsetting highest by SOURCE_COLUMN.
	Add assertions to sanity-check the return value.
	(linemap_position_for_column): Offset to_column by range_bits.
	Update set->highest_location if necessary.
	(linemap_position_for_line_and_column): Add line_maps * param.
	Update the calculation to offset the column by range_bits, and
	conditionalize it on being <= LINE_MAP_MAX_LOCATION_WITH_COLS.
	Bound it by LINEMAPS_MACRO_LOWEST_LOCATION.  Update
	set->highest_location if necessary.
	(linemap_position_for_loc_and_offset): Handle ad-hoc locations;
	pass "set" to linemap_position_for_line_and_column.
	(linemap_macro_map_loc_unwind_toward_spelling): Add line_maps
	param.  Handle ad-hoc locations.
	(linemap_location_in_system_header_p): Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_macro_loc_to_spelling_point): Retain ad-hoc locations.
	Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_resolve_location): Retain ad-hoc locations.  Pass on
	"set" to call to linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_unwind_toward_expansion):  Pass on "set" to call to
	linemap_macro_map_loc_unwind_toward_spelling.
	(linemap_expand_location): Extract the data pointer before
	extracting the location.
	(rich_location::rich_location): Add line_maps param; use it to
	extract the range from the source_location.
	* location-example.txt: Regenerate, showing new representation.

From-SVN: r230331

7 files changed, 622 insertions, 161 deletions

diff --git a/libcpp/ChangeLog b/libcpp/ChangeLog
index 57ac5bfbde1..5a87c1791c4 100644
--- a/libcpp/ChangeLog
+++ b/libcpp/ChangeLog
@@ -1,3 +1,85 @@
+2015-11-13  David Malcolm  <dmalcolm@redhat.com>
+
+	* errors.c (cpp_diagnostic): Pass pfile->line_table to
+	rich_location ctor.
+	(cpp_diagnostic_with_line): Likewise.
+	* include/cpplib.h (struct cpp_token): Update comment for src_loc
+	to indicate that the range of the token is "baked into" the
+	source_location.
+	* include/line-map.h (source_location): Update the descriptive
+	comment to reflect the packing scheme for short ranges, adding
+	worked examples of location encoding.
+	(struct line_map_ordinary): Drop field "column_bits" in favor
+	of field "m_column_and_range_bits"; add field "m_range_bits".
+	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS): Delete.
+	(location_adhoc_data): Add source_range field.
+	(struct line_maps): Add fields "default_range_bits",
+	"num_optimized_ranges" and "num_unoptimized_ranges".
+	(get_combined_adhoc_loc): Add source_range param.
+	(get_range_from_loc): New declaration.
+	(pure_location_p): New prototype.
+	(COMBINE_LOCATION_DATA):  Add source_range param.
+	(SOURCE_LINE): Update for renaming of column_bits.
+	(SOURCE_COLUMN): Likewise.  Shift the column right by the map's
+	range_bits.
+	(LAST_SOURCE_LINE_LOCATION): Update for renaming of column_bits.
+	(linemap_position_for_line_and_column): Add line_maps * params.
+	(rich_location::rich_location): Likewise.
+	* lex.c (_cpp_lex_direct): Capture the range of the token, baking
+	it into token->src_loc via a call to COMBINE_LOCATION_DATA.
+	* line-map.c (LINE_MAP_MAX_COLUMN_NUMBER): Reduce from 1U << 17 to
+	1U << 12.
+	(location_adhoc_data_hash): Add the src_range into
+	the hash value.
+	(location_adhoc_data_eq): Require equality of the src_range
+	values.
+	(can_be_stored_compactly_p): New function.
+	(get_combined_adhoc_loc): Add src_range param, and store it,
+	via a bit-packing scheme for short ranges, otherwise within the
+	lookaside table.  Remove the requirement that data is non-NULL.
+	(get_range_from_adhoc_loc): New function.
+	(get_range_from_loc): New function.
+	(pure_location_p): New function.
+	(linemap_add): Ensure that start_location has zero for the
+	range_bits, unless we're past LINE_MAP_MAX_LOCATION_WITH_COLS.
+	Initialize range_bits to zero.  Assert that the start_location
+	is "pure".
+	(linemap_line_start): Assert that the
+	column_and_range_bits >= range_bits.
+	Update determinination of whether we need to start a new map
+	using the effective column bits, without the range bits.
+	Use the set's default_range_bits in new maps, apart from
+	those with column_bits == 0, which should also have 0 range_bits.
+	Increase the column bits for new maps by the range bits.
+	When adding lines to an existing map, use set->highest_line
+	directly rather than offsetting highest by SOURCE_COLUMN.
+	Add assertions to sanity-check the return value.
+	(linemap_position_for_column): Offset to_column by range_bits.
+	Update set->highest_location if necessary.
+	(linemap_position_for_line_and_column): Add line_maps * param.
+	Update the calculation to offset the column by range_bits, and
+	conditionalize it on being <= LINE_MAP_MAX_LOCATION_WITH_COLS.
+	Bound it by LINEMAPS_MACRO_LOWEST_LOCATION.  Update
+	set->highest_location if necessary.
+	(linemap_position_for_loc_and_offset): Handle ad-hoc locations;
+	pass "set" to linemap_position_for_line_and_column.
+	(linemap_macro_map_loc_unwind_toward_spelling): Add line_maps
+	param.  Handle ad-hoc locations.
+	(linemap_location_in_system_header_p): Pass on "set" to call to
+	linemap_macro_map_loc_unwind_toward_spelling.
+	(linemap_macro_loc_to_spelling_point): Retain ad-hoc locations.
+	Pass on "set" to call to
+	linemap_macro_map_loc_unwind_toward_spelling.
+	(linemap_resolve_location): Retain ad-hoc locations.  Pass on
+	"set" to call to linemap_macro_map_loc_unwind_toward_spelling.
+	(linemap_unwind_toward_expansion):  Pass on "set" to call to
+	linemap_macro_map_loc_unwind_toward_spelling.
+	(linemap_expand_location): Extract the data pointer before
+	extracting the location.
+	(rich_location::rich_location): Add line_maps param; use it to
+	extract the range from the source_location.
+	* location-example.txt: Regenerate, showing new representation.
+
 2015-11-06  David Malcolm  <dmalcolm@redhat.com>
 
 	* errors.c (cpp_diagnostic): Update for change in signature
diff --git a/libcpp/errors.c b/libcpp/errors.c
index c351c112ad1..8790e10f5c8 100644
--- a/libcpp/errors.c
+++ b/libcpp/errors.c
@@ -57,7 +57,7 @@ cpp_diagnostic (cpp_reader * pfile, int level, int reason,
 
   if (!pfile->cb.error)
     abort ();
-  rich_location richloc (src_loc);
+  rich_location richloc (pfile->line_table, src_loc);
   ret = pfile->cb.error (pfile, level, reason, &richloc, _(msgid), ap);
 
   return ret;
@@ -140,7 +140,7 @@ cpp_diagnostic_with_line (cpp_reader * pfile, int level, int reason,
   
   if (!pfile->cb.error)
     abort ();
-  rich_location richloc (src_loc);
+  rich_location richloc (pfile->line_table, src_loc);
   richloc.override_column (column);
   ret = pfile->cb.error (pfile, level, reason, &richloc, _(msgid), ap);
 
diff --git a/libcpp/include/cpplib.h b/libcpp/include/cpplib.h
index a2bdfa0c808..f5c2a21e952 100644
--- a/libcpp/include/cpplib.h
+++ b/libcpp/include/cpplib.h
@@ -237,7 +237,8 @@ struct GTY(()) cpp_identifier {
 /* A preprocessing token.  This has been carefully packed and should
    occupy 16 bytes on 32-bit hosts and 24 bytes on 64-bit hosts.  */
 struct GTY(()) cpp_token {
-  source_location src_loc;	/* Location of first char of token.  */
+  source_location src_loc;	/* Location of first char of token,
+				   together with range of full token.  */
   ENUM_BITFIELD(cpp_ttype) type : CHAR_BIT;  /* token type */
   unsigned short flags;		/* flags - see above */
 
diff --git a/libcpp/include/line-map.h b/libcpp/include/line-map.h
index c9340a6eaef..e7608f1e468 100644
--- a/libcpp/include/line-map.h
+++ b/libcpp/include/line-map.h
@@ -47,7 +47,8 @@ enum lc_reason
 typedef unsigned int linenum_type;
 
 /* The typedef "source_location" is a key within the location database,
-   identifying a source location or macro expansion.
+   identifying a source location or macro expansion, along with range
+   information, and (optionally) a pointer for use by gcc.
 
    This key only has meaning in relation to a line_maps instance.  Within
    gcc there is a single line_maps instance: "line_table", declared in
@@ -69,13 +70,48 @@ typedef unsigned int linenum_type;
              |  ordmap[0]->start_location)   | first line in ordmap 0
   -----------+-------------------------------+-------------------------------
              | ordmap[1]->start_location     | First line in ordmap 1
-             | ordmap[1]->start_location+1   | First column in that line
-             | ordmap[1]->start_location+2   | 2nd column in that line
-             |                               | Subsequent lines are offset by
-             |                               | (1 << column_bits),
-             |                               | e.g. 128 for 7 bits, with a
-             |                               | column value of 0 representing
-             |                               | "the whole line".
+             | ordmap[1]->start_location+32  | First column in that line
+             |   (assuming range_bits == 5)  |
+             | ordmap[1]->start_location+64  | 2nd column in that line
+             | ordmap[1]->start_location+4096| Second line in ordmap 1
+             |   (assuming column_bits == 12)
+             |
+             |   Subsequent lines are offset by (1 << column_bits),
+             |   e.g. 4096 for 12 bits, with a column value of 0 representing
+             |   "the whole line".
+             |
+             |   Within a line, the low "range_bits" (typically 5) are used for
+             |   storing short ranges, so that there's an offset of
+             |     (1 << range_bits) between individual columns within a line,
+             |   typically 32.
+             |   The low range_bits store the offset of the end point from the
+             |   start point, and the start point is found by masking away
+             |   the range bits.
+             |
+             |   For example:
+             |      ordmap[1]->start_location+64    "2nd column in that line"
+             |   above means a caret at that location, with a range
+             |   starting and finishing at the same place (the range bits
+             |   are 0), a range of length 1.
+             |
+             |   By contrast:
+             |      ordmap[1]->start_location+68
+             |   has range bits 0x4, meaning a caret with a range starting at
+             |   that location, but with endpoint 4 columns further on: a range
+             |   of length 5.
+             |
+             |   Ranges that have caret != start, or have an endpoint too
+             |   far away to fit in range_bits are instead stored as ad-hoc
+             |   locations.  Hence for range_bits == 5 we can compactly store
+             |   tokens of length <= 32 without needing to use the ad-hoc
+             |   table.
+             |
+             |   This packing scheme means we effectively have
+             |     (column_bits - range_bits)
+             |   of bits for the columns, typically (12 - 5) = 7, for 128
+             |   columns; longer line widths are accomodated by starting a
+             |   new ordmap with a higher column_bits.
+             |
              | ordmap[2]->start_location-1   | Final location in ordmap 1
   -----------+-------------------------------+-------------------------------
              | ordmap[2]->start_location     | First line in ordmap 2
@@ -127,8 +163,101 @@ typedef unsigned int linenum_type;
   0xffffffff | UINT_MAX                      |
   -----------+-------------------------------+-------------------------------
 
-  To see how this works in practice, see the worked example in
-  libcpp/location-example.txt.  */
+   Examples of location encoding.
+
+   Packed ranges
+   =============
+
+   Consider encoding the location of a token "foo", seen underlined here
+   on line 523, within an ordinary line_map that starts at line 500:
+
+                 11111111112
+        12345678901234567890
+     522
+     523   return foo + bar;
+                  ^~~
+     524
+
+   The location's caret and start are both at line 523, column 11; the
+   location's finish is on the same line, at column 13 (an offset of 2
+   columns, for length 3).
+
+   Line 523 is offset 23 from the starting line of the ordinary line_map.
+
+   caret == start, and the offset of the finish fits within 5 bits, so
+   this can be stored as a packed range.
+
+   This is encoded as:
+      ordmap->start
+         + (line_offset << ordmap->m_column_and_range_bits)
+         + (column << ordmap->m_range_bits)
+         + (range_offset);
+   i.e. (for line offset 23, column 11, range offset 2):
+      ordmap->start
+         + (23 << 12)
+         + (11 << 5)
+         + 2;
+   i.e.:
+      ordmap->start + 0x17162
+   assuming that the line_map uses the default of 7 bits for columns and
+   5 bits for packed range (giving 12 bits for m_column_and_range_bits).
+
+
+   "Pure" locations
+   ================
+
+   These are a special case of the above, where
+      caret == start == finish
+   They are stored as packed ranges with offset == 0.
+   For example, the location of the "f" of "foo" could be stored
+   as above, but with range offset 0, giving:
+      ordmap->start
+         + (23 << 12)
+         + (11 << 5)
+         + 0;
+   i.e.:
+      ordmap->start + 0x17160
+
+
+   Unoptimized ranges
+   ==================
+
+   Consider encoding the location of the binary expression
+   below:
+
+                 11111111112
+        12345678901234567890
+     521
+     523   return foo + bar;
+                  ~~~~^~~~~
+     523
+
+   The location's caret is at the "+", line 523 column 15, but starts
+   earlier, at the "f" of "foo" at column 11.  The finish is at the "r"
+   of "bar" at column 19.
+
+   This can't be stored as a packed range since start != caret.
+   Hence it is stored as an ad-hoc location e.g. 0x80000003.
+
+   Stripping off the top bit gives us an index into the ad-hoc
+   lookaside table:
+
+     line_table->location_adhoc_data_map.data[0x3]
+
+   from which the caret, start and finish can be looked up,
+   encoded as "pure" locations:
+
+     start  == ordmap->start + (23 << 12) + (11 << 5)
+            == ordmap->start + 0x17160  (as above; the "f" of "foo")
+
+     caret  == ordmap->start + (23 << 12) + (15 << 5)
+            == ordmap->start + 0x171e0
+
+     finish == ordmap->start + (23 << 12) + (19 << 5)
+            == ordmap->start + 0x17260
+
+   To further see how source_location works in practice, see the
+   worked example in libcpp/location-example.txt.  */
 typedef unsigned int source_location;
 
 /* A range of source locations.
@@ -217,8 +346,9 @@ struct GTY((tag ("0"), desc ("%h.reason == LC_ENTER_MACRO ? 2 : 1"))) line_map {
    
    Physical source file TO_FILE at line TO_LINE at column 0 is represented
    by the logical START_LOCATION.  TO_LINE+L at column C is represented by
-   START_LOCATION+(L*(1<<column_bits))+C, as long as C<(1<<column_bits),
-   and the result_location is less than the next line_map's start_location.
+   START_LOCATION+(L*(1<<m_column_and_range_bits))+(C*1<<m_range_bits), as
+   long as C<(1<<effective range bits), and the result_location is less than
+   the next line_map's start_location.
    (The top line is line 1 and the leftmost column is column 1; line/column 0
    means "entire file/line" or "unknown line/column" or "not applicable".)
 
@@ -238,8 +368,24 @@ struct GTY((tag ("1"))) line_map_ordinary : public line_map {
      cpp_buffer.  */
   unsigned char sysp;
 
-  /* Number of the low-order source_location bits used for a column number.  */
-  unsigned int column_bits : 8;
+  /* Number of the low-order source_location bits used for column numbers
+     and ranges.  */
+  unsigned int m_column_and_range_bits : 8;
+
+  /* Number of the low-order "column" bits used for storing short ranges
+     inline, rather than in the ad-hoc table.
+     MSB                                                                 LSB
+     31                                                                    0
+     +-------------------------+-------------------------------------------+
+     |                         |<---map->column_and_range_bits (e.g. 12)-->|
+     +-------------------------+-----------------------+-------------------+
+     |                         | column_and_range_bits | map->range_bits   |
+     |                         |   - range_bits        |                   |
+     +-------------------------+-----------------------+-------------------+
+     | row bits                | effective column bits | short range bits  |
+     |                         |    (e.g. 7)           |   (e.g. 5)        |
+     +-------------------------+-----------------------+-------------------+ */
+  unsigned int m_range_bits : 8;
 };
 
 /* This is the highest possible source location encoded within an
@@ -435,15 +581,6 @@ ORDINARY_MAP_IN_SYSTEM_HEADER_P (const line_map_ordinary *ord_map)
   return ord_map->sysp;
 }
 
-/* Get the number of the low-order source_location bits used for a
-   column number within ordinary map MAP.  */
-
-inline unsigned char
-ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (const line_map_ordinary *ord_map)
-{
-  return ord_map->column_bits;
-}
-
 /* Get the filename of ordinary map MAP.  */
 
 inline const char *
@@ -524,9 +661,11 @@ struct GTY(()) maps_info_macro {
   unsigned int cache;
 };
 
-/* Data structure to associate an arbitrary data to a source location.  */
+/* Data structure to associate a source_range together with an arbitrary
+   data pointer with a source location.  */
 struct GTY(()) location_adhoc_data {
   source_location locus;
+  source_range src_range;
   void * GTY((skip)) data;
 };
 
@@ -588,6 +727,12 @@ struct GTY(()) line_maps {
 
   /* True if we've seen a #line or # 44 "file" directive.  */
   bool seen_line_directive;
+
+  /* The default value of range_bits in ordinary line maps.  */
+  unsigned int default_range_bits;
+
+  unsigned int num_optimized_ranges;
+  unsigned int num_unoptimized_ranges;
 };
 
 /* Returns the number of allocated maps so far. MAP_KIND shall be TRUE
@@ -825,11 +970,15 @@ LINEMAPS_LAST_ALLOCATED_MACRO_MAP (const line_maps *set)
 
 extern void location_adhoc_data_fini (struct line_maps *);
 extern source_location get_combined_adhoc_loc (struct line_maps *,
-					       source_location, void *);
+					       source_location,
+					       source_range,
+					       void *);
 extern void *get_data_from_adhoc_loc (struct line_maps *, source_location);
 extern source_location get_location_from_adhoc_loc (struct line_maps *,
 						    source_location);
 
+extern source_range get_range_from_loc (line_maps *set, source_location loc);
+
 /* Get whether location LOC is an ad-hoc location.  */
 
 inline bool
@@ -838,14 +987,21 @@ IS_ADHOC_LOC (source_location loc)
   return (loc & MAX_SOURCE_LOCATION) != loc;
 }
 
+/* Get whether location LOC is a "pure" location, or
+   whether it is an ad-hoc location, or embeds range information.  */
+
+bool
+pure_location_p (line_maps *set, source_location loc);
+
 /* Combine LOC and BLOCK, giving a combined adhoc location.  */
 
 inline source_location
 COMBINE_LOCATION_DATA (struct line_maps *set,
 		       source_location loc,
+		       source_range src_range,
 		       void *block)
 {
-  return get_combined_adhoc_loc (set, loc, block);
+  return get_combined_adhoc_loc (set, loc, src_range, block);
 }
 
 extern void rebuild_location_adhoc_htab (struct line_maps *);
@@ -931,7 +1087,7 @@ inline linenum_type
 SOURCE_LINE (const line_map_ordinary *ord_map, source_location loc)
 {
   return ((loc - ord_map->start_location)
-	  >> ord_map->column_bits) + ord_map->to_line;
+	  >> ord_map->m_column_and_range_bits) + ord_map->to_line;
 }
 
 /* Convert a map and source_location to source column number.  */
@@ -939,7 +1095,7 @@ inline linenum_type
 SOURCE_COLUMN (const line_map_ordinary *ord_map, source_location loc)
 {
   return ((loc - ord_map->start_location)
-	  & ((1 << ord_map->column_bits) - 1));
+	  & ((1 << ord_map->m_column_and_range_bits) - 1)) >> ord_map->m_range_bits;
 }
 
 /* Return the location of the last source line within an ordinary
@@ -949,7 +1105,7 @@ LAST_SOURCE_LINE_LOCATION (const line_map_ordinary *map)
 {
   return (((map[1].start_location - 1
 	    - map->start_location)
-	   & ~((1 << map->column_bits) - 1))
+	   & ~((1 << map->m_column_and_range_bits) - 1))
 	  + map->start_location);
 }
 
@@ -999,7 +1155,8 @@ linemap_position_for_column (struct line_maps *, unsigned int);
 /* Encode and return a source location from a given line and
    column.  */
 source_location
-linemap_position_for_line_and_column (const line_map_ordinary *,
+linemap_position_for_line_and_column (line_maps *set,
+				      const line_map_ordinary *,
 				      linenum_type, unsigned int);
 
 /* Encode and return a source_location starting from location LOC and
@@ -1187,7 +1344,7 @@ class rich_location
   /* Constructors.  */
 
   /* Constructing from a location.  */
-  rich_location (source_location loc);
+  rich_location (line_maps *set, source_location loc);
 
   /* Constructing from a source_range.  */
   rich_location (source_range src_range);
diff --git a/libcpp/lex.c b/libcpp/lex.c
index 7e97bc2b289..d9b428ad8d8 100644
--- a/libcpp/lex.c
+++ b/libcpp/lex.c
@@ -2723,6 +2723,19 @@ _cpp_lex_direct (cpp_reader *pfile)
       break;
     }
 
+  source_range tok_range;
+  tok_range.m_start = result->src_loc;
+  if (result->src_loc >= RESERVED_LOCATION_COUNT)
+    tok_range.m_finish
+      = linemap_position_for_column (pfile->line_table,
+				     CPP_BUF_COLUMN (buffer, buffer->cur));
+  else
+    tok_range.m_finish = tok_range.m_start;
+
+  result->src_loc = COMBINE_LOCATION_DATA (pfile->line_table,
+					   result->src_loc,
+					   tok_range, NULL);
+
   return result;
 }
 
diff --git a/libcpp/line-map.c b/libcpp/line-map.c
index 3c19f93b7a5..c5aa4223b1b 100644
--- a/libcpp/line-map.c
+++ b/libcpp/line-map.c
@@ -27,9 +27,9 @@ along with this program; see the file COPYING3.  If not see
 #include "hashtab.h"
 
 /* Do not track column numbers higher than this one.  As a result, the
-   range of column_bits is [7, 18] (or 0 if column numbers are
+   range of column_bits is [12, 18] (or 0 if column numbers are
    disabled).  */
-const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 17);
+const unsigned int LINE_MAP_MAX_COLUMN_NUMBER = (1U << 12);
 
 /* Do not track column numbers if locations get higher than this.  */
 const source_location LINE_MAP_MAX_LOCATION_WITH_COLS = 0x60000000;
@@ -46,7 +46,7 @@ static const line_map_macro* linemap_macro_map_lookup (struct line_maps *,
 static source_location linemap_macro_map_loc_to_def_point
 (const line_map_macro *, source_location);
 static source_location linemap_macro_map_loc_unwind_toward_spelling
-(const line_map_macro *, source_location);
+(line_maps *set, const line_map_macro *, source_location);
 static source_location linemap_macro_map_loc_to_exp_point
 (const line_map_macro *, source_location);
 static source_location linemap_macro_loc_to_spelling_point
@@ -69,7 +69,10 @@ location_adhoc_data_hash (const void *l)
 {
   const struct location_adhoc_data *lb =
       (const struct location_adhoc_data *) l;
-  return (hashval_t) lb->locus + (size_t) lb->data;
+  return ((hashval_t) lb->locus
+	  + (hashval_t) lb->src_range.m_start
+	  + (hashval_t) lb->src_range.m_finish
+	  + (size_t) lb->data);
 }
 
 /* Compare function for location_adhoc_data hashtable.  */
@@ -81,7 +84,10 @@ location_adhoc_data_eq (const void *l1, const void *l2)
       (const struct location_adhoc_data *) l1;
   const struct location_adhoc_data *lb2 =
       (const struct location_adhoc_data *) l2;
-  return lb1->locus == lb2->locus && lb1->data == lb2->data;
+  return (lb1->locus == lb2->locus
+	  && lb1->src_range.m_start == lb2->src_range.m_start
+	  && lb1->src_range.m_finish == lb2->src_range.m_finish
+	  && lb1->data == lb2->data);
 }
 
 /* Update the hashtable when location_adhoc_data is reallocated.  */
@@ -106,23 +112,103 @@ rebuild_location_adhoc_htab (struct line_maps *set)
 		    set->location_adhoc_data_map.data + i, INSERT);
 }
 
+/* Helper function for get_combined_adhoc_loc.
+   Can the given LOCUS + SRC_RANGE and DATA pointer be stored compactly
+   within a source_location, without needing to use an ad-hoc location.  */
+
+static bool
+can_be_stored_compactly_p (struct line_maps *set,
+			   source_location locus,
+			   source_range src_range,
+			   void *data)
+{
+  /* If there's an ad-hoc pointer, we can't store it directly in the
+     source_location, we need the lookaside.  */
+  if (data)
+    return false;
+
+  /* We only store ranges that begin at the locus and that are sufficiently
+     "sane".  */
+  if (src_range.m_start != locus)
+    return false;
+
+  if (src_range.m_finish < src_range.m_start)
+    return false;
+
+  if (src_range.m_start < RESERVED_LOCATION_COUNT)
+    return false;
+
+  if (locus >= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    return false;
+
+  /* All 3 locations must be within ordinary maps, typically, the same
+     ordinary map.  */
+  source_location lowest_macro_loc = LINEMAPS_MACRO_LOWEST_LOCATION (set);
+  if (locus >= lowest_macro_loc)
+    return false;
+  if (src_range.m_start >= lowest_macro_loc)
+    return false;
+  if (src_range.m_finish >= lowest_macro_loc)
+    return false;
+
+  /* Passed all tests.  */
+  return true;
+}
+
 /* Combine LOCUS and DATA to a combined adhoc loc.  */
 
 source_location
 get_combined_adhoc_loc (struct line_maps *set,
-			source_location locus, void *data)
+			source_location locus,
+			source_range src_range,
+			void *data)
 {
   struct location_adhoc_data lb;
   struct location_adhoc_data **slot;
 
-  linemap_assert (data);
-
   if (IS_ADHOC_LOC (locus))
     locus
       = set->location_adhoc_data_map.data[locus & MAX_SOURCE_LOCATION].locus;
   if (locus == 0 && data == NULL)
     return 0;
+
+  /* Any ordinary locations ought to be "pure" at this point: no
+     compressed ranges.  */
+  linemap_assert (locus < RESERVED_LOCATION_COUNT
+		  || locus >= LINE_MAP_MAX_LOCATION_WITH_COLS
+		  || locus >= LINEMAPS_MACRO_LOWEST_LOCATION (set)
+		  || pure_location_p (set, locus));
+
+  /* Consider short-range optimization.  */
+  if (can_be_stored_compactly_p (set, locus, src_range, data))
+    {
+      /* The low bits ought to be clear.  */
+      linemap_assert (pure_location_p (set, locus));
+      const line_map *map = linemap_lookup (set, locus);
+      const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+      unsigned int int_diff = src_range.m_finish - src_range.m_start;
+      unsigned int col_diff = (int_diff >> ordmap->m_range_bits);
+      if (col_diff < (1U << ordmap->m_range_bits))
+	{
+	  source_location packed = locus | col_diff;
+	  set->num_optimized_ranges++;
+	  return packed;
+	}
+    }
+
+  /* We can also compactly store the reserved locations
+     when locus == start == finish (and data is NULL).  */
+  if (locus < RESERVED_LOCATION_COUNT
+      && locus == src_range.m_start
+      && locus == src_range.m_finish
+      && !data)
+    return locus;
+
+  if (!data)
+    set->num_unoptimized_ranges++;
+
   lb.locus = locus;
+  lb.src_range = src_range;
   lb.data = data;
   slot = (struct location_adhoc_data **)
       htab_find_slot (set->location_adhoc_data_map.htab, &lb, INSERT);
@@ -177,6 +263,60 @@ get_location_from_adhoc_loc (struct line_maps *set, source_location loc)
   return set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 }
 
+/* Return the source_range for adhoc location LOC.  */
+
+static source_range
+get_range_from_adhoc_loc (struct line_maps *set, source_location loc)
+{
+  linemap_assert (IS_ADHOC_LOC (loc));
+  return set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].src_range;
+}
+
+/* Get the source_range of location LOC, either from the ad-hoc
+   lookaside table, or embedded inside LOC itself.  */
+
+source_range
+get_range_from_loc (struct line_maps *set,
+		    source_location loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    return get_range_from_adhoc_loc (set, loc);
+
+  /* For ordinary maps, extract packed range.  */
+  if (loc >= RESERVED_LOCATION_COUNT
+      && loc < LINEMAPS_MACRO_LOWEST_LOCATION (set)
+      && loc <= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    {
+      const line_map *map = linemap_lookup (set, loc);
+      const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+      source_range result;
+      int offset = loc & ((1 << ordmap->m_range_bits) - 1);
+      result.m_start = loc - offset;
+      result.m_finish = result.m_start + (offset << ordmap->m_range_bits);
+      return result;
+    }
+
+  return source_range::from_location (loc);
+}
+
+/* Get whether location LOC is a "pure" location, or
+   whether it is an ad-hoc location, or embeds range information.  */
+
+bool
+pure_location_p (line_maps *set, source_location loc)
+{
+  if (IS_ADHOC_LOC (loc))
+    return false;
+
+  const line_map *map = linemap_lookup (set, loc);
+  const line_map_ordinary *ordmap = linemap_check_ordinary (map);
+
+  if (loc & ((1U << ordmap->m_range_bits) - 1))
+    return false;
+
+  return true;
+}
+
 /* Finalize the location_adhoc_data structure.  */
 void
 location_adhoc_data_fini (struct line_maps *set)
@@ -319,7 +459,19 @@ const struct line_map *
 linemap_add (struct line_maps *set, enum lc_reason reason,
 	     unsigned int sysp, const char *to_file, linenum_type to_line)
 {
-  source_location start_location = set->highest_location + 1;
+  /* Generate a start_location above the current highest_location.
+     If possible, make the low range bits be zero.  */
+  source_location start_location;
+  if (set->highest_location < LINE_MAP_MAX_LOCATION_WITH_COLS)
+    {
+      start_location = set->highest_location + (1 << set->default_range_bits);
+      if (set->default_range_bits)
+	start_location &= ~((1 << set->default_range_bits) - 1);
+      linemap_assert (0 == (start_location
+			    & ((1 << set->default_range_bits) - 1)));
+    }
+  else
+    start_location = set->highest_location + 1;
 
   linemap_assert (!(LINEMAPS_ORDINARY_USED (set)
 		    && (start_location
@@ -398,11 +550,18 @@ linemap_add (struct line_maps *set, enum lc_reason reason,
   map->to_file = to_file;
   map->to_line = to_line;
   LINEMAPS_ORDINARY_CACHE (set) = LINEMAPS_ORDINARY_USED (set) - 1;
-  map->column_bits = 0;
+  map->m_column_and_range_bits = 0;
+  map->m_range_bits = 0;
   set->highest_location = start_location;
   set->highest_line = start_location;
   set->max_column_hint = 0;
 
+  /* This assertion is placed after set->highest_location has
+     been updated, since the latter affects
+     linemap_location_from_macro_expansion_p, which ultimately affects
+     pure_location_p.  */
+  linemap_assert (pure_location_p (set, start_location));
+
   if (reason == LC_ENTER)
     {
       map->included_from =
@@ -549,13 +708,14 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
     SOURCE_LINE (map, set->highest_line);
   int line_delta = to_line - last_line;
   bool add_map = false;
+  linemap_assert (map->m_column_and_range_bits >= map->m_range_bits);
+  int effective_column_bits = map->m_column_and_range_bits - map->m_range_bits;
 
   if (line_delta < 0
       || (line_delta > 10
-	  && line_delta * ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) > 1000)
-      || (max_column_hint >= (1U << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map)))
-      || (max_column_hint <= 80
-	  && ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map) >= 10)
+	  && line_delta * map->m_column_and_range_bits > 1000)
+      || (max_column_hint >= (1U << effective_column_bits))
+      || (max_column_hint <= 80 && effective_column_bits >= 10)
       || (highest > LINE_MAP_MAX_LOCATION_WITH_COLS
 	  && (set->max_column_hint || highest >= LINE_MAP_MAX_SOURCE_LOCATION)))
     add_map = true;
@@ -564,22 +724,27 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
   if (add_map)
     {
       int column_bits;
+      int range_bits;
       if (max_column_hint > LINE_MAP_MAX_COLUMN_NUMBER
 	  || highest > LINE_MAP_MAX_LOCATION_WITH_COLS)
 	{
 	  /* If the column number is ridiculous or we've allocated a huge
-	     number of source_locations, give up on column numbers. */
+	     number of source_locations, give up on column numbers
+	     (and on packed ranges).  */
 	  max_column_hint = 0;
 	  column_bits = 0;
+	  range_bits = 0;
 	  if (highest > LINE_MAP_MAX_SOURCE_LOCATION)
 	    return 0;
 	}
       else
 	{
 	  column_bits = 7;
+	  range_bits = set->default_range_bits;
 	  while (max_column_hint >= (1U << column_bits))
 	    column_bits++;
 	  max_column_hint = 1U << column_bits;
+	  column_bits += range_bits;
 	}
       /* Allocate the new line_map.  However, if the current map only has a
 	 single line we can sometimes just increase its column_bits instead. */
@@ -592,14 +757,14 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
 				ORDINARY_MAP_IN_SYSTEM_HEADER_P (map),
 				ORDINARY_MAP_FILE_NAME (map),
 				to_line)));
-      map->column_bits = column_bits;
+      map->m_column_and_range_bits = column_bits;
+      map->m_range_bits = range_bits;
       r = (MAP_START_LOCATION (map)
 	   + ((to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map))
 	      << column_bits));
     }
   else
-    r = highest - SOURCE_COLUMN (map, highest)
-      + (line_delta << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (map));
+    r = set->highest_line + (line_delta << map->m_column_and_range_bits);
 
   /* Locations of ordinary tokens are always lower than locations of
      macro tokens.  */
@@ -610,6 +775,18 @@ linemap_line_start (struct line_maps *set, linenum_type to_line,
   if (r > set->highest_location)
     set->highest_location = r;
   set->max_column_hint = max_column_hint;
+
+  /* At this point, we expect one of:
+     (a) the normal case: a "pure" location with 0 range bits, or
+     (b) we've gone past LINE_MAP_MAX_LOCATION_WITH_COLS so can't track
+        columns anymore (or ranges), or
+     (c) we're in a region with a column hint exceeding
+        LINE_MAP_MAX_COLUMN_NUMBER, so column-tracking is off,
+	with column_bits == 0.  */
+  linemap_assert (pure_location_p (set, r)
+		  || r >= LINE_MAP_MAX_LOCATION_WITH_COLS
+		  || map->m_column_and_range_bits == 0);
+  linemap_assert (SOURCE_LINE (map, r) == to_line);
   return r;
 }
 
@@ -640,7 +817,8 @@ linemap_position_for_column (struct line_maps *set, unsigned int to_column)
 	  r = linemap_line_start (set, SOURCE_LINE (map, r), to_column + 50);
 	}
     }
-  r = r + to_column;
+  line_map_ordinary *map = LINEMAPS_LAST_ORDINARY_MAP (set);
+  r = r + (to_column << map->m_range_bits);
   if (r >= set->highest_location)
     set->highest_location = r;
   return r;
@@ -650,16 +828,25 @@ linemap_position_for_column (struct line_maps *set, unsigned int to_column)
    column.  */
 
 source_location
-linemap_position_for_line_and_column (const line_map_ordinary *ord_map,
+linemap_position_for_line_and_column (line_maps *set,
+				      const line_map_ordinary *ord_map,
 				      linenum_type line,
 				      unsigned column)
 {
   linemap_assert (ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map) <= line);
 
-  return (MAP_START_LOCATION (ord_map)
-	  + ((line - ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map))
-	     << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (ord_map))
-	  + (column & ((1 << ORDINARY_MAP_NUMBER_OF_COLUMN_BITS (ord_map)) - 1)));
+  source_location r = MAP_START_LOCATION (ord_map);
+  r += ((line - ORDINARY_MAP_STARTING_LINE_NUMBER (ord_map))
+	<< ord_map->m_column_and_range_bits);
+  if (r <= LINE_MAP_MAX_LOCATION_WITH_COLS)
+    r += ((column & ((1 << ord_map->m_column_and_range_bits) - 1))
+	  << ord_map->m_range_bits);
+  source_location upper_limit = LINEMAPS_MACRO_LOWEST_LOCATION (set);
+  if (r >= upper_limit)
+    r = upper_limit - 1;
+  if (r > set->highest_location)
+    set->highest_location = r;
+  return r;
 }
 
 /* Encode and return a source_location starting from location LOC and
@@ -673,6 +860,9 @@ linemap_position_for_loc_and_offset (struct line_maps *set,
 {
   const line_map_ordinary * map = NULL;
 
+  if (IS_ADHOC_LOC (loc))
+    loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+
   /* This function does not support virtual locations yet.  */
   if (linemap_assert_fails
       (!linemap_location_from_macro_expansion_p (set, loc)))
@@ -711,11 +901,11 @@ linemap_position_for_loc_and_offset (struct line_maps *set,
     }
 
   offset += column;
-  if (linemap_assert_fails (offset < (1u << map->column_bits)))
+  if (linemap_assert_fails (offset < (1u << map->m_column_and_range_bits)))
     return loc;
 
   source_location r = 
-    linemap_position_for_line_and_column (map, line, offset);
+    linemap_position_for_line_and_column (set, map, line, offset);
   if (linemap_assert_fails (r <= set->highest_location)
       || linemap_assert_fails (map == linemap_lookup (set, r)))
     return loc;
@@ -893,14 +1083,19 @@ linemap_macro_map_loc_to_def_point (const line_map_macro *map,
    In other words, this returns the xI location presented in the
    comments of line_map_macro above.  */
 source_location
-linemap_macro_map_loc_unwind_toward_spelling (const line_map_macro* map,
+linemap_macro_map_loc_unwind_toward_spelling (line_maps *set,
+					      const line_map_macro* map,
 					      source_location location)
 {
   unsigned token_no;
 
+  if (IS_ADHOC_LOC (location))
+    location = get_location_from_adhoc_loc (set, location);
+
   linemap_assert (linemap_macro_expansion_map_p (map)
 		  && location >= MAP_START_LOCATION (map));
   linemap_assert (location >= RESERVED_LOCATION_COUNT);
+  linemap_assert (!IS_ADHOC_LOC (location));
 
   token_no = location - MAP_START_LOCATION (map);
   linemap_assert (token_no < MACRO_MAP_NUM_MACRO_TOKENS (map));
@@ -1010,7 +1205,7 @@ linemap_location_in_system_header_p (struct line_maps *set,
 
 	      /* It's a token resulting from a macro expansion.  */
 	      source_location loc =
-		linemap_macro_map_loc_unwind_toward_spelling (macro_map, location);
+		linemap_macro_map_loc_unwind_toward_spelling (set, macro_map, location);
 	      if (loc < RESERVED_LOCATION_COUNT)
 		/* This token might come from a built-in macro.  Let's
 		   look at where that macro got expanded.  */
@@ -1183,11 +1378,6 @@ linemap_macro_loc_to_spelling_point (struct line_maps *set,
 				     const line_map_ordinary **original_map)
 {
   struct line_map *map;
-
-  if (IS_ADHOC_LOC (location))
-    location = set->location_adhoc_data_map.data[location
-						 & MAX_SOURCE_LOCATION].locus;
-
   linemap_assert (set && location >= RESERVED_LOCATION_COUNT);
 
   while (true)
@@ -1198,7 +1388,7 @@ linemap_macro_loc_to_spelling_point (struct line_maps *set,
 
       location
 	= linemap_macro_map_loc_unwind_toward_spelling
-	    (linemap_check_macro (map),
+	    (set, linemap_check_macro (map),
 	     location);
     }
 
@@ -1341,10 +1531,11 @@ linemap_resolve_location (struct line_maps *set,
 			  enum location_resolution_kind lrk,
 			  const line_map_ordinary **map)
 {
+  source_location locus = loc;
   if (IS_ADHOC_LOC (loc))
-    loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
+    locus = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 
-  if (loc < RESERVED_LOCATION_COUNT)
+  if (locus < RESERVED_LOCATION_COUNT)
     {
       /* A reserved location wasn't encoded in a map.  Let's return a
 	 NULL map here, just like what linemap_ordinary_map_lookup
@@ -1396,7 +1587,7 @@ linemap_unwind_toward_expansion (struct line_maps *set,
     loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
 
   resolved_location =
-    linemap_macro_map_loc_unwind_toward_spelling (macro_map, loc);
+    linemap_macro_map_loc_unwind_toward_spelling (set, macro_map, loc);
   resolved_map = linemap_lookup (set, resolved_location);
 
   if (!linemap_macro_expansion_map_p (resolved_map))
@@ -1478,9 +1669,9 @@ linemap_expand_location (struct line_maps *set,
   memset (&xloc, 0, sizeof (xloc));
   if (IS_ADHOC_LOC (loc))
     {
-      loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
       xloc.data
 	= set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].data;
+      loc = set->location_adhoc_data_map.data[loc & MAX_SOURCE_LOCATION].locus;
     }
 
   if (loc < RESERVED_LOCATION_COUNT)
@@ -1760,13 +1951,14 @@ line_table_dump (FILE *stream, struct line_maps *set, unsigned int num_ordinary,
 
 /* Construct a rich_location with location LOC as its initial range.  */
 
-rich_location::rich_location (source_location loc) :
+rich_location::rich_location (line_maps *set, source_location loc) :
   m_loc (loc),
   m_num_ranges (0),
   m_have_expanded_location (false)
 {
-  /* Set up the 0th range: */
-  add_range (loc, loc, true);
+  /* Set up the 0th range, extracting any range from LOC.  */
+  source_range src_range = get_range_from_loc (set, loc);
+  add_range (src_range, true);
   m_ranges[0].m_caret = lazily_expand_location ();
 }
 
diff --git a/libcpp/location-example.txt b/libcpp/location-example.txt
index a5f95b2baf8..14b5c2e284a 100644
--- a/libcpp/location-example.txt
+++ b/libcpp/location-example.txt
@@ -30,142 +30,154 @@ RESERVED LOCATIONS
   source_location interval: 0 <= loc < 2
 
 ORDINARY MAP: 0
-  source_location interval: 2 <= loc < 3
+  source_location interval: 32 <= loc < 64
   file: test.c
   starting at line: 1
-  column bits: 7
-test.c:  1|loc:    2|#include "test.h"
-                    |00000001111111111
-                    |34567890123456789
+  column bits: 12
+  range bits: 5
+test.c:  1|loc:   32|#include "test.h"
+                    |69269258258148147
+                    |46802468024680246
 
 ORDINARY MAP: 1
-  source_location interval: 3 <= loc < 4
+  source_location interval: 64 <= loc < 96
   file: <built-in>
   starting at line: 0
   column bits: 0
+  range bits: 0
 
 ORDINARY MAP: 2
-  source_location interval: 4 <= loc < 5
+  source_location interval: 96 <= loc < 128
   file: <command-line>
   starting at line: 0
   column bits: 0
+  range bits: 0
 
 ORDINARY MAP: 3
-  source_location interval: 5 <= loc < 5005
+  source_location interval: 128 <= loc < 160128
   file: /usr/include/stdc-predef.h
   starting at line: 1
-  column bits: 7
+  column bits: 12
+  range bits: 5
 (contents of /usr/include/stdc-predef.h snipped for brevity)
 
 ORDINARY MAP: 4
-  source_location interval: 5005 <= loc < 5006
+  source_location interval: 160128 <= loc < 160160
   file: <command-line>
-  starting at line: 1
-  column bits: 7
+  starting at line: 32
+  column bits: 12
+  range bits: 5
 
 ORDINARY MAP: 5
-  source_location interval: 5006 <= loc < 5134
+  source_location interval: 160160 <= loc < 164256
   file: test.c
   starting at line: 1
-  column bits: 7
-test.c:  1|loc: 5006|#include "test.h"
-                    |55555555555555555
+  column bits: 12
+  range bits: 5
+test.c:  1|loc:160160|#include "test.h"
                     |00000000000000000
-                    |00011111111112222
-                    |78901234567890123
+                    |12223334445556667
+                    |92582581481470470
+                    |24680246802468024
 
 ORDINARY MAP: 6
-  source_location interval: 5134 <= loc < 5416
+  source_location interval: 164256 <= loc < 173280
   file: test.h
   starting at line: 1
-  column bits: 7
-test.h:  1|loc: 5134|extern int foo ();
-                    |555555555555555555
-                    |111111111111111111
-                    |333334444444444555
-                    |567890123456789012
-test.h:  2|loc: 5262|
+  column bits: 12
+  range bits: 5
+test.h:  1|loc:164256|extern int foo ();
+                    |444444444444444444
+                    |233344455566677788
+                    |825814814704703603
+                    |802468024680246802
+test.h:  2|loc:168352|
                     |
                     |
                     |
                     |
-test.h:  3|loc: 5390|#define PLUS(A, B) A + B
-                    |555555555555555555555555
-                    |333333333444444444444444
-                    |999999999000000000011111
-                    |123456789012345678901234
+test.h:  3|loc:172448|#define PLUS(A, B) A + B
+                    |222222222222222223333333
+                    |455566677788889990001112
+                    |814704703603692692582581
+                    |024680246802468024680246
 
 ORDINARY MAP: 7
-  source_location interval: 5416 <= loc < 6314
+  source_location interval: 173280 <= loc < 202016
   file: test.c
   starting at line: 2
-  column bits: 7
-test.c:  2|loc: 5416|
+  column bits: 12
+  range bits: 5
+test.c:  2|loc:173280|
                     |
                     |
                     |
                     |
-test.c:  3|loc: 5544|int
-                    |555
-                    |555
+test.c:  3|loc:177376|int
+                    |777
                     |444
-                    |567
-test.c:  4|loc: 5672|main (int argc, char **argv)
-                    |5555555555555555555555555555
-                    |6666666666666666666666666667
-                    |7777777888888888899999999990
-                    |3456789012345678901234567890
-test.c:  5|loc: 5800|{
+                    |047
+                    |802
+test.c:  4|loc:181472|main (int argc, char **argv)
+                    |1111111111111111222222222222
+                    |5556666777888999000111222333
+                    |0360369269258258148147047036
+                    |4680246802468024680246802468
+test.c:  5|loc:185568|{
                     |5
-                    |8
-                    |0
-                    |1
-test.c:  6|loc: 5928|  int a = PLUS (1,2);
-                    |555555555555555555555
-                    |999999999999999999999
-                    |233333333334444444444
-                    |901234567890123456789
-test.c:  7|loc: 6056|  int b = PLUS (3,4);
-                    |666666666666666666666
-                    |000000000000000000000
-                    |555666666666677777777
-                    |789012345678901234567
-test.c:  8|loc: 6184|  return 0;
-                    |66666666666
-                    |11111111111
-                    |88888999999
-                    |56789012345
-test.c:  9|loc: 6312|}
                     |6
-                    |3
+                    |0
+                    |0
+test.c:  6|loc:189664|  int a = PLUS (1,2);
+                    |999999999900000000000
+                    |677788899900011122233
+                    |926925825814814704703
+                    |680246802468024680246
+test.c:  7|loc:193760|  int b = PLUS (3,4);
+                    |333333344444444444444
+                    |788899900011122233344
+                    |925825814814704703603
+                    |246802468024680246802
+test.c:  8|loc:197856|  return 0;
+                    |77778888888
+                    |89990001112
+                    |82581481470
+                    |80246802468
+test.c:  9|loc:201952|}
                     |1
-                    |3
+                    |9
+                    |8
+                    |4
 
 UNALLOCATED LOCATIONS
-  source_location interval: 6314 <= loc < 2147483633
+  source_location interval: 202016 <= loc < 2147483633
 
 MACRO 1: PLUS (7 tokens)
   source_location interval: 2147483633 <= loc < 2147483640
-test.c:7:11: note: expansion point is location 6067
+test.c:7:11: note: expansion point is location 194115
    int b = PLUS (3,4);
-           ^
+           ^~~~
+
   map->start_location: 2147483633
   macro_locations:
-    0: 6073, 5410
-test.c:7:17: note: token 0 has x-location == 6073
+    0: 194304, 173088
+test.c:7:17: note: token 0 has x-location == 194304
    int b = PLUS (3,4);
                  ^
-test.c:7:17: note: token 0 has y-location == 5410
-    1: 5412, 5412
+
+test.c:7:17: note: token 0 has y-location == 173088
+    1: 173152, 173152
 In file included from test.c:1:0:
-test.h:3:22: note: token 1 has x-location == y-location == 5412
+test.h:3:22: note: token 1 has x-location == y-location == 173152
  #define PLUS(A, B) A + B
                       ^
-    2: 6075, 5414
-test.c:7:19: note: token 2 has x-location == 6075
+
+    2: 194368, 173216
+test.c:7:19: note: token 2 has x-location == 194368
    int b = PLUS (3,4);
                    ^
-test.c:7:19: note: token 2 has y-location == 5414
+
+test.c:7:19: note: token 2 has y-location == 173216
     3: 0, 2947526575
 cc1: note: token 3 has x-location == 0
 cc1: note: token 3 has y-location == 2947526575
@@ -178,26 +190,30 @@ x-location == y-location == 2947526575 encodes token # 800042942
 
 MACRO 0: PLUS (7 tokens)
   source_location interval: 2147483640 <= loc < 2147483647
-test.c:6:11: note: expansion point is location 5939
+test.c:6:11: note: expansion point is location 190019
    int a = PLUS (1,2);
-           ^
+           ^~~~
+
   map->start_location: 2147483640
   macro_locations:
-    0: 5945, 5410
-test.c:6:17: note: token 0 has x-location == 5945
+    0: 190208, 173088
+test.c:6:17: note: token 0 has x-location == 190208
    int a = PLUS (1,2);
                  ^
-test.c:6:17: note: token 0 has y-location == 5410
-    1: 5412, 5412
+
+test.c:6:17: note: token 0 has y-location == 173088
+    1: 173152, 173152
 In file included from test.c:1:0:
-test.h:3:22: note: token 1 has x-location == y-location == 5412
+test.h:3:22: note: token 1 has x-location == y-location == 173152
  #define PLUS(A, B) A + B
                       ^
-    2: 5947, 5414
-test.c:6:19: note: token 2 has x-location == 5947
+
+    2: 190272, 173216
+test.c:6:19: note: token 2 has x-location == 190272
    int a = PLUS (1,2);
                    ^
-test.c:6:19: note: token 2 has y-location == 5414
+
+test.c:6:19: note: token 2 has y-location == 173216
     3: 0, 2947526575
 cc1: note: token 3 has x-location == 0
 cc1: note: token 3 has y-location == 2947526575