ia64.c (stops_p): Added explicit initialization.

2006-03-16  Maxim Kuvyrkov <mkuvyrkov@ispras.ru>

        * config/ia64/ia64.c (stops_p): Added explicit initialization.
        (ia64_first_cycle_multipass_dfa_lookahead_guard_spec,
        ia64_h_i_d_extended, ia64_set_sched_flags, ia64_speculate_insn,
        ia64_needs_block_p, ia64_gen_check, ia64_sched_init_global,
	ia64_sched_finish_global): New static functions to implement
	hooks from gcc_target.sched.
	(spec_check_no, max_uid, pending_data_specs): New static variables.
        (ia64_mode_to_int, ia64_gen_spec_insn, ia64_spec_check_p,
	ia64_spec_check_src_p): New static functions.
        (ia64_adjust_cost): Renamed to ia64_adjust_cost_2.
        (TARGET_SCHED_ADJUST_COST): Removed.
        (TARGET_SCHED_ADJUST_COST_2, TARGET_SCHED_INIT_GLOBAL,
	TARGET_SCHED_FINISH_GLOBAL, TARGET_SCHED_H_I_D_EXTENDED,
	TARGET_SCHED_SPECULATE_INSN, TARGET_SCHED_NEEDS_BLOCK_P,
	TARGET_SCHED_GEN_CHECK,
	TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC,
        TARGET_SCHED_SET_SCHED_FLAGS): New macros.
        (update_set_flags, group_barrier_needed, set_src_needs_barrier):
	Fixed to handle speculation checks.
        (rtx_needs_barrier): Fixed to handle speculative loads and
        their checks.
        (ia64_variable_issue): Added code to count speculative loads and their
        checks.
        (ia64_first_cycle_multipass_dfa_lookahead_guard): Fixed to handle
        speculative loads.
        (enum SPEC_MODES, enum SPEC_GEN_LD_MAP, SPEC_GEN_CHECK_OFFSET):
        New enumerations.
        (SPEC_GEN_EXTEND_OFFSET, SPEC_N, SPEC_GEN_CHECK_MUTATION_OFFSET):
        New constants.
        (ia64_ld_address_bypass_p): Fixed to handle speculative loads.
	(ia64_reorg): Reset stops_p to NULL after it is freed.
        * config/ia64/ia64.md (UNSPEC_LDA, UNSPEC_LDS, UNSPEC_LDSA,
	UNSPEC_LDCCLR, UNSPEC_CHKACLR, UNSPEC_CHKS): New constants.
        (itanium_class): chk_s renamed to chk_s_i.  New constants: chk_s_f,
        chk_a.
        (data_speculative, control_speculative, check_load): New attributes.
        (mov<mode>_advanced, mov<mode>_speculative,
        mov<mode>_speculative_advanced, zero_extend<mode>di2_advanced,
        zero_extend<mode>di2_speculative,
        zero_extend<mode>di2_speculative_advanced): New patterns for
        data and control speculative loads.
        (mov<mode>_clr, zero_extend<mode>di2_clr): New patterns for
        check loads.
        (advanced_load_check_clr_<mode>, speculation_check_<mode>):
        New pattern for data and control speculation checks.
	(MODE, MODE_FOR_EXTEND, output_a, output_s, output_sa, output_c_clr,
	ld_reg_constr, ldc_reg_constr, chk_reg_constr, mem_constr,
	reg_pred_prefix, ld_class, chka_class, chks_class, attr_yes):
	Auxiliary definitions for the patterns.
        * config/ia64/itanium1.md (1_fldc, 1_fldpc, 1_ldc, 1_chk_s_f, 1_chk_a,
        1b_fldc, 1b_fldpc, 1b_ldc, 1b_chk_s_f, 1b_chk_a): New resource
        constraints.
	(1_fld, 1_fldp, 1_ld, 1b_fld, 1b_fldp, 1b_ld): Add a condition
        for speculation.
        (1_chk_s, 1b_chk_s): Renamed to 1_chk_s_i, 1b_chk_s_i.
        * config/ia64/itanium2.md (2_flda, 2_fldc, 2_fldpc, 2_ldc, 2_chk_s_f,
	2_chk_a, 2b_flda, 2b_fldc, 2b_fldpc, 2b_ldc, 2b_chk_s_f, 2b_chk_a):
	New resource constraints.
	(2_fld, 2_fldp, 2_ld, 2b_fld, 2b_fldp, 2b_ld): Add a condition
        for speculation.
        (2_chk_s, 2b_chk_s): Renamed to 2_chk_s_i, 2b_chk_s_i.
        * config/ia64/ia64.opt (msched-br-data-spec, msched-ar-data-spec,
        msched-control-spec, msched-br-in-data-spec, msched-ar-in-data-spec,
        msched-in-control-spec, msched-ldc, msched-control-ldc,
        msched-spec-verbose, msched-prefer-non-data-spec-insns,
        msched-prefer-non-control-spec-insns,
	msched-count-spec-in-critical-path): New flags to tune speculative
	scheduling.
	* doc/invoke.texi (msched-br-data-spec, msched-ar-data-spec,
        msched-control-spec, msched-br-in-data-spec, msched-ar-in-data-spec,
        msched-in-control-spec, msched-ldc, msched-control-ldc,
        msched-spec-verbose, msched-prefer-non-data-spec-insns,
        msched-prefer-non-control-spec-insns,
	msched-count-spec-in-critical-path): Document new flags.

From-SVN: r112129

7 files changed, 1357 insertions, 71 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index b191bcffb60..579f7a3c58f 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,5 +1,82 @@
 2006-03-16  Maxim Kuvyrkov <mkuvyrkov@ispras.ru>
 
+        * config/ia64/ia64.c (stops_p): Added explicit initialization.
+        (ia64_first_cycle_multipass_dfa_lookahead_guard_spec,
+        ia64_h_i_d_extended, ia64_set_sched_flags, ia64_speculate_insn,
+        ia64_needs_block_p, ia64_gen_check, ia64_sched_init_global,
+	ia64_sched_finish_global): New static functions to implement
+	hooks from gcc_target.sched.
+	(spec_check_no, max_uid, pending_data_specs): New static variables.
+        (ia64_mode_to_int, ia64_gen_spec_insn, ia64_spec_check_p,
+	ia64_spec_check_src_p): New static functions.
+        (ia64_adjust_cost): Renamed to ia64_adjust_cost_2.
+        (TARGET_SCHED_ADJUST_COST): Removed.
+        (TARGET_SCHED_ADJUST_COST_2, TARGET_SCHED_INIT_GLOBAL,
+	TARGET_SCHED_FINISH_GLOBAL, TARGET_SCHED_H_I_D_EXTENDED,
+	TARGET_SCHED_SPECULATE_INSN, TARGET_SCHED_NEEDS_BLOCK_P,
+	TARGET_SCHED_GEN_CHECK,
+	TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC,
+        TARGET_SCHED_SET_SCHED_FLAGS): New macros.
+        (update_set_flags, group_barrier_needed, set_src_needs_barrier):
+	Fixed to handle speculation checks.
+        (rtx_needs_barrier): Fixed to handle speculative loads and
+        their checks.
+        (ia64_variable_issue): Added code to count speculative loads and their
+        checks.
+        (ia64_first_cycle_multipass_dfa_lookahead_guard): Fixed to handle
+        speculative loads.
+        (enum SPEC_MODES, enum SPEC_GEN_LD_MAP, SPEC_GEN_CHECK_OFFSET):
+        New enumerations.
+        (SPEC_GEN_EXTEND_OFFSET, SPEC_N, SPEC_GEN_CHECK_MUTATION_OFFSET):
+        New constants.
+        (ia64_ld_address_bypass_p): Fixed to handle speculative loads.
+	(ia64_reorg): Reset stops_p to NULL after it is freed.
+        * config/ia64/ia64.md (UNSPEC_LDA, UNSPEC_LDS, UNSPEC_LDSA,
+	UNSPEC_LDCCLR, UNSPEC_CHKACLR, UNSPEC_CHKS): New constants.
+        (itanium_class): chk_s renamed to chk_s_i.  New constants: chk_s_f,
+        chk_a.
+        (data_speculative, control_speculative, check_load): New attributes.
+        (mov<mode>_advanced, mov<mode>_speculative,
+        mov<mode>_speculative_advanced, zero_extend<mode>di2_advanced,
+        zero_extend<mode>di2_speculative,
+        zero_extend<mode>di2_speculative_advanced): New patterns for
+        data and control speculative loads.
+        (mov<mode>_clr, zero_extend<mode>di2_clr): New patterns for
+        check loads.
+        (advanced_load_check_clr_<mode>, speculation_check_<mode>):
+        New pattern for data and control speculation checks.
+	(MODE, MODE_FOR_EXTEND, output_a, output_s, output_sa, output_c_clr,
+	ld_reg_constr, ldc_reg_constr, chk_reg_constr, mem_constr,
+	reg_pred_prefix, ld_class, chka_class, chks_class, attr_yes):
+	Auxiliary definitions for the patterns.
+        * config/ia64/itanium1.md (1_fldc, 1_fldpc, 1_ldc, 1_chk_s_f, 1_chk_a,
+        1b_fldc, 1b_fldpc, 1b_ldc, 1b_chk_s_f, 1b_chk_a): New resource
+        constraints.
+	(1_fld, 1_fldp, 1_ld, 1b_fld, 1b_fldp, 1b_ld): Add a condition
+        for speculation.
+        (1_chk_s, 1b_chk_s): Renamed to 1_chk_s_i, 1b_chk_s_i.
+        * config/ia64/itanium2.md (2_flda, 2_fldc, 2_fldpc, 2_ldc, 2_chk_s_f,
+	2_chk_a, 2b_flda, 2b_fldc, 2b_fldpc, 2b_ldc, 2b_chk_s_f, 2b_chk_a):
+	New resource constraints.
+	(2_fld, 2_fldp, 2_ld, 2b_fld, 2b_fldp, 2b_ld): Add a condition
+        for speculation.
+        (2_chk_s, 2b_chk_s): Renamed to 2_chk_s_i, 2b_chk_s_i.
+        * config/ia64/ia64.opt (msched-br-data-spec, msched-ar-data-spec,
+        msched-control-spec, msched-br-in-data-spec, msched-ar-in-data-spec,
+        msched-in-control-spec, msched-ldc, msched-control-ldc,
+        msched-spec-verbose, msched-prefer-non-data-spec-insns,
+        msched-prefer-non-control-spec-insns,
+	msched-count-spec-in-critical-path): New flags to tune speculative
+	scheduling.
+	* doc/invoke.texi (msched-br-data-spec, msched-ar-data-spec,
+        msched-control-spec, msched-br-in-data-spec, msched-ar-in-data-spec,
+        msched-in-control-spec, msched-ldc, msched-control-ldc,
+        msched-spec-verbose, msched-prefer-non-data-spec-insns,
+        msched-prefer-non-control-spec-insns,
+	msched-count-spec-in-critical-path): Document new flags.
+
+2006-03-16  Maxim Kuvyrkov <mkuvyrkov@ispras.ru>
+
         * target.h (struct spec_info_def): New opaque declaration.
         (struct gcc_target.sched): New fields: adjust_cost_2, h_i_d_extended,
         speculate_insn, needs_block_p, gen_check,
diff --git a/gcc/config/ia64/ia64.c b/gcc/config/ia64/ia64.c
index 5cfcfd00ab8..2fb1c153366 100644
--- a/gcc/config/ia64/ia64.c
+++ b/gcc/config/ia64/ia64.c
@@ -158,7 +158,17 @@ static void ia64_dependencies_evaluation_hook (rtx, rtx);
 static void ia64_init_dfa_pre_cycle_insn (void);
 static rtx ia64_dfa_pre_cycle_insn (void);
 static int ia64_first_cycle_multipass_dfa_lookahead_guard (rtx);
+static bool ia64_first_cycle_multipass_dfa_lookahead_guard_spec (rtx);
 static int ia64_dfa_new_cycle (FILE *, int, rtx, int, int, int *);
+static void ia64_h_i_d_extended (void);
+static int ia64_mode_to_int (enum machine_mode);
+static void ia64_set_sched_flags (spec_info_t);
+static int ia64_speculate_insn (rtx, ds_t, rtx *);
+static rtx ia64_gen_spec_insn (rtx, ds_t, int, bool, bool);
+static bool ia64_needs_block_p (rtx);
+static rtx ia64_gen_check (rtx, rtx, bool);
+static int ia64_spec_check_p (rtx);
+static int ia64_spec_check_src_p (rtx);
 static rtx gen_tls_get_addr (void);
 static rtx gen_thread_pointer (void);
 static int find_gr_spill (int);
@@ -200,8 +210,10 @@ static void ia64_output_function_epilogue (FILE *, HOST_WIDE_INT);
 static void ia64_output_function_end_prologue (FILE *);
 
 static int ia64_issue_rate (void);
-static int ia64_adjust_cost (rtx, rtx, rtx, int);
+static int ia64_adjust_cost_2 (rtx, int, rtx, int);
 static void ia64_sched_init (FILE *, int, int);
+static void ia64_sched_init_global (FILE *, int, int);
+static void ia64_sched_finish_global (FILE *, int);
 static void ia64_sched_finish (FILE *, int);
 static int ia64_dfa_sched_reorder (FILE *, int, rtx *, int *, int, int);
 static int ia64_sched_reorder (FILE *, int, rtx *, int *, int);
@@ -313,8 +325,8 @@ static const struct attribute_spec ia64_attribute_table[] =
 #undef TARGET_IN_SMALL_DATA_P
 #define TARGET_IN_SMALL_DATA_P  ia64_in_small_data_p
 
-#undef TARGET_SCHED_ADJUST_COST
-#define TARGET_SCHED_ADJUST_COST ia64_adjust_cost
+#undef TARGET_SCHED_ADJUST_COST_2
+#define TARGET_SCHED_ADJUST_COST_2 ia64_adjust_cost_2
 #undef TARGET_SCHED_ISSUE_RATE
 #define TARGET_SCHED_ISSUE_RATE ia64_issue_rate
 #undef TARGET_SCHED_VARIABLE_ISSUE
@@ -323,6 +335,10 @@ static const struct attribute_spec ia64_attribute_table[] =
 #define TARGET_SCHED_INIT ia64_sched_init
 #undef TARGET_SCHED_FINISH
 #define TARGET_SCHED_FINISH ia64_sched_finish
+#undef TARGET_SCHED_INIT_GLOBAL
+#define TARGET_SCHED_INIT_GLOBAL ia64_sched_init_global
+#undef TARGET_SCHED_FINISH_GLOBAL
+#define TARGET_SCHED_FINISH_GLOBAL ia64_sched_finish_global
 #undef TARGET_SCHED_REORDER
 #define TARGET_SCHED_REORDER ia64_sched_reorder
 #undef TARGET_SCHED_REORDER2
@@ -346,6 +362,25 @@ static const struct attribute_spec ia64_attribute_table[] =
 #undef TARGET_SCHED_DFA_NEW_CYCLE
 #define TARGET_SCHED_DFA_NEW_CYCLE ia64_dfa_new_cycle
 
+#undef TARGET_SCHED_H_I_D_EXTENDED
+#define TARGET_SCHED_H_I_D_EXTENDED ia64_h_i_d_extended
+
+#undef TARGET_SCHED_SET_SCHED_FLAGS
+#define TARGET_SCHED_SET_SCHED_FLAGS ia64_set_sched_flags
+
+#undef TARGET_SCHED_SPECULATE_INSN
+#define TARGET_SCHED_SPECULATE_INSN ia64_speculate_insn
+
+#undef TARGET_SCHED_NEEDS_BLOCK_P
+#define TARGET_SCHED_NEEDS_BLOCK_P ia64_needs_block_p
+
+#undef TARGET_SCHED_GEN_CHECK
+#define TARGET_SCHED_GEN_CHECK ia64_gen_check
+
+#undef TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC
+#define TARGET_SCHED_FIRST_CYCLE_MULTIPASS_DFA_LOOKAHEAD_GUARD_SPEC\
+  ia64_first_cycle_multipass_dfa_lookahead_guard_spec
+
 #undef TARGET_FUNCTION_OK_FOR_SIBCALL
 #define TARGET_FUNCTION_OK_FOR_SIBCALL ia64_function_ok_for_sibcall
 #undef TARGET_ARG_PARTIAL_BYTES
@@ -5414,13 +5449,15 @@ update_set_flags (rtx x, struct reg_flags *pflags)
       return;
 
     case IF_THEN_ELSE:
-      /* There are three cases here:
+      /* There are four cases here:
 	 (1) The destination is (pc), in which case this is a branch,
 	 nothing here applies.
 	 (2) The destination is ar.lc, in which case this is a
 	 doloop_end_internal,
 	 (3) The destination is an fp register, in which case this is
 	 an fselect instruction.
+	 (4) The condition has (unspec [(reg)] UNSPEC_LDC), in which case 
+	 this is a check load.
 	 In all cases, nothing we do in this function applies.  */
       return;
 
@@ -5465,18 +5502,29 @@ set_src_needs_barrier (rtx x, struct reg_flags flags, int pred)
       /* X is a conditional branch.  */
       /* ??? This seems redundant, as the caller sets this bit for
 	 all JUMP_INSNs.  */
-      flags.is_branch = 1;
+      if (!ia64_spec_check_src_p (src))
+	flags.is_branch = 1;
       return rtx_needs_barrier (src, flags, pred);
     }
 
-  need_barrier = rtx_needs_barrier (src, flags, pred);
+  if (ia64_spec_check_src_p (src))
+    /* Avoid checking one register twice (in condition 
+       and in 'then' section) for ldc pattern.  */
+    {
+      gcc_assert (REG_P (XEXP (src, 2)));
+      need_barrier = rtx_needs_barrier (XEXP (src, 2), flags, pred);
+		  
+      /* We process MEM below.  */
+      src = XEXP (src, 1);
+    }
+
+  need_barrier |= rtx_needs_barrier (src, flags, pred);
 
   dst = SET_DEST (x);
   if (GET_CODE (dst) == ZERO_EXTRACT)
     {
       need_barrier |= rtx_needs_barrier (XEXP (dst, 1), flags, pred);
       need_barrier |= rtx_needs_barrier (XEXP (dst, 2), flags, pred);
-      dst = XEXP (dst, 0);
     }
   return need_barrier;
 }
@@ -5725,6 +5773,11 @@ rtx_needs_barrier (rtx x, struct reg_flags flags, int pred)
 	case UNSPEC_SETF_EXP:
         case UNSPEC_ADDP4:
 	case UNSPEC_FR_SQRT_RECIP_APPROX:
+	case UNSPEC_LDA:
+	case UNSPEC_LDS:
+	case UNSPEC_LDSA:
+	case UNSPEC_CHKACLR:
+        case UNSPEC_CHKS:
 	  need_barrier = rtx_needs_barrier (XVECEXP (x, 0, 0), flags, pred);
 	  break;
 
@@ -5872,7 +5925,8 @@ group_barrier_needed (rtx insn)
       break;
 
     case JUMP_INSN:
-      flags.is_branch = 1;
+      if (!ia64_spec_check_p (insn))
+	flags.is_branch = 1;
 
       /* Don't bundle a jump following a call.  */
       if ((pat = prev_active_insn (insn))
@@ -6126,7 +6180,15 @@ static state_t prev_cycle_state = NULL;
 /* The following array element values are TRUE if the corresponding
    insn requires to add stop bits before it.  */
 
-static char *stops_p;
+static char *stops_p = NULL;
+
+/* The following array element values are ZERO for non-speculative
+   instructions and hold corresponding speculation check number for
+   speculative instructions.  */
+static int *spec_check_no = NULL;
+
+/* Size of spec_check_no array.  */
+static int max_uid = 0;
 
 /* The following variable is used to set up the mentioned above array.  */
 
@@ -6148,6 +6210,9 @@ static int *clocks;
 
 static int *add_cycles;
 
+/* The following variable value is number of data speculations in progress.  */
+static int pending_data_specs = 0;
+
 static rtx ia64_single_set (rtx);
 static void ia64_emit_insn_before (rtx, rtx);
 
@@ -6197,16 +6262,18 @@ ia64_single_set (rtx insn)
   return ret;
 }
 
-/* Adjust the cost of a scheduling dependency.  Return the new cost of
-   a dependency LINK or INSN on DEP_INSN.  COST is the current cost.  */
+/* Adjust the cost of a scheduling dependency.
+   Return the new cost of a dependency of type DEP_TYPE or INSN on DEP_INSN.
+   COST is the current cost.  */
 
 static int
-ia64_adjust_cost (rtx insn, rtx link, rtx dep_insn, int cost)
+ia64_adjust_cost_2 (rtx insn, int dep_type1, rtx dep_insn, int cost)
 {
+  enum reg_note dep_type = (enum reg_note) dep_type1;
   enum attr_itanium_class dep_class;
   enum attr_itanium_class insn_class;
 
-  if (REG_NOTE_KIND (link) != REG_DEP_OUTPUT)
+  if (dep_type != REG_DEP_OUTPUT)
     return cost;
 
   insn_class = ia64_safe_itanium_class (insn);
@@ -6293,6 +6360,26 @@ ia64_sched_init (FILE *dump ATTRIBUTE_UNUSED,
   init_insn_group_barriers ();
 }
 
+/* We're beginning a scheduling pass.  Check assertion.  */
+
+static void
+ia64_sched_init_global (FILE *dump ATTRIBUTE_UNUSED,
+                        int sched_verbose ATTRIBUTE_UNUSED,
+                        int max_ready ATTRIBUTE_UNUSED)
+{  
+  gcc_assert (!pending_data_specs);
+}
+
+/* Scheduling pass is now finished.  Free/reset static variable.  */
+static void
+ia64_sched_finish_global (FILE *dump ATTRIBUTE_UNUSED,
+			  int sched_verbose ATTRIBUTE_UNUSED)
+{
+  free (spec_check_no);
+  spec_check_no = 0;
+  max_uid = 0;
+}
+
 /* We are about to being issuing insns for this clock cycle.
    Override the default sort algorithm to better slot instructions.  */
 
@@ -6415,6 +6502,16 @@ ia64_variable_issue (FILE *dump ATTRIBUTE_UNUSED,
 		     rtx insn ATTRIBUTE_UNUSED,
 		     int can_issue_more ATTRIBUTE_UNUSED)
 {
+  if (current_sched_info->flags & DO_SPECULATION)
+    /* Modulo scheduling does not extend h_i_d when emitting
+       new instructions.  Deal with it.  */
+    {
+      if (DONE_SPEC (insn) & BEGIN_DATA)
+	pending_data_specs++;
+      if (CHECK_SPEC (insn) & BEGIN_DATA)
+	pending_data_specs--;
+    }
+
   last_scheduled_insn = insn;
   memcpy (prev_cycle_state, curr_state, dfa_state_size);
   if (reload_completed)
@@ -6437,8 +6534,22 @@ static int
 ia64_first_cycle_multipass_dfa_lookahead_guard (rtx insn)
 {
   gcc_assert (insn  && INSN_P (insn));
-  return (!reload_completed
-	  || !safe_group_barrier_needed (insn));
+  return ((!reload_completed
+	   || !safe_group_barrier_needed (insn))
+	  && ia64_first_cycle_multipass_dfa_lookahead_guard_spec (insn));
+}
+
+/* We are choosing insn from the ready queue.  Return nonzero if INSN
+   can be chosen.  */
+
+static bool
+ia64_first_cycle_multipass_dfa_lookahead_guard_spec (rtx insn)
+{
+  gcc_assert (insn  && INSN_P (insn));
+  /* Size of ALAT is 32.  As far as we perform conservative data speculation,
+     we keep ALAT half-empty.  */
+  return (pending_data_specs < 16
+	  || !(TODO_SPEC (insn) & BEGIN_DATA));
 }
 
 /* The following variable value is pseudo-insn used by the DFA insn
@@ -6526,6 +6637,578 @@ ia64_dfa_new_cycle (FILE *dump, int verbose, rtx insn, int last_clock,
   return 0;
 }
 
+/* Implement targetm.sched.h_i_d_extended hook.
+   Extend internal data structures.  */
+static void
+ia64_h_i_d_extended (void)
+{
+  if (current_sched_info->flags & DO_SPECULATION)
+    {
+      int new_max_uid = get_max_uid () + 1;
+
+      spec_check_no = xrecalloc (spec_check_no, new_max_uid,
+				 max_uid, sizeof (*spec_check_no));
+      max_uid = new_max_uid;
+    }
+
+  if (stops_p != NULL) 
+    {
+      int new_clocks_length = get_max_uid () + 1;
+      
+      stops_p = xrecalloc (stops_p, new_clocks_length, clocks_length, 1);
+      
+      if (ia64_tune == PROCESSOR_ITANIUM)
+	{
+	  clocks = xrecalloc (clocks, new_clocks_length, clocks_length,
+			      sizeof (int));
+	  add_cycles = xrecalloc (add_cycles, new_clocks_length, clocks_length,
+				  sizeof (int));
+	}
+      
+      clocks_length = new_clocks_length;
+    }
+}
+
+/* Constants that help mapping 'enum machine_mode' to int.  */
+enum SPEC_MODES
+  {
+    SPEC_MODE_INVALID = -1,
+    SPEC_MODE_FIRST = 0,
+    SPEC_MODE_FOR_EXTEND_FIRST = 1,
+    SPEC_MODE_FOR_EXTEND_LAST = 3,
+    SPEC_MODE_LAST = 8
+  };
+
+/* Return index of the MODE.  */
+static int
+ia64_mode_to_int (enum machine_mode mode)
+{
+  switch (mode)
+    {
+    case BImode: return 0; /* SPEC_MODE_FIRST  */
+    case QImode: return 1; /* SPEC_MODE_FOR_EXTEND_FIRST  */
+    case HImode: return 2;
+    case SImode: return 3; /* SPEC_MODE_FOR_EXTEND_LAST  */
+    case DImode: return 4;
+    case SFmode: return 5;
+    case DFmode: return 6;
+    case XFmode: return 7;
+    case TImode:
+      /* ??? This mode needs testing.  Bypasses for ldfp8 instruction are not
+	 mentioned in itanium[12].md.  Predicate fp_register_operand also
+	 needs to be defined.  Bottom line: better disable for now.  */
+      return SPEC_MODE_INVALID;
+    default:     return SPEC_MODE_INVALID;
+    }
+}
+
+/* Provide information about speculation capabilities.  */
+static void
+ia64_set_sched_flags (spec_info_t spec_info)
+{
+  unsigned int *flags = &(current_sched_info->flags);
+
+  if (*flags & SCHED_RGN
+      || *flags & SCHED_EBB)  
+    {
+      int mask = 0;
+
+      if ((mflag_sched_br_data_spec && !reload_completed)
+	  || (mflag_sched_ar_data_spec && reload_completed))
+	{
+	  mask |= BEGIN_DATA;
+	  
+	  if ((mflag_sched_br_in_data_spec && !reload_completed)
+	      || (mflag_sched_ar_in_data_spec && reload_completed))
+	    mask |= BE_IN_DATA;
+	}
+      
+      if (mflag_sched_control_spec)
+	{
+	  mask |= BEGIN_CONTROL;
+	  
+	  if (mflag_sched_in_control_spec)
+	    mask |= BE_IN_CONTROL;
+	}
+
+      gcc_assert (*flags & USE_GLAT);
+
+      if (mask)
+	{
+	  *flags |= USE_DEPS_LIST | DETACH_LIFE_INFO | DO_SPECULATION;
+	  
+	  spec_info->mask = mask;
+	  spec_info->flags = 0;
+      
+	  if ((mask & DATA_SPEC) && mflag_sched_prefer_non_data_spec_insns)
+	    spec_info->flags |= PREFER_NON_DATA_SPEC;
+
+	  if ((mask & CONTROL_SPEC)
+	      && mflag_sched_prefer_non_control_spec_insns)
+	    spec_info->flags |= PREFER_NON_CONTROL_SPEC;
+
+	  if (mflag_sched_spec_verbose)
+	    {
+	      if (sched_verbose >= 1)
+		spec_info->dump = sched_dump;
+	      else
+		spec_info->dump = stderr;
+	    }
+	  else
+	    spec_info->dump = 0;
+	  
+	  if (mflag_sched_count_spec_in_critical_path)
+	    spec_info->flags |= COUNT_SPEC_IN_CRITICAL_PATH;
+	}
+    }
+}
+
+/* Implement targetm.sched.speculate_insn hook.
+   Check if the INSN can be TS speculative.
+   If 'no' - return -1.
+   If 'yes' - generate speculative pattern in the NEW_PAT and return 1.
+   If current pattern of the INSN already provides TS speculation, return 0.  */
+static int
+ia64_speculate_insn (rtx insn, ds_t ts, rtx *new_pat)
+{  
+  rtx pat, reg, mem, mem_reg;
+  int mode_no, gen_p = 1;
+  bool extend_p;
+  
+  gcc_assert (!(ts & ~BEGIN_SPEC) && ts);
+           
+  pat = PATTERN (insn);
+
+  if (GET_CODE (pat) == COND_EXEC)
+    pat = COND_EXEC_CODE (pat);
+
+  if (GET_CODE (pat) != SET)
+    return -1;
+  reg = SET_DEST (pat);
+  if (!REG_P (reg))
+    return -1;
+
+  mem = SET_SRC (pat);  
+  if (GET_CODE (mem) == ZERO_EXTEND)
+    {
+      mem = XEXP (mem, 0);
+      extend_p = true;      
+    }
+  else
+    extend_p = false;
+
+  if (GET_CODE (mem) == UNSPEC)
+    {
+      int code;
+      
+      code = XINT (mem, 1);
+      if (code != UNSPEC_LDA && code != UNSPEC_LDS && code != UNSPEC_LDSA)
+	return -1;
+
+      if ((code == UNSPEC_LDA && !(ts & BEGIN_CONTROL))
+	  || (code == UNSPEC_LDS && !(ts & BEGIN_DATA))
+	  || code == UNSPEC_LDSA)
+	gen_p = 0;
+
+      mem = XVECEXP (mem, 0, 0);
+      gcc_assert (MEM_P (mem));
+    }
+  if (!MEM_P (mem))
+    return -1;
+  mem_reg = XEXP (mem, 0);
+  if (!REG_P (mem_reg))
+    return -1;
+     
+  /* We should use MEM's mode since REG's mode in presence of ZERO_EXTEND
+     will always be DImode.  */
+  mode_no = ia64_mode_to_int (GET_MODE (mem));
+  
+  if (mode_no == SPEC_MODE_INVALID
+      || (extend_p
+	  && !(SPEC_MODE_FOR_EXTEND_FIRST <= mode_no
+	       && mode_no <= SPEC_MODE_FOR_EXTEND_LAST)))
+    return -1;
+
+  extract_insn_cached (insn);
+  gcc_assert (reg == recog_data.operand[0] && mem == recog_data.operand[1]);
+  *new_pat = ia64_gen_spec_insn (insn, ts, mode_no, gen_p != 0, extend_p);
+
+  return gen_p;
+}
+
+enum
+  {
+    /* Offset to reach ZERO_EXTEND patterns.  */
+    SPEC_GEN_EXTEND_OFFSET = SPEC_MODE_LAST - SPEC_MODE_FOR_EXTEND_FIRST + 1,
+    /* Number of patterns for each speculation mode.  */
+    SPEC_N = (SPEC_MODE_LAST
+              + SPEC_MODE_FOR_EXTEND_LAST - SPEC_MODE_FOR_EXTEND_FIRST + 2)
+  };
+
+enum SPEC_GEN_LD_MAP
+  {
+    /* Offset to ld.a patterns.  */
+    SPEC_GEN_A = 0 * SPEC_N,
+    /* Offset to ld.s patterns.  */
+    SPEC_GEN_S = 1 * SPEC_N,
+    /* Offset to ld.sa patterns.  */
+    SPEC_GEN_SA = 2 * SPEC_N,
+    /* Offset to ld.sa patterns.  For this patterns corresponding ld.c will
+       mutate to chk.s.  */
+    SPEC_GEN_SA_FOR_S = 3 * SPEC_N
+  };
+
+/* These offsets are used to get (4 * SPEC_N).  */
+enum SPEC_GEN_CHECK_OFFSET
+  {
+    SPEC_GEN_CHKA_FOR_A_OFFSET = 4 * SPEC_N - SPEC_GEN_A,
+    SPEC_GEN_CHKA_FOR_SA_OFFSET = 4 * SPEC_N - SPEC_GEN_SA
+  };
+
+/* If GEN_P is true, calculate the index of needed speculation check and return
+   speculative pattern for INSN with speculative mode TS, machine mode
+   MODE_NO and with ZERO_EXTEND (if EXTEND_P is true).
+   If GEN_P is false, just calculate the index of needed speculation check.  */
+static rtx
+ia64_gen_spec_insn (rtx insn, ds_t ts, int mode_no, bool gen_p, bool extend_p)
+{
+  rtx pat, new_pat;
+  int load_no;
+  int shift = 0;
+
+  static rtx (* const gen_load[]) (rtx, rtx) = {
+    gen_movbi_advanced,
+    gen_movqi_advanced,
+    gen_movhi_advanced,
+    gen_movsi_advanced,
+    gen_movdi_advanced,
+    gen_movsf_advanced,
+    gen_movdf_advanced,
+    gen_movxf_advanced,
+    gen_movti_advanced,
+    gen_zero_extendqidi2_advanced,
+    gen_zero_extendhidi2_advanced,
+    gen_zero_extendsidi2_advanced,
+
+    gen_movbi_speculative,
+    gen_movqi_speculative,
+    gen_movhi_speculative,
+    gen_movsi_speculative,
+    gen_movdi_speculative,
+    gen_movsf_speculative,
+    gen_movdf_speculative,
+    gen_movxf_speculative,
+    gen_movti_speculative,
+    gen_zero_extendqidi2_speculative,
+    gen_zero_extendhidi2_speculative,
+    gen_zero_extendsidi2_speculative,
+
+    gen_movbi_speculative_advanced,
+    gen_movqi_speculative_advanced,
+    gen_movhi_speculative_advanced,
+    gen_movsi_speculative_advanced,
+    gen_movdi_speculative_advanced,
+    gen_movsf_speculative_advanced,
+    gen_movdf_speculative_advanced,
+    gen_movxf_speculative_advanced,
+    gen_movti_speculative_advanced,
+    gen_zero_extendqidi2_speculative_advanced,
+    gen_zero_extendhidi2_speculative_advanced,
+    gen_zero_extendsidi2_speculative_advanced,
+
+    gen_movbi_speculative_advanced,
+    gen_movqi_speculative_advanced,
+    gen_movhi_speculative_advanced,
+    gen_movsi_speculative_advanced,
+    gen_movdi_speculative_advanced,
+    gen_movsf_speculative_advanced,
+    gen_movdf_speculative_advanced,
+    gen_movxf_speculative_advanced,
+    gen_movti_speculative_advanced,
+    gen_zero_extendqidi2_speculative_advanced,
+    gen_zero_extendhidi2_speculative_advanced,
+    gen_zero_extendsidi2_speculative_advanced
+  };
+
+  load_no = extend_p ? mode_no + SPEC_GEN_EXTEND_OFFSET : mode_no;
+
+  if (ts & BEGIN_DATA)
+    {
+      /* We don't need recovery because even if this is ld.sa
+	 ALAT entry will be allocated only if NAT bit is set to zero. 
+	 So it is enough to use ld.c here.  */	  
+
+      if (ts & BEGIN_CONTROL)
+	{	      
+	  load_no += SPEC_GEN_SA;
+
+	  if (!mflag_sched_ldc)
+	    shift = SPEC_GEN_CHKA_FOR_SA_OFFSET;
+	}
+      else
+	{
+	  load_no += SPEC_GEN_A;
+
+	  if (!mflag_sched_ldc)		
+	    shift = SPEC_GEN_CHKA_FOR_A_OFFSET;
+	}
+    }
+  else if (ts & BEGIN_CONTROL)
+    {
+      /* ld.sa can be used instead of ld.s to avoid basic block splitting.  */
+      if (!mflag_control_ldc)
+	load_no += SPEC_GEN_S;
+      else
+	{
+	  gcc_assert (mflag_sched_ldc);
+	  load_no += SPEC_GEN_SA_FOR_S;
+	}
+    }
+  else
+    gcc_unreachable ();
+
+  /* Set the desired check index.  We add '1', because zero element in this
+     array means, that instruction with such uid is non-speculative.  */
+  spec_check_no[INSN_UID (insn)] = load_no + shift + 1;
+
+  if (!gen_p)
+    return 0;
+
+  new_pat = gen_load[load_no] (copy_rtx (recog_data.operand[0]),
+			       copy_rtx (recog_data.operand[1]));
+
+  pat = PATTERN (insn);
+  if (GET_CODE (pat) == COND_EXEC)
+    new_pat = gen_rtx_COND_EXEC (VOIDmode, copy_rtx 
+				 (COND_EXEC_TEST (pat)), new_pat);
+
+  return new_pat;
+}
+
+/* Offset to branchy checks.  */
+enum { SPEC_GEN_CHECK_MUTATION_OFFSET = 5 * SPEC_N };
+
+/* Return nonzero, if INSN needs branchy recovery check.  */
+static bool
+ia64_needs_block_p (rtx insn)
+{
+  int check_no;
+
+  check_no = spec_check_no[INSN_UID(insn)] - 1;
+  gcc_assert (0 <= check_no && check_no < SPEC_GEN_CHECK_MUTATION_OFFSET);
+
+  return ((SPEC_GEN_S <= check_no && check_no < SPEC_GEN_S + SPEC_N)
+	  || (4 * SPEC_N <= check_no && check_no < 4 * SPEC_N + SPEC_N));
+}
+
+/* Generate (or regenerate, if (MUTATE_P)) recovery check for INSN.
+   If (LABEL != 0 || MUTATE_P), generate branchy recovery check.
+   Otherwise, generate a simple check.  */
+static rtx
+ia64_gen_check (rtx insn, rtx label, bool mutate_p)
+{
+  rtx op1, pat, check_pat;
+
+  static rtx (* const gen_check[]) (rtx, rtx) = {
+    gen_movbi_clr,
+    gen_movqi_clr,
+    gen_movhi_clr,
+    gen_movsi_clr,
+    gen_movdi_clr,
+    gen_movsf_clr,
+    gen_movdf_clr,
+    gen_movxf_clr,
+    gen_movti_clr,
+    gen_zero_extendqidi2_clr,
+    gen_zero_extendhidi2_clr,
+    gen_zero_extendsidi2_clr,
+
+    gen_speculation_check_bi,
+    gen_speculation_check_qi,
+    gen_speculation_check_hi,
+    gen_speculation_check_si,
+    gen_speculation_check_di,
+    gen_speculation_check_sf,
+    gen_speculation_check_df,
+    gen_speculation_check_xf,
+    gen_speculation_check_ti,
+    gen_speculation_check_di,
+    gen_speculation_check_di,
+    gen_speculation_check_di,
+
+    gen_movbi_clr,
+    gen_movqi_clr,
+    gen_movhi_clr,
+    gen_movsi_clr,
+    gen_movdi_clr,
+    gen_movsf_clr,
+    gen_movdf_clr,
+    gen_movxf_clr,
+    gen_movti_clr,
+    gen_zero_extendqidi2_clr,
+    gen_zero_extendhidi2_clr,
+    gen_zero_extendsidi2_clr,
+
+    gen_movbi_clr,
+    gen_movqi_clr,
+    gen_movhi_clr,
+    gen_movsi_clr,
+    gen_movdi_clr,
+    gen_movsf_clr,
+    gen_movdf_clr,
+    gen_movxf_clr,
+    gen_movti_clr,
+    gen_zero_extendqidi2_clr,
+    gen_zero_extendhidi2_clr,
+    gen_zero_extendsidi2_clr,
+
+    gen_advanced_load_check_clr_bi,
+    gen_advanced_load_check_clr_qi,
+    gen_advanced_load_check_clr_hi,
+    gen_advanced_load_check_clr_si,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_sf,
+    gen_advanced_load_check_clr_df,
+    gen_advanced_load_check_clr_xf,
+    gen_advanced_load_check_clr_ti,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_di,
+
+    /* Following checks are generated during mutation.  */
+    gen_advanced_load_check_clr_bi,
+    gen_advanced_load_check_clr_qi,
+    gen_advanced_load_check_clr_hi,
+    gen_advanced_load_check_clr_si,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_sf,
+    gen_advanced_load_check_clr_df,
+    gen_advanced_load_check_clr_xf,
+    gen_advanced_load_check_clr_ti,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_di,
+
+    0,0,0,0,0,0,0,0,0,0,0,0,
+
+    gen_advanced_load_check_clr_bi,
+    gen_advanced_load_check_clr_qi,
+    gen_advanced_load_check_clr_hi,
+    gen_advanced_load_check_clr_si,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_sf,
+    gen_advanced_load_check_clr_df,
+    gen_advanced_load_check_clr_xf,
+    gen_advanced_load_check_clr_ti,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_di,
+    gen_advanced_load_check_clr_di,
+
+    gen_speculation_check_bi,
+    gen_speculation_check_qi,
+    gen_speculation_check_hi,
+    gen_speculation_check_si,
+    gen_speculation_check_di,
+    gen_speculation_check_sf,
+    gen_speculation_check_df,
+    gen_speculation_check_xf,
+    gen_speculation_check_ti,
+    gen_speculation_check_di,
+    gen_speculation_check_di,
+    gen_speculation_check_di
+  };
+
+  extract_insn_cached (insn);
+
+  if (label)
+    {
+      gcc_assert (mutate_p || ia64_needs_block_p (insn));
+      op1 = label;
+    }
+  else
+    {
+      gcc_assert (!mutate_p && !ia64_needs_block_p (insn));
+      op1 = copy_rtx (recog_data.operand[1]);
+    }
+      
+  if (mutate_p)
+    /* INSN is ld.c.
+       Find the speculation check number by searching for original
+       speculative load in the RESOLVED_DEPS list of INSN.
+       As long as patterns are unique for each instruction, this can be
+       accomplished by matching ORIG_PAT fields.  */
+    {
+      rtx link;
+      int check_no = 0;
+      rtx orig_pat = ORIG_PAT (insn);
+
+      for (link = RESOLVED_DEPS (insn); link; link = XEXP (link, 1))
+	{
+	  rtx x = XEXP (link, 0);
+
+	  if (ORIG_PAT (x) == orig_pat)
+	    check_no = spec_check_no[INSN_UID (x)];
+	}
+      gcc_assert (check_no);
+
+      spec_check_no[INSN_UID (insn)] = (check_no
+					+ SPEC_GEN_CHECK_MUTATION_OFFSET);
+    }
+
+  check_pat = (gen_check[spec_check_no[INSN_UID (insn)] - 1]
+	       (copy_rtx (recog_data.operand[0]), op1));
+    
+  pat = PATTERN (insn);
+  if (GET_CODE (pat) == COND_EXEC)
+    check_pat = gen_rtx_COND_EXEC (VOIDmode, copy_rtx (COND_EXEC_TEST (pat)),
+				   check_pat);
+
+  return check_pat;
+}
+
+/* Return nonzero, if X is branchy recovery check.  */
+static int
+ia64_spec_check_p (rtx x)
+{
+  x = PATTERN (x);
+  if (GET_CODE (x) == COND_EXEC)
+    x = COND_EXEC_CODE (x);
+  if (GET_CODE (x) == SET)
+    return ia64_spec_check_src_p (SET_SRC (x));
+  return 0;
+}
+
+/* Return nonzero, if SRC belongs to recovery check.  */
+static int
+ia64_spec_check_src_p (rtx src)
+{
+  if (GET_CODE (src) == IF_THEN_ELSE)
+    {
+      rtx t;
+
+      t = XEXP (src, 0);
+      if (GET_CODE (t) == NE)
+	{
+	  t = XEXP (t, 0);	    
+
+	  if (GET_CODE (t) == UNSPEC)
+	    {
+	      int code;
+	      
+	      code = XINT (t, 1);
+	     
+	      if (code == UNSPEC_CHKACLR
+		  || code == UNSPEC_CHKS
+		  || code == UNSPEC_LDCCLR)
+		{
+		  gcc_assert (code != 0);
+		  return code;
+		}
+	    }
+	}
+    }
+  return 0;
+}
 
 
 /* The following page contains abstract data `bundle states' which are
@@ -6996,7 +7679,7 @@ get_next_important_insn (rtx insn, rtx tail)
    automata only says that we can issue an insn possibly inserting
    some nops before it and using some template.  Therefore insn
    bundling in this function is implemented by using DFA
-   (deterministic finite automata).  We follows all possible insn
+   (deterministic finite automata).  We follow all possible insn
    sequences by inserting 0-2 nops (that is what the NDFA describe for
    insn scheduling) before/after each insn being bundled.  We know the
    start of simulated processor cycle from insn scheduling (insn
@@ -7086,7 +7769,7 @@ bundling (FILE *dump, int verbose, rtx prev_head_insn, rtx tail)
 	      break;
 	    }
       }
-  /* Froward pass: generation of bundle states.  */
+  /* Forward pass: generation of bundle states.  */
   for (insn = get_next_important_insn (NEXT_INSN (prev_head_insn), tail);
        insn != NULL_RTX;
        insn = next_insn)
@@ -7619,11 +8302,27 @@ ia64_ld_address_bypass_p (rtx producer, rtx consumer)
   gcc_assert (src);
   mem = SET_SRC (src);
   gcc_assert (mem);
+ 
   if (GET_CODE (mem) == UNSPEC && XVECLEN (mem, 0) > 0)
     mem = XVECEXP (mem, 0, 0);
+  else if (GET_CODE (mem) == IF_THEN_ELSE)
+    /* ??? Is this bypass neccessary for ld.c?  */
+    {
+      gcc_assert (XINT (XEXP (XEXP (mem, 0), 0), 1) == UNSPEC_LDCCLR);
+      mem = XEXP (mem, 1);
+    }
+     
   while (GET_CODE (mem) == SUBREG || GET_CODE (mem) == ZERO_EXTEND)
     mem = XEXP (mem, 0);
 
+  if (GET_CODE (mem) == UNSPEC)
+    {
+      int c = XINT (mem, 1);
+
+      gcc_assert (c == UNSPEC_LDA || c == UNSPEC_LDS || c == UNSPEC_LDSA);
+      mem = XVECEXP (mem, 0, 0);
+    }
+
   /* Note that LO_SUM is used for GOT loads.  */
   gcc_assert (GET_CODE (mem) == LO_SUM || GET_CODE (mem) == MEM);
 
@@ -7804,6 +8503,7 @@ ia64_reorg (void)
 	  free (clocks);
 	}
       free (stops_p);
+      stops_p = NULL;
       emit_insn_group_barriers (dump_file);
 
       ia64_final_schedule = 0;
diff --git a/gcc/config/ia64/ia64.md b/gcc/config/ia64/ia64.md
index da7fe6d6674..f25ad76be16 100644
--- a/gcc/config/ia64/ia64.md
+++ b/gcc/config/ia64/ia64.md
@@ -81,6 +81,12 @@
    (UNSPEC_SHRP			29)
    (UNSPEC_COPYSIGN		30)
    (UNSPEC_VECT_EXTR		31)
+   (UNSPEC_LDA                  40)
+   (UNSPEC_LDS                  41)
+   (UNSPEC_LDSA                 42)
+   (UNSPEC_LDCCLR               43)
+   (UNSPEC_CHKACLR              45)
+   (UNSPEC_CHKS                 47)	
   ])
 
 (define_constants
@@ -124,18 +130,19 @@
 
 (define_attr "itanium_class" "unknown,ignore,stop_bit,br,fcmp,fcvtfx,fld,
 	fldp,fmac,fmisc,frar_i,frar_m,frbr,frfr,frpr,ialu,icmp,ilog,ishf,
-	ld,chk_s,long_i,mmalua,mmmul,mmshf,mmshfi,rse_m,scall,sem,stf,
+	ld,chk_s_i,chk_s_f,chk_a,long_i,mmalua,mmmul,mmshf,mmshfi,rse_m,scall,sem,stf,
         st,syst_m0, syst_m,tbit,toar_i,toar_m,tobr,tofr,topr,xmpy,xtd,nop,
         nop_b,nop_f,nop_i,nop_m,nop_x,lfetch,pre_cycle"
   (const_string "unknown"))
 
-;; chk_s has an I and an M form; use type A for convenience.
+;; chk_s_i has an I and an M form; use type A for convenience.
 (define_attr "type" "unknown,A,I,M,F,B,L,X,S"
   (cond [(eq_attr "itanium_class" "ld,st,fld,fldp,stf,sem,nop_m") (const_string "M")
 	 (eq_attr "itanium_class" "rse_m,syst_m,syst_m0") (const_string "M")
 	 (eq_attr "itanium_class" "frar_m,toar_m,frfr,tofr") (const_string "M")
 	 (eq_attr "itanium_class" "lfetch") (const_string "M")
-	 (eq_attr "itanium_class" "chk_s,ialu,icmp,ilog,mmalua")
+         (eq_attr "itanium_class" "chk_s_f,chk_a") (const_string "M")
+	 (eq_attr "itanium_class" "chk_s_i,ialu,icmp,ilog,mmalua")
 	   (const_string "A")
 	 (eq_attr "itanium_class" "fmisc,fmac,fcmp,xmpy") (const_string "F")
 	 (eq_attr "itanium_class" "fcvtfx,nop_f") (const_string "F")
@@ -170,6 +177,12 @@
 ;; when we have full intrinsics support.
 
 (define_attr "first_insn" "no,yes" (const_string "no"))
+
+(define_attr "data_speculative" "no,yes" (const_string "no"))
+
+(define_attr "control_speculative" "no,yes" (const_string "no"))
+
+(define_attr "check_load" "no,yes" (const_string "no"))
 
 ;; DFA descriptions of ia64 processors used for insn scheduling and
 ;; bundling.
@@ -372,6 +385,191 @@
 }
   [(set_attr "itanium_class" "ialu,ialu,long_i,ld,st,frfr,tofr,fmisc,fld,stf,frbr,tobr,frar_i,toar_i,frar_m,toar_m,frpr,topr")])
 
+(define_mode_macro MODE [BI QI HI SI DI SF DF XF TI])
+(define_mode_macro MODE_FOR_EXTEND [QI HI SI])
+
+(define_mode_attr output_a [
+  (BI "ld1.a %0 = %1%P1")
+  (QI "ld1.a %0 = %1%P1")
+  (HI "ld2.a %0 = %1%P1")
+  (SI "ld4.a %0 = %1%P1")
+  (DI
+   "@
+    ld8.a %0 = %1%P1
+    ldf8.a %0 = %1%P1")
+  (SF
+   "@
+    ldfs.a %0 = %1%P1
+    ld4.a %0 = %1%P1")
+  (DF
+   "@
+    ldfd.a %0 = %1%P1
+    ld8.a %0 = %1%P1")
+  (XF "ldfe.a %0 = %1%P1")
+  (TI "ldfp8.a %X0 = %1%P1")])
+
+(define_mode_attr output_s [
+  (BI "ld1.s %0 = %1%P1")
+  (QI "ld1.s %0 = %1%P1")
+  (HI "ld2.s %0 = %1%P1")
+  (SI "ld4.s %0 = %1%P1")
+  (DI
+   "@
+    ld8.s %0 = %1%P1
+    ldf8.s %0 = %1%P1")
+  (SF
+   "@
+    ldfs.s %0 = %1%P1
+    ld4.s %0 = %1%P1")
+  (DF
+   "@
+    ldfd.s %0 = %1%P1
+    ld8.s %0 = %1%P1")
+  (XF "ldfe.s %0 = %1%P1")
+  (TI "ldfp8.s %X0 = %1%P1")])
+
+(define_mode_attr output_sa [
+  (BI "ld1.sa %0 = %1%P1")
+  (QI "ld1.sa %0 = %1%P1")
+  (HI "ld2.sa %0 = %1%P1")
+  (SI "ld4.sa %0 = %1%P1")
+  (DI
+   "@
+    ld8.sa %0 = %1%P1
+    ldf8.sa %0 = %1%P1")
+  (SF
+   "@
+    ldfs.sa %0 = %1%P1
+    ld4.sa %0 = %1%P1")
+  (DF
+   "@
+    ldfd.sa %0 = %1%P1
+    ld8.sa %0 = %1%P1")
+  (XF "ldfe.sa %0 = %1%P1")
+  (TI "ldfp8.sa %X0 = %1%P1")])
+
+(define_mode_attr output_c_clr [
+  (BI "ld1.c.clr%O1 %0 = %1%P1")
+  (QI "ld1.c.clr%O1 %0 = %1%P1")
+  (HI "ld2.c.clr%O1 %0 = %1%P1")
+  (SI "ld4.c.clr%O1 %0 = %1%P1")
+  (DI
+   "@
+    ld8.c.clr%O1 %0 = %1%P1
+    ldf8.c.clr %0 = %1%P1")
+  (SF
+   "@
+    ldfs.c.clr %0 = %1%P1
+    ld4.c.clr%O1 %0 = %1%P1")
+  (DF
+   "@
+    ldfd.c.clr %0 = %1%P1
+    ld8.c.clr%O1 %0 = %1%P1")
+  (XF "ldfe.c.clr %0 = %1%P1")
+  (TI "ldfp8.c.clr %X0 = %1%P1")])
+
+(define_mode_attr ld_reg_constr [(BI "=*r") (QI "=r") (HI "=r") (SI "=r") (DI "=r,*f") (SF "=f,*r") (DF "=f,*r") (XF "=f") (TI "=*x")])
+(define_mode_attr ldc_reg_constr [(BI "+*r") (QI "+r") (HI "+r") (SI "+r") (DI "+r,*f") (SF "+f,*r") (DF "+f,*r") (XF "+f") (TI "+*x")])
+(define_mode_attr chk_reg_constr [(BI "*r") (QI "r") (HI "r") (SI "r") (DI "r,*f") (SF "f,*r") (DF "f,*r") (XF "f") (TI "*x")])
+
+(define_mode_attr mem_constr [(BI "*m") (QI "m") (HI "m") (SI "m") (DI "m,Q") (SF "Q,m") (DF "Q,m") (XF "m") (TI "Q")])
+
+(define_mode_attr reg_pred_prefix [(BI "gr") (QI "gr") (HI "gr") (SI "gr") (DI "grfr") (SF "grfr") (DF "grfr") (XF "fr") (TI "fr")])
+
+(define_mode_attr ld_class [(BI "ld") (QI "ld") (HI "ld") (SI "ld") (DI "ld,fld") (SF "fld,ld") (DF "fld,ld") (XF "fld") (TI "fldp")])
+(define_mode_attr chka_class [(BI "chk_a") (QI "chk_a") (HI "chk_a") (SI "chk_a") (DI "chk_a,chk_a") (SF "chk_a,chk_a") (DF "chk_a,chk_a") (XF "chk_a") (TI "chk_a")])
+(define_mode_attr chks_class [(BI "chk_s_i") (QI "chk_s_i") (HI "chk_s_i") (SI "chk_s_i") (DI "chk_s_i,chk_s_f") (SF "chk_s_f,chk_s_i") (DF "chk_s_f,chk_s_i") (XF "chk_s_f") (TI "chk_s_i")])
+
+(define_mode_attr attr_yes [(BI "yes") (QI "yes") (HI "yes") (SI "yes") (DI "yes,yes") (SF "yes,yes") (DF "yes,yes") (XF "yes") (TI "yes")])
+
+(define_insn "mov<mode>_advanced"
+  [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ld_reg_constr>")
+	(unspec:MODE [(match_operand:MODE 1 "memory_operand" "<mem_constr>")] UNSPEC_LDA))]
+  "ia64_move_ok (operands[0], operands[1])"
+  "<output_a>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "data_speculative" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_advanced"
+  [(set (match_operand:DI 0 "gr_register_operand" "=r")
+	(zero_extend:DI (unspec:MODE_FOR_EXTEND [(match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>")] UNSPEC_LDA)))]
+  ""
+  "<output_a>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "data_speculative" "<attr_yes>")])
+
+(define_insn "mov<mode>_speculative"
+  [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ld_reg_constr>")
+	(unspec:MODE [(match_operand:MODE 1 "memory_operand" "<mem_constr>")] UNSPEC_LDS))]
+  "ia64_move_ok (operands[0], operands[1])"
+  "<output_s>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_speculative"
+  [(set (match_operand:DI 0 "gr_register_operand" "=r")
+	(zero_extend:DI (unspec:MODE_FOR_EXTEND [(match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>")] UNSPEC_LDS)))]
+  ""
+  "<output_s>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "mov<mode>_speculative_advanced"
+  [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ld_reg_constr>")
+	(unspec:MODE [(match_operand:MODE 1 "memory_operand" "<mem_constr>")] UNSPEC_LDSA))]
+  "ia64_move_ok (operands[0], operands[1])"
+  "<output_sa>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "data_speculative" "<attr_yes>")
+   (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_speculative_advanced"
+  [(set (match_operand:DI 0 "gr_register_operand" "=r")
+	(zero_extend:DI (unspec:MODE_FOR_EXTEND [(match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>")] UNSPEC_LDSA)))]
+  ""
+  "<output_sa>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "data_speculative" "<attr_yes>")
+   (set_attr "control_speculative" "<attr_yes>")])
+
+(define_insn "mov<mode>_clr"
+  [(set (match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<ldc_reg_constr>")
+	(if_then_else:MODE (ne (unspec [(match_dup 0)] UNSPEC_LDCCLR) (const_int 0))
+			   (match_operand:MODE 1 "memory_operand" "<mem_constr>")
+			   (match_dup 0)))]
+  "ia64_move_ok (operands[0], operands[1])"
+  "<output_c_clr>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "check_load" "<attr_yes>")])
+
+(define_insn "zero_extend<mode>di2_clr"
+  [(set (match_operand:DI 0 "gr_register_operand" "+r")
+	(if_then_else:DI (ne (unspec [(match_dup 0)] UNSPEC_LDCCLR) (const_int 0))
+			 (zero_extend:DI (match_operand:MODE_FOR_EXTEND 1 "memory_operand" "<mem_constr>"))
+			 (match_dup 0)))]
+  ""
+  "<output_c_clr>"
+  [(set_attr "itanium_class" "<ld_class>")
+   (set_attr "check_load" "<attr_yes>")])
+
+(define_insn "advanced_load_check_clr_<mode>"
+  [(set (pc)
+        (if_then_else (ne (unspec [(match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<chk_reg_constr>")] UNSPEC_CHKACLR) (const_int 0))
+                      (pc)
+                      (label_ref (match_operand 1 "" ""))))]
+  ""
+  "chk.a.clr %0, %l1"
+  [(set_attr "itanium_class" "<chka_class>")])
+
+(define_insn "speculation_check_<mode>"
+  [(set (pc) 
+        (if_then_else (ne (unspec [(match_operand:MODE 0 "<reg_pred_prefix>_register_operand" "<chk_reg_constr>")] UNSPEC_CHKS) (const_int 0))
+                      (pc)
+                      (label_ref (match_operand 1 "" ""))))]
+  ""
+  "chk.s %0, %l1"
+  [(set_attr "itanium_class" "<chks_class>")])
+
 (define_split
   [(set (match_operand 0 "register_operand" "")
 	(match_operand 1 "symbolic_operand" ""))]
@@ -6114,7 +6312,7 @@
   [(trap_if (const_int 1) (match_operand 0 "const_int_operand" ""))]
   ""
   "break %0"
-  [(set_attr "itanium_class" "chk_s")])
+  [(set_attr "itanium_class" "chk_s_i")])
 
 (define_expand "conditional_trap"
   [(trap_if (match_operand 0 "" "") (match_operand 1 "" ""))]
@@ -6130,7 +6328,7 @@
 	    (match_operand 2 "const_int_operand" ""))]
   ""
   "(%J0) break %2"
-  [(set_attr "itanium_class" "chk_s")
+  [(set_attr "itanium_class" "chk_s_i")
    (set_attr "predicable" "no")])
 
 (define_insn "break_f"
diff --git a/gcc/config/ia64/ia64.opt b/gcc/config/ia64/ia64.opt
index 25a2cc6a91b..1166020b0b6 100644
--- a/gcc/config/ia64/ia64.opt
+++ b/gcc/config/ia64/ia64.opt
@@ -96,3 +96,50 @@ mtune=
 Target RejectNegative Joined
 Schedule code for given CPU
 
+msched-br-data-spec
+Target Report Var(mflag_sched_br_data_spec) Init(0)
+Use data speculation before reload
+
+msched-ar-data-spec
+Target Report Var(mflag_sched_ar_data_spec) Init(1)
+Use data speculation after reload
+
+msched-control-spec
+Target Report Var(mflag_sched_control_spec) Init(0)
+Use control speculation
+
+msched-br-in-data-spec
+Target Report Var(mflag_sched_br_in_data_spec) Init(1)
+Use in block data speculation before reload
+
+msched-ar-in-data-spec
+Target Report Var(mflag_sched_ar_in_data_spec) Init(1)
+Use in block data speculation after reload
+
+msched-in-control-spec
+Target Report Var(mflag_sched_in_control_spec) Init(1)
+Use in block control speculation
+
+msched-ldc
+Target Report Var(mflag_sched_ldc) Init(1)
+Use simple data speculation check
+
+msched-control-ldc
+Target Report Var(mflag_control_ldc) Init(0)
+Use simple data speculation check for control speculation
+
+msched-spec-verbose
+Common Report Var(mflag_sched_spec_verbose) Init(0)
+Print information about speculative motions.
+
+msched-prefer-non-data-spec-insns
+Common Report Var(mflag_sched_prefer_non_data_spec_insns) Init(0)
+If set, data speculative instructions will be choosen for schedule only if there are no other choices at the moment 
+
+msched-prefer-non-control-spec-insns
+Common Report Var(mflag_sched_prefer_non_control_spec_insns) Init(0)
+If set, control speculative instructions will be choosen for schedule only if there are no other choices at the moment 
+
+msched-count-spec-in-critical-path
+Common Report Var(mflag_sched_count_spec_in_critical_path) Init(0)
+Count speculative dependencies while calculating priority of instructions
diff --git a/gcc/config/ia64/itanium1.md b/gcc/config/ia64/itanium1.md
index e93475ada7e..f9b7f91a9cd 100644
--- a/gcc/config/ia64/itanium1.md
+++ b/gcc/config/ia64/itanium1.md
@@ -525,14 +525,29 @@
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "fcvtfx"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "1_F")
+
 (define_insn_reservation "1_fld"     9
-  (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "fld"))
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fld"))
+	    (eq_attr "check_load" "no"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_fldc"    0
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fld"))
+	    (eq_attr "check_load" "yes"))
+       (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+
 (define_insn_reservation "1_fldp"    9
-  (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "fldp"))
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "no"))
+       (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_fldpc"   0
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "yes"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+
 (define_insn_reservation "1_fmac"    5
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "fmac"))
@@ -604,8 +619,14 @@
        (eq (symbol_ref "bundling_p") (const_int 0)))
     "1_I+1_not_ui1")
 (define_insn_reservation "1_ld"      2
-  (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "ld"))
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "ld"))
+	    (eq_attr "check_load" "no"))
+       (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_ldc"     0
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "ld"))
+	    (eq_attr "check_load" "yes"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
 (define_insn_reservation "1_long_i"  1
   (and (and (eq_attr "cpu" "itanium")
@@ -696,10 +717,19 @@
             (eq_attr "itanium_class" "xtd"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "1_I")
 
-(define_insn_reservation "1_chk_s"   0
+(define_insn_reservation "1_chk_s_i" 0
   (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "chk_s"))
+            (eq_attr "itanium_class" "chk_s_i"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "1_A")
+(define_insn_reservation "1_chk_s_f" 0
+  (and (and (eq_attr "cpu" "itanium")
+            (eq_attr "itanium_class" "chk_s_f"))
+       (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+(define_insn_reservation "1_chk_a"   0
+  (and (and (eq_attr "cpu" "itanium")
+            (eq_attr "itanium_class" "chk_a"))
+       (eq (symbol_ref "bundling_p") (const_int 0))) "1_M")
+
 (define_insn_reservation "1_lfetch"  0
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "lfetch"))
@@ -943,15 +973,15 @@
 (define_bypass  2 "1_ilog,1_xtd" "1_ld"  "ia64_ld_address_bypass_p")
 (define_bypass  2 "1_ilog,1_xtd" "1_st"  "ia64_st_address_bypass_p")
 
-(define_bypass  3 "1_ld" "1_mmmul,1_mmshf")
+(define_bypass  3 "1_ld,1_ldc" "1_mmmul,1_mmshf")
 (define_bypass  3 "1_ld" "1_ld"  "ia64_ld_address_bypass_p")
 (define_bypass  3 "1_ld" "1_st"  "ia64_st_address_bypass_p")
 
 ;; Intel docs say only LD, ST, IALU, ILOG, ISHF consumers have latency 4,
 ;;      but HP engineers say any non-MM operation.
 (define_bypass  4 "1_mmmul,1_mmshf,1_mmalua"
-     "1_br,1_fcmp,1_fcvtfx,1_fld,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
-      1_frbr,1_frfr,1_frpr,1_ialu,1_icmp,1_ilog,1_ishf,1_ld,1_chk_s,\
+     "1_br,1_fcmp,1_fcvtfx,1_fld,1_fldc,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
+      1_frbr,1_frfr,1_frpr,1_ialu,1_icmp,1_ilog,1_ishf,1_ld,1_ldc,1_chk_s_i,1_chk_s_f,1_chk_a,\
       1_long_i,1_rse_m,1_sem,1_stf,1_st,1_syst_m0,1_syst_m,\
       1_tbit,1_toar_i,1_toar_m,1_tobr,1_tofr,1_topr,1_xmpy,1_xtd")
 
@@ -965,15 +995,15 @@
 (define_bypass  8 "1_fmisc,1_fcvtfx,1_fmac,1_xmpy"  "1_stf")
 
 ;; We don't use here fcmp because scall may be predicated.
-(define_bypass  0 "1_fcvtfx,1_fld,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
+(define_bypass  0 "1_fcvtfx,1_fld,1_fldc,1_fmac,1_fmisc,1_frar_i,1_frar_m,\
                    1_frbr,1_frfr,1_frpr,1_ialu,1_ialu_addr,1_ilog,1_ishf,\
-	           1_ld,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,1_toar_m,\
-		   1_tofr,1_xmpy,1_xtd" "1_scall")
+	           1_ld,1_ldc,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,\
+                   1_toar_m,1_tofr,1_xmpy,1_xtd" "1_scall")
 
 (define_bypass  0 "1_unknown,1_ignore,1_stop_bit,1_br,1_fcmp,1_fcvtfx,\
-                   1_fld,1_fmac,1_fmisc,1_frar_i,1_frar_m,1_frbr,1_frfr,\
-                   1_frpr,1_ialu,1_ialu_addr,1_icmp,1_ilog,1_ishf,1_ld,\
-                   1_chk_s,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,1_nop,\
+                   1_fld,1_fldc,1_fmac,1_fmisc,1_frar_i,1_frar_m,1_frbr,1_frfr,\
+                   1_frpr,1_ialu,1_ialu_addr,1_icmp,1_ilog,1_ishf,1_ld,1_ldc,\
+                   1_chk_s_i,1_chk_s_f,1_chk_a,1_long_i,1_mmalua,1_mmmul,1_mmshf,1_mmshfi,1_nop,\
                    1_nop_b,1_nop_f,1_nop_i,1_nop_m,1_nop_x,1_rse_m,1_scall,\
                    1_sem,1_stf,1_st,1_syst_m0,1_syst_m,1_tbit,1_toar_i,\
                    1_toar_m,1_tobr,1_tofr,1_topr,1_xmpy,1_xtd,1_lfetch"
@@ -1407,14 +1437,29 @@
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "fcvtfx"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "1b_F")
+
 (define_insn_reservation "1b_fld"     9
-  (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "fld"))
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fld"))
+	    (eq_attr "check_load" "no"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_fldc"    0
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fld"))
+	    (eq_attr "check_load" "yes"))
+       (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
 (define_insn_reservation "1b_fldp"    9
-  (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "fldp"))
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "no"))
+       (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_fldpc"   0
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "yes"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
 (define_insn_reservation "1b_fmac"    5
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "fmac"))
@@ -1480,10 +1525,18 @@
             (eq_attr "itanium_class" "ishf"))
        (ne (symbol_ref "bundling_p") (const_int 0)))
   "1b_I+1b_not_ui1")
+
 (define_insn_reservation "1b_ld"      2
-  (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "ld"))
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "ld"))
+	    (eq_attr "check_load" "no"))
+       (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_ldc"     0
+  (and (and (and (eq_attr "cpu" "itanium")
+		 (eq_attr "itanium_class" "ld"))
+	    (eq_attr "check_load" "yes"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
 (define_insn_reservation "1b_long_i"  1
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "long_i"))
@@ -1566,10 +1619,20 @@
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "xtd"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "1b_I")
-(define_insn_reservation "1b_chk_s"   0
+
+(define_insn_reservation "1b_chk_s_i" 0
   (and (and (eq_attr "cpu" "itanium")
-            (eq_attr "itanium_class" "chk_s"))
+            (eq_attr "itanium_class" "chk_s_i"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "1b_A")
+(define_insn_reservation "1b_chk_s_f" 0
+  (and (and (eq_attr "cpu" "itanium")
+            (eq_attr "itanium_class" "chk_s_f"))
+       (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+(define_insn_reservation "1b_chk_a"   0
+  (and (and (eq_attr "cpu" "itanium")
+            (eq_attr "itanium_class" "chk_a"))
+       (ne (symbol_ref "bundling_p") (const_int 0))) "1b_M")
+
 (define_insn_reservation "1b_lfetch"  0
   (and (and (eq_attr "cpu" "itanium")
             (eq_attr "itanium_class" "lfetch"))
diff --git a/gcc/config/ia64/itanium2.md b/gcc/config/ia64/itanium2.md
index f16d185e52b..a10971df7de 100644
--- a/gcc/config/ia64/itanium2.md
+++ b/gcc/config/ia64/itanium2.md
@@ -731,14 +731,38 @@
             (eq_attr "itanium_class" "fcvtfx"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "2_F")
 (define_insn_reservation "2_fld"     6
-  (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "fld"))
-       (eq (symbol_ref "bundling_p") (const_int 0))) "2_M")
+  (and (and (and (and (eq_attr "cpu" "itanium2")
+                      (eq_attr "itanium_class" "fld"))
+                 (eq_attr "data_speculative" "no"))
+            (eq_attr "check_load" "no"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M")
+(define_insn_reservation "2_flda"    6
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "fld"))
+            (eq_attr "data_speculative" "yes"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M_only_um01")
+(define_insn_reservation "2_fldc"    0
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "fld"))
+            (eq_attr "check_load" "yes"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M_only_um01")
+
 (define_insn_reservation "2_fldp"    6
-  (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "fldp"))
+  (and (and (and (eq_attr "cpu" "itanium2")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "no"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M_only_um01")
+(define_insn_reservation "2_fldpc"   0
+  (and (and (and (eq_attr "cpu" "itanium2")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "yes"))
        (eq (symbol_ref "bundling_p") (const_int 0)))
   "2_M_only_um01")
+
 (define_insn_reservation "2_fmac"    4
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "fmac"))
@@ -801,11 +825,19 @@
             (eq_attr "itanium_class" "ishf"))
        (eq (symbol_ref "bundling_p") (const_int 0)))
   "2_I+2_only_ui0")
+
 (define_insn_reservation "2_ld"      1
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "ld"))
+            (eq_attr "check_load" "no"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M_only_um01")
+(define_insn_reservation "2_ldc"     0
   (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "ld"))
+            (eq_attr "check_load" "yes"))
        (eq (symbol_ref "bundling_p") (const_int 0)))
   "2_M_only_um01")
+
 (define_insn_reservation "2_long_i"  1
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "long_i"))
@@ -909,11 +941,22 @@
             (eq_attr "itanium_class" "xtd"))
        (eq (symbol_ref "bundling_p") (const_int 0))) "2_I")
 
-(define_insn_reservation "2_chk_s"   0
+(define_insn_reservation "2_chk_s_i" 0
   (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "chk_s"))
+            (eq_attr "itanium_class" "chk_s_i"))
        (eq (symbol_ref "bundling_p") (const_int 0)))
   "2_I|2_M_only_um23")
+(define_insn_reservation "2_chk_s_f" 0
+  (and (and (eq_attr "cpu" "itanium2")
+            (eq_attr "itanium_class" "chk_s_f"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M_only_um23")
+(define_insn_reservation "2_chk_a"   0
+  (and (and (eq_attr "cpu" "itanium2")
+            (eq_attr "itanium_class" "chk_a"))
+       (eq (symbol_ref "bundling_p") (const_int 0)))
+  "2_M_only_um01")
+
 (define_insn_reservation "2_lfetch"  0
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "lfetch"))
@@ -1025,23 +1068,23 @@
 (define_bypass  0 "2_tbit" "2_br,2_scall")
 (define_bypass  2 "2_ld" "2_ld"  "ia64_ld_address_bypass_p")
 (define_bypass  2 "2_ld" "2_st"  "ia64_st_address_bypass_p")
-(define_bypass  2 "2_ld" "2_mmalua,2_mmmul,2_mmshf")
+(define_bypass  2 "2_ld,2_ldc" "2_mmalua,2_mmmul,2_mmshf")
 (define_bypass  3 "2_ilog" "2_mmalua,2_mmmul,2_mmshf")
 (define_bypass  3 "2_ialu" "2_mmalua,2_mmmul,2_mmshf")
-(define_bypass  3 "2_mmalua,2_mmmul,2_mmshf" "2_ialu,2_ilog,2_ishf,2_st,2_ld")
+(define_bypass  3 "2_mmalua,2_mmmul,2_mmshf" "2_ialu,2_ilog,2_ishf,2_st,2_ld,2_ldc")
 (define_bypass  6 "2_tofr"  "2_frfr,2_stf")
 (define_bypass  7 "2_fmac"  "2_frfr,2_stf")
 
 ;; We don't use here fcmp because scall may be predicated.
-(define_bypass  0 "2_fcvtfx,2_fld,2_fmac,2_fmisc,2_frar_i,2_frar_m,\
-                   2_frbr,2_frfr,2_frpr,2_ialu,2_ilog,2_ishf,2_ld,2_long_i,\
+(define_bypass  0 "2_fcvtfx,2_fld,2_flda,2_fldc,2_fmac,2_fmisc,2_frar_i,2_frar_m,\
+                   2_frbr,2_frfr,2_frpr,2_ialu,2_ilog,2_ishf,2_ld,2_ldc,2_long_i,\
                    2_mmalua,2_mmmul,2_mmshf,2_mmshfi,2_toar_m,2_tofr,\
 		   2_xmpy,2_xtd"
                   "2_scall")
 
-(define_bypass  0 "2_unknown,2_ignore,2_stop_bit,2_br,2_fcmp,2_fcvtfx,2_fld,\
+(define_bypass  0 "2_unknown,2_ignore,2_stop_bit,2_br,2_fcmp,2_fcvtfx,2_fld,2_flda,2_fldc,\
                    2_fmac,2_fmisc,2_frar_i,2_frar_m,2_frbr,2_frfr,2_frpr,\
-                   2_ialu,2_icmp,2_ilog,2_ishf,2_ld,2_chk_s,2_long_i,\
+                   2_ialu,2_icmp,2_ilog,2_ishf,2_ld,2_ldc,2_chk_s_i,2_chk_s_f,2_chk_a,2_long_i,\
 		   2_mmalua,2_mmmul,2_mmshf,2_mmshfi,2_nop,2_nop_b,2_nop_f,\
                    2_nop_i,2_nop_m,2_nop_x,2_rse_m,2_scall,2_sem,2_stf,2_st,\
                    2_syst_m0,2_syst_m,2_tbit,2_toar_i,2_toar_m,2_tobr,2_tofr,\
@@ -1543,14 +1586,38 @@
             (eq_attr "itanium_class" "fcvtfx"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "2b_F")
 (define_insn_reservation "2b_fld"     6
-  (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "fld"))
-       (ne (symbol_ref "bundling_p") (const_int 0))) "2b_M")
+  (and (and (and (and (eq_attr "cpu" "itanium2")
+                      (eq_attr "itanium_class" "fld"))
+                 (eq_attr "data_speculative" "no"))
+            (eq_attr "check_load" "no"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M")
+(define_insn_reservation "2b_flda"    6
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "fld"))
+            (eq_attr "data_speculative" "yes"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M_only_um01")
+(define_insn_reservation "2b_fldc"    0
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "fld"))
+            (eq_attr "check_load" "yes"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M_only_um01")
+
 (define_insn_reservation "2b_fldp"    6
-  (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "fldp"))
+  (and (and (and (eq_attr "cpu" "itanium2")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "no"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M_only_um01")
+(define_insn_reservation "2b_fldpc"   0
+  (and (and (and (eq_attr "cpu" "itanium2")
+		 (eq_attr "itanium_class" "fldp"))
+	    (eq_attr "check_load" "yes"))
        (ne (symbol_ref "bundling_p") (const_int 0)))
   "2b_M_only_um01")
+
 (define_insn_reservation "2b_fmac"    4
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "fmac"))
@@ -1611,11 +1678,20 @@
             (eq_attr "itanium_class" "ishf"))
        (ne (symbol_ref "bundling_p") (const_int 0)))
   "2b_I+2b_only_ui0")
+
 (define_insn_reservation "2b_ld"      1
-  (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "ld"))
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "ld"))
+            (eq_attr "check_load" "no"))
        (ne (symbol_ref "bundling_p") (const_int 0)))
   "2b_M_only_um01")
+(define_insn_reservation "2b_ldc"     0
+  (and (and (and (eq_attr "cpu" "itanium2")
+                 (eq_attr "itanium_class" "ld"))
+            (eq_attr "check_load" "yes"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M_only_um01")
+
 (define_insn_reservation "2b_long_i"  1
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "long_i"))
@@ -1714,11 +1790,23 @@
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "xtd"))
        (ne (symbol_ref "bundling_p") (const_int 0))) "2b_I")
-(define_insn_reservation "2b_chk_s"   0
+
+(define_insn_reservation "2b_chk_s_i" 0
   (and (and (eq_attr "cpu" "itanium2")
-            (eq_attr "itanium_class" "chk_s"))
+            (eq_attr "itanium_class" "chk_s_i"))
        (ne (symbol_ref "bundling_p") (const_int 0)))
   "2b_I|2b_M_only_um23")
+(define_insn_reservation "2b_chk_s_f" 0
+  (and (and (eq_attr "cpu" "itanium2")
+            (eq_attr "itanium_class" "chk_s_f"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M_only_um23")
+(define_insn_reservation "2b_chk_a"   0
+  (and (and (eq_attr "cpu" "itanium2")
+            (eq_attr "itanium_class" "chk_a"))
+       (ne (symbol_ref "bundling_p") (const_int 0)))
+  "2b_M_only_um01")
+
 (define_insn_reservation "2b_lfetch"  0
   (and (and (eq_attr "cpu" "itanium2")
             (eq_attr "itanium_class" "lfetch"))
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index f037f4b0991..533841931b7 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -549,7 +549,13 @@ Objective-C and Objective-C++ Dialects}.
 -minline-sqrt-min-latency -minline-sqrt-max-throughput @gol
 -mno-dwarf2-asm -mearly-stop-bits @gol
 -mfixed-range=@var{register-range} -mtls-size=@var{tls-size} @gol
--mtune=@var{cpu-type} -mt -pthread -milp32 -mlp64}
+-mtune=@var{cpu-type} -mt -pthread -milp32 -mlp64 @gol
+-mno-sched-br-data-spec -msched-ar-data-spec -mno-sched-control-spec @gol
+-msched-br-in-data-spec -msched-ar-in-data-spec -msched-in-control-spec @gol
+-msched-ldc -mno-sched-control-ldc -mno-sched-spec-verbose @gol
+-mno-sched-prefer-non-data-spec-insns @gol
+-mno-sched-prefer-non-control-spec-insns @gol
+-mno-sched-count-spec-in-critical-path}
 
 @emph{M32R/D Options}
 @gccoptlist{-m32r2 -m32rx -m32r @gol
@@ -9734,6 +9740,113 @@ The 32-bit environment sets int, long and pointer to 32 bits.
 The 64-bit environment sets int to 32 bits and long and pointer
 to 64 bits.  These are HP-UX specific flags.
 
+@item -mno-sched-br-data-spec
+@itemx -msched-br-data-spec
+@opindex -mno-sched-br-data-spec
+@opindex -msched-br-data-spec
+(Dis/En)able data speculative scheduling before reload.
+This will result in generation of the ld.a instructions and
+the corresponding check instructions (ld.c / chk.a).
+The default is 'disable'.
+
+@item -msched-ar-data-spec
+@itemx -mno-sched-ar-data-spec
+@opindex -msched-ar-data-spec
+@opindex -mno-sched-ar-data-spec
+(En/Dis)able data speculative scheduling after reload.
+This will result in generation of the ld.a instructions and
+the corresponding check instructions (ld.c / chk.a).
+The default is 'enable'.
+
+@item -mno-sched-control-spec
+@itemx -msched-control-spec
+@opindex -mno-sched-control-spec
+@opindex -msched-control-spec
+(Dis/En)able control speculative scheduling.  This feature is
+available only during region scheduling (i.e. before reload).
+This will result in generation of the ld.s instructions and
+the corresponding check instructions chk.s .
+The default is 'disable'.
+
+@item -msched-br-in-data-spec
+@itemx -mno-sched-br-in-data-spec
+@opindex -msched-br-in-data-spec
+@opindex -mno-sched-br-in-data-spec
+(En/Dis)able speculative scheduling of the instructions that
+are dependent on the data speculative loads before reload.
+This is effective only with @option{-msched-br-data-spec} enabled.
+The default is 'enable'.
+
+@item -msched-ar-in-data-spec
+@itemx -mno-sched-ar-in-data-spec
+@opindex -msched-ar-in-data-spec
+@opindex -mno-sched-ar-in-data-spec
+(En/Dis)able speculative scheduling of the instructions that
+are dependent on the data speculative loads after reload.
+This is effective only with @option{-msched-ar-data-spec} enabled.
+The default is 'enable'.
+
+@item -msched-in-control-spec
+@itemx -mno-sched-in-control-spec
+@opindex -msched-in-control-spec
+@opindex -mno-sched-in-control-spec
+(En/Dis)able speculative scheduling of the instructions that
+are dependent on the control speculative loads.
+This is effective only with @option{-msched-control-spec} enabled.
+The default is 'enable'.
+
+@item -msched-ldc
+@itemx -mno-sched-ldc
+@opindex -msched-ldc
+@opindex -mno-sched-ldc
+(En/Dis)able use of simple data speculation checks ld.c .
+If disabled, only chk.a instructions will be emitted to check
+data speculative loads.
+The default is 'enable'.
+
+@item -mno-sched-control-ldc
+@itemx -msched-control-ldc
+@opindex -mno-sched-control-ldc
+@opindex -msched-control-ldc 
+(Dis/En)able use of ld.c instructions to check control speculative loads.
+If enabled, in case of control speculative load with no speculatively
+scheduled dependent instructions this load will be emitted as ld.sa and
+ld.c will be used to check it.
+The default is 'disable'.
+
+@item -mno-sched-spec-verbose
+@itemx -msched-spec-verbose
+@opindex -mno-sched-spec-verbose
+@opindex -msched-spec-verbose
+(Dis/En)able printing of the information about speculative motions.
+
+@item -mno-sched-prefer-non-data-spec-insns
+@itemx -msched-prefer-non-data-spec-insns
+@opindex -mno-sched-prefer-non-data-spec-insns
+@opindex -msched-prefer-non-data-spec-insns
+If enabled, data speculative instructions will be choosen for schedule
+only if there are no other choices at the moment.  This will make
+the use of the data speculation much more conservative.
+The default is 'disable'.
+
+@item -mno-sched-prefer-non-control-spec-insns
+@itemx -msched-prefer-non-control-spec-insns
+@opindex -mno-sched-prefer-non-control-spec-insns
+@opindex -msched-prefer-non-control-spec-insns
+If enabled, control speculative instructions will be choosen for schedule
+only if there are no other choices at the moment.  This will make
+the use of the control speculation much more conservative.
+The default is 'disable'.
+
+@item -mno-sched-count-spec-in-critical-path
+@itemx -msched-count-spec-in-critical-path
+@opindex -mno-sched-count-spec-in-critical-path
+@opindex -msched-count-spec-in-critical-path
+If enabled, speculative depedencies will be considered during
+computation of the instructions priorities.  This will make the use of the
+speculation a bit more conservative.
+The default is 'disable'.
+
 @end table
 
 @node M32C Options