* doc/passes.texi (Tree-SSA passes): Document SLP pass.

	* tree-pass.h (pass_slp_vectorize): New pass.
	* params.h (SLP_MAX_INSNS_IN_BB): Define.
	* timevar.def (TV_TREE_SLP_VECTORIZATION): Define.
	* tree-vectorizer.c (timevar.h): Include.
	(user_vect_verbosity_level): Declare.
	(vect_location): Fix comment.
	(vect_set_verbosity_level): Update user_vect_verbosity_level
	instead of vect_verbosity_level.
	(vect_set_dump_settings): Add an argument. Ignore user defined
	verbosity if dump flags require higher level of verbosity. Print to
	stderr only for loop vectorization.
	(vectorize_loops): Update call to vect_set_dump_settings.
	(execute_vect_slp): New function.
	(gate_vect_slp): Likewise.
	(struct gimple_opt_pass pass_slp_vectorize): New.
	* tree-vectorizer.h (struct _bb_vec_info): Define along macros to
	access its members.
	(vec_info_for_bb): New function.
	(struct _stmt_vec_info): Add bb_vinfo and a macro for its access.
	(VECTORIZATION_ENABLED): New macro.
	(SLP_ENABLED, SLP_DISABLED): Likewise.
	(vect_is_simple_use): Add bb_vec_info argument.
	(new_stmt_vec_info, vect_analyze_data_ref_dependences,
	vect_analyze_data_refs_alignment, vect_verify_datarefs_alignment,
	vect_analyze_data_ref_accesses, vect_analyze_data_refs,
	vect_schedule_slp, vect_analyze_slp): Likewise.
	(vect_analyze_stmt): Add slp_tree argument.
	(find_bb_location): Declare.
	(vect_slp_analyze_bb, vect_slp_transform_bb): Likewise.
	* tree-vect-loop.c (new_loop_vec_info): Adjust function calls.
	(vect_analyze_loop_operations, vect_analyze_loop,
	get_initial_def_for_induction, vect_create_epilog_for_reduction,
	vect_finalize_reduction, vectorizable_reduction,
	vectorizable_live_operation, vect_transform_loop): Likewise.
	* tree-data-ref.c (dr_analyze_innermost): Update comment,
	skip evolution analysis if analyzing a basic block.
	(dr_analyze_indices): Likewise.
	(initialize_data_dependence_relation): Skip the test whether the
	object is invariant for basic blocks.
	(compute_all_dependences): Skip dependence analysis for data
	references in basic blocks.
	(find_data_references_in_stmt): Don't fail in case of invariant
	access in basic block.
	(find_data_references_in_bb): New function.
	(find_data_references_in_loop): Move code to
	find_data_references_in_bb    and add a call to it.
	(compute_data_dependences_for_bb): New function.
	* tree-data-ref.h (compute_data_dependences_for_bb): Declare.
	* tree-vect-data-refs.c (vect_check_interleaving): Adjust to the case
	that STEP is 0.
	(vect_analyze_data_ref_dependence): Check for interleaving in case of
	unknown dependence in basic block and fail in case of dependence in
	basic block.
	(vect_analyze_data_ref_dependences): Add bb_vinfo argument, get data
	dependence instances from either loop or basic block vectorization
	info.
	(vect_compute_data_ref_alignment): Check if it is loop vectorization
	before calling nested_in_vect_loop_p.
	(vect_compute_data_refs_alignment): Add bb_vinfo argument, get data
	dependence instances from either loop or basic block vectorization
	info.
	(vect_verify_datarefs_alignment): Likewise.
	(vect_enhance_data_refs_alignment): Adjust function calls.
	(vect_analyze_data_refs_alignment): Likewise.
	(vect_analyze_group_access): Fix printing. Skip different checks if
	DR_STEP is 0. Keep strided stores either in loop or basic block
	vectorization data structure. Fix indentation.
	(vect_analyze_data_ref_access): Fix comments, allow zero step in
	basic blocks.
	(vect_analyze_data_ref_accesses): Add bb_vinfo argument, get data
	dependence instances from either loop or basic block vectorization
	info.
	(vect_analyze_data_refs): Update comment. Call
	compute_data_dependences_for_bb to analyze basic blocks.
	(vect_create_addr_base_for_vector_ref): Check for outer loop only in
	case of loop vectorization. In case of basic block vectorization use
	data-ref itself   as  a base.
	(vect_create_data_ref_ptr): In case of basic block vectorization:
	don't advance the pointer, add new statements before the current
	statement.  Adjust function calls.
	(vect_supportable_dr_alignment): Support only aligned accesses in
	basic block vectorization.
	* common.opt (ftree-slp-vectorize): New flag.
	* tree-vect-patterns.c (widened_name_p): Adjust function calls.
	(vect_pattern_recog_1): Likewise.
	* tree-vect-stmts.c (process_use): Likewise.
	(vect_init_vector): Add new statements in the beginning of the basic
	block in case of basic block SLP.
	(vect_get_vec_def_for_operand): Adjust function calls.
	(vect_finish_stmt_generation): Likewise.
	(vectorizable_call): Add assert that it is loop vectorization, adjust
	function calls.
	(vectorizable_conversion, vectorizable_assignment): Likewise.
	(vectorizable_operation): In case of basic block SLP, take
	vectorization factor from statement's type and skip the relevance
	check. Adjust function calls.
	(vectorizable_type_demotion): Add assert that it is loop
	vectorization, adjust function calls.
	(vectorizable_type_promotion): Likewise.
	(vectorizable_store): Check for outer loop only in case of loop
	vectorization. Adjust function calls. For basic blocks, skip the
	relevance check and don't advance pointers.
	(vectorizable_load): Likewise.
	(vectorizable_condition): Add assert that it is loop vectorization,
	adjust function calls.
	(vect_analyze_stmt): Add argument. In case of basic block SLP, check
	that it is not reduction, get vector type, call only supported
	functions, skip loop    specific parts.
	(vect_transform_stmt): Check for outer loop only in case of loop
	vectorization.
	(new_stmt_vec_info): Add new argument and initialize bb_vinfo.
	(vect_is_simple_use): Fix comment, add new argument, fix conditions
	for external definition.
	* passes.c (pass_slp_vectorize): New pass.
	* tree-vect-slp.c (find_bb_location): New function.
	(vect_get_and_check_slp_defs): Add argument, adjust function calls,
	check for patterns only in loops.
	(vect_build_slp_tree): Add argument, adjust function calls, fail in
	case of multiple types in basic block SLP.
	(vect_mark_slp_stmts_relevant): New function.
	(vect_supported_load_permutation_p): Fix comment.
	(vect_analyze_slp_instance): Add argument. In case of basic block
	SLP, take vectorization factor from statement's type, check that
	unrolling factor is 1. Adjust function call. Save SLP instance in
	either loop or basic block vectorization structure. Return FALSE,
	if SLP failed.
	(vect_analyze_slp): Add argument. Get strided stores groups from
	either loop or basic block vectorization structure. Return FALSE
	if basic block SLP failed.
	(new_bb_vec_info): New function.
	(destroy_bb_vec_info, vect_slp_analyze_node_operations,
	vect_slp_analyze_operations, vect_slp_analyze_bb): Likewise.
	(vect_schedule_slp): Add argument. Get SLP instances from either
	loop or basic block vectorization structure. Set vectorization factor
	to be 1 for basic block SLP.
	(vect_slp_transform_bb): New function.
	* params.def (PARAM_SLP_MAX_INSNS_IN_BB): Define.



git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@147829 138bc75d-0d04-0410-961f-82ee72b054a4

1 files changed, 94 insertions, 35 deletions

diff --git a/gcc/tree-data-ref.c b/gcc/tree-data-ref.c
index 583d2c0fd9e..dc79e8a8bc4 100644
--- a/gcc/tree-data-ref.c
+++ b/gcc/tree-data-ref.c
@@ -668,8 +668,9 @@ canonicalize_base_object_address (tree addr)
   return build_fold_addr_expr (TREE_OPERAND (addr, 0));
 }
 
-/* Analyzes the behavior of the memory reference DR in the innermost loop that
-   contains it. Returns true if analysis succeed or false otherwise.  */
+/* Analyzes the behavior of the memory reference DR in the innermost loop or 
+   basic block that contains it. Returns true if analysis succeed or false
+   otherwise.  */
 
 bool
 dr_analyze_innermost (struct data_reference *dr)
@@ -683,6 +684,7 @@ dr_analyze_innermost (struct data_reference *dr)
   int punsignedp, pvolatilep;
   affine_iv base_iv, offset_iv;
   tree init, dinit, step;
+  bool in_loop = (loop && loop->num);
 
   if (dump_file && (dump_flags & TDF_DETAILS))
     fprintf (dump_file, "analyze_innermost: ");
@@ -699,13 +701,24 @@ dr_analyze_innermost (struct data_reference *dr)
     }
 
   base = build_fold_addr_expr (base);
-  if (!simple_iv (loop, loop_containing_stmt (stmt), base, &base_iv, false))
+  if (in_loop)
     {
-      if (dump_file && (dump_flags & TDF_DETAILS))
-	fprintf (dump_file, "failed: evolution of base is not affine.\n");
-      return false;
+      if (!simple_iv (loop, loop_containing_stmt (stmt), base, &base_iv, 
+                      false))
+        {
+          if (dump_file && (dump_flags & TDF_DETAILS))
+	    fprintf (dump_file, "failed: evolution of base is not affine.\n");
+          return false;
+        }
+    }
+  else
+    {
+      base_iv.base = base;
+      base_iv.step = ssize_int (0);
+      base_iv.no_overflow = true;
     }
-  if (!poffset)
+
+  if (!poffset || !in_loop)
     {
       offset_iv.base = ssize_int (0);
       offset_iv.step = ssize_int (0);
@@ -752,17 +765,23 @@ dr_analyze_indices (struct data_reference *dr, struct loop *nest)
   struct loop *loop = loop_containing_stmt (stmt);
   VEC (tree, heap) *access_fns = NULL;
   tree ref = unshare_expr (DR_REF (dr)), aref = ref, op;
-  tree base, off, access_fn;
-  basic_block before_loop = block_before_loop (nest);
-
+  tree base, off, access_fn = NULL_TREE;
+  basic_block before_loop = NULL;
+ 
+  if (nest)
+    before_loop = block_before_loop (nest);
+    
   while (handled_component_p (aref))
     {
       if (TREE_CODE (aref) == ARRAY_REF)
 	{
 	  op = TREE_OPERAND (aref, 1);
-	  access_fn = analyze_scalar_evolution (loop, op);
-	  access_fn = instantiate_scev (before_loop, loop, access_fn);
-	  VEC_safe_push (tree, heap, access_fns, access_fn);
+	  if (nest)
+	    {
+  	      access_fn = analyze_scalar_evolution (loop, op);
+	      access_fn = instantiate_scev (before_loop, loop, access_fn);
+	      VEC_safe_push (tree, heap, access_fns, access_fn);
+	    }
 
 	  TREE_OPERAND (aref, 1) = build_int_cst (TREE_TYPE (op), 0);
 	}
@@ -770,7 +789,7 @@ dr_analyze_indices (struct data_reference *dr, struct loop *nest)
       aref = TREE_OPERAND (aref, 0);
     }
 
-  if (INDIRECT_REF_P (aref))
+  if (nest && INDIRECT_REF_P (aref))
     {
       op = TREE_OPERAND (aref, 0);
       access_fn = analyze_scalar_evolution (loop, op);
@@ -1332,8 +1351,9 @@ initialize_data_dependence_relation (struct data_reference *a,
   /* If the base of the object is not invariant in the loop nest, we cannot
      analyze it.  TODO -- in fact, it would suffice to record that there may
      be arbitrary dependences in the loops where the base object varies.  */
-  if (!object_address_invariant_in_loop_p (VEC_index (loop_p, loop_nest, 0),
-					   DR_BASE_OBJECT (a)))
+  if (loop_nest 
+      && !object_address_invariant_in_loop_p (VEC_index (loop_p, loop_nest, 0),
+     					      DR_BASE_OBJECT (a)))
     {
       DDR_ARE_DEPENDENT (res) = chrec_dont_know;    
       return res;
@@ -4003,7 +4023,8 @@ compute_all_dependences (VEC (data_reference_p, heap) *datarefs,
 	{
 	  ddr = initialize_data_dependence_relation (a, b, loop_nest);
 	  VEC_safe_push (ddr_p, heap, *dependence_relations, ddr);
-	  compute_affine_dependence (ddr, VEC_index (loop_p, loop_nest, 0));
+          if (loop_nest)
+   	    compute_affine_dependence (ddr, VEC_index (loop_p, loop_nest, 0));
 	}
 
   if (compute_self_and_rr)
@@ -4110,9 +4131,10 @@ find_data_references_in_stmt (struct loop *nest, gimple stmt,
       dr = create_data_ref (nest, *ref->pos, stmt, ref->is_read);
       gcc_assert (dr != NULL);
   
-      /* FIXME -- data dependence analysis does not work correctly for objects with
-	 invariant addresses.  Let us fail here until the problem is fixed.  */
-      if (dr_address_invariant_p (dr))
+      /* FIXME -- data dependence analysis does not work correctly for objects 
+         with invariant addresses in loop nests.  Let us fail here until the
+	 problem is fixed.  */
+      if (dr_address_invariant_p (dr) && nest)
 	{
 	  free_data_ref (dr);
 	  if (dump_file && (dump_flags & TDF_DETAILS))
@@ -4129,6 +4151,33 @@ find_data_references_in_stmt (struct loop *nest, gimple stmt,
 
 /* Search the data references in LOOP, and record the information into
    DATAREFS.  Returns chrec_dont_know when failing to analyze a
+   difficult case, returns NULL_TREE otherwise.  */
+
+static tree
+find_data_references_in_bb (struct loop *loop, basic_block bb,
+                            VEC (data_reference_p, heap) **datarefs)
+{
+  gimple_stmt_iterator bsi;
+
+  for (bsi = gsi_start_bb (bb); !gsi_end_p (bsi); gsi_next (&bsi))
+    {
+      gimple stmt = gsi_stmt (bsi);
+
+      if (!find_data_references_in_stmt (loop, stmt, datarefs))
+        {
+          struct data_reference *res;
+          res = XCNEW (struct data_reference);
+          VEC_safe_push (data_reference_p, heap, *datarefs, res);
+
+          return chrec_dont_know;
+        }
+    }
+
+  return NULL_TREE;
+}
+
+/* Search the data references in LOOP, and record the information into
+   DATAREFS.  Returns chrec_dont_know when failing to analyze a
    difficult case, returns NULL_TREE otherwise.
 
    TODO: This function should be made smarter so that it can handle address
@@ -4140,7 +4189,6 @@ find_data_references_in_loop (struct loop *loop,
 {
   basic_block bb, *bbs;
   unsigned int i;
-  gimple_stmt_iterator bsi;
 
   bbs = get_loop_body_in_dom_order (loop);
 
@@ -4148,20 +4196,11 @@ find_data_references_in_loop (struct loop *loop,
     {
       bb = bbs[i];
 
-      for (bsi = gsi_start_bb (bb); !gsi_end_p (bsi); gsi_next (&bsi))
-	{
-	  gimple stmt = gsi_stmt (bsi);
-
-	  if (!find_data_references_in_stmt (loop, stmt, datarefs))
-	    {
-	      struct data_reference *res;
-	      res = XCNEW (struct data_reference);
-	      VEC_safe_push (data_reference_p, heap, *datarefs, res);
-
-	      free (bbs);
-	      return chrec_dont_know;
-	    }
-	}
+      if (find_data_references_in_bb (loop, bb, datarefs) == chrec_dont_know)
+        {
+          free (bbs);
+          return chrec_dont_know;
+        }
     }
   free (bbs);
 
@@ -4298,6 +4337,26 @@ compute_data_dependences_for_loop (struct loop *loop,
   return res;
 }
 
+/* Returns true when the data dependences for the basic block BB have been 
+   computed, false otherwise.
+   DATAREFS is initialized to all the array elements contained in this basic 
+   block, DEPENDENCE_RELATIONS contains the relations between the data
+   references. Compute read-read and self relations if
+   COMPUTE_SELF_AND_READ_READ_DEPENDENCES is TRUE.  */
+bool
+compute_data_dependences_for_bb (basic_block bb,
+                                 bool compute_self_and_read_read_dependences,
+                                 VEC (data_reference_p, heap) **datarefs,
+                                 VEC (ddr_p, heap) **dependence_relations)
+{
+  if (find_data_references_in_bb (NULL, bb, datarefs) == chrec_dont_know)
+    return false;
+
+  compute_all_dependences (*datarefs, dependence_relations, NULL,
+                           compute_self_and_read_read_dependences);
+  return true;
+}
+
 /* Entry point (for testing only).  Analyze all the data references
    and the dependence relations in LOOP.