[gcc]

2015-10-29  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* config/rs6000/rs6000.c (rs6000_init_libfuncs): Split libfunc
	setup into 3 functions: init_float128_ibm, init_float128_ieee, and
	rs6000_init_libfuncs. If -mfloat128, add IFmode functions for all
	of the traditional names that TFmode uses for handling IEEE
	extended double. If -mfloat128, add KFmode functions for all of
	the emulation functions. If -mabi=ieeelongdouble and -mfloat128,
	make TFmode use the same emulation functions as KFmode.
	(init_float128_ibm): Likewise.
	(init_float128_ieee): Likewise.
	(rs6000_generate_compare): For IEEE 128-bit floating point
	comparisons, call the unordered comparison function instead of the
	ordered comparison function.
	(rs6000_expand_float128_convert): Deal with operands that are
	memory operands. Restructure the code to use a switch statement on
	the mode. Add support for TFmode defaulting to either IBM extended
	double or IEEE 128-bit floating point. If the underlying types are
	the same, use a move instead of a conversion function.
	(TARGET_C_MODE_FOR_SUFFIX): Define 'q' and 'Q' as the suffix to
	use for IEEE 128-bit floating point constants with -mfloat128.
	(rs6000_c_mode_for_suffix): Likewise.
	(TARGET_INVALID_BINARY_OP): Do not allow inter-mixing of IEEE
	128-bit floating point with IBM extended double floating point.
	(rs6000_invalid_binary_op): Likewise.
	(rs6000_gen_le_vsx_permute): On little endian systems generate a
	ROTATE insn instead of VEC_SELECT for IEEE 128-bit floating point
	types that can go in vector registers.
	(chain_contains_only_swaps): Properly swap IEEE 128-bit floating
	point types that can go in vector registers on little endian
	PowerPC systems.
	(mark_swaps_for_removal): Likewise.
	(rs6000_analyze_swaps): Likewise.
	(rs6000_mangle_type): Use U10__float128 for IEEE 128-bit floating point.

	* config/rs6000/rs6000.md (FLOAT128_SFDFTF): Delete iterator,
	rework IEEE 128-bit floating point insns to deal with TFmode being
	either IBM extended double or IEEE 128-bit floating point.
	(IFKF): Likewise.
	(IBM128): Update iterator to add condition that the mode is IBM
	extended double.
	(IEEE128): New iterator for IEEE 128-bit floating point.
	(TFIFKF): Rename TFIFKF iterator to FLOAT128.
	(FLOAT128): Likewise.
	(signbit<mode>2): FLOAT128_IBM_P condition test moved into IBM128
	iterator.
	(neg<mode>2): Replace TFIFKF iterator with FLOAT128. Add support
	for TFmode being IEEE 128-bit floating point. Use IEEE128 iterator
	instead of hard coding TFmode or KFmode.
	(negtf2_internal): Likewise.
	(neg<mode>2_internal): Likewise.
	(abs<mode>2): Likewise.
	(abstf2_internal): Likewise.
	(abs<mode>2_internal): Likewise.
	(ieee_128bit_neg<mode>2): Likewise.
	(ieee_128bit_neg<mode>2_internal): Likewise.
	(ieee_128bit_abs<mode>2): Likewise.
	(ieee_128bit_abs<mode>2_internal): Likewise.
	(ieee_128bit_nabs<mode>2): Likewise.
	(ieee_128bit_nabs<mode>2_internal): Likewise.
	(extendiftf2): Add explicit conversions between 128-bit floating
	point types. Drop the old conversions that had become unwieldy.
	(extend<FLOAT128_SFDFTF:mode><IFKF:mode>2): Likewise.
	(extendifkf2): Likewise.
	(trunc<IFKF:mode><FLOAT128_SFDFTF:mode>2): Likewise.
	(extendtfkf2): Likewise.
	(fix_trunc<IFKF:mode><SDI:mode>2): Likewise.
	(trunciftf2): Likewise.
	(fixuns_trunc<IFKF:mode><SDI:mode>2): Likewise.
	(truncifkf2): Likewise.
	(float<SDI:mode><IFKF:mode>2): Likewise.
	(trunckftf2): Likewise.
	(floatuns<SDI:mode><IFKF:mode>2): Likewise.
	(trunctfif2): Likewise.
	(FP iterator): Allow TFmode to be IEEE 128-bit floating point.
	(extenddftf2): Rework 128-bit floating point conversions to
	properly handle -mabi=ieeelongdouble. Merge IFmode, TFmode, and
	KFmode expanders into one function.
	(extenddf<mode>2): Likewise.
	(extenddftf2_fprs): Likewise.
	(extenddf<mode>2_fprs): Likewise.
	(extenddftf2_vsx): Likewise.
	(extenddf<mode>2_vsx): Likewise.
	(extendsftf2): Likewise.
	(extendsf<mode>2): Likewise.
	(trunctfdf2): Likewise.
	(trunc<mode>df2): Likewise.
	(trunctfdf2_internal1): Likewise.
	(trunc<mode>df2_internal1): Likewise.
	(trunctfdf2_internal2): Likewise.
	(trunc<mode>df2_internal2): Likewise.
	(trunctfsf2): Likewise.
	(trunc<mode>sf2): Likewise.
	(trunctfsf2_fprs): Likewise.
	(trunc<mode>sf2_fprs): Likewise.
	(floatsit2f): Likewise.
	(floatsi<mode>2): Likewise.
	(fix_trunc_helper): Likewise.
	(fix_trunc_helper<mode>): Likewise.
	(fix_trunctfsi2): Likewise.
	(fix_trunc<mode>si2): Likewise.
	(fix_trunctfsi2_fprs): Likewise.
	(fix_trunc<mode>si2_fprs): Likewise.
	(fix_trunctfsi2_internal): Likewise.
	(fix_trunc<mode>si2_internal): Likewise.
	(fix_trunctfdi2): Likewise.
	(fix_trunc<mode>di2): Likewise.
	(fixuns_trunctf<mode>2): Likewise.
	(fixuns_trunc<IEEE128:mode><SDI:mode>2): Likewise.
	(floatditf2): Likewise.
	(floatdi<mode>2): Likewise.
	(floatuns<mode>tf2): Likewise.
	(floatuns<SDI:mode><IEEE128:mode>): Likewise.
	(cmptf_internal1): Use a mode iterator to add support for both
	types (IFmode, TFmode) that support IBM extended double.
	(cmp<mode>_internal1): Likewise.
	(cmptf_internal2): Likewise.
	(cmp<mode>_internal2): Likewise.

	* doc/extend.texi (Floating Types): Document __ibm128 and
	__float128 on PowerPC.

	* doc/invoke.texi (RS/6000 and PowerPC Options): Document
	-mfloat128 and -mno-float128.

[gcc/testsuite]
2015-10-29  Michael Meissner  <meissner@linux.vnet.ibm.com>

	* gcc.target/powerpc/float128-call.c: New test for -mfloat128 on
	PowerPC.
	* gcc.target/powerpc/float128-mix.c: Likewise.



git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@229547 138bc75d-0d04-0410-961f-82ee72b054a4

8 files changed, 805 insertions, 283 deletions

diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index ddc9108300c..1f32c9dfa59 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,5 +1,130 @@
 2015-10-29  Michael Meissner  <meissner@linux.vnet.ibm.com>
 
+	* config/rs6000/rs6000.c (rs6000_init_libfuncs): Split libfunc
+	setup into 3 functions: init_float128_ibm, init_float128_ieee, and
+	rs6000_init_libfuncs. If -mfloat128, add IFmode functions for all
+	of the traditional names that TFmode uses for handling IEEE
+	extended double. If -mfloat128, add KFmode functions for all of
+	the emulation functions. If -mabi=ieeelongdouble and -mfloat128,
+	make TFmode use the same emulation functions as KFmode.
+	(init_float128_ibm): Likewise.
+	(init_float128_ieee): Likewise.
+	(rs6000_generate_compare): For IEEE 128-bit floating point
+	comparisons, call the unordered comparison function instead of the
+	ordered comparison function.
+	(rs6000_expand_float128_convert): Deal with operands that are
+	memory operands. Restructure the code to use a switch statement on
+	the mode. Add support for TFmode defaulting to either IBM extended
+	double or IEEE 128-bit floating point. If the underlying types are
+	the same, use a move instead of a conversion function.
+	(TARGET_C_MODE_FOR_SUFFIX): Define 'q' and 'Q' as the suffix to
+	use for IEEE 128-bit floating point constants with -mfloat128.
+	(rs6000_c_mode_for_suffix): Likewise.
+	(TARGET_INVALID_BINARY_OP): Do not allow inter-mixing of IEEE
+	128-bit floating point with IBM extended double floating point.
+	(rs6000_invalid_binary_op): Likewise.
+	(rs6000_gen_le_vsx_permute): On little endian systems generate a
+	ROTATE insn instead of VEC_SELECT for IEEE 128-bit floating point
+	types that can go in vector registers.
+	(chain_contains_only_swaps): Properly swap IEEE 128-bit floating
+	point types that can go in vector registers on little endian
+	PowerPC systems.
+	(mark_swaps_for_removal): Likewise.
+	(rs6000_analyze_swaps): Likewise.
+	(rs6000_mangle_type): Use U10__float128 for IEEE 128-bit floating point.
+
+	* config/rs6000/rs6000.md (FLOAT128_SFDFTF): Delete iterator,
+	rework IEEE 128-bit floating point insns to deal with TFmode being
+	either IBM extended double or IEEE 128-bit floating point.
+	(IFKF): Likewise.
+	(IBM128): Update iterator to add condition that the mode is IBM
+	extended double.
+	(IEEE128): New iterator for IEEE 128-bit floating point.
+	(TFIFKF): Rename TFIFKF iterator to FLOAT128.
+	(FLOAT128): Likewise.
+	(signbit<mode>2): FLOAT128_IBM_P condition test moved into IBM128
+	iterator.
+	(neg<mode>2): Replace TFIFKF iterator with FLOAT128. Add support
+	for TFmode being IEEE 128-bit floating point. Use IEEE128 iterator
+	instead of hard coding TFmode or KFmode.
+	(negtf2_internal): Likewise.
+	(neg<mode>2_internal): Likewise.
+	(abs<mode>2): Likewise.
+	(abstf2_internal): Likewise.
+	(abs<mode>2_internal): Likewise.
+	(ieee_128bit_neg<mode>2): Likewise.
+	(ieee_128bit_neg<mode>2_internal): Likewise.
+	(ieee_128bit_abs<mode>2): Likewise.
+	(ieee_128bit_abs<mode>2_internal): Likewise.
+	(ieee_128bit_nabs<mode>2): Likewise.
+	(ieee_128bit_nabs<mode>2_internal): Likewise.
+	(extendiftf2): Add explicit conversions between 128-bit floating
+	point types. Drop the old conversions that had become unwieldy.
+	(extend<FLOAT128_SFDFTF:mode><IFKF:mode>2): Likewise.
+	(extendifkf2): Likewise.
+	(trunc<IFKF:mode><FLOAT128_SFDFTF:mode>2): Likewise.
+	(extendtfkf2): Likewise.
+	(fix_trunc<IFKF:mode><SDI:mode>2): Likewise.
+	(trunciftf2): Likewise.
+	(fixuns_trunc<IFKF:mode><SDI:mode>2): Likewise.
+	(truncifkf2): Likewise.
+	(float<SDI:mode><IFKF:mode>2): Likewise.
+	(trunckftf2): Likewise.
+	(floatuns<SDI:mode><IFKF:mode>2): Likewise.
+	(trunctfif2): Likewise.
+	(FP iterator): Allow TFmode to be IEEE 128-bit floating point.
+	(extenddftf2): Rework 128-bit floating point conversions to
+	properly handle -mabi=ieeelongdouble. Merge IFmode, TFmode, and
+	KFmode expanders into one function.
+	(extenddf<mode>2): Likewise.
+	(extenddftf2_fprs): Likewise.
+	(extenddf<mode>2_fprs): Likewise.
+	(extenddftf2_vsx): Likewise.
+	(extenddf<mode>2_vsx): Likewise.
+	(extendsftf2): Likewise.
+	(extendsf<mode>2): Likewise.
+	(trunctfdf2): Likewise.
+	(trunc<mode>df2): Likewise.
+	(trunctfdf2_internal1): Likewise.
+	(trunc<mode>df2_internal1): Likewise.
+	(trunctfdf2_internal2): Likewise.
+	(trunc<mode>df2_internal2): Likewise.
+	(trunctfsf2): Likewise.
+	(trunc<mode>sf2): Likewise.
+	(trunctfsf2_fprs): Likewise.
+	(trunc<mode>sf2_fprs): Likewise.
+	(floatsit2f): Likewise.
+	(floatsi<mode>2): Likewise.
+	(fix_trunc_helper): Likewise.
+	(fix_trunc_helper<mode>): Likewise.
+	(fix_trunctfsi2): Likewise.
+	(fix_trunc<mode>si2): Likewise.
+	(fix_trunctfsi2_fprs): Likewise.
+	(fix_trunc<mode>si2_fprs): Likewise.
+	(fix_trunctfsi2_internal): Likewise.
+	(fix_trunc<mode>si2_internal): Likewise.
+	(fix_trunctfdi2): Likewise.
+	(fix_trunc<mode>di2): Likewise.
+	(fixuns_trunctf<mode>2): Likewise.
+	(fixuns_trunc<IEEE128:mode><SDI:mode>2): Likewise.
+	(floatditf2): Likewise.
+	(floatdi<mode>2): Likewise.
+	(floatuns<mode>tf2): Likewise.
+	(floatuns<SDI:mode><IEEE128:mode>): Likewise.
+	(cmptf_internal1): Use a mode iterator to add support for both
+	types (IFmode, TFmode) that support IBM extended double.
+	(cmp<mode>_internal1): Likewise.
+	(cmptf_internal2): Likewise.
+	(cmp<mode>_internal2): Likewise.
+
+	* doc/extend.texi (Floating Types): Document __ibm128 and
+	__float128 on PowerPC.
+
+	* doc/invoke.texi (RS/6000 and PowerPC Options): Document
+	-mfloat128 and -mno-float128.
+
+2015-10-29  Michael Meissner  <meissner@linux.vnet.ibm.com>
+
 	* config/rs6000/rs6000.h (ALTIVEC_VECTOR_MODE): Add IEEE 128-bit
 	floating point modes that can go in vector registers.
 	(MODES_TIEABLE_P): Move tests for vector modes before tests for
diff --git a/gcc/config/rs6000/rs6000.c b/gcc/config/rs6000/rs6000.c
index b08b5ea2dce..21b43476181 100644
--- a/gcc/config/rs6000/rs6000.c
+++ b/gcc/config/rs6000/rs6000.c
@@ -1674,6 +1674,12 @@ static const struct attribute_spec rs6000_attribute_table[] =
 
 #undef TARGET_OFFLOAD_OPTIONS
 #define TARGET_OFFLOAD_OPTIONS rs6000_offload_options
+
+#undef TARGET_C_MODE_FOR_SUFFIX
+#define TARGET_C_MODE_FOR_SUFFIX rs6000_c_mode_for_suffix
+
+#undef TARGET_INVALID_BINARY_OP
+#define TARGET_INVALID_BINARY_OP rs6000_invalid_binary_op
 
 
 /* Processor table.  */
@@ -8461,8 +8467,14 @@ rs6000_const_vec (machine_mode mode)
 rtx
 rs6000_gen_le_vsx_permute (rtx source, machine_mode mode)
 {
-  rtx par = gen_rtx_PARALLEL (VOIDmode, rs6000_const_vec (mode));
-  return gen_rtx_VEC_SELECT (mode, source, par);
+  /* Use ROTATE instead of VEC_SELECT on IEEE 128-bit floating point.  */
+  if (FLOAT128_VECTOR_P (mode))
+    return gen_rtx_ROTATE (mode, source, GEN_INT (64));
+  else
+    {
+      rtx par = gen_rtx_PARALLEL (VOIDmode, rs6000_const_vec (mode));
+      return gen_rtx_VEC_SELECT (mode, source, par);
+    }
 }
 
 /* Emit a little-endian load from vector memory location SOURCE to VSX
@@ -16036,75 +16048,184 @@ rs6000_common_init_builtins (void)
     }
 }
 
+/* Set up AIX/Darwin/64-bit Linux quad floating point routines.  */
 static void
-rs6000_init_libfuncs (void)
+init_float128_ibm (machine_mode mode)
 {
-  if (!TARGET_IEEEQUAD)
-      /* AIX/Darwin/64-bit Linux quad floating point routines.  */
-    if (!TARGET_XL_COMPAT)
-      {
-	set_optab_libfunc (add_optab, TFmode, "__gcc_qadd");
-	set_optab_libfunc (sub_optab, TFmode, "__gcc_qsub");
-	set_optab_libfunc (smul_optab, TFmode, "__gcc_qmul");
-	set_optab_libfunc (sdiv_optab, TFmode, "__gcc_qdiv");
+  if (!TARGET_XL_COMPAT)
+    {
+      set_optab_libfunc (add_optab, mode, "__gcc_qadd");
+      set_optab_libfunc (sub_optab, mode, "__gcc_qsub");
+      set_optab_libfunc (smul_optab, mode, "__gcc_qmul");
+      set_optab_libfunc (sdiv_optab, mode, "__gcc_qdiv");
 
-	if (!(TARGET_HARD_FLOAT && (TARGET_FPRS || TARGET_E500_DOUBLE)))
-	  {
-	    set_optab_libfunc (neg_optab, TFmode, "__gcc_qneg");
-	    set_optab_libfunc (eq_optab, TFmode, "__gcc_qeq");
-	    set_optab_libfunc (ne_optab, TFmode, "__gcc_qne");
-	    set_optab_libfunc (gt_optab, TFmode, "__gcc_qgt");
-	    set_optab_libfunc (ge_optab, TFmode, "__gcc_qge");
-	    set_optab_libfunc (lt_optab, TFmode, "__gcc_qlt");
-	    set_optab_libfunc (le_optab, TFmode, "__gcc_qle");
-
-	    set_conv_libfunc (sext_optab, TFmode, SFmode, "__gcc_stoq");
-	    set_conv_libfunc (sext_optab, TFmode, DFmode, "__gcc_dtoq");
-	    set_conv_libfunc (trunc_optab, SFmode, TFmode, "__gcc_qtos");
-	    set_conv_libfunc (trunc_optab, DFmode, TFmode, "__gcc_qtod");
-	    set_conv_libfunc (sfix_optab, SImode, TFmode, "__gcc_qtoi");
-	    set_conv_libfunc (ufix_optab, SImode, TFmode, "__gcc_qtou");
-	    set_conv_libfunc (sfloat_optab, TFmode, SImode, "__gcc_itoq");
-	    set_conv_libfunc (ufloat_optab, TFmode, SImode, "__gcc_utoq");
-	  }
+      if (!(TARGET_HARD_FLOAT && (TARGET_FPRS || TARGET_E500_DOUBLE)))
+	{
+	  set_optab_libfunc (neg_optab, mode, "__gcc_qneg");
+	  set_optab_libfunc (eq_optab, mode, "__gcc_qeq");
+	  set_optab_libfunc (ne_optab, mode, "__gcc_qne");
+	  set_optab_libfunc (gt_optab, mode, "__gcc_qgt");
+	  set_optab_libfunc (ge_optab, mode, "__gcc_qge");
+	  set_optab_libfunc (lt_optab, mode, "__gcc_qlt");
+	  set_optab_libfunc (le_optab, mode, "__gcc_qle");
 
-	if (!(TARGET_HARD_FLOAT && TARGET_FPRS))
-	  set_optab_libfunc (unord_optab, TFmode, "__gcc_qunord");
-      }
-    else
-      {
-	set_optab_libfunc (add_optab, TFmode, "_xlqadd");
-	set_optab_libfunc (sub_optab, TFmode, "_xlqsub");
-	set_optab_libfunc (smul_optab, TFmode, "_xlqmul");
-	set_optab_libfunc (sdiv_optab, TFmode, "_xlqdiv");
-      }
+	  set_conv_libfunc (sext_optab, mode, SFmode, "__gcc_stoq");
+	  set_conv_libfunc (sext_optab, mode, DFmode, "__gcc_dtoq");
+	  set_conv_libfunc (trunc_optab, SFmode, mode, "__gcc_qtos");
+	  set_conv_libfunc (trunc_optab, DFmode, mode, "__gcc_qtod");
+	  set_conv_libfunc (sfix_optab, SImode, mode, "__gcc_qtoi");
+	  set_conv_libfunc (ufix_optab, SImode, mode, "__gcc_qtou");
+	  set_conv_libfunc (sfloat_optab, mode, SImode, "__gcc_itoq");
+	  set_conv_libfunc (ufloat_optab, mode, SImode, "__gcc_utoq");
+	}
+
+      if (!(TARGET_HARD_FLOAT && TARGET_FPRS))
+	set_optab_libfunc (unord_optab, mode, "__gcc_qunord");
+    }
   else
     {
-      /* 32-bit SVR4 quad floating point routines.  */
+      set_optab_libfunc (add_optab, mode, "_xlqadd");
+      set_optab_libfunc (sub_optab, mode, "_xlqsub");
+      set_optab_libfunc (smul_optab, mode, "_xlqmul");
+      set_optab_libfunc (sdiv_optab, mode, "_xlqdiv");
+    }
+
+  /* Add various conversions for IFmode to use the traditional TFmode
+     names.  */
+  if (mode == IFmode)
+    {
+      set_conv_libfunc (sext_optab, mode, SDmode, "__dpd_extendsdtf2");
+      set_conv_libfunc (sext_optab, mode, DDmode, "__dpd_extendddtf2");
+      set_conv_libfunc (trunc_optab, mode, TDmode, "__dpd_trunctftd2");
+      set_conv_libfunc (trunc_optab, SDmode, mode, "__dpd_trunctfsd2");
+      set_conv_libfunc (trunc_optab, DDmode, mode, "__dpd_trunctfdd2");
+      set_conv_libfunc (sext_optab, TDmode, mode, "__dpd_extendtdtf2");
 
-      set_optab_libfunc (add_optab, TFmode, "_q_add");
-      set_optab_libfunc (sub_optab, TFmode, "_q_sub");
-      set_optab_libfunc (neg_optab, TFmode, "_q_neg");
-      set_optab_libfunc (smul_optab, TFmode, "_q_mul");
-      set_optab_libfunc (sdiv_optab, TFmode, "_q_div");
+      if (TARGET_POWERPC64)
+	{
+	  set_conv_libfunc (sfix_optab, TImode, mode, "__fixtfti");
+	  set_conv_libfunc (ufix_optab, TImode, mode, "__fixunstfti");
+	  set_conv_libfunc (sfloat_optab, mode, TImode, "__floattitf");
+	  set_conv_libfunc (ufloat_optab, mode, TImode, "__floatuntitf");
+	}
+    }
+}
+
+/* Set up IEEE 128-bit floating point routines.  Use different names if the
+   arguments can be passed in a vector register.  The historical PowerPC
+   implementation of IEEE 128-bit floating point used _q_<op> for the names, so
+   continue to use that if we aren't using vector registers to pass IEEE
+   128-bit floating point.  */
+
+static void
+init_float128_ieee (machine_mode mode)
+{
+  if (FLOAT128_VECTOR_P (mode))
+    {
+      set_optab_libfunc (add_optab, mode, "__addkf3");
+      set_optab_libfunc (sub_optab, mode, "__subkf3");
+      set_optab_libfunc (neg_optab, mode, "__negkf2");
+      set_optab_libfunc (smul_optab, mode, "__mulkf3");
+      set_optab_libfunc (sdiv_optab, mode, "__divkf3");
+      set_optab_libfunc (sqrt_optab, mode, "__sqrtkf2");
+      set_optab_libfunc (abs_optab, mode, "__abstkf2");
+
+      set_optab_libfunc (eq_optab, mode, "__eqkf2");
+      set_optab_libfunc (ne_optab, mode, "__nekf2");
+      set_optab_libfunc (gt_optab, mode, "__gtkf2");
+      set_optab_libfunc (ge_optab, mode, "__gekf2");
+      set_optab_libfunc (lt_optab, mode, "__ltkf2");
+      set_optab_libfunc (le_optab, mode, "__lekf2");
+      set_optab_libfunc (unord_optab, mode, "__unordkf2");
+      set_optab_libfunc (cmp_optab, mode, "__cmpokf2");		/* fcmpo */
+      set_optab_libfunc (ucmp_optab, mode, "__cmpukf2");	/* fcmpu */
+
+      set_conv_libfunc (sext_optab, mode, SFmode, "__extendsfkf2");
+      set_conv_libfunc (sext_optab, mode, DFmode, "__extenddfkf2");
+      set_conv_libfunc (trunc_optab, SFmode, mode, "__trunckfsf2");
+      set_conv_libfunc (trunc_optab, DFmode, mode, "__trunckfdf2");
+
+      set_conv_libfunc (sext_optab, mode, IFmode, "__extendtfkf2");
+      if (mode != TFmode && FLOAT128_IBM_P (TFmode))
+	set_conv_libfunc (sext_optab, mode, TFmode, "__extendtfkf2");
+
+      set_conv_libfunc (trunc_optab, IFmode, mode, "__trunckftf2");
+      if (mode != TFmode && FLOAT128_IBM_P (TFmode))
+	set_conv_libfunc (trunc_optab, TFmode, mode, "__trunckftf2");
+
+      set_conv_libfunc (sext_optab, mode, SDmode, "__dpd_extendsdkf2");
+      set_conv_libfunc (sext_optab, mode, DDmode, "__dpd_extendddkf2");
+      set_conv_libfunc (trunc_optab, mode, TDmode, "__dpd_trunckftd2");
+      set_conv_libfunc (trunc_optab, SDmode, mode, "__dpd_trunckfsd2");
+      set_conv_libfunc (trunc_optab, DDmode, mode, "__dpd_trunckfdd2");
+      set_conv_libfunc (sext_optab, TDmode, mode, "__dpd_extendtdkf2");
+
+      set_conv_libfunc (sfix_optab, SImode, mode, "__fixkfsi");
+      set_conv_libfunc (ufix_optab, SImode, mode, "__fixunskfsi");
+      set_conv_libfunc (sfix_optab, DImode, mode, "__fixkfdi");
+      set_conv_libfunc (ufix_optab, DImode, mode, "__fixunskfdi");
+
+      set_conv_libfunc (sfloat_optab, mode, SImode, "__floatsikf");
+      set_conv_libfunc (ufloat_optab, mode, SImode, "__floatunsikf");
+      set_conv_libfunc (sfloat_optab, mode, DImode, "__floatdikf");
+      set_conv_libfunc (ufloat_optab, mode, DImode, "__floatundikf");
+
+      if (TARGET_POWERPC64)
+	{
+	  set_conv_libfunc (sfix_optab, TImode, mode, "__fixkfti");
+	  set_conv_libfunc (ufix_optab, TImode, mode, "__fixunskfti");
+	  set_conv_libfunc (sfloat_optab, mode, TImode, "__floattikf");
+	  set_conv_libfunc (ufloat_optab, mode, TImode, "__floatuntikf");
+	}
+    }
+
+  else
+    {
+      set_optab_libfunc (add_optab, mode, "_q_add");
+      set_optab_libfunc (sub_optab, mode, "_q_sub");
+      set_optab_libfunc (neg_optab, mode, "_q_neg");
+      set_optab_libfunc (smul_optab, mode, "_q_mul");
+      set_optab_libfunc (sdiv_optab, mode, "_q_div");
       if (TARGET_PPC_GPOPT)
-	set_optab_libfunc (sqrt_optab, TFmode, "_q_sqrt");
+	set_optab_libfunc (sqrt_optab, mode, "_q_sqrt");
 
-      set_optab_libfunc (eq_optab, TFmode, "_q_feq");
-      set_optab_libfunc (ne_optab, TFmode, "_q_fne");
-      set_optab_libfunc (gt_optab, TFmode, "_q_fgt");
-      set_optab_libfunc (ge_optab, TFmode, "_q_fge");
-      set_optab_libfunc (lt_optab, TFmode, "_q_flt");
-      set_optab_libfunc (le_optab, TFmode, "_q_fle");
+      set_optab_libfunc (eq_optab, mode, "_q_feq");
+      set_optab_libfunc (ne_optab, mode, "_q_fne");
+      set_optab_libfunc (gt_optab, mode, "_q_fgt");
+      set_optab_libfunc (ge_optab, mode, "_q_fge");
+      set_optab_libfunc (lt_optab, mode, "_q_flt");
+      set_optab_libfunc (le_optab, mode, "_q_fle");
 
-      set_conv_libfunc (sext_optab, TFmode, SFmode, "_q_stoq");
-      set_conv_libfunc (sext_optab, TFmode, DFmode, "_q_dtoq");
-      set_conv_libfunc (trunc_optab, SFmode, TFmode, "_q_qtos");
-      set_conv_libfunc (trunc_optab, DFmode, TFmode, "_q_qtod");
-      set_conv_libfunc (sfix_optab, SImode, TFmode, "_q_qtoi");
-      set_conv_libfunc (ufix_optab, SImode, TFmode, "_q_qtou");
-      set_conv_libfunc (sfloat_optab, TFmode, SImode, "_q_itoq");
-      set_conv_libfunc (ufloat_optab, TFmode, SImode, "_q_utoq");
+      set_conv_libfunc (sext_optab, mode, SFmode, "_q_stoq");
+      set_conv_libfunc (sext_optab, mode, DFmode, "_q_dtoq");
+      set_conv_libfunc (trunc_optab, SFmode, mode, "_q_qtos");
+      set_conv_libfunc (trunc_optab, DFmode, mode, "_q_qtod");
+      set_conv_libfunc (sfix_optab, SImode, mode, "_q_qtoi");
+      set_conv_libfunc (ufix_optab, SImode, mode, "_q_qtou");
+      set_conv_libfunc (sfloat_optab, mode, SImode, "_q_itoq");
+      set_conv_libfunc (ufloat_optab, mode, SImode, "_q_utoq");
+    }
+}
+
+static void
+rs6000_init_libfuncs (void)
+{
+  /* __float128 support.  */
+  if (TARGET_FLOAT128)
+    {
+      init_float128_ibm (IFmode);
+      init_float128_ieee (KFmode);
+    }
+
+  /* AIX/Darwin/64-bit Linux quad floating point routines.  */
+  if (TARGET_LONG_DOUBLE_128)
+    {
+      if (!TARGET_IEEEQUAD)
+	init_float128_ibm (TFmode);
+
+      /* IEEE 128-bit including 32-bit SVR4 quad floating point routines.  */
+      else
+	init_float128_ieee (TFmode);
     }
 }
 
@@ -19989,17 +20110,18 @@ rs6000_generate_compare (rtx cmp, machine_mode mode)
       emit_insn (cmp);
     }
 
-  /* IEEE 128-bit support in VSX registers.  The comparison function (__cmpkf2)
-     returns 0..15 that is laid out the same way as the PowerPC CR register
-     would for a normal floating point comparison.  */
+  /* IEEE 128-bit support in VSX registers.  The comparison functions
+     (__cmpokf2 and __cmpukf2) returns 0..15 that is laid out the same way as
+     the PowerPC CR register would for a normal floating point comparison from
+     the fcmpo and fcmpu instructions.  */
   else if (FLOAT128_IEEE_P (mode))
     {
       rtx and_reg = gen_reg_rtx (SImode);
       rtx dest = gen_reg_rtx (SImode);
-      rtx libfunc = optab_libfunc (cmp_optab, mode);
+      rtx libfunc = optab_libfunc (ucmp_optab, mode);
       HOST_WIDE_INT mask_value = 0;
 
-      /* Values that __cmpkf2 returns.  */
+      /* Values that __cmpokf2/__cmpukf2 returns.  */
 #define PPC_CMP_UNORDERED	0x1		/* isnan (a) || isnan (b).  */
 #define PPC_CMP_EQUAL		0x2		/* a == b.  */
 #define PPC_CMP_GREATER_THEN	0x4		/* a > b.  */
@@ -20170,7 +20292,49 @@ rs6000_generate_compare (rtx cmp, machine_mode mode)
   return gen_rtx_fmt_ee (code, VOIDmode, compare_result, const0_rtx);
 }
 
+
+/* Return the diagnostic message string if the binary operation OP is
+   not permitted on TYPE1 and TYPE2, NULL otherwise.  */
+
+static const char*
+rs6000_invalid_binary_op (int op ATTRIBUTE_UNUSED,
+			  const_tree type1,
+			  const_tree type2)
+{
+  enum machine_mode mode1 = TYPE_MODE (type1);
+  enum machine_mode mode2 = TYPE_MODE (type2);
+
+  /* For complex modes, use the inner type.  */
+  if (COMPLEX_MODE_P (mode1))
+    mode1 = GET_MODE_INNER (mode1);
+
+  if (COMPLEX_MODE_P (mode2))
+    mode2 = GET_MODE_INNER (mode2);
+
+  /* Don't allow IEEE 754R 128-bit binary floating point and IBM extended
+     double to intermix.  */
+  if (mode1 == mode2)
+    return NULL;
+
+  if ((mode1 == KFmode && mode2 == IFmode)
+      || (mode1 == IFmode && mode2 == KFmode))
+    return N_("__float128 and __ibm128 cannot be used in the same expression");
+
+  if (TARGET_IEEEQUAD
+      && ((mode1 == IFmode && mode2 == TFmode)
+	  || (mode1 == TFmode && mode2 == IFmode)))
+    return N_("__ibm128 and long double cannot be used in the same expression");
+
+  if (!TARGET_IEEEQUAD
+      && ((mode1 == KFmode && mode2 == TFmode)
+	  || (mode1 == TFmode && mode2 == KFmode)))
+    return N_("__float128 and long double cannot be used in the same "
+	      "expression");
+
+  return NULL;
+}
 
+
 /* Expand floating point conversion to/from __float128 and __ibm128.  */
 
 void
@@ -20179,60 +20343,121 @@ rs6000_expand_float128_convert (rtx dest, rtx src, bool unsigned_p)
   machine_mode dest_mode = GET_MODE (dest);
   machine_mode src_mode = GET_MODE (src);
   convert_optab cvt = unknown_optab;
+  bool do_move = false;
   rtx libfunc = NULL_RTX;
   rtx dest2;
 
   if (dest_mode == src_mode)
     gcc_unreachable ();
 
+  /* Eliminate memory operations.  */
+  if (MEM_P (src))
+    src = force_reg (src_mode, src);
+
+  if (MEM_P (dest))
+    {
+      rtx tmp = gen_reg_rtx (dest_mode);
+      rs6000_expand_float128_convert (tmp, src, unsigned_p);
+      rs6000_emit_move (dest, tmp, dest_mode);
+      return;
+    }
+
+  /* Convert to IEEE 128-bit floating point.  */
   if (FLOAT128_IEEE_P (dest_mode))
     {
-      if (src_mode == SFmode
-	  || src_mode == DFmode
-	  || FLOAT128_IBM_P (src_mode))
-	cvt = sext_optab;
+      switch (src_mode)
+	{
+	case DFmode:
+	  cvt = sext_optab;
+	  break;
 
-      else if (GET_MODE_CLASS (src_mode) == MODE_INT)
-	cvt = (unsigned_p) ? ufloat_optab : sfloat_optab;
+	case SFmode:
+	  cvt = sext_optab;
+	  break;
 
-      else if (FLOAT128_IEEE_P (src_mode))
-	emit_move_insn (dest, gen_lowpart (dest_mode, src));
+	case KFmode:
+	case IFmode:
+	case TFmode:
+	  if (FLOAT128_IBM_P (src_mode))
+	    cvt = sext_optab;
+	  else
+	    do_move = true;
+	  break;
 
-      else
-	gcc_unreachable ();
+	case SImode:
+	case DImode:
+	  cvt = (unsigned_p) ? ufloat_optab : sfloat_optab;
+	  break;
+
+	default:
+	  gcc_unreachable ();
+	}
     }
 
+  /* Convert from IEEE 128-bit floating point.  */
   else if (FLOAT128_IEEE_P (src_mode))
     {
-      if (dest_mode == SFmode
-	  || dest_mode == DFmode
-	  || FLOAT128_IBM_P (dest_mode))
-	cvt = trunc_optab;
+      switch (dest_mode)
+	{
+	case DFmode:
+	  cvt = trunc_optab;
+	  break;
 
-      else if (GET_MODE_CLASS (dest_mode) == MODE_INT)
-	cvt = (unsigned_p) ? ufix_optab : sfix_optab;
+	case SFmode:
+	  cvt = trunc_optab;
+	  break;
 
-      else
-	gcc_unreachable ();
+	case KFmode:
+	case IFmode:
+	case TFmode:
+	  if (FLOAT128_IBM_P (dest_mode))
+	    cvt = trunc_optab;
+	  else
+	    do_move = true;
+	  break;
+
+	case SImode:
+	case DImode:
+	  cvt = (unsigned_p) ? ufix_optab : sfix_optab;
+	  break;
+
+	default:
+	  gcc_unreachable ();
+	}
     }
 
+  /* Both IBM format.  */
+  else if (FLOAT128_IBM_P (dest_mode) && FLOAT128_IBM_P (src_mode))
+    do_move = true;
+
   else
     gcc_unreachable ();
 
-  gcc_assert (cvt != unknown_optab);
-  libfunc = convert_optab_libfunc (cvt, dest_mode, src_mode);
-  gcc_assert (libfunc != NULL_RTX);
+  /* Handle conversion between TFmode/KFmode.  */
+  if (do_move)
+    emit_move_insn (dest, gen_lowpart (dest_mode, src));
 
-  dest2 = emit_library_call_value (libfunc, dest, LCT_CONST, dest_mode, 1, src,
-				   src_mode);
+  /* Call an external function to do the conversion.  */
+  else if (cvt != unknown_optab)
+    {
+      libfunc = convert_optab_libfunc (cvt, dest_mode, src_mode);
+      gcc_assert (libfunc != NULL_RTX);
 
-  gcc_assert (dest != NULL_RTX);
-  if (!rtx_equal_p (dest, dest2))
-    emit_move_insn (dest, dest2);
+      dest2 = emit_library_call_value (libfunc, dest, LCT_CONST, dest_mode, 1, src,
+				       src_mode);
+
+      gcc_assert (dest2 != NULL_RTX);
+      if (!rtx_equal_p (dest, dest2))
+	emit_move_insn (dest, dest2);
+    }
+
+  else
+    gcc_unreachable ();
 
   return;
 }
 
+
 /* Emit the RTL for an sISEL pattern.  */
 
 void
@@ -29827,6 +30052,21 @@ rs6000_mangle_type (const_tree type)
   if (type == bool_int_type_node) return "U6__booli";
   if (type == bool_long_type_node) return "U6__booll";
 
+  /* Use a unique name for __float128 rather than trying to use "e" or "g". Use
+     "g" for IBM extended double, no matter whether it is long double (using
+     -mabi=ibmlongdouble) or the distinct __ibm128 type.  */
+  if (TARGET_FLOAT128)
+    {
+      if (type == ieee128_float_type_node)
+	return "U10__float128";
+
+      if (type == ibm128_float_type_node)
+	return "g";
+
+      if (type == long_double_type_node && TARGET_LONG_DOUBLE_128)
+	return (TARGET_IEEEQUAD) ? "U10__float128" : "g";
+    }
+
   /* Mangle IBM extended float long double as `g' (__float128) on
      powerpc*-linux where long-double-64 previously was the default.  */
   if (TYPE_MAIN_VARIANT (type) == long_double_type_node
@@ -33136,6 +33376,26 @@ rs6000_vector_mode_supported_p (machine_mode mode)
     return false;
 }
 
+/* Target hook for c_mode_for_suffix.  */
+static machine_mode
+rs6000_c_mode_for_suffix (char suffix)
+{
+  if (TARGET_FLOAT128)
+    {
+      if (suffix == 'q' || suffix == 'Q')
+	return (FLOAT128_IEEE_P (TFmode)) ? TFmode : KFmode;
+
+      /* At the moment, we are not defining a suffix for IBM extended double.
+	 If/when the default for -mabi=ieeelongdouble is changed, and we want
+	 to support __ibm128 constants in legacy library code, we may need to
+	 re-evalaute this decision.  Currently, c-lex.c only supports 'w' and
+	 'q' as machine dependent suffixes.  The x86_64 port uses 'w' for
+	 __float80 constants.  */
+    }
+
+  return VOIDmode;
+}
+
 /* Target hook for invalid_arg_for_unprototyped_fn. */
 static const char *
 invalid_arg_for_unprototyped_fn (const_tree typelist, const_tree funcdecl, const_tree val)
@@ -35605,7 +35865,7 @@ chain_contains_only_swaps (swap_web_entry *insn_entry, struct df_link *link,
 
   for (; link; link = link->next)
     {
-      if (!VECTOR_MODE_P (GET_MODE (DF_REF_REG (link->ref))))
+      if (!ALTIVEC_OR_VSX_VECTOR_MODE (GET_MODE (DF_REF_REG (link->ref))))
 	continue;
 
       if (DF_REF_IS_ARTIFICIAL (link->ref))
@@ -35704,7 +35964,7 @@ mark_swaps_for_removal (swap_web_entry *insn_entry, unsigned int i)
 	{
 	  /* Ignore uses for addressability.  */
 	  machine_mode mode = GET_MODE (DF_REF_REG (use));
-	  if (!VECTOR_MODE_P (mode))
+	  if (!ALTIVEC_OR_VSX_VECTOR_MODE (mode))
 	    continue;
 
 	  struct df_link *link = DF_REF_CHAIN (use);
@@ -36218,10 +36478,11 @@ rs6000_analyze_swaps (function *fun)
 		    mode = V4SImode;
 		}
 
-	      if (VECTOR_MODE_P (mode) || mode == TImode)
+	      if (ALTIVEC_OR_VSX_VECTOR_MODE (mode) || mode == TImode)
 		{
 		  insn_entry[uid].is_relevant = 1;
-		  if (mode == TImode || mode == V1TImode)
+		  if (mode == TImode || mode == V1TImode
+		      || FLOAT128_VECTOR_P (mode))
 		    insn_entry[uid].is_128_int = 1;
 		  if (DF_REF_INSN_INFO (mention))
 		    insn_entry[uid].contains_subreg
@@ -36242,13 +36503,14 @@ rs6000_analyze_swaps (function *fun)
 		 isn't sufficient to ensure we union the call into the
 		 web with the parameter setup code.  */
 	      if (mode == DImode && GET_CODE (insn) == SET
-		  && VECTOR_MODE_P (GET_MODE (SET_DEST (insn))))
+		  && ALTIVEC_OR_VSX_VECTOR_MODE (GET_MODE (SET_DEST (insn))))
 		mode = GET_MODE (SET_DEST (insn));
 
-	      if (VECTOR_MODE_P (mode) || mode == TImode)
+	      if (ALTIVEC_OR_VSX_VECTOR_MODE (mode) || mode == TImode)
 		{
 		  insn_entry[uid].is_relevant = 1;
-		  if (mode == TImode || mode == V1TImode)
+		  if (mode == TImode || mode == V1TImode
+		      || FLOAT128_VECTOR_P (mode))
 		    insn_entry[uid].is_128_int = 1;
 		  if (DF_REF_INSN_INFO (mention))
 		    insn_entry[uid].contains_subreg
diff --git a/gcc/config/rs6000/rs6000.md b/gcc/config/rs6000/rs6000.md
index 78e20f07632..eeeef342228 100644
--- a/gcc/config/rs6000/rs6000.md
+++ b/gcc/config/rs6000/rs6000.md
@@ -347,8 +347,7 @@
    && ((TARGET_FPRS && TARGET_SINGLE_FLOAT) || TARGET_E500_SINGLE)")
   (DF "TARGET_HARD_FLOAT 
    && ((TARGET_FPRS && TARGET_DOUBLE_FLOAT) || TARGET_E500_DOUBLE)")
-  (TF "!TARGET_IEEEQUAD
-   && TARGET_HARD_FLOAT
+  (TF "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE)
    && TARGET_LONG_DOUBLE_128")
   (IF "TARGET_FLOAT128")
@@ -448,24 +447,18 @@
 ; Iterator for just SF/DF
 (define_mode_iterator SFDF [SF DF])
 
-; Iterator for float128 floating conversions
-(define_mode_iterator FLOAT128_SFDFTF [
-    (SF "TARGET_FLOAT128")
-    (DF "TARGET_FLOAT128")
-    (TF "FLOAT128_IBM_P (TFmode)")
-    (IF "TARGET_FLOAT128")])
-
-; Iterator for special 128-bit floating point.  This is for non-default
-; conversions, so TFmode is not used here.
-(define_mode_iterator IFKF [IF KF])
-
 ; Iterator for 128-bit floating point that uses the IBM double-double format
-(define_mode_iterator IBM128 [IF TF])
+(define_mode_iterator IBM128 [(IF "FLOAT128_IBM_P (IFmode)")
+			      (TF "FLOAT128_IBM_P (TFmode)")])
+
+; Iterator for 128-bit floating point that uses IEEE 128-bit float
+(define_mode_iterator IEEE128 [(KF "FLOAT128_IEEE_P (KFmode)")
+			       (TF "FLOAT128_IEEE_P (TFmode)")])
 
 ; Iterator for 128-bit floating point
-(define_mode_iterator TFIFKF [(KF "TARGET_FLOAT128")
-			      (IF "TARGET_FLOAT128")
-			      (TF "TARGET_LONG_DOUBLE_128")])
+(define_mode_iterator FLOAT128 [(KF "TARGET_FLOAT128")
+				(IF "TARGET_FLOAT128")
+				(TF "TARGET_LONG_DOUBLE_128")])
 
 ; SF/DF suffix for traditional floating instructions
 (define_mode_attr Ftrad		[(SF "s") (DF "")])
@@ -4248,8 +4241,7 @@
    	(match_dup 5))
    (set (match_operand:SI 0 "gpc_reg_operand" "")
   	(match_dup 6))]
-  "FLOAT128_IBM_P (<MODE>mode)
-   && TARGET_HARD_FLOAT
+  "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE)"
 {
   operands[2] = gen_reg_rtx (DFmode);
@@ -6470,34 +6462,51 @@
 { rs6000_split_multireg_move (operands[0], operands[1]); DONE; }
   [(set_attr "length" "20,20,16")])
 
-(define_expand "extenddftf2"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "")
-	(float_extend:TF (match_operand:DF 1 "gpc_reg_operand" "")))]
+(define_expand "extenddf<mode>2"
+  [(set (match_operand:FLOAT128 0 "gpc_reg_operand" "")
+	(float_extend:FLOAT128 (match_operand:DF 1 "gpc_reg_operand" "")))]
   "TARGET_HARD_FLOAT && (TARGET_FPRS || TARGET_E500_DOUBLE)
    && TARGET_LONG_DOUBLE_128"
 {
-  if (TARGET_IEEEQUAD)
+  if (FLOAT128_IEEE_P (<MODE>mode))
     rs6000_expand_float128_convert (operands[0], operands[1], false);
   else if (TARGET_E500_DOUBLE)
-    emit_insn (gen_spe_extenddftf2 (operands[0], operands[1]));
+    {
+      gcc_assert (<MODE>mode == TFmode);
+      emit_insn (gen_spe_extenddftf2 (operands[0], operands[1]));
+    }
   else if (TARGET_VSX)
-    emit_insn (gen_extenddftf2_vsx (operands[0], operands[1]));
-  else
+    {
+      if (<MODE>mode == TFmode)
+	emit_insn (gen_extenddftf2_vsx (operands[0], operands[1]));
+      else if (<MODE>mode == IFmode)
+	emit_insn (gen_extenddfif2_vsx (operands[0], operands[1]));
+      else
+	gcc_unreachable ();
+    }
+   else
     {
       rtx zero = gen_reg_rtx (DFmode);
       rs6000_emit_move (zero, CONST0_RTX (DFmode), DFmode);
-      emit_insn (gen_extenddftf2_fprs (operands[0], operands[1], zero));
+
+      if (<MODE>mode == TFmode)
+	emit_insn (gen_extenddftf2_fprs (operands[0], operands[1], zero));
+      else if (<MODE>mode == IFmode)
+	emit_insn (gen_extenddfif2_fprs (operands[0], operands[1], zero));
+      else
+	gcc_unreachable ();
     }
   DONE;
 })
 
 ;; Allow memory operands for the source to be created by the combiner.
-(define_insn_and_split "extenddftf2_fprs"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "=d,d,&d")
-	(float_extend:TF (match_operand:DF 1 "nonimmediate_operand" "d,m,d")))
+(define_insn_and_split "extenddf<mode>2_fprs"
+  [(set (match_operand:IBM128 0 "gpc_reg_operand" "=d,d,&d")
+	(float_extend:IBM128
+	 (match_operand:DF 1 "nonimmediate_operand" "d,m,d")))
    (use (match_operand:DF 2 "nonimmediate_operand" "m,m,d"))]
   "!TARGET_VSX && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
-   && TARGET_LONG_DOUBLE_128 && !TARGET_IEEEQUAD"
+   && TARGET_LONG_DOUBLE_128 && FLOAT128_IBM_P (<MODE>mode)"
   "#"
   "&& reload_completed"
   [(set (match_dup 3) (match_dup 1))
@@ -6506,14 +6515,15 @@
   const int lo_word = LONG_DOUBLE_LARGE_FIRST ? GET_MODE_SIZE (DFmode) : 0;
   const int hi_word = LONG_DOUBLE_LARGE_FIRST ? 0 : GET_MODE_SIZE (DFmode);
 
-  operands[3] = simplify_gen_subreg (DFmode, operands[0], TFmode, hi_word);
-  operands[4] = simplify_gen_subreg (DFmode, operands[0], TFmode, lo_word);
+  operands[3] = simplify_gen_subreg (DFmode, operands[0], <MODE>mode, hi_word);
+  operands[4] = simplify_gen_subreg (DFmode, operands[0], <MODE>mode, lo_word);
 })
 
-(define_insn_and_split "extenddftf2_vsx"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "=d,d")
-	(float_extend:TF (match_operand:DF 1 "nonimmediate_operand" "ws,m")))]
-  "TARGET_LONG_DOUBLE_128 && TARGET_VSX && !TARGET_IEEEQUAD"
+(define_insn_and_split "extenddf<mode>2_vsx"
+  [(set (match_operand:IBM128 0 "gpc_reg_operand" "=d,d")
+	(float_extend:IBM128
+	 (match_operand:DF 1 "nonimmediate_operand" "ws,m")))]
+  "TARGET_LONG_DOUBLE_128 && TARGET_VSX && FLOAT128_IBM_P (<MODE>mode)"
   "#"
   "&& reload_completed"
   [(set (match_dup 2) (match_dup 1))
@@ -6522,47 +6532,48 @@
   const int lo_word = LONG_DOUBLE_LARGE_FIRST ? GET_MODE_SIZE (DFmode) : 0;
   const int hi_word = LONG_DOUBLE_LARGE_FIRST ? 0 : GET_MODE_SIZE (DFmode);
 
-  operands[2] = simplify_gen_subreg (DFmode, operands[0], TFmode, hi_word);
-  operands[3] = simplify_gen_subreg (DFmode, operands[0], TFmode, lo_word);
+  operands[2] = simplify_gen_subreg (DFmode, operands[0], <MODE>mode, hi_word);
+  operands[3] = simplify_gen_subreg (DFmode, operands[0], <MODE>mode, lo_word);
   operands[4] = CONST0_RTX (DFmode);
 })
 
-(define_expand "extendsftf2"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "")
-	(float_extend:TF (match_operand:SF 1 "gpc_reg_operand" "")))]
+(define_expand "extendsf<mode>2"
+  [(set (match_operand:FLOAT128 0 "gpc_reg_operand" "")
+	(float_extend:FLOAT128 (match_operand:SF 1 "gpc_reg_operand" "")))]
   "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE)
    && TARGET_LONG_DOUBLE_128"
 {
-  if (TARGET_IEEEQUAD)
+  if (FLOAT128_IEEE_P (<MODE>mode))
     rs6000_expand_float128_convert (operands[0], operands[1], false);
   else
     {
       rtx tmp = gen_reg_rtx (DFmode);
       emit_insn (gen_extendsfdf2 (tmp, operands[1]));
-      emit_insn (gen_extenddftf2 (operands[0], tmp));
+      emit_insn (gen_extenddf<mode>2 (operands[0], tmp));
     }
   DONE;
 })
 
-(define_expand "trunctfdf2"
+(define_expand "trunc<mode>df2"
   [(set (match_operand:DF 0 "gpc_reg_operand" "")
-	(float_truncate:DF (match_operand:TF 1 "gpc_reg_operand" "")))]
+	(float_truncate:DF (match_operand:FLOAT128 1 "gpc_reg_operand" "")))]
   "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE)
    && TARGET_LONG_DOUBLE_128"
 {
-  if (TARGET_IEEEQUAD)
+  if (FLOAT128_IEEE_P (<MODE>mode))
     {
       rs6000_expand_float128_convert (operands[0], operands[1], false);
       DONE;
     }
 })
 
-(define_insn_and_split "trunctfdf2_internal1"
+(define_insn_and_split "trunc<mode>df2_internal1"
   [(set (match_operand:DF 0 "gpc_reg_operand" "=d,?d")
-	(float_truncate:DF (match_operand:TF 1 "gpc_reg_operand" "0,d")))]
-  "!TARGET_IEEEQUAD && !TARGET_XL_COMPAT
+	(float_truncate:DF
+	 (match_operand:IBM128 1 "gpc_reg_operand" "0,d")))]
+  "FLOAT128_IBM_P (<MODE>mode) && !TARGET_XL_COMPAT
    && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_LONG_DOUBLE_128"
   "@
    #
@@ -6575,39 +6586,44 @@
 }
   [(set_attr "type" "fp")])
 
-(define_insn "trunctfdf2_internal2"
+(define_insn "trunc<mode>df2_internal2"
   [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(float_truncate:DF (match_operand:TF 1 "gpc_reg_operand" "d")))]
-  "!TARGET_IEEEQUAD && TARGET_XL_COMPAT
-   && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
-   && TARGET_LONG_DOUBLE_128"
+	(float_truncate:DF (match_operand:IBM128 1 "gpc_reg_operand" "d")))]
+  "FLOAT128_IBM_P (<MODE>mode) && TARGET_XL_COMPAT && TARGET_HARD_FLOAT
+   && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LONG_DOUBLE_128"
   "fadd %0,%1,%L1"
   [(set_attr "type" "fp")
    (set_attr "fp_type" "fp_addsub_d")])
 
-(define_expand "trunctfsf2"
+(define_expand "trunc<mode>sf2"
   [(set (match_operand:SF 0 "gpc_reg_operand" "")
-	(float_truncate:SF (match_operand:TF 1 "gpc_reg_operand" "")))]
+	(float_truncate:SF (match_operand:FLOAT128 1 "gpc_reg_operand" "")))]
   "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE)
    && TARGET_LONG_DOUBLE_128"
 {
-  if (TARGET_IEEEQUAD)
+  if (FLOAT128_IEEE_P (<MODE>mode))
     rs6000_expand_float128_convert (operands[0], operands[1], false);
   else if (TARGET_E500_DOUBLE)
-    emit_insn (gen_spe_trunctfsf2 (operands[0], operands[1]));
-  else
+    {
+      gcc_assert (<MODE>mode == TFmode);
+      emit_insn (gen_spe_trunctfsf2 (operands[0], operands[1]));
+    }
+  else if (<MODE>mode == TFmode)
     emit_insn (gen_trunctfsf2_fprs (operands[0], operands[1]));
+  else if (<MODE>mode == IFmode)
+    emit_insn (gen_truncifsf2_fprs (operands[0], operands[1]));
+  else
+    gcc_unreachable ();
   DONE;
 })
 
-(define_insn_and_split "trunctfsf2_fprs"
+(define_insn_and_split "trunc<mode>sf2_fprs"
   [(set (match_operand:SF 0 "gpc_reg_operand" "=f")
-	(float_truncate:SF (match_operand:TF 1 "gpc_reg_operand" "d")))
+	(float_truncate:SF (match_operand:IBM128 1 "gpc_reg_operand" "d")))
    (clobber (match_scratch:DF 2 "=d"))]
-  "!TARGET_IEEEQUAD
-   && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT 
-   && TARGET_LONG_DOUBLE_128"
+  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_SINGLE_FLOAT 
+   && TARGET_LONG_DOUBLE_128 && FLOAT128_IBM_P (<MODE>mode)"
   "#"
   "&& reload_completed"
   [(set (match_dup 2)
@@ -6616,56 +6632,69 @@
 	(float_truncate:SF (match_dup 2)))]
   "")
 
-(define_expand "floatsitf2"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "")
-        (float:TF (match_operand:SI 1 "gpc_reg_operand" "")))]
-  "!TARGET_IEEEQUAD
-   && TARGET_HARD_FLOAT
+(define_expand "floatsi<mode>2"
+  [(set (match_operand:FLOAT128 0 "gpc_reg_operand" "")
+        (float:FLOAT128 (match_operand:SI 1 "gpc_reg_operand" "")))]
+  "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE)
    && TARGET_LONG_DOUBLE_128"
 {
-  rtx tmp = gen_reg_rtx (DFmode);
-  expand_float (tmp, operands[1], false);
-  emit_insn (gen_extenddftf2 (operands[0], tmp));
+  if (FLOAT128_IEEE_P (<MODE>mode))
+    rs6000_expand_float128_convert (operands[0], operands[1], false);
+  else
+    {
+      rtx tmp = gen_reg_rtx (DFmode);
+      expand_float (tmp, operands[1], false);
+      if (<MODE>mode == TFmode)
+	emit_insn (gen_extenddftf2 (operands[0], tmp));
+      else if (<MODE>mode == IFmode)
+	emit_insn (gen_extenddfif2 (operands[0], tmp));
+      else
+	gcc_unreachable ();
+    }
   DONE;
 })
 
 ; fadd, but rounding towards zero.
 ; This is probably not the optimal code sequence.
-(define_insn "fix_trunc_helper"
+(define_insn "fix_trunc_helper<mode>"
   [(set (match_operand:DF 0 "gpc_reg_operand" "=d")
-	(unspec:DF [(match_operand:TF 1 "gpc_reg_operand" "d")]
+	(unspec:DF [(match_operand:IBM128 1 "gpc_reg_operand" "d")]
 		   UNSPEC_FIX_TRUNC_TF))
    (clobber (match_operand:DF 2 "gpc_reg_operand" "=&d"))]
-  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT"
+  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT
+   && FLOAT128_IBM_P (<MODE>mode)"
   "mffs %2\n\tmtfsb1 31\n\tmtfsb0 30\n\tfadd %0,%1,%L1\n\tmtfsf 1,%2"
   [(set_attr "type" "fp")
    (set_attr "length" "20")])
 
-(define_expand "fix_trunctfsi2"
+(define_expand "fix_trunc<mode>si2"
   [(set (match_operand:SI 0 "gpc_reg_operand" "")
-	(fix:SI (match_operand:TF 1 "gpc_reg_operand" "")))]
+	(fix:SI (match_operand:FLOAT128 1 "gpc_reg_operand" "")))]
   "TARGET_HARD_FLOAT
    && (TARGET_FPRS || TARGET_E500_DOUBLE) && TARGET_LONG_DOUBLE_128"
 {
-  if (TARGET_IEEEQUAD)
+  if (FLOAT128_IEEE_P (<MODE>mode))
     rs6000_expand_float128_convert (operands[0], operands[1], false);
-  else if (TARGET_E500_DOUBLE)
+  else if (TARGET_E500_DOUBLE && <MODE>mode == TFmode)
     emit_insn (gen_spe_fix_trunctfsi2 (operands[0], operands[1]));
-  else
+  else if (<MODE>mode == TFmode)
     emit_insn (gen_fix_trunctfsi2_fprs (operands[0], operands[1]));
+  else if (<MODE>mode == IFmode)
+    emit_insn (gen_fix_truncifsi2_fprs (operands[0], operands[1]));
+  else
+    gcc_unreachable ();
   DONE;
 })
 
-(define_expand "fix_trunctfsi2_fprs"
+(define_expand "fix_trunc<mode>si2_fprs"
   [(parallel [(set (match_operand:SI 0 "gpc_reg_operand" "")
-		   (fix:SI (match_operand:TF 1 "gpc_reg_operand" "")))
+		   (fix:SI (match_operand:IBM128 1 "gpc_reg_operand" "")))
 	      (clobber (match_dup 2))
 	      (clobber (match_dup 3))
 	      (clobber (match_dup 4))
 	      (clobber (match_dup 5))])]
-  "!TARGET_IEEEQUAD
-   && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_LONG_DOUBLE_128"
+  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_LONG_DOUBLE_128"
 {
   operands[2] = gen_reg_rtx (DFmode);
   operands[3] = gen_reg_rtx (DFmode);
@@ -6673,21 +6702,21 @@
   operands[5] = assign_stack_temp (DImode, GET_MODE_SIZE (DImode));
 })
 
-(define_insn_and_split "*fix_trunctfsi2_internal"
+(define_insn_and_split "*fix_trunc<mode>si2_internal"
   [(set (match_operand:SI 0 "gpc_reg_operand" "=r")
-        (fix:SI (match_operand:TF 1 "gpc_reg_operand" "d")))
+        (fix:SI (match_operand:IBM128 1 "gpc_reg_operand" "d")))
    (clobber (match_operand:DF 2 "gpc_reg_operand" "=d"))
    (clobber (match_operand:DF 3 "gpc_reg_operand" "=&d"))
    (clobber (match_operand:DI 4 "gpc_reg_operand" "=d"))
    (clobber (match_operand:DI 5 "offsettable_mem_operand" "=o"))]
-  "!TARGET_IEEEQUAD
-   && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_LONG_DOUBLE_128"
+  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_LONG_DOUBLE_128"
   "#"
   ""
   [(pc)]
 {
   rtx lowword;
-  emit_insn (gen_fix_trunc_helper (operands[2], operands[1], operands[3]));
+  emit_insn (gen_fix_trunc_helper<mode> (operands[2], operands[1],
+					 operands[3]));
 
   gcc_assert (MEM_P (operands[5]));
   lowword = adjust_address (operands[5], SImode, WORDS_BIG_ENDIAN ? 4 : 0);
@@ -6698,45 +6727,45 @@
   DONE;
 })
 
-(define_expand "fix_trunctfdi2"
-  [(set (match_operand:DI 0 "nonimmediate_operand" "")
-	(fix:DI (match_operand:TF 1 "gpc_reg_operand" "")))]
-  "TARGET_IEEEQUAD && TARGET_LONG_DOUBLE_128"
+(define_expand "fix_trunc<mode>di2"
+  [(set (match_operand:DI 0 "gpc_reg_operand" "")
+	(fix:DI (match_operand:IEEE128 1 "gpc_reg_operand" "")))]
+  "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "fixuns_trunctf<mode>2"
-  [(set (match_operand:SDI 0 "nonimmediate_operand" "")
-	(unsigned_fix:SDI (match_operand:TF 1 "gpc_reg_operand" "")))]
-  "TARGET_IEEEQUAD && TARGET_LONG_DOUBLE_128"
+(define_expand "fixuns_trunc<IEEE128:mode><SDI:mode>2"
+  [(set (match_operand:SDI 0 "gpc_reg_operand" "")
+	(unsigned_fix:SDI (match_operand:IEEE128 1 "gpc_reg_operand" "")))]
+  "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], true);
   DONE;
 })
 
-(define_expand "floatditf2"
-  [(set (match_operand:TF 0 "nonimmediate_operand" "")
-	(float:TF (match_operand:DI 1 "gpc_reg_operand" "")))]
-  "TARGET_IEEEQUAD && TARGET_LONG_DOUBLE_128"
+(define_expand "floatdi<mode>2"
+  [(set (match_operand:IEEE128 0 "gpc_reg_operand" "")
+	(float:IEEE128 (match_operand:DI 1 "gpc_reg_operand" "")))]
+  "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "floatuns<mode>tf2"
-  [(set (match_operand:TF 0 "nonimmediate_operand" "")
-	(unsigned_float:TF (match_operand:SDI 1 "gpc_reg_operand" "")))]
-  "TARGET_IEEEQUAD && TARGET_LONG_DOUBLE_128"
+(define_expand "floatuns<SDI:mode><IEEE128:mode>2"
+  [(set (match_operand:IEEE128 0 "gpc_reg_operand" "")
+	(unsigned_float:IEEE128 (match_operand:SDI 1 "gpc_reg_operand" "")))]
+  "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], true);
   DONE;
 })
 
 (define_expand "neg<mode>2"
-  [(set (match_operand:TFIFKF 0 "gpc_reg_operand" "")
-	(neg:TFIFKF (match_operand:TFIFKF 1 "gpc_reg_operand" "")))]
+  [(set (match_operand:FLOAT128 0 "gpc_reg_operand" "")
+	(neg:FLOAT128 (match_operand:FLOAT128 1 "gpc_reg_operand" "")))]
   "FLOAT128_IEEE_P (<MODE>mode)
    || (FLOAT128_IBM_P (<MODE>mode)
        && TARGET_HARD_FLOAT
@@ -6746,7 +6775,14 @@
   if (FLOAT128_IEEE_P (<MODE>mode))
     {
       if (TARGET_FLOAT128)
-	emit_insn (gen_ieee_128bit_vsx_neg<mode>2 (operands[0], operands[1]));
+	{
+	  if (<MODE>mode == TFmode)
+	    emit_insn (gen_ieee_128bit_vsx_negtf2 (operands[0], operands[1]));
+	  else if (<MODE>mode == KFmode)
+	    emit_insn (gen_ieee_128bit_vsx_negkf2 (operands[0], operands[1]));
+	  else
+	    gcc_unreachable ();
+	}
       else
 	{
 	  rtx libfunc = optab_libfunc (neg_optab, <MODE>mode);
@@ -6761,9 +6797,9 @@
     }
 }")
 
-(define_insn "negtf2_internal"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "=d")
-	(neg:TF (match_operand:TF 1 "gpc_reg_operand" "d")))]
+(define_insn "neg<mode>2_internal"
+  [(set (match_operand:IBM128 0 "gpc_reg_operand" "=d")
+	(neg:IBM128 (match_operand:IBM128 1 "gpc_reg_operand" "d")))]
   "TARGET_HARD_FLOAT && TARGET_FPRS && FLOAT128_IBM_P (TFmode)"
   "*
 {
@@ -6776,8 +6812,8 @@
    (set_attr "length" "8")])
 
 (define_expand "abs<mode>2"
-  [(set (match_operand:TFIFKF 0 "gpc_reg_operand" "")
-	(abs:TFIFKF (match_operand:TFIFKF 1 "gpc_reg_operand" "")))]
+  [(set (match_operand:FLOAT128 0 "gpc_reg_operand" "")
+	(abs:FLOAT128 (match_operand:FLOAT128 1 "gpc_reg_operand" "")))]
   "FLOAT128_IEEE_P (<MODE>mode)
    || (FLOAT128_IBM_P (<MODE>mode)
        && TARGET_HARD_FLOAT
@@ -6790,7 +6826,12 @@
     {
       if (TARGET_FLOAT128)
 	{
-	  emit_insn (gen_ieee_128bit_vsx_abs<mode>2 (operands[0], operands[1]));
+	  if (<MODE>mode == TFmode)
+	    emit_insn (gen_ieee_128bit_vsx_abstf2 (operands[0], operands[1]));
+	  else if (<MODE>mode == KFmode)
+	    emit_insn (gen_ieee_128bit_vsx_abskf2 (operands[0], operands[1]));
+	  else
+	    FAIL;
 	  DONE;
 	}
       else
@@ -6798,22 +6839,26 @@
     }
 
   label = gen_label_rtx ();
-  if (TARGET_E500_DOUBLE)
+  if (TARGET_E500_DOUBLE && <MODE>mode == TFmode)
     {
       if (flag_finite_math_only && !flag_trapping_math)
 	emit_insn (gen_spe_abstf2_tst (operands[0], operands[1], label));
       else
 	emit_insn (gen_spe_abstf2_cmp (operands[0], operands[1], label));
     }
-  else
+  else if (<MODE>mode == TFmode)
     emit_insn (gen_abstf2_internal (operands[0], operands[1], label));
+  else if (<MODE>mode == TFmode)
+    emit_insn (gen_absif2_internal (operands[0], operands[1], label));
+  else
+    FAIL;
   emit_label (label);
   DONE;
 }")
 
-(define_expand "abstf2_internal"
-  [(set (match_operand:TF 0 "gpc_reg_operand" "")
-	(match_operand:TF 1 "gpc_reg_operand" ""))
+(define_expand "abs<mode>2_internal"
+  [(set (match_operand:IBM128 0 "gpc_reg_operand" "")
+	(match_operand:IBM128 1 "gpc_reg_operand" ""))
    (set (match_dup 3) (match_dup 5))
    (set (match_dup 5) (abs:DF (match_dup 5)))
    (set (match_dup 4) (compare:CCFP (match_dup 3) (match_dup 5)))
@@ -6821,8 +6866,7 @@
 			   (label_ref (match_operand 2 "" ""))
 			   (pc)))
    (set (match_dup 6) (neg:DF (match_dup 6)))]
-  "!TARGET_IEEEQUAD
-   && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
+  "TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT 
    && TARGET_LONG_DOUBLE_128"
   "
 {
@@ -6830,8 +6874,8 @@
   const int lo_word = LONG_DOUBLE_LARGE_FIRST ? GET_MODE_SIZE (DFmode) : 0;
   operands[3] = gen_reg_rtx (DFmode);
   operands[4] = gen_reg_rtx (CCFPmode);
-  operands[5] = simplify_gen_subreg (DFmode, operands[0], TFmode, hi_word);
-  operands[6] = simplify_gen_subreg (DFmode, operands[0], TFmode, lo_word);
+  operands[5] = simplify_gen_subreg (DFmode, operands[0], <MODE>mode, hi_word);
+  operands[6] = simplify_gen_subreg (DFmode, operands[0], <MODE>mode, lo_word);
 }")
 
 
@@ -6864,14 +6908,14 @@
 ;; neg/abs to create the constant just once.
 
 (define_insn_and_split "ieee_128bit_vsx_neg<mode>2"
-  [(set (match_operand:TFIFKF 0 "register_operand" "=wa")
-	(neg:TFIFKF (match_operand:TFIFKF 1 "register_operand" "wa")))
+  [(set (match_operand:IEEE128 0 "register_operand" "=wa")
+	(neg:IEEE128 (match_operand:IEEE128 1 "register_operand" "wa")))
    (clobber (match_scratch:V16QI 2 "=v"))]
-  "TARGET_FLOAT128 && FLOAT128_IEEE_P (<MODE>mode)"
+  "TARGET_FLOAT128"
   "#"
   "&& 1"
   [(parallel [(set (match_dup 0)
-		   (neg:TFIFKF (match_dup 1)))
+		   (neg:IEEE128 (match_dup 1)))
 	      (use (match_dup 2))])]
 {
   if (GET_CODE (operands[2]) == SCRATCH)
@@ -6884,8 +6928,8 @@
    (set_attr "type" "vecsimple")])
 
 (define_insn "*ieee_128bit_vsx_neg<mode>2_internal"
-  [(set (match_operand:TFIFKF 0 "register_operand" "=wa")
-	(neg:TFIFKF (match_operand:TFIFKF 1 "register_operand" "wa")))
+  [(set (match_operand:IEEE128 0 "register_operand" "=wa")
+	(neg:IEEE128 (match_operand:IEEE128 1 "register_operand" "wa")))
    (use (match_operand:V16QI 2 "register_operand" "=v"))]
   "TARGET_FLOAT128"
   "xxlxor %x0,%x1,%x2"
@@ -6893,14 +6937,14 @@
 
 ;; IEEE 128-bit absolute value
 (define_insn_and_split "ieee_128bit_vsx_abs<mode>2"
-  [(set (match_operand:TFIFKF 0 "register_operand" "=wa")
-	(abs:TFIFKF (match_operand:TFIFKF 1 "register_operand" "wa")))
+  [(set (match_operand:IEEE128 0 "register_operand" "=wa")
+	(abs:IEEE128 (match_operand:IEEE128 1 "register_operand" "wa")))
    (clobber (match_scratch:V16QI 2 "=v"))]
   "TARGET_FLOAT128 && FLOAT128_IEEE_P (<MODE>mode)"
   "#"
   "&& 1"
   [(parallel [(set (match_dup 0)
-		   (abs:TFIFKF (match_dup 1)))
+		   (abs:IEEE128 (match_dup 1)))
 	      (use (match_dup 2))])]
 {
   if (GET_CODE (operands[2]) == SCRATCH)
@@ -6913,8 +6957,8 @@
    (set_attr "type" "vecsimple")])
 
 (define_insn "*ieee_128bit_vsx_abs<mode>2_internal"
-  [(set (match_operand:TFIFKF 0 "register_operand" "=wa")
-	(abs:TFIFKF (match_operand:TFIFKF 1 "register_operand" "wa")))
+  [(set (match_operand:IEEE128 0 "register_operand" "=wa")
+	(abs:IEEE128 (match_operand:IEEE128 1 "register_operand" "wa")))
    (use (match_operand:V16QI 2 "register_operand" "=v"))]
   "TARGET_FLOAT128"
   "xxlandc %x0,%x1,%x2"
@@ -6922,16 +6966,16 @@
 
 ;; IEEE 128-bit negative absolute value
 (define_insn_and_split "*ieee_128bit_vsx_nabs<mode>2"
-  [(set (match_operand:TFIFKF 0 "register_operand" "=wa")
-	(neg:TFIFKF
-	 (abs:TFIFKF
-	  (match_operand:TFIFKF 1 "register_operand" "wa"))))
+  [(set (match_operand:IEEE128 0 "register_operand" "=wa")
+	(neg:IEEE128
+	 (abs:IEEE128
+	  (match_operand:IEEE128 1 "register_operand" "wa"))))
    (clobber (match_scratch:V16QI 2 "=v"))]
   "TARGET_FLOAT128 && FLOAT128_IEEE_P (<MODE>mode)"
   "#"
   "&& 1"
   [(parallel [(set (match_dup 0)
-		   (abs:TFIFKF (match_dup 1)))
+		   (abs:IEEE128 (match_dup 1)))
 	      (use (match_dup 2))])]
 {
   if (GET_CODE (operands[2]) == SCRATCH)
@@ -6944,70 +6988,78 @@
    (set_attr "type" "vecsimple")])
 
 (define_insn "*ieee_128bit_vsx_nabs<mode>2_internal"
-  [(set (match_operand:TFIFKF 0 "register_operand" "=wa")
-	(neg:TFIFKF
-	 (abs:TFIFKF
-	  (match_operand:TFIFKF 1 "register_operand" "wa"))))
+  [(set (match_operand:IEEE128 0 "register_operand" "=wa")
+	(neg:IEEE128
+	 (abs:IEEE128
+	  (match_operand:IEEE128 1 "register_operand" "wa"))))
    (use (match_operand:V16QI 2 "register_operand" "=v"))]
   "TARGET_FLOAT128"
   "xxlor %x0,%x1,%x2"
   [(set_attr "type" "vecsimple")])
 
 ;; Float128 conversion functions.  These expand to library function calls.
+;; We use expand to convert from IBM double double to IEEE 128-bit
+;; and trunc for the opposite.
+(define_expand "extendiftf2"
+  [(set (match_operand:TF 0 "gpc_reg_operand" "")
+	(float_extend:TF (match_operand:IF 1 "gpc_reg_operand" "")))]
+  "TARGET_FLOAT128"
+{
+  rs6000_expand_float128_convert (operands[0], operands[1], false);
+  DONE;
+})
 
-(define_expand "extend<FLOAT128_SFDFTF:mode><IFKF:mode>2"
-  [(set (match_operand:IFKF 0 "nonimmediate_operand" "")
-	(float_extend:IFKF
-	 (match_operand:FLOAT128_SFDFTF 1 "gpc_reg_operand" "")))]
+(define_expand "extendifkf2"
+  [(set (match_operand:KF 0 "gpc_reg_operand" "")
+	(float_extend:KF (match_operand:IF 1 "gpc_reg_operand" "")))]
   "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "trunc<IFKF:mode><FLOAT128_SFDFTF:mode>2"
-  [(set (match_operand:FLOAT128_SFDFTF 0 "nonimmediate_operand" "")
-	(float_truncate:FLOAT128_SFDFTF
-	 (match_operand:IFKF 1 "gpc_reg_operand" "")))]
+(define_expand "extendtfkf2"
+  [(set (match_operand:KF 0 "gpc_reg_operand" "")
+	(float_extend:KF (match_operand:TF 1 "gpc_reg_operand" "")))]
   "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "fix_trunc<IFKF:mode><SDI:mode>2"
-  [(set (match_operand:SDI 0 "nonimmediate_operand" "")
-	(fix:SDI (match_operand:IFKF 1 "gpc_reg_operand" "")))]
+(define_expand "trunciftf2"
+  [(set (match_operand:IF 0 "gpc_reg_operand" "")
+	(float_truncate:IF (match_operand:TF 1 "gpc_reg_operand" "")))]
   "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "fixuns_trunc<IFKF:mode><SDI:mode>2"
-  [(set (match_operand:SDI 0 "nonimmediate_operand" "")
-	(unsigned_fix:SDI (match_operand:IFKF 1 "gpc_reg_operand" "")))]
+(define_expand "truncifkf2"
+  [(set (match_operand:IF 0 "gpc_reg_operand" "")
+	(float_truncate:IF (match_operand:KF 1 "gpc_reg_operand" "")))]
   "TARGET_FLOAT128"
 {
-  rs6000_expand_float128_convert (operands[0], operands[1], true);
+  rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "float<SDI:mode><IFKF:mode>2"
-  [(set (match_operand:IFKF 0 "nonimmediate_operand" "")
-	(float:KF (match_operand:SDI 1 "gpc_reg_operand" "")))]
+(define_expand "trunckftf2"
+  [(set (match_operand:TF 0 "gpc_reg_operand" "")
+	(float_truncate:TF (match_operand:KF 1 "gpc_reg_operand" "")))]
   "TARGET_FLOAT128"
 {
   rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
-(define_expand "floatuns<SDI:mode><IFKF:mode>2"
-  [(set (match_operand:IFKF 0 "nonimmediate_operand" "")
-	(unsigned_float:IFKF (match_operand:SDI 1 "gpc_reg_operand" "")))]
+(define_expand "trunctfif2"
+  [(set (match_operand:IF 0 "gpc_reg_operand" "")
+	(float_truncate:IF (match_operand:TF 1 "gpc_reg_operand" "")))]
   "TARGET_FLOAT128"
 {
-  rs6000_expand_float128_convert (operands[0], operands[1], true);
+  rs6000_expand_float128_convert (operands[0], operands[1], false);
   DONE;
 })
 
@@ -10524,20 +10576,20 @@
    (set (match_dup 0) (plus:SI (match_dup 1) (match_dup 4)))])
 
 ;; Only need to compare second words if first words equal
-(define_insn "*cmptf_internal1"
+(define_insn "*cmp<mode>_internal1"
   [(set (match_operand:CCFP 0 "cc_reg_operand" "=y")
-	(compare:CCFP (match_operand:TF 1 "gpc_reg_operand" "d")
-		      (match_operand:TF 2 "gpc_reg_operand" "d")))]
-  "!TARGET_IEEEQUAD && !TARGET_XL_COMPAT
+	(compare:CCFP (match_operand:IBM128 1 "gpc_reg_operand" "d")
+		      (match_operand:IBM128 2 "gpc_reg_operand" "d")))]
+  "!TARGET_XL_COMPAT && FLOAT128_IBM_P (<MODE>mode)
    && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LONG_DOUBLE_128"
   "fcmpu %0,%1,%2\;bne %0,$+8\;fcmpu %0,%L1,%L2"
   [(set_attr "type" "fpcompare")
    (set_attr "length" "12")])
 
-(define_insn_and_split "*cmptf_internal2"
+(define_insn_and_split "*cmp<mode>_internal2"
   [(set (match_operand:CCFP 0 "cc_reg_operand" "=y")
-	(compare:CCFP (match_operand:TF 1 "gpc_reg_operand" "d")
-		      (match_operand:TF 2 "gpc_reg_operand" "d")))
+	(compare:CCFP (match_operand:IBM128 1 "gpc_reg_operand" "d")
+		      (match_operand:IBM128 2 "gpc_reg_operand" "d")))
     (clobber (match_scratch:DF 3 "=d"))
     (clobber (match_scratch:DF 4 "=d"))
     (clobber (match_scratch:DF 5 "=d"))
@@ -10547,7 +10599,7 @@
     (clobber (match_scratch:DF 9 "=d"))
     (clobber (match_scratch:DF 10 "=d"))
     (clobber (match_scratch:GPR 11 "=b"))]
-  "!TARGET_IEEEQUAD && TARGET_XL_COMPAT
+  "TARGET_XL_COMPAT && FLOAT128_IBM_P (<MODE>mode)
    && TARGET_HARD_FLOAT && TARGET_FPRS && TARGET_DOUBLE_FLOAT && TARGET_LONG_DOUBLE_128"
   "#"
   "&& reload_completed"
@@ -10571,10 +10623,10 @@
   const int lo_word = LONG_DOUBLE_LARGE_FIRST ? GET_MODE_SIZE (DFmode) : 0;
   const int hi_word = LONG_DOUBLE_LARGE_FIRST ? 0 : GET_MODE_SIZE (DFmode);
 
-  operands[5] = simplify_gen_subreg (DFmode, operands[1], TFmode, hi_word);
-  operands[6] = simplify_gen_subreg (DFmode, operands[1], TFmode, lo_word);
-  operands[7] = simplify_gen_subreg (DFmode, operands[2], TFmode, hi_word);
-  operands[8] = simplify_gen_subreg (DFmode, operands[2], TFmode, lo_word);
+  operands[5] = simplify_gen_subreg (DFmode, operands[1], <MODE>mode, hi_word);
+  operands[6] = simplify_gen_subreg (DFmode, operands[1], <MODE>mode, lo_word);
+  operands[7] = simplify_gen_subreg (DFmode, operands[2], <MODE>mode, hi_word);
+  operands[8] = simplify_gen_subreg (DFmode, operands[2], <MODE>mode, lo_word);
   operands[12] = gen_label_rtx ();
   operands[13] = gen_label_rtx ();
   real_inf (&rv);
diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi
index fdb1547ccc8..9f8785eb145 100644
--- a/gcc/doc/extend.texi
+++ b/gcc/doc/extend.texi
@@ -929,6 +929,7 @@ examine and set these two fictitious variables with your debugger.
 @cindex additional floating types
 @cindex @code{__float80} data type
 @cindex @code{__float128} data type
+@cindex @code{__ibm128} data type
 @cindex @code{w} floating point suffix
 @cindex @code{q} floating point suffix
 @cindex @code{W} floating point suffix
@@ -941,19 +942,39 @@ Support for additional types includes the arithmetic operators:
 add, subtract, multiply, divide; unary arithmetic operators;
 relational operators; equality operators; and conversions to and from
 integer and other floating types.  Use a suffix @samp{w} or @samp{W}
-in a literal constant of type @code{__float80} and @samp{q} or @samp{Q}
-for @code{_float128}.  You can declare complex types using the
-corresponding internal complex type, @code{XCmode} for @code{__float80}
-type and @code{TCmode} for @code{__float128} type:
+in a literal constant of type @code{__float80} or type
+@code{__ibm128}.  Use a suffix @samp{q} or @samp{Q} for @code{_float128}.
+
+On the i386, x86_64, IA-64, and HP-UX targets, you can declare complex
+types using the corresponding internal complex type, @code{XCmode} for
+@code{__float80} type and @code{TCmode} for @code{__float128} type:
 
 @smallexample
 typedef _Complex float __attribute__((mode(TC))) _Complex128;
 typedef _Complex float __attribute__((mode(XC))) _Complex80;
 @end smallexample
 
+On PowerPC Linux, Freebsd and Darwin systems, the default for
+@code{long double} is to use the IBM extended floating point format
+that uses a pair of @code{double} values to extend the precision.
+This means that the mode @code{TCmode} was already used by the
+traditional IBM long double format, and you would need to use the mode
+@code{KCmode}:
+
+@smallexample
+typedef _Complex float __attribute__((mode(KC))) _Complex128;
+@end smallexample
+
 Not all targets support additional floating-point types.  @code{__float80}
 and @code{__float128} types are supported on x86 and IA-64 targets.
-The @code{__float128} type is supported on hppa HP-UX targets.
+The @code{__float128} type is supported on hppa HP-UX.
+The @code{__float128} type is supported on PowerPC systems by default
+if the vector scalar instruction set (VSX) is enabled.
+
+On the PowerPC, @code{__ibm128} provides access to the IBM extended
+double format, and it is intended to be used by the library functions
+that handle conversions if/when long double is changed to be IEEE
+128-bit floating point.
 
 @node Half-Precision
 @section Half-Precision Floating Point
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 76fdc31769d..70dc5d0b197 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -946,7 +946,8 @@ See RS/6000 and PowerPC Options.
 -mquad-memory-atomic -mno-quad-memory-atomic @gol
 -mcompat-align-parm -mno-compat-align-parm @gol
 -mupper-regs-df -mno-upper-regs-df -mupper-regs-sf -mno-upper-regs-sf @gol
--mupper-regs -mno-upper-regs}
+-mupper-regs -mno-upper-regs @gol
+-mfloat128 -mno-float128}
 
 @emph{RX Options}
 @gccoptlist{-m64bit-doubles  -m32bit-doubles  -fpu  -nofpu@gol
@@ -19525,6 +19526,17 @@ floating point register set, depending on the model of the machine.
 If the @option{-mno-upper-regs} option is used, it turns off both
 @option{-mupper-regs-sf} and @option{-mupper-regs-df} options.
 
+@item -mfloat128
+@itemx -mno-float128
+@opindex mfloat128
+@opindex mno-float128
+Enable/disable the @var{__float128} keyword for IEEE 128-bit floating point
+and use software emulation for IEEE 128-bit floating point.
+
+The VSX instruction set (@option{-mvsx}, @option{-mcpu=power7}, or
+@option{-mcpu=power8}) must be enabled to use the @option{-mfloat128}
+option.
+
 @item -mfloat-gprs=@var{yes/single/double/no}
 @itemx -mfloat-gprs
 @opindex mfloat-gprs
diff --git a/gcc/testsuite/ChangeLog b/gcc/testsuite/ChangeLog
index 0ec41025170..eea5a20f335 100644
--- a/gcc/testsuite/ChangeLog
+++ b/gcc/testsuite/ChangeLog
@@ -1,3 +1,9 @@
+2015-10-29  Michael Meissner  <meissner@linux.vnet.ibm.com>
+
+	* gcc.target/powerpc/float128-call.c: New test for -mfloat128 on
+	PowerPC.
+	* gcc.target/powerpc/float128-mix.c: Likewise.
+
 2015-10-29  Steven G. Kargl  <kargl@gcc.gnu.org>
 
 	PR fortran/68054
diff --git a/gcc/testsuite/gcc.target/powerpc/float128-call.c b/gcc/testsuite/gcc.target/powerpc/float128-call.c
new file mode 100644
index 00000000000..382b2926e90
--- /dev/null
+++ b/gcc/testsuite/gcc.target/powerpc/float128-call.c
@@ -0,0 +1,27 @@
+/* { dg-do compile { target { powerpc*-*-linux* } } } */
+/* { dg-skip-if "" { powerpc*-*-darwin* } { "*" } { "" } } */
+/* { dg-require-effective-target powerpc_vsx_ok } */
+/* { dg-skip-if "do not override -mcpu" { powerpc*-*-* } { "-mcpu=*" } { "-mcpu=power7" } } */
+/* { dg-options "-O2 -mcpu=power7 -mfloat128 -mno-regnames" } */
+
+#ifndef __FLOAT128__
+#error "-mfloat128 is not supported."
+#endif
+
+#ifdef __LONG_DOUBLE_IEEE128__
+#define TYPE long double
+#define ONE  1.0L
+
+#else
+#define TYPE __float128
+#define ONE  1.0Q
+#endif
+
+/* Test to make sure vector registers are used for passing IEEE 128-bit
+   floating point values and returning them. Also make sure the 'q' suffix is
+   handled.  */
+TYPE one (void) { return ONE; }
+void store (TYPE a, TYPE *p) { *p = a; }
+
+/* { dg-final { scan-assembler "lxvd2x 34"  } } */
+/* { dg-final { scan-assembler "stxvd2x 34" } } */
diff --git a/gcc/testsuite/gcc.target/powerpc/float128-mix.c b/gcc/testsuite/gcc.target/powerpc/float128-mix.c
new file mode 100644
index 00000000000..466be907524
--- /dev/null
+++ b/gcc/testsuite/gcc.target/powerpc/float128-mix.c
@@ -0,0 +1,17 @@
+/* { dg-do compile { target { powerpc*-*-linux* } } } */
+/* { dg-skip-if "" { powerpc*-*-darwin* } { "*" } { "" } } */
+/* { dg-require-effective-target powerpc_vsx_ok } */
+/* { dg-skip-if "do not override -mcpu" { powerpc*-*-* } { "-mcpu=*" } { "-mcpu=power7" } } */
+/* { dg-options "-O2 -mcpu=power7 -mfloat128" } */
+
+
+/* Test to make sure that __float128 and __ibm128 cannot be combined together.  */
+__float128 add (__float128 a, __ibm128 b)
+{
+  return a+b;	/* { dg-error "__float128 and __ibm128 cannot be used in the same expression" "" } */
+}
+
+__ibm128 sub (__ibm128 a, __float128 b)
+{
+  return a-b;	/* { dg-error "__float128 and __ibm128 cannot be used in the same expression" "" } */
+}