Merge #7070

7070: Improve cross repo integration r=deivid-rodriguez a=deivid-rodriguez

### What was the end-user problem that led to this PR?

The problem was that we had on `rubygems`, we're not testing bundler against the rubygems checkout of each PR, but against master. That is confusing because developers assume that the PRs tests will run against the PRs code, but this is currently not true.

### What was your diagnosis of the problem?

My diagnosis was that on rubygems we are [specifying `RGV=master`](https://github.com/rubygems/rubygems/blob/fa6e547330e97b417ed11f262d68d03c57abeeda/.travis.yml#L12) in our TravisCI matrix. And what `bundler` does with that is to clone rubygems master under `tmp/rubygems` and use that. Thus your rubygems PR is not really tested.

### What is your fix for the problem, implemented in this PR?

My fix is to rework the setup to accept an `RGV` environment variable when it contains. In that case, no git operations will be performed, and that path will be used directly and assumed to contain a rubygems checkout.

I also refactored a few things about the setup while at it.

### Why did you choose this fix out of the possible options?

There were existing tasks to test bundler against a rubygems checkout, but they were unused and they expected a `RG` environment variable instead. I decided to reuse these tasks, but make them pick up the `RGV` as long as it contains a valid path. I chose this fix because it works and it doesn't make the existing setup too different.

Co-authored-by: David Rodríguez <deivid.rodriguez@riseup.net>

5 files changed, 29 insertions, 42 deletions

diff --git a/.travis.yml b/.travis.yml
index c87c1305c9..673a6ad7b5 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,7 +5,6 @@ before_script:
   - travis_retry rake -E 'module ::Bundler; VERSION = "0.0.0"; end' override_version
   - travis_retry rake spec:travis:deps
   - travis_retry rake man:build
-  - travis_retry rake spec:rubygems:clone_rubygems_$RGV
 
 branches:
   only:
diff --git a/Rakefile b/Rakefile
index 53830aebc0..2b038c1127 100644
--- a/Rakefile
+++ b/Rakefile
@@ -3,8 +3,7 @@
 $:.unshift File.expand_path("../lib", __FILE__)
 require "benchmark"
 
-NULL_DEVICE = (Gem.win_platform? ? "NUL" : "/dev/null")
-RUBYGEMS_REPO = if `git -C "#{File.expand_path("..")}" remote --verbose 2> #{NULL_DEVICE}` =~ /rubygems/i
+RUBYGEMS_REPO = if `git -C "#{File.expand_path("..")}" remote --verbose 2> #{IO::NULL}` =~ /rubygems/i
   File.expand_path("..")
 else
   File.expand_path("tmp/rubygems")
@@ -115,14 +114,13 @@ namespace :spec do
 
   # RubyGems specs by version
   namespace :rubygems do
-    rubyopt = ENV["RUBYOPT"]
     # When editing this list, also edit .travis.yml!
     branches = %w[master]
     releases = %w[v2.5.2 v2.6.14 v2.7.9 v3.0.3]
     (branches + releases).each do |rg|
       desc "Run specs with RubyGems #{rg}"
       task rg do
-        sh("bin/rspec --format progress --color")
+        sh("bin/rspec --format progress")
       end
 
       # Create tasks like spec:rubygems:v1.8.3:sudo to run the sudo specs
@@ -131,43 +129,26 @@ namespace :spec do
         task :realworld => ["set_realworld", rg]
       end
 
-      task "clone_rubygems_#{rg}" do
-        unless File.directory?(RUBYGEMS_REPO)
-          system("git clone https://github.com/rubygems/rubygems.git tmp/rubygems")
-        end
-        hash = nil
-
-        if RUBYGEMS_REPO.start_with?(Dir.pwd)
-          Dir.chdir(RUBYGEMS_REPO) do
-            system("git remote update")
-            if rg == "master"
-              system("git checkout origin/master")
-            else
-              system("git checkout #{rg}") || raise("Unknown RubyGems ref #{rg}")
-            end
-            hash = `git rev-parse HEAD`.chomp
-          end
-        elsif rg != "master"
-          raise "need to be running against master with bundler as a submodule"
-        end
-
-        puts "Checked out rubygems '#{rg}' at #{hash}"
+      task "set_#{rg}" do
         ENV["RGV"] = rg
       end
 
-      task rg => ["clone_rubygems_#{rg}"]
+      task rg => ["set_#{rg}"]
       task "rubygems:all" => rg
     end
 
-    desc "Run specs under a RubyGems checkout (set RG=path)"
+    desc "Run specs under a RubyGems checkout (set RGV=path)"
     task "co" do
-      sh("bin/rspec")
+      sh("bin/rspec --format progress")
+    end
+
+    namespace "co" do
+      task :sudo => ["set_sudo", "co", "clean_sudo"]
+      task :realworld => ["set_realworld", "co"]
     end
 
     task "setup_co" do
-      rg = File.expand_path ENV["RG"]
-      puts "Running specs against RubyGems in #{rg}..."
-      ENV["RUBYOPT"] = "-I#{rg} #{rubyopt}"
+      ENV["RGV"] = RUBYGEMS_REPO
     end
 
     task "co" => "setup_co"
@@ -178,6 +159,8 @@ namespace :spec do
   task :travis do
     rg = ENV["RGV"] || raise("RubyGems version is required on Travis!")
 
+    rg = "co" if File.directory?(File.expand_path(ENV["RGV"]))
+
     # disallow making network requests on CI
     ENV["BUNDLER_SPEC_PRE_RECORDED"] = "TRUE"
 
diff --git a/bin/with_rubygems b/bin/with_rubygems
index d16c289822..72edc1c928 100755
--- a/bin/with_rubygems
+++ b/bin/with_rubygems
@@ -9,15 +9,21 @@ def run(*cmd)
 end
 
 version = ENV.delete("RGV")
-rubygems_path = Pathname.new(__FILE__).join("../../tmp/rubygems").expand_path
+rubygems_path = Pathname.new(version).expand_path
 unless rubygems_path.directory?
-  rubygems_path.parent.mkpath unless rubygems_path.directory?
-  run("git", "clone", "https://github.com/rubygems/rubygems.git", rubygems_path.to_s)
+  rubygems_path = Pathname.new("tmp/rubygems").expand_path
+  unless rubygems_path.directory?
+    rubygems_path.parent.mkpath unless rubygems_path.directory?
+    run("git", "clone", "https://github.com/rubygems/rubygems.git", rubygems_path.to_s)
+  end
+  Dir.chdir(rubygems_path) do
+    run("git remote update")
+    version = "v#{version}" if version =~ /\A\d/
+    run("git", "checkout", version, "--quiet")
+    hash = `git rev-parse HEAD`.chomp
+    puts "Checked out rubygems '#{version}' at #{hash}"
+  end
 end
-Dir.chdir(rubygems_path) do
-  version = "v#{version}" if version =~ /\A\d/
-  run("git", "checkout", version, "--quiet")
-end if version
 
 rubygems_lib = rubygems_path + "lib"
 ENV["RUBYOPT"] = %(-I#{rubygems_lib} #{ENV["RUBYOPT"]})
diff --git a/spec/install/security_policy_spec.rb b/spec/install/security_policy_spec.rb
index 6e94c3f7ce..ad76162aac 100644
--- a/spec/install/security_policy_spec.rb
+++ b/spec/install/security_policy_spec.rb
@@ -32,8 +32,7 @@ RSpec.describe "policies with unsigned gems" do
     expect(err).to include("security policy didn't allow")
   end
 
-  # This spec will fail on RubyGems 2 rc1 due to a bug in policy.rb. the bug is fixed in rc3.
-  it "will fail with Medium Security setting due to presence of unsigned gem", :unless => ENV["RGV"] == "v2.0.0.rc.1" do
+  it "will fail with Medium Security setting due to presence of unsigned gem" do
     bundle "install --trust-policy=MediumSecurity"
     expect(err).to include("security policy didn't allow")
   end
diff --git a/spec/runtime/setup_spec.rb b/spec/runtime/setup_spec.rb
index d0bc44ca98..87ef96565d 100644
--- a/spec/runtime/setup_spec.rb
+++ b/spec/runtime/setup_spec.rb
@@ -1213,7 +1213,7 @@ end
 
     describe "default gem activation" do
       let(:exemptions) do
-        if Gem::Version.new(Gem::VERSION) >= Gem::Version.new("2.7") || ENV["RGV"] == "master"
+        if Gem::Version.new(Gem::VERSION) >= Gem::Version.new("2.7")
           []
         else
           %w[io-console openssl]