Bundler should give a better error when the Gemfile itself activates a conflicting gem

[Bandes]

I have been running into a strange situation which I believe is due to differing versions of the built-in gem set in different docker images.

My application runs on ruby 3.0.6 currently. My CI is CircleCI, and I use the cimg/ruby:3.0.6-browsers which appears to use set:1.0.3 However my devops team is running my application on Docker, using ruby:3.0.6alpine3.16 which appears to use set:1.0.1

Whichever of those versions I specify in Gemfile.lock will cause an error similar to this:

`check_for_activated_spec!': You have already activated set 1.0.1, but your Gemfile requires set 1.0.3. Since set is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports set as a default gem. (Gem::LoadError)

Similar things have happened to me on other projects that use more up-to-date versions of ruby.

I would have expected that bundler would be able to force the use of the version of the gem in Gemfile.lock, it hadn't occurred to me that I would need to inspect which versions were built-in to my images.

[deivid-rodriguez]

We should be supporting gemfied set since Bundler 2.2.8, which first included #4297. Would it be possible that you are using an older Bundler?

[Bandes]

I don't think so. My Gemfile.lock was built with 2.4.22

RUBY VERSION
   ruby 3.0.6p216

BUNDLED WITH
   2.4.22

[deivid-rodriguez]

Not a good guess then, thanks for confirming.

Please provide repro steps so that we can take a look at this.

[Bandes]

I'm having trouble finding ways to describe how to reproduce the issue without linking to our specific code or our specific images. I do have this which shows the docker versions we are using - the base image has 2.2.33 which we are updating to 2.4.22 before we bundle install

╰─ docker run --rm -it public.ecr.aws/docker/library/ruby:3.0.6-alpine3.16 sh                                                                                                                                                                                                                                                                                          ─╯
/ # bundle --version
Bundler version 2.2.33
/ # gem install bundler:2.4.22
Fetching bundler-2.4.22.gem
Successfully installed bundler-2.4.22
1 gem installed
/ # bundle --version
Bundler version 2.4.22
/ # gem list | grep set
set (default: 1.0.1)

[deivid-rodriguez]

Maybe extracting only dependency manifests to a separate repository, removing your private code and private gems, and see if it still reproduces?

[indirect]

I have suddenly started seeing this problem on a new machine, but I'm having trouble creating another copy of the repo with the same problem. The symptom I can observe is this error during Bundler.setup:

❯ b ruby -e "puts 'hi'"
/Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated date 3.3.4, but your Gemfile requires date 3.3.3. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:25:in `block in setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `map'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler.rb:162:in `setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `block in <top (required)>'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:159:in `with_level'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:111:in `silence'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `<top (required)>'
        from <internal:/Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems.rb:1370:in `<top (required)>'
        from <internal:gem_prelude>:2:in `require'
        from <internal:gem_prelude>:2:in `<internal:gem_prelude>'

The error only appears when installing to system gems, so I can make it appear and disappear repeatedly by running bundle config path .bundle (gone) and then bundle config unset path (error returns).

My system gems include both the locked version of date and a newer version:

❯ gem list date

*** LOCAL GEMS ***

date (3.3.4, default: 3.3.3)

[indirect]

oh no 🤦🏻 I figured it out. my Rails app contains a (relatively) complicated gemfile that loads some ruby to evaluate whether to include other gems. The "configuration" class has this at the top:

require "erb"
require "open-uri"
require "psych"
require "set"

Turns out open-uri requires time, which requires date. So ultimately what this boils down to is exactly what the error message says: somehow, you have a newer version of this gem installed, and merely evaluating your Gemfile has required a version that does not match your lock. So Bundler explodes when it gets around to trying to load your locked version of the gem (whether that gem is set or date).

We should probably turn this into one of the advanced Bundler error messages, and say something like this:

Your Gemfile is locked to date version 3.3.3. When Bundler tried to require date, it discovered that other Ruby code had already required date version 3.3.4. As a result, it is impossible to set up your Gemfile.

This is most likely caused by your Gemfile running Ruby code that requires gems, which is not supported. You will need to remove whatever is inside your Gemfile requiring gems, so that Bundler can take care of loading the versions you need.

Maybe this is also an opportunity for us to overwrite require to raise an exception while we eval Gemfiles? At least then it would fail early, and let users know exactly where the bad code is. 😅 cc @deivid-rodriguez @martinemde

[indirect]

The good news is that figuring all of this out suggests a workaround:

1. bundle config path .bundle to create a separate space for gems just for the given app.
2. bundle install to install gems just for this app, without any newer gems available to be required while evaluating the gemfile.
3. bundle clean after every bundle update, to make sure that newer/older gems are not also installed into the separate app gem space to be picked up by the require lines in the gemfile.

I have tested this on my example app, and it seems to work.

[deivid-rodriguez]

The idea of giving a better error message when a require fails with this error during Gemfile evaluation makes sense to me 👍.

[martinemde]

I made a branch with a failing test for this case. I'm not confident about its approach, but it does trigger the error: https://github.com/rubygems/rubygems/compare/require-conflict-in-gemfile

[Bandes]

Hi this repo https://github.com/modolabs/bundler-set-error-example should demonstrate the issue I am having. It seems like it might have something to do with spring because if I remove those gems from the gemfile I can no longer reproduce the issue. It might also be a problem with the dockerfile, but I'm not sure.

If these commands are executed in that repo, the error will occur for me

docker build --no-cache --build-arg BUNDLE_WITHOUT=nil . -t test-project
docker run -t test-project bundle exec rails runner '1+1'

[deivid-rodriguez]

In your case, you're using a very old spring version that does not include rails/spring#659, that's why you run into this error. If you upgrade (or remove) spring, you should be fine!

[stanhu]

This might be a separate issue, but I ran into a similar issue with the uri gem that is loaded by RubyGems only when:

1. A local gemspec is used.
2. The gemspec has a home page URL: https://github.com/ruby/ruby/blob/10b9679fa61f40c08ec28ec34251f7ec8648054d/lib/rubygems/specification_policy.rb#L430.

Consider this Gemfile:

source 'https://rubygems.org'

gem 'uri', '= 0.12.2'
gem 'test', path: 'test'

In test/test.gemspec:

# frozen_string_literal: true

Gem::Specification.new do |s|
  s.name = "test"
  s.version = "1.0.0"
  s.summary = "test"
  s.authors = ['John Doe']
  s.homepage = 'https://example.com'
end

If I simply run bundle exec irb, I see:

/Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated uri 0.13.0, but your Gemfile requires uri 0.12.2. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:25:in `block in setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `map'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler.rb:162:in `setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `block in <top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:159:in `with_level'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:111:in `silence'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `<top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:56:in `require_relative'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:56:in `kernel_load'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:23:in `run'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:492:in `exec'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:34:in `dispatch'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:28:in `start'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/exe/bundle:37:in `block in <top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/exe/bundle:29:in `<top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `load'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `<main>'

In this example, the user doesn't have a whole lot of control over the loading of uri.

The only workaround I can see is to update uri to v0.13.0.