REXML gem specifies ruby > 0 but does not support < 2

[alextwoods]

Since rexml has been moved from a default to a bundled gem in Ruby 3.0, we are working on adding it as an explicit dependency for the aws-sdk-ruby. However, adding it as a dependency breaks in Ruby 1.9.3 (anything < 2) because some of the code uses syntax that is incompatible with 1.9 (example commit where incompatibility was introduced).

Would it be possible to either:

1. Specify supported ruby versions in the gemspec or
2. Fix the syntax (use of the double splat) to ensure compatability

[mullermp]

@hsbt @kou @nurse Any opinion or recommendation on this? The AWS SDK unfortunately still supports Ruby 1.9.3.

[hsbt]

There is no good solution yet. Because solution.1 is not resolve this issue. The current Gem::Specification didn't have ruby version selector for runtime dependency.

Unfortunately, We can choose only solution.2.

[mullermp]

My understanding is that (following this commit) we can simply change XPathParser to be initialized with XPathParser.new(options) and change the definition to be something like:

def initialize(options = {})
   ...
  @strict = options[:strict] || false
end

Would you agree? If you're able to handle this, I can leave it to you, otherwise, we're happy to create a starter PR, hopefully there isn't other code that needs these kinds of changes..

[kou]

Does REXML 3.1.7.3 work on Ruby 1.9.3?

BTW, why do you want to support Ruby 1.9.3?

[kou]

You'll be able to disable gemified REXML on Ruby 1.9.3 by the following change:

diff --git a/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser.rb b/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser.rb
index 72b43b26d..dd11a527e 100644
--- a/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser.rb
+++ b/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser.rb
@@ -82,7 +82,7 @@ module Aws
 
         def try_load_engine(name)
           load_engine(name)
-        rescue LoadError
+        rescue LoadError, SyntaxError
           false
         end
 

[kou]

Or you will be able to force using system REXML on Ruby 1.9.3:

diff --git a/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser/engines/rexml.rb b/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser/engines/rexml.rb
index e491364df..1154094de 100644
--- a/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser/engines/rexml.rb
+++ b/gems/aws-sdk-core/lib/aws-sdk-core/xml/parser/engines/rexml.rb
@@ -1,7 +1,13 @@
 # frozen_string_literal: true
 
+use_system_rexml = ((RUBY_VERSION <=> "2.0.0") < 0)
+if use_system_rexml
+  require "rbconfig"
+  $LOAD_PATH.unshift(RbConfig::CONFIG["rubylibdir"])
+end
 require 'rexml/document'
 require 'rexml/streamlistener'
+$LOAD_PATH.shift if use_system_rexml
 
 module Aws
   module Xml

[mullermp]

BTW, why do you want to support Ruby 1.9.3?

We're unfortunately committed to doing so, by some stakeholders, as well as our maintenance policy which suggests we should only drop runtime versions in our next SDK major version.

As far as your suggestions, thank you for providing those! We will try these shortly.

[alextwoods]

Using the oldest published rexml (3.1.7.3) does not work and gives the same errors:

SyntaxError: /usr/share/rvm/gems/ruby-1.9.3-p551/gems/rexml-3.1.7.3/lib/rexml/xpath.rb:34: syntax error, unexpected tPOW, expecting ')'
      parser = XPathParser.new(**options)
                                 ^

The first of @kou proposed changes (adding SyntaxError to the exceptions that are caught) may break existing customers that were relying on rexml as their xml engine. However, the second change (using system rexml for ruby <2) does work and is backwards compatible.

However, the question for me is whether that is the right change to make, or if we should ensure that rexml supports the rubies that its gemspec defines (ie, all of them). I'm not a big fan of using old syntax to ensure support for EOL rubies, but because the gemspec never specified rubies, I'd argue for using the older syntax in rexml to ensure support and if future breaking changes are made, setting the minimum ruby version in the gemspec.

[alextwoods]

As an update here: we have decided to not add an explicit dependency on rexml and add the patch to use system rexml for Ruby version < 2, since many other gems, eg crack) are adding explicit dependencies.

It looks like there are other unreleased changes in master (ie: #37 ) that further break compatability with older rubies. I'd recommend updating the gemspec to specify the minimum supported ruby version (with the use of refine I believe that now would be 2.1).

[kou]

We don't want to care about EOL-ed Rubies to reduce maintenance cost.
We'll add Ruby version constrain explicitly if we drop support for not EOL-ed Rubies.

It's better that we improve RubyGems to add support for specifying dependencies for each Ruby version. It'll be useful for a similar case in the future.
Python's pip support it: https://pip.pypa.io/en/stable/reference/pip_install/#requirement-specifiers

If you're interested in the improvement, could you open an issue at https://github.com/rubygems/rubygems/issues ?

[mullermp]

We don't want to care about EOL-ed Rubies to reduce maintenance cost.

I totally agree with this. However, shouldn't we work within the current limitations of gemspecs, and shouldn't the gemspec be reliable? Any user of Ruby 1.9 or 2.0 would not receive an error on gem install. If rexml is included with stdlib up until Ruby 3, could we not declare that this gem requires Ruby >= 3?

[kou]

Any user of Ruby 1.9 or 2.0

We don't want to care about EOL-ed Rubies...

could we not declare that this gem requires Ruby >= 3?

We can use this with all maintained Rubies such as Ruby 2.7.