test_ssl test_options fails on ubuntu 16.04

[zhangyangyu]

BPO	26867
Nosy	@doko42, @pitrou, @larryhastings, @giampaolo, @tiran, @benjaminp, @ned-deily, @alex, @vadmium, @dstufft, @zhangyangyu
PRs	[2.7] bpo-26867: Ubuntu's openssl OP_NO_SSLv3 is forced on by default; fix test. #374

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/tiran'
closed_at = <Date 2017-03-01.07:36:50.318>
created_at = <Date 2016-04-27.03:34:14.203>
labels = ['expert-SSL', 'type-bug', 'library']
title = 'test_ssl test_options fails on ubuntu 16.04'
updated_at = <Date 2017-03-24.23:31:27.560>
user = 'https://github.com/zhangyangyu'

bugs.python.org fields:

activity = <Date 2017-03-24.23:31:27.560>
actor = 'xiang.zhang'
assignee = 'christian.heimes'
closed = True
closed_date = <Date 2017-03-01.07:36:50.318>
closer = 'xiang.zhang'
components = ['Library (Lib)', 'SSL']
creation = <Date 2016-04-27.03:34:14.203>
creator = 'xiang.zhang'
dependencies = []
files = []
hgrepos = []
issue_num = 26867
keywords = []
message_count = 20.0
messages = ['264342', '264414', '264417', '267472', '267485', '267488', '267503', '268322', '268325', '268327', '268425', '268426', '268706', '269165', '269216', '269261', '269341', '270154', '284713', '290355']
nosy_count = 14.0
nosy_names = ['doko', 'janssen', 'pitrou', 'larry', 'giampaolo.rodola', 'christian.heimes', 'benjamin.peterson', 'ned.deily', 'alex', 'SilentGhost', 'python-dev', 'martin.panter', 'dstufft', 'xiang.zhang']
pr_nums = ['374']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26867'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[zhangyangyu]

test_options in test_ssl fails on Ubuntu 16.04. I don't know this is due to the newest ubuntu or a recent code change. But I checkout revision 90000 and then rebuild and test, test_option still fails.

The traceback is:

FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/angwer/cpython/Lib/test/test_ssl.py", line 847, in test_options
    self.assertEqual(0, ctx.options)
AssertionError: 0 != 33554432

[zhangyangyu]

After some test, I think the reason causing this error is due to SSL_CTX_clear_options.

With OPENSSL_VERSION_NUMBER 268443775, SSL_CTX_clear_options(self->ctx, 2248147967) returns 33554432, where SSL_CTX_get_options returns 2248147967. From the manpage of SSL_CTX_clear_options, it seems it should return 0.

[zhangyangyu]

From the source code (get from apt-get source) of openssl-1.0.2g, I find

SSL_CTX_clear_options(ctx, op):
op &= ~SSL_OP_NO_SSLv3
return (ctx->options &= ~op)
SSL_CTX_set_options(ctx, op):
op |= SSL_OP_NO_SSLv3
return (ctx->options |= op)

which differs from the official code repos:

SSL_CTX_clear_options(ctx, op):
return (ctx->options &= ~op)
SSL_CTX_set_options(ctx, op):
return (ctx->options |= op)

This difference is introduced by debian-specific patch:

 case SSL_CTRL_OPTIONS:

+ larg|=SSL_OP_NO_SSLv3;
return (ctx->options |= larg);
case SSL_CTRL_CLEAR_OPTIONS:
+ larg&=~SSL_OP_NO_SSLv3;
return (ctx->options &= ~larg);

[ned-deily]

Can we close this as an Ubuntu-specific problem?

[vadmium]

This test is already decorated with @skip_if_broken_ubuntu_ssl. I’m not sure Python should go too far out of its way to handle downstream patches, but it seems there is a precedent here.

[ned-deily]

I just spoke with @Doko about this here at PyCon. I think we came to the conclusion it might be time to consider removing the old @skip_if_broken_ubuntu_ssl decorator and focus on making the tests work with the most recent releases since pretty much every distributor and current Python releases have moved to disabling the old compromised ssl/tls versions.

[zhangyangyu]

@skip_if_broken_ubuntu_ssl doesn't work in this case. hasattr(ssl, 'PROTOCOL_SSLv2') returns False.

[larryhastings]

I got this when testing 3.5.2rc1 on my Ubuntu 16.04 machine. CAs Xiang Zhang showed, this is Ubuntu doing something crazy. I ignored the failure and shipped 3.5.2rc1, however I would be interested in suppressing the test for 3.5.2 final. That way it has a chance of passing the whole test suite on user's Linux machines...!

[doko42]

ubuntu doesn't do anything crazy, but just disables oldish, deprecated und probably now unsecure ssl protocols. This is done by other vendors as well. From my point of of view this skip_if_ubuntu stuff should be replaced by proper feature tests. I'll see if I can come up with another work around.

[larryhastings]

This still affects 3.4 and 3.5. It'd be lovely if it could be fixed in all the still-alive versions. (Yes, this is technically a "bug fix", but I'd still like it fixed in 3.4.)

[doko42]

Description: properly handle Ubuntu's openssl having OP_NO_SSLv3 forced on by default
Author: Marc Deslauriers <marc.deslauriers@canonical.com>
Forwarded: yes, http://bugs.python.org/issue25724

Index: b/Lib/test/test_ssl.py
===================================================================

--- a/Lib/test/test_ssl.py
+++ b/Lib/test/test_ssl.py
@@ -821,7 +821,8 @@ class ContextTests(unittest.TestCase):
             self.assertEqual(ssl.OP_ALL | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv3,
                              ctx.options)
             ctx.options = 0
-            self.assertEqual(0, ctx.options)
+            # Ubuntu has OP_NO_SSLv3 forced on by default
+            self.assertEqual(0, ctx.options & ~ssl.OP_NO_SSLv3)
         else:
             with self.assertRaises(ValueError):
                 ctx.options = 0

[larryhastings]

That does seem like it'd make the test failure go away. But the fix seems a little Ubuntu-specific. Is it reasonable to do that when testing on every platform?

[vadmium]

FWIW I imagine Ubuntu overriding the option will break the example code in the documentation of clearing SSL_OP_NO_SSLv3: <https://docs.python.org/3.5/library/ssl.html#ssl.create_default_context\>. If we keep that documentation, I think we should continue to test that clearing the option works, which conflicts with the proposed patch.