Supporting native backup facility of SQLite

[lelit]

BPO	27645
Nosy	@vstinner, @cedk, @bitdancer, @berkerpeksag, @serhiy-storchaka, @lelit, @Vgr255, @AnishShah, @palaviv
PRs	bpo-27645: Supporting native backup facility of SQLite #377 bpo-27645: Supporting native backup facility of SQLite #4238 [3.7] bpo-27645: Add support for native backup facility of SQLite (GH-4238) #6064 bpo-27645: Skip test_bad_target_in_transaction if SQLite == 3.8.7.1 #6067 [3.7] bpo-27645: Skip test_bad_target_in_transaction if SQLite == 3.8.7.1 (GH-6067) #6068 bpo-27645: Fix sqlite backup check if database in transaction to be up to 3.8.7 (inclusive) #6131 [3.7] bpo-27645: Fix version number in 'database in transaction' fallback (GH-6131) #6138 bpo-27645, sqlite: Fix integer overflow on sleep #6594 [3.7] bpo-9566: Fix compiler warnings on Windows #12920
Files	issue27645.patch: Preliminar patch issue27645-doc.patch: Documentation

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-03-18.20:01:03.020>
created_at = <Date 2016-07-28.17:48:18.134>
labels = ['extension-modules', '3.8', 'type-feature', '3.7']
title = 'Supporting native backup facility of SQLite'
updated_at = <Date 2019-04-23.08:26:14.694>
user = 'https://github.com/lelit'

bugs.python.org fields:

activity = <Date 2019-04-23.08:26:14.694>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-03-18.20:01:03.020>
closer = 'berker.peksag'
components = ['Extension Modules']
creation = <Date 2016-07-28.17:48:18.134>
creator = 'lelit'
dependencies = []
files = ['43928', '43930']
hgrepos = []
issue_num = 27645
keywords = ['patch']
message_count = 44.0
messages = ['271574', '271575', '271576', '271578', '271580', '271582', '271584', '271676', '272848', '288632', '288723', '288750', '295455', '295472', '298868', '303782', '303789', '304789', '304894', '305444', '307647', '307840', '308022', '308068', '308073', '310669', '313558', '313560', '313561', '313562', '313563', '313564', '313569', '313570', '313571', '313635', '313991', '313994', '313995', '314018', '314026', '314052', '315946', '340701']
nosy_count = 10.0
nosy_names = ['ghaering', 'vstinner', 'ced', 'r.david.murray', 'berker.peksag', 'serhiy.storchaka', 'lelit', 'abarry', 'anish.shah', 'palaviv']
pr_nums = ['377', '4238', '6064', '6067', '6068', '6131', '6138', '6594', '12920']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27645'
versions = ['Python 3.7', 'Python 3.8']

[lelit]

It would be nice if the sqlite3 stdlib module could expose the SQLite Online Backup API.

I'm willing to implement it, as encouraged by Paul Moore.

See also: https://mail.python.org/pipermail/python-dev/2016-July/145570.html

[lelit]

Here is a preliminary implementation: lelit@b7456eb

[Vgr255]

That's really nice, thank you for doing this! To get your code reviewed, though, you should upload a patch here. With git, you can do 'git diff > my_patch_file_name.patch' in your local repo, then you can upload the resulting file here. If you haven't done so yet, you'll need to sign a contributor agreement as well: https://www.python.org/psf/contrib/contrib-form/ :)

(Also, you might want to remove the 'index ...' lines in the patch file that git generates for each modified file; since there are three files, there should be three lines to remove. I'm not sure if you really have to, but I always do it just to be sure :)

[berkerpeksag]

Thanks for the patch! I haven't had a chance to review the patch yet, but we also need documentation updates to Doc/library/sqlite3.rst.

[lelit]

For the documentation see lelit@bd82f8d or the attached patch.

[lelit]

WRT to the agreement form, I guess I'll have to compile it even if I already contributed to Python decades ago (ObjC, readline, NeXT support...), right?

Will try to do whatever is needed in the next days...

[bitdancer]

If you have a copy of your original agreement you could fax it to the PSF, but it is probably easier to just to sign the electronic one.

[lelit]

Ok, the agreement is fullfilled.

[lelit]

I guess the chance of getting this merged before the 3.6 betas is very low, but if there is *anything* I could do to raise it, please tell :-)

[lelit]

Now that we are is officially on GH, would you welcome a PR rebasing this patch on top of the master branch?

[palaviv]

I actually looked at the patch and have a few comments:

1. You need to put Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS before the sqlite3 calls (especially the sleep).
2. I think that the pysqlite_connection_backup function will look a lot better if you will have a cleanup/error label.

I am not a core developer but I think you should open the PR as it will be easier for the CR.

[lelit]

Thank you Aviv, I applied your suggestions and opened a PR.

[lelit]

Is there any chance this could be accepted for Python 3.7?