Skip to content

Comments

Enum to varchar#963

Merged
timvaillancourt merged 40 commits intogithub:masterfrom
openark:enum-to-varchar
Jun 10, 2021
Merged

Enum to varchar#963
timvaillancourt merged 40 commits intogithub:masterfrom
openark:enum-to-varchar

Conversation

@shlomi-noach
Copy link
Contributor

@shlomi-noach shlomi-noach commented May 2, 2021

Related issue: #642

Description

Looking into -alter converting an enum into varchar. Per #642 this copies the numeric value of the enum into the varchar column, where we want the textual value.

Initial commit: test suite to confirm error.

@shlomi-noach
Merge pull request #2 from openark/workflow-upload-artifact
9b2a04d 
Actions/Workflow: upload artifact
@shlomi-noach
Merge pull request #5 from openark/parse-alter-statement
9ccde4f 
Support a complete ALTER TABLE statement in --alter
@shlomi-noach
merged conflict
b59a8ed
@shlomi-noach
Merge pull request #8 from openark/ajm188-handle_driver_timeout_error
6012e80 
handle driver timeout error
@shlomi-noach
v1.1.0
ae22d84
@shlomi-noach
Merge remote-tracking branch 'upstream/master' into updates-from-upst…
ca0ca5a 
…ream-2020-10
@shlomi-noach
Merge pull request #11 from openark/updates-from-upstream-2020-10
e9f9af2 
 Updates from upstream 2020 10
@shlomi-noach
WIP: copying AUTO_INCREMENT value to ghost table
294d43b 
Initial commit: towards setting up a test suite

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
greping for 'expect_table_structure' content
26f7602
@shlomi-noach
Adding simple test for 'expect_table_structure' scenario
75009db
@shlomi-noach
adding tests for AUTO_INCREMENT value after row deletes. Should initi…
eeab264 
…ally fail
@shlomi-noach
clear event beforehand
2d0281f
@shlomi-noach
parsing AUTO_INCREMENT from alter query, reading AUTO_INCREMENT from …
af20211 
…original table, applying AUTO_INCREMENT value onto ghost table if applicable and user has not specified AUTO_INCREMENT in alter statement
@shlomi-noach
support GetUint64
31069ae 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
minor update to test
3d4dfaa 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
adding test for user defined AUTO_INCREMENT statement
63219ab
@shlomi-noach
Merge branch 'master' into copy-auto-increment
525a80d
@shlomi-noach
Merge pull request #12 from openark/copy-auto-increment
ff82140 
Copying AUTO_INCREMENT value to ghost table
@shlomi-noach
Generated column as part of UNIQUE (or PRIMARY) KEY
7202076 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
skip analysis of generated column data type in unique key
b7b3bfb 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Merge pull request #13 from openark/unique-key-generated-column
253658d 
Generated column as part of UNIQUE (or PRIMARY) KEY
@shlomi-noach
Merge pull request #14 from ccoffey/cathal/safer_cut_over
4a36e24 
Cut-over should wait for heartbeat lag to be low enough to succeed
@shlomi-noach
All MySQL DBs limited to max 3 concurrent/idle connections
710c9dd 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Merge branch 'master' into limit-mysql-connetions
dea8d54
@shlomi-noach
Merge pull request #15 from openark/limit-mysql-connetions
2b5cf78 
All MySQL DBs limited to max 3 concurrent/idle connections
@shlomi-noach
hooks: reporting GH_OST_ETA_SECONDS. ETA stored as part of migration …
54000ab 
…context

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
GH_OST_ETA_NANOSECONDS
51719a2 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
N/A denoted by negative value
76b9c16 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
ETAUnknown constant
b688c58 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
Merge pull request #17 from openark/hooks-eta-seconds
33516f4 
hooks: reporting GH_OST_ETA_SECONDS. ETA as part of migration context
@shlomi-noach
test: not null
9bb2daa 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
first attempt at setting enum-to-string right
939b898 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
fix insert query
95ee9e2 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
store enum values, use when populating
e80ddb4 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
apply EnumValues to mapped column
6e5b665 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@shlomi-noach
fix compilation error
82bdf06 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@timvaillancourt
Copy link
Collaborator

timvaillancourt commented May 4, 2021

@shlomi-noach any concerns if I re-open this PR? I think this includes some PRs we haven't merged but this would be useful to have in this repo

@shlomi-noach
Copy link
Contributor Author

shlomi-noach commented May 4, 2021

@timvaillancourt let me open a new one, since there's been more commits to this branch than are present here.

@timvaillancourt
Copy link
Collaborator

timvaillancourt commented May 4, 2021

@shlomi-noach oh, thanks a lot!

We can handle getting this PR in-sync if you'd like, I think we'll just need to merge a few dependant PRs to get this one mergeable. Totally up to you, thanks!

@shlomi-noach
Copy link
Contributor Author

shlomi-noach commented May 4, 2021

@timvaillancourt I see some merge conflicts locally, looks like I need to pull from upstream. Will do so tomorrow morning!

@shlomi-noach shlomi-noach reopened this May 4, 2021
@shlomi-noach
Copy link
Contributor Author

shlomi-noach commented May 4, 2021

@timvaillancourt well that was short. The PR is all yours!

@shlomi-noach
gofmt
48b0283 
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
@timvaillancourt
Copy link
Collaborator

timvaillancourt commented May 4, 2021 •
edited
Loading

@shlomi-noach thanks a lot!

This support will unblock some varchar -> enum migrations we wanted to do but were afraid we couldn't roll back ❤️🎉

@shlomi-noach
Copy link
Contributor Author

shlomi-noach commented May 4, 2021

unblock some varchar -> enum

I hope you mean enum -> varchar 😛

@timvaillancourt
Copy link
Collaborator

timvaillancourt commented May 4, 2021

unblock some varchar -> enum

I hope you mean enum -> varchar 😛

@shlomi-noach I think the blocked migrations are varchar -> enum (which already worked?) but we were worried we couldn't reverse the migration without trashing the data if we needed to

I believe we would have numbers instead of real varchar values if we reversed a migration to enum

@shlomi-noach
Copy link
Contributor Author

shlomi-noach commented May 4, 2021

ahh, got it.

@timvaillancourt timvaillancourt modified the milestones: v1.2.0, v1.1.2 May 5, 2021
@shlomi-noach
Copy link
Contributor Author

shlomi-noach commented May 25, 2021

branch updated after recent merge of downstream contribution.

@timvaillancourt
Copy link
Collaborator

timvaillancourt commented Jun 10, 2021

This PR passed integration testing 100%, merging. Thanks @shlomi-noach!

@timvaillancourt timvaillancourt merged commit 9bc508f into github:master Jun 10, 2021
@shlomi-noach shlomi-noach deleted the enum-to-varchar branch June 10, 2021 17:47
@timvaillancourt timvaillancourt mentioned this pull request Jun 16, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

data-integrity enhancement needs-testing

Projects

None yet

Milestone

v1.1.2

Development

Successfully merging this pull request may close these issues.

2 participants

@shlomi-noach @timvaillancourt