Skip to content

Added CLIP module and redesigned tokenizer apis #81

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
marscrazy merged 82 commits into from
Aug 29, 2022
Merged

Added CLIP module and redesigned tokenizer apis #81

marscrazy merged 82 commits into from
Aug 29, 2022

Conversation

@BAAI-OpenPlatform
Copy link
Collaborator

@BAAI-OpenPlatform BAAI-OpenPlatform commented Aug 26, 2022

No description provided.

Anhforth and others added 30 commits June 29, 2022 13:44
@Anhforth
saved workd
83fceae 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
saved workd
f113f7c 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
saved work on 6.29
708ce15
@Anhforth
transformed tokenizer: progressing
0d1b079 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@920232796 @marscrazy
Opt 30b (#16)
2763b5d 
* clean codes 
Co-authored-by: Zac Liu <liuguang@baai.ac.cn>
@Anhforth
fix bert tokenizer issue (#18)
3e52907 
* fix bert tokenizer issue
* updated t5, opt and roberta tokenizers
* fixed doc 404 error
Signed-off-by: ZhaodongYan1 <yzdrogeryan@gmail.com>
@ZhaodongYan1
reconstruct the tokenizer structure
deb2612 
Signed-off-by: ZhaodongYan1 <yzdrogeryan@gmail.com>
@Anhforth
tested the new tokenizer
c2c6e9d 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
removed some redundant codes and added sp model
fc2b5d8 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@ZhaodongYan1
updated the tokenizer
7da1757 
Signed-off-by: ZhaodongYan1 <yzdrogeryan@gmail.com>
@Anhforth
saved work
7c8c0b1 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@920232796 @marscrazy
Opt 66b (#19)
3a0c8cb 
* autoloader for opt
* opt-66b inference
* Update train.py
* Load data from example dir
* add readme of multi GPU inference

Co-authored-by: Zac Liu <liuguang@baai.ac.cn>
@Anhforth
saved work on 7.6
265d35a 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
updated release version
4f8d715 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
fix tokenizer issue
efc1310 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
temp save
59531e7 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
tokenizer test passed
3b6c16a 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
fixed some errors
a7ff8f3 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
test of tokenizer transform
f4ff1a8 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
fixed conflicts
811d9e9 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
fixed error
1406d89 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
add encode_plus
b30eefa 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@920232796
fix bug multi_gpu_training
9b81869
@Anhforth
Merge pull request #21 from baai-open-internal/fix_multi_gpu_training
7ad38a0 
fix bug multi_gpu_training
@Anhforth
changed the version
72ffd6a 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@920232796
fix_validation_bug (#24)
e6f89a6
@Anhforth
updated the version
29ea850 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
updated
4c68936 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
modified encoder_plus
4834f23 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@920232796
add vit and examples
8d44329
marscrazy and others added 23 commits August 17, 2022 10:44
@marscrazy
Merge pull request #52 from baai-open-internal/add_clip
4a4b003 
Add clip
@Anhforth
Merge branch 'master' into transform_tokenizer
ce5fd31
@marscrazy
Update train.py
8353cd3
@marscrazy
Delete tst_superglue.py
5d5e135
@BAAI-OpenPlatform
updated according to comments
4c6ba56 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@BAAI-OpenPlatform
Merge pull request #50 from baai-open-internal/bert_model
6076287 
fix bug bert model
@BAAI-OpenPlatform
merged the clip tokenizer
c11e232 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@BAAI-OpenPlatform
merged clip tokenizer
6e135ef 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@marscrazy
Update inference_clip.py
fd06e4d
@marscrazy
Update auto_loader.py
b61b708
@marscrazy
Update glm_10b_en_tokenizer.py
25b659b
@marscrazy
Merge pull request #20 from baai-open-internal/transform_tokenizer
8cffa38 
Transform tokenizer
@920232796
swinv1v2
9117f78 
Signed-off-by: zhaohu xing <920232796@qq.com>
@marscrazy
Merge pull request #58 from baai-open-internal/swinv1v2_checkpoint_ac…
f3186d9 
…tivations

add swinv1v2
@Anhforth
updated the version
4bd211d 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@Anhforth
updated the requirement packages list
6ef4190 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@BAAI-OpenPlatform
fixed some issues
036e337 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@BAAI-OpenPlatform
fixed some issues
edfd518 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@BAAI-OpenPlatform
tried to fix the data directory not found error
497d709 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@BAAI-OpenPlatform
fixed issues in running glm_seq2seq
1ac43c0 
Signed-off-by: BAAI-OpenPlatform <open.platform@baai.ac.cn>
@marscrazy
Update test_glm_seq2seq.py
351fba7
@marscrazy
Merge pull request #59 from baai-open-internal/fix_issues
35b5d9a 
fixed some issues
@Anhforth
merged upstream
d71ee8d 
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
@marscrazy marscrazy merged commit dee25b7 into master Aug 29, 2022
@BAAI-OpenPlatform BAAI-OpenPlatform deleted the develop branch August 29, 2022 07:54
@marscrazy marscrazy mentioned this pull request Aug 29, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@marscrazy marscrazy marscrazy approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@BAAI-OpenPlatform @marscrazy @Anhforth @920232796 @ZhaodongYan1 @jongjyh @wchh-2000 @xuanricheng @shunxing1234