Skip to content

thirdparty.mbedtls: upgrade to v3.6.3.1#24602

Merged
spytheman merged 11 commits into
vlang:masterfrom
kbkpbot:upgrade-thirdparty-mbedtls
May 31, 2025
Merged

thirdparty.mbedtls: upgrade to v3.6.3.1#24602
spytheman merged 11 commits into
vlang:masterfrom
kbkpbot:upgrade-thirdparty-mbedtls

Conversation

@kbkpbot

@kbkpbot kbkpbot commented May 29, 2025

Copy link
Copy Markdown
Contributor

required by #24557

Update thirdparty/mbedtls/update.vsh, add fix_config().

// This script can be used to update the thirdparty/mbedtls/ folder to a future version of mbedtls.
import os

fn do(cmd string) {
        println(cmd)
        res := os.system(cmd)
        if res != 0 {
                panic('failed at: `${cmd}`')
        }
}

fn fix_config(config_file string) ! {
        mut content := os.read_file(config_file)!
        content += '\n
#if defined(__TINYC__)
#undef MBEDTLS_HAVE_ASM
#undef MBEDTLS_AESNI_C
#undef MBEDTLS_PADLOCK_C
#else // __TINYC__
#define MBEDTLS_HAVE_ASM
#define MBEDTLS_AESNI_C
#define MBEDTLS_PADLOCK_C
#endif // __TINYC__

#if ( defined(__linux__) || defined(__FreeBSD__) ) || defined (__OpenBSD__)
#define MBEDTLS_THREADING_PTHREAD
#define MBEDTLS_THREADING_C
#else
#undef MBEDTLS_THREADING_PTHREAD
#undef MBEDTLS_THREADING_C
#endif
'
        os.write_file(config_file, content)!
}

os.chdir(@VEXEROOT)!

do('rm -rf mbedtls/')
do('git clone --depth=1 --recursive --shallow-submodules --single-branch -b v3.6.3.1 https://github.com/Mbed-TLS/mbedtls.git mbe
dtls/')
do('rsync -a --delete mbedtls/include/  thirdparty/mbedtls/include/')
do('rsync -a --delete mbedtls/library/  thirdparty/mbedtls/library/')
do('rsync -a --delete mbedtls/3rdparty/ thirdparty/mbedtls/3rdparty/')
do('rsync -a          mbedtls/LICENSE   thirdparty/mbedtls/LICENSE')
do('rsync -a          mbedtls/README.md thirdparty/mbedtls/README.md')
do("find thirdparty/mbedtls/ -name '*.txt' -or -name '*.inc' -or -name '.gitignore' -or -name 'Makefile' |xargs rm -f")
do('rm -rf mbedtls/')
fix_config('thirdparty/mbedtls/include/mbedtls/mbedtls_config.h')!

@huly-for-github

Copy link
Copy Markdown

Connected to Huly®: V_0.6-22963

@spytheman spytheman left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent, thank you 🙇🏻 .

@spytheman

Copy link
Copy Markdown
Contributor

The recent changes in thirdparty/mbedtls/include/mbedtls/mbedtls_config.h are not reflected here. Are they not needed anymore?

@spytheman

Copy link
Copy Markdown
Contributor
   │ 52c7130 2025-05-25 12:19 +0200 LCheylus  ∙ <weekly.2025.22> net.mbedtls: enable MBEDTLS_THREADING_C and MBEDTLS_THREADING_PTHREAD on OpenBSD (#24572)
   │ 22c327f 2025-05-24 10:07 +0200 LCheylus  ∙ net.mbedtls: disable AES-NI on OpenBSD with tcc (fix #22239) (#24560)
   │ dacc738 2025-02-10 21:18 +0200 DAngelov  ∙ thirdparty: improve `-cstrict -cc clang-18 -prod` compatibility for programs that do `import net.mbedtls`
  5│ b995e64 2024-11-21 01:30 +0200 DAngelov  ∙ thirdparty: enable MBEDTLS_THREADING_C and MBEDTLS_THREADING_PTHREAD on FreeBSD too
   │ 75ae93d 2024-11-21 01:13 +0200 DAngelov  ∙ ci,thirdparty: enable MBEDTLS_THREADING_PTHREAD and MBEDTLS_THREADING_C only on Linux for now (fix windows CI)
   │ 4b015c8 2024-11-20 23:42 +0200 DAngelov  ∙ net.mbedtls: define MBEDTLS_THREADING_PTHREAD, in mbedtls_config.h; call C.mbedtls_ssl_conf_read_timeout expli
   │ c2788f9 2024-09-15 15:00 +0300 DAngelov  ∙ thirdparty: fix compilation of thirdparty/mbedtls with tcc on macos m1
   │ 64a3369 2024-03-28 07:46 +0100 Turiiya   ∙ thirdparty,net.mbedtls: update `mbedtls` to latest compatible version v3.3.0 (#21118)

@lcheylus

lcheylus commented May 29, 2025

Copy link
Copy Markdown
Contributor

The recent changes in thirdparty/mbedtls/include/mbedtls/mbedtls_config.h are not reflected here. Are they not needed anymore?

Yes, the fixes in thirdparty/mbedtls/include/mbedtls/mbedtls_config.h are needed and are not included here.

@kbkpbot Your fix in fix_config function / script thirdparty/mbedtls/update.vsh is wrong.

You must backport fixes (list by @spytheman above) in .c/.h files.

@kbkpbot

kbkpbot commented May 29, 2025

Copy link
Copy Markdown
Contributor Author

-cstrict -cc clang-18 -prod

OK , I will verify them.

@lcheylus

Copy link
Copy Markdown
Contributor

@kbkpbot Do you update thirdparty/mbedtls from official v3.6.3.1 tag or last commit from GH repository ?

@kbkpbot

kbkpbot commented May 29, 2025

Copy link
Copy Markdown
Contributor Author
git clone --depth=1 --recursive --shallow-submodules --single-branch -b v3.6.3.1 https://github.com/Mbed-TLS/mbedtls.git mbe
dtls/

git clone --depth=1 --recursive --shallow-submodules --single-branch -b v3.6.3.1 https://github.com/Mbed-TLS/mbedtls.git mbe
dtls/

@lcheylus

Copy link
Copy Markdown
Contributor
git clone --depth=1 --recursive --shallow-submodules --single-branch -b v3.6.3.1 https://github.com/Mbed-TLS/mbedtls.git mbedtls/

OK, the sources are correctly imported from the v3.6.3.1 tag :)

@lcheylus

Copy link
Copy Markdown
Contributor

@kbkpbot Could you add a .patch file (and some doc) with all our modifications towards mbedtls v3.6.3.1 sources ? It will be simple to use for a future update. Your .vsh file is not sufficient to do it => backport for #dacc738 is not included in it.

@kbkpbot

kbkpbot commented May 30, 2025

Copy link
Copy Markdown
Contributor Author

dacc738

I am working on it

@kbkpbot

kbkpbot commented May 30, 2025

Copy link
Copy Markdown
Contributor Author

How to skip MemorySanitizer for mbedtls library?
As "MemorySanitizer does not support assembly implementation" if we defined ASM implement in config.

Also, due to a bad implementation of RtlSecureZeroMemory in tcc/windows, it need a fallback to memset.
in include/winapi/winnt.h

    __CRT_INLINE PVOID RtlSecureZeroMemory(PVOID ptr,SIZE_T cnt) {
      volatile char *vptr =(volatile char *)ptr;
#ifdef __x86_64
      __stosb((PBYTE)((DWORD64)vptr),0,cnt);           // this is for MSVC only. So tcc need a fallback
#else
      while(cnt) {
	*vptr = 0;
	vptr++;
	cnt--;
      }
#endif
      return ptr;
    }

@spytheman

Copy link
Copy Markdown
Contributor

Please restore the thirdparty/mbedtls/update.vsh script.

The script already did most of the instructions, that are now described in the fix.md file, but the difference between them, is that you can not run the descriptions in the .md file automatically (you have to follow them), but you can run a .vsh one, and it will do more or less the same, but without the possibility of human error.

The goal with the patches and scripts, is to have an easy/automated way to replicate the changes, so that they can be both:

  1. verified independently
  2. redone in the future, when we update to a new mbedtls version

@spytheman

spytheman commented May 31, 2025

Copy link
Copy Markdown
Contributor

The .vsh script can also apply the .patch file.

@spytheman

Copy link
Copy Markdown
Contributor

The update process passed locally for me now too:
image

@spytheman spytheman merged commit 1274cdc into vlang:master May 31, 2025
64 checks passed
@kbkpbot kbkpbot deleted the upgrade-thirdparty-mbedtls branch May 31, 2025 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants