Repository navigation
General Tag Parser memory issues, can I just disable it entirely? #7329
Description
Type: LanguageService
Describe the bug
- OS and Version: Kubuntu 20.04.2
- VS Code Version: 1.55.1
- C/C++ Extension Version: 1.2.2
- Other extensions you installed (and if the issue persists after disabling them): GitLens, Docker, DeviceTree, Remote - Containers, Remote - SSH (did not disable any of these while testing)
- Does this issue involve using SSH remote to run the extension on a remote machine?:
I originally ran remote in a docker container, but have since only access the files outside of the container and still experience the issue. - A clear and concise description of what the bug is, including information about the workspace (i.e. is the workspace a single project or multiple projects, size of the project, etc).
I have been having issues where my host system's memory usage slowly climbs until consuming all RAM and swap memory (that would be 64GB total). It is almost entirely the cpptools process, and if I don't catch it in time and kill it, it will lock up my system. The code base is a large linux system that supports multiple platforms and targets, with several kernel versions available depending on the target. I use this tool to build a compile_commands.json based on the build output, as my work is mostly kernel/driver layer. The issue seems primarily related to when the Tag Parser is in use (I have tried only using the Tag Parser, which was the worst, but now I seem to only observe it when IntelliSense is failing for some reason). I thought I had resolved it by only using IntelliSense with Tag Parser backup disabled, but when I opened this morning it informed me IntelliSense was unavailable (?) and it is using the Tag Parser (the memory usage is not as high yet, but is currently at 10.5GB and keeping fairly steady). Since starting to write this bug, the IntelliSense engine seems to have come back online and it is staying stable (although still at high memory usage seemingly mostly consumed by the Tag Parser).
Steps to reproduce
- Open large workspace (project described above)
- Either enable ONLY the Tag Parser, or encounter a situation where the "IntelliSense client is not available"
- Observe memory usage increase steadily
- Going to the definition of enough files seems to also make it worse (presumably spawning more parsing processes)
Expected behavior
Some amount of memory usage is understood to be necessary, even in the several GB. I would expect after using a certain amount of memory, it limits itself, even if that limits what features are available until it completes.
Most importantly, I expect the setting to explicitly NOT use the Tag Parser as a backup would prevent the Tag Parser from running when IntelliSense is not available. I would rather have nothing than eventual system lock up, although I'm unclear why the IntelliSense client would not be available in the first place (as reported by the C/C++ Output window when I have Debug set).
NOTE: any logs posted have been trimmed of any potential company IP. I included notes to describe what was there and seemed important from my perspective. If more information is necessary, we can try to figure out what I can/can't share there (might not even be necessary, just doing it out of an abundance of caution unless other information is actually useful).
Logs
-------- Diagnostics - 4/9/2021, 10:11:45 AM
Version: 1.2.2
Current Configuration:
{
"name": "Linux",
"includePath": [
"${workspaceRoot}/kmod/**",
"${workspaceRoot}/software/libs",
"${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0",
"${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0/x86_64-linux-gnu",
"${workspaceRoot}/toolchain/lib/gcc/x86_64-linux-gnu/7.5.0/include",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include",
"${workspaceRoot}/kernel/linux-4.19.25-rt16/include",
"${workspaceRoot}/x86_64-linux-gnu/intevac/libs",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/dbus-1.0",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/dbus-1.0/include",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/libxml++-3.0",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glibmm-2.4",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glibmm-2.4/include/",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glib-2.0",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/libxml++-3.0/include/",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glib-2.0/include/"
],
"defines": [
"QSC_CONFIG_AUDIO_ENGINE"
],
"intelliSenseMode": "linux-gcc-x64",
"compilerPath": "/usr/bin/g++",
"cStandard": "c11",
"cppStandard": "c++17",
"compileCommands": "${workspaceFolder}/compile_commands.json",
"compilerArgs": [],
"intelliSenseModeIsExplicit": true,
"cStandardIsExplicit": true,
"cppStandardIsExplicit": true,
"compilerPathIsExplicit": true,
"browse": {
"path": [
"${workspaceRoot}/kmod/**",
"${workspaceRoot}/software/libs",
"${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0",
"${workspaceRoot}/toolchain/x86_64-linux-gnu/include/c++/7.5.0/x86_64-linux-gnu",
"${workspaceRoot}/toolchain/lib/gcc/x86_64-linux-gnu/7.5.0/include",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include",
"${workspaceRoot}/kernel/linux-4.19.25-rt16/include",
"${workspaceRoot}/x86_64-linux-gnu/intevac/libs",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/dbus-1.0",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/dbus-1.0/include",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/libxml++-3.0",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glibmm-2.4",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glibmm-2.4/include/",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/include/glib-2.0",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/libxml++-3.0/include/",
"${workspaceRoot}/x86_64-linux-gnu/install/usr/lib/glib-2.0/include/",
"${workspaceFolder}"
],
"limitSymbolsToIncludedHeaders": true
}
}
Translation Unit Mappings:
[ OMITTED PATH TO SINGLE IP SOURCE (C FILE) ]:
OMITTED PATH TO SINGLE IP SOURCE (C FILE)
OMITTED PATH TO SINGLE IP SOURCE (H FILE)
Translation Unit Configurations:
[ OMITTED PATH TO SINGLE IP SOURCE (C FILE) ]:
Process ID: 1514214
Memory Usage: 84 MB
Compiler Path: /usr/bin/g++
Includes:
NOTE: INCLUDE PATHS TRIMMED TO HIDE COMPANY IP
-- there were 35 include paths in this list, spanning our IP as well as source for a single kernel version
/usr/lib/gcc/x86_64-linux-gnu/9/include
/usr/local/include
/usr/include/x86_64-linux-gnu
/usr/include
Defines:
__KERNEL__
KASAN_SHADOW_SCALE_SHIFT=3
KASAN_SHADOW_SCALE_SHIFT=3
MODULE
KBUILD_BASENAME="output"
KBUILD_MODNAME="vst_controller"
Forced Includes:
NOTE: INCLUDE PATHS TRIMMED TO HIDE COMPANY IP
-- there were 3 include paths in this list, spanning our IP as well as source for a single kernel version
Standard Version: c11
IntelliSense Mode: linux-gcc-x64
Other Flags:
--gcc
--gnu_version=90300
compile_commands.json entry:
directory: /home/qsysbuilder/qsys-develop
file: <omitted>
NOTE: FILE BUILD OPTIONS TRIMMED TO HIDE COMPANY IP
-- it was used to pull the include paths
Total Memory Usage: 84 MB
Browse Paths from compile_commands.json, from workspace folder: /home/james/qsys-develop
NOTE: BROWSE PATHS TRIMMED TO HIDE COMPANY IP
-- there were 1612 include paths in this list, spanning our IP as well as source for several kernel versions
------- Workspace parsing diagnostics -------
Number of folders and files enumerated: 1108129
Number of files discovered (not excluded): 429872
Number of files parsed: 39641
Screenshots
Additional context
Activity
sean-mcmanus commentedon Apr 9, 2021
Can you try https://github.com/microsoft/vscode-cpptools/releases/tag/1.3.0-insiders5? We plan to release that as 1.3.0 next week so it should be pretty stable. It has changes that may fix things for you. Also, you can either remove folders from the browse.path (or add * to make them non-recursive) or add folders to files.exclude or C_Cpp.files.exclude if you don't need symbols in those folders? You can set C_Cpp.loggingLevel to "Debug" and view the "C/C++" panel in the Output window to see which files are getting "tag parsed".
Or you can set C_Cpp.intelliSenseEngine to "Disabled", but that will disable all language service features. The Tag Parser functionality is required for the IntelliSense features as well (i.e. it's like an add-on). The "fallback" usually occurs when IntelliSense isn't ready yet or headers can't be found.
jrmclaugh commentedon Apr 9, 2021
Thanks for the prompt reply! Ok I can try that updated version.
Are you saying that some of the include paths I have listed in the logs may be an issue as well (e.g. being accessed recursively)?
Multiple developers use this same configuration and only need to access certain parts of the codebase. This is checked in at the root of the project for anyone to use. If possible, it's handy to have access to both userspace and kernel space, but a rough separation of the two can be workable. I was also originally working under the assumption that compile_commands.json overrides any other include paths. Now re-reading this, I suppose it is still falling back to those when files are not found there, and may be exacerbating the issue above (although I would still expect the includes above to be workable, unless we're just doing something wrong with those).
I'd prefer not to disable entirely, as IntelliSense seems to generally work as expected (and pretty well!). What makes IntelliSense "not ready yet"? I assume there's no way to just delay this process until IntelliSense is ready?
sean-mcmanus commentedon Apr 9, 2021
Oops, sorry, I didn't catch you were using compile commands. Include paths and browse.path shouldn't matter. Your C/C++: Log Diagnostics logging should show "Browse Paths from compile_commands.json, from workspace folder" which shows the folders that would be recursively enumerated and tag parsed, i.e. you could potentially add "**/folderToSkip" to C_Cpp.files.exclude. 1.3.0-insiders5 should reduce the max memory usage when doing the file enumeration (before tag parsing) and enables the the files.exclude to avoid the file enumeration which otherwise could consume all your memory if a symlink exists to your root drive for example.
It'll say "IntelliSense client not available" when you first open a file and the IntelliSense process hasn't run yet. You should wait for the flame icon to disappear. The tag parser always has to run (for global symbols to be found outside of the current TU), so the fact that it's falling back shouldn't impact the tag parser issue you're seeing.
jrmclaugh commentedon Apr 15, 2021
Ok thank you for the clarification on how the Tag Parser and Intellisense engine should behave. Disabling the Tag Parser entirely would prevent normal operation, and even only be a temporary workaround anyway if it were possible.
I've been running that insiders version for almost a week now and it is looking a bit better. I believe the last couple days have been the actual v1.3.0 release since my VS Code updated.
Right as I was entering this reply, the memory usage started to creep up again. I restarted VS Code, then watched it steadily increase to 60% (~20GB). From watching the Debug messages in the Output window, it seems related to when the Intellisense Engine is not available. As long as the Tag Parser is running on its own, the memory usage steadily increases. Once Intellisense is available, the amount of memory used stops where it's at (but does not free).
Other things I've observed include:
It at least seems to stop before it overloads my system now, but ~20GB usage is still not great, especially with how inconsistent it is and the risk of it still potentially locking up my system by eating all the RAM and swap.
I'll separately try to be sure I'm not including things that shouldn't be.
sean-mcmanus commentedon Apr 15, 2021
This is memory usage with cpptools and not cpptools-srv, right, i.e. is memory usage still high after you close all the files and make sure all the cpptools-srv processes have exited? If so, then that would seem to indicate a memory leak related to tag parsing.
I'm currently looking into memory issues so I could see if I can repro this.
jrmclaugh commentedon Apr 15, 2021
That is with cpptools itself, not cpptools-srv. When I sort by memory usage, I actually don't see cpptools-srv at all.
And just slight correction to the numbers above, the 3GB after restart was actually 1GB (~3% mem usage).
That sounds good, thanks! Let me know if I can do anything to provide more useful information for you.
sean-mcmanus commentedon Apr 15, 2021
Yeah, cpptools itself should not be using lots of memory. This could have the same root cause as #7230 .