Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Protobuf C++ Arena - Memory Clean Slow
I have a fairly large C++ application that is reading Protobuf messages from the network using the Google Cloud CPP BigQuery API.
If the data is too much, the application starts to use up a lot of memory very very fast and after some time, the machine becomes unresponsive.
In my implementation, I hold onto the Protobuf messages in a list and clean up the list in the destructor.
I thought there was a memory leak, but after taking gperftool dumps, I saw that the memory was being cleaned up.
The weird thing that is happening is that when the gperftool is active, the application doesn't go out of memory and completes the processing, but without gperftool, I see the problems. The only difference between the 2 ways, is that with gperftools, things are just slower.
I'm suspecting that the memory release process is getting time to "free" up the arena memory of Protobuf messages when gperftool is on.
Is this an appropriate assumption? What else could be different/wrong?
1 comment thread