docker stats doesn't report memory usage on arm64

[SelfhostedPro]

• This is a bug report
• This is a feature request
• I searched existing issues before opening this one

Expected behavior

For docker stats to provide information on memory usage on arm64

Actual behavior

docker stats provides 0 information on arm as per this issue:
https://github.com/SelfhostedPro/Yacht/issues/115#issuecomment-699164648

Steps to reproduce the behavior

On a raspberry pi run docker stats

[nidua42]

Here’s the info

Raspberry pi 4b 4GB version 1.2

Linux raspberrypi 5.4.51-v7l+ #1333 SMP Mon Aug 10 16:51:40 BST 2020 armv7l GNU/Linux

Docker version
Client:
Version: 18.09.1
API version: 1.39
Go version: go1.11.6
Git commit: 4c52b90
Built: Fri, 13 Sep 2019 10:45:43 +0100
OS/Arch: linux/arm
Experimental: false

Server:
Engine:
Version: 18.09.1
API version: 1.39 (minimum version 1.12)
Go version: go1.11.6
Git commit: 4c52b90
Built: Fri Sep 13 09:45:43 2019
OS/Arch: linux/arm
Experimental: false

[Griefed]

Two more examples of docker stats not working, also mentioned in https://github.com/SelfhostedPro/Yacht/issues/115 :
Raspberry Pi 3B+: https://pastebin.com/CnkzVtKx
Raspberry Pi 3B: https://pastebin.com/nm24pWkC

[cpuguy83]

Is your cgroup memory controller enabled? What's the output of "docker info"?

[nidua42]

Docker info output:

Containers: 16
Running: 12
Paused: 0
Stopped: 4
Images: 19
Server Version: 18.09.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce
runc version: 1.0.0~rc6+dfsg1-3
init version: v0.18.0 (expected: fec3683b971d9c3ef73f284f176672c44b448662)
Security Options:
seccomp
Profile: default
Kernel Version: 5.4.51-v7l+
Operating System: Raspbian GNU/Linux 10 (buster)
OSType: linux
Architecture: armv7l
CPUs: 4
Total Memory: 3.688GiB
Name: raspberrypi
ID: COTX:PTT5:F2AA:CV34:UDEU:WUZC:WE73:Z522:TUW3:DKJK:2K65:NLC5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support

[Griefed]

Here's my docker info output for the Pi3B: https://pastebin.com/QNRDaiah
Pi3B+: https://pastebin.com/b65Tmghe
Both mention cgroupfs drivers, but I am not sure wheather that means that the cgroup memory controller is enabled.

[cpuguy83]

Yeah, looks like your cgroup memory controller is disabled.

The key here is these two warnings:

WARNING: No memory limit support
WARNING: No swap limit support

It's likely that you need to add "cgroup_enable=memory swapaccount=1" to your kernel command line... but I'm just guessing here.

[SelfhostedPro]

@cpuguy83 would that essentially be a sysctl command?

[cpuguy83]

No that's in your boot loader config.

[SelfhostedPro]

Ah. I believe this is it: https://www.raspberrypi.org/forums/viewtopic.php?t=203128

Can you try that @Griefed @nidua42 ?

[nidua42]

Yep, this solved the problem. I added cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 to cmdline.txt and now I’m getting mem and cpu graphs in yacht.

[SelfhostedPro]

Awesome. Thanks for your help @cpuguy83 ! I'll add documentation to Yacht for this.