Comment by AnotherGoodName | Hacker News Reader

Comment by AnotherGoodName | Hacker News Reader

Parent Full thread

AnotherGoodName

I appreciate the note on virtual memory not being reliable. This is what Windows task manager reports by default and it's terrible. Resident size is the most reliable metric. Anything else can be wrongfully inflated by things like harmless memory mapped files that won't actually hurt anything. eg. memory map 2GB of logfiles, it'll only be paged in if reading that portion of the logfile so isn't really using memory but users look at the processes and claim "OMG why does this app use so much memory". It doesn't. It uses very little. You're reading the memory usage wrong. Chrome actually had this problem for a while and they moved away from using memory mapped files. Not because memory mapped files are a bad thing but because users will read the memory usage and go crazy over what they see even though it's not really using that much actual physical memory.

There's actually guides out there on the web that tell people judge usage by virtual memory allocated too :(. At least this article gets it right :).

> This is what Windows task manager reports

Just to clarify, Windows Task Manager uses Private Working Set by default for process memory usage which does NOT include shared pages with other processes such as libraries or memory mapped files (hence the name “private”). It only shows the memory that maps to privately allocated physical memory per process. It’s probably closer to Resident Set on Unix.

You probably meant the memory usage in performance tab but I wanted to clarify in case people mistake it for all memory usage fields.

> Resident size is the most reliable metric

Actually, Proportional Set Size is more accurate than RSS. See: https://en.wikipedia.org/wiki/Proportional_set_size

`procs display` (mentioned elsethread https://news.ycombinator.com/item?id=48788167 ) supports PSS via its %M format code.

One issue with that relative to RSS is permissions. Historically, all procs could see the RSS used by procs of all other users (at least if they could see the PIDs at all). So, RSS requires no special permissions, but the Linux kernel team decided PSS should not be as promiscuous for whatever reasons (I didn't do a deep dive). So, I'm always having to do (the equivalent of) `sudo pu`.

If you use memory-mapped files, cached pages count towards the resident set size of your process. If you use ordinary file I/O, they don't. That behavior has amusing consequences in HPC clusters that monitor the memory usage of each job and kill them if they use more memory than they requested.

Resident set size is not the amount of memory that the process wants, it’s how much the OS is willing to give it. So once memory pressure kicks in it stops being representative. I’ve seen this misunderstanding lead to bad decisions a few times. I even went so far to remove this value form charts because a team member was going left when he should have gone right.

Reply on news.ycombinator.com