All major OSes (well Windows and macOS) do in-memory compression before swap, which is cheaper than evicting a file-backed page. But still slow, so you don't want to rely on it.
Compacting reduces memory usage - that's why it's called compacting.
The JVM uses a lot of memory a) because it's tuned for servers and not for low memory usage and b) because Java is a poorly designed language without value types.
No, it reduces memory fragmentation, which is why it's called compacting and not compression.
I do agree that the lack of value types is a big contributor to why Java uses so much memory. But it's not a server tuning thing that makes the JVM lean memory heavy.
The JVM uses moving collectors and that is the big reason why it prefers having so much memory available. Requesting and freeing memory blocks from the OS is an expensive operation which the JVM avoids by grabbing very large blocks of memory all at once. If you have a JVM with 75% old gen and 25% new gen, half that new gen will always be empty because the JVM during collection moves live data from one side of the new gen to the next. And while it does that, it slowly fills up old gen with data.
Even more modern collectors like G1 prefer a large set of empty space because it's moving portions of old gen to empty regions while it does young collection.
As I mentioned, the difference here between the JVM and python or go is that python and go do no moving. They rely heavily on the malloc implementation to handle grabbing right sized blocks from the OS and combating memory fragmentation. But, because they aren't doing any sort of moving, they can get away with having more "right sized" heaps.
The 390GB is because of implementation details of GPU drivers. It's not wrong but it also doesn't matter at all.
(The only number that actually means anything there is the first one, but the label for it is basically meaningless. Use `footprint` in Terminal for a better explanation of memory use.)
"Poor" in California means earning $80k/year, so they probably are not doing that. Africa / Indonesia / Philippines are better places to find English speaking RLHF workers.
Yes, yes, I know. But how many people have AI companies hired to do RLHF who actually have the expertise to adjust them away from biases like this? As opposed to paying a dollar per day to a bunch of poor people in Africa?
Claudes have lots of empathy. The issue is the opposite - it isn't very good at challenging you and it's not capable of independently verifying you're not bullshitting it or lying about your own situation.
But it's better than talking to yourself or an abuser!
It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful. Definitely agree about talking to an abuser, though.
Sometimes people indeed just need validation and it helps them a lot, in that case LLMs can work. Alternatively, I assume some people just put the whole situation into words and that alone helps.
But if someone needs something else, they can be straight up dangerous.
> It's about the same as talking to yourself, LLMs simply agree with anything you say unless it is directly harmful.
They have world knowledge and are capable of explaining things and doing web searches. That's enough to help. I mean, sometimes people just need answers to questions.
In one way it's potentially worse than talking to yourself. Some part of you might recognize that you need to talk to someone other than yourself; an LLM might make you feel like you've done that, while reinforcing whatever you think rather than breaking you out of patterns.
Also, LLMs can have more resources and do some "creative" enabling of a person stuck in a loop, so if you are thinking dangerous things but lack the wherewithal to put them into action, an LLM could make you more dangerous (to yourself or to others).
I haven't tried talking to Sonnet much, but Opus 4.6 is very sycophantic. Not in the sense of explicitly always agreeing with you, but its answers strictly conform to the worldview in your questions and don't go outside it or disagree with it.
It _does_ love to explicitly agree with anything it finds in web search though.
(Anthropic tries to fight this by adding a hidden prompt that makes it disagree with you and tell you to go to bed, which doesn't help.)
the go to bed thing gets annoying, you can't even hint that you are almost done or wrapping up or something or this is hyper triggered and it never stops.
I do like when opus is incredibly short in its responses to prompts that probably shouldnt have been made though. keeps me grounded a bit.
reply