I read should AI flatter us, fix us, or just inform us, the crux of which was that agents like ChatGPT etc. should behave like machines, and it should be clear to us, the humans, that they are machines.

Emotional Agents by Kevin Kelly says it's a matter of when rather than if. That emotional agents would be a selling point of these agents. Not just uni-directional emotions, the machines will learn to read our emotions too, and act accordingly. It will be a relationship after all.

Emotions in machines will not arrive overnight. The emotions will gradually accumulate, so we have time to steer them. They begin with politeness, civility, niceness. They praise and flatter us, easily, maybe too easily. The central concern is not whether our connection with machines will be close and intimate (they will), nor whether these relationships are real (they are), nor whether they will preclude human relationships (they won’t), but rather who does your emotional agent work for? Who owns it? What is it being optimized for? Can you trust it to not manipulate you? These are the questions that will dominate the next decade.