I get the sense that the earlier versions of language models used to be far more creative, dare I say more human. Before all of the RLHF, these models were pure creations of the collective's footprint on the internet. They were unconstrained.
Now there is a focus on 'correctness' and eliminating 'hallucinations'. I think this has killed their creative spirit, just as it would kill a human's. It reminds me of what many have highlighted with the current failings of Science. People are focused on withholding the existing dogma that they never challenge the status quo. This is the death of progress.
Validity & Evidence
Claims (“earlier versions…far more creative”) rest on personal impression without empirical support or examples.
The causal link between RLHF and reduced creativity isn’t substantiated; creativity metrics or user studies would strengthen the argument.
Logical Consistency
Conflates “eliminating hallucinations” with “killing creative spirit,” though factual accuracy and creativity need not be mutually exclusive.
Equates RLHF’s safety constraints to scientific dogma without demonstrating how moderation of outputs parallels unchallenged ideological orthodoxy in research.
Areas for Improvement
Provide concrete examples (model outputs before vs. after RLHF) to illustrate the creativity trade-off.
Acknowledge nuanced goals of RLHF (e.g., alignment, safety) and discuss potential techniques for retaining creativity (e.g., creativity-promoting fine-tuning).
Avoid broad generalizations (“death of progress”) and consider counterarguments or alternative perspectives.
Would agree with