At this point everyone knows that these LLM’s don’t know what they were trained on.

Just to emphasize this, machine learning algorithms doesn’t know anything. All training it does is calibrate adjust constants in an equation.

Like Jon Snow, it knows nothing, and if you ask it for something complicated, it will put that on full display.