The observer himself can be observed.
If I observe an observer, I see things that the observer does not. I deduce, by reasonable analogy, that therefore there are also things that I, as an observer, do not see. I want a theory of physics that accounts for the structure of the universe, that clarifies what it is to be an observer in the universe, not a theory that makes the universe depend on me observing it.