In "Canonical models of Shimura curves" by J.S. Milne (avaliable at https://www.jmilne.org/math/articles/2003a.pdf), he explains the definition of quaternion Shimura curve, and explains the modern definition of Shimura varieties is the best (at Section 4 Page 33):

"More significantly, many Shimura varieties are not moduli varieties, not even conjecturally, and so Definition A doesn’t apply to such Shimura varieties."

And he writes in Section 6, Page 37:

"Conjecturally, the Shimura variety is a moduli variety (in general for motives) when $w_X$ is defined over $\mathbb Q$, and it is not a moduli variety when $w_X$ is not defined over $\mathbb Q$."

Here $w_X : \mathbb G_m \rightarrow G_{\mathbb R}$ is the weight homomorphism. The moduli variety is for general motives, and Milne proved in 1994 that all Shimura varieties of abelian type with rational weight are moduli varieties for abelian motives.

Why do we believe such conjecture? I can't find a reference and don't know whether it's proved. If the weight is not rational, what is the obstruction of our Shimura variety to have a moduli interpretation ? On the other hand, for non-abelian type Shimura variety with rational weight, what objects will it parametrize? Thanks for any enlightening example.