Heh. I finally bothered to look this up in the FDA classification database.
The answer is for toothbrushes is yes. Product Code EFW, Class I, GMP exempt, regulation = 872.6855. As always, patient contact materials are expected to conform to ISO 10993 per guidance memo G95-1 and other guidance info.
The answer for drinking cups is no, unless they're a DAAD (daily activity assist device) for a person with abnormal physical limitations, in which case they're Product Code ILC, Class I, GMP exempt, regulation = 890.5050.