The usual trick works - just replace the "foo" with an asterisk while leaving the rest alone, including the backslashes:
Which ties back into those expectations from other languages that Marc mentioned. It's important to remember that, unlike in those languages, quotation marks are not token delimiters in the shell. They don't terminate the current shell
word (what other languages would just call a "string"); you can go in and out of quotes, switch kinds of quotes, etc. as often as you like within a single word.
So .*.'~undo-tree~' is still just one string, even though only part of it is in quotation marks. The part in quotes is not subject to glob expansion; the part not in quotes is.
FWIW, if you want to do the same thing but recursing into subdirectories, you can use the special recursive glob sequence **, as in **/.*.'~undo-tree~'