You shouldn't need to specify the server in each filter.  Have you tried
this?:

-*.phtml*

Also, does it make a difference with the assume option on or off?