Our aim was to retrospectively evaluate the results of ultrasound (US)-guided vacuum-assisted excision (US-VAE) of phyllodes tumors (PTs). A total of 41 PTs diagnosed at US-VAE followed by surgery (n = 27) or at least 2 y of US monitoring (n = 14) were included. By comparison of US-VAE pathology with surgical histology or follow-up US results, cases were divided into upgraded (malignant) and non-upgraded (benign) groups. These two groups were compared with respect to clinical, procedural and US features. Among 27 surgical cases, 2 (8.7%) of 23 benign PTs were upgraded to malignant PTs. The Breast Imaging Reporting and Data System category was retrospectively assigned as 4a (50%) or 4b (50%) in the upgraded group (n = 2) and 3 (64%) or 4a (36%) in the non-upgraded group (n = 39) (p = 0.018). Residual tumor was observed at the site of US-VAE in 15 of 27 surgical cases and 0 of 14 US follow-up cases (36.6%, 15/41). Given the rates of upgrade to malignancy (8.7%) and residual tumor (36.6%), PTs diagnosed after US-VAE should be surgically excised.