Surgical navigation systems enhance surgical efficiency and outcomes, especially in minimally invasive surgeries by providing 3D anatomy models from computed tomography (CT) scans. We introduce the Surgical Phase Recognition-based Navigation Adjustment System (SP-NAS) for distal gastrectomy, which uses surgical phase recognition for workflow-based adjustments. This system eliminates the need for real-time camera adjustments and challenging registrations by defining three reference views for ten surgical phases. We employed recent action recognition models to identify surgical phases, training and evaluating them on 146 robotic distal gastrectomy cases using 6-fold cross-validation. Our system demonstrated effective phase recognition and post-processed predictions, resulting in a deployable real-time solution.