Despite the promising capabilities of diffusion models in speech enhancement, their application in Acoustic Echo Cancellation (AEC) has been limited. In this paper, we introduce Fewer Step Diffusion, a framework specifically designed for AEC, which addresses computational efficiency concerns, making it particularly suitable for deployment on edge devices. Unlike traditional approaches, FSD uses a novel score model, which substantially boosts processing efficiency. Additionally, we present a unique noise generation technique that leverages far-end signals, utilizing both far-end and near-end signals to enhance the accuracy of the score model. We evaluate our proposed method using the ICASSP2023 Microsoft Deep Echo Cancellation Challenge dataset, where FSD demonstrates superior performance compared to several end-to-end methods and other diffusion-based echo cancellation techniques.