This paper presents LESSER, the deep learning-based mobile gaze tracking system for the diagnosis of dyslexia. Dyslexia has been studied extensively across the domain since it is one of the most common learning disabilities found in general population. However, strenuous gaze tracking step with bulky and costly equipment has been hindering the timely diagnosis of dyslexia. With the use of the minimal input which includes coordinate information of both pupil, LESSER outperforms existing approaches which use complete images of both eyes in terms of capability of generalization.