Gaze estimation is vital in various applications, but factors like poor lighting and lowresolution images challenge the performance of estimation model. We introduce, for thefirst time a Eye Gaze Estimation with Self-Improving Features (EG-SIF) method. EG-SIFsegregates images based on their quality, generates a pair of good and adverse images, andapplies multitask training with image enhancement using the generated pairs, where thetask is to reconstruct given a poor image. This innovative approach outperforms existingmethods, significantly improving gaze estimation angular error on challenging datasets likeMPIIGaze from 4.64 to 4.53 and in RTGene from 7.44 to 7.41.