Magnetic Resonance Imaging (MRI) can accurately identify pathologies, their use for population-level screening of diseases has remained infeasible due to the high costs associated with their operations. A large portion of the cost is attributed to the slow process of acquiring enough data to generate images for the human eye to read and identify clinically relevant variables. Existing methods focus on reducing costs by accelerating the data acquisition process. However, the requirement to generate a high-fidelity image imposes certain constraints on the acquisition process, limiting the speedups achievable. We propose the AcceleRated MRScreener (ARMS), which learns to infer the clinically relevant variable directly from raw measurements acquired by the scanner, achieving a speedup of 20x in the data acquisition process, thereby bringing the technology closer to screening. We test the efficacy of our method on the task of identifying clinically significant prostate tumors in the MR scans of the abdomen and in identifying ACL sprains and Meniscus tears in the knee MR scans.