In drug discovery, structure-based high-throughput virtual screening (vHTS) campaigns aim to identify bioactive ligands or 'hits' for therapeutic protein targets from docked poses at specific binding sites. However, while generally successful at this task, many deep learning methods are known to be insensitive to protein-ligand interactions, decreasing the reliability of hit detection and hindering discovery at novel binding sites. Here, we overcome this limitation by introducing a class of models with two key features: 1) we condition bioactivity on pose quality score, and 2) we present poor poses of true binders to the model as negative examples. The conditioning forces the model to learn details of physical interactions. We evaluate these models on a new benchmark designed to detect pose-sensitivity.