Learning methods for optimizing strategies in very large multi-agent settings (a.k.a. games) often abstract the space of all possible strategies of the opponents to a small portfolio. Existing scalable methods for constructing these portfolios are either hand-designed for specific domains or heuristics without understood guarantees. To enable a more fundamental approach, this paper studies the problem in small normal form games. We formally define the optimization problem of finding the optimal portfolio for approximating the Nash equilibrium in two-player zero-sum games. We propose a method to approximate the optimal solution in small games. Even though this approximation is provably suboptimal, it is enough to demonstrate that the portfolios constructed by a recent scalable heuristic are very far from the optimum. We study the reasons and propose some improvements, but we are still far from closing the gap.