Abstract: We investigate the use of Physics-Informed Neural Networks (PINNs) for ice shelf hardness inversion, focusing on the effect of the relative weighting between equation and data components in the PINN objective function on its predictive performance. In the objective function we use a hyperparameter gamma which adjusts the relative priority given to the fit of the PINN to known physical laws and its fit to the training data. We train the PINN with a range of gamma, and training data with varying magnitudes of injected noise. We find that the PINN solutions converge to two different clusters in the prediction error space; one cluster corresponds to accurate, "low-error" solutions, while the other consists of "high-error" solutions that were likely trapped in a local minimum of the PINN objective function and fit poorly to the ground truth datasets. We call this the PINN clustering behaviour, which persists for a wide range of gamma, noise level, and even with clean data. Using k-means clustering, we filter out the PINN solutions in the high-error clusters. The accuracy of the solutions in the low-error cluster varies with gamma and the data noise. We find that the value of $\gamma$ that minimizes the error of PINN-predicted ice hardness varies significantly with the data noise. With the optimal choice of gamma, the PINN can remove the noise in the data and successfully predict the noise-free velocity, thickness and the ice hardness. The clustering phenomenon is observed for a wide range of parameter settings and is of practical, as well as theoretical interest.