Proximity to magnetic order as well as low dimensionality are both beneficial to superconductivity at elevated temperatures. Materials on the border of magnetism display a wide range of novel and potentially useful phenomena: high Tcs, heavy fermions, coexistence of magnetism and superconductivity and giant magnetoresistance. Low dimensionality is linked to enhanced fluctuations and, in the case of heavy fermions, has been experimentally shown to be beneficial for the fluctuations that are responsible for the rich abundance of novel emergent phases. This experimental strategy motivated us to explore 2D insulating magnets with a view to investigate phase evolution across metal-insulator and magnetic-non-magnetic boundaries. This has been a fruitful venture with totally novel results different to our expectations. We present results from 2 distinct systems. The MPS3 family are highly anisotropic in both their crystal and magnetic structures. FePS3 in particular is a model insulating honeycomb antiferromagnet. We find that the application of pressure to FePS3 induces an insulator to metal transition. The second system, Cs2CuCl4, is a highly-frustrated quantum spin liquid at low temperature. The competition of the 3 relevant exchange couplings is delicately balanced. It has been shown to become antiferromagnetic at very low temperatures (~1 K). We have found that the application of pressure for 3 days or more followed by a return to ambient pressure stabilises a totally distinct magnetic ground state.