From what I understand, this setup will work only "if" the IP's you use on both systems are on the same layer 3 subnet
Correct.
will this not defeat the purpose of using VLANS, though?
Usually, yes. There are some instances where such a setup would be useful temporarily, for example to merge two independent networks. But like I said, it's not generally recommended.
