Quickest way to compare two generic lists for differences

What is the quickest (and least resource intensive) to compare two massive (>50.000 items) and as a result have two lists like the ones below:

1. items that show up in the first list but not in the second
2. items that show up in the second list but not in the first

Currently I’m working with the List or IReadOnlyCollection and solve this issue in a linq query:

var list1 = list.Where(i => !list2.Contains(i)).ToList();
var list2 = list2.Where(i => !list.Contains(i)).ToList();

But this doesn’t perform as good as i would like.
Any idea of making this quicker and less resource intensive as i need to process a lot of lists?