I have a script reading in a csv file with very huge fields:
# example from http://docs.python.org/3.3/library/csv.html?highlight=csv%20dictreader#examples
import csv
with open('some.csv', newline="") as f:
reader = csv.reader(f)
for row in reader:
print(row)
However, this throws the following error on some csv files:
_csv.Error: field larger than field limit (131072)
How can I analyze csv files with huge fields? Skipping the lines with huge fields is not an option as the data needs to be analyzed in subsequent steps.